TW201937412A - Integrated circuit chip device and related product has the advantages of small amount of calculation and low power consumption - Google Patents

Abstract

Disclosed are an integrated circuit chip device and a related product. The integrated circuit chip device comprises a main processing circuit and a plurality of basic processing circuits. The main processing circuit comprises a first mapping circuit. At least one of the plurality of basic processing circuits comprises a second mapping circuit. The first mapping circuit and the second mapping circuit are both used for executing compression processing of each data in neural network operation. The plurality of basic processing circuits are distributed in an array mode. Each basic processing circuit is connected with other adjacent basic processing circuits. The main processing circuit is connected with n basic processing circuits of the first row, n basic processing circuits of the m row, and m basic processing circuits of the first column. The technical scheme provided by the invention has the advantages of small amount of calculation and low power consumption.

Description

Integrated circuit chip device and related products

The present disclosure relates to the field of neural networks, and in particular, to an integrated circuit chip device and related products.

Artificial neural network (Artificial Neural Network, ANN) is a research hotspot that has emerged in the field of artificial intelligence since the 1980s. It abstracts the human brain neuron network from the perspective of information processing, establishes some simple model, and forms different networks according to different connection methods. In engineering and academia, it is often referred to as neural network or neural network. A neural network is a computing model that consists of a large number of nodes (or neurons) connected to each other. The existing neural network operations are based on a central processing unit (CPU) or a graphics processor (Graphics Processing Unit, GPU) to implement the operations of the neural network. Such operations have a large amount of calculation and high power consumption.

The embodiments of the present disclosure provide an integrated circuit chip device and related products, which can increase the processing speed and efficiency of a computing device.

According to a first aspect, an integrated circuit chip device is provided. The integrated circuit chip device includes a main processing circuit and a plurality of basic processing circuits. The main processing circuit includes a first mapping circuit and at least one of the plurality of basic processing circuits. The circuit (that is, part or all of the basic processing circuit) includes a second mapping circuit, and the first mapping circuit and the second mapping circuit are both used to perform compression processing of each data in a neural network operation;

The multiple basic processing circuits are distributed in an array; each basic processing circuit is connected to other adjacent basic processing circuits, and the main processing circuit is connected to n basic processing circuits in the first row and n basic processing in the m row. Circuit and m basic processing circuits in column 1;

The main processing circuit is configured to perform each continuous operation in a neural network operation and transmit data to the basic processing circuit connected thereto;

The plurality of basic processing circuits are configured to perform operations in the neural network in a parallel manner according to the transmitted data, and transmit the operation results to the main processing circuit through the basic processing circuit connected to the main processing circuit.

In a second aspect, a neural network computing device is provided. The neural network computing device includes one or more integrated circuit chip devices provided in the first aspect.

According to a third aspect, a combined processing device is provided. The combined processing device includes a neural network computing device, a universal interconnection interface, and a universal processing device provided in the second aspect;

The neural network computing device is connected to the universal processing device through the universal interconnection interface.

According to a fourth aspect, a chip is provided, which integrates the device of the first aspect, the device of the second aspect, or the device of the third aspect.

According to a fifth aspect, an electronic device is provided, and the electronic device includes a chip according to the fourth aspect.

According to a sixth aspect, a method for calculating a neural network is provided. The method is applied in an integrated circuit chip device. The integrated electrical chip device includes the integrated circuit chip device according to the first aspect. The integrated circuit chip device is used for: Perform operations on neural networks.

It can be seen that, according to the embodiment of the present disclosure, a mapping circuit is provided to compress data blocks and then perform operations, which saves transmission resources and computing resources, so it has the advantages of low power consumption and small calculation amount.

In order to enable those skilled in the art to better understand the disclosure scheme, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are merely These embodiments are part of, but not all of the embodiments of this disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by persons with ordinary knowledge in the technical field without making creative labor fall into the scope of protection of the present disclosure.

In the apparatus provided by the first aspect, the main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction; Splitting the horizontal data block and the pre-stored identification data block associated with the horizontal data block to obtain multiple basic data blocks and the identification data block associated with the basic data block; dividing the multiple basic data blocks and The identification data blocks associated with each of the plurality of basic data blocks are distributed to a basic processing circuit connected thereto; the vertical data block and the identification data block associated with the vertical data block are broadcast to a basic processing circuit connected thereto. The identification data block may be specifically expressed by a direct index or a step index. Alternatively, a list of lists (LIL), a coordinate list (COO), and a compressed sparse row (Compressed) are optional. Sparse Row (CSR), Compressed Sparse Column (CSC), ELL (ELL Pack), and Hybrid (Hybird, HYB) are used to indicate, this application is not limited.

Taking the identification data block as an example of direct indexing, the identification data block may specifically be a data block composed of 0 and 1, where 0 represents data contained in the data block (such as weight or input nerve). The absolute value of the element is less than or equal to the first threshold. 1 means that the absolute value of the data (such as weights or input neurons) contained in the data block is greater than the first threshold. The first threshold is a user- or device-side custom random setting. , Such as 0.05, 0, and so on.

In order to save data transmission amount and improve data transmission efficiency, in the process that the main processing circuit sends data to the basic processing circuit, the target data in the multiple basic data blocks and the multiple basic data may be specifically The identification data block associated with each block is distributed to the basic processing circuit connected to it; optionally, the target data in the processed vertical data block and the identification data block associated with the vertical data block can also be broadcast to the connection to it Basic processing circuit. The target data refers to data whose absolute value in the data block is greater than the first threshold value, or non-zero data in the data block (here, specifically, a processed horizontal data block or a processed vertical data block). .

Accordingly, the basic processing circuit is configured to start the second mapping circuit to obtain a connection identification data block according to the identification data block associated with the vertical data block and the identification data associated with the basic data block; according to the connection identification Data block processing the vertical data block and the basic data block to obtain a processed vertical data block and a basic data block; performing an inner product operation on the processed vertical data block and the basic data block to obtain an operation As a result, the operation result is sent to the main processing circuit;

The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of an operation instruction.

For example, the horizontal data block is a matrix of M1 rows and N1 columns, and the basic data block is a matrix of M2 rows and N2 columns, where M1> M2 and N1> N2. Correspondingly, the identification data block associated with the horizontal data block is also a matrix with M1 rows and N1 columns, and the identification data block associated with the basic data block is also a matrix with M2 rows and N2 columns. Taking a matrix with a basic data block of 2 * 2 as an example, set , The first threshold is 0.05, then the identification data block associated with the basic data block is . The processing of the data blocks by the first mapping circuit and the second mapping circuit will be specifically described later.

In the apparatus provided by the first aspect, the main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction. ; Starting the first mapping circuit to process the horizontal data block and the vertical data block to obtain a processed horizontal data block and an identification data block associated with the horizontal data block, the processed vertical data block, and the The identification data block associated with the vertical data block; splitting the processed horizontal data block and the identification data block associated with the horizontal data block to obtain a plurality of basic data blocks and identification data associated with each of the basic data blocks Block, distributing the plurality of basic data blocks and identification data blocks each associated with the plurality of basic data blocks to a basic processing circuit connected thereto, and distributing the vertical data block and identification data associated with the vertical data block Block broadcast to the underlying processing circuit connected to it;

The basic processing circuit is configured to start the second mapping circuit to obtain a connection identification data block according to the identification data block associated with the vertical data block and the identification data associated with the basic data block; according to the connection identification data block Processing the vertical data block and the basic data block to obtain a processed vertical data block and a basic data block; performing an inner product operation on the processed vertical data block and the basic data block to obtain an operation result, Sending the operation result to the main processing circuit;

The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of an operation instruction.

In an optional embodiment, the main processing circuit is further specifically configured to split the vertical data block or the processed vertical data block and the identification data block associated with the vertical data block to obtain multiple data blocks. A portion of the vertical data block and the identification data block associated with each of the plurality of partial vertical data blocks; passing the identification data block associated with the plurality of partial vertical data blocks and the plurality of partial vertical data blocks once Or broadcast to the basic processing circuit multiple times; wherein the plurality of vertical data blocks are combined to form the vertical data block or the processed vertical data block.

Accordingly, the basic processing circuit is specifically configured to start the second mapping circuit to obtain a connection identification data block according to the identification data block associated with the partial vertical data block and the identification data block associated with the basic data block; And processing, by the connection identification data, the partial vertical data block and the basic data block to obtain a processed partial vertical data block and a processed basic data block; the processed partial vertical data block and The processed basic data block performs an inner product operation.

The connection identification data block is a data block obtained by performing an element-wise AND operation on the identification data block associated with the basic data block and the identification data block associated with the partial vertical data block. Optionally, the connection identification data block is used to represent data in which the data in both data blocks (specifically, the basic data block and the vertical data block) is greater than an absolute value. The details will be described later.

For example, the identification data block associated with the horizontal data block is a 2 * 3 matrix , Part of the vertical data block is associated with a 2 * 2 matrix , The corresponding connection identification data block is .

In the apparatus provided by the first aspect, the main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction. ; Start the first mapping circuit to process the horizontal data block to obtain a processed horizontal data block and an identification data block associated with the horizontal data block, or start the first mapping circuit according to the pre-stored horizontal data block The associated identification data block processes the horizontal data block to obtain a processed horizontal data block; and splits the processed horizontal data block and the identification data block associated with the horizontal data block to obtain multiple basic data blocks. And an identification data block associated with each of the basic data blocks, distributing the plurality of basic data blocks and the identification data blocks associated with each of the plurality of basic data blocks to a basic processing circuit connected thereto, and transmitting the vertical data Block broadcast to the underlying processing circuit connected to it;

The basic processing circuit is configured to start the second mapping circuit to process the vertical data block according to the identification data block associated with the basic data block to obtain a processed vertical data block; after the processing, Performing an inner product operation on the vertical data block and the processed basic data block to obtain an operation result, and sending the operation result to the main processing circuit;

The main processing circuit is configured to process the operation result to obtain an instruction result.

In an optional embodiment, the main processing circuit is further specifically configured to perform split processing on the vertical data block to obtain multiple partial vertical data blocks; pass the multiple vertical data blocks once or Broadcast to the basic processing circuit multiple times; wherein the plurality of partial vertical data blocks are combined to form the vertical data block or the processed vertical data block.

Correspondingly, the basic processing circuit is specifically configured to process the partial vertical data block according to the identification data block associated with the basic data block to obtain a processed partial vertical data block; Perform the inner product operation on the processed vertical data blocks.

In the apparatus provided by the first aspect, the main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction. Start the first mapping circuit to process the vertical data block to obtain a processed vertical data block and an identification data block associated with the vertical data block, or start the first mapping circuit according to a pre-stored data block. The identification data block associated with the vertical data block processes the vertical data block to obtain a processed vertical data block; performs split processing on the horizontal data block to obtain a plurality of basic data blocks; The basic data block is distributed to a basic processing circuit connected thereto, and the processed vertical data block and the identification data block associated with the vertical data block are broadcast to the basic processing circuit connected thereto;

The basic processing circuit is configured to start the second mapping circuit to process the basic data block according to the identification data block associated with the vertical data block to obtain a processed basic data block; to the processed vertical data block; Performing an inner product operation on a data block and the processed basic data block to obtain an operation result, and sending the operation result to the main processing circuit;

The main processing circuit is configured to process the operation result to obtain an instruction result.

In an optional embodiment, the main processing circuit is further specifically configured to perform split processing on the processed vertical data block and an identification data block associated with the vertical data block to obtain multiple partial vertical data blocks. And the identification data block associated with the plurality of partial vertical data blocks; and broadcasting the identification data block associated with each of the plurality of vertical data blocks and the plurality of partial vertical data blocks to one or more broadcasts through one or more times. The basic processing circuit; wherein the plurality of partial vertical data blocks are combined to form the vertical data block or the processed vertical data block.

Accordingly, the basic processing circuit is specifically configured to process the basic data block according to the identification data block associated with the partial vertical data block to obtain a processed basic data block; to the processed basic data block and The partial vertical data block performs an inner product operation.

In the device provided by the first aspect, the main processing circuit is specifically configured to send the vertical data block (specifically, the vertical data block or the processed vertical data block) to a connection with the vertical data block through one broadcast. The basic processing circuit.

In the device provided by the first aspect, the basic processing circuit is specifically configured to perform an inner product of the basic data block (the same can be the basic data block or the processed basic data block) and the vertical data block. The inner product processing result is obtained by processing, the inner product processing result is accumulated to obtain an operation result, and the operation result is sent to the main processing circuit.

In the device provided by the first aspect, the main processing circuit is configured to, when the operation result is a result of an inner product processing, accumulate the operation result to obtain an accumulation result, and arrange the accumulation result to obtain the result. The data block to be calculated and the instruction result of the operation instruction.

In the device provided by the first aspect, the main processing circuit is specifically configured to divide the vertical data block into multiple partial vertical data blocks, and broadcast the multiple partial vertical data blocks to all The basic processing circuit is described; the plurality of partial vertical data blocks are combined to form the vertical data block.

In the device provided by the first aspect, the basic processing circuit is specifically configured to execute the part of the vertical data block (specifically, the part of the vertical data block or the processed part of the vertical data block) and the basic data block. An inner product processing result is obtained after one inner product processing, the inner product processing result is accumulated to obtain a partial operation result, and the partial operation result is sent to the main processing circuit.

In the device provided by the first aspect, the basic processing circuit is specifically configured to multiplex the partial vertical data block n times and execute the inner product operation of the partial vertical data block and the n basic data blocks to obtain n partial processes. As a result, n partial processing results are accumulated after obtaining n partial computing results, and the n partial computing results are sent to the main processing circuit, where n is an integer greater than or equal to two.

In the apparatus provided by the first aspect, the main processing circuit includes: a main register or a main on-chip cache circuit;

The basic processing circuit includes a basic register or a basic on-chip buffer circuit.

In the device provided by the first aspect, the main processing circuit includes: a vector operator circuit, an arithmetic logic unit circuit, an accumulator circuit, a matrix transposition circuit, a direct memory access circuit, a first mapping circuit, or a data rearrangement circuit One or any combination.

In the device provided in the first aspect, the basic processing circuit is further specifically configured to forward the vertical data block and the basic data block to other basic processing circuits to perform data processing first and then perform an inner product operation to obtain an operation result. Sending the operation result to the main processing circuit;

The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of an operation instruction.

In the device provided by the first aspect, the data block may be represented by a tensor, which may specifically be one or any combination of a vector, a matrix, a three-dimensional data block, a four-dimensional data block, and an n-dimensional data block.

In the apparatus provided by the first aspect, if the operation instruction is a multiplication instruction, the main processing circuit determines that the multiplier data block is a vertical data block, and the multiplicand data block is a horizontal data block;

Alternatively, if the operation instruction is a convolution instruction, the main processing circuit determines that the convolution input data block is a vertical data block, and the convolution kernel is a horizontal data block.

In the method provided by the sixth aspect, the operations of the neural network include: convolution operations, matrix multiplication matrix operations, matrix multiplication vector operations, paranoid operations, fully connected operations, general matrix multiplication (GEMM) operations, One or any combination of General Matrix Vector Multiplication (GEMV) operations and activation operations.

Referring to FIG. 1a, FIG. 1a is an integrated circuit chip device provided by the present disclosure. The integrated circuit chip device includes a main processing circuit and a plurality of basic processing circuits. The plurality of basic processing circuits are arranged in an array (m * n array). ), Where the range of m and n is an integer greater than or equal to 1 and at least one of m and n is greater than or equal to 2. For a plurality of basic processing circuits distributed in an m * n array, each basic processing circuit is connected to an adjacent basic processing circuit, and the main processing circuit is connected to k basic processing circuits of the multiple basic processing circuits, and the k basic processing circuits are connected to each other. The processing circuits may be: n basic processing circuits in the first row, n basic processing circuits in the m row, and m basic processing circuits in the first column. As shown in the integrated circuit chip device shown in FIG. 1 a, the main processing circuit includes a first mapping circuit, and the first mapping circuit is configured to perform compression processing on data to obtain processed data and identification data. The identification data is used to indicate whether an absolute value of the data is greater than a first threshold. Further, the main processing circuit may send only the processed data (specifically, the data whose absolute value is greater than the first threshold) and the identification data associated with the data to the basic processing circuit. The advantage is: reduce the amount of data sent to the basic processing circuit for data processing, and improve the data processing rate. The first threshold is a user- or device-side customized setting, such as 0.05, 0.5, and the like, and is not limited.

For example, the input data of the main processing circuit is a matrix data block , After processing by the first mapping circuit, the processed matrix data block can be obtained as , The identification data block associated with the matrix data block is . The specific processing of the first mapping circuit will be described in detail later.

Correspondingly, when the main processing circuit distributes data to the basic processing circuit, only two data of 1 and 0.5 can be sent, instead of sending the processed matrix data block, 8 data; at the same time, the identification of the matrix data block needs to be associated. The data blocks are sent to the basic processing circuit together, so that the basic processing circuit correspondingly learns that the two data are located in the original matrix data block according to the received identification data block and the two received data (1 and 0.5). That is, the basic processing circuit may correspondingly restore the processed matrix data block in the main processing circuit according to the received identification data block and the received data.

At least one basic processing circuit (that is, part or all of the basic processing circuits) of the plurality of basic circuits may include a second mapping circuit. Specifically, some of the plurality of basic processing circuits may include a second mapping circuit. For example, in an alternative solution, k basic processing circuits may be configured with the second mapping circuit, so that n basic processing circuits may be respectively responsible for Perform compression processing steps on the data of the m basic processing circuits in this column. This setting can improve the operation efficiency and reduce the power consumption, because for the n basic processing circuits in the first row, since it first receives the data sent by the main processing circuit, the compression processing of the received data can be reduced. The calculation amount of the subsequent basic processing circuit and the data transmission amount of the subsequent basic processing circuit are the same. For the m basic processing circuits in the first column, the configuration of the second mapping circuit also has the advantages of small calculation amount and low power consumption. In addition, according to this structure, the main processing circuit can adopt a dynamic data transmission strategy. For example, the main processing circuit broadcasts data to the m basic processing circuits in the first column, and the main processing circuit sends and distributes the data to the n basic processing circuits in the first row. data. The specific processing of the second mapping circuit will be described in detail later.

The main processing circuit is configured to perform each continuous operation in a neural network operation and transmit data to the basic processing circuit connected thereto; the above-mentioned continuous operation is not limited to: an accumulation operation, an ALU operation, an activation operation, and the like .

The plurality of basic processing circuits are configured to perform operations in the neural network in a parallel manner according to the transmitted data, and transmit the operation results to the main processing circuit through the basic processing circuit connected to the main processing circuit. The above-mentioned parallel manner for performing operations in the neural network includes, but is not limited to, inner product operations, matrix or vector multiplication operations, and the like.

The main processing circuit may include: a data sending circuit, a data receiving circuit, or an interface. The data sending circuit may integrate a horizontal data distribution circuit and a vertical data distribution circuit. Of course, in practical applications, the horizontal data distribution circuit and the vertical data distribution circuit also Can be set separately. For horizontal data, that is, data that needs to be sent to each basic processing circuit in a row direction (or horizontal direction), as shown in FIG. 1a, the horizontal data is sent to any one or more of the m lines of basic processing circuits. For vertical data, that is, data that needs to be selectively sent to some basic processing circuits in the column direction (or vertical), specifically, such as convolution operations, convolution input data of convolution operations need to be sent to all basic processing The circuit is all vertical data. The convolution kernel needs to be selectively sent to some basic data blocks, so the convolution kernel is horizontal data. The way in which the horizontal data is specifically selected for sending to that basic processing circuit can be determined specifically by the main processing circuit based on the load and other distribution methods. For vertical or horizontal data transmission, the data can be sent to each basic processing circuit in a broadcast form. (In practical applications, horizontal / vertical data is sent to each basic processing circuit through one broadcast, and horizontal / vertical data can also be sent to each basic processing circuit through multiple broadcasts. This disclosure specifically The embodiment does not limit the number of broadcasts described above). Optionally, for the above horizontal / vertical data, the main processing circuit may also be selectively sent to some basic processing circuits.

The main processing circuit (as shown in FIG. 1d) may include a register and / or an on-chip cache circuit. The main processing circuit may further include a control circuit, a vector operator circuit, an arithmetic and logic unit (ALU) circuit, and an accumulation circuit. Circuit, direct memory access (DMA) circuit, etc. Of course, in practical applications, the above main processing circuit can also be added, conversion circuit (such as matrix transposition circuit), data rearrangement circuit or activation circuit, etc. And other circuits.

Each basic processing circuit may include a basic register and / or a basic on-chip cache circuit; each basic processing circuit may further include one or any combination of an inner product operator circuit, a vector operator circuit, an accumulator circuit, and the like. The inner product operator circuit, the vector operator circuit, and the accumulator circuit may all be integrated circuits, and the inner product operator circuit, the vector operator circuit, and the accumulator circuit may also be separately provided circuits.

Optionally, the accumulator circuit for the nth basic processing circuit in the mth row can perform the accumulation operation of the inner product operation, because for the mth basic processing circuit, it can receive the product of all the basic processing circuits in this column. As a result, the accumulation operation of the inner product operation is performed by the n basic processing circuits in the m-th row, so that the calculation resources can be effectively allocated, which has the advantage of saving power consumption. This technical solution is especially applicable when the number of m is large.

For data compression processing, the main processing circuit can be used to allocate and execute the circuit. Specifically, the display or implicit mode can be used to allocate and execute the circuit. For the display mode, the main processing circuit can be configured with a special instruction or instruction. When the processing circuit receives the special instruction or instruction, it determines to perform data compression processing. For example, when the basic processing circuit does not receive the special instruction or instruction, it determines to not perform data compression processing. As another example, it can be performed in an implied manner. For example, when the basic processing circuit receives sparse data (that is, contains zero, or includes data less than a preset threshold greater than a preset amount) and determines that an inner product operation needs to be performed, the sparse The data is compressed. For the display configuration mode, a special instruction or instruction can be configured with a decreasing sequence. The value of the decreasing sequence decreases by 1 each time the decreasing sequence passes through the basic processing circuit, and the basic processing circuit reads the value of the decreasing sequence. If the value is greater than zero, it executes Data compression processing. If the value is equal to or less than zero, no data compression processing is performed. This setting is configured according to the basic processing circuit allocated by the array. For example, for the m basic processing circuits in the i-th column, the main processing circuit needs the first five basic processing circuits to perform data compression processing, and then the main processing circuit issues a A special instruction that contains a decreasing sequence. The initial value of the decreasing sequence can be 5, and the value of the decreasing sequence is reduced by 1 after each basic processing circuit. When the fifth basic processing circuit is reached, the The value is 1. When the sixth basic processing circuit is reached, the decrementing sequence is 0. At this time, the sixth basic processing circuit will not perform the data compression processing. In this way, the main processing circuit can dynamically configure the data compression processing. And the number of executions.

An embodiment of the present disclosure provides an integrated circuit chip device, including a main processing circuit (also referred to as a main unit) and a plurality of basic processing circuits (also referred to as a basic unit); the structure of the embodiment is shown in FIG. 1b; The dashed box is the internal structure of the neural network computing device; the gray-filled arrows indicate the data transmission path between the main processing circuit and the basic processing circuit array, and the hollow arrows indicate the basic processing circuits (phases) in the basic processing circuit array. (Neighboring basic processing circuits). The length, width, and length of the basic processing circuit array may be different, that is, the values of m and n may be different, and of course, they may be the same. The present disclosure does not limit the specific values of the foregoing values.

The circuit structure of the basic processing circuit is shown in Figure 1c; the dashed box in the figure indicates the boundary of the basic processing circuit, and the thick arrows crossing the dashed box indicate the data input and output channels (pointing to the dashed box is the input channel, and the dotted box is the output channel ); The rectangular box in the dashed box indicates the storage unit circuit (register and / or on-chip cache), including input data 1, input data 2, multiplication or inner product results, and accumulated data; diamond box indicates the arithmetic circuit, including multiplication or internal Product operator, adder.

In this embodiment, the neural network computing device includes a main processing circuit and 16 basic processing circuits (the 16 basic processing circuits are for illustration only, and other values may be used in practical applications);

In this embodiment, the basic processing circuit has two data input interfaces and two data output interfaces; in the subsequent description of this example, the horizontal input interface (the horizontal arrow pointing to this unit in Figure 1b) is called input 0, The vertical input interface (the vertical arrow pointing to this unit in Figure 1b) is called input 1; each horizontal data output interface (the horizontal arrow pointed out from this unit in Figure 1b) is called output 0, and the vertical The data output interface (the vertical arrow pointed out from this unit in Figure 1b) is called output 1.

The data input interface and data output interface of each basic processing circuit can be connected to different units, including the main processing circuit and other basic processing circuits;

In this example, the input 0 of the four basic processing circuits 0, 4, 8, 12 (numbered as shown in Figure 1b) is connected to the data output interface of the main processing circuit;

In this example, the input 1 of the four basic processing circuits 0,1,2,3 is connected to the data output interface of the main processing circuit;

In this example, the outputs 1 of the four basic processing circuits 12, 13, 14, 15 are connected to the data input interface of the main processing circuit;

In this example, the connection between the output interface of the basic processing circuit and the input interface of other basic processing circuits is shown in Figure 1b, which is no longer listed one by one;

Specifically, the output interface S1 of the S unit is connected to the input interface P1 of the P unit, which means that the P unit can receive data from its P1 interface and the data sent by the S unit to its S1 interface.

This embodiment includes a main processing circuit, which is connected to an external device (that is, an input interface also has an output interface), a part of the data output interface of the main processing circuit is connected to a part of the data input interface of the basic processing circuit; the main processing circuit A part of the data input interface is connected to a part of the data output interface of the basic processing circuit.

Method for using integrated circuit chip device

The data involved in the usage methods provided in this disclosure may be data after compression processing. It should be noted that the data in this application may be input neurons or weights in a neural network, which may specifically be matrix data or vector data, which is not limited in this application. That is, the data or data blocks described later in this application may be input neurons or weights in a neural network, and they may be embodied in the form of a matrix or a vector.

The data compression processing involved in this application is specifically performed in the first mapping circuit and the second mapping circuit described above. It should be understood that since the neural network is an algorithm with a high calculation amount and a high memory access, the more weights, the calculation amount and memory access will increase. In particular, for cases where the weight is small (such as 0 or a weight smaller than a set value), in order to increase the calculation rate and reduce the overhead, it is necessary to compress the data with smaller weights. In practical applications, data compression processing is applied in sparse neural networks with the most obvious effects, such as reducing the data calculation workload, reducing the data overhead, and increasing the data calculation rate.

Taking input data as an example, the specific embodiment involved in data compression processing is explained. The input data includes, but is not limited to, at least one input neuron and / or at least one weight.

In the first embodiment:

After the first mapping circuit receives the first input data (specifically, a data block to be calculated sent by the main processing circuit, such as a horizontal data block or a vertical data block, etc.), the first mapping circuit may The input data is processed to obtain the processed first input data and an identification mask data associated with the first input data, where the mask data is used to indicate whether an absolute value of the first input data is greater than a first threshold, such as 0.5, 0 and many more.

Specifically, when the absolute value of the first input data is greater than a first threshold, the input data is retained; otherwise, the first input data is deleted or the first input data is set to 0. For example, the input matrix data block is , The first threshold is 0.05, and the processed matrix data block can be obtained after the first mapping circuit processes , The identification data block (also known as the mask matrix) associated with the matrix data block is .

Further, in order to reduce the amount of data transmission, when the main processing circuit distributes data to the basic processing circuit connected to the main processing circuit, it may send the target data in the processed matrix data block (in this example, it is 1.0.06). And 0.5) and the identification data block associated with the matrix data block. In specific implementation, the main processing circuit may distribute the target data in the processed matrix data block to the basic processing circuit according to a set rule, for example, sequentially sending in row order or column order, etc. Be limited. Correspondingly, after receiving the target data and the identification data block corresponding to the target data, the basic processing circuit restores the target data to a processed matrix data block according to a set rule (such as a row order). For example, in this example, the basic processing circuit can identify the data block based on the received data (1,0.06 and 0.5) and , It can be known that the matrix data block corresponding to the data (that is, the matrix data block processed by the first mapping circuit in the main processing circuit) is .

In the embodiment of the present invention, the first input data may be a horizontal data block and / or a vertical data block.

Correspondingly, the second mapping circuit may use the identification data associated with the first input data to process the second input data to obtain the processed second input data; wherein the first input data is different from the second input data. For example, when the first input data is at least one weight value, the second input data may be at least one input neuron; or when the first input data is at least one input neuron, the The second input data may be at least one weight value.

In the embodiment of the present invention, the second input data is different from the first input data, and the second input data may be any one of the following: a horizontal data block, a basic data block, a vertical data block, and a part of a vertical data block. To the data block.

For example, when the first input data is a horizontal data block, the second input data is a partial vertical data block. Assume that the second input data is a matrix data block And correspondingly use the mask matrix in the above example After processing, the processed vertical data block is obtained as . Because in the actual application, the dimension of the matrix data block involved in the input data is relatively large, this application is only for illustration here, and it is not a limitation.

In the second embodiment:

The first mapping circuit may be configured to process the first input data and the second input data to obtain processed first input data and first identification mask data associated with the first input data, and processed second data. The input data and the second identification mask data associated with the second input data. The first mask data or the second mask data is used to indicate whether the absolute value of the first or second input data is greater than a second threshold, where the second threshold is set by the user or the device, such as 0.05, 0 and so on.

The processed first input data or the second input data may be processed input data or unprocessed input data. For example, the first input data is a horizontal data block, such as the matrix data block in the above example. . After processing by the first mapping circuit, a processed horizontal data block can be obtained. The processed horizontal data block can be the original matrix data block. , Or a matrix data block after compression processing . It should be understood that, in order to reduce the amount of data transmitted and the data processing efficiency in the basic processing circuit, it is preferable that the processed input data (such as the processed basic data block or part of the vertical data block) should be compressed After the data. Preferably, the data sent by the main processing circuit to the basic processing circuit may specifically be target data in the processed input data, and the target data may specifically be data having an absolute value greater than a preset threshold, or may be non-zero Data and more.

Accordingly, in the basic processing circuit, the second mapping circuit may obtain connection identification data according to the first identification data associated with the first input data and the second identification data associated with the second input data; the connection identification data is used for Data indicating that the absolute value of both the first input data and the second input data is greater than a third threshold, where the third threshold is a user- or device-side custom setting, such as 0.05, 0, and the like. Further, the second mapping circuit may process the received first input data and the second input data respectively according to the connection identification data, so as to obtain the processed first input data and the processed second input data.

For example, the first input data is a matrix data block , The second input data block is also a matrix data block . After processing by the first mapping circuit, a first identification data block associated with the first input data can be obtained , And the processed first input data block ; Correspondingly obtain the second identification data block associated with the second input data , The processed second input data block is . Correspondingly, in order to improve the data transmission rate, the main processing circuit can only send the target data in the processed first input data block 1,0.06 and 0.5, and the first identification data block associated with the first input data block to the main processing circuit. The basic processing circuit; at the same time, the target data 1,1.1, 0.6, 0.3, and 0.5 in the processed second input data block and the second identification data block associated with the second input data block are sent to the basic processing circuit.

Correspondingly, after receiving the data, the basic processing circuit may perform element-by-element AND operation on the first identification data block and the second identification data block through the second mapping circuit to obtain the connection identification data block. . Correspondingly, the second mapping circuit uses the connection identification data block to separately process the processed first input data block and the processed second input data block, thereby obtaining the processed first input data block as , The processed second input data block is . The basic processing circuit may determine the first data block corresponding to the target data according to the first identification data block and the target data in the received first data block (that is, the first data block processed by the first mapping circuit). (Data block); correspondingly, according to the second identification data block and the target data in the received second data block, determine the second data block corresponding to the target data (that is, the second data processed by the first mapping circuit) Block); then, after the second mapping circuit learns the connection identification data block, use the connection identification data block to perform an element-wise AND operation with the determined first data block and the determined second data block, respectively, to obtain The first data block processed by the circuit and the second data block processed.

In the third embodiment:

The first processing circuit is not provided in the main processing circuit, but the main processing circuit may send the third input data and the third identification data associated with the pre-stored third input data to the basic processing circuit connected thereto. in. A second mapping circuit is provided in the basic processing circuit. A specific embodiment of the data compression processing involved in the second mapping circuit is described below.

It should be understood that the third input data includes, but is not limited to, a basic data block, a partial vertical data block, a vertical data block, and the like. Similarly, in the neural network processor, the third input data may also be at least one weight value and / or at least one input nerve, which is not limited in this application.

In the second mapping circuit, the second mapping circuit may process the third input data according to the third identification data associated with the received third input data, so as to obtain the processed third input data for subsequent subsequent The processed third input data performs related operation operations, such as an inner product operation.

For example, the third input data received by the second mapping circuit is a matrix data block. , And the corresponding pre-stored third identification data block (also referred to as a mask matrix data block) of the third input data is . Further, the second mapping circuit processes the third input data block according to the third identification data block to obtain a processed third input data block, which is specifically: .

In addition, the input neuron and output neuron mentioned in the embodiments of the present invention do not refer to the neurons in the input layer and the neurons in the output layer of the entire neural network, but to any two adjacent layers in the neural network. Neuron, the neuron in the lower layer of the network feedforward operation is the input neuron, and the neuron in the upper layer of the network feedforward operation is the output neuron. Taking a convolutional neural network as an example, suppose a convolutional neural network has L layers, K = 1,2,3 ... L-1. For the Kth and K + 1th layers, the Kth layer is called the input Layer, the neurons in this layer are the above-mentioned input neurons, the K + 1 layer is called the input layer, and the neurons in this layer are the above-mentioned output neurons, that is, each layer can be used as the input layer except the top layer The next layer is the corresponding output layer.

In the fourth implementation:

A mapping circuit is not provided in the main processing circuit, and a first mapping circuit and a second mapping circuit are provided in the basic processing circuit. For the data processing of the first mapping circuit and the second mapping circuit, reference may be made to the foregoing first embodiment to the third embodiment, and details are not described herein again.

Optionally, there is a fifth embodiment. In a fifth embodiment, a mapping circuit is not provided in the basic processing circuit, and both the first mapping circuit and the second mapping circuit are provided in a main processing circuit. Regarding the first mapping circuit and the second mapping circuit For specific data processing, refer to the foregoing first embodiment to the third embodiment, and details are not described herein again. That is, the data compression processing is completed in the main processing circuit, and the processed input data is sent to the basic processing circuit, so that the basic processing circuit uses the processed input data (specifically, it can be processed neurons and processed weights) Perform the corresponding operation.

The specific structure diagram of the mapping circuit involved in the present application is described below. Figures 4a and 4b show two possible mapping circuits. The mapping circuit shown in FIG. 4a includes a comparator and a selector. The number of the comparators and selectors is not limited in this application. As shown in Fig. 4a, a comparator and two selectors are shown, wherein the comparator is used to determine whether the input data meets a preset condition. The preset condition may be set by the user or the device. For example, the absolute value of the input data described above in this application is greater than or equal to a preset threshold. If the preset condition is satisfied, the comparator may determine that the input data is allowed to be output, and the input data corresponds to the associated identification data is 1; otherwise, it may be determined that the input data is not output or the input data is 0 by default. Accordingly, at this time, the corresponding identification data corresponding to the input data is 0. That is, after the comparator, the identification data associated with the input data can be obtained.

Further, after the comparator determines the preset conditions on the input data, the obtained identification data may be input into the selector, so that the selector uses the identification data to decide whether to output corresponding input data, that is, to obtain processing. After the input data.

As shown in FIG. 4a, taking the input data as a matrix data block as an example, each of the data in the matrix data block can be determined by a comparator, so as to obtain an identification data block associated with the matrix data block ( mask matrix). Further, in the first selector, the identification data block may be used to filter the matrix data block, and data having an absolute value greater than or equal to a preset threshold (that is, meeting a preset condition) in the matrix data block is retained. , The remaining data is deleted to output the processed matrix data block. Optionally, in the second selector, the identification data block may also be used to process other input data (for example, the second matrix data block), for example, perform an element-wise AND operation to absolute values in the second matrix data block. Data that is greater than or equal to a preset threshold is retained to output a processed second matrix data block.

It should be understood that, corresponding to the foregoing first and second embodiments, the specific structure of the first mapping circuit may include at least one comparator and at least one selector, such as the comparator and the first in FIG. 4a in the above example. Selector; the specific result of the second mapping circuit may include one or more selectors, such as the second selector in FIG. 4a in the above example.

As shown in FIG. 4b, a schematic structural diagram of another mapping circuit is shown. As shown in FIG. 4b, the mapping circuit includes a selector, and the number of the selectors is not limited, and may be one or multiple. Specifically, the selector is configured to select the inputted input data according to the identification data associated with the inputted input data to output data whose absolute value is greater than or equal to a preset threshold, and the rest The data is deleted / not output to obtain the processed input data.

Taking the input data as a matrix data block as an example, the matrix data block and an identification data block associated with the matrix data block are input to the mapping circuit, and the selector may select the matrix data block according to the identification data block. The data whose absolute value is greater than or equal to 0 is output, and the remaining data is not output, so that the processed matrix data block is output.

It should be understood that the structure shown in FIG. 4b may be applied to the second mapping circuit in the third embodiment, that is, the specific result of the second mapping circuit in the third embodiment may include at least one selector. Similarly, the first mapping circuit and the second mapping circuit designed in the main processing circuit and the basic processing circuit may be cross-combined or divided according to the functional components shown in FIG. 4a and FIG. 4b, which is not limited in this application.

Based on the foregoing embodiments, the operation processing to be completed in the main processing circuit and the basic processing circuit will be specifically described below, which can be performed using the following methods:

The main processing circuit first enables the first mapping circuit to process the first input data to obtain the processed first input data and the first identification data associated with the first input data; then, the processed first input data and The first identification data associated with the first input data is transmitted to a basic processing circuit for calculation. For example, the main processing circuit can process the data to be calculated (such as horizontal data blocks / vertical data blocks) before transmitting them to the basic processing circuit. The advantages are that the bit width of the transmitted data can be reduced, and the total number of bits transmitted can be reduced. The basic processing circuit performs data operations with smaller bit widths with higher efficiency and lower power consumption.

The basic processing circuit enables the second mapping circuit to use the first identification data to process the received second input data to obtain the processed second input data and then perform related operations on the processed first input data and the second input data. operating. For example, the basic processing circuit receives the second input data (such as sparse data and vertical data blocks) transmitted from the main processing circuit, and then compresses it before performing operations to improve the operation efficiency and reduce power consumption.

Optionally, the main processing circuit may first convert the first input data (such as a basic data block), the first identification data associated with the first input data, the second input data (such as a partial vertical data block, etc.), and the second input data. The associated second identification data is first transmitted to the basic processing circuit for operation.

Accordingly, after receiving the data, the basic processing circuit may first enable the second mapping circuit to obtain the connection identification data block according to the first identification data and the second identification data, and then use the connection identification data to pair the first input data and the second input data. By performing processing, further, the operation of the first input data and the second input data after the processing can be completed in the basic processing circuit, which has the advantages of reducing the amount of data operation, improving the operation efficiency, and reducing the power consumption.

Optionally, the first identification data associated with the first input data and the second identification data associated with the second input data sent by the main processing circuit are stored in the main processing circuit in advance, or the first processing data is enabled for the main processing circuit. A mapping circuit is obtained through the first / second input data, which is not limited in this application.

How to use the basic processing circuit (see Figure 2a);

The main processing circuit receives input data to be calculated from outside the device;

Optionally, the main processing circuit uses various arithmetic circuits, vector arithmetic circuits, inner product arithmetic circuits, and accumulator circuits of the unit to perform arithmetic processing on the data;

The main processing circuit sends data to the basic processing circuit array (referring to the collection of all basic processing circuits as the basic processing circuit array) through the data output interface (as shown in Figure 2b);

The method for sending data here may be sending data directly to a part of the basic processing circuit, that is, multiple broadcast mode;

The way of sending data here can send different data to different basic processing circuits, that is, the distribution mode;

The basic processing circuit array calculates the data;

The basic processing circuit performs operations after receiving the input data;

Optionally, after receiving the data, the basic processing circuit transmits the data from the data output interface of the unit; (transmit to other basic processing circuits that do not receive data directly from the main processing circuit.)

Optionally, the basic processing circuit transmits the operation result from the data output interface; (intermediate calculation result or final calculation result)

The main processing circuit receives the output data returned from the basic processing circuit array;

Optionally, the main processing circuit continues to process the data received from the basic processing circuit array (such as an accumulation or activation operation);

The main processing circuit finishes processing and transmits the processing result from the data output interface to the outside of the device.

Use the circuit device to complete the tensor multiplication tensor operation. The tensor is the same as the data block described above, and it can be any of a matrix, a vector, a three-dimensional data block, a four-bit data block, and a high-dimensional data block. A combination of terms or multiple items; the specific implementation methods of matrix multiplying vectors and matrix multiplying matrix operations are shown in Figures 2c and 2f, respectively.

Use the circuit device to complete matrix multiplying vector operations; (matrix multiplying vectors can be the inner product operation of each row in the matrix with the vector, and these results are placed into a vector in the order of the corresponding rows.)

The following describes the calculation of a multiplication of a matrix S of size M rows and L columns and a vector C of length L, as shown in Figure 2c below.

This method uses all or part of the basic processing circuits of the neural network computing device, assuming that K basic processing circuits are used;

The main processing circuit sends data in part or all of the rows of the matrix S to each of the k basic processing circuits;

In an optional solution, the control circuit of the main processing circuit sends data of a row in the matrix S one number or a part of the number to a basic processing circuit at a time; (for example, for sending a number each time, it can be A certain basic processing circuit sends the first number of the third row of data for the first time, the second number of the third row of data for the second time, the third number of the third row for the third time ..., or Each time a part of the number is sent, the first two numbers of the third line (ie, the first and second numbers) are sent for the first time, the third and third numbers of the third line are sent for the second time, and the third and third lines are sent. 5 and 6th numbers ...;)

In an optional solution, the control circuit of the main processing circuit sends data of some rows in the matrix S to each basic processing circuit at a time; (for example, for a basic processing circuit, the first Send the 3rd, 4th, and 5th lines of the first number each time, send the 2nd number of the 3rd, 4th, and 5th lines each time, and send the 3rd, 4th, and 5th lines each time The third number of ........., or the first time to send the first two numbers of each line 3,4,5, the second time to send the third and fourth numbers of each line 3,4,5, the third Send the 3rd, 4th, and 5th lines 5th and 6th numbers of each line ...)

The control circuit of the main processing circuit sequentially sends the data in the vector C to the 0th basic processing circuit;

After the 0th basic processing circuit receives the data of the vector C, it sends the data to the next basic processing circuit connected to it, namely, the basic processing circuit 1.

Specifically, some basic processing circuits cannot directly obtain all the data required for calculation from the main processing circuit. For example, the basic processing circuit 1 in FIG. 2d has only one data input interface connected to the main processing circuit, so it can only be obtained directly from the main processing circuit. The main processing circuit obtains the data of the matrix S, and the data of the vector C needs to be output to the basic processing circuit 1 by the basic processing circuit 0. Similarly, the basic processing circuit 1 also continues to output the data of the vector C after receiving the data. To the base processing circuit 2.

Each basic processing circuit performs operations on the received data, including, but not limited to, inner product operations, multiplication operations, addition operations, and the like;

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result is transmitted from the data output interface (that is, to other basic processing circuits connected to it);

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

The main processing circuit receives the result of the inner product operation of each basic processing circuit, and processes the result to obtain the final result (the processing may be an accumulation operation or an activation operation, etc.).

An embodiment of a method for implementing a matrix multiplication vector by using the above calculation device:

In an optional solution, the multiple basic processing circuits used in the method are arranged in the manner shown in FIG. 2d or FIG. 2e below;

As shown in FIG. 2c, the main processing circuit can respectively obtain a mask matrix corresponding to each of the matrix S and the matrix P (that is, the identification data / identification data block described above). Specifically, the respective mask matrices of the matrix S and the matrix P may be stored in a high-speed memory in the main processing circuit in advance; or the main processing circuit may enable the first mapping circuit to obtain the corresponding corresponding ones obtained from the matrix S and the matrix P, respectively. mask matrix. The control circuit of the main processing unit divides the M rows of data of the matrix S into K groups, and the i-th basic processing circuit is responsible for the operation of the i-th group (the set of rows in this group is denoted as Ai); accordingly, the main processing unit's The control circuit also divides the M rows of the first mask matrix corresponding to the matrix S into K groups, and sends them to the corresponding basic processing circuit together with the newly formed matrix after the matrix S is divided into K groups. The processing circuit completes the operation of related data.

The method for grouping M rows of data here is any grouping method that will not be repeatedly allocated;

In an optional solution, the following allocation method is adopted: the jth row is allocated to the j% K (% is the remainder operation) basic processing circuits;

In an alternative, for a case where the grouping cannot be evenly divided, a part of the rows may be evenly distributed first, and the remaining rows may be distributed in an arbitrary manner.

The control circuit of the main processing circuit sends the data in some or all of the rows of the matrix S to the corresponding basic processing circuit in turn each time; correspondingly, the control circuit also corresponds to the first mask matrix of the rows of data in the matrix S The identification data in is sent to the corresponding basic processing circuit together.

For example, if the matrix S is a matrix data block of 50 * 50, the main processing circuit can divide the matrix S into 10 small matrices, and the size of each small matrix is 5 * 50, then the main processing circuit can divide the first small matrix S0 (5 rows and 50 columns) and the identification data block (5 rows and 50 columns) associated with the small matrix S0 are sent to the first basic processing circuit together to complete the related data calculation processing in the first basic processing circuit.

In an optional solution, the control circuit of the main processing circuit sends one or more data in a row of data of the i-th group of data Mi to the i-th basic processing circuit each time, and the i-th group of data Mi may Data in the matrix S, or data in the first mask matrix corresponding to the matrix S;

In an optional solution, the control circuit of the main processing circuit sends one or more data of each of some or all of the i-th group of data Mi to the i-th basic processing circuit each time;

The control circuit of the main processing circuit sequentially sends the data in the vector C to the first basic processing circuit; correspondingly, the control circuit of the main processing circuit can also sequentially send the data in the second mask matrix associated with the vector C to the first 1 basic processing circuit.

In an optional solution, the control circuit of the main processing circuit may send one or more data in the vector C or the second mask matrix associated with the vector C each time;

After receiving the data of the vector C or the second mask matrix, the i-th basic processing circuit may also send it to the i + 1-th basic processing circuit connected to it;

After each basic processing circuit receives one or more data from a row or rows in the matrix S and one or more data from the vector C, it performs operations (including but not limited to multiplication or addition);

In specific implementation, each basic processing circuit receives the data in the matrix S and the first identification data associated with the data in the first mask matrix, the data in the vector C, and the second associated with the data in the second mask data. After the identification data, the connection identification data may be obtained first according to the first identification data and the second identification data; and then the connection identification data is used to decide whether to perform related operation operations on the data in the matrix P and the data in the vector C. The connection identification data is obtained by performing an AND operation on the first identification data and the second identification data, and may be 0 or 1, where 1 indicates that data at a position in the matrix S and data at the same position in the vector C are both Data whose absolute value is greater than a preset threshold; on the contrary, 0 indicates that data at the same position in the matrix S and / or data at the same position in the vector C are data whose absolute value is less than or equal to the preset threshold.

That is, each basic processing circuit starts a second mapping circuit to select the identification data in the same position as 1 according to the first mask matrix of the matrix S and the second mask matrix of the vector C, and perform related operations on the data in the matrix S and the vector C. Operations, such as multiplication, addition, and so on. That is, the correspondence between the first mask matrix and the second mask matrix is used to select data in the same position in the matrix S and the matrix P whose absolute value is greater than a preset threshold to perform a related operation, such as a multiplication operation.

For example, the basic processing circuit receives the data of some two rows in the matrix S as the matrix S0 Corresponding to the first mask matrix associated with this matrix S0 ; Received some data in vector C is vector C0 , The second mask vector associated with this vector C0 ; Further basic processing circuit can enable the second mapping circuit to first with Perform element-by-element AND operation to obtain the connection mask matrix , Further use the connection mask matrix to process the received matrix S0 and the vector C0 to obtain a processed matrix S0 And the processed vector C0 So that the basic processing circuit performs related operation operations on the processed matrix S0 and the processed vector C0.

In an optional solution, if the data received in each basic processing circuit (specifically, it may be a data block to be calculated, such as data of some rows / columns in matrix S or vector C, and identification data corresponding to the mask matrix) ) When the amount of data exceeds a preset threshold, the basic processing circuit will no longer receive new input data. For example, the main processing circuit will send the data of some rows / columns of matrix S or vector C and the data corresponds to the mask matrix. The identification data in the server, etc., until there is sufficient buffer / storage space in the basic processing circuit, and then receive the newly sent data from the main processing circuit.

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the data received by the basic processing circuit may also be intermediate results, which are stored in registers and / or on-chip buffers;

The basic processing circuit transmits the local calculation result to the next basic processing circuit or main processing circuit connected to it;

In an optional solution, corresponding to the structure of FIG. 2d, only the output interface of the last basic processing circuit of each column is connected to the main processing circuit. In this case, only the last basic processing circuit can directly connect the local The calculation results are transmitted to the main processing circuit. The calculation results of other basic processing circuits must be passed to their next basic processing circuit, and the next basic processing circuit is passed to the next basic processing circuit until all of them are transmitted to the last basic processing circuit. The last basic processing circuit performs an accumulation calculation on the local calculation result and the results of other basic processing circuits received in this column to obtain an intermediate result, and sends the intermediate result to the main processing circuit. Of course, the last basic processing circuit can also send The results of the other basic circuits in this column and the local processing results are sent directly to the main processing circuit.

In an optional solution, corresponding to the structure of FIG. 2e, each basic processing circuit has an output interface connected to the main processing circuit. In this case, each basic processing circuit directly transmits the local calculation result to Main processing circuit

After the basic processing circuit receives the calculation results transmitted from other basic processing circuits, it transmits it to the next basic processing circuit or main processing circuit connected to it.

The main processing circuit receives the results of the M inner product operations as the operation results of the matrix multiplication vector.

Using the circuit device to complete a matrix multiplication matrix operation;

The following describes the calculation of a multiplication of a matrix S whose size is M rows and L columns and a matrix P whose size is L rows and N columns. (Each row in matrix S has the same length as each column of matrix P, as shown in Figure 2f.)

The method is described by using the embodiment of the device as shown in FIG. 1b;

The first mapping circuit of the main processing circuit obtains the identification mask matrices corresponding to the matrix S and the matrix P respectively. For example, the first mapping circuit is activated to process the matrix S and the matrix P to obtain the first mask matrix corresponding to the matrix S and the matrix. A second mask matrix corresponding to P;

The control circuit of the main processing circuit sends the data in some or all of the rows of the matrix S to those basic processing circuits that are directly connected to the main processing circuit through the horizontal data input interface (for example, the gray-filled vertical data at the top in Figure 1b) Path); at the same time, the control circuit will also send the identification data corresponding to some or all of the rows in the first mask matrix to the basic processing circuit connected to it. For example, the control circuit sends the first two rows of data in the matrix S and the first two rows of identification data corresponding to the first two rows of data in the first mask matrix to the basic circuit connected to the main processing circuit.

In an optional solution, the control circuit of the main processing circuit sends data of a row in the matrix S one number or a part of the number to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, the first transmission The first number in the third line, the second number in the third line of data is sent for the second time, the third number in the third line is sent in the third time ..., or the first two numbers in the third line are sent for the first time , Send the 3rd and 4th numbers of the 3rd line for the second time, send the 5th and 6th numbers of the 3rd line for the third time ...;)

Correspondingly, the control circuit also sends the identification data corresponding to the corresponding row in the first mask matrix in the matrix S to the basic processing circuit one or a part of the identification data at a time.

In an optional solution, the control circuit of the main processing circuit sends the data of some rows in the matrix S and the identification data corresponding to the corresponding rows in the first mask matrix to the basic processing at a time. Circuit; (for example, for a basic processing circuit, the first number of each line of 3,4,5 is sent for the first time, and the second number of each line of 3,4,5 is sent for the second time, Send the 3rd number of 3, 4, 5 lines for the third time ... or send the first 2 numbers of 3, 4, 5 lines for the first time, and send the 3, 4, 5 for the second time 3rd and 4th numbers per line, 3rd, 4th, 5th lines send 5th and 6th numbers per line ...;)

The control circuit of the main processing circuit sends data in some or all of the columns in the matrix P to those basic processing circuits that are directly connected to the main processing circuit through the vertical data input interface (for example, the gray to the left of the basic processing circuit array in Figure 1b) Filled horizontal data path); at the same time, the control circuit will also send the identification data corresponding to some or all of the rows in the second mask matrix to the basic processing circuit connected to it. For example, the control circuit sends the first two rows of data in the matrix P and the first two rows of identification data corresponding to the first two rows in the second mask matrix to the basic circuit connected to the main processing circuit.

In an optional solution, the control circuit of the main processing circuit sends data of a column in the matrix P one number or a part of the number to a basic processing circuit at a time; (for example, for a basic processing circuit, the first sending The first number in the third column, the second number in the third column of data is sent for the second time, the third number in the third column is sent for the third time ..., or the first two numbers in the third column are sent for the first time. , Send the 3rd and 3rd numbers of the 3rd column for the second time, and send the 5th and 6th numbers of the 3rd column for the third time ...;) correspondingly, the control circuit will also correspond to the row in the matrix P at the same time The identification data of the corresponding row in the second mask matrix sends one or a part of the identification data to a certain basic processing circuit at a time.

In an optional solution, the control circuit of the main processing circuit sends the data of some columns in the matrix P and the identification data corresponding to the corresponding rows in the second mask matrix to each of them one at a time to a certain basic processing. Circuit; (for example, for a basic processing circuit, the first number of columns 3, 4, 5 is sent for the first time, and the second number of columns 3, 4, 5 is sent for the second time, Send the 3rd number of each 3rd, 4th, 5th column for the third time ... or send the first 2 numbers of each 3rd, 4th, 5th column for the first time, and the 3rd, 4th, 5th for the second time 3rd and 4th numbers per column, 3rd, 4th, 5th columns each send 5th and 6th numbers ...;)

After receiving the data of the matrix S and the identification data of the first mask matrix associated with the matrix S, the basic processing circuit passes the data (specifically, the data of the matrix S and the identification data corresponding to the data in the first mask matrix). The horizontal data output interface is transmitted to its next basic processing circuit (for example, the white filled horizontal data path in the middle of the basic processing circuit array in Figure 1b); after receiving the data of matrix P, the basic processing circuit sends the data Transmitted through its vertical data output interface to the next basic processing circuit connected to it (for example, the white-filled vertical data path in the middle of the basic processing circuit array in Figure 1b);

Each basic processing circuit performs operations on the received data; specifically, each basic processing circuit receives data of a certain row or rows in the matrix S and the data corresponds to the first identification data associated with the first mask matrix, After the data of one or more columns in the matrix P and the data correspond to the second identification data associated in the second mask data; the connection identification data can be obtained according to the first identification data and the second identification data; and then the connection identification is used The data decides whether to perform a correlation operation on the data in the matrix S and the data in the matrix P. The connection identification data is obtained by performing an AND operation on the first identification data and the second identification data, and may be 0 or 1, where 1 indicates that data at a position in the matrix S and data at the same position in the matrix P are both Data whose absolute value is greater than a preset threshold; conversely, 0 indicates that data at a certain position in matrix S and / or data at the same position in matrix P are data whose absolute value is less than or equal to a preset threshold. For details, refer to the foregoing embodiments, and details are not described herein again.

That is, each basic processing circuit activates a second mapping circuit to select the data with the identification data of 1 in the same position according to the first mask matrix of the matrix S and the second mask matrix of the matrix P to perform related arithmetic operations, such as multiplication and addition operations, etc. Wait.

In an optional solution, if the data received in each basic processing circuit (specifically, it may be a data block to be calculated, such as data of some rows / columns in matrix S or matrix P, and identification data corresponding to the mask matrix) ) When the amount of data exceeds a preset threshold, the basic processing circuit will no longer receive new input data. For example, the main processing circuit will send the data of some rows / columns of matrix S or matrix P and the data corresponds to the mask matrix. The identification data in the server, etc., until there is sufficient buffer / storage space in the basic processing circuit, and then receive the newly sent data from the main processing circuit.

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result can be transmitted from the data output interface;

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

Specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface; if not, the result is output to the direction of the basic processing circuit that can directly output to the main processing circuit (for example, FIG. In 1b, the bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transmit the calculation results downward from the vertical output interface).

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

Output the result in a direction that can be directly output to the main processing circuit (for example, in Figure 1b, the bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transfer the calculation from the vertical output interface downward. result);

The main processing circuit receives the result of the inner product operation of each basic processing circuit and can obtain the output result.

An embodiment of the "matrix by matrix" method:

The method uses a basic processing circuit array arranged in a manner as shown in FIG. 1b;

The first mapping circuit of the main processing circuit obtains the identification mask matrices corresponding to the matrix S and the matrix P respectively. For example, the first mapping circuit is activated to process the matrix S and the matrix P to obtain the first mask matrix corresponding to the matrix S and the matrix. The second mask matrix corresponding to P is optional, and the processed matrix S and matrix P can also be obtained. Assuming that the processed matrix S has h rows, the control circuit that processes the main processing circuit divides the data of h rows of the matrix S into h Group, the i-th basic processing circuit is responsible for the operation of the i-th group (the set of rows in the set of data is denoted as Hi); at the same time, the control circuit will also correspond to the data in some or all of the rows in the first mask matrix. The identification data is sent to the underlying processing circuit connected to it. For example, the control circuit sends the first two rows of data in the matrix S and the first two rows of identification data corresponding to the first two rows of data in the first mask matrix to the basic circuit connected to the main processing circuit.

Here, the method for grouping the data of h rows is any grouping method that will not be repeatedly allocated;

In an optional solution, the following allocation method is adopted: the control circuit of the main processing circuit allocates the jth row to the j% h basic processing circuits;

In an alternative, for a case where the grouping cannot be evenly divided, a part of the rows may be evenly distributed first, and the remaining rows may be distributed in an arbitrary manner.

The control circuit of the main processing circuit divides the W column data of the matrix P into w groups, and the i-th basic processing circuit is responsible for the operation of the i-th group (the set of rows in the data is denoted as Wi); accordingly, the control circuit also simultaneously The identification data corresponding to the corresponding column in the second mask matrix in the matrix P is sent one or a part of the identification data to a certain basic processing circuit at a time.

The method for grouping W column data here is any grouping method that will not be repeatedly assigned;

In an optional solution, the following allocation method is adopted: the control circuit of the main processing circuit divides the jth row to the j% wth basic processing circuits;

In an optional solution, for a case where the grouping cannot be evenly divided, a part of the columns may be evenly distributed first, and the remaining columns may be allocated in an arbitrary manner.

The control circuit of the main processing circuit sends data in part or all of the rows of the matrix S to the first basic processing circuit of each row in the basic processing circuit array;

In an alternative, the control circuit of the main processing circuit sends one or more of a row of data in the i-th group of data Hi to the first basic processing circuit in the i-th row of the basic processing circuit array each time. Data; at the same time, the identification data corresponding to the i-th group of data Hi in the mask matrix can also be sent to the first basic processing circuit by using the same method;

In an optional solution, the control circuit of the main processing circuit sends each of some or all of the i-th group of data Hi to the first basic processing circuit in the i-th row of the basic processing circuit array. At the same time, the identification data corresponding to the i-th group of data Hi in the mask matrix can also be sent to the first basic processing circuit by using the same method;

The control circuit of the main processing circuit sends data in some or all of the columns of the matrix P to the first basic processing circuit of each column in the basic processing circuit array; at the same time, the control circuit will also send the portion corresponding to the second mask matrix Or the identification data in all rows is sent to the underlying processing circuit connected to it. For example, the control circuit sends the first two rows of data in the matrix P and the first two rows of identification data corresponding to the first two rows in the second mask matrix to the basic circuit connected to the main processing circuit.

In an optional solution, the control circuit of the main processing circuit sends one or more of a column of data in the i-th group of data Wi that it is responsible to the first basic processing circuit in the i-th column of the basic processing circuit array. data;

In an optional solution, the control circuit of the main processing circuit sends each of some or all of the i-th group of data Ni to the first basic processing circuit in the i-th column of the basic processing circuit array. One or more data;

After the basic processing circuit receives the data of the matrix S, it transmits the data to its next basic processing circuit through its horizontal data output interface (for example, the white-filled horizontal data path in the middle of the basic processing circuit array in Figure 1b). ); After receiving the data of matrix P, the basic processing circuit transmits the data to the next basic processing circuit connected to it through its vertical data output interface (for example, the white filled pad in the middle of the basic processing circuit array in Figure 1b) Vertical data path);

Each basic processing circuit performs operations on the received data; specifically, each basic processing circuit receives data of a certain row or rows in the matrix S and the data corresponds to the first identification data associated with the first mask matrix, After the data of one or more columns in the matrix P and the data correspond to the second identification data associated in the second mask data; the connection identification data can be obtained according to the first identification data and the second identification data; and then the connection identification is used The data decides whether to perform a correlation operation on the data in the matrix S and the data in the matrix P. The connection identification data is obtained by performing an AND operation on the first identification data and the second identification data, and may be 0 or 1, where 1 indicates that data at a position in the matrix S and data at the same position in the matrix P are both Data whose absolute value is greater than a preset threshold; conversely, 0 indicates that data at a certain position in matrix S and / or data at the same position in matrix P are data whose absolute value is less than or equal to a preset threshold. For details, refer to the foregoing embodiments, and details are not described herein again.

That is, each basic processing circuit activates a second mapping circuit to select the data with the identification data of 1 in the same position according to the first mask matrix of the matrix S and the second mask matrix of the matrix P to perform related arithmetic operations, such as multiplication and addition operations, etc. Wait.

In an optional solution, if the data received in each basic processing circuit (specifically, it may be a data block to be calculated, such as data of some rows / columns in matrix S or matrix P, and identification data corresponding to the mask matrix) ) When the amount of data exceeds a preset threshold, the basic processing circuit will no longer receive new input data. For example, the main processing circuit will send the data of some rows / columns of matrix S or matrix P and the data corresponds to the mask matrix. The identification data in the server, etc., until there is sufficient buffer / storage space in the basic processing circuit, and then receive the newly sent data from the main processing circuit.

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result can be transmitted from the data output interface;

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

Specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface; if not, the result is output to the basic processing circuit that can directly output to the main processing circuit (for example, the most The bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transmit the calculation results downward from the vertical output interface).

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

Output the result in a direction that can be directly output to the main processing circuit (for example, the bottom row of the basic processing circuit outputs its output result directly to the main processing circuit, and the other basic processing circuits transmit the calculation result downward from the vertical output interface);

The main processing circuit receives the result of the inner product operation of each basic processing circuit and can obtain the output result.

The words "horizontal" and "vertical" used in the above description are just to describe the example shown in Figure 1b. In actual use, only the "horizontal" and "vertical" interfaces of each unit need to be distinguished to represent two different interfaces. can.

Use the circuit device to complete a fully connected operation:

If the input data of the fully connected layer is a vector (ie, the input of the neural network is a single sample), then the weight matrix of the fully connected layer is used as the matrix S, and the input vector is used as the vector C. Perform operations in a vector method;

If the input data of the fully connected layer is a matrix (that is, when the input of the neural network is multiple samples), then the weight matrix of the fully connected layer is used as the matrix S, and the input vector is used as the matrix P, or the weight of the fully connected layer is used. The value matrix is used as the matrix P, the input vector is used as the matrix S, and the operation is performed according to the matrix of the device multiplied by the matrix;

Use the circuit device to complete a convolution operation:

The convolution operation is described below. A square in the figure below represents a piece of data, and the input data is represented in Figure 3a (N samples, each sample has C channels, and the feature map of each channel has a height of H and a width of W). The weights, ie, the convolution kernels, are shown in Figure 3b (there are M convolution kernels, each convolution kernel has C channels, and the height and width are KH and KW, respectively). For N samples of the input data, the rules of the convolution operation are the same. The process of performing the convolution operation on one sample is explained below. On one sample, each of the M convolution kernels must be the same. Operation, each convolution kernel operates to obtain a planar feature map, and M convolution kernels finally calculate to obtain M planar feature maps (for a sample, the output of the convolution is M feature maps), for a convolution kernel , To perform an inner product operation at each plane position of a sample, and then slide in the H and W directions. For example, Figure 3c shows a convolution kernel that performs an inner product operation at the lower right corner of a sample of input data. Correspondence map; Figure 3d shows the position of the convolution slide one grid to the left and Figure 3e shows the position of the convolution slide one grid up.

The method is described by using the embodiment of the device as shown in FIG. 1b;

The first mapping circuit of the main processing circuit may process data in part or all of the weighted convolution kernel to obtain corresponding mask data and processed weighted data (that is, part or all of the processed weighted convolution) Data in the core).

The control circuit of the main processing circuit sends part or all of the data in the convolution kernel (the data can be the original weight data or the processed weight data) to the direct connection to the main processing circuit through the horizontal data input interface Those basic processing circuits (for example, the gray-filled vertical data path at the top of Figure 1b); at the same time, the control circuit also sends the mask data corresponding to the data to the basic processing circuit connected to the main processing circuit. ;

In an optional solution, the control circuit of the main processing circuit sends data of a certain convolution kernel in the weight value to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, the first Send the first number of the 3rd line once, send the 2nd number of the 3rd line data for the 2nd time, send the 3rd number of the 3rd line for the 3rd time ... or before the 3rd line of the 1st time Two numbers, the 3rd and 4th numbers of the 3rd line are sent for the second time, and the 5th and 6th numbers of the 3rd line are sent for the third time ...) At the same time, the control circuit convolves a certain one of the weights The mask data corresponding to the core also uses the above-mentioned occurrence of a number or a part of the data to that basic processing circuit;

In another alternative, the control circuit of the main processing circuit sends the data of some convolution kernels in the weight value to the basic processing circuit one by one each time; (for example, for a certain A basic processing circuit that sends the first number of each line 3,4,5 for the first time, the second number of each line 3,4,5 for the second time, and the third number of 3, The 3rd number of each line 4,5 ... or the first two numbers of 3,4,5 lines are sent for the first time, the 3rd and 4th of 5th lines are sent for the second time 4 numbers, send the 3rd, 4th, 5th lines for the 5th and 6th numbers of each line for the third time ...;) Correspondingly, the control circuit will associate the masks corresponding to some convolution kernels in the weight The data also uses the same method described above to generate a number or a part of the data each time to that basic processing circuit;

The control circuit of the main processing circuit divides the input data according to the position of the convolution. The control circuit of the main processing circuit sends the data in some or all of the convolution positions in the input data to the main processing circuit directly through the vertical data input interface. Those connected basic processing circuits (for example, the gray-filled horizontal data path on the left of the basic processing circuit array in Figure 1b); correspondingly, the control circuit will also divide the mask data associated with the input data according to the position of the convolution Correspondingly, the control circuit also sends the mask data corresponding to the data in some or all of the convolution positions in the input data to the basic processing circuit electrically connected to the main processing circuit together;

In an optional solution, the control circuit of the main processing circuit sends data of a convolution position in the input data and mask data corresponding to the data to a basic processing circuit at a time or a part of the data; (for example, For a basic processing circuit, the first number in the third column is transmitted for the first time, the second number in the third column of data is transmitted for the second time, and the third number in the third column is transmitted for the third time ..., Or send the first two numbers in the third column for the first time, send the third and fourth numbers in the third column for the second time, and send the fifth and sixth numbers in the third column for the third time ...;)

In another alternative, the control circuit of the main processing circuit sends the data of some convolution positions in the input data and the mask data corresponding to the data each time to send a number or a part of the number to a certain Basic processing circuit; (for example, for a basic processing circuit, the first number of each column of 3, 4, 5 is sent for the first time, and the second number of each column of 3, 4, 5 is sent for the second time Number, the 3rd number of each column in the 3rd, 4th, and 5th columns is sent for the 3rd time ... or the first 2nd number of each column in the 3rd, 4th, and 5th columns is sent, and the 3rd and 4th numbers are sent for the second time , 5th and 3rd and 4th numbers in each column, 3rd, 4th, 5th and 5th and 6th numbers in each column ...;)

After the basic processing circuit receives the weight data (specifically, the data of the convolution kernel in the weight (referred to as the weight data) or the mask data corresponding to the weight data), it outputs the data through its horizontal data. The interface transmits to its next basic processing circuit (for example, the white filled horizontal data path in the middle of the basic processing circuit array in Figure 1b); the basic processing circuit receives the input data (the data can be sent by the main processing circuit) Input data and the identification mask data associated with the input data), then transmit the data to the next basic processing circuit connected to it through its vertical data output interface (for example, the white in the middle of the basic processing circuit array in Figure 1b) Filled vertical data path);

Specifically, the control circuit of the main processing circuit may send the input data and the mask data associated with the input data to the base processing circuit, and the base processing circuit receives the input data and the mask data associated with the input data;

Each basic processing circuit performs operations on the received data. Specifically, the basic processing circuit may enable the second mapping circuit to associate the mask data associated with the input data and the mask data associated with the weight data (that is, the values associated with the convolution kernel in the weight). Mask data) to obtain the connection identification data; and then use the connection identification data to select the input data and the data in the weight data whose absolute value is greater than a preset threshold value for multiplication;

In an optional solution, if the data received in each basic processing circuit (specifically may be a data block to be calculated, such as data in a convolution kernel in weights and mask data associated with the data, input data, or the input When the amount of data associated with the mask data) exceeds a preset threshold, the basic processing circuit will no longer receive new input data, such as the data in several convolution kernels in the weights that the main processing circuit will subsequently send and the The data corresponds to the associated mask data, etc., until there is sufficient buffer / storage space in the basic processing circuit, and then the newly sent data from the main processing circuit is received.

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result can be transmitted from the data output interface;

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

Specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface; if not, the result is output to the direction of the basic processing circuit that can directly output to the main processing circuit (for example, FIG. In 1b, the bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transmit the calculation results downward from the vertical output interface).

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

Output the result in a direction that can be directly output to the main processing circuit (for example, the bottom row of the basic processing circuit outputs its output result directly to the main processing circuit, and the other basic processing circuits transmit the calculation result downward from the vertical output interface);

The main processing circuit receives the result of the inner product operation of each basic processing circuit and can obtain the output result.

In one embodiment, the present invention discloses a neural network computing device, which includes a functional unit corresponding to all or part of the implementation methods provided in the method embodiments described above.

In one embodiment, the present invention discloses a chip (as shown in FIG. 4) for performing all or part of the implementation manners provided in the method embodiments described above.

In one embodiment, the present invention discloses an electronic device including a functional unit for performing all or part of the method embodiments described above.

Electronic devices include data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cameras, camcorders, projectors, watches, headphones, mobile Storage, wearables, vehicles, home appliances, and / or medical devices.

The vehicles include airplanes, ships, and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, cooker hoods, and the medical equipment includes nuclear magnetic resonance Instrument, type B ultrasound scanner and / or electrocardiograph.

The specific embodiments described above further describe the purpose, technical solution and beneficial effects of the present disclosure. It should be understood that the above are only specific embodiments of the present disclosure and are not used to limit the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of this disclosure shall be included in the protection scope of this disclosure.

S, P‧‧‧ Matrix

C‧‧‧ vector

FIG. 1a is a schematic structural diagram of an integrated circuit chip device.

FIG. 1b is a schematic structural diagram of another integrated circuit chip device.

Figure 1c is a schematic structural diagram of a basic processing circuit.

FIG. 1d is a schematic structural diagram of a main processing circuit.

Figure 2a is a schematic diagram of a method of using a basic processing circuit.

Figure 2b is a schematic diagram of data transmission by a main processing circuit.

Figure 2c is a schematic diagram of a matrix multiplied by a vector.

FIG. 2d is a schematic structural diagram of an integrated circuit chip device.

FIG. 2e is a schematic structural diagram of another integrated circuit chip device.

Figure 2f is a schematic diagram of a matrix multiplied by a matrix.

Figure 3a is a schematic diagram of convolution input data.

Figure 3b is a schematic diagram of a convolution kernel.

FIG. 3c is a schematic diagram of a calculation window of a three-dimensional data block of input data.

FIG. 3d is a schematic diagram of another operation window of a three-dimensional data block of input data.

FIG. 3e is another schematic view of a calculation window of a three-dimensional data block of input data.

4a-5b are schematic structural diagrams of two types of mapping circuits provided by embodiments of the present application.

Claims (14)

An integrated circuit chip device, wherein the integrated circuit chip device includes: a main processing circuit and a plurality of basic processing circuits; the main processing circuit includes a first mapping circuit; at least one of the plurality of basic processing circuits includes a first Two mapping circuits, the first mapping circuit and the second mapping circuit are both used to perform compression processing of each data in a neural network operation; The plurality of basic processing circuits are distributed in an array; each basic processing circuit is connected to other adjacent basic processing circuits, and the main processing circuit is connected to n basic processing circuits in the first row, n basic processing circuits in the m row, and M basic processing circuits in the first column; The main processing circuit is configured to perform each continuous operation in a neural network operation and transmit data to the basic processing circuit connected thereto; The plurality of basic processing circuits are configured to perform operations in the neural network in a parallel manner according to the transmitted data, and transmit the operation results to the main processing circuit through the basic processing circuit connected to the main processing circuit. The integrated circuit chip device according to the first patent application scope, wherein, The main processing circuit is configured to obtain a data block to be calculated and an operation instruction. According to the operation instruction, the data block to be calculated is divided into a horizontal data block and a vertical data block. A data block distributed to the basic processing circuit connected to the main processing circuit in a direction, the vertical data block being a data block distributed to the basic processing circuit connected to the main processing circuit in a vertical direction; starting the first mapping circuit to the The horizontal data block and the vertical data block are processed to obtain a processed horizontal data block and an identification data block associated with the horizontal data block, the processed vertical data block and the identification data block associated with the vertical data block; The processed horizontal data block and the identification data block associated with the horizontal data block are split and processed to obtain multiple basic data blocks and the identification data blocks associated with the basic data block, respectively. The multiple basic data blocks and the multiple basic data blocks are split. The identification data blocks associated with the data blocks are distributed to the underlying processing circuit connected to them, and the processed vertical data blocks and Data identifying the associated vertical block data blocks is broadcast to its base connected to the processing circuit; The basic processing circuit is configured to start the second mapping circuit to obtain a connection identification data block according to the identification data block associated with the vertical data block and the identification data associated with the basic data block, and to obtain the connection identification data block according to the connection identification data block. Process the data block and the basic data block to obtain a processed vertical data block and the basic data block; perform an inner product operation on the processed vertical data block and the basic data block to obtain an operation result, and send the operation result To the main processing circuit; The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of the operation instruction. The integrated circuit chip device according to the first patent application scope, wherein, The main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction; and start the first mapping circuit pair. Process the horizontal data block to obtain a processed horizontal data block and an identification data block associated with the horizontal data block, or start the first mapping circuit to process the horizontal data block according to a pre-stored identification data block associated with the horizontal data block. Obtaining a processed horizontal data block; splitting the processed horizontal data block and an identification data block associated with the horizontal data block to obtain a plurality of basic data blocks and identification data blocks that are respectively associated with the plurality of basic data blocks, Distributing the plurality of basic data blocks and the identification data blocks respectively associated with the plurality of basic data blocks to a basic processing circuit connected thereto, and broadcasting the vertical data block to the basic processing circuit connected thereto; The basic processing circuit is used to start the second mapping circuit to process the vertical data block according to the identification data block associated with the basic data block, to obtain a processed vertical data block; and to process the processed vertical data block. Perform an inner product operation with the processed basic data block to obtain an operation result, and send the operation result to the main processing circuit; The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of the operation instruction. The integrated circuit chip device according to the first patent application scope, wherein, The main processing circuit is configured to obtain a data block to be calculated and an operation instruction, and divide the data block to be calculated into a horizontal data block and a vertical data block according to the operation instruction; and start the first mapping circuit pair. The vertical data block is processed to obtain a processed vertical data block and an identification data block associated with the vertical data block, or the first mapping circuit is started according to a pre-stored identification data block associated with the vertical data block. The vertical data block is processed to obtain a processed vertical data block; the horizontal data block is split to obtain multiple basic data blocks; the multiple basic data blocks are distributed to a basic processing circuit connected thereto, and the processing is performed. The subsequent vertical data block and the identification data block associated with the vertical data block are broadcast to the basic processing circuit connected to the vertical data block; The basic processing circuit is used to start the second mapping circuit to process the basic data block according to the identification data block associated with the vertical data block to obtain a processed basic data block; the processed vertical data block and the An inner product operation is performed on the processed basic data block to obtain an operation result, and the operation result is sent to the main processing circuit; The main processing circuit is configured to process the operation result to obtain the data block to be calculated and an instruction result of the operation instruction. The integrated circuit chip device according to any one of claims 1 to 4, wherein the data block to be calculated includes at least one weight, and / or at least one input neuron. The integrated circuit chip device according to item 5 of the scope of patent application, wherein the identification data block is a matrix data block composed of 0 and 1, where 0 represents the weight or the absolute value of the input neuron is less than or equal to a first A threshold, 1 means that the weight or the absolute value of the input neuron is greater than the first threshold. The integrated circuit chip device according to item 6 of the scope of patent application, wherein the connection identification data block is obtained by performing element-by-element and operation on the identification data associated with the vertical data block and the identification data block associated with the basic data block. The integrated circuit chip device according to any one of claims 2-4, wherein, The basic processing circuit is specifically used for performing inner product processing on the basic data block and the vertical data block to obtain an inner product processing result, accumulating the inner product processing result to obtain an operation result, and sending the operation result to the main process. Circuit The main processing circuit is configured to, when the operation result is the result of the inner product processing, accumulate the operation result to obtain an accumulation result, and arrange the accumulation result to obtain the data block to be calculated and the instruction result of the operation instruction. . The integrated circuit chip device according to any one of claims 2-4, wherein, The main processing circuit is specifically configured to divide the vertical data block into multiple partial vertical data blocks, and broadcast the multiple partial vertical data blocks to the basic processing circuit through multiple times; the multiple partial vertical data blocks Combine to form the vertical data block; The basic processing circuit is specifically configured to start the second mapping circuit to process the partial vertical data block according to the identification data block associated with the basic data block to obtain a processed partial vertical data block; the basic data block and the The processed vertical data block performs an inner product operation. The integrated circuit chip device according to any one of claims 2-4, wherein, The main processing circuit is specifically configured to divide the processed vertical data block and the identification data block associated with the vertical data block into a plurality of partial vertical data blocks and the identification data block associated with the vertical data block. The multiple vertical data blocks and the identification data blocks associated with the multiple vertical data blocks are broadcast to the basic processing circuit multiple times; the multiple vertical data blocks are combined to form the vertical data block; The basic processing circuit is specifically configured to start the second mapping circuit to obtain a connection identification data block according to the identification data block associated with the basic data block and the identification data block associated with the partial vertical data block; according to the connection identification data block pair, The basic data block and the vertical data block are processed to obtain a processed basic data block and a processed partial broadcast data; and an inner product operation is performed on the processed basic data block and the processed vertical data block; Alternatively, the basic processing circuit is specifically configured to start the second mapping circuit to process the basic data block according to the identification data block associated with the vertical data block of the part to obtain a processed basic data block; and to process the processed basic data The inner product is performed on the block and the part of the vertical data block. The integrated circuit chip device according to any one of claims 2-4, wherein, The basic processing circuit is specifically configured to perform an inner product process on the partial vertical data block and the basic data block to obtain an inner product processing result, accumulate the inner product processing result to obtain a partial operation result, and perform the partial operation result. Sent to the main processing circuit; or, The basic processing circuit is specifically configured to multiplex the partial vertical data blocks n times and execute the inner product operation of the partial vertical data blocks and n basic data blocks to obtain n partial processing results, and divide the n partial processing results respectively. After the accumulation, n partial operation results are obtained, and the n partial operation results are sent to the main processing circuit, where n is an integer greater than or equal to 2. A chip, wherein the chip integrates a device such as any one of items 1 to 11 of the scope of patent application. An electronic device, wherein the electronic device includes a chip as in item 12 of the scope of patent application. A method for calculating a neural network, wherein the method is applied in an integrated circuit chip device, and the integrated circuit chip device includes: an integrated circuit chip device according to any one of claims 1 to 11 of the scope of patent application; For performing operations on neural networks; Among them, the operation of the neural network includes one or any of convolution operation, matrix multiplication matrix operation, matrix multiplication vector operation, paranoid operation, fully connected operation, general matrix multiplication GEMM operation, general matrix vector multiplication GEMV operation, and activation operation. combination.