TWI767098B - Method for neural network forward computation and related product - Google Patents

Abstract

The present disclosure provides a neural network forward computing method executed on an integrated circuit chip device. The neural network includes multiple layers, wherein the method includes the following steps: receiving a first calculation instruction, parsing the first calculation instruction to obtain the first calculation the first operation included in the i-th layer of the forward operation and the input data and weight data corresponding to the first calculation instruction; determine the first complexity of the first operation according to the input data, weight data and the first operation degree, determine the first data type of the input data and the weight data when the first operation is performed according to the first complexity, the first data type includes: floating point type or fixed point type; The data performs the first operation included in the first layer of the forward operation in the first data type. The technical solution provided by the present disclosure has the advantages of small calculation amount and low power consumption.

Description

Neural network forward operation method and related products

The present disclosure relates to the field of neural networks, and in particular, to a neural network forward computing method and related products.

Artificial Neural Network (ANN) has been a research hotspot in the field of artificial intelligence since the 1980s. It abstracts the human brain neuron network from the perspective of information processing, establishes a certain simple model, and forms different networks according to different connection methods. In engineering and academia, it is often simply referred to as neural network or neural-like network. Neural network is an operation model, which is composed of a large number of nodes (or neurons) connected with each other. The operation of the existing neural network is based on a CPU (Central Processing Unit, central processing unit) or a GPU (Graphics Processing Unit, graphics processing unit) to realize the forward operation of the neural network, such a forward operation has a large amount of calculation and high power consumption .

Embodiments of the present disclosure provide a neural network forward computing method and related products, which can improve the processing speed and efficiency of a computing device.

A first aspect provides a neural network forward computing method executed on an integrated circuit chip device, the neural network includes multiple layers, and the method includes the steps of: receiving a first calculation instruction, parsing the first calculation instruction to obtain the first calculation The first operation included in the i-th layer of the forward operation and the input data and weight data corresponding to the first calculation instruction; the value range of the i is an integer greater than or equal to 1, if the i is greater than or equal to 1 is equal to 2, the input data is the output data of the i-1th layer; the first complexity of the first operation is determined according to the input data, the weight data and the first operation, and the input data is determined according to the first complexity and the first data type of the weight data when the first operation is performed, the first data type includes: a floating point type or a fixed-point type; the input data and the weight data are executed in the forward direction with the first data type The first operation contained in the first layer of operations.

In a second aspect, an integrated circuit chip device is provided, the integrated circuit chip device is used to perform forward operation of a neural network, the neural network includes multiple layers, and the device includes: a processing circuit and an external interface; the external interface, is used to receive a first calculation instruction; the processing circuit is used to parse the first calculation instruction to obtain the first calculation and the input corresponding to the first calculation instruction included in the i-th layer of the forward operation of the first calculation instruction data and weight data; the value range of the i is an integer greater than or equal to 1, if the i is greater than or equal to 2, the input data is the output data of the i-1th layer; the processing circuit also uses determining the first complexity of the first operation according to the input data, the weight data and the first operation, and determining the input data and the weight according to the first complexity The first data type of the data when the first operation is performed, the first data type includes: floating-point type or fixed-point type; the processing circuit is further configured to execute the input data and the weight data in the first data type. The first operation included in the i-th layer of the forward operation.

In a third aspect, a neural network computing device is provided, and the neural network computing device includes one or more integrated circuit chip devices provided in the second aspect.

In a fourth aspect, a combined processing device is provided, the combined processing device includes: the neural network computing device, the universal interconnection interface, and the universal processing device provided in the third aspect; the neural network computing device communicates with the neural network computing device through the universal interconnection interface. Universal processing unit connection.

A fifth aspect provides a wafer that integrates the device of the second aspect, the device of the third aspect, or the device of the fourth aspect.

In a sixth aspect, an electronic device is provided, the electronic device comprising the wafer of the fourth aspect.

It can be seen that, through the embodiments of the present disclosure, a data conversion operation circuit is provided to perform operations after converting the types of data blocks, which saves transmission resources and computing resources, so it has the advantages of low power consumption and small calculation amount.

A, B, S: Matrix

P: vector

S401b, S402b, S403b, S401, S402, S403, S404: Steps

10: Neural network processor card board

11: Neural network chip package structure

12: First electrical and non-electrical connection device

13: The first substrate

111: Neural Network Chip

112: Second electrical and non-electrical connection device

113: Second substrate

1111: storage unit

1112: Direct Memory Access Unit

1113: Instruction cache unit

1114: Weight cache unit

1115: Input neuron buffer unit

1116: output neuron buffer unit

1117: Control Unit

1118: arithmetic unit

21: Neural Network Chip

22: Pad

23: Solder Balls

24: Second substrate

25: Connection point on the second substrate 24

26: pin

27: Insulation filler

28: Thermal paste

29: Metal shell heat sink

Figure 1 is a schematic diagram of the forward operation of a neural network.

FIG. 1a is a schematic structural diagram of a fixed-point data type.

Figure 2a is a schematic diagram of convolution input data.

Figure 2b is a schematic diagram of the convolution kernel.

FIG. 2c is a schematic diagram of an operation window of a three-dimensional data block of input data.

FIG. 2d is a schematic diagram of another operation window of a three-dimensional data block of input data.

Figure 2e is a schematic diagram of another operation window of a three-dimensional data block of input data.

Figure 3a is a schematic structural diagram of a neural network chip.

Figure 3b is a schematic structural diagram of another neural network chip.

Figure 4a is a schematic diagram of matrix multiplication by matrix.

Figure 4b is a flow chart of a method of multiplying a matrix by a matrix.

Figure 4c is a schematic diagram of multiplying a matrix by a vector.

Figure 4d is a flow chart of a method of multiplying a matrix by a vector.

FIG. 5a also discloses a schematic structural diagram of a combined processing device for the present disclosure.

FIG. 5b also discloses another structural schematic diagram of a combined processing device for the present disclosure.

5c is a schematic structural diagram of a neural network processor board provided by an embodiment of the disclosure; FIG. 5d is a schematic structural diagram of a neural network chip packaging structure provided by an embodiment of the disclosure; A schematic diagram of the structure of a neural network chip; FIG. 6 is a schematic diagram of a neural network chip packaging structure provided by an embodiment of the disclosure; FIG. 6a is a schematic diagram of another neural network chip packaging structure provided by the embodiment of the disclosure.

In order for those skilled in the art to better understand the disclosed solutions, the technical solutions in the disclosed embodiments will be clearly and completely described below with reference to the drawings in the disclosed embodiments. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those with ordinary knowledge in the technical field without creative work shall fall within the protection scope of the present disclosure.

In the method provided by the first aspect, determining the first data type of the input data and the weight data when performing the first operation according to the first complexity includes: comparing the first complexity with a preset threshold , if the first complexity is higher than the preset threshold, determine that the first data type is a fixed-point type, and if the first complexity is lower than or equal to the preset threshold, determine the first data type The type is a floating point type.

In the method provided in the first aspect, after determining the first data type of the input data and the weight data when the first operation is performed according to the first complexity, the method further includes: determining the input data and the weight data belongs to a second data type, if the second data type is different from the first data type, convert the input data belonging to the second data type and the weight data belonging to the second data type into the input data belonging to the first data type and the weight data belonging to the first data type.

In the method provided in the first aspect, if the first operation is a convolution operation, the input data is convolution input data, the weight data is a convolution kernel, and the first complexity=α*C*kW *kH*M*N*W*C*H; Among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kH, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data; If the first complexity is greater than the set threshold, determine whether the convolution input data and the convolution kernel are floating-point data, if the convolution input data and the convolution kernel are not floating-point data, the convolution input The data is converted to floating-point data, the convolution kernel is converted to floating-point data, and then the convolution input data and the convolution kernel are convolved with the floating-point data type.

In the method provided in the first aspect, if the first operation is a matrix multiplication matrix operation, the input data is the first matrix of the matrix multiplication matrix operation, and the weight value is the matrix multiplication matrix operation. The second matrix; the first complexity=β*F1*G*E*F2; where, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, E, F2 is the row and column values of the second matrix; if the first complexity is greater than the set threshold, determine whether the first matrix and the second matrix are floating point data, such as the first matrix and the second matrix are not floating point data data, convert the first matrix into floating-point data, convert the second matrix into floating-point data, and then perform a matrix-matrix-matrix operation on the first matrix and the second matrix in the floating-point data type.

In the method provided in the first aspect, if the first operation is a matrix multiplication vector operation, the input data is the first matrix of the matrix multiplication vector operation, and the weight is the matrix multiplication vector operation. Vector; first complexity=β*F1*G*F2; where, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, and F2 is the column value of the vector; If the first complexity is greater than the set threshold, determine whether the first matrix and vector are floating-point data, and if the first matrix and vector are not floating-point data, convert the first matrix to floating-point data point data, convert the vector to floating point data, and then perform a matrix multiplication vector operation on the first matrix, vector with the floating point data type.

In the method provided in the first aspect, the i-th layer may further include one or any combination of the following operations: bias operation, fully connected operation, GEMM operation, GEMV operation, and activation operation.

In the device provided in the second aspect, the processing circuit is specifically configured to compare the first complexity with a preset threshold, and if the first complexity is higher than the preset threshold, the computing device determines the The first data type is a fixed-point type, and if the first complexity is lower than or equal to the preset threshold, the computing device determines that the first data type is a floating-point type.

In the device provided in the second aspect, the integrated circuit chip device further includes: a data type conversion circuit; the processing circuit is further configured to determine the second data type to which the input data and the weight data belong, as described above The second data type is different from the first data type, and a conversion command is sent to the data type conversion circuit, and the data type conversion circuit is configured to convert the input data belonging to the second data type and The weight data belonging to the second data type is converted into the input data belonging to the first data type and the weight data belonging to the first data type.

In the device provided in the second aspect, if the first operation is a convolution operation, the input data is convolution input data, the weight data is a convolution kernel, and the processing circuit is configured to calculate the first Complexity, the first complexity = α*C*kW*kH*M*N*W*C*H; among them, α is the convolution coefficient, the value range is greater than 1; C, kW, kH, M are the volume The value of the four dimensions of the product kernel, N, W, C, and H are the values of the four dimensions of the convolution input data; The processing circuit is further configured to determine whether the convolution input data and the convolution kernel are floating-point data if the first complexity is greater than a set threshold, for example, the convolution input data and the convolution kernel are not floating-point data data, convert the convolution input data to floating point data, convert the convolution kernel to floating point data, and then perform the convolution operation on the convolution input data and the convolution kernel with the floating point data type.

In the device provided in the second aspect, the first operation is: a matrix multiplication matrix operation, the input data is the first matrix of the matrix multiplication matrix operation, and the weight is the matrix multiplication matrix operation. The second matrix; the processing circuit is used to calculate the first complexity; the first complexity=β*F1*G*E*F2; wherein, β is the matrix coefficient, and the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, and E and F2 are the row and column values of the second matrix; the processing circuit is further configured to determine the first matrix and the second matrix if the first complexity is greater than the set threshold Whether the matrix is floating-point data, if the first matrix and the second matrix are not floating-point data, convert the first matrix to floating-point data, convert the second matrix to floating-point data, and then convert the first matrix to floating-point data. The first matrix and the second matrix perform matrix-by-matrix operations in floating point data types.

In the device provided in the second aspect, if the first operation is a matrix multiplication vector operation, the input data is the first matrix of the matrix multiplication vector operation, and the weight is the matrix multiplication vector operation. vector; the processing circuit is used to calculate the first complexity; the first complexity=β*F1*G*F2; wherein, β is the matrix coefficient, and the value range is greater than or equal to 1, and F1 and G are the first matrix The row and column values of , and F2 is the column value of the vector; the processing circuit is also used to determine whether the first matrix and the vector are floating-point data if the first complexity is greater than the set threshold, such as the first Matrices and vectors are not floating point data, The first matrix is converted into floating-point data, the vector is converted into floating-point data, and then a matrix-multiply-vector operation is performed on the first matrix and the vector in the floating-point data type.

In the apparatus provided in the second aspect, the i-layer may further include one or any combination of the following operations: a bias operation, a fully connected operation, a GEMM operation, a GEMV operation, and an activation operation.

As shown in FIG. 1 , in the forward operation of a neural network provided by the embodiment of the present disclosure, each layer uses its own input data and weights to calculate corresponding output data according to the operation rules specified by the type of the layer; The forward operation process (also called inference) is the process of processing the input data of each layer layer by layer, and obtaining the output data after a certain calculation. It has the following characteristics: the input of a certain layer: the input of a certain layer can be the input of the neural network data; the input of a certain layer can be the output of other layers; the input of a certain layer can be the output of this layer at the previous moment (corresponding to the case of a recurrent neural network); a certain layer can obtain input from multiple above-mentioned input sources at the same time; The output of one layer: the output of a certain layer can be used as the output of the neural network; the output of a certain layer can be the input of other layers; the output of a certain layer can be the input of this layer at the next moment (in the case of a recurrent neural network); a certain layer The output can output results to the above multiple output directions; Specifically, the types of operations of the layers in the neural network include but are not limited to the following: convolutional layers (that is, performing convolution operations); fully-connected layers (that is, performing fully-connected operations); normalization (regularization) Layer: including LRN (Local Response Normalization) layer, BN (Batch Normalization) layer and other types; pooling layer; activation layer: including but not limited to the following types of Sigmoid layer, ReLU layer, PReLu layer, LeakyReLu layer, Tanh layer; Reverse operation, the reverse operation of each layer needs to perform two parts of operation: one part is to use the gradient of the output data that may be sparsely represented and the input data that may be sparsely represented to calculate the gradient of the weights (used in "weight update" step to update the weights of this layer), and the other part is to use the output data gradient that may be sparsely represented and the weights that may be sparse representation to calculate the input data gradient (used as the output data gradient of the next layer in the reverse operation to for the reverse operation); the reverse operation follows the reverse order of the forward operation, starting from the last layer and passing the gradient backwards.

In an optional solution, the gradient of the output data obtained by the reverse calculation of a certain layer can come from: the gradient returned by the final loss function (lost function or cost function) of the neural network; the input data gradient of other layers; The gradient of the input data at the previous moment of this layer (corresponding to the case of the recurrent neural network); a certain layer can obtain the gradient of the output data from multiple above-mentioned sources at the same time; after the reverse operation of the neural network is performed, the Gradient of the weight, in this step, the first input buffer and the second input buffer of the device are respectively used to store the weight of the layer and the gradient of the weight, and then use the gradient of the weight in the computing unit to measure the weight Update; the operations mentioned above are all operations of one layer in the neural network. For a multi-layer neural network, the implementation process is that in the forward operation, after the execution of the upper layer of artificial neural network is completed, the operation instructions of the next layer The output data calculated in the operation unit is used as the input data of the next layer for operation (or some operations are performed on the output data and then used as the input data of the next layer), and at the same time, the weights are also replaced with the weights of the next layer. value; in the reverse operation, when the reverse operation of the artificial neural network of the previous layer is executed, the operation instruction of the next layer will calculate the gradient of the input data calculated in the operation unit as the gradient of the output data of the next layer. The input data gradient performs some operations and then serves as the output data gradient of the next layer), and at the same time replaces the weights with the weights of the next layer; (represented by the following figure, the dotted arrow in the figure below represents the reverse operation, the solid line The arrows represent forward operations, and the labels below each figure represent the meaning of the figures)

Representation of fixed-point data

The fixed-point method refers to converting the data representation of a data block in the network into a specific data representation with a fixed decimal point position (mapped to the 0/1 bit placement of the data on the circuit device); In an optional solution, multiple pieces of data are formed into a data block as a whole using the same fixed-point representation method for fixed-point representation; FIG. 1a shows fixed-point data with short bits for storing data according to an embodiment of the present invention A concrete representation of the structure. Among them, 1Bit is used to represent the symbol, M bits are used to represent the integer part, and N bits are used to represent the fractional part; compared with the 32-bit floating point data representation, the short-bit fixed-point data representation adopted in the present invention is used in addition to occupying bits In addition to fewer digits, for the same layer and the same type of data in the neural network, such as the ownership value data of the first convolutional layer, a flag bit Point location is additionally set to record the position of the decimal point, so that it can be used according to the actual data. The distribution adjusts the precision of the data representation and the range of representable data.

The floating-point number is represented by 32 bits, but for this technical solution, the use of fixed-point numbers can reduce the number of bits of a numerical value, thereby reducing the amount of data transmitted and the amount of data calculated.

The input data is represented by Figure 2a (N samples, each sample has C channels, the height of the feature map of each channel is H, and the width is W), and the weights, that is, the convolution kernel, are represented by Figure 2b (there are M Convolution kernel, each convolution kernel has C channels, the height and width are KH and KW respectively). For N samples of input data, the rules of the convolution operation are the same. The following explains the process of performing the convolution operation on a sample. On a sample, each of the M convolution kernels must perform the same Operation, each convolution kernel operation obtains a plane feature map, and M convolution kernels finally calculate M plane feature maps, (for a sample, the output of the convolution is M feature maps), for a convolution kernel , to perform the inner product operation at each plane position of a sample, and then slide along the H and W directions. For example, Figure 2c shows that a convolution kernel is processed at the lower right corner of a sample of the input data. Corresponding diagram of the in-line product operation; Figure 2d shows that the position of the convolution is slid one cell to the left and Figure 2e shows that the position of the convolution is slid one cell up.

When the first operation is a convolution operation, the input data is the convolution input data, the weight data is the convolution kernel, and the first complexity=α*C*kW*kH*M*N*W*C* H; among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kH, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the four dimensions of the convolution input data. value; if the first complexity is greater than the set threshold, determine whether the convolution input data and the convolution kernel are floating-point data, if the convolution input data and the convolution kernel are not floating-point data, the convolution input data and the convolution kernel are not floating-point data. Convert the convolution input data to floating point data, convert the convolution kernel to floating point data, and then perform the convolution operation on the convolution input data and the convolution kernel with the floating point data type.

Specifically, the convolution processing method can adopt the wafer structure processing as shown in FIG. 3a, and the data conversion operation circuit of the main processing circuit (also referred to as the main unit) can convert the weight of the weight when the first complexity is greater than the set threshold. Part or all of the data in the convolution kernel of the value is converted into fixed-point type data, and the control circuit of the main processing circuit sends part or all of the data in the convolution kernel of the weight to the main processing circuit through the horizontal data input interface. Those basic processing circuits (which may also be referred to as basic units) (for example, the uppermost gray-filled vertical data path in Figure 3b); in an alternative, the control circuit of the main processing circuit will The data of the convolution kernel is sent one number or part of the number to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, the first number of the third line is sent for the first time, and the data of the third line is sent for the second time. The 2nd number in the 3rd time, the 3rd number of the 3rd line..., or the 1st time to send the first two numbers of the 3rd line, the second time to send the 3rd and 3rd line of the 3rd line 4 numbers, 3rd time sending 3rd line 5th and 6th numbers... ;) In an alternative solution, another situation is that the control circuit of the main processing circuit sends the data of certain convolution kernels in the weights each time a number or a part of the number to a certain basic processing circuit; (for example, for a certain basic processing circuit; A basic processing circuit that sends the first number of lines 3, 4, and 5 for the first time, sends the second number of lines 3, 4, and 5 for the second time, and sends the third number for the third time. The 3rd number of each line of line 4,5..., or the first two numbers of lines 3,4,5 are sent for the first time, and the first two numbers of lines 3,4,5 are sent for the second time The 3rd and 4th numbers, the third time the 3rd, 4th, and 5th lines are sent. The 5th and 6th numbers in each line... ;) The control circuit of the main processing circuit arranges the input data according to the position of the convolution To divide, the control circuit of the main processing circuit sends the data in some or all of the convolution positions in the input data to those basic processing circuits that are directly connected to the main processing circuit through the vertical data input interface (for example, the basic processing circuit in Figure 3b). The gray-filled horizontal data path on the left side of the circuit array); in an alternative, the control circuit of the main processing circuit sends the data of a certain convolution position in the input data one number or part of the number at a time to a certain basic processing circuit ;(For example, for a certain basic processing circuit, the first number in the third column is sent for the first time, the second number in the third column is sent for the second time, and the third number in the third column is sent for the third time. ..., or the first two numbers of column 3 are sent for the first time, the third and fourth numbers of column 3 are sent for the second time, and the fifth and sixth numbers of column 3 are sent for the third time. .....;) In an alternative solution, another situation is that the control circuit of the main processing circuit sends a number or a part of the data at a certain convolution position in the input data to a certain basic processing. Circuit; (For example, for a certain basic processing circuit, the first number of columns 3, 4, and 5 is sent for the first time, and the second number of columns 3, 4, and 5 is sent for the second time. The 3rd time to send the 3rd number of each column of the 3rd, 4th, 5th column..., or the 1st time to send the first two numbers of each column of the 3rd, 4th, and 5th column, and the second time to send the 3rd number of each column ,4,5 columns 3rd and 4th numbers in each column, 3rd send 3rd,4,5th column 5th and 6th numbers in each column... ;) After the basic processing circuit receives the weight data, it transmits the data to the next basic processing circuit connected to it through its horizontal data output interface (for example, the white-filled horizontal data path in the middle of the basic processing circuit array in Figure 3b). ); after the basic processing circuit receives the data of the input data, it transmits the data to the next basic processing circuit connected to it through its vertical data output interface (for example, the white-filled ones in the middle of the basic processing circuit array in FIG. 3b vertical data path); each base processing circuit operates on the data received; in an alternative, the base processing circuit computes the multiplication of one or more sets of two data at a time, and then accumulates the results into a register and/or on-chip cache; in an alternative, the base processing circuit computes the inner product of one or more sets of two vectors at a time, and then accumulates the results into registers and/or on-chip cache; the base processing circuit calculates After the result, the result can be transmitted from the data output interface; in an optional solution, the calculation result can be the final result or the intermediate result of the inner product operation; The connected output interface transmits the result from this interface, if not, it outputs the result in the direction of the basic processing circuit that can output directly to the main processing circuit (for example, in Figure 3b, the basic processing circuit in the bottom row outputs its output result directly. To the main processing circuit, other basic processing circuits transmit the operation results downward from the vertical output interface).

After the basic processing circuit receives the calculation results from other basic processing circuits, it transmits the data to other basic processing circuits or main processing circuits connected to it; Output the results in a direction that can be directly output to the main processing circuit (for example, the basic processing circuits in the bottom row directly output their output results to the main processing circuit, and other basic processing circuits transmit the operation results downward from the vertical output interface); The processing circuit receives the result of the inner product operation of each basic processing circuit, and then the output result can be obtained.

Referring to FIG. 4a, FIG. 4a is a matrix-by-matrix operation. For example, the first operation is: a matrix-by-matrix operation, the input data is the first matrix of the matrix-by-matrix operation, and the weight is The second matrix of the matrix multiplication matrix operation; the first complexity=β*F1*G*E*F2; wherein, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the rows of the first matrix, Column value, E and F2 are the row and column values of the second matrix; if the first complexity is greater than the set threshold, determine whether the first matrix and the second matrix are floating-point data, such as the first matrix and the first matrix. The second matrix is not floating point data, convert the first matrix to floating point data, convert the second matrix to floating point data, and then perform matrix multiplication of the first matrix and the second matrix with floating point data type Matrix Operations.

Referring to Figure 4b, use the device as shown in Figure 3b to complete the operation of matrix multiplication matrix; Describe below the calculation size is the operation of the multiplication of the matrix S of M rows and L columns and the size of the matrix P of L rows and N columns, (in the matrix S Each row of is the same length as each column of matrix P, as shown in Figure 2d) The neural network computing device has K basic processing circuits:

Step S401b, the main processing circuit converts the matrix S and the matrix P into fixed-point type data when the first complexity is greater than the set threshold, and the control circuit of the main processing circuit distributes each row of data in the matrix S to K basic processing circuits On one of these, the underlying processing circuit will receive the number of The data is stored in the on-chip cache and/or register; specifically, it can be sent to the basic processing circuit connected to the main processing circuit among the K basic processing circuits.

In an alternative solution, if the number of rows of S is M<=K, the control circuit of the main processing circuit distributes a row of the S matrix to the M basic processing circuits respectively; in an alternative solution, if the number of rows of S is M >K, the control circuit of the main processing circuit distributes data of one or more rows in the S matrix to each basic processing circuit.

There are Mi rows in S that are distributed to the i-th basic processing circuit, and the set of Mi rows is called Ai. Figure 2e shows the computation to be performed on the i-th basic processing circuit.

In an optional solution, in each basic processing circuit, such as the ith basic processing circuit: the received matrix Ai distributed by the main processing circuit is stored in the ith basic processing circuit register and/or In the on-chip cache; the advantage is that the amount of subsequent data transmission is reduced, the computing efficiency is improved, and the power consumption is reduced.

In step S402b, the control circuit of the main processing circuit transmits each part in the matrix P to each basic processing circuit by broadcasting; in an optional solution, each part in the matrix P can be broadcast only once to the registers of each basic processing circuit Or in the on-chip cache, the i-th basic processing circuit fully multiplexes the data of the matrix P obtained this time, and completes the inner product operation corresponding to each row in the matrix Ai; the multiplexing in this embodiment can be based on The processing circuit is used repeatedly in the calculation, for example, the multiplexing of the data of the matrix P may be used for the data of the matrix P multiple times.

In an optional solution, the control circuit of the main processing circuit may broadcast each part of the matrix P to the registers or the on-chip cache of each basic processing circuit for many times, and the i-th basic processing circuit The data of the matrix P obtained each time is not multiplexed, and the inner product operation corresponding to each row in the matrix Ai is completed in stages; in an optional solution, the control circuit of the main processing circuit can The part is broadcast to the registers or on-chip cache of each basic processing circuit for many times, and the i-th basic processing circuit performs partial multiplexing on the data of the matrix P obtained each time, and completes the inner product operation corresponding to each row in the matrix Ai; In an optional solution, each basic processing circuit, such as the ith basic processing circuit, calculates the inner product of the data of the matrix Ai and the data of the matrix P;

Step S403b, the accumulator circuit of each basic processing circuit accumulates the result of the inner product operation and transmits it back to the main processing circuit.

In an optional solution, the basic processing circuit may transmit the partial sums obtained by performing the inner product operation each time back to the main processing circuit for accumulation; in an optional solution, the inner product operation performed by the basic processing circuit each time may also be The obtained partial sum is stored in the register and/or on-chip cache of the basic processing circuit, and is transmitted back to the main processing circuit after the accumulation; In some cases, the sum is stored in the register and/or on-chip cache of the basic processing circuit for accumulation, and in some cases, it is transmitted to the main processing circuit for accumulation, and is transmitted back to the main processing circuit after the accumulation is completed.

Referring to FIG. 4c, it is a schematic diagram of an operation of multiplying a matrix by a vector. If the first operation is: matrix multiplication vector operation, the input data is the first matrix of the matrix multiplication vector operation, and the weight value is the vector of the matrix multiplication vector operation; The first complexity=β*F1*G*F2; where, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, and F2 is the column value of the vector; If the first complexity is greater than the set threshold, determine whether the first matrix and vector are floating-point data, if the first matrix and vector are not floating-point data, convert the first matrix into floating-point data, and convert The vector is converted into floating point data, and then a matrix multiplication vector operation is performed on the first matrix, the vector, with the floating point data type.

Referring to Figure 4d, Figure 4d provides an implementation method of multiplying a matrix by a vector, which may specifically include:

Step S401, the data conversion operation circuit of the main processing circuit converts each row of data in the matrix S into fixed-point data, the control circuit of the main processing circuit distributes it to one of the K basic processing circuits, and the basic processing circuit will receive the data. The received distribution data is stored in the on-chip cache and/or register of the basic processing circuit; in an optional solution, if the number of rows M<=K of the matrix S, the control circuit of the main processing circuit gives the K basic processing circuits respectively. Distribute one row of the S matrix; in an optional solution, if the number of rows of the matrix S M>K, the control circuit of the main processing circuit distributes the data of one or more rows in the S matrix to each basic processing circuit.

The set of rows in S distributed to the i-th basic processing circuit is Ai, and there are Mi rows in total. Figure 2c shows the computation to be performed on the i-th basic processing circuit.

In an optional solution, in each basic processing circuit, such as the ith basic processing circuit, the received distribution data, such as the matrix Ai, may be stored in a register and/or an on-chip cache of the ith basic processing circuit ; The advantage is that the data transmission amount of the distributed data is reduced, the calculation efficiency is improved, and the power consumption is reduced.

Step S402, the data type operation circuit of the main processing circuit converts the vector P into fixed-point type data, and the control circuit of the main processing circuit transmits each part of the fixed-point type vector P to K basic processing circuits by broadcasting; In an alternative solution, the control circuit of the main processing circuit can broadcast each part of the vector P only once to the registers or the on-chip cache of each basic processing circuit, and the i-th basic processing circuit can fully perform the data of the vector P obtained this time. Ground multiplexing to complete the inner product operation corresponding to each row in the matrix Ai. The advantage is that the data transmission amount of the repeated transmission of the vector P from the main processing circuit to the basic processing circuit is reduced, the execution efficiency is improved, and the transmission power consumption is reduced.

In an optional solution, the control circuit of the main processing circuit can broadcast each part of the vector P to the registers or the on-chip cache of each basic processing circuit for multiple times, and the i-th basic processing circuit can evaluate the data of the vector P obtained each time. No multiplexing is performed, and the inner product operation corresponding to each row in the matrix Ai is completed in stages; the advantage is that the data transmission amount of the vector P transmitted in a single transmission inside the basic processing circuit can be reduced, and the basic processing circuit cache and/or can be reduced. Or the capacity of the register, improve the execution efficiency, reduce the transmission power consumption, and reduce the cost.

In an optional solution, the control circuit of the main processing circuit can broadcast each part of the vector P to the registers or the on-chip cache of each basic processing circuit for multiple times, and the i-th basic processing circuit can evaluate the data of the vector P obtained each time. Partial multiplexing is performed to complete the inner product operation corresponding to each row in the matrix Ai; the advantage is that it reduces the amount of data transmission from the main processing circuit to the basic processing circuit, and also reduces the amount of data transmission inside the basic processing circuit, improving execution efficiency. , reduce the transmission power consumption.

Step S403, the inner product operator circuit of the K basic processing circuits calculates the inner product of the data of the matrix S and the vector P, such as the i-th basic processing circuit, calculates the inner product of the data of the matrix Ai and the data of the vector P;

Step S404 , the accumulator circuits of the K basic processing circuits accumulate the results of the inner product operation to obtain an accumulation result, and transmit the accumulation result back to the main processing circuit in the form of a fixed-point type.

In an optional solution, the partial sum (the partial sum is a part of the accumulated result) obtained by each time the basic processing circuit performs the inner product operation, for example, the accumulated result is: F1*G1+F2*G2+F3*G3+F4* G4+F5*G5, then the partial sum can be: the value of F1*G1+F2*G2+F3*G3) is transmitted back to the main processing circuit for accumulation; the advantage is that it reduces the amount of operations inside the basic processing circuit and improves the basic processing The operational efficiency of the circuit.

In an optional solution, the partial sum obtained by the inner product operation performed by the basic processing circuit each time can also be stored in the register and/or the on-chip cache of the basic processing circuit, and transferred back to the main processing circuit after the accumulation is completed; the advantage is that, The data transmission amount between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, and the power consumption of data transmission is reduced.

In an optional solution, the partial sum obtained by the inner product operation performed by the basic processing circuit each time can also be stored in the register and/or on-chip cache of the basic processing circuit for accumulation in some cases, and transferred to the main The processing circuit performs accumulation, and after the accumulation is completed, it is transmitted back to the main processing circuit; the advantage is that the amount of data transmission between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, the power consumption of data transmission is reduced, and the basic processing circuit is reduced. The amount of internal operation improves the operation efficiency of the basic processing circuit.

The present disclosure also provides an integrated circuit chip device, the integrated circuit chip device is used to perform forward operation of a neural network, the neural network includes multiple layers, and the device includes: a processing circuit and an external interface; upon receiving the first calculation instruction; The processing circuit is configured to parse the first calculation instruction to obtain the first calculation included in the i-th layer of the forward operation, the input data and weight data corresponding to the first calculation instruction; the i The value of can be 1. If it is 1, the input data can be the original input data. When i is greater than or equal to 2, the input data can be the output data of the previous layer, such as the output data of the i-1 layer.

The processing circuit is further configured to determine the first complexity of the first operation according to the input data, the weight data and the first operation, and determine that the input data and the weight data are performing the first operation according to the first complexity The first data type at the time, the first data type includes: floating-point type or fixed-point type; the processing circuit is further configured to perform the first data type of the forward operation on the input data and the weight data in the first data type The first operation contained in layer i.

The present disclosure also discloses a neural network computing device, which includes one or more chips as shown in FIG. 3a or FIG. 3b, for obtaining data to be computed and control information from other processing devices, and executing a specified neural network Operation, the execution result is passed to the peripheral device through the I/O interface. Peripherals such as camera, monitor, mouse, keyboard, network card, wifi interface, server. When more than one chip as shown in Figure 3a or Figure 3b is included, the chips shown in Figure 3a or Figure 3b can be linked and transmitted through a specific structure, for example, interconnected and transmitted through the PCIE bus data to support larger-scale neural network operations. At this time, the same control system can be shared, or there can be independent control systems; memory can be shared, or each accelerator can have its own memory. In addition, the interconnection method can be any interconnection topology.

The neural network computing device has high compatibility and can be connected with various types of servers through the PCIE interface.

The present disclosure also discloses a combined processing device, which includes the above-mentioned neural network computing device, a universal interconnection interface, and other processing devices (ie, general-purpose processing devices). The neural network computing device interacts with other processing devices to jointly complete the operation specified by the user. Such as 5a is a schematic diagram of the combined treatment device.

Other processing devices include one or more processor types among general-purpose/special-purpose processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processor. The number of processors included in other processing devices is not limited. Other processing devices serve as the interface between the neural network computing device and external data and control, including data transfer, to complete the basic control of starting and stopping the neural network computing device; other processing devices can also cooperate with the neural network computing device to complete computing tasks.

The universal interconnection interface is used to transmit data and control instructions between the neural network computing device and other processing devices. The neural network computing device obtains the required input data from other processing devices, and writes it into the memory device on-chip of the neural network computing device; it can obtain control instructions from other processing devices and write it into the control cache on the neural network computing device chip; The data in the memory module of the neural network computing device can be read and transmitted to other processing devices.

As shown in Figure 5b, optionally, the structure further includes a storage device for storing the data required by the operation unit/operation device or other operation units, especially suitable for the data required for operation in the neural network operation device. or other data that cannot be fully stored in the internal storage of the processing device.

The combined processing device can be used as a SOC system for mobile phones, robots, drones, video surveillance equipment and other equipment, effectively reducing the core area of the control part, improving processing speed, reducing Low overall power consumption. In this case, the general interconnection interface of the combined processing device is connected to some components of the device. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.

Please refer to FIG. 5c, which is a schematic structural diagram of a neural network processor board according to an embodiment of the present disclosure. As shown in FIG. 5 c , the above-mentioned neural network processor board 10 includes a neural network chip package structure 11 , a first electrical and non-electrical connection device 12 and a first substrate 13 .

The present disclosure does not limit the specific structure of the neural network chip package structure 11. Optionally, as shown in FIG. 5d, the neural network chip package structure 11 includes: a neural network chip 111, a second electrical and non-electrical connection device 112, a first Two substrates 113 .

The specific form of the neural network chip 111 involved in the present disclosure is not limited. The above-mentioned neural network chip 111 includes but is not limited to a neural network chip that integrates a neural network processor. The above-mentioned chip can be made of silicon material, germanium material, quantum material or molecular material. materials etc. According to the actual situation (for example: harsh environment) and different application requirements, the above-mentioned neural network chip can be packaged, so that most of the neural network chip is wrapped, and the pins on the neural network chip are passed through gold wires. The other conductors are connected to the outside of the package structure for circuit connection with the outer layers.

The present disclosure does not limit the specific structure of the neural network chip 111. Optionally, please refer to the device shown in FIG. 1a.

The present disclosure does not limit the types of the first substrate 13 and the second substrate 113, which may be a printed circuit board (PCB) or a printed wiring board (PWB), and may also be other circuit boards. The material for making the PCB is also not limited.

The second substrate 113 involved in the present disclosure is used to carry the above-mentioned neural network chip 111 , and the above-mentioned neural network chip 111 and the second substrate 113 are connected to each other through the second electrical and non-electrical connection device 112 . The neural network chip package structure 11 obtained by row connection is used to protect the neural network chip 111 and facilitate further packaging of the neural network chip package structure 11 and the first substrate 13 .

The packaging method and the structure corresponding to the packaging method of the above-mentioned specific second electrical and non-electrical connection device 112 are not limited, and an appropriate packaging method can be selected and simply improved according to the actual situation and different application requirements, for example: flip chip Ball Grid Array Package (Flip Chip Ball Grid Array Package, FCBGAP), Low-profile Quad Flat Package (LQFP), Quad Flat Package with Heat Sink (HQFP), None Pin quad flat package (Quad Flat Non-lead Package, QFN) or small pitch quad flat package (Fine-pitch Ball Grid Package, FBGA) and other packaging methods.

Flip Chip is suitable for situations where the area after packaging is high or is sensitive to the inductance of wires and the transmission time of signals. In addition, a wire bonding (Wire Bonding) packaging method can be used to reduce the cost and improve the flexibility of the packaging structure.

Ball Grid Array (Ball Grid Array) can provide more pins, and the average lead length of the pins is short, which has the function of high-speed signal transmission. Among them, the package can be packaged with Pin Grid Array (PGA) , zero insertion force (Zero Insertion Force, ZIF), single edge contact connection (Single Edge Contact Connection, SECC), contact array (Land Grid Array, LGA), etc. instead.

Optionally, the neural network chip 111 and the second substrate 113 are packaged by a flip chip ball grid array (Flip Chip Ball Grid Array). As shown in Figure 6, the above-mentioned neural network chip package structure package It includes: neural network chip 21 , pads 22 , solder balls 23 , second substrate 24 , connection points 25 and pins 26 on the second substrate 24 .

Wherein, the pad 22 is connected to the neural network chip 21, and solder balls 23 are formed by welding between the pad 22 and the connection point 25 on the second substrate 24, and the neural network chip 21 and the second substrate 24 are connected. Packaging of the neural network chip 21 .

The pin 26 is used to connect with the external circuit of the package structure (for example, the first substrate 13 on the neural network processor board 10 ), which can realize the transmission of external data and internal data, and is convenient for the neural network chip 21 or the neural network chip 21 The corresponding neural network processor processes the data. The present disclosure also does not limit the type and quantity of pins, and different pin forms can be selected according to different packaging technologies, and are arranged in accordance with certain rules.

Optionally, the above-mentioned neural network chip package structure further includes insulating fillers, which are placed in the gaps between the pads 22 , the solder balls 23 and the connection points 25 to prevent interference between the solder balls.

Wherein, the material of the insulating filler can be silicon nitride, silicon oxide or silicon oxynitride; the interference includes electromagnetic interference, inductive interference and the like.

Optionally, the above-mentioned neural network chip packaging structure further includes a heat dissipation device for dissipating heat during operation of the neural network chip 21 . Wherein, the heat dissipation device may be a metal sheet, a heat sink or a heat sink with good thermal conductivity, such as a fan.

For example, as shown in FIG. 6a, the neural network chip package structure 11 includes: a neural network chip 21, pads 22, solder balls 23, a second substrate 24, connection points 25 on the second substrate 24, pins 26, Insulation filler 27, thermal paste 28 and metal shell heat sink 29. Among them, the heat dissipation paste 28 and the metal shell heat dissipation fins 29 are used to dissipate the heat of the neural network chip 21 during operation.

Optionally, the above-mentioned neural network chip package structure 11 further includes a reinforcing structure, which is connected to the pads 22 and embedded in the solder balls 23 to enhance the connection strength between the solder balls 23 and the pads 22 .

Wherein, the reinforcing structure may be a metal wire structure or a columnar structure, which is not limited herein.

The present disclosure does not limit the specific form of the first electrical and non-electrical device 12, and reference can be made to the description of the second electrical and non-electrical device 112, that is, the neural network chip package structure 11 is packaged by welding, or a connection can be used. The way of connecting the second substrate 113 and the first substrate 13 by wire connection or plugging is convenient for subsequent replacement of the first substrate 13 or the neural network chip package structure 11 .

Optionally, the first substrate 13 includes an interface or the like for a memory unit used to expand the storage capacity, for example: a synchronous dynamic random access memory (Synchronous Dynamic Random Access Memory, SDRAM), a double-rate synchronous dynamic random access memory (Double Date Rate SDRAM, DDR), etc., the processing power of the neural network processor is improved by expanding the memory.

The first substrate 13 may further include a Peripheral Component Interconnect-Express (PCI-E or PCIe) interface, a Small Form-factor Pluggable (SFP) interface, an Ethernet interface, A controller area network bus (Controller Area Network, CAN) interface, etc., is used for data transmission between the package structure and the external circuit, which can improve the operation speed and the convenience of operation.

The neural network processor is packaged as a neural network chip 111, the neural network chip 111 is packaged into a neural network chip package structure 11, and the neural network chip package structure 11 is packaged as a neural network processor board 10, through the interface ( socket or ferrule) for data interaction with external circuits (such as computer motherboards), that is, directly through the use of the neural network processor board 10 The function of the neural network processor and protect the neural network chip 111. In addition, other modules can be added to the neural network processor board 10, which improves the application scope and operation efficiency of the neural network processor.

In one embodiment, the present disclosure discloses an electronic device including the above-mentioned neural network processor board 10 or neural network chip package structure 11 .

Electronic devices include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cameras, video cameras, projectors, watches, headphones, mobile storage , wearable devices, vehicles, home appliances, and/or medical devices.

The vehicles include airplanes, ships and/or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lamps, gas stoves, and range hoods; the medical equipment includes nuclear magnetic resonance device, B-ultrasound and/or electrocardiograph.

The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present disclosure in further detail. It should be understood that the above are only specific embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this disclosure shall be included within the protection scope of this disclosure.

Claims (18)

A neural network forward computing method executed on an integrated circuit chip device, the neural network includes multiple layers, wherein the method includes the following steps: receiving a first calculation instruction, parsing the first calculation instruction to obtain that the first calculation instruction is in the forward direction The first operation included in the i-th layer of the operation and an input data and a weight data corresponding to the first calculation instruction, the value range of the i is an integer greater than or equal to 1, if the i is greater than or equal to 2, the input data is the output data of the i-1th layer; the first complexity of the first operation is determined according to the input data, the weight data and the first operation, and the input data and the weight data are determined according to the first complexity during execution The first data type during the first operation, the first data type includes: floating-point type or fixed-point type; the input data and the weight data are included in the i-th layer that performs the forward operation with the first data type first operation. The method according to claim 1, wherein determining the first data type of the input data and the weight data when performing the first operation according to the first complexity includes: combining the first complexity with a predetermined Set a threshold for comparison, if the first complexity is higher than the preset threshold, determine that the first data type is a fixed-point type, if the first complexity is lower than or equal to the preset threshold, determine that the first data type is floating point type. The method according to claim 2, wherein after determining the first data type of the input data and the weight data when the first operation is performed according to the first complexity, the method further comprises: determining the input data And the weight data belongs to a second data type, if the second data type is different from the first data type, the input data belonging to the second data type and the second data type The weight data of the type is converted into the input data of the first data type and the weight data of the first data type. The method according to claim 1, wherein, if the first operation is a convolution operation, the input data is a convolution input data, the weight data is a convolution kernel, and the first complexity=α*C *kW*kH*M*N*W*C*H; among them, α is the convolution coefficient, the value range is greater than 1; C, kW, kH, M are the values of the four dimensions of the convolution kernel, N, W, C, and H are the values of the four dimensions of the convolution input data; if the first complexity is greater than a set threshold, determine whether the convolution input data and the convolution kernel are floating-point data, such as the convolution The input data and the convolution kernel are not floating-point data, the convolution input data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data, the convolution The kernel performs convolution operations with floating point data types. The method according to claim 1, wherein, if the first operation is a matrix-by-matrix operation, the input data is the first matrix of the matrix-by-matrix operation, and the weight is the second of the matrix-by-matrix operation. Matrix; first complexity=β*F1*G*E*F2; where, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, E, F2 are The row and column values of the second matrix; if the first complexity is greater than a preset threshold, determine whether the first matrix and the second matrix are floating point data, such as the first matrix and the second matrix are not floating point data data, convert the first matrix into floating-point data, convert the second matrix into floating-point data, and then perform a matrix-matrix-matrix operation on the first matrix and the second matrix in the floating-point data type. The method according to claim 1, wherein, if the first operation is a matrix multiplication vector operation, the input data is the first matrix of the matrix multiplication vector operation, and the weight value is the vector of the matrix multiplication vector operation; The first complexity=β*F1*G*F2; wherein, β is the matrix coefficient, the value range is greater than or equal to 1, F1, G are the row and column values of the first matrix, and F2 is the column value of the vector; If the first complexity is greater than a set threshold, determine whether the first matrix and the vector are floating-point data; if the first matrix and the vector are not floating-point data, convert the first matrix to floating-point data data, convert the vector to floating-point data, and then perform a matrix-multiply-vector operation on the first matrix, the vector, in the floating-point data type. The method according to any one of items 1 to 6 in the scope of the patent application, wherein the i-th layer further includes one or any combination of bias operation, full connection operation, GEMM operation, GEMV operation, and activation operation. An integrated circuit chip device, wherein the integrated circuit chip device is used to perform forward operation of a neural network, the neural network includes multiple layers, the device includes: a processing circuit and an external interface; the external interface is used for receiving a first calculation instruction ; The processing circuit is used to analyze the first calculation instruction to obtain the first operation that the first calculation instruction includes in the i-th layer of the forward operation, an input data corresponding to the first calculation instruction and a weight data, the The value range of i is an integer greater than or equal to 1, if the i is greater than or equal to 2, the input data is the output data of the i-1th layer; The processing circuit is further configured to determine the first complexity of the first operation according to the input data, the weight data and the first operation, and determine that the input data and the weight data are performing the first operation according to the first complexity The first data type at the time, the first data type includes: floating-point type or fixed-point type; the processing circuit is further configured to perform the first data type of the forward operation on the input data and the weight data with the first data type The first operation that a layer contains. The integrated circuit chip device according to claim 8, wherein the processing circuit is specifically configured to compare the first complexity with a preset threshold, and if the first complexity is higher than the preset threshold, the computing device It is determined that the first data type is a fixed-point type, and if the first complexity is lower than or equal to the preset threshold, the computing device determines that the first data type is a floating-point type. The integrated circuit chip device according to claim 9, wherein the integrated circuit chip device further includes a data type conversion circuit; the processing circuit is further configured to determine the second data type to which the input data and the weight data belong , if the second data type is different from the first data type, send a conversion command to the data type conversion circuit, and the data type conversion circuit is used to convert the input data belonging to the second data type and the input data belonging to the second data type according to the conversion command to the data type conversion circuit. The weight data of the second data type is converted into the input data of the first data type and the weight data of the first data type. The integrated circuit chip device according to claim 8, wherein, if the first operation is a convolution operation, the input data is a convolution input data, the weight data is a convolution kernel, and the processing circuit uses For calculating the first complexity, the first complexity=α*C*kW*kH*M*N*W*C*H; Among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kH, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the four dimensions of the convolution input data. value; the processing circuit is also used to determine whether the convolution input data and the convolution kernel are floating-point data if the first complexity is greater than a set threshold, if the convolution input data and the convolution kernel are not Floating point data, convert the convolution input data to floating point data, convert the convolution kernel to floating point data, then perform the convolution on the convolution input data, the convolution kernel with the floating point data type Product operation. The integrated circuit chip device according to claim 8, wherein, if the first operation is a matrix-by-matrix operation, the input data is a first matrix of the matrix-by-matrix operation, and the weight is the matrix-by-matrix operation the second matrix of are the row and column values of the first matrix, and E and F2 are the row and column values of the second matrix; the processing circuit is also used to determine the first matrix and the Whether the second matrix is floating-point data, if the first matrix and the second matrix are not floating-point data, convert the first matrix to floating-point data, and convert the second matrix to floating-point data , and then perform a matrix-matrix-matrix operation on the first matrix and the second matrix with floating-point data types. The integrated circuit chip device according to claim 8, wherein, if the first operation is a matrix multiplication vector operation, the input data is a first matrix of the matrix multiplication vector operation, and the weight is the matrix multiplication vector operation The processing circuit is used to calculate the first complexity; the first complexity=β*F1*G*F2; wherein, β is the matrix coefficient, and the value range is greater than or equal to 1, and F1 and G are the first The row and column values of a matrix, F2 is the column value of the vector; The processing circuit is further configured to determine whether the first matrix and the vector are floating-point data if the first complexity is greater than a set threshold, and if the first matrix and the vector are not floating-point data, determine whether the first matrix and the vector are floating-point data. Convert the first matrix to floating point data, convert the vector to floating point data, and then perform a matrix multiply-vector operation on the first matrix, the vector, in the floating point data type. The integrated circuit chip device according to any one of items 8 to 13 in the scope of the application, wherein the i-th layer further includes: one or any combination of bias operation, full connection operation, GEMM operation, GEMV operation, and activation operation . A neural network computing device, wherein, the neural network computing device includes one or more integrated circuit chip devices according to any one of items 8 to 14 of the patent application scope. A combined processing device, wherein the combined processing device includes: a neural network computing device, a general interconnection interface, and a general processing device as claimed in item 15 of the patent application scope; the neural network computing device communicates with the general purpose through the general interconnection interface Handling device connection. A wafer, wherein the wafer integrates a device as in any one of claims 8-14 of the claimed scope. An electronic device, wherein the electronic device includes a chip as claimed in claim 17 of the patented scope.

Images