TW201928794A - Neural network training method and related product having the advantages of small calculation amount and low power consumption - Google Patents

Abstract

The present disclosure provides a neural network training method performed on an integrated circuit chip device and a related product, the neural network comprising multiple layers, wherein the method includes the following steps: receiving a training instruction, and determining first layer input data and first layer weight data according to the training instruction, the computing device performing the nth layer forward operation of the neural network through the first layer input data and the first layer weight data to obtain the nth output result; obtaining the nth output result gradient according to the nth output result, and obtaining an nth inverse operation of the nth layer inverse operation according to the training instruction, and the computing device obtaining nth inverse operation complexity according to the nth output result gradient, the nth layer input data, the nth layer weight group data, and the nth inverse operation. The technical solution provided by the present disclosure has the advantages of small calculation amount and low power consumption.

Description

Neural network training method and related products

The present disclosure relates to the field of neural networks, and in particular, to a neural network training method and related products.

Artificial neural network (Artificial Neural Network, ANN) is a research hotspot that has emerged in the field of artificial intelligence since the 1980s. It abstracts the human brain neuron network from the perspective of information processing, establishes some simple model, and forms different networks according to different connection methods. In engineering and academia, it is often referred to as neural network or neural network. A neural network is a computing model that consists of a large number of nodes (or neurons) connected to each other. The existing neural network operations are based on the CPU (Central Processing Unit, Central Processing Unit) or GPU (Graphics Processing Unit, Graphics Processor) to implement the forward operation of the neural network. This forward operation has a large amount of calculation and high power consumption. .

The embodiments of the present disclosure provide a neural network training method and related products, which can improve the processing speed and efficiency of a computing device.

According to a first aspect, a method for training a neural network executed on an integrated circuit chip device is provided. The neural network includes n layers, and the method includes the following steps:

Receive a training instruction, determine the first layer of input data and the first layer of weight group data according to the training instruction, and the computing device executes the n-layer forward operation of the neural network to obtain a positive value through the first layer of input data and the first layer of weight group data. Output the result to the nth operation;

An n-th output result gradient is obtained according to the n-th output result, an n-th inverse operation of an n-th layer inverse operation is obtained according to the training instruction, an n-th output result gradient, an n-th input data, and an n-th weight are obtained according to the n-th output result gradient. The value group data and the nth inverse operation obtain the nth inverse operation complexity, and the nth output result gradient, the nth layer input data, and the nth layer weight group data corresponding to the nth output result gradient are determined according to the nth inverse operation complexity. The nth inverse data type, which uses the nth output result gradient, the nth layer input data, and the nth layer weight group data to perform the nth layer inverse operation of the neural network using the nth inverse data type to obtain the nth layer weight Group gradient and n-th input data gradient; applying the n-th weight group gradient to update the n-th weight group data; the n-th reverse data type includes: fixed-point type or floating-point type;

Use the n-th input data gradient as the n-1th output result of the n-1th layer. Perform the n-1 layer direction operation to get the n-1 layer weight group gradient. Apply the n-1 layer weight group gradient to update the corresponding layer. Weight group data, the weight group data includes; at least two weights.

According to a second aspect, an integrated circuit chip device is provided for performing a training operation of a neural network, the neural network includes n layers; the integrated circuit chip device includes: a processing circuit and an external interface;

The external interface is used to receive training instructions;

The processing circuit is configured to determine the first layer input data and the first layer weight group data according to the training instruction, and the computing device executes the n layer forward direction of the neural network through the first layer input data and the first layer weight group data. The operation obtains the n-th output result of the forward operation;

The processing circuit is further configured to obtain the n-th output result gradient according to the n-th output result, obtain the n-th reverse operation of the n-th layer reverse operation according to the training instruction, and obtain the n-th output result gradient, n Layer input data, nth layer weight group data, and nth inverse operation to obtain the nth inverse operation complexity, and determine the nth output result gradient, nth layer input data, and nth inverse operation complexity according to the nth inverse operation complexity. The n-th reverse data type corresponding to the n-th weight group data. The n-th output result gradient, the n-th input data, and the n-th weight group data are executed as the n-th reverse data type of the neural network. The operation obtains the n-th weight group gradient and the n-th input data gradient; applying the n-th weight group gradient to update the n-th weight group data; the n-th reverse data type includes: fixed point Type or floating point type;

The processing circuit is further configured to use the n-th input data gradient as the n-1th output result gradient of the n-1th layer, perform the n-1 layer direction operation to obtain the n-1 layer weight group gradient, and apply n-1 The layer weight group gradient updates the weight group data of the corresponding layer, the weight group data includes; at least two weights.

In a third aspect, a neural network computing device is provided. The neural network computing device includes one or more integrated circuit chip devices provided in the second aspect.

According to a fourth aspect, a combined processing device is provided. The combined processing device includes: a neural network computing device, a universal interconnection interface, and a universal processing device provided in the third aspect;

The neural network computing device is connected to the universal processing device through the universal interconnection interface.

According to a fifth aspect, a chip is provided, and the chip integrates the device of the second aspect, the device of the third aspect, or the device of the fourth aspect.

According to a sixth aspect, an electronic device is provided, and the electronic device includes the chip of the fourth aspect.

It can be seen that, according to the embodiment of the present disclosure, a data conversion operation circuit is provided to perform a conversion operation on the type of the data block, which saves transmission resources and calculation resources, so it has the advantages of low power consumption and small calculation amount.

In order to enable those skilled in the art to better understand the disclosure scheme, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are merely These embodiments are part of, but not all of the embodiments of this disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by persons with ordinary knowledge in the technical field without making creative labor fall into the scope of protection of the present disclosure.

In the method provided by the first aspect, determining the n-th reverse data type corresponding to the n-th output result gradient, the n-th input data, and the n-th weight group data according to the n-th reverse operation complexity, include:

Comparing the complexity of the n-th reverse operation with a preset threshold, if the complexity of the n-th reverse operation is higher than the preset threshold, determining that the n-th reverse data type is a fixed-point type, such as the n-th The complexity of the reverse operation is lower than or equal to the preset threshold, and the computing device determines that the n-th reverse data type is a floating point type.

In the method provided by the first aspect, the method determines the n-th output result gradient, the n-th input data, and the n-th reverse data corresponding to the n-th weight group data according to the n-th reverse operation complexity. After the type also includes:

Determine the n + 1th reverse data type to which the nth output result gradient, nth layer input data, and nth layer weight group data belong, such as the n + 1th reverse data type and the nth inverse data type The data types are different, and the n-th output result gradient, the n-th input data, and the n-th weight group data that belong to the n + 1-th reverse data type are converted into the n-th reverse data type. n output result gradient, n-th input data, and n-th weight group data.

In the method provided by the first aspect, if the n-layer inverse operation is a convolution operation, the convolution input data is the n-th layer input data, and the convolution kernel is the n-th output result gradient,

The nth reverse operation complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

If the complexity is greater than a set threshold, determine that the n-th reverse data type is a floating point data type, and determine whether the convolution input data and the convolution kernel are floating point data, such as the convolution input data and the convolution. The kernel is not floating-point data. The convolution input data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data and the convolution kernel are performed as floating-point data types. Product operation.

In the method provided by the first aspect, as the nth inverse operation is: matrix multiplication matrix operation, the input data is nth layer input data, and the weight value is the nth output result gradient;

Complexity = β * F * G * E * F; Among them, β is a matrix coefficient, the value range is 1 or more, F, G are the row and column values of the nth layer of input data, and E and F are weighted Row and column values;

If the complexity is greater than a set threshold, determine that the n-th reverse data type is a floating-point data type, and determine whether the n-th input data and weights are floating-point data, such as the n-th input data and weights. The value is not floating-point data. The n-th layer of input data is converted to floating-point data, the weights are converted to floating-point data, and then the n-th layer of input data and weights are executed as a floating-point data type. Multiplication matrix operation.

In the method provided by the first aspect, as the n-th reverse operation is: a matrix multiplication vector operation, the input data is n-th layer input data, and the weight is the n-th output result gradient;

Complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F and G are the row and column values of the input data of the nth layer, and F is the column of the nth output result gradient value;

If the complexity is greater than a set threshold, determine that the n-th reverse data type is a floating-point data type, and determine whether the n-th input data and weights are floating-point data, such as the n-th input data and weights. The value is not floating-point data. The n-th layer of input data is converted to floating-point data, the weights are converted to floating-point data, and then the n-th layer of input data and weights are executed as a floating-point data type. Multiply vector operations.

In the method provided by the first aspect, the n-th reverse operation may further include one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and activation operation.

In the apparatus provided by the second aspect, the processing circuit specifically compares the complexity of the n-th reverse operation with a preset threshold, and determines that the complexity of the n-th reverse operation is higher than the preset threshold. The n-th reverse data type is a fixed-point type. If the complexity of the n-th reverse operation is lower than or equal to the preset threshold, it is determined that the n-th reverse data type is a floating-point type.

In the device provided by the second aspect, the integrated circuit chip device further includes: a data type conversion circuit;

The processing circuit is further configured to determine an n + 1th reverse data type to which the nth output result gradient, the nth layer input data, and the nth layer weight group data belong, such as the n + 1th reverse A data type different from the n-th reverse data type, and sending a conversion command to the data type conversion circuit,

The data type conversion circuit is configured to convert the n-th output result gradient, the n-th input data, and the n-th weight group data that belong to the (n + 1) th reverse data type to the n-th reverse data type The n-th output result gradient, n-th input data, and n-th weight group data.

In the device provided by the second aspect, if the n-layer inverse operation is a convolution operation, the convolution input data is the n-th layer input data, and the convolution kernel is the n-th output result gradient,

The processing circuit is configured to calculate an n-th reverse operation complexity,

The nth reverse operation complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

The processing circuit is further configured to determine that the n-th reverse data type is a floating-point data type, and determine whether the convolution input data and the convolution kernel are floating-point data if the complexity is greater than a set threshold; The convolution input data and the convolution kernel are not floating-point data. The convolution input data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data and the convolution kernel are converted. Performs a convolution operation as a floating-point data type.

In the apparatus provided by the second aspect, as the n-th reverse operation is: matrix multiplication matrix operation, the input data is n-th layer input data, and the weight is the n-th output result gradient;

The processing circuit is configured to calculate an n-th reverse operation complexity,

The nth inverse operation complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F and G are the row and column values of the input data of the nth layer, E, F is the row and column value of the weight;

The processing unit is configured to determine that the n-th reverse data type is a floating-point data type, and determine whether the n-th-level input data and the weight are floating-point data, if the complexity is greater than a set threshold, such as The n-th input data and weights are not floating-point data. The n-th input data is converted to floating-point data, the weights are converted to floating-point data, and then the n-th input data and weights are Floating-point data types perform matrix-by-matrix operations.

In the apparatus provided by the second aspect, as the n-th inverse operation is: a matrix multiplying vector operation, the input data is n-th layer input data, and the weight is the n-th output result gradient;

The processing circuit is configured to calculate an n-th reverse operation complexity,

The complexity of the n-th reverse operation = β * F * G * F; where β is a matrix coefficient, and the value range is greater than or equal to 1, F and G are the row and column values of the input data of the nth layer, and F is the nth Output the column values of the gradient;

The processing circuit is further configured to determine that the n-th reverse data type is a floating-point data type, and determine whether the n-th input data and the weight are floating-point data, if the complexity is greater than a set threshold, such as The n-th layer of input data and weights are not floating-point data, the n-th layer of input data is converted to floating-point data, the weights are converted to floating-point data, and then the n-th layer of input data and weights Performs matrix multiplication vector operations on floating point data types.

As shown in Figure 1, the steps of neural network training include:

The layers in a (multi-layer) neural network perform forward operations in order;

Perform the inverse operation in turn in the order of the opposite layers to obtain the weight gradient;

Use the calculated weight gradient to update the weight of the forward operation;

This is the sequential iteration of the training of the neural network, and the entire training process needs to be repeatedly performed (that is, multiple iterative calculations). This process is repeated multiple times;

As shown in FIG. 1a, a neural network forward operation provided by an embodiment of the present disclosure, each layer uses its own input data and weights to calculate corresponding output data according to the operation rules specified by the layer type;

The neural network's forward computing process (also called inference) is a process of processing the input data of each layer layer by layer and obtaining the output data after a certain calculation. It has the following characteristics:

Inputs for a layer:

The input of a certain layer can be the input data of a neural network;

The input of one layer can be the output of other layers;

The input of a certain layer can be the output of the previous moment in this layer (corresponding to the case of recurrent neural network);

A layer can obtain input from multiple input sources at the same time;

Output of a layer:

The output of a layer can be used as the output of a neural network;

The output of one layer can be the input of other layers;

The output of a layer can be the input of this layer at the next moment (in the case of a recurrent neural network);

The output of a certain layer can output results to the above multiple output directions;

Specifically, the types of operations of the layers in the neural network include but are not limited to the following:

Convolutional layer (ie performing a convolution operation);

Fully connected layer (that is, performing a fully connected operation);

Normalization (regularization) layer: including LRN (Local Response Normalization) layer, BN (Batch Normalization) layer and other types;

Pooling layer

Activation layer: including but not limited to the following types of Sigmoid layer, ReLU layer, PReLu layer, LeakyReLu layer, Tanh layer;

Layer inversion operation, each layer needs to perform two parts of the operation: one is to use the output data gradient that may be a sparse representation and the input data that may be a sparse representation to calculate the weight gradient (used in the "weight Update "step to update the weights of this layer), the other part is to calculate the gradient of the input data using the output data gradient that may be sparse representation and the weight value that may be sparse representation (for the output data of the next layer in the inverse operation) Gradient for inverse operation);

The reverse operation transfers gradients in the reverse order from the last layer, starting from the last layer.

In an optional solution, the gradient of the output data obtained by inverse calculation of a certain layer can come from:

The gradient returned by the last loss function or cost function of the neural network;

Input data gradients of other layers;

The input data gradient at the previous moment in this layer (corresponding to the case of recurrent neural network);

A certain layer can obtain output data gradients from multiple sources at the same time;

After performing the inverse operation of the neural network, the gradient of the weights of each layer is calculated. In this step, the first input buffer and the second input buffer of the device are used to store the weights and Weight gradient, and then use the weight gradient in the arithmetic unit to update the weight;

The operations mentioned above are all one-level operations in neural networks. For multi-layer neural networks, the implementation process is that in the forward operation, after the execution of the artificial neural network in the previous layer is completed, the operation instructions in the next layer will calculate the operation. The output data calculated in the unit is used as the input data of the next layer (or some operation is performed on the output data and then used as the input data of the next layer), and the weight is also replaced by the weight of the next layer; In the inverse operation, after the inverse operation of the artificial neural network in the previous layer is completed, the operation instructions in the next layer will calculate the input data gradient calculated in the operation unit as the output data gradient in the next layer (or the input data). The gradient performs some operations and then acts as the output data gradient of the next layer), and at the same time replaces the weight with the weight of the next layer; (represented by the following figure, the dotted arrow in the following figure represents the reverse operation, and the solid arrow represents (Forward operation, the label below each figure indicates the meaning of the figure)

Representation of fixed-point data

The fixed-point method refers to converting the data representation of a data block in the network into a specific data representation of a fixed decimal point position (mapped to the 0/1 bit position of the data on the circuit device);

In an optional solution, a plurality of data is combined into a data block as a whole to perform fixed-point representation using the same fixed-point representation method;

FIG. 1b shows a specific representation method of a short-bit fixed-point data structure for storing data according to an embodiment of the present invention. Among them, 1Bit is used to represent a symbol, M is used to represent an integer part, and N is used to represent a decimal part. Compared to a 32-bit floating-point data representation, the short-bit fixed-point data representation used in the present invention is in addition to occupying bits In addition to fewer digits, for the data of the same layer and the same type in the neural network, such as the ownership value data of the first convolution layer, a flag point location is also set to record the position of the decimal point, which can be based on the actual data. The accuracy and range of data that can be represented by the distribution adjustment data.

The floating-point number is represented by 32bit, but for this technical solution, the use of fixed-point numbers can reduce the number of bits of a numerical value, thereby reducing the amount of data transmitted and the amount of data calculated.

The input data is shown in Figure 2a (N samples, each sample has C channels, and the height of the feature map of each channel is H and width W), and the weight, that is, the convolution kernel is shown in Figure 2b (M Convolution kernel, each convolution kernel has C channels, height and width are KH and KW respectively). For N samples of the input data, the rules of the convolution operation are the same. The process of performing the convolution operation on one sample is explained below. On one sample, each of the M convolution kernels must be the same. Operation, each convolution kernel operates to obtain a planar feature map, and M convolution kernels finally calculate to obtain M planar feature maps (for a sample, the output of the convolution is M feature maps), for a convolution kernel , To perform an inner product operation at each plane position of a sample, and then slide along the H and W directions. For example, Figure 2c shows a convolution kernel that performs an inner product operation at the lower right corner of a sample of input data. Correspondence map; Figure 2d shows the position of the convolution slide one grid to the left and Figure 2e shows the position of the convolution slide one grid up.

When the first operation is a convolution operation, the input data is convolution input data, and the weight data is a convolution kernel,

First complexity = α * C * kW * kW * M * N * W * C * H;

Among them, α is the convolution coefficient, and the value range is greater than 1. C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data;

If the first complexity is greater than a set threshold, determine whether the convolution input data and the convolution kernel are floating point data. If the convolution input data and the convolution kernel are not floating point data, input the convolution input. The data is converted into floating-point data, the convolution kernel is converted into floating-point data, and then the convolution input data and the convolution kernel are used to perform convolution operations in the floating-point data type.

Specifically, the convolution processing method may adopt a chip structure processing as shown in FIG. 3a. The data conversion operation circuit of the main processing circuit (also referred to as the main unit) may convert the weight when the first complexity is greater than a set threshold. The data in some or all of the convolution kernels is converted into fixed-point data. The control circuit of the main processing circuit sends the data in some or all of the weighted convolution kernels to be directly connected to the main processing circuit through the horizontal data input interface. Those basic processing circuits (also called basic units) (for example, the gray-filled vertical data path at the top in Figure 3b);

In an optional solution, the control circuit of the main processing circuit sends data of a certain convolution kernel in the weight value to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, the first Send the first number of the 3rd line once, send the 2nd number of the 3rd line data for the 2nd time, send the 3rd number of the 3rd line for the 3rd time ... or before the 3rd line of the 1st time Two numbers, the second time sends the 3rd and 4th numbers of the 3rd line, the third time sends the 5th and 6th numbers of the 3rd line ...;)

In another alternative, the control circuit of the main processing circuit sends the data of some convolution kernels in the weight value to the basic processing circuit one by one each time; (for example, for a certain A basic processing circuit that sends the first number of each line 3,4,5 for the first time, the second number of each line 3,4,5 for the second time, and the third number of 3, The 3rd number of each line 4,5 ... or the first two numbers of 3,4,5 lines are sent for the first time, the 3rd and 4th of 5th lines are sent for the second time 4 numbers, 3rd, 4th, 5th lines send 5th and 6th numbers per line ...;)

The control circuit of the main processing circuit divides the input data according to the position of the convolution. The control circuit of the main processing circuit sends the data in some or all of the convolution positions in the input data to the main processing circuit directly through the vertical data input interface. Connected basic processing circuits (for example, the gray-filled horizontal data path to the left of the basic processing circuit array in Figure 3b);

In an optional solution, the control circuit of the main processing circuit sends data of a convolution position in the input data to a basic processing circuit at a time or a portion of the data each time; (for example, for a basic processing circuit, the first Send the first number in the third column once, send the second number in the third column data for the second time, send the third number in the third column for the third time ..., or before the first column 3 Two numbers, the third and third numbers are sent in the third column, the third and third and fifth numbers are sent in the third column ...;)

In another optional solution, the control circuit of the main processing circuit sends data of some convolution positions in the input data to a basic processing circuit each time or a part of the data; (for example, for a certain A basic processing circuit that sends the first number of each of the 3, 4, and 5 columns for the first time, sends the second number of each of the 3, 4, 5 columns for the second time, and sends the third number of 3, The 3rd number of each column in 4,5 columns ..., or the first two numbers of 3,4,5 columns are sent for the first time, and the 3rd and 4th of 5th columns are sent for the second time. 4 numbers, send the 3rd, 4th, 5th columns for the third time, 5th and 6th numbers for each column ...;)

After the basic processing circuit receives the weighted data, it transmits the data to its next basic processing circuit through its horizontal data output interface (for example, the white-filled horizontal data path in the middle of the basic processing circuit array in Figure 3b). ); After receiving the input data, the basic processing circuit transmits the data to the next basic processing circuit connected to it through its vertical data output interface (for example, the white filled pad in the middle of the basic processing circuit array in Figure 3b) Vertical data path);

Each basic processing circuit performs operations on the received data;

In an optional solution, the basic processing circuit calculates a multiplication of one or more sets of two data at a time, and then accumulates the results in a register and / or an on-chip buffer;

In an optional solution, the basic processing circuit calculates an inner product of one or more groups of two vectors at a time, and then accumulates the results in a register and / or an on-chip buffer;

After the basic processing circuit calculates the result, the result can be transmitted from the data output interface;

In an optional solution, the calculation result may be a final result or an intermediate result of the inner product operation;

Specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface; if not, the result is output to the direction of the basic processing circuit that can directly output to the main processing circuit (for example, FIG. In 3b, the bottom line of the basic processing circuit directly outputs its output result to the main processing circuit, and the other basic processing circuits transmit the calculation results downward from the vertical output interface).

After receiving the calculation results from other basic processing circuits, the basic processing circuit transmits the data to other basic processing circuits or main processing circuits connected to the data;

Output the result in a direction that can be directly output to the main processing circuit (for example, the bottom row of the basic processing circuit outputs its output result directly to the main processing circuit, and the other basic processing circuits transmit the calculation result downward from the vertical output interface);

The main processing circuit receives the result of the inner product operation of each basic processing circuit and can obtain the output result.

Referring to FIG. 4a, FIG. 4a is a matrix-by-matrix operation. For example, the first operation is: matrix-by-matrix operation, the input data is the first matrix of the matrix-by-matrix operation, and the weight is The second matrix of matrix multiplication matrix operation;

The first complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is 1 or more, F and G are the row and column values of the first matrix, and E and F are the second matrix Row and column values

If the first complexity is greater than a set threshold, determine whether the first matrix and the second matrix are floating-point data. If the first matrix and the second matrix are not floating-point data, convert the first matrix into For floating-point data, the second matrix is converted into floating-point data, and then the first matrix and the second matrix are performed as matrix floating-matrix operations in the floating-point data type.

Referring to FIG. 4b, the matrix multiplication matrix operation is completed using the apparatus shown in FIG. 3b;

The following describes the calculation of a multiplication of a matrix S whose size is M rows and L columns and a matrix P whose size is L rows and N columns. (Each row in matrix S has the same length as each column of matrix P, as shown in Figure 2d.) The neural network computing device has K basic processing circuits:

Step S401b: When the first complexity is greater than a set threshold, the main processing circuit converts the matrix S and the matrix P into fixed-point type data, and the control circuit of the main processing circuit distributes each row of data in the matrix S to the K basic processing circuits. On one of them, the basic processing circuit stores the received data in an on-chip buffer and / or register; specifically, it can be sent to the basic processing circuits connected to the main processing circuit among the K basic processing circuits.

In an optional solution, if the number of rows of S M <= K, the control circuit of the main processing circuit distributes one row of the S matrix to the M basic processing circuits respectively;

In an alternative, if the number of rows of S is M> K, the control circuit of the main processing circuit distributes one or more rows of data in the S matrix to each basic processing circuit.

The Mi line in S is distributed to the i-th basic processing circuit. This set of Mi lines is called Ai, and Figure 2e shows the calculation to be performed on the i-th basic processing circuit.

In an alternative, in each basic processing circuit, for example, in the i-th basic processing circuit:

The received matrix Ai distributed by the main processing circuit stores the matrix Ai in the i-th basic processing circuit register and / or on-chip cache; the advantage is that the subsequent data transmission amount is reduced, the calculation efficiency is improved, and the power consumption is reduced.

Step S402b: The control circuit of the main processing circuit transmits each part of the matrix P to each basic processing circuit in a broadcast manner;

In an optional solution, each part of the matrix P may be broadcast only once to a register or an on-chip buffer of each basic processing circuit, and the i-th basic processing circuit fully multiplexes the data of the matrix P obtained this time. Complete the inner product operation corresponding to each row in the matrix Ai; the multiplexing in this embodiment may specifically be used repeatedly by the basic processing circuit in the calculation, for example, the multiplexing of the data of the matrix P may be the multiple of the data of the matrix P Times of use.

In an optional solution, the control circuit of the main processing circuit may broadcast the various parts of the matrix P to the registers or on-chip buffers of each basic processing circuit multiple times. Without multiplexing, the inner product operation corresponding to each row in the matrix Ai is completed in stages;

In an optional solution, the control circuit of the main processing circuit may broadcast the various parts of the matrix P to the registers or on-chip buffers of each basic processing circuit multiple times. Perform partial multiplexing to complete the inner product operation corresponding to each row in the matrix Ai;

In an optional solution, each basic processing circuit, for example, the i-th basic processing circuit, calculates an inner product of the data of the matrix Ai and the data of the matrix P;

Step S403b: The accumulator circuit of each basic processing circuit accumulates the result of the inner product operation and transmits the result to the main processing circuit.

In an optional solution, the basic processing circuit may transfer the part obtained by performing the inner product operation each time and transfer it to the main processing circuit for accumulation;

In an optional solution, the part obtained by the inner product operation performed by each basic processing circuit may also be stored in a register and / or an on-chip buffer of the basic processing circuit, and transferred to the main processing circuit after the accumulation is completed;

In an optional solution, the part obtained by the inner product operation performed by the basic processing circuit and in some cases may be stored in a register of the basic processing circuit and / or an on-chip buffer for accumulation, and may be transmitted to the host in some cases. The processing circuit performs accumulation, and after the accumulation is completed, it is transmitted back to the main processing circuit.

Refer to FIG. 4c, which is a schematic diagram of a matrix multiplying a vector. For example, the first operation is: a matrix multiplying vector operation, the input data is a first matrix of the matrix multiplying vector operation, and the weight value is a vector of the matrix multiplying vector operation;

The first complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F, G are the row and column values of the first matrix, and F is the column value of the vector;

If the first complexity is greater than a set threshold, determine whether the first matrix and vector are floating point data. If the first matrix and vector are not floating point data, convert the first matrix to floating point data. , Convert the vector to floating-point data, and then perform matrix multiplication vector operations on the first matrix and vector with floating-point data type.

Referring to FIG. 4d, FIG. 4d provides a method for implementing a matrix multiplication vector, which may specifically include:

Step S401: The data conversion operation circuit of the main processing circuit converts each row of data in the matrix S into data of a fixed point type, and the control circuit of the main processing circuit is distributed to one of the K basic processing circuits, and the basic processing circuit will receive The obtained distribution data is stored in the on-chip buffer and / or register of the basic processing circuit;

In an optional solution, if the number of rows of the matrix S is M <= K, the control circuit of the main processing circuit distributes one row of the S matrix to the K basic processing circuits respectively;

In an alternative, if the number of rows M of the matrix S is greater than K, the control circuit of the main processing circuit distributes one or more rows of data in the S matrix to each basic processing circuit.

The set of rows distributed to S in the i-th basic processing circuit is Ai, with a total of Mi rows. As shown in FIG. 2c, the calculation to be performed on the i-th basic processing circuit is shown.

In an optional solution, in each basic processing circuit, for example, the i-th basic processing circuit, the received distribution data, such as a matrix Ai, may be stored in a register and / or an on-chip buffer of the i-th basic processing circuit. ; The advantage is that it reduces the data transmission amount of the distributed data in the future, improves the calculation efficiency, and reduces the power consumption.

Step S402: The data type operation circuit of the main processing circuit converts the vector P into data of a fixed point type, and the control circuit of the main processing circuit transmits the parts of the vector P of the fixed point type to the K basic processing circuits in a broadcast manner;

In an optional solution, the control circuit of the main processing circuit may broadcast each part of the vector P to the registers or on-chip buffers of the basic processing circuits only once, and the data of the vector P obtained this time by the ith basic processing circuit Perform sufficient multiplexing to complete the inner product operation corresponding to each row in the matrix Ai. The advantage is that the data transmission amount of repeated transmission of the vector P from the main processing circuit to the basic processing circuit is reduced, the execution efficiency is improved, and the transmission power consumption is reduced.

In an optional solution, the control circuit of the main processing circuit may broadcast the parts of the vector P to the registers or on-chip buffers of each basic processing circuit multiple times. Without multiplexing, the inner product operation corresponding to each row in the matrix Ai is completed in stages; the advantage is that the data transmission amount of a single transmission vector P inside the basic processing circuit is reduced, and the basic processing circuit cache and / Or register capacity, improve execution efficiency, reduce transmission power consumption, and reduce costs.

In an optional solution, the control circuit of the main processing circuit may broadcast the parts of the vector P to the registers or on-chip buffers of each basic processing circuit multiple times, and the i-th basic processing circuit may obtain the data of the vector P each time. Perform partial multiplexing to complete the inner product operation corresponding to each row in the matrix Ai; the advantage is that it reduces the amount of data transmission from the main processing circuit to the basic processing circuit, and also reduces the amount of data transmission inside the basic processing circuit, improving execution efficiency To reduce transmission power consumption.

Step S403: The inner product operator circuit of the K basic processing circuits calculates the inner product of the data of the matrix S and the vector P, for example, the i-th basic processing circuit calculates the inner product of the data of the matrix Ai and the data of the vector P;

Step S404: The accumulator circuits of the K basic processing circuits accumulate the results of the inner product operation to obtain an accumulation result, and transmit the accumulation result to the main processing circuit in a fixed-point type.

In an optional solution, the partial sum (partial sum is a part of the accumulated result) obtained when the inner product operation is performed each time by the basic processing circuit, for example, the accumulated result is: F1 * G1 + F2 * G2 + F3 * G3 + F4 * G4 + F5 * G5, then the partial sum can be: the value of F1 * G1 + F2 * G2 + F3 * G3) is transmitted back to the main processing circuit for accumulation; the advantage is that the internal calculation amount of the basic processing circuit is reduced and the operation efficiency of the basic processing circuit is improved.

In an optional solution, the part obtained by the inner product operation performed by each basic processing circuit may be stored in a register and / or an on-chip buffer of the basic processing circuit, and then transferred back to the main processing circuit after the accumulation is completed; the advantage is, The data transmission amount between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, and the data transmission power consumption is reduced.

In an optional solution, the part obtained by the inner product operation performed by the basic processing circuit and in some cases may be stored in a register of the basic processing circuit and / or an on-chip buffer for accumulation, and may be transmitted to the host in some cases. The processing circuit accumulates and transmits back to the main processing circuit after the accumulation; the advantage is that the data transmission amount between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, the data transmission power consumption is reduced, and the basic processing circuit is reduced The internal calculation amount improves the calculation efficiency of the basic processing circuit.

Neural network training method

All the data involved in the neural network training process can use different data representation methods;

Specifically, the data representation method includes, but is not limited to, the following situations:

Floating point numbers with different bit widths;

Number of fixed points with different bit widths, and number of fixed points with different fixed point positions;

Different moments in the training process (specifically, different times of iteration or initialization), different stages in the training process (that is, forward or reverse operations), different layers, different data blocks in the same layer (that is, multiple Input data blocks, output data blocks), or sub-data blocks of different parts divided in the same data block, can:

You can use fixed or floating point separately;

For fixed points:

Use different fixed point widths;

Use different fixed-point offset values (that is, fixed-point positions);

The following uses a practical example to illustrate the specific implementation of neural network training. Figure 1a shows a specific calculation diagram of neural network training for a single-layer operation. As shown in Figure 1a, the input data and weights or parameters are used to execute this method. Layer operation. The technical solution provided in the embodiment of the present application determines whether to convert the input data and the type of the weight according to the input data, the weight value, and the forward operation amount of the layer. The specific method may be: such as the input data and The register or memory space occupied by the weight storage is greater than the set threshold and the forward operation amount of this layer is greater than the set operation amount. When it is determined that the input data and weight data are floating point data, the input data and weight data are converted Into fixed-point data. If the input data and the weight or the register or memory space occupied by the weight storage is less than the set threshold, if the input data and weight data are fixed-point data, the input data and weight data are converted into floating-point data, and then perform the calculation at this layer. .

The principle of the above data type conversion is explained in detail in this application. As shown in FIG. 1b, it is a method for expressing fixed-point data. For a computing system, the storage digits of a floating-point data are 32 bits. For fixed-point data, In particular, the floating-point data shown in Figure 1b is used to represent the data. The storage digits of one fixed-point data can be less than 16Bit, so for this conversion, the number of calculators can be greatly reduced. Transmission overhead. In addition, for a calculator, the storage space for data with fewer bits is smaller, that is, the storage overhead will be smaller, and the amount of calculation will be reduced. That is, the calculation overhead will be reduced, so the calculation overhead and storage can be reduced. Overhead, but also requires some overhead for the conversion of data types, referred to below as conversion overhead. For data with a large amount of calculation and large amount of data storage, the conversion overhead is relative to subsequent calculation overhead, storage overhead, and transmission overhead. It is almost negligible, so for data with a large amount of calculation and a large amount of data storage, this application uses the data type The technical solution for converting to fixed-point data. Conversely, for data with a small amount of calculation and small data storage, the calculation, storage, and transmission costs are relatively small at this time. If fixed-point data is used at this time, due to fixed-point data The accuracy will be slightly lower than the floating-point data. Under the premise of a small amount of calculation, the accuracy of the calculation needs to be guaranteed, so the fixed-point data is converted into floating-point data here, that is, the calculation is improved by adding a small overhead. The purpose of precision.

The following uses a practical example to illustrate. As shown in FIG. 4e, the operation method of this layer is matrix multiplication, and the input data and weights are both matrices. For the convenience of explanation, the input data here is matrix I and the weights are matrix W For example, as shown in Figure 4e, the output data = matrix I * matrix W; if the sum of the number of columns and rows of matrix I and matrix W is large, the above matrix I and matrix W can be considered to be in memory and / or The register takes up a lot of space and has a large amount of calculation. In this way, if matrix I and matrix W are floating-point data, matrix I and matrix W are converted into fixed-point data, and then matrix multiplication is performed.

For example, if matrix I is a matrix of 1000 * 1000, and matrix W is also a matrix of 1000 * 1000, then the sum of the number of columns and rows is 2000, which is a large number, and the corresponding calculation is greater. The matrix is multiplied by the matrix The multiplication of the inner product operation is 109 times. For this technical solution, due to the large number of matrix I and matrix W, it is impossible to transmit all the data at once, so the same data may be transmitted multiple times, assuming fixed-point data. Transmission, can greatly reduce the amount of data transmitted, and thus reduce transmission overhead. Relative to the calculation and storage of fewer bits can also reduce the calculation overhead and storage overhead.

For the technical solution of converting fixed-point data into floating-point data, take the reverse operation as an example, and the upward arrow direction of the calculation structure shown in FIG. 4g is a reverse operation. Take the inverse operation as an example. For the direction operation, the direction operation is the output data gradient. The output data gradient can be, for example, the output data gradient is the last layer of this iterative calculation, and the output data gradient is iteratively calculated. The output data of the last layer is obtained by a preset operation (the preset operation can be set by the manufacturer according to his own needs, and the specific operation steps of the preset operation are not limited here) to obtain the output data gradient. If the output data gradient is not the original The last layer of the iteration calculation, for example, the output data gradient is the nth layer of this iteration calculation, then the output data gradient is the input data gradient calculated by the inverse operation of the n + 1th layer.

The following uses a practical example to illustrate, as shown in Figure 4g, the operation method of this layer is matrix multiplication, the input data is a matrix, and the weight is a scalar. For the convenience of explanation, the input data here is matrix I, and the weight is scalar. C as an example, as shown in Figure 4g, the output data = matrix I * C; at this time, because the weight value is scalar data, the amount of data calculation is small, so if the matrix I is fixed-point data, the matrix I is converted to floating point The data is then subjected to a matrix multiply scalar operation.

For example, if the matrix I is a 10 * 10 matrix and the scalar is C, then for the sum of the number of columns and the number of rows is 20, the number is small, (assuming that greater than 100 is considered larger, less than 100 is considered smaller, The number of 100 can be arbitrarily set by those skilled in the art.) The corresponding calculation amount is very small. The multiplication of matrix multiplication by the inner product operation of the matrix is 102 times. Because the calculation amount is small, if you still use fixed-point data for calculation, it will It has an impact on its accuracy. In order to make the calculation accuracy higher, under the premise of a small amount of calculation, the calculation accuracy can be improved by floating-point data calculation.

In an optional solution, each data block of each layer in the network may adopt a fixed fixed-point bit width, but its fixed-point position changes with the training iteration cycle;

Specifically, during the training process, the data representation method of a certain data block can be set as follows;

Specifically, at the beginning of training, an arbitrary data representation method can be selected for a certain data block;

In an alternative, a floating point representation method with a specific bit width can be selected;

In an alternative, a specific form of fixed-point representation can be selected;

Can choose a specific fixed-point bit width;

Can choose a specific fixed-point position;

In an optional solution, the fixed-point position can be set according to the maximum value of the absolute value of all the data in the data block;

In an optional solution, the fixed-point position may be set according to the minimum value of the absolute values of all the data in the data block;

In an optional solution, the fixed-point position of the data block during initialization may be determined according to the fixed-point positions of other data blocks;

In an optional solution, the fixed-point position of the data block can be set according to the experience value;

Specifically, during the training process, the data representation method of a certain data block can be changed at any number of iteration cycles;

In an optional solution, no adjustment is required for a certain data block;

In an optional solution, adjustment can be performed every certain number of iterations;

In an alternative, it can be adjusted every certain training epoch number;

In an optional solution, adjustments can be made according to an irregular number of iterations;

In an alternative, the number of training epochs can be adjusted at irregular intervals;

Specifically, during the training process, when adjusting the representation method of a certain data block, it can be adjusted to any data representation method;

In an optional solution, if a data block is represented by a fixed fixed-point bit-width fixed-point number, the adjustment method of the fixed-point position represented by the data may be:

In an optional solution, the fixed-point position is set each time according to a method of initializing the fixed-point position;

In an optional solution, if the fixed-point position calculated by a data block according to the initial setting method of the fixed-point position increases in an iteration period compared to the previous iteration period, then the fixed-point position in the current period is increased. The method changes; otherwise, it changes in a decreasing direction.

The present disclosure also provides an integrated circuit chip device for performing training of a neural network. The neural network includes multiple layers. The integrated circuit chip device includes a processing circuit and an external interface.

The external interface is used to receive training instructions;

The processing circuit is configured to determine the first-layer input data and the first-layer weight data according to the training instruction, and perform the n-layer forward operation of the neural network to obtain the n-th through the first-layer input data and the first-layer weight data. Output result

The processing circuit is further configured to obtain the n-th output result gradient according to the n-th output result, obtain the n-th reverse operation of the n-th layer reverse operation according to the training instruction, and obtain the n-th output result gradient and the n-th input according to the n-th output result. Data, the nth layer weight group data, and the nth inverse operation to obtain the nth inverse operation complexity. The nth output result gradient, the nth layer input data, and the nth layer are determined according to the nth inverse operation complexity. The nth inverse data type corresponding to the weight group data. The nth output result gradient, the nth layer input data, and the nth layer weight group data are obtained by performing the nth layer inverse operation of the neural network with the nth inverse data type. n weight gradients for n-layer operations; the n-th reverse data type includes: a fixed-point type or a floating-point type;

The processing circuit is further configured to update the n weights of the n-layer operations by applying the n weight gradients.

This disclosure also discloses a neural network computing device, which includes one or more chips as shown in FIG. 3a or FIG. 3b, for obtaining data to be calculated and control information from other processing devices, and executing a specified neural network. The calculation and execution results are passed to the peripheral device through the I / O interface. Peripheral equipment such as camera, monitor, mouse, keyboard, network card, wifi interface, server. When more than one chip shown in Figure 3a or Figure 3b is included, the chips shown in Figure 3a or Figure 3b can be linked and transmitted through a specific structure, for example, interconnected and transmitted through the PCIE bus. Data to support larger-scale neural network operations. At this time, you can share the same control system, or you can have separate control systems; you can share memory, or each accelerator can have its own memory. In addition, its interconnection method can be any interconnection topology.

The neural network computing device has high compatibility and can be connected to various types of servers through a PCIE interface.

The present disclosure also discloses a combined processing device, which includes the aforementioned neural network computing device, a universal interconnection interface, and other processing devices (ie, general processing devices). The neural network computing device interacts with other processing devices to complete a user-specified operation. For example, 5a is a schematic diagram of a combined processing device.

Other processing devices include one or more types of processors such as a central processing unit CPU, a graphics processor GPU, and a neural network processor. The number of processors included in other processing devices is not limited. Other processing devices serve as the interface between the neural network computing device and external data and control, including data transfer, to complete the basic control of the neural network computing device, such as start and stop; other processing devices can also cooperate with the neural network computing device to complete computing tasks.

A universal interconnection interface for transmitting data and control instructions between the neural network computing device and other processing devices. The neural network computing device obtains required input data from other processing devices and writes it to a storage device on the neural network computing device chip; it can obtain control instructions from other processing devices and write it to the control buffer on the neural network computing device chip; also The data in the storage module of the neural network computing device can be read and transmitted to other processing devices.

As shown in FIG. 5b, optionally, the structure further includes a storage device for storing data required by the operation unit / operation device or other operation units, and particularly suitable for the data required for operation in the neural network operation device. Or all data that cannot be saved in the internal storage of other processing devices.

The combined processing device can be used as an SOC system-on-chip for devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of the control section, increasing processing speed, and reducing overall power consumption. In this case, the universal interconnection interface of the combined processing device is connected to some parts of the equipment. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.

Please refer to FIG. 5c, which is a schematic structural diagram of a neural network processor board provided by an embodiment of the present disclosure. As shown in FIG. 5 c, the neural network processor board 10 includes a neural network chip package structure 11, a first electrical and non-electrical connection device 12, and a first substrate 13.

The present disclosure does not limit the specific structure of the neural network chip package structure 11. Optionally, as shown in FIG. 5d, the aforementioned neural network chip package structure 11 includes: a neural network chip 111, a second electrical and non-electrical connection device 112, a first Two substrates 113.

The specific form of the neural network chip 111 involved in this disclosure is not limited. The aforementioned neural network chip 111 includes, but is not limited to, a neural network chip that integrates a neural network processor. The above chip may be made of silicon material, germanium material, quantum material, or molecule. Materials. According to the actual situation (for example: harsh environment) and different application requirements, the above neural network chip can be packaged so that most of the neural network chip is wrapped, and the pins on the neural network chip are passed through gold wires. The isoconductor is connected to the outer side of the package structure for circuit connection with the outer layer.

The present disclosure does not limit the specific structure of the neural network chip 111. For optional, please refer to the device shown in FIG. 1a or FIG. 1b.

The disclosure does not limit the types of the first substrate 13 and the second substrate 113, and may be a printed circuit board (PCB) or a printed wiring board (PWB), or may be other circuit boards. There are no restrictions on the materials used to make the PCB.

The second substrate 113 according to the present disclosure is used to carry the neural network chip 111, and a neural network chip package structure 11 obtained by connecting the neural network chip 111 and the second substrate 113 through a second electrical and non-electrical connection device 112. It is used to protect the neural network chip 111 and facilitate the further packaging of the neural network chip packaging structure 11 and the first substrate 13.

There is no limitation on the above-mentioned specific packaging method of the second electrical and non-electrical connection device 112 and the corresponding structure of the packaging method. A suitable packaging method can be selected and simply improved according to the actual situation and different application needs, such as flip chip Flip Chip Ball Grid Array Package (FCBGAP), Low-profile Quad Flat Package (LQFP), Quad Flat Package with Heat sink (HQFP), None Packaging methods such as Quad Flat Non-lead Package (QFN) or Fine-pitch Ball Grid Package (FBGA).

Flip chip (Flip Chip) is suitable for the case where the area after packaging is high or the inductance of the wire and the signal transmission time are sensitive. In addition, wire bonding can be used to reduce the cost and improve the flexibility of the packaging structure.

Ball Grid Array, which can provide more pins, and the average lead length of the pins is short, which has the function of transmitting signals at high speed. Among them, the package can be packaged with a pin grid array (Pin Grid Array, PGA) , Zero Insertion Force (ZIF), Single Edge Contact Connection (SECC), and Land Grid Array (LGA).

Optionally, the Flip Chip Ball Grid Array (Flip Chip Ball Grid Array) packaging method is used to package the neural network chip 111 and the second substrate 113. For a schematic diagram of a specific neural network chip packaging structure, refer to FIG. 6. As shown in FIG. 6, the aforementioned neural network chip package structure includes: a neural network chip 21, a pad 22, a solder ball 23, a second substrate 24, a connection point 25 on the second substrate 24, and a pin 26.

Among them, the pad 22 is connected to the neural network chip 21, and a solder ball 23 is formed by welding between the pad 22 and the connection point 25 on the second substrate 24, and the neural network chip 21 and the second substrate 24 are connected. Packaging of the neural network chip 21.

Pin 26 is used to connect with the external circuit of the package structure (for example, the first substrate 13 on the neural network processor board 10), and can realize the transmission of external data and internal data, which is convenient for the neural network chip 21 or the neural network chip 21 The corresponding neural network processor processes the data. The type and quantity of pins are not limited in this disclosure. Different pin forms can be selected according to different packaging technologies and arranged in accordance with certain rules.

Optionally, the aforementioned neural network chip package structure further includes an insulating filler placed in a gap between the pad 22, the solder ball 23, and the connection point 25 to prevent interference between the solder ball and the solder ball.

The material of the insulating filler may be silicon nitride, silicon oxide, or silicon oxynitride; interference includes electromagnetic interference, inductive interference, and the like.

Optionally, the aforementioned neural network chip package structure further includes a heat dissipation device for dissipating heat during operation of the neural network chip 21. The heat dissipation device may be a metal sheet, a heat sink, or a heat sink with good thermal conductivity, such as a fan.

For example, as shown in FIG. 6a, the neural network chip package structure 11 includes: a neural network chip 21, a pad 22, a solder ball 23, a second substrate 24, a connection point 25, a pin 26, The insulating filler 27, the heat dissipation paste 28, and the metal case heat sink 29. Among them, the heat dissipation paste 28 and the metal shell heat sink 29 are used to dissipate heat during the operation of the neural network chip 21.

Optionally, the aforementioned neural network chip package structure 11 further includes a reinforcing structure connected to the pad 22 and embedded in the solder ball 23 to enhance the connection strength between the solder ball 23 and the pad 22.

The reinforcing structure may be a metal wire structure or a columnar structure, which is not limited herein.

The present disclosure also does not limit the specific form of the first electrical and non-electrical device 12, and may refer to the description of the second electrical and non-electrical device 112, that is, the neural network chip packaging structure 11 is packaged by soldering, and connection may also be adopted. The method of connecting the second substrate 113 and the first substrate 13 in a line connection or plugging manner is convenient for subsequent replacement of the first substrate 13 or the neural network chip package structure 11.

Optionally, the first substrate 13 includes an interface of a memory unit for expanding the storage capacity, such as: synchronous dynamic random access memory (SDRAM), double-rate synchronous dynamic random access memory (Double Date Rate SDRAM, DDR), etc., to improve the processing capacity of the neural network processor by expanding the memory.

The first substrate 13 may further include a Peripheral Component Interconnect-Express (PCI-E or PCIe) interface, a Small Form-factor Pluggable (SFP) interface, and an Ethernet interface. , Controller Area Network (CAN) interface, etc., used for data transmission between the package structure and external circuits, which can improve the operation speed and the convenience of operation.

The neural network processor is packaged as a neural network chip 111, the neural network chip 111 is packaged as a neural network chip package structure 11, the neural network chip package structure 11 is packaged as a neural network processor board 10, and an interface on the board ( Slots or inserts) perform data interaction with external circuits (for example, computer motherboards), that is, the function of the neural network processor is realized directly by using the neural network processor board 10 and the neural network chip 111 is protected. In addition, other modules can be added to the neural network processor board 10, which improves the application range and operation efficiency of the neural network processor.

In one embodiment, the present disclosure discloses an electronic device including the neural network processor board 10 or the neural network chip package structure 11 described above.

Electronic devices include data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cameras, camcorders, projectors, watches, headphones, mobile storage , Wearables, vehicles, home appliances, and / or medical devices.

The vehicles include airplanes, ships, and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, cooker hoods, and the medical equipment includes nuclear magnetic resonance Instrument, type B ultrasound instrument and / or electrocardiograph.

The specific embodiments described above further describe the purpose, technical solution and beneficial effects of the present disclosure. It should be understood that the above are only specific embodiments of the present disclosure and are not used to limit the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of this disclosure shall be included in the protection scope of this disclosure.

A, B, S‧‧‧ Matrix

P‧‧‧ vector

S401b, S402b, S403b, S401, S402, S403, S404‧‧‧ steps

10‧‧‧ Neural Network Processor Card Board

11‧‧‧ neural network chip packaging structure

12‧‧‧First electrical and non-electrical connection device

13‧‧‧First substrate

111‧‧‧ neural network chip

112‧‧‧Second electrical and non-electrical connection device

113‧‧‧second substrate

1111‧‧‧Storage Unit

1112‧‧‧Direct Memory Access Unit

1113‧‧‧Instruction cache unit

1114‧‧‧ Rights Cache Unit

1115‧‧‧Input neuron buffer unit

1116‧‧‧Output neuron buffer unit

1117‧‧‧Control Unit

1118‧‧‧ Computing Unit

21‧‧‧Neural Network Chip

22‧‧‧ pad

23‧‧‧Solder Ball

24‧‧‧second substrate

25‧‧‧ Connection points on the second substrate 24

26‧‧‧pin

27‧‧‧Insulation filler

28‧‧‧ Thermal Paste

29‧‧‧ metal case heat sink

Figure 1 is a schematic diagram of a neural network training method.

Figure 1a is a schematic diagram of the forward operation of a neural network.

FIG. 1b is a schematic structural diagram of a fixed-point data type.

Figure 2a is a schematic diagram of convolution input data.

Figure 2b is a schematic diagram of a convolution kernel.

FIG. 2c is a schematic diagram of a calculation window of a three-dimensional data block of input data.

FIG. 2d is a schematic diagram of another operation window of a three-dimensional data block of input data.

Figure 2e is a schematic diagram of another calculation window of a three-dimensional data block of the input data.

Figure 3a is a schematic structural diagram of a neural network chip.

FIG. 3b is a schematic structural diagram of another neural network chip.

Figure 4a is a schematic diagram of matrix multiplication by matrix.

FIG. 4b is a flowchart of a method of matrix by matrix.

Figure 4c is a schematic diagram of a matrix multiplied by a vector.

FIG. 4d is a flowchart of a method of matrix multiplication by a vector.

Figure 4e is a schematic diagram of a neural network training.

Figure 4f is another schematic diagram of neural network training.

Figure 4g is a schematic diagram of the forward and reverse operations of the neural network.

Figure 4h is a schematic diagram of a multilayer structure for neural network training.

FIG. 5a is a schematic structural diagram of a combination processing device disclosed in the present disclosure.

FIG. 5b also discloses another structure diagram of a combined processing device in the present disclosure.

5c is a schematic structural diagram of a neural network processor board provided in an embodiment of the present disclosure;

5d is a schematic structural diagram of a neural network chip package structure provided by an embodiment of the present disclosure;

5e is a schematic structural diagram of a neural network chip according to an embodiment of the present disclosure;

6 is a schematic diagram of a neural network chip package structure provided by the embodiment of the present disclosure;

FIG. 6a is a schematic diagram of another neural network chip package structure provided by the embodiment of the present disclosure.

Claims (18)

A training method for a neural network executed on an integrated circuit chip device. The neural network includes n layers, where the value of n is an integer greater than or equal to 2, wherein the method includes the following steps: receiving a training instruction according to the training instruction Determine the first layer input data and the first layer weight group data, and the computing device performs the n-layer forward operation of the neural network by using the first layer input data and the first layer weight group data to obtain the n-th output of the forward operation. Result; Obtain the n-th output result gradient according to the n-th output result, obtain the n-th inverse operation of the n-th layer inverse operation according to the training instruction, according to the n-th output result gradient, the n-th input data, and the n-th layer The weight group data and the n-th inverse operation obtain an n-th inverse operation complexity. The n-th output result gradient, the n-th layer input data, and the n-th weight are determined according to the n-th inverse operation complexity. The nth inverse data type corresponding to the value group data, the nth output result gradient, the nth layer input data, and the nth layer weight group data are executed in the nth inverse data type to perform the nth in the neural network. Layer reverse Calculate the n-th weight group gradient and the n-th input data gradient; apply the n-th weight group gradient to update the n-th weight group data; the n-th reverse data type includes fixed-point type or floating Point type: Use the n-th input data gradient as the n-1th output result of the n-1th layer. Perform the n-1 layer direction operation to get the n-1 layer weight group gradient. Apply the n-1 layer weight. The group gradient updates a weight group data of a corresponding layer, and the weight group data includes at least two weights. The method according to item 1 of the scope of patent application, wherein the n-th output result gradient, the n-th layer input data, and the n-th weight group data corresponding to the n-th output result gradient are determined according to the n-th inverse operation complexity. The reverse data type includes: comparing the nth reverse operation complexity with a preset threshold, and if the nth reverse operation complexity is higher than the preset threshold, determining the nth reverse data type as a fixed-point type If the complexity of the n-th reverse operation is lower than or equal to the preset threshold, the computing device determines that the n-th reverse data type is a floating point type. The method according to item 2 of the scope of patent application, wherein the method determines the n-th output result gradient, the n-th layer input data, and the n-th weight group data corresponding to the n-th inverse operation complexity. The n-th reverse data type also includes: determining the n-th output result gradient, the n-th input data, and the n + 1-th reverse data type to which the n-th weight group data belongs, such as the n + 1 The reverse data type is different from the nth reverse data type. The nth output result gradient, the nth layer input data, and the nth weight group data that belong to the n + 1th reverse data type are converted into The n-th output result gradient, the n-th input data, and the n-th weight group data belonging to the n-th reverse data type. The method according to item 1 of the scope of patent application, wherein if the n-layer inverse operation is a convolution operation, the convolution input data is the n-layer input data, the convolution kernel is the n-th output result gradient, and the n-th Complexity of inverse calculation = α * C * kW * kW * M * N * W * C * H; where α is the convolution coefficient, and the value range is greater than 1; C, kW, kW, and M are the convolutions Kernel the values of the four dimensions, N, W, C, H are the values of the four dimensions of the convolutional input data; if the complexity of the nth reverse operation is greater than a set threshold, determine that the nth reverse data type is floating Point data type, to determine whether the convolution input data and the convolution kernel are floating point data. If the convolution input data and the convolution kernel are not floating point data, convert the convolution input data to floating point data. Point data, convert the convolution kernel into floating point data, and then input the convolution input data, and the convolution kernel performs a convolution operation in the floating point data type. The method according to item 1 of the scope of patent application, wherein if the nth inverse operation is a matrix multiplication matrix operation, the input data is the nth layer input data, and the weight is the nth output result gradient; n Inverse operation complexity = β * F * G * E * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F and G are the row and column values of the input data of the nth layer, and E, F is the row and column value of the weight; if the complexity of the n-th reverse operation is greater than a set threshold, determine that the n-th reverse data type is a floating-point data type, determine the n-th input data and the weight Whether the value is floating-point data, such as the n-th input data and the weight is not floating-point data, convert the n-th input data to floating-point data, and convert the weight to floating-point data , And then perform the matrix-by-matrix operation on the n-th input data and the weights as a floating-point data type. The method according to item 1 of the scope of patent application, wherein if the nth inverse operation is a matrix multiplication vector operation, the input data is the nth layer input data, and the weight is the nth output result gradient; n Inverse operation complexity = β * F * G * F; where β is a matrix coefficient and the value range is greater than or equal to 1, F and G are the row and column values of the input data of the nth layer, and F is the first The column value of the n output result gradient; if the complexity of the n-th reverse operation is greater than a set threshold, determine that the n-th reverse data type is a floating-point data type, determine whether the n-th input data and whether the weight is Floating point data, such as the nth layer input data and the weight value is not floating point data, the nth layer input data is converted into floating point data, the weight value is converted into floating point data, and then The n-th layer of input data and the weights perform the matrix multiplication vector operation in a floating-point data type. The method according to any one of claims 1 to 6, wherein the n-layer inverse operation further includes one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and activation operation. An integrated circuit chip device, wherein the integrated circuit chip device is used to perform a training operation of a neural network, the neural network includes n layers; the integrated circuit chip device includes: a processing circuit and an external interface; the external interface is used for receiving a Training instruction; the processing circuit is configured to determine the first layer input data and the first layer weight group data according to the training instruction, and the computing device executes the neural network through the first layer input data and the first layer weight group data The n-layer forward operation obtains the n-th output result of the forward operation; the processing circuit is further configured to obtain the n-th output result gradient according to the n-th output result, and obtain the n-th inverse operation of the n-th layer reverse operation according to the training instruction. Directional operation, according to the nth output result gradient, nth layer input data, nth layer weight group data, and the nth inverse operation to obtain an nth inverse operation complexity, according to the nth inverse operation complexity Determine the n-th output result gradient, the n-th input data, and the n-th reverse data type corresponding to the n-th weight group data, the n-th output result gradient, and the n-th input The data, the n-th weight group data, and the n-th inverse data type to perform the n-th layer inverse operation of the neural network to obtain the n-th weight group gradient and the n-th input data gradient; apply the n-th layer The weight group gradient updates the n-th layer weight group data; the n-th reverse data type includes: a fixed-point type or a floating-point type; the processing circuit is further configured to use the n-th input data gradient as the n-th The n-1th output result gradient of the -1 layer performs the n-1 layer direction operation to obtain the n-1 layer weight group gradient, and applies the n-1 layer weight group gradient to update the weight group data of the corresponding layer, and the weight value Group data includes; at least two weights. According to the integrated circuit chip device according to item 8 of the scope of patent application, the processing circuit specifically compares the nth inverse operation complexity with a preset threshold, such as the nth inverse operation complexity is higher than the preset A threshold value, determining that the n-th reverse data type is a fixed-point type, and if the complexity of the n-th reverse data operation is lower than or equal to the preset threshold, determining the n-th reverse data type as a floating-point type. The integrated circuit chip device according to item 9 of the scope of patent application, wherein the integrated circuit chip device further includes: a data type conversion circuit; the processing circuit is further configured to determine the n-th output result gradient, the n-th input data, The n + 1th reverse data type to which the nth layer weight group data belongs. If the n + 1th reverse data type is different from the nth reverse data type, a conversion command is sent to the data type conversion circuit. A data type conversion circuit, configured to convert the n-th output result gradient, the n-th layer input data, and the n-th weight group data that belong to the n + 1-th reverse data type into the n-th reverse data type The n-th output result gradient, the n-th input data, and the n-th weight group data. According to the integrated circuit chip device of the eighth patent application scope, if the n-layer inverse operation is a convolution operation, the convolution input data is the n-layer input data, and the convolution kernel is the n-th output result gradient, The processing circuit is configured to calculate the nth reverse operation complexity, where the nth reverse operation complexity = α * C * kW * kW * M * N * W * C * H; where α is a convolution coefficient , The value range is greater than 1; C, kW, kW, M are the values of the four dimensions of the convolution kernel, N, W, C, H are the values of the four dimensions of the convolution input data; the processing circuit, also It is used to determine if the complexity of the n-th reverse operation is greater than a set threshold, determine that the n-th reverse data type is a floating-point data type, determine whether the convolution input data and whether the convolution kernel is floating-point data; The convolution input data and the convolution kernel are not floating point data, the convolution input data is converted into floating point data, the convolution kernel is converted into floating point data, and then the convolution input data, The convolution kernel performs convolution operations on floating-point data types. According to the integrated circuit chip device of the eighth patent application scope, if the nth inverse operation is a matrix multiplication matrix operation, the input data is the nth layer input data, and the weight is the nth output result gradient The processing circuit is configured to calculate the complexity of the n-th reverse operation, where the complexity of the n-th reverse operation = β * F * G * E * F; where β is a matrix coefficient and the value range is greater than or equal to 1 , F, G are the row and column values of the nth layer of input data, and E and F are the row and column values of the weight; the processing unit is configured to, if the complexity of the nth inverse operation is greater than a set threshold, Determine that the n-th reverse data type is a floating-point data type, and determine whether the n-th input data and the weight are floating-point data, such as the n-th input data and the weight is not floating-point data , Convert the n-th input data to floating-point data, convert the weight to floating-point data, and then perform the matrix multiplication matrix operation on the n-th input data and the weight as a floating-point data type . According to the integrated circuit chip device of the eighth patent application scope, if the nth inverse operation is a matrix multiplication vector operation, the input data is the nth layer input data, and the weight is the nth output result gradient The processing circuit is configured to calculate the complexity of the n-th reverse operation, where the complexity of the n-th reverse operation = β * F * G * F; where β is a matrix coefficient, and the value range is greater than or equal to 1, F And G are row and column values of the n-th layer of input data, and F is a column value of the n-th output result gradient; the processing circuit is further configured to determine the complexity of the n-th reverse operation greater than a set threshold to determine the The n-th reverse data type is a floating-point data type. Determine whether the n-th input data and the weight are floating-point data. If the n-th input data and the weight are not floating-point data, The nth layer input data is converted into floating point data, the weight value is converted into floating point data, and then the nth layer input data and the weight value are used to perform the matrix multiplication vector operation in a floating point data type. The integrated circuit chip device according to any one of claims 8-13, wherein the n reverse operation further includes one or any combination of paranoid operation, fully connected operation, GEMM operation, GEMV operation, and activation operation. A neural network computing device, wherein the neural network computing device includes one or more integrated circuit chip devices according to any one of claims 8-14 of the scope of the patent application. A combined processing device, wherein the combined processing device includes: a neural network computing device, a universal interconnection interface, and a universal processing device, such as item 15 of the scope of patent application; the neural network computing device communicates with the universal through the universal interconnection interface The processing device is connected. A chip, wherein the chip integrates a device such as any one of items 8-14 of the scope of patent application. An electronic device, wherein the electronic device includes a chip as in item 17 of the scope of patent application.