CN112766472B

CN112766472B - Data processing method, device, computer equipment and storage medium

Info

Publication number: CN112766472B
Application number: CN201911060473.XA
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Cambricon Technologies Corp Ltd
Current assignee: Cambricon Technologies Corp Ltd
Priority date: 2019-11-01
Filing date: 2019-11-01
Publication date: 2024-04-12
Anticipated expiration: 2039-11-01
Also published as: WO2021082653A1; CN112766472A

Abstract

The present disclosure relates to a data processing method, apparatus, computer device, and storage medium. The disclosed board card comprises: a memory device, an interface device, and a control device, and an artificial intelligence chip including a data processing device; the artificial intelligent chip is respectively connected with the storage device, the control device and the interface device; a memory device for storing data; the interface device is used for realizing data transmission between the artificial intelligent chip and external equipment; and the control device is used for monitoring the state of the artificial intelligent chip. The data processing method, the data processing device, the computer equipment and the storage medium provided by the embodiment of the disclosure can improve the quantization precision, save the calculation time of the Winograd convolution and reduce the energy consumption.

Description

Data processing method, device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, apparatus, computer device, and storage medium.

Background

In the field of artificial intelligence technology, a neural network algorithm is a machine learning algorithm which is very popular recently, and has very good effects in various fields, such as image recognition, voice recognition, natural language processing and the like. With the development of neural network algorithms, the complexity of the algorithms is also higher and higher, and in order to improve the recognition degree, the scale of the model is also gradually increased. Processing with the GPU and CPU takes a significant amount of computation time and power consumption to start these large-scale models.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data processing method, apparatus, computer device, and storage medium that can save computation time, reduce power consumption, and improve computation accuracy.

According to an aspect of the present disclosure, there is provided a data processing method, the method including:

carrying out quantization processing on the data to be quantized according to the determined quantization parameters to obtain quantized data, wherein the quantization parameters are determined according to the corresponding statistical result and data bit width of the data to be quantized;

continuing to execute the wingrad convolution processing according to the quantized data to obtain a quantized wingrad convolution result;

and performing inverse quantization processing on the quantized Winograd convolution result to obtain a Winograd convolution result.

According to another aspect of the present disclosure, there is provided a data processing apparatus, the apparatus comprising:

the data quantization module is used for carrying out quantization processing on the data to be quantized according to the determined quantization parameters to obtain quantized data, wherein the quantization parameters are determined according to the corresponding statistical result and data bit width of the data to be quantized;

the convolution processing module is used for continuously executing the Winograd convolution processing according to the quantized data to obtain quantized Winograd convolution results;

And the dequantization processing module is used for performing dequantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result.

According to another aspect of the present disclosure, there is provided an artificial intelligence chip comprising a data processing apparatus as claimed in any one of the preceding claims.

According to another aspect of the present disclosure, there is provided an electronic device comprising an artificial intelligence chip as described above.

According to another aspect of the present disclosure, there is provided a board including: a memory device, an interface device and a control device, and an artificial intelligence chip as described above;

wherein the artificial intelligent chip is respectively connected with the storage device, the control device and the interface device;

the storage device is used for storing data;

the interface device is used for realizing data transmission between the artificial intelligent chip and external equipment;

the control device is used for monitoring the state of the artificial intelligent chip.

According to another aspect of the present disclosure, there is provided an electronic device including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the instructions stored by the memory to perform the method of any of the preceding claims.

According to another aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions, characterized in that the computer program instructions, when executed by a processor, implement the method of any of the preceding claims.

According to the data processing method, the device, the computer equipment and the storage medium, quantization processing is carried out on the data to be quantized according to the determined quantization parameters, so as to obtain quantized data, wherein the quantization parameters are determined according to the corresponding statistical result and data bit width of the data to be quantized; continuing to execute the wingrad convolution processing according to the quantized data to obtain a quantized wingrad convolution result; and performing dequantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result. The calculation time of the Winograd convolution is saved and the energy consumption is reduced while the quantization precision is improved.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a flow chart of a data processing method according to an embodiment of the present disclosure.

FIG. 2 illustrates a schematic diagram of a symmetric fixed-point number representation, according to an embodiment of the present disclosure.

FIG. 3 illustrates a schematic diagram of a fixed point number representation of an introduced offset in accordance with an embodiment of the present disclosure.

Fig. 4a and fig. 4b are graphs of the fluctuation range of the weight data of the neural network in the training process.

Fig. 5 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

Fig. 6 shows a block diagram of a board according to an embodiment of the present disclosure.

Fig. 7 illustrates a block diagram of an electronic device 800, according to an embodiment of the disclosure.

Fig. 8 illustrates a block diagram of an electronic device 1900 according to an embodiment of the disclosure.

Detailed Description

The following description of the technical solutions in the embodiments of the present disclosure will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments. Based on the embodiments in this disclosure, all other embodiments that a person skilled in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.

It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of this disclosure are used for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of this disclosure are taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the present disclosure is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in this disclosure and in the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present disclosure and claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in this specification and the claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

The data processing method according to the embodiment of the present disclosure may be applied to a processor, which may be a general-purpose processor such as a CPU (Central Processing Unit ) or an artificial Intelligence Processor (IPU) for performing an artificial intelligence operation. The artificial intelligence operations may include machine learning operations, brain-like operations, and the like. The machine learning operation comprises neural network operation, k-means operation, support vector machine operation and the like. The artificial intelligence processor may include, for example, one or a combination of GPU (Graphics Processing Unit ), NPU (Neural-Network Processing Unit, neural network processing unit), DSP (Digital Signal Process, digital signal processing unit), field-programmable gate array (Field-Programmable Gate Array, FPGA) chips. The present disclosure is not limited by the specific type of processor.

In one possible implementation, the processors referred to in this disclosure may include multiple processing units, each of which may independently execute various tasks assigned thereto, such as: convolution operation task, pooling task or full connection task, etc. The present disclosure is not limited to the tasks that the processing unit operates on.

The processor includes a plurality of processing units for executing sequences of instructions, and a memory unit for storing data, which may include a random access memory (RAM, random Access Memory) and a register file. Multiple processing units in a processor may share some memory space, such as shared part of RAM memory space and register files, as well as having separate memory spaces.

Winograd convolution is a convolution acceleration implementation based on a polynomial interpolation algorithm. It passes through two inputs to the convolution operation: and (3) dividing the neuron and the weight to a certain scale, respectively performing linear transformation (winograd forward transformation), performing para-multiplication on the transformed neuron and weight, and performing linear transformation (winograd inverse transformation) on the para-multiplication result again to obtain a convolution result equivalent to the original convolution operation.

The expression of the wingrad transformation is as follows:

for one-dimensional neurons and weights: s=a ^T ((Gg)⊙(B ^T d))

For neurons and weights in two dimensions: s=a ^T ((GgG ^T )⊙(B ^T dB))A

Wherein G represents the weight, G represents the left-by-positive transformation matrix corresponding to the weight, G ^T Represents the right-by-positive transformation matrix corresponding to the weight, d represents the input neuron, and B represents the right-by-positive transformation corresponding to the input neuronMatrix, B ^T Indicates the corresponding left-by-positive transformation matrix of the input neuron, as well as the para-multiplication operation, A indicates the right-by-inverse transformation matrix, A ^T Representing the inverse left-hand transform matrix. For input neurons of different dimensions, there are B and B corresponding thereto ^T The method comprises the steps of carrying out a first treatment on the surface of the Similarly, for weights of different dimensions, there are G and G corresponding thereto ^T 。

Substitution of the original convolution operation by the winograd convolution can bring greater benefits in terms of hardware energy efficiency ratio and operation time, and can realize higher neural network performance without increasing or increasing less hardware cost. However, the disadvantage of the winograd convolution is still relatively obvious, and a large number of multiplication operations still consume a relatively long operation time in the calculation process.

In order to solve the technical problems, the present disclosure provides a data processing method, a device, a computer device and a storage medium, which can disassemble multiplication operation in a trigrad convolution process into addition operation, so as to save calculation time, reduce energy consumption, and perform quantization processing on data in the trigrad convolution process, thereby further improving calculation performance.

Fig. 1 shows a flow chart of a data processing method according to an embodiment of the present disclosure. As shown in fig. 1, the method is applied to a processor, and the method includes steps S11 to S13.

In step S11, quantization processing is performed on the data to be quantized according to the determined quantization parameter, so as to obtain quantized data, where the quantization parameter is determined according to the corresponding statistical result and data bit width of the data to be quantized.

In one possible implementation, the method may further include: counting the data to be quantized, and determining the counting result of each type of data to be quantized; and determining corresponding quantized data by using the statistical result of each type of data to be quantized and the data bit width.

In one possible implementation, the statistics may include any of the following: the absolute value of each maximum value in the data to be quantized, the distance of the maximum value and the minimum value in each data to be quantized,

the quantization parameter may include one or more of a point location parameter, a scaling factor and an offset,

wherein the absolute value maximum is the absolute value of the maximum or minimum value in each type of data to be quantized.

In one possible implementation, the scaling factor is determined according to the point location parameter, the statistics, the data bit width.

In one possible implementation, the offset is determined according to statistics of each type of data to be quantized.

In one possible implementation, the point location parameter is determined from the statistics and the data bit width.

In one possible implementation, the quantization parameter may be divided into the following six cases. First case: the quantization parameter is a point location parameter s. In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following formula (1) _x ：

Wherein s is a point location parameter, I _x Representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and (3) adopting rounding operations such as rounding upwards, rounding downwards, rounding to zero and the like to replace round rounding operation in the formula (1). At this time, the maximum value A of the floating point number, which can be represented by the n-bit fixed point number, is 2 ^s (2 ^n-1 -1), then the n-bit fixed point number may represent a maximum of 2 in the number domain of the data to be quantized ^s (2 ^n-1 -1) the n-bit fixed point number may represent a minimum value of-2 in the number domain of the data to be quantized ^s (2 ^n-1 -1). From equation (1), it can be seen that the quantization parameter corresponding to the first case is used to quantize the data to be quantizedAt the time, the quantization interval is 2 ^s The quantization interval is denoted as C.

Let Z be the absolute maximum of all floating point numbers in the domain of the data to be quantized, then A needs to contain Z, and Z is greater thanThere is therefore the following constraint of equation (2):

2 ^s (2 ^n-1 -1)≥Z＞2 ^s-1 (2 ^n-1 -1) (2)

thus, the first and second substrates are bonded together,obtain->

The quantized n-bit binary representation value I of data x according to equation (3) _x Performing inverse quantization to obtain inverse quantized dataWherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

Second case: the quantization parameter is a first scaling factor f ₁ . In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following equation (4) _x ：

Wherein f ₁ For the first scaling factor, I _x Representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and (3) adopting rounding operations such as rounding upwards, rounding downwards, rounding to zero and the like to replace round rounding operation in the formula (4). As can be seen from the equation (4), when the quantization parameter corresponding to the second case is adopted to quantize the data to be quantized, the quantization interval is f ₁ The quantization interval is denoted as C.

For the first scaling factor f ₁ In terms of this, there is a case where: the point position parameter s is a fixed known value, no change occurs anymore, set 2 ^s T, T is a fixed value, then the maximum value a that can be expressed as a floating point number with an n-bit fixed point number is (2 ^n-1 -1) x T. In this case, the maximum value a depends on the data bit width n. Let Z be the maximum absolute value of all numbers in the number domain of the data to be quantizedAt this time z= (2 ^n-1 -1)×f ₁ . The n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ^n-1 -1)×f ₁ The n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×f ₁ . In yet another case, in engineering applications, 2 ^s ×f ₂ As a whole, as a first scaling factor f ₁ . At this time, it can be regarded that there is no independent point location parameter s. Wherein f ₂ Is the second scaling factor. Let Z be the absolute maximum of all numbers in the number domain of the data to be quantized, then +.>At this time z= (2 ^n-1 -1)×f ₁ . The n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ^n-1 -1)×f ₁ The n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×f ₁ 。

The quantized n-bit binary representation value I of the data x according to (5) _x Performing inverse quantization to obtain inverse quantized data Wherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

Third case: the quantization parameter is a point location parameter s and a second scaling factor f ₂ . In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following equation (6) _x ：

Wherein s is a point location parameter, f ₂ For the second scaling factor is,I _x representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and (3) adopting rounding operations such as rounding upwards, rounding downwards, rounding to zero and the like to replace round rounding operation in the formula (6). The maximum value a in the number domain of the data to be quantized, which can be represented by n-bit fixed-point number, is 2 ^s (2 ^n-1 -1). As can be seen from the formula (6), the third case is adoptedWhen the quantization parameter quantizes the data to be quantized, the quantization interval is 2 ^s ×f ₂ The quantization interval is denoted as C.

Let Z be the absolute maximum of all numbers in the number domain of the data to be quantized, at this time, it is available according to equation (2):

I.e. < ->

When, according to equation (2), Z can be represented with lossless accuracy. When f ₂ When=1, formula (6) and formula (1), ++>The n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ⁿ ^-1 -1)×2 ^s ×f ₂ The n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×2 ^s ×f ₂ 。

N-bit binary representation of quantized data x according to equation (7) I _x Performing inverse quantization to obtain inverse quantized dataWherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

In the present embodiment, FIG. 2 shows a solid according to the present disclosureA schematic representation of a symmetric fixed point number representation of an embodiment. The number field of the data to be quantized as shown in fig. 2 is distributed with "0" as the center of symmetry. Z is the absolute maximum value of all floating-point numbers in the number domain of the data to be quantized, A is the maximum value of the floating-point number which can be represented by n-bit fixed-point numbers in FIG. 2, and the floating-point number A is converted into fixed-point number 2 ^n-1 -1. In order to avoid overflow, a needs to contain Z. In actual operation, floating point data in the neural network operation process tends to be normally distributed in a certain determined section, but the distribution taking "0" as a symmetry center is not necessarily satisfied, and when the floating point data is represented by a fixed point number, overflow is easy to occur. To improve this, an offset is introduced in the quantization parameter. FIG. 3 illustrates a schematic diagram of a fixed point number representation of an introduced offset in accordance with an embodiment of the present disclosure. As shown in fig. 3. The number domain of the data to be quantized is not distributed by taking 0 as the symmetry center, Z _min Is the minimum value of all floating point numbers in the number domain of the data to be quantized, Z _max Is the maximum of all floating point numbers in the number domain of the data to be quantized. P is Z _min ～Z _max And (3) the center point is used for integrally shifting the number domain of the data to be quantized, so that the number domain of the data to be quantized after the shifting is distributed by taking 0 as a symmetrical center, and the maximum value of the absolute value in the number domain of the data to be quantized after the shifting is Z. As can be seen from fig. 3, the offset is the horizontal distance between the "0" point and the "P" point, which is referred to as offset O. Wherein,

based on the above description about the offset O, a case of the fourth quantization parameter occurs. Fourth case: the quantization parameters include point location parameters and offsets. In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following equation (8) _x ：

Wherein s is a point location parameter, O is an offset,I _x representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and replacing round rounding operation in the formula (8) by rounding operations such as rounding up, rounding down and the like. At this time, the maximum value A of the floating point number, which can be represented by the n-bit fixed point number, is 2 ^s (2 ^n-1 -1), then the n-bit fixed point number may represent a maximum of 2 in the number domain of the data to be quantized ^s (2 ^n-1 -1) +o, the n-bit fixed point number may represent a minimum value of-2 in the number domain of the data to be quantized ^s (2 ^n-1 -1) +o. As can be seen from equation (8), when the quantization parameter corresponding to the fourth case is used to quantize the data to be quantized, the quantization interval is 2 ^s The quantization interval is denoted as C.

Let Z be the absolute maximum of all floating point numbers in the domain of the data to be quantized,then A needs to contain Z, and Z is greater than +.>Obtain +.>And then obtain

The quantized n-bit binary representation value I of data x according to equation (9) _x Performing inverse quantization to obtain inverse quantized dataWherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

Based on the above description about the offset O, a case of a fifth quantization parameter occurs. Fifth case: the quantization parameter includes a first scaling factor f ₁ And an offset O. In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following formula (10) _x ：

Wherein f ₁ For the first scaling factor, O is the offset, I _x Representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and replacing round rounding operation in the formula (10) by rounding operations such as rounding up, rounding down and the like. At this time, there is a case that: the point position parameter s is a fixed known value, no change occurs anymore, set 2 ^s =t, T being a fixed value. Then, the maximum value A of the floating point number, which can be expressed by the n-bit fixed point number, is (2 ^n-1 -1) x T. In this case, the maximum value a depends on the data bit width n. Let Z be the maximum absolute value of all numbers in the number domain of the data to be quantizedAt this time z= (2 ^n-1 -1)×f ₁ . The n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ^n-1 -1)×f ₁ The n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×f ₁ . In yet another case, in engineering applications, 2 ^s ×f ₂ As a whole, as a first scaling factor f ₁ . At this time, it can be regarded that there is no independent point location parameter s. Wherein f ₂ Is the second scaling factor. Let Z be the absolute maximum of all numbers in the number domain of the data to be quantized, then +.>At this time z= (2 ^n-1 -1)×f ₁ . The n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ^n-1 -1)×f ₁ The +O, n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×f ₁ +O。

As can be seen from the equation (10), when the quantization parameter corresponding to the fifth case is used for quantizing the data to be quantized, the quantization interval is f ₁ The quantization interval is denoted as C.

The quantized n-bit binary representation value I of data x according to equation (11) _x Performing inverse quantization to obtain inverse quantized data Wherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

Based on the above description about the offset O, a case of a sixth quantization parameter occurs. Sixth case: the quantization parameter includes a point position parameter, a second scaling factor f ₂ And an offset O. In this case, the quantized data I can be obtained by quantizing the data to be quantized using the following formula (12) _x ：

Wherein s is the point location parameter, the offset O, f ₂ For the second scaling factor is,I _x representing the value for n-bit binary representation after quantization of data x, F _x For the floating point value before quantization of data x, round is the rounding operation performed for rounding. It should be noted that, the method is not limited to round, and other rounding methods may be used, for example: and replacing round rounding operation in the formula (12) by rounding operations such as rounding up, rounding down and the like. The maximum value a in the number domain of the data to be quantized, which can be represented by n-bit fixed-point number, is 2 ^s (2 ^n-1 -1). As can be seen from the equation (12), when the quantization parameter corresponding to the sixth case is used to quantize the data to be quantized, the quantization interval is 2 ^s ×f ₂ The quantization interval is denoted as C.

i.e. < ->

When, according to equation (2), Z can be represented with lossless accuracy. When f ₂ When the number of the codes is =1,the n-bit fixed point number may represent that the maximum value in the number domain of the data to be quantized is (2 ^n-1 -1)×2 ^s ×f ₂ The +O, n-bit fixed point number may represent that the minimum value in the number domain of the data to be quantized is- (2) ^n-1 -1)×2 ^s ×f ₂ +O。

The quantized n-bit binary representation value I of data x according to equation (13) _x Performing inverse quantization to obtain inverse quantized dataWherein the dequantized data +.>Data format of (2) and corresponding pre-quantization data F _x The data formats of the data are the same and are all floating point values.

In this embodiment, as can be seen from the formulas (1) to (13), the point location parameter and the scaling factor are both related to the data bit width. Different data bit widths result in different point location parameters and scaling factors, thereby affecting quantization accuracy. The quantization is a process of converting a high-precision number expressed by 32 bits or 64 bits into a fixed point number occupying less memory space, and the process of converting the high-precision number into the fixed point number causes a certain loss in precision. In the training or fine tuning process, the same data bit width quantization is used within a certain iteration (iterations) frequency range, so that the overall accuracy of the neural network operation is not greatly influenced. Beyond a certain number of iterations, the requirement of training or fine tuning on precision cannot be met by using the same data bit width quantization. This requires an adjustment of the data bit width n with the training or fine tuning process. Simply, the data bit width n may be set to a preset value by human. And calling the corresponding data bit width n which is set in advance in different iteration frequency ranges.

In one possible implementation, the method may further include:

adjusting the data bit width according to the quantization error corresponding to the data bit width to determine quantization parameters by using the adjusted data bit width,

wherein the quantization error is determined according to quantized data in the corresponding layer and corresponding pre-quantized data.

In one possible implementation, adjusting the data bit width according to the quantization error corresponding to the data bit width may include: comparing the quantization error with a threshold value, and adjusting the data bit width according to the comparison result. Wherein the threshold may comprise at least one of a first threshold and a second threshold. The first threshold is greater than the second threshold.

In one possible implementation, comparing the quantization error with a threshold, and adjusting the data bits according to the comparison result may include any one of:

increasing the data bit width when the quantization error is greater than or equal to the first threshold;

reducing the data bit width when the quantization error is less than or equal to the second threshold;

the data bit width remains unchanged while the quantization error is between the first threshold and the second threshold.

In this implementation, the first threshold and the second threshold may be empirical values or may be variable super parameters. The conventional super-parameter optimization method is suitable for the first threshold value and the second threshold value, and the super-parameter optimization scheme is not repeated here.

It should be emphasized that the data bit width can be adjusted according to a fixed bit number step, or according to a difference between the quantization error and the error threshold, the data bit width can be adjusted according to a variable adjustment step, and finally, according to the actual requirement of the neural network operation process, the data bit width is adjusted to be longer or shorter. Such as: the data bit width n of the current convolutional layer is 16, and is adjusted to 12 according to the quantization error. That is, in practical application, the data bit width n can meet the requirement of the neural network operation on the precision without 16, so that the fixed-point operation speed can be greatly improved within the precision allowable range, and the resource utilization rate of the artificial intelligent processor chip is improved.

In one possible implementation, the method may further include: performing inverse quantization on the quantized data to obtain inverse quantized data, wherein the data format of the inverse quantized data is the same as the data format of the corresponding data before quantization; and determining quantization errors according to the quantized data and the corresponding inverse quantized data.

In one possible implementation, the pre-quantized data may be the data to be quantized.

In one possible implementation, the processor may calculate the quantization error from the data to be quantized and its corresponding inverse quantized data. Let the data to be quantized be z= [ Z ] ₁ ,z ₂ …,z _m ]The inverse quantization data corresponding to the data to be quantized is Z ⁽ⁿ⁾ ＝[z ₁ ⁽ⁿ⁾ ,z ₂ ⁽ⁿ⁾ …,z _m ⁽ⁿ⁾ ]. The processor can quantize the data Z according to the data Z to be quantized and the corresponding inverse quantization data Z ⁽ⁿ⁾ An error term is determined and a quantization error is determined from the error term.

In one possible implementation, the processor may calculate the data Z to be quantized and the corresponding inverse quantized data Z, respectively ⁽ⁿ⁾ Obtaining m differences, and taking the sum of the m differences as an error term. The processor may then determine a quantization error from the error term. The specific quantization error can be determined according to the following formula:

wherein i is the index of the ith data to be quantized in the data set to be quantized. i is an integer greater than or equal to 1 and less than or equal to m.

It should be understood that the above determination method of the quantization error is only one example of the present disclosure, and those skilled in the art may set the determination method of the quantization error according to actual needs, which is not limited by the present disclosure.

For the data bit width, fig. 4a and fig. 4b are graphs of the fluctuation range of the weight data of the neural network in the training process. In fig. 4a and 4b, the abscissa represents the number of iterations and the ordinate represents the maximum value of the logarithm of the weight. The weight data fluctuation range curve shown in fig. 4a shows the weight data fluctuation conditions corresponding to different iterations in the same period (epoch) of any convolution layer of the neural network. In fig. 4B, the conv0 layer corresponds to the weight data fluctuation width curve a, the conv1 layer corresponds to the weight data fluctuation width curve B, the conv2 layer corresponds to the weight data fluctuation width curve C, the conv3 layer corresponds to the weight data fluctuation width curve D, and the conv4 layer corresponds to the weight data fluctuation width curve e. As can be seen from fig. 4a and 4b, the weight change range per iteration is relatively large in the initial training period (epoch). The change amplitude of the weight value of each iteration is not too large in the middle and later stages of training. In this case, in the middle and later stages of training, because the variation amplitude of the weight data before and after each iteration is not large, the weight data of the corresponding layers of each generation have similarity in a certain iteration interval, and the data bit width used in the quantization of the corresponding layers in the last iteration can be adopted in the quantization of the data involved in each layer in the neural network training process. However, in the initial training stage, since the variation amplitude of the weight data before and after each iteration is relatively large, in order to meet the precision of floating point operation required by quantization, the weight data of the corresponding layer of the current generation is quantized by using the data bit width adopted in the quantization of the corresponding layer of the previous generation, or the weight data of the current layer is quantized based on the preset data bit width n of the current layer, so as to obtain the quantized fixed point number. And determining quantization errors according to the quantized weight data and the corresponding weight data before quantization, adjusting the data bit width adopted in the quantization of the corresponding layer of the previous generation or the preset data bit width of the current layer according to the comparison result of the quantization errors and the threshold value, and applying the adjusted data bit width to the quantization of the weight data of the corresponding layer of the current generation. Further, in the training or fine tuning process, the weight data of each layer of the neural network are independent of each other and have no similarity. The neuron data among each layer are independent of each other because the weight data do not have similarity, and the weight data do not have similarity. Thus, during the neural network training or trimming process, the data bit width of each layer within each iteration of the neural network is applied to the corresponding layer. In the training or fine tuning process of the neural network, the data bit widths corresponding to the neuron data and the gradient data respectively are also taken as examples of the weight data, and are not repeated here.

In one possible implementation, the pre-quantized data is data to be quantized that is involved in a weight update iteration process within a target iteration interval. The target iteration interval comprises at least one weight updating iteration, and the same data bit width is adopted in the quantization process in the same target iteration interval.

In one possible implementation, the target iteration interval is determined according to a trend value of a point location parameter of the data to be quantized involved in the pre-determination time point weight update iteration process. Or the target iteration interval is determined according to the change trend value of the point position parameter and the change trend value of the data bit width of the data to be quantized, which are involved in the pre-judging time point weight updating iteration process. The pre-judging time point is used for judging whether the data bit width needs to be adjusted or not, and corresponds to the time point when the weight updating iteration is completed.

In one possible implementation, the determining of the target iteration interval may include:

determining a change trend value of the position parameter of the corresponding point of the data to be quantized in the weight iterative process at a pre-judging time point;

and determining a corresponding target iteration interval according to the change trend value of the point position parameter.

In the implementation manner, according to the formula (15), the change trend value of the point position parameter is determined according to the sliding average value of the point position parameter in the weight iteration process corresponding to the current pre-judgment time point and the sliding average value of the point position parameter in the weight iteration process corresponding to the last pre-judgment time point, or according to the point position parameter in the weight iteration process corresponding to the current pre-judgment time point and the sliding average value of the point position parameter in the weight iteration process corresponding to the last pre-judgment time point. The expression of formula (15) is:

diff _update1 ＝|M ^(t) -M ^(t-1) |＝α|s ^(t) -M ^(t-1) | (15)

in 15M is the running average of the point location parameter s as the training iterations increase. Wherein M is ^(t) For the point position parameter s corresponding to the t < th > pre-judgment time point, obtaining M according to a formula (16) along with the increasing sliding average value of training iteration ^(t) 。s ^(t) And the point position parameter s corresponding to the t < th > pre-judging time point is obtained. M is M ^(t-1) And the value is a sliding average value of point position parameters s corresponding to the t-1 th pre-judging time point, and alpha is a super parameter. diff (diff) _update1 Measuring the change trend of the point position parameter s, wherein the change of the point position parameter s also changes the phase and represents the maximum value Z of the data in the current data to be quantized _max Is a variation of (2). diff (diff) _update1 The larger the value range, the more widely varying the value range, requiring a shorter interval of update frequency, i.e., a smaller target iteration interval.

M ^(t) ←α×s ^(t-1) +(1-α)×M ^(t-1) (16)

In this implementation, the target iteration interval is determined according to equation (17). For the target iteration interval, the same data bit width is adopted in the quantization process in the same target iteration interval, and the data bit widths adopted in the quantization process in different target iteration intervals can be the same or different.

In formula (17), I is a target iteration interval. Beta and gamma are super parameters. diff (diff) _update1 Is the variation trend value of the point location parameter.

In this implementation, the pre-determined time points include a first pre-determined time point, which is determined according to the target iteration interval. Specifically, at the t-th pre-judgment time point in the training or fine tuning process, the weight data of the corresponding layer of the current generation is quantized by utilizing the data bit width adopted in the quantization of the corresponding layer of the previous generation, the number of fixed points after quantization is obtained, and the quantization error is determined according to the weight data before quantization and the corresponding weight data before quantization. Comparing the quantization error with a first threshold and a second threshold, respectively, and determining whether to pair the previous generation by using the comparison resultThe data bit width used in the layer quantization is adjusted. If: the t first prejudgement time point corresponds to the 100 th generation, and the data bit width used by the 99 th generation is n ₁ . At generation 100, according to data bit width n ₁ Confirming the quantization error, and comparing the quantization error with a first threshold value and a second threshold value to obtain a comparison result. If the data bit width n is confirmed based on the comparison result ₁ Without changing, the target iteration interval is confirmed to be 8 generations by using the formula (17), when the 100 th generation is used as the initial iteration in the current target iteration interval, the 100 th to 107 th generation are used as the current target iteration interval, and when the 100 th generation is used as the last iteration of the previous target iteration interval, the 101 st to 108 th generation are used as the current target iteration interval. Each generation still extends the data bit width n used in the last target iteration interval when quantizing within the current target iteration interval ₁ . In this case, the data bit width used in quantization between different target iteration intervals may be the same. If take 100 th generation to 107 th generation as the present goal iteration interval, then the 108 th generation in the next goal iteration interval is regarded as t+1st first prejudgement time point, if 101 th generation to 108 th generation are regarded as the present goal iteration interval, then the 108 th generation in the present goal iteration interval is regarded as t+1st first prejudgement time point. At the t+1st first pre-judging time point, according to the data bit width n ₁ Confirming the quantization error, and comparing the quantization error with a first threshold value and a second threshold value to obtain a comparison result. Determining the data bit width n based on the comparison result ₁ Need to be changed to n ₂ And confirming that the target iteration interval is 55 generations by using the formula (17). Then the 108 th to 163 th or 109 th to 163 th generation is used as the target iteration interval, and the data bit width n is used for each generation during quantization in the target iteration interval ₂ . In this case, the data bit width used in quantization may be different between different target iteration intervals.

In this implementation, regardless of whether the first predetermined time point is the initial iteration or the last iteration within the target iteration interval, equation (15) is adapted to obtain the trend value of the change in the point position parameter. If the first pre-determined time point at the current time is the initial iteration of the current target iteration interval, then(15) Wherein M is ^(t) A sliding average value s of point location parameter s corresponding to a point of time corresponding to a starting iteration of a current target iteration interval along with the increase of training iteration ^(t) Point location parameter s, M corresponding to the point in time corresponding to the starting iteration of the current target iteration interval ^(t-1) And the point position parameter s corresponding to the point in time corresponding to the initial iteration of the last target iteration interval is a sliding average value which increases along with the training iteration. If the first predetermined point in time at the current time is the last iteration of the current target iteration interval, then M in equation (15) ^(t) A sliding average value s of point location parameter s corresponding to the time point corresponding to the last iteration of the current target iteration interval along with the increase of training iteration ^(t) Point location parameter s, M corresponding to the time point corresponding to the last iteration of the current target iteration interval ^(t-1) The point position parameter s corresponding to the time point corresponding to the last iteration of the last target iteration interval is a sliding average value which increases along with the training iteration.

In this implementation, the pre-determination time point may further include a second pre-determination time point on the basis of including the first pre-determination time point. The second pre-determination time point is determined according to the data fluctuation range curve. Based on the data fluctuation range condition of big data in the neural network training process, a data fluctuation range curve shown in fig. 4a is obtained.

Taking weight data as an example, as can be seen from the data fluctuation width curve shown in fig. 4a, the data fluctuation width is very large every time the weight is updated in the iteration interval period from the training start to the T generation. At the current prejudging time point, during quantization, the current generation utilizes the data bit width n of the previous generation ₁ Performing quantization to obtain quantization result and corresponding quantization error determined by the data before quantization, comparing the quantization error with a first threshold and a second threshold respectively, and comparing the data bit width n according to the comparison result ₁ Adjusting to obtain data bit width n ₂ . By means of the data bit width n ₂ And quantizing the weight data to be quantized related to the current generation. Then determining a target iteration interval according to equation (17) to determine a first predetermined time point at which it is further determined whether to adjust the data bit widthAnd how to adjust and determine the next target iteration interval according to formula (17) to obtain the next first pre-determined point in time. In the period from the beginning of training to the iteration interval of the T generation, the change amplitude of the weight data before and after each iteration is very large, so that the weight data of the corresponding layers of each generation are not similar, in order to meet the precision problem, the data of each layer of the current generation cannot be delayed by the corresponding quantization parameter of the corresponding layer of the previous generation during quantization, the data bit width can be adjusted in the previous generation, at the moment, the data bit width used in each generation in the previous generation during quantization is different, and the target iteration interval is 1 generation. For optimal utilization of resources of the artificial intelligence processor chip, the target iteration interval of the previous T generation may be preset in advance according to the rule disclosed by the data fluctuation range graph shown in fig. 4a, namely: and directly presetting a target iteration interval of the previous generation T of the data fluctuation amplitude curve without confirming a time point when the weight updating iteration corresponding to each generation of the previous generation T is completed through a formula (17) as a second pre-judging time point. Thereby making the resource of the artificial intelligent processor chip more reasonably utilized. The data fluctuation range curve shown in fig. 4a has small fluctuation range from the T generation, the quantization parameter is reconfirmed without generation in the middle and later stages of training, the quantization error is determined by using the data before quantization and the data after quantization corresponding to the current generation in the T generation or the t+1 generation, whether the data bit width needs to be adjusted and how the data bit width needs to be adjusted are determined according to the quantization error, and the target iteration interval is also determined according to the formula (17). If the confirmed target iteration interval is 55 generations, this requires that the corresponding time point from the T generation or the t+1 generation later by 55 generations is taken as the first pre-judgment time point to judge whether to adjust the data bit width and how to adjust, and the next target iteration interval is determined according to the formula (17), so as to determine the next first pre-judgment time point until all generation operations in the same period (epoch) are completed. On the basis, after each period (epoch), adaptive adjustment is carried out on the data bit width or the quantization parameter, and finally the quantized data is used to obtain the neural network with the accuracy meeting the expectations.

In this implementation, if: according to the weight data fluctuation range graph shown in fig. 4a, it is determined that the value of T is 130 (this value does not correspond to fig. 4a, for convenience of description, it is only assumed that the value of T is 130, and is not limited to the assumed value.) then the 130 th generation in the training process is taken as the second pre-determination time point, the current first pre-determination time point is the 100 th generation in the training process, and at the 100 th generation, the target iteration interval is determined to be 35 generations through the formula (17). Training to 130 th generation in the target iteration interval, reaching a second pre-judging time point, determining whether the data bit width needs to be adjusted and how to adjust at the time point corresponding to 130 th generation, and determining the target iteration interval according to a formula (17). If the target iteration interval determined in this case is 42 generations. And taking the 130 th generation to the 172 th generation as a target iteration interval, wherein the 135 th generation corresponding to the first pre-judging time point determined when the target iteration interval is 35 th generation is within 42 th generation of the target iteration interval, and judging whether the data bit width needs to be adjusted or not and how to adjust according to a formula (17) in the 135 th generation. Instead of making the evaluation prediction at the 135 th generation, the evaluation of whether or not the data bit width needs to be adjusted and how to adjust can be directly performed until the 172 th generation. In summary, whether or not the evaluation and prognosis are performed at generation 135 is appropriate for the technical solution provided in the present disclosure.

In summary, a second preset time point is preset in advance according to the data fluctuation range curve, the target iteration interval is determined without spending resources of an artificial intelligent processor chip in the initial stage of training or fine tuning, the data bit width is directly adjusted according to the quantization error at the preset second preset time point, and the data to be quantized related to the current generation is quantized by utilizing the adjusted data bit width. In the middle and later stages of training or fine tuning, a target iteration interval is obtained according to formula (17), thereby determining corresponding first pre-determination time points, and determining whether and how to adjust the data bit width at each first pre-determination time point. Therefore, the precision of floating point operation required by neural network operation can be met, the resources of the artificial intelligent processor chip are reasonably utilized, and the efficiency in quantization is greatly improved.

In one possible implementation, not only the trend value of the point position parameter but also the trend value of the point position parameter and the trend value of the data bit width may be considered at the same time in order to obtain a more accurate target iteration interval of the data bit width. The step of determining the target iteration interval may comprise:

determining a change trend value of position parameters of corresponding points of data to be quantized and a change trend value of data bit width in a weight iteration process at a pre-judging time point; the pre-judging time point is used for judging whether the data bit width needs to be adjusted or not, and corresponds to the time point when the weight updating iteration is completed;

And determining a corresponding target iteration interval according to the change trend value of the point position parameter and the change trend value of the data bit width.

In this implementation, the trend value of the data bit width may be determined using the corresponding quantization error according to equation (18).

In the formula (19), delta is a super parameter, diff _bit Is quantization error; diff (diff) _update2 Is the variation trend value of the data bit width. diff (diff) _update2 Measuring change trend of data bit width n adopted in quantization, diff _update2 The larger the bit width of the update setpoint, the more likely it is that a shorter interval of update frequency is required.

In this implementation, the trend value of the point position parameter is still obtainable according to the formula (15), for M in the formula (15) ^(t) Obtained according to equation (16). diff (diff) _update1 Measuring the change trend of the point position parameter s, wherein the change of the point position parameter s also changes the phase and represents the maximum value Z of the data in the current data to be quantized _max Is a variation of (2). diff (diff) _update1 The larger the value range, the more widely varying the value range, requiring a shorter interval of update frequency, i.e., a smaller target iteration interval.

In this implementation, the target iteration interval is determined according to equation (19). For the target iteration interval, the same data bit width is adopted in the quantization process in the same target iteration interval, and the data bit widths adopted in the quantization process in different target iteration intervals can be the same or different.

In formula (19), I is a target iteration interval. Beta and gamma are super parameters. diff (diff) _update1 Is the variation trend value of the point location parameter. diff (diff) _update2 Is the variation trend value of the data bit width.

In this implementation, diff _update1 Is used to measure the change in the point location parameter s, but the change in the point location parameter s due to the change in the data bit width n is ignored. Since this is already at diff _update2 The variation of the data bit width n is reflected. If at diff _update1 If the target iteration interval I determined according to the formula (19) is inaccurate, resulting in excessive first predetermined time points, and if and how to update the data bit width n is easy to frequently perform in the training or fine tuning process, thereby resulting in unreasonable utilization of resources of the artificial intelligence processor chip.

In this implementation, diff _update1 According to M ^(t) And (5) determining. Assume that the data bit width corresponding to the t-1 th prejudgement time point is n ₁ The corresponding point location parameter is s ₁ The sliding average value of the point position parameter increased along with training iteration is m ₁ . By means of the data bit width n ₁ And quantizing the data to be quantized to obtain the quantized fixed-point number. Determining quantization error diff based on pre-quantization data and corresponding post-quantization data _bit According to quantization error diff _bit Comparing the data bit width n with the threshold value ₁ Adjusted to n ₂ Data bit width is adjusted by |n ₁ -n ₂ The data bit width used in the quantification of the t-th pre-judgment time point is n ₂ . In order to ignore the change of point position parameters caused by the change of data bit width, M is determined ^t ) One of the following two optimization modes can be selected. The first way is: if the data bit width increases by |n ₁ -n ₂ The I bit is s ^(t-1) Take the value s ₁ -|n ₁ -n ₂ |，M ^(t-1) Take the value of m ₁ -|n ₁ -n ₂ I, will s ^(t-1) 、M ^(t-1) Substituting into formula (16) to obtain M ^(t) And the point position parameter corresponding to the t < th > pre-judging time point is a sliding average value which increases along with training iteration. If the data bit width is reduced by |n ₁ -n ₂ The I bit is s ^(t-1) Take the value s ₁ +|n ₁ -n ₂ |，M ^(t-1) Take the value of m ₁ +|n ₁ -n ₂ I, will s ^(t-1) 、M ^(t-1) Substituting into formula (16) to obtain M ^(t) And the point position parameter corresponding to the t < th > pre-judging time point is a sliding average value which increases along with training iteration. The second way is: regardless of whether the data bit width is increased by |n ₁ -n ₂ The i bit is also reduced by n ₁ -n ₂ |，s ^(t-1) Take the value s ₁ ，M ^(t-1) Take the value of m ₁ Will s ^(t-1) 、M ^(t-1) Substituting into formula (16) to obtain M ^(t) . Increase in data bit width by |n ₁ -n ₂ When in the I bit, M is ^(t) Subtracting |n ₁ -n ₂ I, decrease in data bit width by |n ₁ -n ₂ When in the I bit, M is ^(t) Plus |n ₁ -n ₂ And the result is taken as a sliding average value of the point position parameter corresponding to the t < th > pre-judging time point along with the increase of training iteration. The two modes are equivalent, and the change of the point position parameters caused by the change of the data bit width can be ignored, so that a more accurate target iteration interval is obtained, and the resource utilization rate of the artificial intelligence processor chip is improved.

In practical applications, the data bit width n and the point location parameter s have a great influence on quantization, and the scaling factor f and the offset O in the quantization parameter have little influence on quantization. It is therefore also a significant matter to determine the target iteration interval for the point location parameter s, regardless of whether the data bit width n is changed or not, and the point location parameter s is variable.

In one possible implementation, the process of determining the target iteration interval may include the steps of:

determining a change trend value of the position parameter of the corresponding point of the data to be quantized, which is involved in the weight iterative process, at a pre-judging time point; the pre-judging time point is used for judging whether the quantization parameter needs to be adjusted or not, and corresponds to the time point when the weight updating iteration is completed;

In this implementation, the quantization parameter is preferably a point location parameter.

In step S12, a wingrad convolution process is continuously performed according to the quantized data, so as to obtain a quantized wingrad convolution result.

In step S13, inverse quantization processing is performed on the quantized wingrad convolution result, so as to obtain a wingrad convolution result.

In one possible implementation, the wingrad convolution processing procedure includes:

resolving the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain a wingrad forward transformation result of each data in the data to be operated;

performing para-multiplication operation between the wingrad positive conversion results of corresponding data in the data to be operated to obtain para-multiplication results;

resolving the inverse wingrad transform of the para-multiplication result into sum operation to obtain the wingrad convolution result,

the data to be quantized may be any one of the data to be operated, a winograd positive transformation result of the data to be operated, and the para-multiplication result.

In one possible implementation, the data to be operated on includes at least one of an input neuron, a weight, and a gradient.

The data to be quantized, the data to be operated, and the like mentioned in the embodiments of the present disclosure may be data that can occur during actual data processing, and may be data corresponding to data such as image data, video data, audio data, text data, and the like. For example, the method provided by the present disclosure is used for image processing, video processing, audio processing, and other scene applications. Taking the data to be operated as image data as an example, the data to be operated may be represented in the form of NHWC (batch, height, width, channels), N represents the number of images, HW represents the number of pixels in the height and width directions, respectively, C may represent the number of channels, for example, C may represent three channels of RGB (Red, green, blue), which is only one example of the present disclosure, and the present disclosure is not limited thereto. It should be noted that the above method may be applied to any data that may be quantized and subjected to a winograd convolution operation, and those skilled in the art may set the type of the data and the corresponding application scenario according to actual needs, which is not limited in this disclosure.

For example, the data to be quantized may be quantized, thereby increasing the processing speed of the winograd convolution. In some embodiments, the data to be quantized may be a 32-bit floating point number. Alternatively, the data to be quantized may be floating point numbers of other bits, or other data types.

In one possible implementation manner, the quantizing processing is performed on the data to be quantized according to the determined pair of cut-off thresholds, so as to obtain quantized data, which includes any one of the following operations:

before the wingrad forward conversion of the data to be operated is disassembled into summation operation, the data to be operated is used as the data to be quantized to carry out quantization processing;

before carrying out the para-multiplication operation, taking a wingrad positive conversion result of each data in the data to be operated as the data to be quantized to carry out quantization processing;

and before performing inverse winograd transform and decomposition to sum operation, performing quantization processing on the para-multiplication result as data to be quantized.

Illustratively, if the data to be quantized is data to be calculated, the wingrad convolution process may be:

quantizing the data to be operated by adopting the determined pair of cut-off thresholds to obtain quantized data to be operated; resolving the Winograd positive transformation of the quantized data to be operated into sum operation, and calculating to obtain the Winograd positive transformation result of the quantized data to be operated; performing para-multiplication operation of the quantized wingrad forward conversion result of the data to be operated to obtain a para-multiplication result; and decomposing the inverse Winograd transform of the para-multiplication result into a sum operation to obtain the quantized Winograd convolution result, and performing inverse quantization processing on the quantized Winograd convolution result to obtain the Winograd convolution result.

For example, if the data to be quantized is a wingrad forward transformation result of the data to be operated, the wingrad convolution process may be:

resolving the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain a wingrad forward transformation result of the data to be operated; quantizing a wingrad positive transformation result of the data to be operated by adopting the determined pair of cut-off thresholds to obtain a quantized wingrad positive transformation result of the data to be operated; performing para-multiplication operation of the quantized wingrad forward conversion result of the data to be operated to obtain a para-multiplication result; and carrying out inverse quantization processing on the quantized Winograd convolution result to obtain the Winograd convolution result.

Illustratively, if the data to be quantized is a para-multiplication result, the wingrad convolution process may be:

resolving the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain a wingrad forward transformation result of the data to be operated; performing para-multiplication operation of a wingrad forward conversion result of the data to be operated to obtain a para-multiplication result; quantizing the para-multiplication result by adopting the determined pair of cut-off thresholds to obtain a quantized para-multiplication result; and decomposing the inverse Winograd transform of the quantized para-multiplication result into a sum operation to obtain a quantized Winograd convolution result. And performing inverse quantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result.

In one possible implementation manner, the disassembling the wingrad forward transformation of the data to be operated into sum operation, and performing calculation to obtain a wingrad forward transformation result of each data in the data to be operated includes:

respectively disassembling each data in the data to be operated into a plurality of first sub-tensors, performing wingrad forward transformation on the plurality of first sub-tensors of each data in the data to be operated and summing to obtain a wingrad forward transformation result of each data in the data to be operated,

the number of the plurality of first sub-tensors split out of each data is the same as the number of elements which are not 0 in the corresponding data, one element in each first sub-tensor is the same as the element in the corresponding position in the corresponding data, and other elements are all 0.

For example, assume that the input neuron is represented as:

the input neuron is a 4×4 matrix including 16 elements, and thus, the data to be operated on can be disassembled into 16 first sub-tensors.

Then, according to the disassembly of the present disclosure, the 16 first sub-tensors are respectively:

one element in each first sub-tensor is the same as the element in the corresponding position in the data to be operated, and the other elements are all 0, which means that: with a first sub-tensor d ₀₀ For example, the element at the first row and first column position is the same as the element of the input neuron at the first row and first column position, the other elements are all 0, and the other first sub-tensors have the same attribute.

It should be noted that the above disassembly is only some examples of the disclosure, and the disclosure is not limited in any way, for example, if the data to be operated has an element with a value of 0, the number of first sub-tensors obtained by disassembly may be less than the number of elements of the data to be operated, for example, the number of the plurality of first sub-tensors is the same as the number of elements of the data to be operated other than 0.

In one possible implementation manner, performing a wingrad forward transformation on a plurality of first sub-tensors of each data in the data to be operated on and summing to obtain a wingrad forward transformation result of each data in the data to be operated on may include:

acquiring a wingrad positive transformation result of a first unitary tensor corresponding to the first tensor; the first meta-tensor corresponding to the first tensor is: the value of the element of the first position in the first meta-tensor is 1, wherein the position of the first position in the first meta-tensor is the same as the position of the non-0 element in the first tensor;

Taking the element value which is not 0 in the first sub-tensor as a coefficient to multiply the corresponding wingrad positive transformation result of the first meta-tensor to obtain the wingrad positive transformation result of the first sub-tensor;

and adding the wingrad positive conversion results of the first sub-tensors to obtain the wingrad positive conversion result of the data to be operated.

Still with the first sub-tensor d ₀₀ For example, d ₀₀ The corresponding first element tensor may beThat is, the first element tensor is extracted from the non-0 element values in the first tensor, and the non-0 element values can be used as coefficients of the first element tensor.

The wingrad positive transformation result of the first meta-tensor corresponding to the first tensor may be obtained in advance through the following process: and for each first sub-tensor, multiplying the left side of the first meta-tensor corresponding to the first sub-tensor by a positive transformation left multiplication matrix and multiplying the right side of the first meta-tensor by a positive transformation right multiplication matrix to obtain a winograd positive transformation result of the first meta-tensor.

For matrices of different sizes, the form of the corresponding first element tensor is determined, as are the corresponding forward left-hand and forward right-hand matrices.

Thus, the first can be calculated in advanceThe detailed procedure of the wingrad forward conversion result of the meta-sub tensor is as described above. For example, still at d ₀₀ For example, the corresponding wingrad forward transform result of the first meta-tensor is:

for another example, as d ₀₁ For example, the corresponding wingrad forward transform result of the first meta-tensor is:

since the element values of the forward left-hand matrix and the forward right-hand matrix are both 0, ±1, the element value of the first meta-tensor is 0 or 1, and the element in the wingrad forward result of the first meta-tensor is also 0, ±1. Thus, the matrix multiplication operation can be disassembled into addition operations.

The process of calculating the winnrad forward conversion result of the first element tensor involves more multiplication operations, and by the method of the disclosure, the winnrad forward conversion result of the first element tensor with various pre-calculated scales can be stored in the operation device, so that the calculation result can be directly obtained without repeated operation in the actual operation process, thereby shortening the calculation time and saving the calculation resources.

And obtaining a wingrad positive conversion result of the first unitary tensor corresponding to the first tensor, multiplying the element value which is not 0 in the first tensor by the wingrad positive conversion result of the corresponding first unitary tensor, thereby obtaining the wingrad positive conversion result of the first tensor. For example, still at d ₀₀ For example, the corresponding wingrad forward transform result is:

for another example, as d ₀₁ For example, d ₀₁ The wingrad positive transformation result of (2) is

And calculating through the process to obtain the wingrad positive conversion results of all the first sub-tensors, and adding the wingrad positive conversion results of the plurality of first sub-tensors to obtain the wingrad positive conversion results of the data to be operated.

Since the elements in the wingrad positive transform result of the first meta-tensor obtained by the conversion are also 0, ±1, the right side of the above equations (20), (21) involves only the summation operation.

According to the above embodiments of the present disclosure, the data to be calculated is disassembled to obtain a plurality of first sub-tensors, and the sum operation is performed according to the pre-calculated wingrad forward conversion result of the first unitary sub-tensor corresponding to the first sub-tensor and the non-0 element value of the first sub-tensor to obtain the wingrad forward conversion result of the data to be calculated.

After the above-mentioned disassembly is adopted to obtain the wingrad forward conversion result of the input neuron, the wingrad forward conversion result of the weight can be calculated, wherein the calculation mode of the wingrad forward conversion result of the weight can be calculated by adopting the traditional matrix multiplication, and the wingrad forward conversion result can be calculated by referring to the above-mentioned disassembly to obtain the wingrad forward conversion result by adopting the summation operation.

After obtaining the wingrad positive conversion result of the data to be operated (input neuron, weight and gradient), the para-multiplication operation of the wingrad positive conversion result of the data to be operated can be continuously executed, and the para-multiplication result is obtained. The para-multiplication may be a value obtained by multiplying data corresponding to positions of two tensors as a corresponding position in a para-multiplication result.

Suppose win of an input neuronResults of the forward conversion of ograd B ^T d _4×4 B may be represented as:

winograd positive conversion result of weightCan be expressed as: />

Then the p-multiplication result may be:

the Winograd convolution result of the data to be operated on can be expressed as S _4×4 ＝A ^T (G _4×4 ⊙D _4×4 ) A, the slave function processing unit of the present disclosure can process A ^T (G _4×4 ⊙D _4×4 ) And A, disassembling the data to be calculated into sum operation, and calculating to obtain a winograd convolution result of the data to be calculated, so that the calculation time can be further saved, and the energy consumption can be reduced.

In one possible implementation manner, the decomposing the inverse winograd transform of the para-multiplication result into a sum operation, to obtain the winograd convolution result may include:

disassembling the para-multiplication result into a plurality of second sub-tensors, performing inverse Winograd transformation on the plurality of second sub-tensors, and summing to obtain a Winograd convolution result of the data to be operated;

The number of the second sub-tensors is the same as the number of elements which are not 0 in the para-position multiplication result, one element in each of the second sub-tensors is the same as the element in the corresponding position in the para-position multiplication result, and other elements are all 0.

Assume that the p-multiplication result is:

the para-multiplication result is disassembled into a plurality of second sub-tensors, for example, the para-multiplication result can be disassembled into 16, and the 16 second sub-tensors are respectively:

after the disassembly is completed, the inverse winograd transform can be performed on the plurality of second sub-tensors, and the winograd convolution results of the data to be operated can be obtained through summation.

In one possible implementation manner, performing inverse winograd transform on the plurality of second sub-tensors and summing to obtain a winograd convolution result of the data to be calculated may include the following processes:

obtaining a wingrad inverse transformation result of a second sub-tensor corresponding to the second sub-tensor; wherein the second sub-tensor corresponding to the second sub-tensor is: the value of the element of the second position in the second sub-tensor is 1, wherein the position of the second position in the second sub-tensor is the same as the position of the non-0 element in the second sub-tensor;

Taking the element value which is not 0 in the second sub-tensor as a coefficient to multiply the corresponding inverse wingrad result of the second sub-tensor to obtain the inverse wingrad result of the second sub-tensor;

and adding the inverse wingrad transformation results of the plurality of second sub-tensors to obtain the wingrad convolution result of the data to be operated.

The manner of determining the second element tensor corresponding to the second element tensor is the same as the manner of determining the first element tensor above, and will not be described again. Wherein, the inverse winograd transform result of the second sub-tensor is obtained in advance through the following process: and for each second sub-tensor, multiplying the left side of the second sub-tensor corresponding to the second sub-tensor by an inverse transformation left-multiplication matrix and multiplying the right side of the second sub-tensor by an inverse transformation right-multiplication matrix to obtain a winograd inverse transformation result of the second sub-tensor.

For matrices of different sizes, the form of the corresponding second sub-tensor is determined, as are the corresponding inverse transform left-multiplication matrix and inverse transform right-multiplication matrix. Thus, the inverse winograd transform result of the second sub-tensor can be calculated in advance, with the specific procedure described above.

For the example listed hereinabove, the inverse transform squaring matrix is a 2 x 4 matrix, which may be, for example:

The inverse transform right-multiplication matrix is a 4×2 matrix, which may be, for example:

the dimensions of the inverse transform matrix may be determined from the dimensions of the input neurons and the dimensions of the weights and convolution steps, the above being just one example and not limiting the disclosure in any way.

The inverse transform matrix is formed from 0,1, so the inverse transformed matrix multiplication operation can be broken down into an addition and shift operation implementation. Multiplying the inverse transformation matrix by the second sub-tensor to obtain a wingrad inverse transformation result of the second sub-tensor, wherein the element value in the wingrad inverse transformation result of the second sub-tensor is 0, <'>1, etc., the score can be calculated by a simple shift operation, and the calculation time can be saved compared to the multiplication operation.

Obtaining a wingrad inverse transformation result of the second sub-tensor by multiplying the element value which is not 0 in the second sub-tensor as a coefficient by the corresponding wingrad inverse transformation result of the second sub-tensor; the specific process of adding the wingrad inverse transformation results of the plurality of second sub-tensors to obtain the wingrad convolution result of the data to be operated can refer to the above, except that the wingrad inverse transformation results of the second sub-tensors are not completely calculated by 0, ±1, but the score can be calculated through simple shift operation, and compared with multiplication operation, the method can still realize the effects of saving calculation time and reducing energy consumption after disassembling the common inverse transformation process.

According to the above embodiments of the present disclosure, the second sub-tensor is obtained by disassembling the para-multiplication result, and the wingrad convolution result of the data to be calculated can be obtained by performing the summation operation according to the wingrad inverse transformation result of the second sub-tensor corresponding to the second sub-tensor obtained by the pre-calculation and the non-0 element value of the second sub-tensor.

According to the data processing method, quantization processing is carried out on the data to be quantized according to the determined quantization parameters, so that quantized data is obtained, wherein the quantization parameters are determined according to the corresponding statistical result and data bit width of the data to be quantized; continuing to execute the wingrad convolution processing according to the quantized data to obtain a quantized wingrad convolution result; and performing dequantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result. The calculation time of the Winograd convolution is saved and the energy consumption is reduced while the quantization precision is improved.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present disclosure is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present disclosure. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments, and that the acts and modules referred to are not necessarily required by the present disclosure.

It should be further noted that, although the steps in the flowchart of fig. 1 are sequentially shown as indicated by arrows, the steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 1 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of other steps or sub-steps of other steps.

Fig. 5 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure. As shown in fig. 5, the apparatus includes: a data quantization module 41, a convolution processing module 42 and an inverse quantization processing module 43. The data quantization module 41 performs quantization processing on the data to be quantized according to the determined quantization parameters, so as to obtain quantized data, wherein the quantization parameters are determined according to the corresponding statistical result and data bit width of the data to be quantized. And the convolution processing module 42 continues to execute the wingrad convolution processing according to the quantized data to obtain quantized wingrad convolution results. And the dequantization processing module 43 performs dequantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result.

In one possible implementation, the wingrad convolution processing procedure includes: resolving the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain a wingrad forward transformation result of each data in the data to be operated; performing para-multiplication operation between the wingrad positive conversion results of corresponding data in the data to be operated to obtain para-multiplication results; and decomposing the inverse Winograd transform of the para-position multiplication result into sum operation to obtain the Winograd convolution result, wherein the data to be quantized is any one of the data to be operated, the Winograd forward transform result of the data to be operated and the para-position multiplication result.

In one possible implementation manner, the quantization processing is performed on the data to be quantized according to the determined quantization parameter, so as to obtain quantized data, which includes any one of the following operations: before the wingrad forward conversion of the data to be operated is disassembled into summation operation, the data to be operated is used as the data to be quantized to carry out quantization processing; before carrying out the para-multiplication operation, taking a wingrad positive conversion result of each data in the data to be operated as the data to be quantized to carry out quantization processing; and before performing inverse winograd transform and decomposition to sum operation, performing quantization processing on the para-multiplication result as data to be quantized.

In one possible implementation manner, the disassembling the wingrad forward transformation of the data to be operated into sum operation, and performing calculation to obtain a wingrad forward transformation result of each data in the data to be operated includes: and respectively disassembling each data in the data to be operated into a plurality of first sub-tensors, performing the wingrad forward conversion on the plurality of first sub-tensors of each data in the data to be operated, and summing to obtain a wingrad forward conversion result of each data in the data to be operated, wherein the number of the plurality of first sub-tensors which are disassembled in each data is the same as the number of elements which are not 0 in the corresponding data, and one element in each first sub-tensor is the same as the element in the corresponding position in the corresponding data and the other elements are all 0.

In one possible implementation manner, the decomposing the inverse winograd transform of the para-multiplication result into a sum operation, to obtain the winograd convolution result, includes: disassembling the para-multiplication result into a plurality of second sub-tensors, performing inverse Winograd transformation on the plurality of second sub-tensors, and summing to obtain a Winograd convolution result of the data to be operated; the number of the second sub-tensors is the same as the number of elements which are not 0 in the para-position multiplication result, one element in each of the second sub-tensors is the same as the element in the corresponding position in the para-position multiplication result, and other elements are all 0.

In one possible implementation, the statistics include any one of the following: the method comprises the steps of obtaining a maximum absolute value in each piece of data to be quantized, and obtaining a half distance between the maximum absolute value and the minimum absolute value in each piece of data to be quantized, wherein the quantization parameters comprise one or more of point position parameters, scaling factors and offsets, and the maximum absolute value is the absolute value of the maximum absolute value or the minimum absolute value in each piece of data to be quantized.

In one possible implementation, the apparatus further includes: and the bit width adjusting module is used for adjusting the data bit width according to the quantization error corresponding to the data bit width so as to determine a quantization parameter by utilizing the adjusted data bit width, wherein the quantization error is determined according to quantized data in a corresponding layer and corresponding pre-quantized data.

In one possible implementation, the pre-quantization data is data to be quantized involved in a weight update iteration process within a target iteration interval; the target iteration interval comprises at least one weight updating iteration, and the same data bit width is adopted in the quantization process in the same target iteration interval.

It should be understood that the above-described apparatus embodiments are merely illustrative and that the apparatus of the present disclosure may be implemented in other ways. For example, the division of the units/modules in the above embodiments is merely a logic function division, and there may be another division manner in actual implementation. For example, multiple units, modules, or components may be combined, or may be integrated into another system, or some features may be omitted or not performed.

In addition, each functional unit/module in the embodiments of the present disclosure may be integrated into one unit/module, or each unit/module may exist alone physically, or two or more units/modules may be integrated together, unless otherwise specified. The integrated units/modules described above may be implemented either in hardware or in software program modules.

The integrated units/modules, if implemented in hardware, may be digital circuits, analog circuits, etc. Physical implementations of hardware structures include, but are not limited to, transistors, memristors, and the like. The artificial intelligence processor may be any suitable hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc., unless otherwise specified. The Memory unit may be any suitable magnetic or magneto-optical storage medium, such as resistive Random Access Memory RRAM (Resistive Random Access Memory), dynamic Random Access Memory DRAM (Dynamic Random Access Memory), static Random Access Memory SRAM (Static Random-Access Memory), enhanced dynamic Random Access Memory EDRAM (Enhanced Dynamic Random Access Memory), high-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid Memory cube HMC (Hybrid Memory Cube), etc., unless otherwise indicated.

The integrated units/modules may be stored in a computer readable memory if implemented in the form of software program modules and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in the various embodiments of the present disclosure. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In one possible implementation, an artificial intelligence chip is also disclosed, which includes the above-described data processing apparatus.

In one possible implementation, a board is also disclosed, which includes a memory device, an interface device, and a control device, and the artificial intelligence chip described above; wherein the artificial intelligent chip is respectively connected with the storage device, the control device and the interface device; the storage device is used for storing data; the interface device is used for realizing data transmission between the artificial intelligent chip and external equipment; the control device is used for monitoring the state of the artificial intelligent chip.

Fig. 6 shows a block diagram of a board according to an embodiment of the present disclosure, and referring to fig. 6, the board may include other mating components in addition to the chip 389, including but not limited to: a memory device 390, an interface device 391 and a control device 392;

the memory device 390 is connected to the artificial intelligence chip through a bus for storing data. The memory device may include multiple sets of memory cells 393. Each group of storage units is connected with the artificial intelligent chip through a bus. It is understood that each set of memory cells may be DDR SDRAM (English: double Data Rate SDRAM, double Rate synchronous dynamic random Access memory).

DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on both the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM. In one embodiment, the memory device may include 4 sets of the memory cells. Each set of the memory cells may include a plurality of DDR4 particles (chips). In one embodiment, the artificial intelligence chip may include 4 72-bit DDR4 controllers therein, where 64 bits of the 72-bit DDR4 controllers are used to transfer data and 8 bits are used for ECC verification. It is understood that the theoretical bandwidth of data transfer can reach 25600MB/s when DDR4-3200 granules are employed in each set of memory cells.

In one embodiment, each set of memory cells includes a plurality of double rate synchronous dynamic random access memories arranged in parallel. DDR can transfer data twice in one clock cycle. And a controller for controlling DDR is arranged in the chip and is used for controlling data transmission and data storage of each storage unit.

The interface device is electrically connected with the artificial intelligent chip. The interface device is used for realizing data transmission between the artificial intelligent chip and an external device (such as a server or a computer). For example, in one embodiment, the interface device may be a standard PCIE interface. For example, the data to be processed is transferred from the server to the chip through the standard PCIE interface, so as to implement data transfer. Preferably, when using PCIE3.0X interface transmission, the theoretical bandwidth can reach 16000MB/s. In another embodiment, the interface device may be another interface, and the disclosure is not limited to the specific form of the other interface, and the interface unit may be capable of implementing a switching function. In addition, the results of the computation of the artificial intelligence chip are still transmitted back to the external device (e.g., server) by the interface device.

The control device is electrically connected with the artificial intelligence chip. The control device is used for monitoring the state of the artificial intelligent chip. Specifically, the artificial intelligent chip and the control device can be electrically connected through an SPI interface. The control device may comprise a single chip microcomputer (Micro Controller Unit, MCU). The artificial intelligent chip can comprise a plurality of processing chips, a plurality of processing cores or a plurality of processing circuits, and can drive a plurality of loads. Therefore, the artificial intelligent chip can be in different working states such as multi-load and light-load. The control device can regulate and control the working states of a plurality of processing chips, a plurality of processing circuits and/or a plurality of processing circuits in the artificial intelligent chip.

In one possible implementation, an electronic device is disclosed that includes the artificial intelligence chip described above. The electronic device includes a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet, an intelligent terminal, a cell phone, a vehicle recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, a video camera, a projector, a watch, an earphone, a mobile storage, a wearable device, a vehicle, a household appliance, and/or a medical device. The vehicle comprises an aircraft, a ship and/or a vehicle; the household appliances comprise televisions, air conditioners, microwave ovens, refrigerators, electric cookers, humidifiers, washing machines, electric lamps, gas cookers and range hoods; the medical device includes a nuclear magnetic resonance apparatus, a B-mode ultrasonic apparatus, and/or an electrocardiograph apparatus.

The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a non-volatile computer readable storage medium.

The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

Fig. 7 illustrates a block diagram of an electronic device 800, according to an embodiment of the disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 7, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.

Fig. 8 illustrates a block diagram of an electronic device 1900 according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server. Referring to fig. 8, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments. The technical features of the foregoing embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the foregoing embodiments are not described, however, all of the combinations of the technical features should be considered as being within the scope of the disclosure.

The foregoing may be better understood in light of the following clauses:

clause a1. A data processing method, the method comprising:

Clause a2 the method of clause A1, the wingrad convolution process comprising:

the data to be quantized is any one of the data to be operated, a wingrad positive transformation result of the data to be operated and the para-multiplication result.

Clause a3. According to the method of clause A2, performing quantization processing on the data to be quantized according to the determined quantization parameter to obtain quantized data, including any one of the following operations:

Clause a4. According to the method of clause A2, the disassembling the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain the wingrad forward transformation result of each data in the data to be operated, including:

Clause a5. The method of clause A2, wherein the decomposing the inverse winograd transform of the para-multiply result into a sum operation, obtaining the winograd convolution result, comprises:

Clause a6 the method of clause A1, wherein the statistics comprise any one of: the absolute value of each maximum value in the data to be quantized, the distance of the maximum value and the minimum value in each data to be quantized,

the quantization parameter includes one or more of a point location parameter, a scaling factor and an offset,

Clause A7. the method of clause A6, wherein the scaling factor is determined based on the point location parameter, the statistics, and the data bit width.

Clause A8. the method of clause A6, wherein the offset is determined based on statistics of each type of data to be quantified.

Clause A9. the method of clause A6, wherein the point location parameter is determined based on the statistics and the data bit width.

Clause a10 the method of clause A6, further comprising:

Clause a11 the method of clause a10, wherein the pre-quantization data is data to be quantized involved in a weight update iteration process within a target iteration interval;

the target iteration interval comprises at least one weight updating iteration, and the same data bit width is adopted in the quantization process in the same target iteration interval.

Clause a12 the method of clause A2, wherein the data to be calculated comprises at least one of an input neuron, a weight, and a gradient.

Clause a13 a data processing device, the device comprising:

Clause a14 the apparatus of clause a13, the wingrad convolution process comprising:

Clause a15 the apparatus according to clause a14, wherein the quantizing process is performed on the quantized data according to the determined quantizing parameters to obtain quantized data, including any one of the following operations:

Clause a16 the device according to clause a14, wherein the disassembling the wingrad forward transformation of the data to be operated into sum operation, and calculating to obtain the wingrad forward transformation result of each data in the data to be operated comprises:

Clause a17 the apparatus of clause a14, wherein the decomposing the inverse winograd transform of the para-multiply result into a sum operation results in the winograd convolution result, comprises:

Clause a18 the apparatus of clause a13, wherein the statistics comprise any one of: the absolute value of each maximum value in the data to be quantized, the distance of the maximum value and the minimum value in each data to be quantized,

Clause a19 the apparatus of clause a18, the scaling factor is determined based on the point location parameter, the statistics, and the data bit width.

Clause a20 the apparatus of clause a18, wherein the offset is determined based on statistics of each type of data to be quantified.

Clause a21 the apparatus of clause a18, wherein the point location parameter is determined based on the statistics and the data bit width.

Clause a22 the apparatus of clause a18, further comprising:

a bit width adjusting module for adjusting the data bit width according to the quantization error corresponding to the data bit width to determine quantization parameters by using the adjusted data bit width,

Clause a23 the device of clause a22, wherein the pre-quantization data is data to be quantized involved in a weight update iteration process within a target iteration interval;

Clause a24 the apparatus of clause a14, wherein the data to be operated on includes at least one of an input neuron, a weight, and a gradient.

An artificial intelligence chip comprising the data processing apparatus of any one of clauses a13 to a24.

Clause a26 an electronic device comprising the artificial intelligence chip of clause a25.

Clause a27. A board card, the board card comprising: a memory device, interface means and control device, an artificial intelligence chip as set forth in clause a 25;

the storage device is used for storing data;

Clause a28 the board of clause a27,

the memory device includes: each group of storage units is connected with the artificial intelligent chip through a bus, and the storage units are as follows: DDR SDRAM;

the chip comprises: the DDR controller is used for controlling data transmission and data storage of each storage unit;

the interface device is as follows: standard PCIE interfaces.

Clause a29 an electronic device comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the instructions stored by the memory to perform the method of any of clauses A1-a 12.

Clause a30 a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method of any of clauses A1 to a 12.

The foregoing has outlined rather closely the embodiments of the present disclosure, and detailed description of the principles and embodiments of the present disclosure have been presented herein with the application of specific examples, the description of the examples above being merely intended to facilitate an understanding of the method of the present disclosure and its core ideas. Meanwhile, those skilled in the art will recognize that modifications or variations made on the basis of the specific embodiments and application scope of the present disclosure are within the scope of the protection of the present disclosure in light of the ideas of the present disclosure. In view of the foregoing, this description should not be construed as limiting the disclosure.

Claims

1. A method of data processing, the method being applied to a processor, the method comprising:

carrying out quantization processing on data to be quantized according to the determined quantization parameters to obtain quantized data, wherein the quantization parameters are determined according to a statistical result and a data bit width of the corresponding data to be quantized, the data to be quantized is data used in the process that the processor executes an operation task by using a neural network model, the data to be quantized is derived from any one of image data, video data, audio data and text data, the operation task comprises any one of image recognition, voice recognition and natural language processing, and the quantization parameters are used for improving quantization precision of the data to be quantized used in the process that the neural network model executes the operation task;

performing inverse quantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result;

the quantized wingrad convolution result is used for improving the calculation performance of the neural network model when executing the operation task;

wherein, the Winograd convolution processing process comprises the following steps:

resolving the inverse Winograd transform of the para-multiplication result into a sum operation to obtain the quantized Winograd convolution result,

2. The method according to claim 1, wherein the quantization of the data to be quantized according to the determined quantization parameter results in quantized data, comprising any one of the following operations:

3. The method according to claim 1, wherein the disassembling the wingrad forward transform of the data to be operated into a sum operation, and calculating to obtain the wingrad forward transform result of each data in the data to be operated, includes:

4. The method of claim 1, wherein the decomposing the inverse winograd transform of the para-multiply result into a sum operation results in the winograd convolution result, comprising:

5. The method of claim 1, wherein the statistics comprise any one of: the absolute value of each maximum value in the data to be quantized, the distance of the maximum value and the minimum value in each data to be quantized,

6. The method of claim 5, wherein the scaling factor is determined based on the point location parameter, the statistics, and the data bit width.

7. The method of claim 5, wherein the offset is determined based on statistics of each type of data to be quantized.

8. The method of claim 5, wherein the point location parameter is determined based on the statistics and the data bit width.

9. The method of claim 5, wherein the method further comprises:

10. The method of claim 9, wherein the pre-quantized data is data to be quantized involved in a weight update iteration process within a target iteration interval;

11. The method of claim 1, wherein the data to be operated on includes at least one of input neurons, weights, and gradients.

12. A data processing apparatus, the apparatus being for use with a processor, the apparatus comprising:

the data quantization module is used for carrying out quantization processing on data to be quantized according to the determined quantization parameters to obtain quantized data, wherein the quantization parameters are determined according to a statistical result and a data bit width of the corresponding data to be quantized, the data to be quantized are used in the process that the processor executes an operation task by using a neural network model, the data to be quantized are derived from any one of image data, video data, audio data and text data, the operation task comprises any one of image recognition, voice recognition and natural language processing, and the quantization parameters are used for improving quantization precision of the data to be quantized used in the process that the neural network model executes the operation task;

The dequantization processing module is used for performing dequantization processing on the quantized wingrad convolution result to obtain a wingrad convolution result;

13. An artificial intelligence chip, characterized in that the chip comprises a data processing device according to claim 12.

14. An electronic device comprising the artificial intelligence chip of claim 13.

15. A board, characterized in that, the board includes: a memory device, interface means and control device, an artificial intelligence chip according to claim 13;

the storage device is used for storing data;

16. The board card of claim 15, wherein the board card comprises,

the interface device is as follows: standard PCIE interfaces.

17. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 11.

18. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 11.