CN114648101A - Transformer structure-based softmax function quantization realization method and device - Google Patents
Transformer structure-based softmax function quantization realization method and device Download PDFInfo
- Publication number
- CN114648101A CN114648101A CN202210517307.3A CN202210517307A CN114648101A CN 114648101 A CN114648101 A CN 114648101A CN 202210517307 A CN202210517307 A CN 202210517307A CN 114648101 A CN114648101 A CN 114648101A
- Authority
- CN
- China
- Prior art keywords
- data
- input data
- row
- mapping
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a transformer structure-based softmax function quantization realization method and device, which are applied to a softmax module in a transformer structure and comprise the following steps: according to the bit width of input data of the global shared index mapping table, performing data adjustment and truncation processing on each row of input data in an input matrix transmitted from the MatMul module to the softmax module to obtain truncation data of the row of input data; searching a global shared index mapping table to determine index mapping data corresponding to truncated data of each row of input data; adding the exponential mapping data corresponding to the truncation data of each row of input data to obtain the sum of the exponential mapping data corresponding to the row of input data; determining a reciprocal mapping value of the sum of the exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function; and performing multiplication and shift operation on the exponent mapping data corresponding to the stage data of each row of input data and the reciprocal mapping value of the sum of the exponent mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data.
Description
Technical Field
The invention relates to the technical field of neural networks, in particular to a transformer structure-based softmax function quantization implementation method and device.
Background
The translation model transformer based on the self-attention mechanism does not use the methods and modules of CNN and RNN, but creatively takes the attention mechanism as the core construction of a codec to execute the translation operation.
Referring to fig. 1, fig. 1 is a schematic diagram of a prior art transformer structure, as shown in fig. 1, the transformer structure sequentially includes: the system comprises a MatMul module used for carrying out matrix multiplication operation on input parameters Q (query) and K (key) of a transform structure, a Scale module used for carrying out scaling on output data of the MatMul module, a Mask module used for shielding partial output data of the Scale module, a SoftMax module used for carrying out Softmax operation on the output data of the Mask module, and a MatMul module used for multiplying the output data of the SoftMax module and the input parameters V (value) of the transform structure. It can be seen that the operation of the softmax module in the transformer structure is preceded by the operation of the MatMul module, and the operation of the softmax module is substantially performed on the matrix output after the operation of the MatMul module line by line.
However, due to unbalanced data distribution between rows in the input matrix, the difference between actual scales (the difference between the maximum value and the minimum value in one row of data) is relatively large, and the quantization results of two rows of data are consistent due to quantization errors, for example, [41.45, 42.15, 42.84, 43.89] and [40.81, 41.93, 43.36, 44.27] are quantized to obtain the same quantization result [41, 42, 43, 44] (scale =1), so that the quantization accuracy is not high.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for implementing softmax function quantization based on a transform structure, which are simple in calculation and capable of effectively improving quantization precision.
In order to achieve the purpose, the invention provides the following technical scheme:
a softmax function quantization realization method based on a transform structure is applied to a softmax module in the transform structure and comprises the following steps:
acquiring an input matrix transmitted by a MatMul module in a transform structure;
performing data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to the bit width of the input data of the global shared index mapping table configured for the transform structure in advance to obtain adjustment data of the row of input data;
cutting off the adjustment data of each row of input data to obtain cut-off data of the row of input data;
searching the global shared index mapping table to determine index mapping data corresponding to truncated data of each row of input data;
adding the exponential mapping data corresponding to the truncation data of each row of input data to obtain the sum of the exponential mapping data corresponding to the row of input data;
determining a reciprocal mapping value of the sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for a transformer structure in advance;
performing multiplication and shift operation on the exponential mapping data corresponding to the truncation data of each row of input data and the reciprocal mapping value of the sum of the exponential mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data;
and outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
A transformer structure-based softmax function quantization implementation device is applied to a softmax module in a transformer structure, and comprises the following components:
the acquisition unit is used for acquiring an input matrix transmitted by a MatMul module in a transformer structure;
the adjusting unit is used for performing data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to the bit width of the input data of the global shared index mapping table configured for the transformer structure in advance to obtain the adjustment data of the row of input data; cutting off the adjustment data of each row of input data to obtain cut-off data of the row of input data;
the index unit is used for searching the global shared index mapping table to determine index mapping data corresponding to the truncated data of each row of input data;
the summing unit is used for summing the exponential mapping data corresponding to the truncation data of each row of input data to obtain the sum of the exponential mapping data corresponding to the row of input data;
the reciprocal unit is used for determining a reciprocal mapping value of the sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for the transformer structure in advance;
the multiplication unit is used for carrying out multiplication and shift operation on the exponential mapping data corresponding to the truncation data of each row of input data and the reciprocal mapping value of the sum of the exponential mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data;
and the output unit is used for outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
According to the technical scheme, after data adjustment and truncation processing are carried out on each row of input data in the input matrix transmitted from the MatMul module to the softmax module in the transform structure, the index operation and reciprocal operation are achieved through the globally shared index mapping table and reciprocal mapping function which are configured for the transform structure in advance, calculation is simple, the problem that data distribution among different rows in the input matrix transmitted from the MatMul module to the softmax module is unbalanced, and therefore quantization accuracy is not high can be effectively improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a schematic diagram of a prior art transformer structure;
fig. 2 is a schematic diagram of an operation process of the softmax module in the transform structure according to the embodiment of the present invention;
FIG. 3 is a flowchart of a method for implementing quantization of a softmax function based on a transform structure according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for implementing quantization of softmax function based on a transform structure according to a second embodiment of the present invention;
FIG. 5 is an exploded view of input data of the softmax module according to an embodiment of the invention;
FIG. 6 is a diagram of a prior art exp exponential function image;
FIG. 7 is a flowchart of a method for implementing quantization of a softmax function based on a transform structure according to a third embodiment of the present invention;
FIG. 8 is a schematic diagram of a pipeline mode according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a device for implementing softmax function quantization based on a transform structure according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.
The technical solution of the present invention will be described in detail with specific examples. Several of the following embodiments may be combined with each other and some details of the same or similar concepts or processes may not be repeated in some embodiments.
In the invention, a globally shared index mapping table and a reciprocal mapping function are configured for the transformer structure in advance, so that the softmax module in the transformer structure can realize index operation and reciprocal operation by using the globally shared index mapping table and reciprocal mapping function, the calculation is simple, the hardware resource consumption can be reduced, and the power consumption is reduced; and the problem of low quantization precision caused by inconsistent data distribution among different rows in the input matrix transmitted from the MatMul module to the softmax module can be solved, and the quantization precision can be effectively improved.
Fig. 2 is a schematic diagram of an operation process of the softmax module in the transform structure according to an embodiment of the present invention, and as shown in fig. 2, assuming that each row of input data of the input matrix transferred from the MatMul module to the softmax module in the transform structure includes n +1 input data, the input data [ X ] of the i +1 th row of input data of the input matrix transferred from the MatMul module to the softmax module is inputi0,……,Xin]The operation process of (2) is as follows:
in this embodiment, the input data bit width of the softmax module is already indicated in fig. 2, and is i _ bits, which is generally 16 bits.
In this embodiment, the input data bit width t _ bits and the output data bit width e _ bits of the global shared index mapping table are already indicated in fig. 2, where the value of t _ bits is generally 8bits, and the value of e _ bits may also be appropriately increased as needed.
in this embodiment, the bit width of the output data of the global shared exponent mapping function is already indicated in fig. 2, and is r _ bits, which is generally 16bits, and the value of r _ bits may also be appropriately adjusted as needed.
In this embodiment, the output data bit width of the softmax module is already indicated in fig. 2, which is o _ bits, and generally takes a value of 8bits, and the value of o _ bits can be increased to 16bits, but this increases the bandwidth consumed by the whole network.
The input data bit width of the method can be 16bits, the precision of the implementation scheme is high, and the method can adapt to various scenes. Such as object classification, object monitoring, etc.
The method for realizing the softmax function quantization based on the transform structure provided by the invention is described in detail with reference to specific embodiments as follows:
referring to fig. 3, fig. 3 is a flowchart of a method for implementing softmax function quantization based on a transform structure according to an embodiment of the present invention, where the method is applied to a softmax module in a transform structure, and as shown in fig. 3, the method mainly includes the following steps:
and 307, outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
As can be seen from the method shown in fig. 3, in this embodiment, after data adjustment and truncation processing are performed on each row of input data in an input matrix transmitted from the MatMul module to the softmax module in the transform structure, an index operation and a reciprocal operation are implemented by using a globally shared index mapping table and reciprocal mapping function configured for the transform structure in advance, so that a calculation process is simplified, hardware resource consumption can be reduced, power consumption can be reduced, the problem of inconsistency between data distribution and data precision between different rows in the input matrix transmitted from the MatMul module to the softmax module can be solved, and quantization precision can be effectively improved.
Referring to fig. 4, fig. 4 is a flowchart of a method for implementing softmax function quantization based on a transform structure according to a second embodiment of the present invention, where the method is applied to a softmax module in a transform structure, and as shown in fig. 4, the method mainly includes the following steps:
in this embodiment, a specific value of the input data bit width t _ bits of the global shared index mapping table may be 8 bits.
In this embodiment, determining an adjustment offset according to the input data bit width t _ bits of the global shared index mapping table may specifically be implemented by using the following formula: offset = (1< < t _ bits) -1, for example, when t _ bits =8, offset =255 can be determined by calculation according to the above formula.
in this embodiment, the difference is adjusted by using the adjustment value, specifically, the difference is added to the adjustment value.
In this embodiment, for each input in the i +1 th row of input dataEnter data Xij(j =0, 1, 2, … …, n), calculating a difference between the input data and the maximum value, and adjusting the difference by the adjustment value to obtain adjustment data X 'of the input data'ijSpecifically, the following formula can be adopted: x'ij=Xij-MAX+offset。
In this embodiment, the adjustment data of all the input data in the row of input data constitutes the adjustment data corresponding to the row of input data.
The above steps 4011 to 4013 are a detailed refinement of step 301 shown in fig. 3.
in this embodiment, the input data bit width i _ bits of the softmax module may be 16 bits. The input data of 16bits can be decomposed into three parts, as shown in fig. 5, the first part is a sign bit and includes 1bit, the second part and the third part are respectively an integer bit and a decimal bit, and occupy 15bits in total, wherein the more bits occupied by the decimal bit, the higher the precision of corresponding representation is, but the representation capability of the integer is reduced.
In practical applications, the value of the exp exponential function drops relatively quickly, as shown in fig. 6, the value of exp (-7) is already small and can be basically ignored, so long as the difference between the maximum value and the minimum value in the input data of the softmax module is kept at about 7.
Therefore, in this embodiment, after calculating the difference between the input data and the maximum value and using the adjustment value to adjust the difference to obtain the adjustment data of the input data, the truncation processing may be continued to be performed on the adjustment data of the input data to obtain the truncated data of the input data, so that only 3bits of integer bits are retained in the truncated data of the input data, and it is ensured that the difference between the maximum value and the minimum value in the truncated data is maintained at about 7. In addition, in this embodiment, the number of the decimal place may be preset according to the precision requirement, for example, the number of the decimal place may be 5bits, 6bits, or 7 bits.
In the present embodiment, each input data X is obtained by dividing each input data XijAdjusted and cut off to obtain X'ij， X′ijHas a value range of [0, offset]X 'taking t _ bits =8 as an example'ijHas a value range of [0, 255 ]]。
The above step 402 is a detailed refinement of step 302 shown in fig. 3.
in this embodiment, the global shared index mapping table may be generated according to a preset minimum scale S that can be represented after fixed-point quantization, an input data bit width t _ bits and an output data bit width e _ bits of the shared index mapping table, and includes 1<<t _ bits table entries, the occupied storage space is (1)<<t _ bits) × e _ bits/8 bytes "<<"is the left shift operator. Wherein, the value of S can be preset according to the digit k of decimal place set by precision requirement, k is positive integer, S =1/2k. For example, when the number of decimal places is 5, the value of S is 1/32; when the decimal digit is 6, the value of S is 1/64; when the number of decimal places is 7, S takes on the value 1/128.
In this embodiment, a specific generation method of the global shared index mapping table may specifically be: input data X to softmax moduleijTruncated data X 'of'ijValue range of [0, offset ]]Each integer value u in (d) is calculated as its corresponding exponential mapping value v using the following formula:
v = Round (exp ((u-offset) × S) × ((1< < e _ bits) -1)), where the function Round (x) is used to Round the value x in brackets. Through the calculation formula, the mapping relations of offset u to v can be calculated, and the mapping relations form a global shared index mapping table.
in this embodiment, it is assumed that the i +1 th row of input data includes n +1 input data, wherein the jth input data XijAdjusted data X 'of (2)'ijCorresponding exponential mapping data as QijThen, the sum of the exponential mapping data corresponding to the input data of the i +1 th row is sum = Qi0+Qi1+……+Qin。
in this embodiment, the global shared reciprocal mapping function may be a reciprocal mapping table generated according to a preset function, a linear function obtained by linearly fitting the preset function, or a polynomial function generated by polynomial fitting the preset function; wherein the preset function is f (x) =2r_bitsThe value range of/x, x is a preset value interval, preferably, the preset value interval is [256, 512 ], where r _ bits is the output data bit width of the reciprocal mapping function, and usually takes 16 bits.
and step 407, outputting the final result to a neural network layer which is the next neural network layer of the neural network layer to which the softmax module belongs.
As can be seen from the method shown in fig. 4, in this embodiment, data adjustment and truncation processing are performed on each input data in each row of input data in an input matrix transmitted from the MatMul module to the softmax module to obtain truncated data of the row of input data, an index mapping table shared globally is used to implement an index operation to obtain index mapping data corresponding to the truncated data of the row of input data, a sum of the index mapping data corresponding to the row of input data is obtained through summation, a reciprocal mapping value corresponding to the sum of the index mapping data is determined according to a reciprocal mapping function shared globally, and a final result corresponding to the row of input data is obtained through multiplication and shift operation. It can be seen that, in this embodiment, by using the globally shared index mapping table and reciprocal mapping function, the calculation process is simplified, hardware resource consumption can be reduced, power consumption is reduced, the problem of inconsistent data distribution and data precision between different rows in the input matrix transmitted from the MatMul module to the softmax module can be solved, and the quantization precision can be effectively improved.
Referring to fig. 7, fig. 7 is a flowchart of a method for implementing softmax function quantization based on a transform structure according to a third embodiment of the present invention, where the method is applied to a softmax module in a transform structure, as shown in fig. 7, the method mainly includes the following steps:
in this embodiment, when data is transferred from the MatMul module to the softmax module, a pipeline mode may be adopted for implementation, fig. 8 shows a schematic diagram of the pipeline mode, as shown in fig. 8, a gray part represents operations of the MatMul module, and a white part represents operations of the softmax module, and in the pipeline mode, the MatMul module only transfers partial data of the input matrix to the softmax module at a time, instead of transferring the entire input matrix to the softmax module at one time. The implementation mode enables the MatMul module and the softmax module to be strongly coupled, data output by the MatMul module is not required to be transmitted to the DDR, but is directly transmitted to the softmax module, bandwidth occupation can be reduced, and the speed of the softmax module for acquiring the data is increased.
In this embodiment, when data is transferred from the MatMul module to the softmax module, pipeline is used, and the softmax module does not obtain a final result corresponding to the entire input matrix by performing operations at the same time, but obtains the final result line by line.
7051, performing data conversion on the sum of the exponential mapping data corresponding to each row of input data;
in this embodiment, the data conversion of the sum of the exponent mapping data corresponding to each row of input data may specifically include:
s11, determining a first effective bit number required to occupy the sum of the index mapping values corresponding to the row of input data and a second effective bit number required to occupy the left boundary value of the preset value interval, and determining a difference between the first effective bit number and the second effective bit number as a first right shift number required to map the sum of the index mapping values corresponding to the row of input data to the preset value interval;
in this embodiment, a specific method for determining the valid bit number bits (bit) (val) required to be occupied by a value val may be calculated by using the following formula bits (val) = floor (log2(val)) +1, where the function floor (x) is used to round the value x in parentheses downward.
Assuming that the sum of the exponent mapping values is 1600 and the preset value interval is [256, 512) as an example, it can be known from the above formula that the first significant bit number = floor (log2(1600)) +1=11 required for representing the sum of the exponent mapping values, and the second significant bit number = floor (log2(256)) +1=9 required for representing the left boundary value 256 of the preset value interval [256, 512), so that the right shift number bits =11-9=2 required for mapping the sum of the exponent mapping values to the preset value interval can be determined.
And S12, right shifting the exponent mapping data sum corresponding to the row of input data according to the first right shift number, mapping the right shifting result to a numerical value in a value range determined by the output data bit width of the exponent mapping table, and taking the numerical value as conversion data of the exponent mapping data sum corresponding to the row of input data.
In this embodiment, the sum of the exponent mapping values corresponding to the row of input data is right-shifted according to the first right-shift number, that is, the sum of the exponent mapping values corresponding to the row of input data is right-shifted by the first right-shift number. And mapping the right shift result to a numerical value in a value range determined by the output data bit width of the index mapping table, wherein the left boundary value in a preset value interval is subtracted from the right shift result. Taking the preset value interval as [256, 512) as an example, the following formula can be adopted to obtain the conversion data sum' = (sum > > bits) -256 corresponding to the sum of the exponent mapping values corresponding to the row of input data, and ">", which is a right shift operator.
7052, inputting the conversion result of the sum of the exponential mapping data corresponding to each row of input data into a global shared reciprocal mapping function configured for a transformer structure in advance to obtain a reciprocal mapping value of the sum of the exponential mapping data corresponding to the row of input data;
in this embodiment, the inverse mapping function may be: generating a reciprocal mapping table according to a preset function, a linear function obtained by linearly fitting the preset function, or a polynomial function generated by polynomial fitting the preset function; wherein the preset function is f (x) =2r_bitsThe value range of/x, x ∈ is a preset value interval, preferably, the preset value interval is [256, 512 ], where r _ bits is the output data bit width of the reciprocal mapping function, and usually takes a value of 16.
The above steps 7051 to 7052 are specific refinements of step 305 shown in fig. 3.
In this embodiment, the performing multiplication and shift operation on the inverse mapping value of the sum of the exponent mapping data corresponding to the truncated data of each row of input data and the exponent mapping data corresponding to the row of input data to obtain the final result corresponding to the row of input data may specifically include:
s21, for each input data in the row of input data, performing the following operations S22 to S23:
s22, calculating the product of the inverse mapping value of the index mapping data corresponding to the truncation data of the input data and the sum of the index mapping data corresponding to the input data of the row, and rounding the product to obtain an intermediate result corresponding to the input data;
and S23, determining a second right shift number required for mapping the intermediate result corresponding to the input data to a value range determined by the output data bit width of the softmax module according to the first right shift number, the output data bit width of the global shared reciprocal mapping function and the output data bit width of the softmax module, and performing right shift on the intermediate result corresponding to the input data according to the second right shift number to obtain a final result corresponding to the input data.
In the step S23, the intermediate result corresponding to the input data is determined to be mapped to the second right shift number bits2 required for the value range determined by the output data bit width of the softmax module according to the first right shift number bits1, the output data bit width r _ bits of the global shared reciprocal mapping function, and the output data bit width o _ bits of the softmax module, which may specifically be implemented by using the following formula: bits2= r _ bits-o _ bits + bits 1.
And step 707, outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
As can be seen from the method shown in fig. 7, in this embodiment, after data adjustment and truncation processing are performed on each row of input data in an input matrix transmitted from the MatMul module to the softmax module, an index mapping table shared globally is used to perform index operation to obtain index mapping data corresponding to truncated data of the row of input data, a sum of the index mapping data corresponding to the row of input data is obtained through summation, the sum of the index mapping data is subjected to data conversion and then is input to a global shared reciprocal mapping function to obtain a reciprocal mapping value corresponding to the sum of the index mapping data, and finally, a final result corresponding to the row of input data is obtained through multiplication and shift operation. It can be seen that, in this embodiment, by using the globally shared index mapping table and reciprocal mapping function, the calculation process is simplified, hardware resource consumption can be reduced, power consumption is reduced, the problem of inconsistent data distribution and data precision between different rows in the input matrix transmitted from the MatMul module to the softmax module can be solved, and the quantization precision can be effectively improved.
The above describes in detail a method for implementing softmax function quantization based on a transform structure in the embodiment of the present invention, and the embodiment of the present invention also provides a device for implementing softmax function quantization based on a transform structure, which is described in detail below with reference to fig. 9.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an apparatus for implementing softmax function quantization based on a transform structure according to an embodiment of the present invention, where the apparatus is applied to a softmax module in a transform structure, and as shown in fig. 9, the apparatus includes:
an obtaining unit 900, configured to obtain an input matrix transmitted by a MatMul module in a transformer structure;
an adjusting unit 901, configured to perform data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to an input data bit width of a global shared index mapping table configured for a transformer structure in advance, to obtain adjustment data of the row of input data; cutting off the adjustment data of each row of input data to obtain cut-off data of the row of input data;
an index unit 902, configured to search the global shared index mapping table to determine index mapping data corresponding to truncated data of each row of input data;
a summing unit 903, configured to add the exponential mapping data corresponding to the truncated data of each row of input data to obtain an exponential mapping data sum corresponding to the row of input data;
a reciprocal unit 904, configured to determine a reciprocal mapping value of a sum of exponent mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for the transformer structure in advance;
a multiplication unit 905, configured to perform multiplication and shift operation on the inverse mapping value of the sum of the exponent mapping data corresponding to the truncated data of each row of input data and the exponent mapping data corresponding to the row of input data, to obtain a final result corresponding to the row of input data;
the output unit 906 is configured to output the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
In the arrangement shown in figure 9 of the drawings,
the adjusting unit 901 performs data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to an input data bit width of a global shared index mapping table configured for a transformer structure in advance, including:
determining the maximum value in the row of input data, and determining an adjusting value according to the bit width of the input data of the shared index mapping table;
calculating the difference value between the input data and the maximum value aiming at each input data in the row of input data, and adjusting the difference value by using the adjusting value to obtain the adjusting data of the input data;
the adjusting unit 901 performs truncation processing on the adjustment data of each line of input data to obtain truncated data of the line of input data, and includes:
and for each input data in the row of input data, if the adjustment data of the input data is less than 0, setting the truncation data of the input data to be 0, otherwise, setting the truncation data of the input data to be the adjustment data of the input data.
In the arrangement shown in figure 9 of the drawings,
the global shared index mapping table is generated according to a preset minimum scale S which can be expressed after fixed point quantization, an input data bit width t _ bits and an output data bit width e _ bits of the global shared index mapping table, and comprises 1< < t _ bits table entries, and the occupied storage space is (1< < t _ bits) multiplied by e _ bits divided by 8 bytes.
In the arrangement shown in figure 9 of the drawings,
the global shared reciprocal mapping function is a reciprocal mapping table generated according to a preset function, a linear function obtained by linearly fitting the preset function, or a polynomial function generated by polynomial fitting the preset function; wherein the preset function is f (x) =2 r_bitsThe value range of x is a preset value range, and r _ bits is the output value bit width of the reciprocal mapping function;
the reciprocal unit 904 determines a reciprocal mapping value of a sum of exponent mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for a transformer structure in advance, including:
carrying out data conversion on the sum of the exponential mapping data corresponding to the row of input data;
and inputting the conversion result into the global shared reciprocal mapping function to obtain a reciprocal mapping value of the sum of the exponential mapping data corresponding to the input data of the row.
In the arrangement shown in figure 9 of the drawings,
the reciprocal unit 904 performs data conversion on the sum of the exponent mapping data corresponding to the row of input data, and includes:
determining a first effective bit number required to be occupied by the index mapping value sum corresponding to the row of input data and a second effective bit number required to be occupied by the left boundary value representing a preset value interval, and determining a difference value between the first effective bit number and the second effective bit number as a first right shift number required to map the index mapping value sum corresponding to the row of input data to the preset value interval;
and right shifting the exponent mapping data sum corresponding to the row of input data according to the first right shift number, mapping the right shift result to a value in a value range determined by the bit width of the output data of the exponent mapping table, and taking the value as conversion data of the exponent mapping data sum corresponding to the row of input data.
In the arrangement shown in figure 9 of the drawings,
the multiplication unit 905 performs multiplication and shift operation on the inverse mapping value of the sum of the exponent mapping data corresponding to the truncated data of each row of input data and the exponent mapping data corresponding to the row of input data to obtain the final result corresponding to the row of input data, and includes:
for each input data in the row of input data, performing the following operations:
calculating the product of the inverse mapping value of the index mapping data corresponding to the truncation data of the input data and the sum of the index mapping data corresponding to the input data of the row, and rounding the product to obtain an intermediate result corresponding to the input data;
and determining a second right shift number required for mapping the intermediate result corresponding to the input data to a value range determined by the output data bit width of the softmax module according to the first right shift number, the output data bit width of the global shared reciprocal mapping function and the output data bit width of the softmax module, and performing right shift on the intermediate result corresponding to the input data according to the second right shift number to obtain a final result corresponding to the input data.
In the arrangement shown in figure 9 of the drawings,
when the input matrix is transmitted to the softmax module from the MatMul module, a pipeline mode is adopted for realization;
and the softmax module is used for temporarily storing a final result corresponding to each row of input data in the input matrix in a Static Random Access Memory (SRAM) through operation.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A transformer structure-based softmax function quantization implementation method is applied to a softmax module in a transformer structure, and is characterized by comprising the following steps:
acquiring an input matrix transmitted by a MatMul module in a transform structure;
performing data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to the bit width of the input data of the global shared index mapping table configured for the transformer structure in advance to obtain adjustment data of the row of input data;
cutting off the adjustment data of each line of input data to obtain cut-off data of the line of input data;
searching the global shared index mapping table to determine index mapping data corresponding to truncated data of each row of input data;
adding the exponential mapping data corresponding to the truncation data of each row of input data to obtain the sum of the exponential mapping data corresponding to the row of input data;
determining a reciprocal mapping value of the sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for a transformer structure in advance;
performing multiplication and shift operation on the exponential mapping data corresponding to the truncation data of each row of input data and the reciprocal mapping value of the sum of the exponential mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data;
and outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
2. The method of claim 1,
according to the input data bit width of a global shared index mapping table configured for a transformer structure in advance, performing data adjustment on each row of input data in an input matrix transmitted from a MatMul module to a softmax module, wherein the data adjustment comprises the following steps:
determining the maximum value in the row of input data, and determining an adjusting value according to the bit width of the input data of the global shared index mapping table;
calculating the difference value between the input data and the maximum value aiming at each input data in the row of input data, and adjusting the difference value by using the adjusting value to obtain the adjusting data of the input data;
the method for intercepting the adjustment data of each line of input data to obtain the intercepted data of the line of input data comprises the following steps:
and for each input data in the row of input data, if the adjustment data of the input data is less than 0, setting the truncation data of the input data to be 0, otherwise, setting the truncation data of the input data to be the adjustment data of the input data.
3. The method of claim 1,
the global shared index mapping table is generated according to a preset minimum scale S which can be expressed after fixed point quantization, an input data bit width t _ bits and an output data bit width e _ bits of the global shared index mapping table, and comprises 1< < t _ bits table entries, the occupied storage space is (1< < t _ bits) × e _ bits/8 bytes, and < <' is a left shift operator.
4. The method of claim 1,
the global shared reciprocal mapping function is a reciprocal mapping table generated according to a preset function, a linear function obtained by linearly fitting the preset function, or a polynomial function generated by polynomial fitting the preset function; wherein the preset function is f (x) =2r_bitsThe value range of x is a preset value interval, and r _ bits is the output data bit width of the global shared reciprocal mapping function;
determining a reciprocal mapping value of the sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for a transformer structure in advance, wherein the reciprocal mapping value comprises the following steps:
carrying out data conversion on the sum of the exponential mapping data corresponding to the row of input data;
and inputting the conversion result into the global shared reciprocal mapping function to obtain a reciprocal mapping value of the sum of the exponential mapping data corresponding to the input data of the row.
5. The method of claim 4,
and performing data conversion on the index mapping data sum corresponding to the row of input data, wherein the data conversion comprises the following steps:
determining a first effective bit number required to be occupied by the index mapping value sum corresponding to the row of input data and a second effective bit number required to be occupied by the left boundary value representing a preset value interval, and determining a difference value between the first effective bit number and the second effective bit number as a first right shift number required to map the index mapping value sum corresponding to the row of input data to the preset value interval;
and performing right shift on the index mapping data sum corresponding to the input data of the row according to the first right shift number, mapping the right shift result to a numerical value in a value range determined by the output data bit width of the global shared index mapping table, and taking the numerical value as conversion data of the index mapping data sum corresponding to the input data of the row.
6. The method of claim 5,
performing multiplication and shift operation on the exponent mapping data corresponding to the truncation data of each row of input data and the reciprocal mapping value of the sum of the exponent mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data, wherein the method comprises the following steps:
for each input data in the row of input data, performing the following operations:
calculating the product of the inverse mapping value of the index mapping data corresponding to the truncation data of the input data and the sum of the index mapping data corresponding to the input data of the row, and rounding the product to obtain an intermediate result corresponding to the input data;
and determining a second right shift number required for mapping the intermediate result corresponding to the input data to a value range determined by the output data bit width of the softmax module according to the first right shift number, the output data bit width of the global shared reciprocal mapping function and the output data bit width of the softmax module, and performing right shift on the intermediate result corresponding to the input data according to the second right shift number to obtain a final result corresponding to the input data.
7. The method of claim 1,
when the input matrix is transmitted to the softmax module from the MatMul module, a pipeline mode is adopted for realization;
and the softmax module is used for temporarily storing a final result corresponding to each row of input data in the input matrix in a Static Random Access Memory (SRAM) in an operation mode.
8. A transformer structure-based softmax function quantization implementation device is applied to a softmax module in a transformer structure, and is characterized by comprising the following steps:
the acquisition unit is used for acquiring an input matrix transmitted by a MatMul module in a transformer structure;
the adjusting unit is used for performing data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to the bit width of the input data of the global shared index mapping table configured for the transformer structure in advance to obtain the adjustment data of the row of input data; cutting off the adjustment data of each row of input data to obtain cut-off data of the row of input data;
the index unit is used for searching the global shared index mapping table to determine index mapping data corresponding to the truncation data of each row of input data;
the summing unit is used for summing the exponential mapping data corresponding to the truncation data of each row of input data to obtain the sum of the exponential mapping data corresponding to the row of input data;
the reciprocal unit is used for determining a reciprocal mapping value of the sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for the transformer structure in advance;
the multiplication unit is used for carrying out multiplication and shift operation on the exponential mapping data corresponding to the truncation data of each row of input data and the reciprocal mapping value of the sum of the exponential mapping data corresponding to the row of input data to obtain a final result corresponding to the row of input data;
and the output unit is used for outputting the final result to a subsequent neural network layer of the neural network layer to which the softmax module belongs.
9. The apparatus of claim 8,
the adjusting unit performs data adjustment on each row of input data in an input matrix transmitted from the MatMul module to the softmax module according to an input data bit width of a global shared index mapping table configured for a transformer structure in advance, and includes:
determining the maximum value in the row of input data, and determining an adjusting value according to the bit width of the input data of the shared index mapping table;
calculating the difference value between the input data and the maximum value aiming at each input data in the row of input data, and adjusting the difference value by using the adjusting value to obtain the adjusting data of the input data;
the adjusting unit is used for intercepting the adjusting data of each line of input data to obtain the intercepted data of the line of input data, and comprises:
and for each input data in the row of input data, if the adjustment data of the input data is less than 0, setting the truncation data of the input data to be 0, otherwise, setting the truncation data of the input data to be the adjustment data of the input data.
10. The apparatus of claim 8,
the global shared index mapping table is generated according to a preset minimum scale S which can be expressed after fixed point quantization, an input data bit width t _ bits and an output data bit width e _ bits of the global shared index mapping table, and comprises 1< < t _ bits table entries, the occupied storage space is (1< < t _ bits) × e _ bits/8 bytes, and < <' is a left shift operator.
11. The apparatus of claim 8,
the global shared reciprocal mapping function is a reciprocal mapping table generated according to a preset function, a linear function obtained by linearly fitting the preset function, or multiple functions generated by polynomial fitting of the preset functionA polynomial function; wherein the preset function is f (x) =2 r_bitsThe value range of the/x is a preset value interval, and the r _ bits is the output value bit width of the reciprocal mapping function;
the reciprocal unit determines a reciprocal mapping value of a sum of exponential mapping data corresponding to each row of input data according to a global shared reciprocal mapping function configured for a transformer structure in advance, and includes:
carrying out data conversion on the sum of the exponential mapping data corresponding to the row of input data;
and inputting the conversion result into the global shared reciprocal mapping function to obtain a reciprocal mapping value of the sum of the exponential mapping data corresponding to the input data of the row.
12. The apparatus of claim 11,
the reciprocal unit performs data conversion on the sum of the exponential mapping data corresponding to the row of input data, and includes:
determining a first effective bit number required to be occupied by the index mapping value sum corresponding to the row of input data and a second effective bit number required to be occupied by the left boundary value representing a preset value interval, and determining a difference value between the first effective bit number and the second effective bit number as a first right shift number required to map the index mapping value sum corresponding to the row of input data to the preset value interval;
and right shifting the exponent mapping data sum corresponding to the row of input data according to the first right shift number, mapping the right shift result to a numerical value in a value range determined by the output data bit width of the exponent mapping table, and taking the numerical value as conversion data of the exponent mapping data sum corresponding to the row of input data.
13. The apparatus of claim 12,
the multiplication unit performs multiplication and shift operation on the inverse mapping value of the sum of the exponent mapping data corresponding to the truncated data of each row of input data and the exponent mapping data corresponding to the row of input data to obtain the final result corresponding to the row of input data, and includes:
for each input data in the row of input data, performing the following operations:
calculating the product of the inverse mapping value of the index mapping data corresponding to the truncation data of the input data and the sum of the index mapping data corresponding to the input data of the row, and rounding the product to obtain an intermediate result corresponding to the input data;
and determining a second right shift number required for mapping the intermediate result corresponding to the input data to a value range determined by the output data bit width of the softmax module according to the first right shift number, the output data bit width of the global shared reciprocal mapping function and the output data bit width of the softmax module, and performing right shift on the intermediate result corresponding to the input data according to the second right shift number to obtain a final result corresponding to the input data.
14. The apparatus of claim 8,
when the input matrix is transmitted to the softmax module from the MatMul module, a pipeline mode is adopted for realization;
and the softmax module is used for temporarily storing a final result corresponding to each row of input data in the input matrix in a Static Random Access Memory (SRAM) in an operation mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210517307.3A CN114648101B (en) | 2022-05-13 | 2022-05-13 | Transformer structure-based softmax function quantization realization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210517307.3A CN114648101B (en) | 2022-05-13 | 2022-05-13 | Transformer structure-based softmax function quantization realization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114648101A true CN114648101A (en) | 2022-06-21 |
| CN114648101B CN114648101B (en) | 2022-08-12 |
Family
ID=81997334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210517307.3A Active CN114648101B (en) | 2022-05-13 | 2022-05-13 | Transformer structure-based softmax function quantization realization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114648101B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115469829A (en) * | 2022-10-28 | 2022-12-13 | 摩尔线程智能科技(北京)有限责任公司 | Arithmetic device and exponential operation method using arithmetic circuit |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021537A (en) * | 2018-01-05 | 2018-05-11 | 南京大学 | A kind of softmax implementations based on hardware platform |
| CN109308520A (en) * | 2018-09-26 | 2019-02-05 | 阿里巴巴集团控股有限公司 | Realize the FPGA circuitry and method that softmax function calculates |
| CN110135086A (en) * | 2019-05-20 | 2019-08-16 | 合肥工业大学 | The variable softmax function hardware circuit of computational accuracy and its implementation |
| CN111258839A (en) * | 2020-02-16 | 2020-06-09 | 苏州浪潮智能科技有限公司 | AI accelerator card simulation test system based on ResNet50 network and working method thereof |
| US20200211576A1 (en) * | 2018-12-28 | 2020-07-02 | Electronics And Telecommunications Research Institute | Method and device for determining loss function for audio signal |
| CN112686031A (en) * | 2020-12-24 | 2021-04-20 | 北京有竹居网络技术有限公司 | Text feature extraction model quantification method, device, equipment and storage medium |
| CN112685693A (en) * | 2020-12-31 | 2021-04-20 | 南方电网科学研究院有限责任公司 | Device for realizing Softmax function |
| CN112699284A (en) * | 2021-01-11 | 2021-04-23 | 四川大学 | Bus stop optimization visualization method based on multi-source data |
| CN112804726A (en) * | 2021-01-06 | 2021-05-14 | 南京理工大学 | Multi-agent reinforcement learning routing algorithm based on geographical position |
| CN113011571A (en) * | 2021-03-03 | 2021-06-22 | 华南理工大学 | INT8 offline quantization and integer inference method based on Transformer model |
| CN113052252A (en) * | 2021-03-31 | 2021-06-29 | 北京字节跳动网络技术有限公司 | Hyper-parameter determination method, device, deep reinforcement learning framework, medium and equipment |
| CN113377332A (en) * | 2021-05-28 | 2021-09-10 | 南京大学 | Softmax hardware implementation method based on linear segmentation |
| CN113449863A (en) * | 2021-07-14 | 2021-09-28 | 中国人民解放军国防科技大学 | Neural network quantization method based on table lookup |
| WO2021262053A1 (en) * | 2020-06-25 | 2021-12-30 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and system for image compressing and coding with deep learning |
| CN114090740A (en) * | 2021-11-19 | 2022-02-25 | 北京有竹居网络技术有限公司 | Intention recognition method and device, readable medium and electronic equipment |
| CN114169513A (en) * | 2022-02-11 | 2022-03-11 | 深圳比特微电子科技有限公司 | Neural network quantization method and device, storage medium and electronic equipment |
| CN114386469A (en) * | 2020-10-22 | 2022-04-22 | 珠海格力电器股份有限公司 | Method and device for quantizing convolutional neural network model and electronic equipment |
| CN114418929A (en) * | 2021-11-19 | 2022-04-29 | 东北大学 | Weld defect identification method based on consistency multi-scale metric learning |
| CN114463591A (en) * | 2022-02-13 | 2022-05-10 | 北京师范大学 | Deep neural network image classification method, device, equipment and storage medium |
-
2022
- 2022-05-13 CN CN202210517307.3A patent/CN114648101B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021537A (en) * | 2018-01-05 | 2018-05-11 | 南京大学 | A kind of softmax implementations based on hardware platform |
| CN109308520A (en) * | 2018-09-26 | 2019-02-05 | 阿里巴巴集团控股有限公司 | Realize the FPGA circuitry and method that softmax function calculates |
| US20200211576A1 (en) * | 2018-12-28 | 2020-07-02 | Electronics And Telecommunications Research Institute | Method and device for determining loss function for audio signal |
| CN110135086A (en) * | 2019-05-20 | 2019-08-16 | 合肥工业大学 | The variable softmax function hardware circuit of computational accuracy and its implementation |
| CN111258839A (en) * | 2020-02-16 | 2020-06-09 | 苏州浪潮智能科技有限公司 | AI accelerator card simulation test system based on ResNet50 network and working method thereof |
| WO2021262053A1 (en) * | 2020-06-25 | 2021-12-30 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and system for image compressing and coding with deep learning |
| CN114386469A (en) * | 2020-10-22 | 2022-04-22 | 珠海格力电器股份有限公司 | Method and device for quantizing convolutional neural network model and electronic equipment |
| CN112686031A (en) * | 2020-12-24 | 2021-04-20 | 北京有竹居网络技术有限公司 | Text feature extraction model quantification method, device, equipment and storage medium |
| CN112685693A (en) * | 2020-12-31 | 2021-04-20 | 南方电网科学研究院有限责任公司 | Device for realizing Softmax function |
| CN112804726A (en) * | 2021-01-06 | 2021-05-14 | 南京理工大学 | Multi-agent reinforcement learning routing algorithm based on geographical position |
| CN112699284A (en) * | 2021-01-11 | 2021-04-23 | 四川大学 | Bus stop optimization visualization method based on multi-source data |
| CN113011571A (en) * | 2021-03-03 | 2021-06-22 | 华南理工大学 | INT8 offline quantization and integer inference method based on Transformer model |
| CN113052252A (en) * | 2021-03-31 | 2021-06-29 | 北京字节跳动网络技术有限公司 | Hyper-parameter determination method, device, deep reinforcement learning framework, medium and equipment |
| CN113377332A (en) * | 2021-05-28 | 2021-09-10 | 南京大学 | Softmax hardware implementation method based on linear segmentation |
| CN113449863A (en) * | 2021-07-14 | 2021-09-28 | 中国人民解放军国防科技大学 | Neural network quantization method based on table lookup |
| CN114090740A (en) * | 2021-11-19 | 2022-02-25 | 北京有竹居网络技术有限公司 | Intention recognition method and device, readable medium and electronic equipment |
| CN114418929A (en) * | 2021-11-19 | 2022-04-29 | 东北大学 | Weld defect identification method based on consistency multi-scale metric learning |
| CN114169513A (en) * | 2022-02-11 | 2022-03-11 | 深圳比特微电子科技有限公司 | Neural network quantization method and device, storage medium and electronic equipment |
| CN114463591A (en) * | 2022-02-13 | 2022-05-10 | 北京师范大学 | Deep neural network image classification method, device, equipment and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| DANYANG ZHU 等: "Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning", 《JOURNAL OF LATEX CLASS FILES》 * |
| ZHEXIN LI 等: "Fixed-point Quantization for Vision Transformer", 《2021 CHINA AUTOMATION CONGRESS (CAC)》 * |
| ZHIHANG YUAN 等: "PTQ4ViT: Post-Training Quantization Framework for Vision Transformers", 《ARXIV:2111.12293V1》 * |
| 刘海章 等: "浅析激活函数SoftMax的设计与实现", 《西部广播电视》 * |
| 周维康 等: "采用软硬件协同设计的SoftMax函数的FPGA实现", 《信息通信》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115469829A (en) * | 2022-10-28 | 2022-12-13 | 摩尔线程智能科技(北京)有限责任公司 | Arithmetic device and exponential operation method using arithmetic circuit |
| CN115469829B (en) * | 2022-10-28 | 2023-07-04 | 摩尔线程智能科技(北京)有限责任公司 | Arithmetic device and exponent arithmetic method based on arithmetic circuit |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114648101B (en) | 2022-08-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hildebrandt et al. | On linear functional operations and the moment problem for a finite interval in one or several dimensions | |
| CN110070178A (en) | A kind of convolutional neural networks computing device and method | |
| CN110688088A (en) | General nonlinear activation function computing device and method for neural network | |
| CN107633298B (en) | Hardware architecture of recurrent neural network accelerator based on model compression | |
| CN114648101B (en) | Transformer structure-based softmax function quantization realization method and device | |
| CN111062610B (en) | Power system state estimation method and system based on information matrix sparse solution | |
| CN110377875B (en) | Matrix inversion method, apparatus, device and computer readable storage medium | |
| CN101655803A (en) | Method and mobile terminal for implementing mathematical model simulation | |
| Buchholz et al. | Complexity of Kronecker operations on sparse matrices with applications to the solution of Markov models | |
| CN105103152A (en) | Approximate query processing | |
| Wu et al. | Efficient dynamic fixed-point quantization of CNN inference accelerators for edge devices | |
| CN209708122U (en) | A kind of computing unit, array, module, hardware system | |
| CN111563598A (en) | Method and system for predicting quantum computation simulation time | |
| Rekha et al. | FPGA implementation of exponential function using cordic IP core for extended input range | |
| CN110837624B (en) | Approximation calculation device for sigmoid function | |
| CN115827555B (en) | Data processing method, computer device, storage medium, and multiplier structure | |
| CN109460535B (en) | Finite field matrix inversion device and inversion method based on cloud | |
| CN110633447B (en) | Spherical distance fixed-point calculation method based on FPGA and calculation device thereof | |
| US9213639B2 (en) | Division of numerical values based on summations and memory mapping in computing systems | |
| CN113778681B (en) | Data processing method and device based on cloud computing and storage medium | |
| CN115408061A (en) | Hardware acceleration method, device, chip and storage medium for complex matrix operation | |
| Lin et al. | Hybrid dynamic fixed point quantization methodology for AI accelerators | |
| CN110413647B (en) | High-dimensional vector unequal length sequence similarity rapid calculation system | |
| CN114385540A (en) | Data unit conversion method and device | |
| WO2023028884A1 (en) | Floating-point number computing circuit and floating-point number computing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |