CN115062777A - Method for quantizing convolutional neural network, device, and storage medium - Google Patents
Method for quantizing convolutional neural network, device, and storage medium Download PDFInfo
- Publication number
- CN115062777A CN115062777A CN202210985406.4A CN202210985406A CN115062777A CN 115062777 A CN115062777 A CN 115062777A CN 202210985406 A CN202210985406 A CN 202210985406A CN 115062777 A CN115062777 A CN 115062777A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- quantization
- convolutional
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention is suitable for the technical field of computers, and provides a quantization method of a convolutional neural network, the neural network, equipment and a storage medium, wherein the method comprises the following steps: a. pre-training a convolutional neural network to be quantized to obtain a first convolutional neural network; b. quantizing the first convolutional neural network to obtain a second convolutional neural network; c. calculating the contribution degree of each convolution layer which is not subjected to residue compensation quantization in the second convolution neural network; d. according to the obtained contribution degree, acquiring a convolution layer to be compensated which needs to be subjected to residue compensation quantization in the second convolutional neural network, and performing residue compensation quantization on the convolution layer to be compensated to obtain a third convolutional neural network; e. and c, testing the accuracy of the third convolutional neural network, outputting the third convolutional neural network if the accuracy reaches the preset accuracy, otherwise, setting the third convolutional neural network as the first convolutional neural network, and jumping to the step b until the accuracy of the third convolutional neural network reaches the preset accuracy.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a quantization method of a convolutional neural network, the neural network, equipment and a storage medium.
Background
Deep learning is used as a branch of artificial intelligence and is closely related to a deep learning algorithm in deep learning, such as a convolutional neural network. With more and more attention paid to artificial intelligence, the convolutional neural network is rapidly developed, is a deep neural network with a convolutional structure, and is widely applied to the fields of image and voice recognition and the like.
However, the convolutional neural network has more parameters after training is completed, for example, there are generally hundreds, thousands, or even tens of thousands of parameters, so that the whole computing process of the neural network needs to consume a large amount of storage and computing resources, and as the convolutional neural network is deeply researched, the convolutional neural network develops towards a deeper and more complex hierarchy, which puts higher requirements on the storage and computing resources of the operating platform. At present, a solution for reducing the complexity of a convolutional neural network on the premise of not reducing the accuracy of the convolutional neural network is generally implemented by quantizing parameters of the convolutional neural network, however, the accuracy of the convolutional neural network is reduced by using the existing quantization method, and the user experience of various convolutional neural network applications is affected.
Disclosure of Invention
The invention aims to provide a method for quantizing a convolutional neural network, the neural network, equipment and a storage medium, and aims to solve the problem that the accuracy of the quantized convolutional neural network is not high because the prior art cannot provide an effective method for quantizing the convolutional neural network.
In one aspect, the present invention provides a quantization method of a convolutional neural network, the method comprising the steps of:
a. pre-training a convolutional neural network to be quantized to obtain a first convolutional neural network;
b. quantizing the first convolutional neural network to obtain a second convolutional neural network corresponding to the preset quantization bit width constraint;
c. calculating the contribution degree of each convolution layer which is not subjected to remainder compensation quantization in the second convolution neural network according to the network parameters of the first convolution neural network and the second convolution neural network;
d. according to the calculated contribution degree, acquiring a convolution layer to be compensated, which needs to be subjected to remainder compensation quantization, in the second convolutional neural network, and performing remainder compensation quantization on the convolution layer to be compensated to obtain a corresponding third convolutional neural network;
e. and c, testing the accuracy of the third convolutional neural network, outputting the third convolutional neural network if the accuracy of the third convolutional neural network reaches preset accuracy, otherwise setting the third convolutional neural network as the first convolutional neural network, and skipping to the step b to continue to carry out residue compensation quantization on convolutional layers which are not subjected to residue compensation quantization until the accuracy of the third convolutional neural network reaches the preset accuracy.
In another aspect, the present invention provides a convolutional neural network, comprising:
a quantization module, configured to quantize an input of a kth convolutional layer of the convolutional neural network and a weight of the kth convolutional layer, where k is a positive integer;
the convolution module is used for performing convolution calculation on the input of the quantized kth layer of convolution layer and the quantized weight to obtain a convolution result of the kth layer of convolution layer; and
and the residual compensation module is used for performing residual compensation quantization on the convolution result of the kth layer of convolution layer.
In another aspect, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method when executing the computer program.
In another aspect, the present invention also provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method as described above.
The invention quantizes a first convolutional neural network obtained by pre-training to obtain a second convolutional neural network corresponding to the constraint of a preset quantization bit width, calculates the contribution degree of each convolutional layer which is not subjected to the remainder compensation quantization in the second convolutional neural network according to network parameters of the first convolutional neural network and the second convolutional neural network, obtains a convolutional layer to be compensated which is required to be subjected to the remainder compensation quantization in the second convolutional neural network according to the contribution degree, performs the remainder compensation quantization on the convolutional layer to be compensated to obtain a third convolutional neural network, tests the accuracy of the third convolutional neural network, outputs the third convolutional neural network if the accuracy reaches the preset accuracy, otherwise sets the third convolutional neural network as the first convolutional neural network to continue to perform the remainder compensation quantization on the convolutional layer which is not subjected to the remainder compensation quantization until the accuracy of the third convolutional neural network reaches the preset accuracy, therefore, the quantized convolutional layer is dynamically and differentially compensated based on the contribution degree of the convolutional layer, and the accuracy of the quantized convolutional neural network is improved.
Drawings
Fig. 1 is a flowchart illustrating an implementation of a quantization method of a convolutional neural network according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a convolutional neural network according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1 shows an implementation flow of a quantization method of a convolutional neural network according to an embodiment of the present invention, and for convenience of description, only the parts related to the embodiment of the present invention are shown, which are detailed as follows:
in step S101, pre-training a convolutional neural network to be quantized to obtain a first convolutional neural network;
embodiments of the present invention are applicable to electronic devices, which may be, by way of example, any of desktop, mobile, or portable computer system devices. In particular, the electronic device may be a mobile or smart phone, a portable internet appliance, a personal desktop computer or a server. The convolutional neural network to be quantized is pre-trained on the electronic device to obtain a pre-trained convolutional neural network, which is referred to as a first convolutional neural network for convenience of description, and may be a deep residual network, for example, ResNet-50.
In step S102, quantizing the first convolutional neural network to obtain a second convolutional neural network corresponding to the preset quantization bit width constraint;
in the embodiment of the invention, when the first convolution neural network is quantized, floating point weight values of most operations in the first convolution neural network are converted into fixed point integer representations, and operations of input parameters (data) are replaced by operations capable of executing similar fixed point integer operations, so that most floating point operations in the network are replaced by fixed point integer calculation. And quantizing the first convolutional neural network to obtain a quantized convolutional neural network corresponding to the preset quantization bit width constraint, wherein for convenience of description, the convolutional neural network is referred to as a second convolutional neural network. By way of example, the quantization bit width may be 4 bits, i.e.: the floating point weights in the first convolutional neural network are quantized into 4-bit fixed point integers, and the operation of the input parameters is replaced by the operation capable of executing similar 4-bit fixed point integer operation.
In one embodiment, a formula is used
And
the first convolutional neural network is quantized, wherein,
representing the result after quantization of the neural network parameters (including the weight parameters and the input parameters),
representing the parameters before quantization of the neural network parameters, s representing the quantization step size, is represented by
Is obtained wherein
And
is that
The median minimum value and the maximum value,bindicating the width of the bit to be quantized,zrepresents an offset constant, which can be represented by
The calculation results in that,
representing a rounding function that maps real numbers to nearest integers,clampthe function of the truncation is represented by,
for inverse quantization.
In step S103, calculating a contribution of each convolutional layer in the second convolutional neural network without performing remainder compensation quantization according to the network parameters of the first convolutional neural network and the second convolutional neural network;
in the embodiment of the present invention, the contribution of each convolutional layer in the second convolutional neural network, which is not subjected to the residue compensation quantization, is calculated according to the network parameters of the first convolutional neural network and the second convolutional neural network, that is, the contribution of each convolutional layer in the convolutional neural network, which is not subjected to the residue compensation quantization, is calculated according to the network parameters of the convolutional neural network before quantization and the convolutional neural network after quantization. The contribution degree is the information loss degree of each convolution layer before and after the convolution neural network is quantized, the higher the information loss degree is, the greater the accuracy contribution of the convolution neural network is, and the lower the information loss degree is, the smaller the accuracy contribution of the convolution neural network is. The remainder compensation quantization refers to that remainders of input and weight parameters are quantized and then compensated to be output of convolution in the convolution process, so that the response efficiency of the convolution neural network can be improved through the quantization of the neural network, and meanwhile, the accuracy of the quantized convolution neural network is improved through the remainder compensation quantization.
Therefore, when the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network is calculated, the information loss degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network can be calculated according to the network parameters of the first convolutional neural network and the second convolutional neural network, and the information loss degree is set to be the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization, so that the contribution of each convolutional layer to the accuracy of the convolutional neural network is accurately obtained. In a preferred embodiment, a formula is used in calculating the information loss of each convolutional layer in the second convolutional neural network that is not residue compensation quantized
Calculating the information loss degree of each convolution layer, wherein S represents the information loss degree,y i representing the output of the convolutional layer of the first convolutional neural network,
representing the output of the convolutional layer of the second convolutional neural network,p represents the norm and n is the quantity of quantization parameters, so that the contribution degree of each convolution layer without remainder compensation quantization can be quickly and accurately obtained.
In step S104, according to the calculated contribution, a convolution layer to be compensated, which needs to be subjected to remainder compensation quantization, in the second convolutional neural network is obtained, and remainder compensation quantization is performed on the convolution layer to be compensated, so as to obtain a corresponding third convolutional neural network;
in the embodiment of the present invention, when the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently in the second convolutional neural network, is obtained according to the calculated contribution degree, only the convolution layer with the largest contribution degree may be obtained as the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently, and the convolution layers with the preset number of layers may also be obtained as the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently. In a preferred embodiment, when convolution layers to be compensated, which need to be subjected to residue compensation quantization, in the second convolutional neural network are obtained, the calculated contribution degrees are sorted, and convolution layers corresponding to a preset number of contribution degrees in front of a sorted contribution degree queue are set as convolution layers to be compensated, so that the residue compensation quantization efficiency of the convolutional neural network is improved.
After a convolutional layer to be compensated, which needs to be subjected to residue compensation quantization, in the second convolutional neural network is obtained, the residue compensation quantization is performed on the convolutional layer to be compensated, so as to obtain a convolutional neural network corresponding to the convolutional layer to be compensated, which is subjected to residue compensation quantization. In a preferred embodiment, when performing the residue compensation quantization on the convolutional layer to be compensated, the compensation value of the convolutional layer to be compensated is calculated according to the weight of the convolutional layer to be compensated, the input, the quantization step and the quantization bit width corresponding to the weight, and the quantization step and the quantization bit width corresponding to the input, and the residue compensation quantization is performed on the convolutional layer to be compensated by using the compensation value, so as to improve the accuracy of the residue compensation quantization and further improve the accuracy of the quantized convolutional neural network. Further preferably, the compensation value of the convolution layer to be compensated is calculated using the following formula:
In step S105, testing the accuracy of the third convolutional neural network;
in the embodiment of the present invention, the accuracy of the third convolutional neural network is tested by using a preset sample to determine whether the accuracy of the third convolutional neural network reaches the preset accuracy, that is, whether the accuracy of the convolutional neural network quantized and compensated and quantized by the current remainder of the convolutional layer reaches the preset accuracy is determined.
In step S106, it is determined whether the accuracy of the third convolutional neural network reaches the preset accuracy, if so, step S107 is performed, otherwise, step S108 is performed, and then step S102 is performed, so as to continue performing the remainder compensation quantization on the convolutional layer on which the remainder compensation quantization is not performed until the accuracy of the third convolutional neural network reaches the preset accuracy;
in step S107, a third convolutional neural network is output;
in step S108, the third convolutional neural network is set as the first convolutional neural network to continue the residue compensation quantization of the convolutional layer on which the residue compensation quantization is not performed.
In the embodiment of the invention, after the residue compensation quantization is carried out on the convolution layer to be compensated, if the accuracy of the obtained third convolutional neural network reaches the preset accuracy, the third convolutional neural network is output, the third convolutional neural network is the quantized final convolutional neural network, the storage and calculation resources of the application equipment are reduced by the convolutional neural network, and meanwhile, the accuracy of the convolutional neural network is improved through the residue compensation quantization. If the accuracy of the obtained third convolutional neural network does not reach the preset accuracy, the third convolutional neural network is set as the first convolutional neural network, and then the step S102 is skipped to continue to perform the residue compensation quantization on the convolutional layer which is not subjected to the residue compensation quantization, so that the quantized convolutional neural network can finally reach the user requirement.
In the embodiment of the invention, a first convolutional neural network obtained by pre-training is quantized to obtain a second convolutional neural network corresponding to the constraint of a preset quantization bit width, the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network is calculated according to network parameters of the first convolutional neural network and the second convolutional neural network, convolutional layers to be compensated which are required to be subjected to the residue compensation quantization in the second convolutional neural network are obtained according to the contribution degree, the residue compensation quantization is carried out on the convolutional layers to be compensated to obtain a third convolutional neural network, the accuracy of the third convolutional neural network is tested, if the accuracy reaches the preset accuracy, the third convolutional neural network is output, otherwise, the third convolutional neural network is set as the first convolutional neural network so as to continuously carry out the residue compensation quantization on the convolutional layers which are not subjected to the residue compensation quantization, and until the accuracy of the third convolutional neural network reaches the preset accuracy, dynamically compensating the quantized convolutional layer based on the contribution degree of the convolutional layer, and improving the accuracy of the quantized convolutional neural network.
Example two:
fig. 2 shows a structure of a convolutional neural network provided in a second embodiment of the present invention, and for convenience of explanation, only the parts related to the second embodiment of the present invention are shown, which include:
a quantization module 21, configured to quantize an input of a kth convolutional layer of the convolutional neural network and a weight of the kth convolutional layer, where k is a positive integer;
a convolution module 22, configured to perform convolution calculation on the input of the quantized kth convolutional layer and the quantized weight to obtain a convolution result of the kth convolutional layer; and
and a residual compensation module 23, configured to perform residual compensation quantization on the convolution result of the kth convolutional layer.
In an embodiment of the present invention, when performing the remainder compensation quantization on the convolution result of the kth convolutional layer, the remainder compensation module 23 calculates a compensation value of the kth convolutional layer according to the weight of the kth convolutional layer, the input, the quantization step and the quantization bit width corresponding to the weight, and the quantization step and the quantization bit width corresponding to the input, and performs the remainder compensation quantization on the kth convolutional layer by using the compensation value.
In a preferred embodiment, when performing residue compensation quantization on convolution results of a kth convolutional layer, according to network parameters of an unquantized convolutional neural network (first convolutional neural network) and a quantized convolutional neural network (second convolutional neural network), calculating a contribution degree of each convolutional layer, which is not subjected to residue compensation quantization, in the second convolutional neural network, obtaining a convolutional layer to be compensated (kth convolutional layer) which needs to be subjected to residue compensation quantization in the second convolutional neural network according to the calculated contribution degree, and performing residue compensation quantization on the convolutional layer to be compensated to obtain a corresponding third convolutional neural network, so that response efficiency of the convolutional neural network can be improved through quantization of the neural network, and meanwhile, accuracy of the convolutional neural network is improved through residue compensation quantization.
In the embodiment of the present invention, the contribution of each convolutional layer in the second convolutional neural network, which is not subjected to the residue compensation quantization, is calculated according to the network parameters of the first convolutional neural network and the second convolutional neural network, that is, the contribution of each convolutional layer in the convolutional neural network, which is not subjected to the residue compensation quantization, is calculated according to the network parameters of the convolutional neural network before quantization and the convolutional neural network after quantization. The contribution degree is the information loss degree of each convolution layer before and after the convolution neural network is quantized, the higher the information loss degree is, the greater the accuracy contribution of the convolution neural network is, and the lower the information loss degree is, the smaller the accuracy contribution of the convolution neural network is. The remainder compensation quantization refers to that remainders of input and weight parameters are quantized and then compensated to be output of convolution in the convolution process, so that the response efficiency of the convolution neural network can be improved through the neural network quantization, and meanwhile, the accuracy of the convolution neural network is improved through the remainder compensation quantization.
Therefore, when the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network is calculated, the information loss degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network can be calculated according to the network parameters of the first convolutional neural network and the second convolutional neural network, and the information loss degree is set to be the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization, so that the contribution of each convolutional layer to the accuracy of the convolutional neural network is accurately obtained. In a preferred embodiment, a formula is used in calculating the information loss of each convolutional layer in the second convolutional neural network that is not residue compensation quantized
Calculating the information loss degree of each convolution layer, wherein S represents the information loss degree,y i represents the output of the convolutional layer of the first convolutional neural network,
the output of the convolutional layer of the second convolutional neural network is represented, p represents the norm, and n represents the number of quantization parameters, so that the contribution degree of each convolutional layer which is not subjected to residue compensation quantization can be quickly and accurately obtained.
In the embodiment of the present invention, when the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently in the second convolutional neural network, is obtained according to the calculated contribution degree, only the convolution layer with the largest contribution degree may be obtained as the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently, and the convolution layers with the preset number of layers may also be obtained as the convolution layer to be compensated, which needs to be subjected to the residue compensation quantization currently. In a preferred embodiment, when convolution layers to be compensated, which need to be subjected to residue compensation quantization, in the second convolutional neural network are obtained, the calculated contribution degrees are sorted, and convolution layers corresponding to a preset number of contribution degrees in front of a sorted contribution degree queue are set as convolution layers to be compensated, so that the residue compensation quantization efficiency of the convolutional neural network is improved.
After a convolutional layer to be compensated, which needs to be subjected to residue compensation quantization, in the second convolutional neural network is obtained, the residue compensation quantization is performed on the convolutional layer to be compensated, so as to obtain a convolutional neural network corresponding to the convolutional layer to be compensated, which is subjected to residue compensation quantization. In a preferred embodiment, when performing the residue compensation quantization on the convolutional layer to be compensated, the compensation value of the convolutional layer to be compensated is calculated according to the weight of the convolutional layer to be compensated, the input, the quantization step and the quantization bit width corresponding to the weight, and the quantization step and the quantization bit width corresponding to the input, and the residue compensation quantization is performed on the convolutional layer to be compensated by using the compensation value, so as to improve the accuracy of the residue compensation quantization and further improve the accuracy of the quantized convolutional neural network. Further preferably, the compensation value of the convolution layer to be compensated is calculated using the following formula:
Example three:
fig. 3 shows a structure of an electronic device according to a third embodiment of the present invention, and for convenience of description, only the portions related to the third embodiment of the present invention are shown.
The electronic device 3 of an embodiment of the invention comprises a processor 30, a memory 31 and a computer program 32 stored in the memory 31 and executable on the processor 30. The processor 30, when executing the computer program 32, implements the steps in the above-described quantization method embodiment of the convolutional neural network, such as the steps S101 to S108 shown in fig. 1.
In the embodiment of the invention, a first convolutional neural network obtained by pre-training is quantized to obtain a second convolutional neural network corresponding to the constraint of a preset quantization bit width, the contribution degree of each convolutional layer which is not subjected to the residue compensation quantization in the second convolutional neural network is calculated according to network parameters of the first convolutional neural network and the second convolutional neural network, a convolutional layer to be compensated which is required to be subjected to the residue compensation quantization in the second convolutional neural network is obtained according to the contribution degree, the residue compensation quantization is performed on the convolutional layer to be compensated to obtain a third convolutional neural network, the accuracy of the third convolutional neural network is tested, if the accuracy reaches the preset accuracy, the third convolutional neural network is output, otherwise, the third convolutional neural network is set as the first convolutional neural network so as to continue to perform the residue compensation quantization on the convolutional layer which is not subjected to the residue compensation quantization, and until the accuracy of the third convolutional neural network reaches the preset accuracy, performing dynamic and differential compensation on the quantized convolutional layer based on the contribution degree of the convolutional layer, and improving the accuracy of the quantized convolutional neural network.
The electronic device of embodiments of the present invention may be any of a desktop, mobile, or portable computer system device. For the steps implemented when the processor 30 in the electronic device 3 executes the computer program 32 to implement the quantization method of the convolutional neural network, reference may be made to the description of the foregoing method embodiments, and details are not repeated here.
Example four:
in an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, which when executed by a processor implements the steps in the above-described quantization method embodiment of the convolutional neural network, for example, steps S101 to S108 shown in fig. 1.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A method for quantizing a convolutional neural network, comprising the steps of:
a. pre-training a convolutional neural network to be quantized to obtain a first convolutional neural network;
b. quantizing the first convolutional neural network to obtain a second convolutional neural network corresponding to the preset quantization bit width constraint;
c. calculating the contribution of each convolution layer which is not subjected to remainder compensation quantization in the second convolution neural network according to the network parameters of the first convolution neural network and the second convolution neural network;
d. according to the calculated contribution degree, acquiring a convolution layer to be compensated, which needs to be subjected to remainder compensation quantization, in the second convolutional neural network, and performing remainder compensation quantization on the convolution layer to be compensated to obtain a corresponding third convolutional neural network;
e. and c, testing the accuracy of the third convolutional neural network, outputting the third convolutional neural network if the accuracy of the third convolutional neural network reaches preset accuracy, otherwise setting the third convolutional neural network as the first convolutional neural network, and skipping to the step b to continue to carry out residue compensation quantization on convolutional layers which are not subjected to residue compensation quantization until the accuracy of the third convolutional neural network reaches the preset accuracy.
2. The method of claim 1, wherein the step of calculating the contribution of each convolutional layer in the second convolutional neural network that is not residue compensation quantized based on the network parameters of the first convolutional neural network and the second convolutional neural network comprises:
and calculating the information loss degree of each convolution layer which is not subjected to residue compensation quantization in the second convolutional neural network according to the network parameters of the first convolutional neural network and the second convolutional neural network, and setting the information loss degree as the contribution degree.
3. The method of claim 2, wherein the step of calculating the degree of information loss for each convolutional layer in the second convolutional neural network that is not residue compensation quantized comprises:
using the formula
Calculating the information loss degree, wherein S represents the information loss degree,y i representing an output of the convolutional layer of the first convolutional neural network,
represents the output of the convolutional layer of the second convolutional neural network, p represents the norm, and n is the number of quantization parameters.
4. The method of claim 1, wherein the step of obtaining the convolutional layer to be compensated in the second convolutional neural network, which needs to be subjected to residue compensation quantization, according to the contribution degree obtained by calculation comprises:
and sequencing the calculated contribution degrees, and setting the convolutional layers corresponding to the contribution degrees of the preset number in the front of the sequenced contribution degree queue as convolutional layers to be compensated.
5. The method of claim 1, wherein the step of performing residue compensation quantization on the convolutional layer to be compensated comprises:
and calculating a compensation value of the convolution layer to be compensated according to the weight and the input of the convolution layer to be compensated, the quantization step size and the quantization bit width corresponding to the weight and the quantization bit width corresponding to the input, and performing remainder compensation quantization on the convolution layer to be compensated by using the compensation value.
6. The method of claim 5, wherein the compensation value for the convolutional layer to be compensated is calculated using the following equation:
7. A convolutional neural network, comprising:
a quantization module, configured to quantize an input of a kth convolutional layer of the convolutional neural network and a weight of the kth convolutional layer, where k is a positive integer;
the convolution module is used for performing convolution calculation on the input of the quantized kth layer of convolutional layer and the quantized weight to obtain a convolution result of the kth layer of convolutional layer; and
and the residual compensation module is used for performing residual compensation quantization on the convolution result of the kth layer of convolution layer.
8. The convolutional neural network of claim 7, wherein the residue compensation module calculates a compensation value for the kth convolutional layer according to the weight of the kth convolutional layer, an input, a quantization step size and a quantization bit width corresponding to the weight, and a quantization step size and a quantization bit width corresponding to the input, and performs residue compensation quantization on the kth convolutional layer using the compensation value.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 6 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210985406.4A CN115062777B (en) | 2022-08-17 | 2022-08-17 | Quantization method, quantization device, equipment and storage medium of convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210985406.4A CN115062777B (en) | 2022-08-17 | 2022-08-17 | Quantization method, quantization device, equipment and storage medium of convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115062777A true CN115062777A (en) | 2022-09-16 |
| CN115062777B CN115062777B (en) | 2022-11-22 |
Family
ID=83208399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210985406.4A Active CN115062777B (en) | 2022-08-17 | 2022-08-17 | Quantization method, quantization device, equipment and storage medium of convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115062777B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112488291A (en) * | 2020-11-03 | 2021-03-12 | 珠海亿智电子科技有限公司 | Neural network 8-bit quantization compression method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596143A (en) * | 2018-05-03 | 2018-09-28 | 复旦大学 | Face identification method based on residual quantization convolutional neural networks and device |
| CN108898168A (en) * | 2018-06-19 | 2018-11-27 | 清华大学 | The compression method and system of convolutional neural networks model for target detection |
| US20180350109A1 (en) * | 2017-06-06 | 2018-12-06 | Via Alliance Semiconductor Co., Ltd. | Method and device for data quantization |
| CN111260022A (en) * | 2019-11-22 | 2020-06-09 | 中国电子科技集团公司第五十二研究所 | Method for fixed-point quantization of complete INT8 of convolutional neural network |
| CN111985495A (en) * | 2020-07-09 | 2020-11-24 | 珠海亿智电子科技有限公司 | Model deployment method, device, system and storage medium |
| WO2021159023A1 (en) * | 2020-02-07 | 2021-08-12 | The Regents Of The University Of California | Query optimization for deep convolutional neural network inferences |
| WO2021179631A1 (en) * | 2020-09-23 | 2021-09-16 | 平安科技（深圳）有限公司 | Convolutional neural network model compression method, apparatus and device, and storage medium |
| CN113409773A (en) * | 2021-08-18 | 2021-09-17 | 中科南京智能技术研究院 | Binaryzation neural network voice awakening method and system |
-
2022
- 2022-08-17 CN CN202210985406.4A patent/CN115062777B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180350109A1 (en) * | 2017-06-06 | 2018-12-06 | Via Alliance Semiconductor Co., Ltd. | Method and device for data quantization |
| CN108596143A (en) * | 2018-05-03 | 2018-09-28 | 复旦大学 | Face identification method based on residual quantization convolutional neural networks and device |
| CN108898168A (en) * | 2018-06-19 | 2018-11-27 | 清华大学 | The compression method and system of convolutional neural networks model for target detection |
| CN111260022A (en) * | 2019-11-22 | 2020-06-09 | 中国电子科技集团公司第五十二研究所 | Method for fixed-point quantization of complete INT8 of convolutional neural network |
| WO2021159023A1 (en) * | 2020-02-07 | 2021-08-12 | The Regents Of The University Of California | Query optimization for deep convolutional neural network inferences |
| CN111985495A (en) * | 2020-07-09 | 2020-11-24 | 珠海亿智电子科技有限公司 | Model deployment method, device, system and storage medium |
| WO2021179631A1 (en) * | 2020-09-23 | 2021-09-16 | 平安科技（深圳）有限公司 | Convolutional neural network model compression method, apparatus and device, and storage medium |
| CN113409773A (en) * | 2021-08-18 | 2021-09-17 | 中科南京智能技术研究院 | Binaryzation neural network voice awakening method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112488291A (en) * | 2020-11-03 | 2021-03-12 | 珠海亿智电子科技有限公司 | Neural network 8-bit quantization compression method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115062777B (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108510067B (en) | Convolutional neural network quantification method based on engineering realization | |
| EP3619652B1 (en) | Adaptive bit-width reduction for neural networks | |
| KR102602195B1 (en) | Quantization of trained long-short-term memory neural networks | |
| CN108460028B (en) | Domain adaptation method for integrating sentence weight into neural machine translation | |
| CN115238893B (en) | Neural network model quantification method and device for natural language processing | |
| CN111985495A (en) | Model deployment method, device, system and storage medium | |
| TWI744724B (en) | Method of processing convolution neural network | |
| CN115631261B (en) | Training method of image generation model, image generation method and device | |
| CN115062777B (en) | Quantization method, quantization device, equipment and storage medium of convolutional neural network | |
| Huai et al. | Zerobn: Learning compact neural networks for latency-critical edge systems | |
| CN114677548A (en) | Neural network image classification system and method based on resistive random access memory | |
| CN115393633A (en) | Data processing method, electronic device, storage medium, and program product | |
| Huai et al. | Latency-constrained DNN architecture learning for edge systems using zerorized batch normalization | |
| CN112257466A (en) | Model compression method applied to small machine translation equipment | |
| CN111797220A (en) | Dialog generation method and device, computer equipment and storage medium | |
| CN113408702B (en) | Music neural network model pre-training method, electronic device and storage medium | |
| CN110276448B (en) | Model compression method and device | |
| Liu et al. | Block-Wise Dynamic-Precision Neural Network Training Acceleration via Online Quantization Sensitivity Analytics | |
| CN113361707A (en) | Model compression method, system and computer readable medium | |
| CN113159318A (en) | Neural network quantification method and device, electronic equipment and storage medium | |
| US20230042275A1 (en) | Network quantization method and network quantization device | |
| CN115496200B (en) | Neural network quantization model training method, device and equipment | |
| CN111737440B (en) | Question generation method and device | |
| CN116611494A (en) | Training method and device for electric power defect detection model, computer equipment and medium | |
| CN116956997A (en) | LSTM model quantization retraining method, system and equipment for time sequence data processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |