CN114048853A - Neural network quantification method and device, computer equipment and storage medium - Google Patents

Neural network quantification method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN114048853A
CN114048853A CN202111433360.7A CN202111433360A CN114048853A CN 114048853 A CN114048853 A CN 114048853A CN 202111433360 A CN202111433360 A CN 202111433360A CN 114048853 A CN114048853 A CN 114048853A
Authority
CN
China
Prior art keywords
quantization
iteration cycle
objects
determining
coefficient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111433360.7A
Other languages
Chinese (zh)
Inventor
史丽坤
胡英俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Power Tensors Intelligent Technology Co Ltd
Original Assignee
Shanghai Power Tensors Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Power Tensors Intelligent Technology Co Ltd filed Critical Shanghai Power Tensors Intelligent Technology Co Ltd
Priority to CN202111433360.7A priority Critical patent/CN114048853A/en
Publication of CN114048853A publication Critical patent/CN114048853A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present disclosure provides a quantization method, apparatus, computer device and storage medium for a neural network, wherein the method comprises: performing in each iteration cycle of a plurality of iteration cycles: determining quantization coefficients corresponding to a plurality of quantization objects for quantizing a target network layer in a first neural network; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results of the plurality of quantization objects respectively corresponding to the current iteration cycle; determining quantization losses of the plurality of quantization objects in the current iteration cycle respectively based on quantization results of the plurality of quantization objects in the current iteration cycle respectively, and determining loss weights corresponding to the quantization losses of the plurality of quantization objects respectively based on the quantization losses; and determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.

Description

Neural network quantification method and device, computer equipment and storage medium
Technical Field
The present disclosure relates to the field of neural network technologies, and in particular, to a quantization method and apparatus for a neural network, a computer device, and a storage medium.
Background
With the development of the neural network, the neural network has larger scale and higher complexity, which results in more computing resources and storage resources required when the neural network is used to perform processing tasks. However, the terminal device deploying the neural network has limited computing resources and storage resources, and in order to deploy the neural network to the terminal device, the neural network needs to be compressed; quantization is a compression processing method for a neural network, which quantizes network layer internal parameters of the neural network and data transmitted in the network from floating point numbers to fixed point numbers with fixed bit positions, and requires fewer computing resources and fewer storage resources by fixed point number operation than by floating point number operation. When the neural network is quantized, the accuracy of the quantized neural network is greatly reduced.
Disclosure of Invention
The embodiment of the disclosure at least provides a quantization method and device of a neural network, computer equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a quantization method for a neural network, including: performing in each iteration cycle of a plurality of iteration cycles: determining quantization coefficients corresponding to a plurality of quantization objects for quantizing a target network layer in a first neural network; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle; determining quantization losses of the quantization objects in the current iteration cycle based on quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses; and determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
In an alternative embodiment, the quantization object includes at least one of: input data, output data and network layer internal parameters of the target network layer.
In an optional embodiment, the quantization object includes a network layer internal parameter of the target network layer; the determining a current iteration cycle to quantize a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects include: responding to the fact that the current iteration cycle is the first iteration cycle, and determining an original quantization coefficient of the network layer internal parameter based on a preset quantization coefficient space; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the network layer internal parameter.
In an alternative embodiment, the quantization object includes: input data of the target network layer and/or output data of the target network layer; the determining a current iteration cycle to quantize a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects include: determining first distribution characteristic information of the input data based on the input data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the input data based on the first distribution characteristic information; and/or determining second distribution characteristic information of the output data based on the output data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the output data based on the second distribution characteristic information; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the input data and/or the original quantization coefficient of the output data.
In an optional embodiment, the quantization object comprises input data, network layer internal parameters and output data of the target network layer; the quantized coefficients include: a first quantized coefficient of the input data and a second quantized coefficient of the network layer internal parameter; the performing quantization operations on the multiple quantization objects based on the quantization coefficients to obtain quantization results of the multiple quantization objects in a current iteration cycle respectively includes: for a first quantization object with the type of input data, performing quantization operation on the first quantization object based on a first quantization coefficient to obtain a first quantization result of the first quantization object in a current iteration cycle; for a second quantization object with the type of network layer internal parameters, performing quantization operation on the second quantization object based on a second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for a third quantization object of which the type is output data, performing quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or determining a third quantization result corresponding to the third quantization object in the current iteration cycle based on the first quantization result and the second quantization result.
In an optional embodiment, the determining, based on the quantization results of the plurality of quantization objects in the current iteration cycle, the quantization losses of the plurality of quantization objects in the current iteration cycle respectively includes: for each quantization object in a plurality of quantization objects, performing inverse quantization operation on a quantization result corresponding to each quantization object in a current iteration cycle based on the quantization coefficient to obtain an inverse quantization result corresponding to each quantization object; and obtaining the quantization loss of each quantization object in the current iteration cycle based on each quantization object and the corresponding inverse quantization result.
In an optional embodiment, the determining, based on the quantization loss, loss weights respectively corresponding to the quantization losses of the plurality of quantization objects includes: carrying out normalization processing on the quantization losses respectively corresponding to the quantization objects belonging to the same type to obtain the loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or respectively processing the quantization objects belonging to the same type based on a preset monotonic function to obtain loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or, carrying out weight prediction processing on quantization objects belonging to the same type by adopting a pre-trained weight prediction neural network to obtain loss weights corresponding to quantization losses of the quantization objects belonging to the same type respectively; or, for each quantization object, determining a target value interval to which the quantization loss of each quantization object belongs in a plurality of value intervals; and determining a loss weight corresponding to the quantization loss of each quantization object based on the weight corresponding to the target value interval.
In an optional implementation manner, after determining quantization losses corresponding to the quantization objects in the current iteration cycle based on the quantization results corresponding to the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects in the current iteration cycle based on the quantization losses, the method further includes: determining whether the current iteration cycle meets an iteration stop condition based on the loss weights respectively corresponding to the multiple quantitative objects; entering a next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; the determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient includes: and in response to that the current iteration cycle meets the iteration stop condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.
In an alternative embodiment, the iteration stop condition includes at least one of: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on loss weights respectively corresponding to the plurality of quantization objects; the difference value between the total quantization losses corresponding to the target iteration cycles is smaller than a preset difference value threshold value; wherein the target iteration cycle comprises: the current iteration cycle, and at least one historical iteration cycle that is continuous with the current iteration cycle.
In an optional embodiment, the quantizing the first neural network by using the target quantization coefficient to generate a second neural network includes: quantizing the quantization object in the target network layer by using the target quantization coefficient to obtain a target quantization object; generating the second neural network based on the target quantization object.
In a second aspect, an embodiment of the present disclosure further provides a quantization apparatus for a neural network, including: a processing module to perform, in each of a plurality of iteration cycles: determining a current iteration cycle to quantize a target network layer in a first neural network, wherein the quantization coefficients correspond to a plurality of quantization objects; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle; determining quantization losses of the quantization objects in the current iteration cycle based on quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses; and the quantization module is used for determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
In an alternative embodiment, the quantization object includes at least one of: input data, output data and network layer internal parameters of the target network layer.
In an optional embodiment, the quantization object includes a network layer internal parameter of the target network layer; the processing module is configured to, when determining that a current iteration cycle quantizes a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects: responding to the fact that the current iteration cycle is the first iteration cycle, and determining an original quantization coefficient of the network layer internal parameter based on a preset quantization coefficient space; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the network layer internal parameter.
In an alternative embodiment, the quantization object includes: input data of the target network layer and/or output data of the target network layer; the processing module is configured to, when determining that a current iteration cycle quantizes a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects: determining first distribution characteristic information of the input data based on the input data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the input data based on the first distribution characteristic information; and/or determining second distribution characteristic information of the output data based on the output data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the output data based on the second distribution characteristic information; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the input data and/or the original quantization coefficient of the output data.
In an optional embodiment, the quantization object comprises input data, network layer internal parameters and output data of the target network layer; the quantized coefficients include: a first quantized coefficient of the input data and a second quantized coefficient of the network layer internal parameter; the processing module, when performing quantization operation on the multiple quantization objects based on the quantization coefficients to obtain quantization results corresponding to the multiple quantization objects in a current iteration cycle, is configured to: for a first quantization object with the type of input data, performing quantization operation on the first quantization object based on a first quantization coefficient to obtain a first quantization result of the first quantization object in a current iteration cycle; for a second quantization object with the type of network layer internal parameters, performing quantization operation on the second quantization object based on a second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for a third quantization object of which the type is output data, performing quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or determining a third quantization result corresponding to the third quantization object in the current iteration cycle based on the first quantization result and the second quantization result.
In an optional embodiment, when determining quantization losses of the plurality of quantization objects in the current iteration cycle based on quantization results of the plurality of quantization objects in the current iteration cycle, the processing module is configured to: for each quantization object in a plurality of quantization objects, performing inverse quantization operation on a quantization result corresponding to each quantization object based on the quantization coefficient to obtain an inverse quantization result corresponding to each quantization object in a current iteration cycle; and obtaining the quantization loss of each quantization object in the current iteration cycle based on each quantization object and the corresponding inverse quantization result.
In an optional embodiment, when determining the loss weights corresponding to the quantization losses of the plurality of quantization objects respectively based on the quantization losses, the processing module is configured to: carrying out normalization processing on the quantization losses respectively corresponding to the quantization objects belonging to the same type to obtain the loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or respectively processing the quantization objects belonging to the same type based on a preset monotonic function to obtain loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or, carrying out weight prediction processing on quantization objects belonging to the same type by adopting a pre-trained weight prediction neural network to obtain loss weights corresponding to quantization losses of the quantization objects belonging to the same type respectively; or, for each quantization object, determining a target value interval to which the quantization loss of each quantization object belongs in a plurality of value intervals; and determining a loss weight corresponding to the quantization loss of each quantization object based on the weight corresponding to the target value interval.
In an optional embodiment, after determining quantization losses corresponding to the quantization objects in the current iteration cycle based on quantization results corresponding to the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects in the current iteration cycle based on the quantization losses, the processing module is further configured to: determining whether the current iteration cycle meets an iteration stop condition based on the loss weights respectively corresponding to the multiple quantitative objects; entering a next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; when the quantization coefficient determined in the last iteration cycle is determined as the target quantization coefficient, the processing module is configured to: and in response to that the current iteration cycle meets the iteration stop condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.
In an alternative embodiment, the iteration stop condition includes at least one of: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on loss weights respectively corresponding to the plurality of quantization objects; the difference value between the total quantization losses corresponding to the target iteration cycles is smaller than a preset difference value threshold value; wherein the target iteration cycle comprises: the current iteration cycle, and at least one historical iteration cycle that is continuous with the current iteration cycle.
In an optional embodiment, the quantization module, when performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network, is configured to: quantizing the quantization object in the target network layer by using the target quantization coefficient to obtain a target quantization object; generating the second neural network based on the target quantization object.
In a third aspect, this disclosure also provides a computer device, a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions are executed by the processor to perform the steps in the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.
For the description of the effects of the quantization apparatus, the computer device, and the computer-readable storage medium of the neural network, reference is made to the description of the quantization method of the neural network, and details are not repeated here.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 illustrates a flow chart of a quantization method of a neural network provided by an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a quantization apparatus of a neural network provided in an embodiment of the present disclosure;
fig. 3 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
Research shows that the neural network contains more network layer internal parameters (parameters) and processing data (such as input data and output data of each network layer), and the network layer internal parameters and the processing data of the neural network are both in a floating point number system; if the processing task is directly performed by using the floating-point number system, the neural network consumes more computing resources and storage resources in the process of performing the processing task due to more internal parameters of the network layer and more bits occupied by the processing data. In order to reduce the computing resources and storage resources consumed by the neural network and ensure the performance (performance) of the neural network, the network layer internal parameter processing data in the model can be quantized, and the data represented by floating point numbers is converted into the data represented by fixed point numbers with fixed bit positions, so that the bit positions of the network layer internal parameter processing data are reduced. Since the operation of quantizing the data inevitably causes the precision of the data to be reduced, and further affects the precision of the neural network, the quantization parameter during quantization is usually determined by measuring the distribution difference between the quantized neural network and the original neural network. Specifically, the quantization difference of each layer of the network layer between the quantized neural network and the initial neural network can be directly determined, and the quantization losses of the neural network during quantization can be determined by averaging the quantization differences respectively corresponding to the multiple network layers, and then the quantization coefficients can be determined by using the quantization losses. The method easily ignores the influence of partial parameters with larger quantization difference on quantization loss in the neural network, and enhances the influence of partial parameters with smaller quantization difference on quantization loss; such imbalance causes a large decrease in the accuracy of the quantized neural network obtained by quantizing the neural network with the quantization coefficients after determining the quantization coefficients with the quantization loss.
Based on the above research, the present disclosure provides a quantization method for a neural network, in which corresponding quantization weights are determined according to magnitudes of quantization losses corresponding to respective quantization objects, and the quantization losses of a network layer are determined by using the quantization weights, so that degrees of influence of quantization objects with different quantization differences on the quantization losses can be equalized, and thus, after a quantization coefficient is obtained by using the quantization loss, the quantized neural network obtained based on the quantization coefficient has higher accuracy.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
To facilitate understanding of the present embodiment, first, a detailed description is given of a quantization method of a neural network disclosed in an embodiment of the present disclosure, and an execution subject of the quantization method of the neural network provided in the embodiment of the present disclosure is generally a computer device with certain computing power, where the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the quantization method of the neural network may be implemented by a processor calling computer-readable instructions stored in a memory.
The following describes a method for quantifying a neural network provided by an embodiment of the present disclosure.
Referring to fig. 1, a flowchart of a quantization method of a neural network provided in an embodiment of the present disclosure is shown, where the quantization method of the neural network includes the following steps S101 to S102:
s101: the following steps S1011 to S1013 are performed in each of a plurality of iteration cycles, wherein:
s1011: determining a current iteration cycle to quantize a target network layer in a first neural network, wherein the quantization coefficients correspond to a plurality of quantization objects; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles;
s1012: respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle;
s1013: and determining quantization losses of the quantization objects in the current iteration cycle based on the quantization results of the quantization objects in the current iteration cycle, and determining loss weights of the quantization objects based on the quantization losses.
S102: and after the iteration of at least one iteration cycle is completed, determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
The quantization loss determining method and the quantization loss determining device in the embodiment of the disclosure determine quantization coefficients corresponding to a plurality of quantization objects when a target network layer in a first neural network performs quantization, perform quantization operation on the plurality of quantization objects by using the quantization coefficients respectively to obtain quantization results corresponding to the plurality of quantization objects respectively, and determine quantization loss and loss weight corresponding to the quantization loss according to the quantization results correspondingly. Thus, for the quantization coefficient corresponding to each quantization object, the quantization weight determined for the quantization object and the obtained quantization loss may be used to calculate the quantization coefficient corresponding to the next iteration cycle. The corresponding quantization weight is determined according to the quantization loss magnitude corresponding to each quantization object, the quantization loss of the network layer is determined by using the quantization weight, the influence degree of quantization objects with different quantization differences on the quantization loss can be balanced, and after the quantization coefficients are obtained by using the quantization losses, the quantized neural network obtained on the basis of the quantization coefficients has higher precision.
The following describes the details of S101 to S102.
First, S101 of the embodiment of the present disclosure is described in detail by taking the first iteration cycle as an example.
In the above S1011, the first neural network described therein corresponds to different types in different application scenarios. Illustratively, in an application scenario of image processing, the corresponding first Neural network includes, for example, a Convolutional Neural Network (CNN); in an application scenario of natural language processing, the corresponding first Neural Network includes, for example, a Recurrent Neural Network (RNN). By way of example, the following describes a quantization method of a neural network provided by an embodiment of the present disclosure, taking the first neural network as a convolutional neural network as an example.
In a possible case, any neural network to be deployed may be processed by the quantization method provided by the embodiment of the present disclosure, so that the model parameters and processing data in the neural network to be deployed are quantized into data occupying fewer bits, thereby meeting the deployment requirement of a hardware device, for example, deployed in a small device such as an embedded device.
In a specific implementation, when the first neural network is quantized, for example, a target network layer therein may be quantized. The specifically selected target network layer may be determined according to actual conditions, for example, only the convolutional layer and the fully-connected layer with more model parameters in the first neural network are quantized, and then, part or all of the convolutional layer and the fully-connected layer in the first neural network may be used as the target network layer. For another example, the device to be deployed only supports a neural network quantized to a certain number of bits, and all network layers in the first neural network may also be quantized correspondingly as target network layers.
In the case of determining a target network layer in the first neural network, a plurality of quantization objects can be determined accordingly, depending on the target network layer. Specifically, the quantization object includes at least one of: input data, output data and network layer internal parameters of the target network layer.
For the case that the quantization object includes a network layer internal reference of the target network layer, for example, if the target network layer includes a convolution layer, the corresponding quantization object may include, for example, each convolution factor in the convolution kernel; for example, the convolution kernel is a convolution kernel of 3 × 5, and each element therein becomes a convolution factor.
In a specific implementation, in the case that the quantization object includes a network layer internal parameter of a target network layer, when determining quantization coefficients corresponding to a plurality of quantization objects in a current iteration cycle, for example, the following manner may be adopted: responding to the fact that the current iteration cycle is the first iteration cycle, and determining an original quantization coefficient of the network layer internal parameter based on a preset quantization coefficient space; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the network layer internal parameter.
Specifically, since the network layer internal parameters of each network layer in the first neural network are determined before quantization, if a quantized bit (bit) is determined, for each network layer internal parameter, the selectable quantization coefficient may also be correspondingly determined within a range. For example, the range of quantized coefficients may be determined empirically, e.g., for a first convolution factor b in the target network layer in the first neural network, which corresponds to a value of 0.1, then when quantized to 3 bits, the possible quantized coefficients are in the range of (0, 0.1).
For example, when determining the preset quantization coefficient space, a uniform quantization mode or a non-uniform quantization mode may be used. For any kind of quantization object, for example, the first convolution factor in the convolution kernel, taking uniform quantization as an example, a plurality of intervals may be determined according to the size range of the quantization result of the convolution factor after quantization, for example, the size range of the convolution factor is 0 to 1, and the convolution factor is quantized into 5 intervals in an average division manner, where [0,0.2], (0.2,0.4], (0.4,0.6], (0.6,0.8], (0.8,1] in the case of quantizing different convolution factors into 5 different intervals, a corresponding one of the quantization coefficients may be determined to be 0.2 for the 5 different intervals, specifically, in the case of determining the quantization coefficient corresponding to the quantization interval, different values may be selected, and whether the selected value can quantize the quantization object in a certain value range into the plurality of divided intervals is determined in a calculation manner, if the selected quantization coefficient cannot divide the quantization object into the plurality of divided intervals, new quantized coefficients are determined until quantized coefficients meeting the quantization requirements can be determined. .
Specifically, when the quantization coefficient in the uniform quantization mode is determined, the quantization coefficient may be determined by obtaining the distribution of the original data corresponding to the quantization object and mapping the distribution to the target quantization interval. Illustratively, the maximum value and the minimum value in the original data are obtained and are only mapped to an 8-bit quantization interval, and the obtained quotient can be used as a quantization coefficient by means of quotient calculation between the difference value between the maximum value and the minimum value and the power of 8 of 2.
Here, when the quantization coefficient is determined by uniform quantization, the partition of the interval is simple and convenient during uniform quantization, so that the quantization coefficient space can be determined more simply.
In the non-uniform quantization method, for example, a plurality of sections may be determined according to the numerical distribution of the quantization target, specifically, for example, a section with a large number of sections may be divided into a section with a large number of numerical distributions, and a section with a small number of sections may be divided into a section with a small number of sections with a small number of numerical distributions. Taking the convolution factor as an example, the distribution of the convolution factors in values can be determined from the convolution factors of a plurality of convolution operations, for example, more are distributed in the interval of [0,0.2], and the rest are distributed in the intervals of (0.2,0.5], (0.5, 1), since the values distributed in the interval of [0,0.2] are more, the interval of [0,0.2] can be further refined, for example, into [0,0.025], (0.025,0.1], (0.1,0.2], when the corresponding quantization coefficient is determined for each interval, for example, the corresponding quantization coefficient 0.005 is set for the interval [0,0.025], the corresponding quantization coefficient 0.015 is set for the interval (0.025, 0.1), the corresponding quantization coefficient 0.02 is set for the interval (0.2, 0.5) is set for the interval (0.06), 1] set the corresponding quantization factor to 0.1. The method of determining the quantization coefficient in the non-uniform quantization mode may be the same as the uniform quantization, or may be performed by first performing a non-linear transformation (e.g., log) on the original data of the quantization object, then determining the distribution of the original data after the non-linear transformation, and then similarly obtaining the quantization coefficient using the uniform quantization method.
Here, when the quantization coefficient is determined in the non-uniform quantization manner, the distribution of the specific value of the quantization coefficient is considered, so that the quantization coefficient can be determined in the partition and for each partition more in accordance with the actual quantization requirement, and the obtained preset quantization coefficient space is more reasonable.
In a specific implementation, regarding the above-mentioned manners of uniform quantization and non-uniform quantization, for a quantization object, a quantization coefficient determined in a range corresponding to a possible value of the quantization object may constitute a quantization coefficient space for the quantization object, that is, the preset quantization coefficient space described above. Taking the quantization object b as an example, in the quantization coefficient space, according to the section where the quantization object b is located, the quantization coefficient corresponding to the section can be correspondingly determined, and the determined quantization coefficient is used as the original quantization coefficient corresponding to the quantization object b. For example, if the value of the quantization target b is 0.06, the corresponding quantization coefficient may be determined to be 0.2 in the uniform quantization example, and the corresponding quantization coefficient may be determined to be 0.015 in the non-uniform quantization example.
Similarly, for other network layer internal parameters in the target network layer, the original quantized coefficients of the network layer internal parameters can also be determined in a similar manner. In one possible case, for a first iteration cycle, an original quantized coefficient determined from the quantized coefficient space may be directly determined as a quantized coefficient corresponding to the first iteration cycle.
In another embodiment of the present disclosure, the quantization object may further include input data and/or output data of the target network layer.
In a specific implementation, in the case that the quantization object includes input data and/or output data of a target network layer, when determining quantization coefficients corresponding to a plurality of quantization objects in quantizing a target network layer in the first neural network, for example, the following manner may be adopted: determining first distribution characteristic information of the input data based on the input data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the input data based on the first distribution characteristic information; and/or determining second distribution characteristic information of the output data based on the output data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the output data based on the second distribution characteristic information; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the input data and/or the original quantization coefficient of the output data.
For any one of the target network layers, the corresponding input data may include, for example, an original image or a feature map (feature map) output by a previous network layer of the network layer; the corresponding output data is, for example, another characteristic diagram obtained by processing the input data. Since the first neural network is different in the plurality of images input to the first neural network, taking the processing of the image as an example, the output data obtained through the same network layer of the first neural network is different for each image, and the input data is different for the same network layer. Because the input data may be different due to a difference between the original image and a feature map output by a previous network layer, when determining the quantization coefficient corresponding to the input data, first distribution feature information of the input data may be determined first, where the first distribution feature information includes, for example, a value range and a value distribution of the input data, and a plurality of intervals corresponding to the value range of the input data and a quantization coefficient corresponding to each interval may be determined by using the first distribution feature information, or a manner similar to the determination of the quantization coefficient space of the parameter in the network layer described above may be correspondingly employed to determine a plurality of intervals corresponding to the value range of the input data and a quantization coefficient corresponding to each interval, so as to determine a corresponding original quantization coefficient for the input data having a specific value.
In addition, if the first network layer of the first neural network is used as the target network layer, for example, the first neural network may be used to perform a first processing on the input data of the target network layer (i.e., the input data of the entire first neural network) for the output data, so as to obtain the output data of the target network layer. For each network layer of the first neural network, when determining first distribution characteristic information corresponding to input data and second distribution characteristic information corresponding to output data, the input data and the output data of the network layer are determined by taking the network layer as a reference, and further the corresponding distribution characteristic information is determined. Similar to the above input data, when determining the corresponding quantization coefficient for the output data of the target network layer, second distribution characteristic information of the output data may be determined first, and the corresponding original quantization coefficient may be determined for the output data by using the second distribution characteristic information.
In another embodiment of the present disclosure, when the quantization object is directly quantized by using the quantization coefficient, the value of the quantization object with a value of 0 is correspondingly changed, and in fact, the value of the quantization object with a value of 0 after quantization should still be 0. Therefore, while the corresponding quantization coefficient is determined for the quantization object, a zero value (zero) may be determined accordingly and used as a part of the quantization coefficient. The zero value can be used to process the quantization object with the value of 0, so that the value after quantization is kept unchanged. That is, corresponding to any one quantization object, a corresponding one quantization coefficient and one zero value may be determined for it, respectively.
For the above-described plurality of quantization objects, it is exemplary that the first quantization coefficient s may be determined for the input data1And zero value z1(ii) a Determining a second quantized coefficient s for a network layer internal parameter2And zero value z2(ii) a And determining a third quantized coefficient s for the output data3And zero value z3。
In the above S1012, when the corresponding quantization coefficient is determined for each quantization object, the quantization operation may be performed on each of the plurality of quantization objects by using the quantization coefficient to obtain the quantization result corresponding to each of the plurality of quantization objects.
In a specific implementation, the quantization object includes an inputIn the case of data, the quantization coefficient S corresponding thereto can be determined in accordance with the above S1011And zero value z1I.e. the first quantized coefficients. When quantization processing is performed by using a first quantization object corresponding to the first quantization coefficient (i.e., a quantization object whose kind is input data), a first quantization result of the first quantization object can be obtained accordingly. Illustratively, the obtained first quantization result satisfies, for example, the following formula (1):
Figure BDA0003380972950000121
wherein x islInput data representing the l-th layer, i.e., a first quantization object;
Figure BDA0003380972950000122
representing a first quantization object xlIn order to distinguish between data before and after quantization, the result obtained after quantization is denoted by a subscript "int" to indicate that an object to be quantized is quantized into a small number of bits. b represents a quantized bit, and can be set according to actual conditions. round (·) denotes a round operation, with the round operation, and a zero value z1The first quantized object may be processed and mapped to [0, 2] using clamp (-)b-1]In the interval, completing quantization to obtain a first quantization result
Figure BDA0003380972950000123
Similarly, in the case where the quantization object includes the network layer internal parameter, the quantization coefficient S corresponding thereto can be determined according to the above S1012And zero value z2I.e. the second quantized coefficients. When the quantization processing is performed by using the second quantization object corresponding to the second quantization coefficient (i.e., the quantization object whose kind is the network layer internal parameter), the second quantization result of the second quantization object can be obtained accordingly. Illustratively, the obtained second quantization result satisfies, for example, the following formula (2):
Figure BDA0003380972950000124
wherein, wlRepresenting a network layer internal parameter, namely a second quantization object;
Figure BDA0003380972950000125
representing a second quantization object wlThe results obtained after quantification. The remaining items have meanings similar to those expressed in the above (1), and are not repeated herein.
In contrast, in the case where the quantization target includes output data, since the output data of the previous network layer is input data of the next network layer for the adjacent network layer, when the output data is represented, it can be represented as x, for examplel+1I.e. the third quantization object.
In determining a third quantization object xl+1For example, but not limited to, the following (a1) or the following (a2) may be adopted for the corresponding third quantization result:
(A1) the method comprises the following steps And performing quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object.
Similarly to the above-described first quantization result for determining the first quantization object and the second quantization result for determining the second quantization object, in the case where the quantization objects include output data, the corresponding quantization coefficient S may be determined according to the above-described S1013And zero value z3I.e. the third quantized coefficients. When quantization processing is performed by using a third quantization object corresponding to the third quantization coefficient (i.e., the quantization object whose type is the output data), a third quantization result of the third quantization object can be obtained accordingly. Illustratively, the obtained third quantization result satisfies the following formula (3), for example:
Figure BDA0003380972950000131
wherein, wl+1Representing the output data, i.e. the third quantization object;
Figure BDA0003380972950000132
representing a third quantization object wl+1The results obtained after quantification. The remaining terms have similar meanings to those expressed by the above formulas (1) and (2), and are not repeated herein.
(A2) The method comprises the following steps And determining a third quantization result corresponding to the third quantization object based on the first quantization result and the second quantization result.
Due to the first quantization object xlSecond quantization object wlAnd a third quantization object xl+1Satisfies the following formula (4):
xl+1＝wlxl (4)
thus, a third quantization result corresponding to the third quantization object
Figure BDA0003380972950000133
First quantization result with first quantization object
Figure BDA0003380972950000134
And a second quantization result corresponding to the second quantization object
Figure BDA0003380972950000135
May satisfy the following formula (5):
Figure BDA0003380972950000136
that is, the third quantization result of the third quantization object may be directly determined using the first quantization result of the first quantization object and the second quantization result of the second quantization object.
According to the above-described manner of determining the quantization results, the quantization results corresponding to the quantization objects corresponding to the target network layer can be determined accordingly.
In the case of determining the quantization results corresponding to the plurality of quantization objects in S1013, the corresponding quantization loss may be determined for each of the plurality of quantization objects. Specifically, for example, for each quantization object in a plurality of quantization objects, an inverse quantization operation may be performed on a quantization result corresponding to each quantization object based on the quantization coefficient, so as to obtain an inverse quantization result corresponding to each quantization object; and obtaining the quantization loss of each quantization object based on each quantization object and the corresponding inverse quantization result.
Next, a description will be given of a procedure for determining a quantization loss for each of the types of input data, network layer internal parameters, and quantization objects of output data.
For a quantization target of which the type is input data, using a first quantization coefficient corresponding to the input data: quantized coefficient s1And zero value z1The first quantization result can be obtained
Figure BDA0003380972950000141
And performing inverse quantization operation to obtain a first inverse quantization result. Wherein the first dequantization result satisfies, for example, the following formula (6):
Figure BDA0003380972950000142
wherein the content of the first and second substances,
Figure BDA0003380972950000143
representing a first dequantization result.
In the case that the selection of the first quantization coefficient is accurate, the difference between the first inverse quantization result obtained by performing inverse quantization operation on the first quantization result and the first quantization object should be small, so as to ensure that the loss of the first quantization object in precision is small. Therefore, the quantization loss of the quantization object can be measured by using the following formula (7), thereby reflecting whether the selection of the first quantization coefficient is accurate or not:
Figure BDA0003380972950000144
in fact, for the target network layer, there are a plurality of input data corresponding to the target network layer, for example, N input data are included. For the ith input data of the N input data, specifically in the calculation, the following formula (8) may be used to determine the corresponding quantization loss, for example:
Figure BDA0003380972950000145
where res (·) represents a difference function. Obtained
Figure BDA0003380972950000146
Namely, the first quantization loss corresponding to the ith first quantization object.
Aiming at the quantization object with the type of network layer internal parameter, utilizing a second quantization coefficient corresponding to the network layer internal parameter: quantized coefficient s2And zero value z2The second quantization result can be compared
Figure BDA0003380972950000147
And performing inverse quantization operation to obtain a second inverse quantization result. Wherein the second dequantization result satisfies, for example, the following formula (9):
Figure BDA0003380972950000148
wherein the content of the first and second substances,
Figure BDA0003380972950000149
representing a second dequantization result.
Under the condition that the selection of the second quantization coefficient is accurate, the difference between the second inverse quantization result obtained by performing inverse quantization operation on the second quantization result and the second quantization object is small, so that the loss of the second quantization object in precision is small. Therefore, the quantization loss of the quantization object can be measured by using the following formula (10), thereby reflecting whether the selection of the second quantization coefficient is accurate or not:
Figure BDA00033809729500001410
in fact, for the target network layer, there are a plurality of network layers corresponding to the target network layer, for example, J network layers are included. For the jth network layer internal parameter of the J network layer internal parameters, specifically in the calculation, the corresponding quantization loss may be determined by using the following formula (11), for example:
Figure BDA0003380972950000151
wherein, obtained
Figure BDA0003380972950000152
I.e. the second quantization loss corresponding to the jth second quantization object.
First, a description will be given of a mode of determining a third dequantization result of a third quantization target whose type is output data. Specifically, in determining the third dequantization result, for example, the following manner (B1) or (B2) may be adopted, but not limited to:
(B1) the method comprises the following steps And utilizing a third quantized coefficient corresponding to the output data: quantized coefficient s3And zero value z3The third quantization result can be obtained
Figure BDA0003380972950000153
And performing inverse quantization operation to obtain a third inverse quantization result. Wherein the third dequantization result, for example, satisfies the following formula (12):
Figure BDA0003380972950000154
wherein the content of the first and second substances,
Figure BDA0003380972950000155
representing a third inverse quantization result.
(B2) The method comprises the following steps Using the first dequantized result
Figure BDA0003380972950000156
(i.e., the inverse quantization result of the input data), and a second inverse quantization result
Figure BDA0003380972950000157
(i.e., the dequantization result of the network layer internal parameter), determining a third dequantization result
Figure BDA0003380972950000158
Specifically, a first dequantization result corresponding to the first quantization object may be obtained according to the above equation (6)
Figure BDA0003380972950000159
In addition, a second inverse quantization result corresponding to the second quantization object can be obtained according to the above formula (9)
Figure BDA00033809729500001510
According to the above formula (4) and formula (5), for example, the first dequantization result can be determined accordingly
Figure BDA00033809729500001511
Second dequantized result
Figure BDA00033809729500001512
And a third dequantized result
Figure BDA00033809729500001513
Satisfies the following formula (13):
Figure BDA00033809729500001514
similarly, in determining the third quantization result
Figure BDA00033809729500001515
Thereafter, a quantization loss corresponding to the third quantization result may be determined. For the third quantization objectSpecifically, if the third quantization target includes H, for the H-th output data of the H output data, the following (14) may be used to determine the corresponding quantization loss, specifically in the calculation:
Figure BDA00033809729500001516
wherein the content of the first and second substances,
Figure BDA00033809729500001517
representing the quantization loss for h output data.
In the case where quantization losses corresponding to the plurality of quantization objects, respectively, are determined, loss weights corresponding to the plurality of quantization objects, respectively, may be determined based on the quantization losses.
Specifically, since the quantization loss corresponding to each quantization object is different, or a case that the quantization loss corresponding to a certain or some quantization objects is large may occur, in order to reflect the difference of different quantization losses, corresponding loss weights may be determined for different quantization losses. In particular, when determining the loss weight, the following different approaches may be adopted, but not limited to:
A. and carrying out normalization processing on the quantization losses respectively corresponding to the quantization objects belonging to the same type to obtain the loss weights respectively corresponding to the quantization objects belonging to the same type.
Specifically, for quantization objects of the same type, since the values of the quantization objects are different and the corresponding quantization coefficients are also different, the loss weight corresponding to each quantization object is also different. When determining the loss weights for different quantization losses, the distribution of the weight losses in the [0,1] space can be determined in a normalized manner. And determining a smaller loss weight for smaller quantization losses and a larger weight for larger quantization losses according to the distribution. Therefore, the quantization object with larger quantization loss can be highlighted by using larger loss weight, and the quantization coefficient corresponding to the quantization object is emphasized to be adjusted in the next iteration cycle, so that the quantization is more accurate.
B. And respectively processing the quantization objects belonging to the same type based on a preset monotonic function to obtain loss weights respectively corresponding to the quantization objects belonging to the same type.
Similarly to the above-mentioned manner in a, the loss weight can be correspondingly determined according to different quantization losses by using a monotonic function. Due to e in a monotonic functionxThe corresponding loss weights can be correspondingly determined for the quantization losses with different sizes.
For example, taking the first quantization object as an example, when determining the corresponding loss weight according to the first quantization loss corresponding to the first quantization object, the following formula (15) may be adopted, for example:
Figure BDA0003380972950000161
wherein the content of the first and second substances,
Figure BDA0003380972950000162
and a loss weight corresponding to a first quantization loss of an ith first quantization object among the N first quantization objects is represented. Due to exIs a monotone increasing function, so that if the first quantization loss of the ith first quantization object is larger in value, the loss weight calculated correspondingly
Figure BDA0003380972950000163
The value of (A) is also large; if the first quantization loss of the ith first quantization object is smaller, the loss weight calculated correspondingly
Figure BDA0003380972950000164
The value of (a) is also large.
C. And performing weight prediction processing on the quantization objects belonging to the same type by adopting a pre-trained weight prediction neural network to obtain loss weights respectively corresponding to the quantization objects belonging to the same type.
When the weight prediction neural network is trained in advance, taking a quantization target belonging to input data as an example, for example, a sample data pair may be determined by using quantization coefficients corresponding to input data in the quantized neural network and loss weights corresponding to the quantization coefficients, respectively, and the weight prediction neural network may be trained by using the sample data pair to obtain a trained weight prediction neural network. And processing the first quantization loss corresponding to the input data in the first neural network by using the weight prediction neural network, and determining the loss weight corresponding to each first quantization loss.
D. Determining a target value interval of each quantization object in a plurality of value intervals for each quantization object; and determining the loss weight corresponding to each quantization object based on the weight corresponding to the target value interval.
Specifically, since the purpose of setting the loss weight is to emphasize a part of items with large quantization losses corresponding to the adjustment when the quantization coefficient is adjusted, the weight may also be set by setting a plurality of value intervals, specifically, the plurality of value intervals may be determined according to a possible value range of the quantization losses of the quantization object, in response to that the quantization losses fall into any one of the value intervals, the value interval may be used as a target value interval, and the weight corresponding to the target value interval may be used as the loss weight corresponding to the quantization losses of the quantization object.
For example, for a quantization object belonging to input data, for example, first quantization losses corresponding to N input data may be obtained. From the distribution of the first quantization losses, a distribution difference between the resulting quantization losses for the first quantization object may be determined. For example, for N input data, the corresponding quantization losses are distributed centrally over p intervals a1、a2、……、ap(p is a positive integer), a corresponding quantization loss may be determined for each bin, e.g., the qth bin aqDetermining a corresponding quantization loss rq. After the ith first quantization loss
Figure BDA0003380972950000171
When a corresponding quantization loss is determined, it can be determinedQuantization loss
Figure BDA0003380972950000172
Falling into intervals, e.g. the q-th interval aqAnd the interval a is dividedqCorresponding quantization loss rqAs the ith first quantization loss
Figure BDA0003380972950000173
Corresponding quantization loss.
For example, the loss weight of the quantization loss corresponding to the input data can be expressed as
Figure BDA0003380972950000174
The loss weight of the quantization loss referred to in the network layer may be expressed, for example, as
Figure BDA0003380972950000175
The loss weight of the quantization loss corresponding to the output data can be expressed as
Figure BDA0003380972950000176
After the quantization losses corresponding to the quantization objects in the first iteration cycle and the loss weights corresponding to the quantization losses are determined, the quantization losses are processed by using the loss weights corresponding to the quantization losses for any quantization object, so that the quantization losses determined by using the quantization object and the corresponding loss weights can be obtained correspondingly, and the total quantization loss can be determined.
For example, the total quantization loss corresponding to the input data can be represented by the following formula (16):
Figure BDA0003380972950000177
wherein the content of the first and second substances,
Figure BDA0003380972950000178
representing the corresponding total quantization loss of the input data. In computing input data pairsWhen the loss is to be summed, the quantization loss corresponding to the ith input data is multiplied by the corresponding loss weight.
Similarly, for network layer internal parameters, the corresponding total quantization loss can be expressed by the following formula (17), for example:
Figure BDA0003380972950000179
and, for the output data, the corresponding total quantization loss can be expressed, for example, by the following equation (18):
Figure BDA0003380972950000181
when the total quantization loss is determined, in a possible case, because a total quantization loss is determined for each quantization object in the target network layer, on one hand, because the number of the quantization objects is large, the obtained total quantization loss is correspondingly large, so that the consumed calculation power is large, and the efficiency is low; on the other hand, since for a quantization object, its corresponding quantization loss may determine a corresponding quantization weight, and it may be evaluated whether the quantization result of the quantization object is superior among quantization objects belonging to the same type, particularly in that whether the value of the loss weight determined for the quantization object is small. Therefore, when the total quantization loss is determined, the total quantization loss determined by the plurality of quantization objects in one network layer is determined, for example, with one network layer in the target network layer as a minimum unit. Specifically, the total quantization loss is determined based on loss weights respectively corresponding to a plurality of quantization objects, and the corresponding quantization loss is weighted by the loss weights, for example. In this way, the number of identified total quantization losses is small, and the determination is easy, and the obtained total quantization losses can be reflected well to the quantization result of the quantization target in the layer network layer.
For example, for one of the target network layers, a first total loss corresponding to the first quantization object (input data), a second total loss corresponding to the second quantization object (network layer internal parameter), and a third total loss corresponding to the third quantization object (output data) may be obtained, for example. In this way, since quantization coefficients corresponding to quantization objects belonging to the same type are relatively similar for different quantization objects, the quantization coefficients can be determined for the quantization objects of different types one by independently determining the total loss corresponding to the quantization objects of different types.
In a possible case, if the quantization loss corresponding to the quantization object is small, the quantization weight determined correspondingly is also small, and the total quantization loss obtained is also correspondingly small, then the quantization coefficient set for the quantization object can be considered to be reasonable and can be selected. In another possible case, if the quantization loss corresponding to the quantization object is large, the quantization weight determined accordingly is also large, and the total quantization loss obtained is also large accordingly, in this case, it may be determined that the quantization coefficient set for the quantization object is not reasonable, which may cause the accuracy of the obtained second neural network to be low, and therefore the determined quantization coefficient cannot be selected, and further adjustment needs to be performed on the set quantization coefficient.
In a specific implementation, for example, whether the quantization coefficient set in the current iteration cycle is reasonable or not may be determined by setting an iteration stop condition, and may be used as the target quantization coefficient. Specifically, when determining the target quantized coefficient using the iteration stop condition, the following manner may be adopted: determining whether the current iteration cycle meets an iteration stop condition based on the loss weights respectively corresponding to the multiple quantitative objects; entering a next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; the determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient includes: and in response to that the current iteration cycle meets the iteration stop condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.
The iteration stop condition may include, for example: the total quantization loss is less than a preset loss threshold.
Specifically, the loss threshold may be set to Ls, for example. Specifically, because the input data, the network layer internal parameters, and the output data have different requirements for quantization accuracy and different distributions of values, different loss thresholds can be determined for different types of quantization objects. For example, a loss threshold is determined for the input data
Figure BDA0003380972950000191
Determining loss thresholds for network layer internal parameters
Figure BDA0003380972950000192
And determining a loss threshold for the output data
Figure BDA0003380972950000193
For any type of quantization object, if the determined total quantization loss is greater than or equal to the corresponding loss threshold, it is considered that the quantization coefficient determined in the current iteration cycle is not accurate, and the quantization coefficient used in the next iteration cycle can be determined according to the difference between the determined total quantization loss and the corresponding loss threshold. And if the determined total quantization loss is less than the loss threshold, determining that the quantization coefficient set in the current iteration period can meet the actual quantization requirement, and taking the quantization coefficient set in the current iteration period as a target quantization coefficient.
For example, in the first iteration cycle, for the input data, the network layer internal parameters and the output data, there is no case that the total quantization loss corresponding to any type of quantization object is less than the corresponding loss threshold, and it is determined to enter the processing of the second iteration cycle.
And when the processing of the second iteration cycle is carried out, changing the current iteration cycle into the second iteration cycle. For example, the quantization coefficient corresponding to each quantization object in each type of quantization objects in the current iteration cycle may be determined according to the processing result of the previous iteration cycle corresponding to the current iteration cycle, that is, the total quantization loss corresponding to each quantization object of different types. For example, for input data, if the total quantization loss corresponding to the input data exceeds the corresponding loss threshold and the exceeded value is large, when the quantization coefficient of the first iteration cycle is used to determine the quantization coefficient corresponding to the current iteration cycle, the quantization coefficient of the first iteration cycle may be adjusted to a large extent, and the adjusted quantization coefficient is used as the quantization coefficient corresponding to the current iteration cycle. If the exceeded value is smaller, when the quantization coefficient of the first iteration period is adjusted, the adjustment with a smaller amplitude can be performed, so that the adjusted quantization coefficient is used as the quantization coefficient corresponding to the second iteration period.
In the second iteration cycle, in the case of determining the quantization coefficients corresponding to the plurality of quantization objects, in a manner similar to the first iteration cycle described above, the quantization results corresponding to the plurality of quantization objects in the second iteration cycle may also be obtained, and further, the quantization loss and the quantization weight may be obtained. In this way, it can also be determined whether to proceed with the processing of the next iteration cycle in a corresponding manner by determining the total quantization loss. And if the processing process of the next iteration cycle is not carried out any more, taking the current iteration cycle as the last iteration cycle, and correspondingly taking the quantization coefficient corresponding to the current iteration cycle as the target quantization coefficient.
In another embodiment of the present disclosure, since the loss threshold is artificially initially set, it may occur that the set value is too small, and even if the total quantization loss reaches a lower value through the processing process of multiple iteration cycles, the iteration stop condition that the total quantization loss is smaller than the preset loss threshold cannot be satisfied; or the loss threshold is set to be too large, so that the progress of the second neural network obtained after the first neural network is quantized by the determined target quantization coefficient is low. Therefore, the iteration stop condition may also be set as: the difference value between the total quantization losses corresponding to the target iteration cycles is smaller than a preset difference value threshold value; wherein the target iteration cycle comprises: the current iteration cycle, and at least one historical iteration cycle that is continuous with the current iteration cycle.
In this case, if the total quantization loss corresponding to the current iteration cycle and the continuous at least one historical iteration cycle fluctuates around a value after a plurality of iterations, it may be considered that the quantization coefficient is adjusted by the next iteration cycle, the possible adjustment space is relatively small, and the quantization loss of the quantization object is not greatly affected. Therefore, the iterative process may be stopped, and the quantization coefficient corresponding to the last iteration cycle may be used as the target quantization coefficient.
In another embodiment of the present disclosure, specifically when the computer device is used to determine the target quantized coefficient, the above iterative process may also be performed by using the following formula (19):
Figure BDA0003380972950000201
wherein the content of the first and second substances,
Figure BDA0003380972950000202
representing quantized coefficient s included in first quantized coefficient1And zero value z1In which the total quantization loss is determined
Figure BDA0003380972950000203
The smallest quantized coefficients and the zero value. In a similar manner to that described above,
Figure BDA0003380972950000204
representing quantized coefficient s included in the second quantized coefficient2And zero value z2In which the total quantization loss is determined
Figure BDA0003380972950000205
The smallest quantized coefficients and the zero value. Based on the network connection relation, the output of the current network layer is the input of the next network layer, so the third quantized coefficient corresponding to the output data is related to the first quantized coefficient and the second quantized coefficient, and therefore the third quantized coefficient is processedIn line search, corresponding joint search space and
Figure BDA0003380972950000206
and
Figure BDA0003380972950000207
related, is represented as
Figure BDA0003380972950000208
Alternatively, specifically, when the target quantized coefficient is determined by using a computer device, for example, the iterative process may be performed by using the following formula (20):
Figure BDA0003380972950000209
wherein the content of the first and second substances,
Figure BDA00033809729500002010
representing quantized coefficient s included in the third quantized coefficient3And zero value z3In which the total quantization loss is determined
Figure BDA00033809729500002011
The smallest quantized coefficients and the zero value. In a possible case, if any one of the operations of quantization and inverse quantization of the third quantization object is performed by using the third quantization coefficient in the process of S101, a search space may be correspondingly determined for the third quantization coefficient to determine a specific value of the corresponding third quantization coefficient when the total loss corresponding to the third quantization object is minimum.
For the above S102: after the quantization coefficient corresponding to the last iteration cycle is determined and is used as a target quantization coefficient, the target quantization coefficient may be used to perform quantization processing on the first neural network to generate a second neural network.
In a specific implementation, for example, the following may be used: quantizing the quantization object in the target network layer by using the target quantization coefficient to obtain a target quantization object; generating the second neural network based on the target quantization object.
Specifically, in the case where the target quantization coefficients corresponding to the quantization objects of the target network layer are determined in the last iteration cycle, the determined target quantization coefficients may be determined and used accordingly, and the quantization loss of the quantization objects in the target network layer may be made smaller, so that the quantized quantization objects after quantization are less accurate than the quantized quantization objects before quantization, but the reaction is less numerically different. Therefore, by using the target quantization coefficient, the quantization object in the first neural network can be correspondingly processed to obtain the target quantization object. Here, although the accuracy of the target quantization target obtained is lowered, the difference in the numerical value from that before quantization is not large, and the number of bits occupied is small, so that the calculation effort required for calculation is small and the memory space is small. And replacing the quantization object at the corresponding position in the first neural network by using the target quantization object, so as to obtain a second neural network.
In another embodiment of the present disclosure, the obtained second neural network may be determined to be applied in the determined application scenario according to different categories of the corresponding first neural network in the corresponding application scenario. For example, when it is determined that the first neural network includes a convolutional neural network and is applied to an application scenario of image processing, the second neural network processed by using the embodiment of the present disclosure may be used for image classification, object recognition, visual positioning, and the like in image processing.
In implementation, the image data to be processed may be input into the neural network generated by quantization using the neural network quantization method disclosed in the present application, and the image data to be processed may be processed to obtain a processing result.
For example, in the case where the second neural network is applied to object recognition, when the second neural network performs image processing on a plurality of acquired images, since the quantization object in the second neural network requires less computation power and less storage space during computation, the efficiency when performing image processing on a plurality of images is reduced compared to the first neural network before quantizing the quantization object; in addition, the second neural network is applicable to various application fields as desired because the accuracy of the object to be quantized is lowered, but the object recognition result of recognition obtained by processing the image data using the second neural network can satisfy the accuracy condition of recognition in terms of accuracy.
In addition, the second neural network requires less computing power and less storage space in practical application, and therefore, the second neural network can be suitable for being deployed on small processing equipment, such as mobile equipment such as vehicles and mobile phones. Therefore, the higher limitation and requirement of the first neural network on computing power and storage space during deployment can be correspondingly eliminated, and the method is suitable for more application scenes.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, the embodiment of the present disclosure further provides a quantization apparatus of a neural network corresponding to the quantization method of the neural network, and since the principle of the apparatus in the embodiment of the present disclosure for solving the problem is similar to the quantization method of the neural network described above in the embodiment of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not repeated.
Referring to fig. 2, there is shown a schematic diagram of a quantization apparatus of a neural network according to an embodiment of the present disclosure, the apparatus includes: a processing module 21, and a quantization module 22; wherein the content of the first and second substances,
a processing module 21 configured to perform, in each iteration cycle of a plurality of iteration cycles: determining quantization coefficients corresponding to a plurality of quantization objects when a target network layer in a first neural network is quantized; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle; determining quantization losses of the quantization objects in the current iteration cycle based on quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses;
and a quantization module 22, configured to determine a quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and perform quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
In an alternative embodiment, the quantization object includes at least one of: input data, output data and network layer internal parameters of the target network layer.
In an optional embodiment, the quantization object includes a network layer internal parameter of the target network layer; when determining that the current iteration cycle quantizes the target network layer in the first neural network, and the quantization coefficients corresponding to the multiple quantization objects are used to: responding to the fact that the current iteration cycle is the first iteration cycle, and determining an original quantization coefficient of the network layer internal parameter based on a preset quantization coefficient space; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the network layer internal parameter.
In an alternative embodiment, the quantization object includes: input data of the target network layer and/or output data of the target network layer; when determining that the current iteration cycle quantizes the target network layer in the first neural network, and the quantization coefficients corresponding to the multiple quantization objects are used to: determining first distribution characteristic information of the input data based on the input data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the input data based on the first distribution characteristic information; and/or determining second distribution characteristic information of the output data based on the output data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the output data based on the second distribution characteristic information; and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the input data and/or the original quantization coefficient of the output data.
In an optional embodiment, the quantization object comprises input data, network layer internal parameters and output data of the target network layer; the quantized coefficients include: a first quantized coefficient of the input data and a second quantized coefficient of the network layer internal parameter; when the quantization operation is performed on the multiple quantization objects based on the quantization coefficients, respectively, to obtain quantization results corresponding to the multiple quantization objects in a current iteration cycle, the processing module 21 is configured to: for a first quantization object with the type of input data, performing quantization operation on the first quantization object based on a first quantization coefficient to obtain a first quantization result of the first quantization object in a current iteration cycle; for a second quantization object with the type of network layer internal parameters, performing quantization operation on the second quantization object based on a second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for a third quantization object of which the type is output data, performing quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or determining a third quantization result corresponding to the third quantization object in the current iteration cycle based on the first quantization result and the second quantization result.
In an optional embodiment, when determining quantization losses of the quantization objects corresponding to the current iteration cycle based on quantization results of the quantization objects corresponding to the current iteration cycle, the processing module 21 is configured to: for each quantization object in a plurality of quantization objects, performing inverse quantization operation on a quantization result corresponding to each quantization object in a current iteration cycle based on the quantization coefficient to obtain an inverse quantization result corresponding to each quantization object; and obtaining the quantization loss of each quantization object in the current iteration cycle based on each quantization object and the corresponding inverse quantization result.
In an optional embodiment, the processing module 21, when determining the loss weights corresponding to the quantization losses of the multiple quantization objects respectively based on the quantization losses, is configured to: carrying out normalization processing on the quantization losses respectively corresponding to the quantization objects belonging to the same type to obtain the loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or respectively processing the quantization objects belonging to the same type based on a preset monotonic function to obtain loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type; or, carrying out weight prediction processing on quantization objects belonging to the same type by adopting a pre-trained weight prediction neural network to obtain loss weights corresponding to quantization losses of the quantization objects belonging to the same type respectively; or, for each quantization object, determining a target value interval to which the quantization loss of each quantization object belongs in a plurality of value intervals; and determining a loss weight corresponding to the quantization loss of each quantization object based on the weight corresponding to the target value interval.
In an optional embodiment, the processing module 21, after determining quantization losses corresponding to the quantization objects in the current iteration cycle based on the quantization results corresponding to the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects in the current iteration cycle based on the quantization losses, is further configured to: determining whether the current iteration cycle meets an iteration stop condition based on the loss weights respectively corresponding to the multiple quantitative objects; entering a next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; when the quantized coefficient determined in the last iteration cycle is determined as the target quantized coefficient, the processing module 21 is configured to: and in response to that the current iteration cycle meets the iteration stop condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.
In an alternative embodiment, the iteration stop condition includes at least one of: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on loss weights respectively corresponding to the plurality of quantization objects; the difference value between the total quantization losses corresponding to the target iteration cycles is smaller than a preset difference value threshold value; wherein the target iteration cycle comprises: the current iteration cycle, and at least one historical iteration cycle that is continuous with the current iteration cycle.
In an alternative embodiment, the quantization module 22, when performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network, is configured to: quantizing the quantization object in the target network layer by using the target quantization coefficient to obtain a target quantization object; generating the second neural network based on the target quantization object.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
An embodiment of the present disclosure further provides a computer device, as shown in fig. 3, which is a schematic structural diagram of the computer device provided in the embodiment of the present disclosure, and the computer device includes:
a processor 10 and a memory 20; the memory 20 stores machine-readable instructions executable by the processor 10, the processor 10 being configured to execute the machine-readable instructions stored in the memory 20, the processor 10 performing the following steps when the machine-readable instructions are executed by the processor 10:
performing in each iteration cycle of a plurality of iteration cycles: determining quantization coefficients corresponding to a plurality of quantization objects for quantizing a target network layer in a first neural network; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle; determining quantization losses of the quantization objects in the current iteration cycle based on quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses; and determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
The storage 20 includes a memory 210 and an external storage 220; the memory 210 is also referred to as an internal memory, and temporarily stores operation data in the processor 10 and data exchanged with the external memory 220 such as a hard disk, and the processor 10 exchanges data with the external memory 220 through the memory 210.
For the specific execution process of the above instruction, reference may be made to the steps of the quantization method of the neural network described in the embodiments of the present disclosure, and details are not described here.
The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the quantization method of a neural network described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the neural network quantization method in the foregoing method embodiments, which may be referred to specifically for the foregoing method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims (13)

1. A method of quantifying a neural network, comprising: performing in each iteration cycle of a plurality of iteration cycles:
determining a current iteration cycle to quantize a target network layer in a first neural network, wherein the quantization coefficients correspond to a plurality of quantization objects; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles;
respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle;
determining quantization losses of the quantization objects in the current iteration cycle based on quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses;
and determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
2. The quantization method of claim 1, wherein the quantization object comprises at least one of: input data, output data and network layer internal parameters of the target network layer.
3. The quantization method according to claim 1 or 2, wherein the quantization object includes a network layer internal parameter of the target network layer;
the determining a current iteration cycle to quantize a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects include:
responding to the fact that the current iteration cycle is the first iteration cycle, and determining an original quantization coefficient of the network layer internal parameter based on a preset quantization coefficient space;
and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the network layer internal parameter.
4. The quantization method according to any one of claims 1-3, wherein the quantization object comprises: input data of the target network layer and/or output data of the target network layer;
the determining a current iteration cycle to quantize a target network layer in a first neural network, and the quantization coefficients corresponding to a plurality of quantization objects include:
determining first distribution characteristic information of the input data based on the input data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the input data based on the first distribution characteristic information; and/or determining second distribution characteristic information of the output data based on the output data in response to the current iteration cycle being the first iteration cycle; determining an original quantization coefficient of the output data based on the second distribution characteristic information;
and determining the quantization coefficient of the current iteration cycle based on the original quantization coefficient of the input data and/or the original quantization coefficient of the output data.
5. The quantization method according to any one of claims 1 to 4, wherein the quantization object includes input data, network layer internal parameters, and output data of the target network layer;
the quantized coefficients include: a first quantized coefficient of the input data and a second quantized coefficient of the network layer internal parameter;
the performing quantization operations on the multiple quantization objects based on the quantization coefficients to obtain quantization results of the multiple quantization objects in a current iteration cycle respectively includes:
for a first quantization object with the type of input data, performing quantization operation on the first quantization object based on a first quantization coefficient to obtain a first quantization result of the first quantization object in a current iteration cycle;
for a second quantization object with the type of network layer internal parameters, performing quantization operation on the second quantization object based on a second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle;
for a third quantization object of which the type is output data, performing quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or determining a third quantization result corresponding to the third quantization object in the current iteration cycle based on the first quantization result and the second quantization result.
6. The quantization method according to any one of claims 1 to 5, wherein the determining, based on the quantization results of the plurality of quantization objects respectively corresponding to the current iteration cycle, the quantization losses of the plurality of quantization objects respectively corresponding to the current iteration cycle comprises:
for each quantization object in a plurality of quantization objects, performing inverse quantization operation on a quantization result corresponding to each quantization object in a current iteration cycle based on the quantization coefficient to obtain an inverse quantization result corresponding to each quantization object;
and obtaining the quantization loss of each quantization object in the current iteration cycle based on each quantization object and the corresponding inverse quantization result.
7. The quantization method according to any one of claims 1 to 6, wherein the determining loss weights corresponding to the quantization losses of the quantization objects, respectively, based on the quantization losses comprises:
carrying out normalization processing on the quantization losses respectively corresponding to the quantization objects belonging to the same type to obtain the loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type;
alternatively, the first and second electrodes may be,
respectively processing the quantization objects belonging to the same type based on a preset monotonic function to obtain loss weights respectively corresponding to the quantization losses of the quantization objects belonging to the same type;
alternatively, the first and second electrodes may be,
adopting a pre-trained weight prediction neural network to carry out weight prediction processing on quantization objects belonging to the same type to obtain loss weights respectively corresponding to quantization losses of the quantization objects belonging to the same type;
alternatively, the first and second electrodes may be,
determining a target value interval to which the quantization loss of each quantization object belongs in a plurality of value intervals aiming at each quantization object; and determining a loss weight corresponding to the quantization loss of each quantization object based on the weight corresponding to the target value interval.
8. The quantization method according to any one of claims 1 to 7, wherein after determining quantization losses of the quantization objects in the current iteration cycle based on the quantization results of the quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the quantization objects based on the quantization losses, further comprising:
determining whether the current iteration cycle meets an iteration stop condition based on the loss weights respectively corresponding to the multiple quantitative objects;
entering a next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition;
the determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient includes:
and in response to that the current iteration cycle meets the iteration stop condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.
9. The quantization method of claim 8, wherein the iteration stop condition comprises at least one of:
the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on loss weights respectively corresponding to the plurality of quantization objects;
the difference value between the total quantization losses corresponding to the target iteration cycles is smaller than a preset difference value threshold value; wherein the target iteration cycle comprises: the current iteration cycle, and at least one historical iteration cycle that is continuous with the current iteration cycle.
10. The quantization method according to any one of claims 1 to 9, wherein the quantizing the first neural network using the target quantization coefficient to generate a second neural network comprises:
quantizing the quantization object in the target network layer by using the target quantization coefficient to obtain a target quantization object;
generating the second neural network based on the target quantization object.
11. An apparatus for quantizing a neural network, comprising:
a processing module for performing at least one of the following iteration cycles, and in each iteration cycle: determining quantization coefficients corresponding to a plurality of quantization objects when a target network layer in a first neural network is quantized; wherein the quantization coefficients are determined based on original quantization coefficients or based on quantization losses and loss weights determined over historical iteration cycles; respectively carrying out quantization operation on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects; determining quantization losses corresponding to the quantization objects respectively based on quantization results corresponding to the quantization objects respectively, and determining loss weights corresponding to the quantization losses of the quantization objects respectively based on the quantization losses;
and the quantization module is used for determining the quantization coefficient determined in the last iteration cycle as a target quantization coefficient, and performing quantization processing on the first neural network by using the target quantization coefficient to generate a second neural network.
12. A computer device, comprising: a processor, a memory storing machine-readable instructions executable by the processor, the processor for executing the machine-readable instructions stored in the memory, the processor performing the steps of the method of quantifying a neural network of any one of claims 1 to 10 when the machine-readable instructions are executed by the processor.
13. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when executed by a computer device, performs the steps of the quantization method of a neural network according to any one of claims 1 to 10.
CN202111433360.7A 2021-11-29 2021-11-29 Neural network quantification method and device, computer equipment and storage medium Pending CN114048853A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111433360.7A CN114048853A (en) 2021-11-29 2021-11-29 Neural network quantification method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111433360.7A CN114048853A (en) 2021-11-29 2021-11-29 Neural network quantification method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114048853A true CN114048853A (en) 2022-02-15

Family

ID=80211685

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111433360.7A Pending CN114048853A (en) 2021-11-29 2021-11-29 Neural network quantification method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114048853A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118133924A (en) * 2024-05-08 2024-06-04 深圳鲲云信息科技有限公司 Method and computing device for convolutional network equalization quantization

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118133924A (en) * 2024-05-08 2024-06-04 深圳鲲云信息科技有限公司 Method and computing device for convolutional network equalization quantization

Similar Documents

Publication Publication Date Title
EP3619652B1 (en) Adaptive bit-width reduction for neural networks
CN110555450B (en) Face recognition neural network adjusting method and device
TW201918939A (en) Method and apparatus for learning low-precision neural network
CN110555508B (en) Artificial neural network adjusting method and device
CN110598839A (en) Convolutional neural network system and method for quantizing convolutional neural network
KR20190034985A (en) Method and apparatus of artificial neural network quantization
CN111598237B (en) Quantization training, image processing method and device, and storage medium
KR20210064303A (en) Quantization of Trained Long Short Memory Neural Networks
CN112149797B (en) Neural network structure optimization method and device and electronic equipment
CN111178258B (en) Image identification method, system, equipment and readable storage medium
CN112513886A (en) Information processing method, information processing apparatus, and information processing program
CN110826685A (en) Method and device for convolution calculation of neural network
CN113780549A (en) Quantitative model training method, device, medium and terminal equipment for overflow perception
Liberis et al. Differentiable neural network pruning to enable smart applications on microcontrollers
CN114048853A (en) Neural network quantification method and device, computer equipment and storage medium
CN112085175B (en) Data processing method and device based on neural network calculation
CN111126557B (en) Neural network quantization, application method, device and computing equipment
KR102368590B1 (en) Electronic apparatus and control method thereof
CN116306879A (en) Data processing method, device, electronic equipment and storage medium
CN115983324A (en) Neural network quantization method and device and electronic equipment
CN110276448B (en) Model compression method and device
CN114492754A (en) Neural network generation method, neural network generation device, data processing method, data processing device, electronic device and medium
US20230058500A1 (en) Method and machine learning system to perform quantization of neural network
CN113157453B (en) Task complexity-based high-energy-efficiency target detection task dynamic scheduling method
CN113052290B (en) Neural network generation method, neural network data processing method, neural network generation device, neural network data processing device, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination