WO2021082851A1 - 一种神经网络模型的量化方法、装置及计算机可读存储介质 - Google Patents

一种神经网络模型的量化方法、装置及计算机可读存储介质 Download PDF

Info

Publication number
WO2021082851A1
WO2021082851A1 PCT/CN2020/118673 CN2020118673W WO2021082851A1 WO 2021082851 A1 WO2021082851 A1 WO 2021082851A1 CN 2020118673 W CN2020118673 W CN 2020118673W WO 2021082851 A1 WO2021082851 A1 WO 2021082851A1
Authority
WO
WIPO (PCT)
Prior art keywords
convolution kernel
coefficients
abnormal
distribution
target
Prior art date
Application number
PCT/CN2020/118673
Other languages
English (en)
French (fr)
Inventor
徐兵
张楠赓
Original Assignee
嘉楠明芯(北京)科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 嘉楠明芯(北京)科技有限公司 filed Critical 嘉楠明芯(北京)科技有限公司
Priority to US17/771,969 priority Critical patent/US20220398413A1/en
Publication of WO2021082851A1 publication Critical patent/WO2021082851A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/2163—Partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention belongs to the field of neural network computing, and specifically relates to a quantization method, device and computer-readable storage medium of a neural network model.
  • the embodiment of the present invention proposes a quantization method and device of a neural network model, and a computer-readable storage medium. By using this method and device, the above-mentioned problems can be solved.
  • a quantification method for a neural network model including: determining the target convolution kernel with abnormal coefficient distribution in the neural network model; splitting the target convolution kernel to obtain multiple subconvolution kernels; The product kernels are respectively quantized to obtain multiple quantized convolution kernels, and multiple quantized convolution kernels are used to replace the target convolution kernel.
  • determining the target convolution kernel with abnormal coefficient distribution in the neural network model further includes: obtaining any one of the convolution kernels to be quantized in the neural network model; according to the multiple coefficients contained in the convolution kernel to be quantized
  • the first threshold is determined by the standard deviation of the coefficient distribution of the coefficients; the difference between the maximum and/or minimum and the mean of the multiple coefficients contained in the convolution kernel to be quantized is determined; when the difference exceeds the first threshold, the volume to be quantized is determined
  • the product kernel is the target convolution kernel.
  • splitting the target convolution kernel further includes: determining an abnormal distribution coefficient in the target convolution kernel; and splitting the target convolution kernel according to the abnormal distribution coefficient.
  • determining the abnormal distribution coefficient in the target convolution kernel further includes: determining the second threshold according to the standard deviation of the coefficient distribution of the multiple coefficients included in the target convolution kernel; and according to the determined second threshold Determine the abnormal distribution coefficient in the target convolution kernel.
  • splitting the target convolution kernel according to the abnormal distribution coefficients further includes: decomposing the abnormal distribution coefficients in the target convolution kernel to obtain multiple non-abnormal decomposition coefficients; using multiple decompositions The coefficients form multiple groups, where each group contains any one of the multiple decomposition coefficients; the multiple non-anomalous distribution coefficients in the target convolution kernel are divided into multiple groups, and each group contains at least one non-anomalous Distribution coefficient; use non-anomalous distribution coefficients and decomposition coefficients in multiple groups to form multiple sub-convolution kernels.
  • decomposing the abnormal distribution coefficient in the target convolution kernel to obtain multiple non-anomalous decomposition coefficients further includes: decomposing the abnormal distribution coefficient in half by using the dichotomy method at least once, until the non-abnormal distribution coefficient is obtained.
  • dividing the multiple non-anomalous distribution coefficients in the target convolution kernel into multiple groups respectively includes: dividing the multiple non-anomalous distribution coefficients into multiple groups based on a uniform extraction operation, so that The coefficients in the multiple sub-convolution kernels are evenly distributed.
  • quantizing the multiple sub-convolution kernels separately further includes: determining the quantization boundary according to the coefficient distribution of the multiple sub-convolution kernels; and performing the multiple sub-convolution according to the determined quantization boundary and the target quantization bit width.
  • the nuclei are respectively linearly quantified.
  • a quantization device for a neural network model including: a determination module for determining a target convolution kernel with abnormal coefficient distribution in the neural network model; a splitting module for splitting the target convolution kernel, Obtain multiple sub-convolution kernels; a quantization module is used to quantize multiple sub-convolution kernels to obtain multiple quantized convolution kernels, and replace the target convolution kernel with multiple quantized convolution kernels.
  • the determining module is further used to: obtain any one of the to-be-quantized convolution kernels in the neural network model; determine the first threshold according to the coefficient distribution standard deviations of the multiple coefficients contained in the to-be-quantized convolution kernel; determine; The difference between the maximum value and/or the minimum value among the multiple coefficients included in the convolution kernel to be quantized and the mean value; when the difference exceeds the first threshold, the convolution kernel to be quantized is determined to be the target convolution kernel.
  • the splitting module is further used to: determine the abnormal distribution coefficient in the target convolution kernel; and split the target convolution kernel according to the abnormal distribution coefficient.
  • the splitting module is further used to: determine the second threshold according to the standard deviation of the coefficient distribution of the multiple coefficients contained in the target convolution kernel; and determine the second threshold according to the determined second threshold. Abnormal distribution coefficient.
  • the splitting module is also used to: decompose the abnormal distribution coefficients in the target convolution kernel to obtain multiple non-anomalous decomposition coefficients; use the multiple decomposition coefficients to form multiple groups, each of which The group contains any one of the multiple decomposition coefficients; the multiple non-anomalous distribution coefficients in the target convolution kernel are divided into multiple groups, where each group contains at least one non-anomalous distribution coefficient; Non-anomalous distribution coefficients and decomposition coefficients form multiple sub-convolution kernels.
  • the splitting module is further used to: use dichotomy to decompose the abnormal distribution coefficients in half at least once, until multiple non-abnormal decomposition coefficients are obtained, wherein the second threshold is used to determine whether the multiple decomposition coefficients are Not abnormal.
  • the splitting module is also used to divide the multiple non-abnormal distribution coefficients into multiple groups based on a uniform extraction operation, so that the coefficients in the multiple sub-convolution kernels are evenly distributed.
  • the quantization module is further configured to: determine the quantization boundary according to the coefficient distribution of the multiple subconvolution kernels; and perform linear quantization on the multiple subconvolution kernels respectively according to the determined quantization boundary and the target quantization bit width.
  • a quantization device for a neural network model including: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are At least one processor executes, so that at least one processor can execute: determine the target convolution kernel with abnormal coefficient distribution in the neural network model; split the target convolution kernel to obtain multiple subconvolution kernels; perform multiple subconvolutions The kernels are respectively quantized to obtain multiple quantized convolution kernels, and multiple quantized convolution kernels are used to replace the target convolution kernel.
  • a computer-readable storage medium stores a program.
  • the program is executed by a multi-core processor, the multi-core processor executes the method as in the first aspect.
  • the above-mentioned at least one technical solution adopted in the embodiments of the application can achieve the following beneficial effects: by performing convolution kernel splitting on the target convolution kernel with abnormal parameter distribution in the neural network model, and then separately splitting the multiple subconvolution kernels obtained by the splitting. Perform quantization to achieve the fixed point of the target convolution kernel, thereby reducing the quantization error.
  • Fig. 1 is a schematic flowchart of a quantification method of a neural network model according to an embodiment of the present invention
  • Fig. 2 is a schematic structural diagram of a schematic target convolution kernel
  • FIG. 3 shows a schematic diagram of performing direct linear quantization on the target convolution kernel shown in FIG. 2;
  • FIG. 4 shows a schematic diagram of quantization after splitting the target convolution kernel shown in FIG. 2;
  • FIG. 5 is a schematic flowchart of a quantification method of a neural network model according to another embodiment of the present invention.
  • Fig. 6 is a schematic flowchart of a quantification method of a neural network model according to another embodiment of the present invention.
  • Fig. 7 is a schematic structural diagram of a quantization device of a neural network model according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a quantization device of a neural network model according to another embodiment of the present invention.
  • the trained neural network may have floating-point type parameters, such as weight parameters with 32-bit floating-point precision.
  • floating-point type parameters require a relatively large number of operations and relatively frequent memory accesses.
  • the floating-point type parameters in the neural network model can be quantified, and the floating-point type number can be converted into a fixed-point type parameter with a specific number of bits, for example, a weight parameter with an accuracy of 8 to a specific point.
  • the device deploying the neural network may be, for example, autonomous vehicles, robots, smart phones, tablet devices, augmented reality (AR) devices, Internet of Things (IoT) devices, etc. that perform voice recognition, image recognition, etc., by using neural networks, but The present disclosure is not limited to this.
  • Fig. 1 schematically shows a flow chart of a method 100 for quantifying a neural network model according to an embodiment of the present invention.
  • the method 100 includes:
  • Step 101 Determine the target convolution kernel with abnormal coefficient distribution in the neural network model
  • Step 102 Split the target convolution kernel to obtain multiple sub-convolution kernels
  • Step 103 quantize multiple sub-convolution kernels respectively to obtain multiple quantized convolution kernels, and use multiple quantized convolution kernels to replace the target convolution kernel.
  • the neural network model includes multiple convolution kernels to be quantized, and a target convolution kernel with abnormal coefficient distribution can be determined from the multiple convolution kernels to be quantized.
  • the abnormal coefficient distribution may mean that the coefficients included in the target convolution kernel do not show a normal distribution, or the distribution of the coefficients included in the target convolution kernel is uneven or not concentrated.
  • quantization may refer to linear quantization.
  • the usual method of linear quantization is to map all the coefficients of the target convolution kernel from a relatively dense data space (such as 32-bit floating point numbers, hereinafter referred to as F32) to a certain mapping relationship.
  • F32 32-bit floating point numbers
  • INT8 the quantization method is generally to find the distribution range of all the coefficients contained in the target convolution kernel [X min , X max ], assuming the quantized target quantization
  • the bit width is INT8.
  • Figure 2 shows a 3 ⁇ 3 ⁇ 3 target convolution kernel 20, which includes (X 1 , X 2 , X 3 , ...) and other 27 floating-point type coefficients (not shown in the figure)
  • the coefficients X 10 ⁇ X 27 assuming that the maximum value X max of the 27 floating-point type coefficients is X 9 , and the minimum value X min is X 1 , and X 9 of the 27 coefficients is compared with other coefficients
  • the distribution is relatively discrete.
  • FIG. 3 shows a schematic diagram of directly linearly quantizing the 27 floating-point type coefficients included in the target convolution kernel 20. It can be seen that the linear quantization of multiple coefficients of the target convolution kernel is directly performed.
  • the target convolution kernel 20 can be split into a subconvolution kernel 21 and a subconvolution kernel 22, so that the sum of the two is equal to the target convolution kernel 20, and
  • the coefficient 0 represents the position of the null coefficient in each sub-convolution kernel.
  • the sub-convolution kernel 21 and the sub-convolution kernel 22 with uniform distribution can be obtained by extracting one by one as shown in FIG. 4.
  • other splitting methods can also be used.
  • the multiple coefficients of the target convolution kernel are divided into multiple groups according to the coefficient size, and the sub-convolution kernels are determined according to the coefficients of each group. There is no specific limitation.
  • the target convolution kernel 20 can also be split into more subconvolution kernels, and the number of subconvolution kernels is not specifically limited in this embodiment.
  • the sub-convolution kernel 21 and the sub-convolution kernel 22 can be quantized respectively to obtain the quantized convolution kernel 31 and the quantized convolution kernel 32, which are compared with the original target convolution kernel 20.
  • the coefficient distribution range, the coefficient distribution range of each sub-convolution kernel is at least partially reduced, so the quantized convolution kernel obtained by quantizing each sub-convolution kernel has a smaller quantization error.
  • the multiple sub-convolution kernels obtained by the split are quantized to obtain multiple quantized convolution kernels.
  • the quantization of the target convolution kernel thereby reducing the quantization error of the target convolution kernel with abnormal coefficient distribution.
  • some embodiments of the present application also provide some specific implementation schemes and extension schemes of the quantization processing method, which will be described below.
  • the above step 101 may further include: obtaining any one of the to-be-quantized convolution kernels in the neural network model; and determining the first threshold value according to the coefficient distribution standard deviations of the multiple coefficients included in the to-be-quantized convolution kernel; Determine the difference between the maximum value and/or minimum value of the multiple coefficients included in the convolution kernel to be quantized and the mean value; when the difference exceeds the first threshold, determine the convolution kernel to be quantized as the target convolution kernel.
  • the above-mentioned first threshold is preferably set to a value of 3 to 6 times the standard deviation of the coefficient distribution of the multiple coefficients included in the convolution kernel to be quantized, so as not to cause significant performance loss.
  • the foregoing step 102 may further include:
  • Step 501 Determine the abnormal distribution coefficient in the target convolution kernel
  • Step 502 Split the target convolution kernel according to the abnormal distribution coefficient.
  • one or more coefficients that do not obey the normal distribution among the multiple coefficients of the target convolution kernel can be determined as the abnormal distribution coefficients.
  • the abnormal distribution coefficient in the target convolution kernel 20 may be X 9 .
  • the above step 501 may further include: determining a second threshold value according to the standard deviation of the coefficient distribution of multiple coefficients included in the target convolution kernel; determining the second threshold value in the target convolution kernel according to the determined second threshold value Abnormal distribution coefficient.
  • the above-mentioned second threshold may be 3-6 times the standard deviation of the coefficient distribution of the multiple coefficients included in the target convolution kernel, or the second threshold may be determined directly according to the first threshold.
  • the first threshold of the convolution kernel A to be quantized can be directly used as the second threshold of the target convolution kernel to avoid repeated operations.
  • Determining the abnormal distribution coefficient in the target convolution kernel according to the determined second threshold may include: determining the difference between each coefficient included in the target convolution kernel and the mean value of the multiple coefficients, and determining whether the difference exceeds
  • the above second threshold is used to find one or more abnormal distribution parameters in the target convolution kernel.
  • the foregoing step 502 may further include:
  • Step 601 Decompose the abnormal distribution coefficients in the target convolution kernel to obtain multiple non-anomalous decomposition coefficients
  • Step 602 Use multiple decomposition coefficients to form multiple groups, where each group contains any one of the multiple decomposition coefficients;
  • Step 603 Divide the multiple non-anomalous distribution coefficients in the target convolution kernel into multiple groups, where each group contains at least one non-anomalous distribution coefficient;
  • Step 604 Use the non-anomalous distribution coefficients and decomposition coefficients in the multiple groups to form multiple sub-convolution kernels.
  • the target convolution kernel 20 contains an abnormal distribution coefficient X 9 and multiple non-abnormal distribution coefficients.
  • the abnormal distribution coefficient X 9 can be decomposed to obtain two non-abnormal decomposition coefficients 1 /2X 9 , divide the multiple coefficients contained in the target convolution kernel 20 into two groups, and each group contains a 1/2X 9 , and then divide each non-anomalous distribution coefficient into any group to form Sub-convolution kernel 21 and sub-convolution kernel 22.
  • the above step 601 may further include: using a dichotomy to decompose the abnormal distribution coefficient at least once, until a plurality of non-abnormal decomposition coefficients are obtained, wherein the second threshold is used to determine the plurality of decomposition coefficients. Whether it is not abnormal.
  • the above step 603 may further include: dividing the multiple non-abnormal distribution coefficients into multiple groups based on a uniform extraction operation, so that the coefficients in the multiple sub-convolution kernels are evenly distributed.
  • the subconvolution kernel 21 may include: (X 1 ,0,X 3 ,0,X 5 ,0,X 7 ,0,1/2X 9 ,...); subconvolution The core 21 may include: (0,X 2 ,0,X 4 ,0,X 6 ,0,X 8 ,1/2X 9 ,...);
  • the foregoing step 103 may further include: determining a quantization boundary according to the coefficient distribution of the multiple subconvolution kernels; and quantizing the multiple subconvolution kernels respectively according to the determined quantization boundary and the target quantization bit width.
  • linear quantization may be performed on the sub-convolution kernel 21 to obtain a quantized convolution kernel 31, and linear quantization may be performed on the sub-convolution kernel 22 to obtain a quantized convolution kernel 32.
  • the quantized convolution kernel 31 and the quantized convolution kernel 32 are used to replace the original target convolution kernel 20 in the neural network model to perform fixed-point neural network operations.
  • an embodiment of the present invention also provides a neural network model quantization device, which is used to implement the neural network model quantization method provided in any of the foregoing embodiments.
  • FIG. 7 is a schematic structural diagram of a quantization device of a neural network model provided by an embodiment of the present invention.
  • the device 700 includes:
  • the determining module 701 is used to determine the target convolution kernel with abnormal coefficient distribution in the neural network model
  • the splitting module 702 is used to split the target convolution kernel to obtain multiple sub-convolution kernels
  • the quantization module 703 is configured to quantize multiple sub-convolution kernels respectively to obtain multiple quantized convolution kernels, and replace the target convolution kernel with multiple quantized convolution kernels.
  • the determining module 701 may also be used to: obtain any convolution kernel to be quantized in the neural network model; determine the first threshold according to the coefficient distribution standard deviations of multiple coefficients contained in the convolution kernel to be quantized ; Determine the difference between the maximum and/or minimum of the multiple coefficients included in the convolution kernel to be quantized and the mean; when the difference exceeds the first threshold, determine the convolution kernel to be quantized as the target convolution kernel.
  • the splitting module 702 may also be used to: determine the abnormal distribution coefficient in the target convolution kernel; and split the target convolution kernel according to the abnormal distribution coefficient.
  • the splitting module 702 may also be used to: determine a second threshold according to the standard deviation of the coefficient distribution of multiple coefficients included in the target convolution kernel; determine the target convolution kernel according to the determined second threshold The coefficient of anomalous distribution in.
  • the splitting module 702 can also be used to: decompose the abnormal distribution coefficients in the target convolution kernel to obtain multiple non-abnormal decomposition coefficients; use the multiple decomposition coefficients to form multiple groups, where Each group contains any one of the multiple decomposition coefficients; divide the multiple non-anomalous distribution coefficients in the target convolution kernel into multiple groups, where each group contains at least one non-anomalous distribution coefficient; use multiple groups The non-anomalous distribution coefficients and decomposition coefficients in, form multiple sub-convolution kernels.
  • the splitting module 702 may also be used to: use a dichotomy to decompose the abnormal distribution coefficients at least once until a plurality of non-abnormal decomposition coefficients are obtained, wherein the second threshold is used to determine the plurality of decompositions. Whether the coefficient is not abnormal.
  • the splitting module 702 may also be used to divide multiple non-abnormal distribution coefficients into multiple groups based on a uniform extraction operation, so that the coefficients in the multiple sub-convolution kernels are evenly distributed.
  • the quantization module 703 may also be used to: determine the quantization boundary according to the coefficient distributions of the multiple subconvolution kernels; and perform linear quantization on the multiple subconvolution kernels respectively according to the determined quantization boundary and the target quantization bit width .
  • the quantization device of the neural network model in the embodiment of the present application can implement the various processes of the foregoing embodiment of the quantization method of the neural network model, and achieve the same effects and functions, which will not be repeated here.
  • FIG. 8 is a schematic diagram of a quantization device for a neural network model according to an embodiment of the present application, which is used to execute the quantization method for a neural network model as shown in FIG. 1, and the device includes: at least one processor; and, and At least one processor is a memory communicatively connected to the processor; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor, so that at least one processor can execute: Determine the abnormal coefficient distribution in the neural network model Target convolution kernel; split the target convolution kernel to obtain multiple sub-convolution kernels; quantize the multiple sub-convolution kernels to obtain multiple quantized convolution kernels, and use multiple quantized convolution kernels to replace the target volume Product core.
  • An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program, and when the program is executed by a multi-core processor, the multi-core processor is caused to execute: determine that the coefficient distribution in the neural network model is abnormal Target convolution kernel; split the target convolution kernel to obtain multiple sub-convolution kernels; quantize the multiple sub-convolution kernels to obtain multiple quantized convolution kernels, and use multiple quantized convolution kernels to replace the target volume Product core.
  • the device and the computer-readable storage medium provided in the embodiments of the present application correspond to the method one-to-one. Therefore, the device, the device, and the computer-readable storage medium also have beneficial technical effects similar to the corresponding method.
  • the beneficial technical effects have been described in detail, and therefore, the beneficial technical effects of the device, equipment and computer-readable storage medium will not be repeated here.
  • the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-permanent memory in a computer readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
  • the information can be computer-readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other memory technology
  • CD-ROM

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

一种神经网络模型的量化方法、装置及计算机可读存储介质，该方法包括：确定神经网络模型中系数分布异常的目标卷积核(101)；对目标卷积核进行拆分，得到多个子卷积核(102)；对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核(103)。利用上述方法，能够减少量化误差。

Description

一种神经网络模型的量化方法、装置及计算机可读存储介质 技术领域
本发明属于神经网络计算领域，具体涉及一种神经网络模型的量化方法、装置及计算机可读存储介质。
背景技术
本部分旨在为权利要求书中陈述的本发明的实施方式提供背景或上下文。此处的描述不因为包括在本部分中就承认是现有技术。
近年来，深度神经网络的规模越来越大，进而导致训练出的神经网络模型中包含大量的参数，这些参数的存储和传输使得运算成本和功耗非常大。这些缺点尤其会限制将神经网络模型部署或应用到边缘设备或低功耗设备。
为了解决神经网络模型的运算量过大的问题，通常可以采用对训练出的神经网络模型的系数进行量化的方法。然而现有技术中，对神经网络模型的量化方案仍然不够完善，容易导致量化误差过大等问题，影响神经网络模型的运算效果。
发明内容
针对上述现有技术对神经网络模型进行量化时容易导致量化误差过大的问题。本发明实施例提出了一种神经网络模型的量化方法、装置及计算机可读存储介质，利用这种方法及装置，能够解决上述问题。
本发明的实施例中提供了以下方案。
一方面，提供了一种神经网络模型的量化方法，包括：确定神经网络模型中系数分布异常的目标卷积核；对目标卷积核进行拆分，得到多个子卷积核；对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
在一些可能的实施方式中，确定神经网络模型中系数分布异常的目标卷积核，还包括：获取神经网络模型中的任意一个待量化卷积核；根据待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；确定待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；当差值超过第一阈值，确定待量化卷积核为目标卷积核。
在一些可能的实施方式中，对目标卷积核进行拆分，还包括：确定目标卷积核中的异常分布系数；根据异常分布系数对目标卷积核进行拆分。
在一些可能的实施方式中，确定目标卷积核中的异常分布系数，还包括：根据目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；根据所确定的第二阈值确定目标卷积核中的异常分布系数。
在一些可能的实施方式中，根据异常分布系数对目标卷积核进行拆分，还包括：将目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；利用多个分解系数形成多个分组，其中每个分组包含多个分解系数的任意一个分解系数；将目标卷积核中的多个非异常分布系数分别划分至多个分组中，其中每个分组包含至少一个非异常分布系数；利用多个分组中的非异常分布系数和分解系数形成多个子卷积核。
在一些可能的实施方式中，将目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数，还包括：利用二分法对异常分布系数进行至少一次对半分解，直至得到非异常的多个分解系数，其中利用第二阈值判断多个分解系数是否非异常。
在一些可能的实施方式中，将目标卷积核中的多个非异常分布系数分别划分至多个分组中，还包括：基于均匀抽取操作将多个非异常分布系数分别划分至多个分组中，使多个子卷积核中的系数分布均匀。
在一些可能的实施方式中，对多个子卷积核分别进行量化，还包括：根据多个子卷积核的系数分布确定量化边界；根据所确定的量化边界和目标量化位宽对多个子卷积核分别进行线性量化。
第二方面，提供一种神经网络模型的量化装置，包括：确定模块，用于确定神经网络模型中系数分布异常的目标卷积核；拆分模块，用于对目标卷积核进行拆分，得到多个子卷积核；量化模块，用于对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
在一些可能的实施方式中，确定模块还用于：获取神经网络模型中的任意一个待量化卷积核；根据待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；确定待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；当差值超过第一阈值，确定待量化卷积核为目标卷积核。
在一些可能的实施方式中，拆分模块还用于：确定目标卷积核中的异常分布系数；根据异常分布系数对目标卷积核进行拆分。
在一些可能的实施方式中，拆分模块还用于：根据目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；根据所确定的第二阈值确定目标卷积核中的异常分布系数。
在一些可能的实施方式中，拆分模块还用于：将目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；利用多个分解系数形成多个分组，其中每个分组包含多个分解系数的任意一个分解系数；将目标卷积核中的多个非异常分布系数分别划分至多个分组中，其中每个分组包含至少一个非异常分布系数；利用多个分组中的非异常分布系数和分解系数形成多个子卷积核。
在一些可能的实施方式中，拆分模块还用于：利用二分法对异常分布系数进行至少一次对半分解，直至得到非异常的多个分解系数，其中利用第二阈值判断多个分解系数是否非异常。
在一些可能的实施方式中，拆分模块还用于：基于均匀抽取操作将多个非异常分布系数分别划分至多个分组中，使多个子卷积核中的系数分布均匀。
在一些可能的实施方式中，量化模块还用于：根据多个子卷积核的系数分布确定量化边界；根据所确定的量化边界和目标量化位宽对多个子卷积核分别进行线性量化。
第三方面，提供一种神经网络模型的量化装置，包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行：确定神经网络模型中系数分布异常的目标卷积核；对目标卷积核进行拆分，得到多个子卷积核；对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
第四方面，提供一种计算机可读存储介质，计算机可读存储介质存储有程序，当程序被多核处理器执行时，使得多核处理器执行如第一方面的方法。
本申请实施例采用的上述至少一个技术方案能够达到以下有益效果：通过对神经网络模型中参数分布异常的目标卷积核执行卷积核拆分，再对拆分得到的多个子卷积核分别进行量化以实现目标卷积核的定点化，从而减少了量化误差。
应当理解，上述说明仅是本发明技术方案的概述，以便能够更清楚地了解本发明的技术手段，从而可依照说明书的内容予以实施。为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举例说明本发明的具体实施方式。
附图说明
通过阅读下文的示例性实施例的详细描述，本领域普通技术人员将明白本文所述的有点和益处以及其他优点和益处。附图仅用于示出示例性实施例的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的标号表示相同的部件。在附图中：
图1为根据本发明一实施例的神经网络模型的量化方法的流程示意图；
图2为一种示意性的目标卷积核的结构示意图；
图3示出了对图2所示出的目标卷积核执行直接线性量化的示意图；
图4示出了对图2所示出的目标卷积核执行拆分后量化的示意图；
图5为根据本发明另一实施例的神经网络模型的量化方法的流程示意图；
图6为根据本发明又一实施例的神经网络模型的量化方法的流程示意图；
图7为根据本发明一实施例的神经网络模型的量化装置的结构示意图；
图8为根据本发明另一实施例的神经网络模型的量化装置的结构示意图。
在附图中，相同或对应的标号表示相同或对应的部分。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
在本发明中，应理解，诸如“包括”或“具有”等术语旨在指示本说明书中所公开的特征、数字、步骤、行为、部件、部分或其组合的存在，并且不旨在排除一个或多个其他特征、数字、步骤、行为、部件、部分或其组合存在的可能性。
另外还需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本发明。
经训练的神经网络可能具有浮点类型参数，例如具有32比特浮点精度的权值参数。然而，浮点类型参数相对于定点类型参数需要相对大量的运算和相对频繁的存储器访问。基于此，为了减少运算量，可以对在神经网络模型中的浮点类型参数进行量化，将浮点类型数转换为具有特定比特数的定点类型参数，例如具有8比特定点精度的权值参数。其中，部署神经网络的设备例如可以是通过使用神经网络来执行语音识别、图像识别等的自主车辆、机器人、智能电话、平板设备、增强现实(AR)设备、物联网(IoT)设备等，但是本公开不限于此。
本发明实施例提出一种神经网络模型的量化方法。图1示意性地示出了根据本发明实施例的神经网络模型的量化方法100的流程示意图。
如图1所示，该方法100包括：
步骤101：确定神经网络模型中系数分布异常的目标卷积核；
步骤102：对目标卷积核进行拆分，得到多个子卷积核；
步骤103：对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
具体地，神经网络模型中包含多个待量化卷积核，可以从该多个待量化卷积核中确定系数分布异常的目标卷积核。其中，该系数分布异常可以指目标卷积核中包含的多个系数并非呈现正态分布，或该目标卷积核包含的多个系数的分布不平均或分布不集中。
本发明实施例中，量化可以指线性量化，线性量化的通常做法是将目标卷积核的所有系数按照某种映射关系从较为密集的数据空间(比如32比特浮点数，以下简称F32)映射到较为稀疏的空间(比如8比特定点数，以下简称INT8))进行量化，量化的方法一般是找到目标卷积核包含的所有系数的分布范围[X min,X max]，假设量化后的目标量化位宽为INT8，由于INT8的表达范围为[0,255]，那么上述线性量化相当于将[X min,X max]均分为255个单元，并将落入每一个单元的浮点系数转换为对应的定点数表达，比如X min转换后为0，X max转换后为255。
举例来说，图2示出了一种3×3×3的目标卷积核20，其包括(X 1、X 2、X 3、…)等27个浮点类型的系数(图中未示出系数X 10～X 27)，假设27个浮点类型的系数中的最大值X max为X 9，最小值X min,为X 1，且该27个系数中的X 9相较其他系数则分布地较为离散。进一步地，图3示出了将该目标卷积核20中包括的27个浮点类型系数直接进行线性量化的示意图，可以看出，在直接对目标卷积核的多个系数进行线性量化的情况下，为了使X max＝X 9能用正确的定点表示，在不增加量化位宽的情况下，必然会导致其他正常分布的多个系数的精度降低。而如果直接对X max做饱和处理，又会造成性能下降。
本发明实施例中，如图4所示，可以将目标卷积核20拆分为子卷积核21与子卷积核22，使二者相加之和与目标卷积核20等同，并以系数0表示各个子卷积核中的空系数位置。其中，可以如图4所示通过逐个抽取的方式得到分布均匀的子卷积核21与子卷积核22。可选地，也可以使用其他拆分方式，比如将目标卷积核的多个系数按系数大小分为多组，并根据每个分组的系数确定子卷积核，本实施例对拆分方式不作具体限定。可选地，目标卷积核20也可以拆分为更多的子卷积核，本实施例对子卷积核的数量不作具体限制。进一步，如图4所示，可以对子卷积核21和子卷积核22分别进行量化，得到量化卷积核31和量化卷积核32，由于相较于原有的目标卷积核20的系数分布范围，各个子卷积核的系数分布范围至少部分地减小，因此对各个子卷积核进行量化所得到的量化卷积核具有更小的量化误差。由此可以看出，对于系数分布异常的目标卷积核来说，通过上述拆分卷积核，再对拆分得到的多个子卷积核分别进行量化以得到多个量化卷积核，用于实现目标卷积核的量化，从而减少了系数分布异常的目标卷积核的量化误差。
基于图1的神经网络模型的量化方法，本申请的一些实施例还提供了该量化处理方法的一些具体实施方案，以及扩展方案，下面进行说明。
在一些可能的实施方式中，上述步骤101还可以包括：获取神经网络模型中的任意一个待量化卷积核；根据待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；确定待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；当差值超过第一阈值，确定待量化卷积核为目标卷积核。
可选地，上述第一阈值优选取值为待量化卷积核包含的多个系数的系数分布标准差的3～6倍，从而不产生明显的性能损失。
可选地，还可以对任意一个待量化卷积核包含的多个系数的系数分布直方图进行分析以判断其是否为目标卷积核。
在一些可能的实施方式中，如图5所示，上述步骤102还可以包括：
步骤501：确定目标卷积核中的异常分布系数；
步骤502：根据异常分布系数对目标卷积核进行拆分。
具体地，可以确定目标卷积核的多个系数中不服从正态分布的一个或多个系数作为异常分布系数。举例来说，如图2所示，目标卷积核20中的异常分布系数可以是X 9。
在一些可能的实施方式中，上述步骤501还可以包括：根据目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；根据所确定的第二阈值确定目标卷积核中的异常分布系数。
具体地，上述第二阈值可以取值为目标卷积核包含的多个系数的系数分布标准差的3～6倍，或直接根据第一阈值确定该第二阈值。举例来说，在确定待量化卷积核A为目标卷积核之后，可以直接将待量化卷积核A的第一阈值作为该目标卷积核的第二阈值以避免重复运算，二者可以是等同的。根据所确定的第二阈值确定目标卷积核中的异常分布系数可以包括：确定目标卷积核包含的每个系数与多个系数的均值之间的差值，并通过判断该差值是否超过上述第二阈值而找到目标卷积核中的一个或多个异常分布参数。
在一些可能的实施方式中，如图6所示，上述步骤502还可以包括：
步骤601：将目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；
步骤602：利用多个分解系数形成多个分组，其中每个分组包含多个分解系数的任意一个分解系数；
步骤603：将目标卷积核中的多个非异常分布系数分别划分至多个分组中，其中每个分组包含至少一个非异常分布系数；
步骤604：利用多个分组中的非异常分布系数和分解系数形成多个子卷积核。
举例来说，如图2所示，目标卷积核20中包含异常分布系数X 9和多个非异常分布的系数，可以将异常分布系数X 9进行分解，得到两个非异常的分解系数1/2X 9，将目标卷积核20包含的多个系数分为两个分组，每个分组中包含一个1/2X 9，然后将每个非异常分布系数各自划分到任一个分组中，从而形成子卷积核21和子卷积核22。
在一些可能的实施方式中，上述步骤601，还可以包括：利用二分法对异常分布系数进行至少一次对半分解，直至得到非异常的多个分解系数，其中利用第二阈值判断多个分解系数是否非异常。
举例来说，利用二分法对异常分布系数X 9进行第一次对半分解，得到两个1/2X 9的分解参数。然而在一些情况中1/2X 9可能仍属于异常分布的系数，此时可以对1/2X 9进行再次对半分解，得到四个1/2X 9的分解参数，诸如此类，直至得到非异常分布的分解参数。
在一些可能的实施方式中，上述步骤603还可以包括：基于均匀抽取操作将多个非异常分布系数分别划分至多个分组中，使多个子卷积核中的系数分布均匀。
举例来说，参见图4，子卷积核21可以包括：(X 1,0,X 3,0,X 5,0,X 7,0,1/2X 9,...)；子卷积核21中可以包括：(0,X 2,0,X 4,0,X 6,0,X 8,1/2X 9,...)；
在一些可能的实施方式中，上述步骤103还可以包括：根据多个子卷积核的系数分布确定量化边界；根据所确定的量化边界和目标量化位宽对多个子卷积核分别进行量化。
举例来说，参见图4，可以对子卷积核21执行线性量化得到量化卷积核31，对子卷积核22执行线性量化得到量化卷积核32。并利用量化卷积核31和量化卷积核32在神经网络模型中替代原有的目标卷积核20以执行定点类型的神经网络运算。
基于相同的技术构思，本发明实施例还提供一种神经网络模型的量化装置，用于执行上述任一实施例所提供的神经网络模型的量化方法。图7为本发明实施例提供的一种神经网络模型的量化装置的结构示意图。
如图7所示，该装置700包括：
确定模块701，用于确定神经网络模型中系数分布异常的目标卷积核；
拆分模块702，用于对目标卷积核进行拆分，得到多个子卷积核；
量化模块703，用于对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
在一些可能的实施方式中，确定模块701还可以用于：获取神经网络模型中的任意一个待量化卷积核；根据待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；确定待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；当差值超过第一阈值，确定待量化卷积核为目标卷积核。
在一些可能的实施方式中，拆分模块702还可以用于：确定目标卷积核中的异常分布系数；根据异常分布系数对目标卷积核进行拆分。
在一些可能的实施方式中，拆分模块702还可以用于：根据目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；根据所确定的第二阈值确定目标卷积核中的异常分布系数。
在一些可能的实施方式中，拆分模块702还可以用于：将目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；利用多个分解系数形成多个分组，其中每个分组包含多个分解系数的任意一个分解系数；将目标卷积核中的多个非异常分布系数分别划分至多个分组中，其中每个分组包含至少一个非异常分布系数；利用多个分组中的非异常分布系数和分解系数形成多个子卷积核。
在一些可能的实施方式中，拆分模块702还可以用于：利用二分法对异常分布系数进行至少一次对半分解，直至得到非异常的多个分解系数，其中利用第二阈值判断多个分解系数是否非异常。
在一些可能的实施方式中，拆分模块702还可以用于：基于均匀抽取操作将多个非异常分布系数分别划分至多个分组中，使多个子卷积核中的系数分布均匀。
在一些可能的实施方式中，量化模块703还可以用于：根据多个子卷积核的系数分布确定量化边界；根据所确定的量化边界和目标量化位宽对多个子卷积核分别进行线性量化。
需要说明的是，本申请实施例中的神经网络模型的量化装置可以实现前述神经网络模型的量化方法的实施例的各个过程，并达到相同的效果和功能，这里不再赘述。
图8为根据本申请一实施例的一种神经网络模型的量化装置的示意图，用于执行如图1所示出的神经网络模型的量化方法，该装置包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行：确定神经网络模型中系数分布异常的目标卷积核；对目标卷积核进行拆分，得到多个子卷积核；对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
本申请一实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有程序，当程序被多核处理器执行时，使得该多核处理器执行：确定神经网络模型中系数分布异常的目标卷积核；对目标卷积核进行拆分，得到多个子卷积核；对多个子卷积核分别进行量化以得到多个量化卷积核，并利用多个量化卷积核替代目标卷积核。
本申请中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、设备和计 算机可读存储介质实施例而言，由于其基本相似于方法实施例，所以其描述进行了简化，相关之处可参见方法实施例的部分说明即可。
本申请实施例提供的装置和计算机可读存储介质与方法是一一对应的，因此，装置、设备和计算机可读存储介质也具有与其对应的方法类似的有益技术效果，由于上面已经对方法的有益技术效果进行了详细说明，因此，这里不再赘述装置、设备和计算机可读存储介质的有益技术效果。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随 机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。此外，尽管在附图中以特定顺序描述了本发明方法的操作，但是，这并非要求或者暗示必须按照该特定顺序来执行这些操作，或是必须执行全部所示的操作才能实现期望的结果。附加地或备选地，可以省略某些步骤，将多个步骤合并为一个步骤执行，和/或将一个步骤分解为多个步骤执行。
虽然已经参考若干具体实施方式描述了本发明的精神和原理，但是应该理解，本发明并不限于所公开的具体实施方式，对各方面的划分也不意味着这些方面中的特征不能组合以进行受益，这种划分仅是为了表述的方便。本发明旨在涵盖所附权利要求的精神和范围内所包括的各种修改和等同布置。
虽然已经参考若干具体实施方式描述了本发明的精神和原理，但是应该理解，本发明并不限于所公开的具体实施方式，对各方面的划分也不意味着这些方面中的特征不能组合以进行受益，这种划分仅是为了表述的方便。本发明旨在涵盖所附权利要求的精神和范围内所包括的各种修改和等同布置。

Claims (18)

  1. 一种神经网络模型的量化方法，其特征在于，包括：
    确定神经网络模型中系数分布异常的目标卷积核；
    对所述目标卷积核进行拆分，得到多个子卷积核；
    对所述多个子卷积核分别进行量化以得到多个量化卷积核，并利用所述多个量化卷积核替代所述目标卷积核。
  2. 由权利要求1所述的方法，其特征在于，所述确定神经网络模型中系数分布异常的目标卷积核，还包括：
    获取所述神经网络模型中的任意一个待量化卷积核；
    根据所述待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；
    确定所述待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；
    当所述差值超过所述第一阈值，确定所述待量化卷积核为所述目标卷积核。
  3. 由权利要求1所述的方法，其特征在于，对所述目标卷积核进行拆分，还包括：
    确定所述目标卷积核中的异常分布系数；
    根据所述异常分布系数对所述目标卷积核进行拆分。
  4. 由权利要求3所述的方法，其特征在于，确定所述目标卷积核中的异常分布系数，还包括：
    根据所述目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；
    根据所确定的第二阈值确定所述目标卷积核中的异常分布系数。
  5. 由权利要求3所述的方法，其特征在于，根据所述异常分布系数对所述目标卷积核进行拆分，还包括：
    将所述目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；
    利用所述多个分解系数形成多个分组，其中每个分组包含所述多个分解系数的任意一个分解系数；
    将所述目标卷积核中的多个非异常分布系数分别划分至所述多个分组中，其中每个分组包含至少一个所述非异常分布系数；
    利用所述多个分组中的所述非异常分布系数和所述分解系数形成所述多个子卷积核。
  6. 由权利要求5所述的方法，其特征在于，将所述目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数，还包括：
    利用二分法对所述异常分布系数进行至少一次对半分解，直至得到所述非异常的多个分解系数，其中利用所述第二阈值判断所述多个分解系数是否非异常。
  7. 由权利要求5所述的方法，其特征在于，将所述目标卷积核中的多个非异常分布系数分别划分至所述多个分组中，还包括：
    基于均匀抽取操作将所述多个非异常分布系数分别划分至所述多个分组中，使所述多个子卷积核中的系数分布均匀。
  8. 由权利要求1所述的方法，其特征在于，对所述多个子卷积核分别进行量化，还包括：
    根据所述多个子卷积核的系数分布确定量化边界；
    根据所确定的量化边界和目标量化位宽对所述多个子卷积核分别进行线性量化。
  9. 一种神经网络模型的量化装置，其特征在于，包括：
    确定模块，用于确定神经网络模型中系数分布异常的目标卷积核；
    拆分模块，用于对所述目标卷积核进行拆分，得到多个子卷积核；
    量化模块，用于对所述多个子卷积核分别进行量化以得到多个量化卷积核，并利用所述多个量化卷积核替代所述目标卷积核。
  10. 由权利要求9所述的装置，其特征在于，所述确定模块还用于：
    获取所述神经网络模型中的任意一个待量化卷积核；
    根据所述待量化卷积核包含的多个系数的系数分布标准差确定第一阈值；
    确定所述待量化卷积核包含的多个系数中的最大值和/或最小值与均值之间的差值；
    当所述差值超过所述第一阈值，确定所述待量化卷积核为所述目标卷积核。
  11. 由权利要求9所述的装置，其特征在于，所述拆分模块还用于：
    确定所述目标卷积核中的异常分布系数；
    根据所述异常分布系数对所述目标卷积核进行拆分。
  12. 由权利要求11所述的装置，其特征在于，所述拆分模块还用于：
    根据所述目标卷积核包含的多个系数的系数分布标准差而确定第二阈值；
    根据所确定的第二阈值确定所述目标卷积核中的异常分布系数。
  13. 由权利要求11所述的装置，其特征在于，所述拆分模块还用于：
    将所述目标卷积核中的异常分布系数进行分解，得到非异常的多个分解系数；
    利用所述多个分解系数形成多个分组，其中每个分组包含所述多个分解系数的任意一个分解系数；
    将所述目标卷积核中的多个非异常分布系数分别划分至所述多个分组中，其中每个分组包含至少一个所述非异常分布系数；
    利用所述多个分组中的所述非异常分布系数和所述分解系数形成所述多个子卷积核。
  14. 由权利要求13所述的装置，其特征在于，所述拆分模块还用于：
    利用二分法对所述异常分布系数进行至少一次对半分解，直至得到所述非异常的多个分解系数，其中利用所述第二阈值判断所述多个分解系数是否非异常。
  15. 由权利要求13所述的装置，其特征在于，所述拆分模块还用于：
    基于均匀抽取操作将所述多个非异常分布系数分别划分至所述多个分组中，使所述多个子卷积核中的系数分布均匀。
  16. 由权利要求9所述的装置，其特征在于，所述量化模块还用于：
    根据所述多个子卷积核的系数分布确定量化边界；
    根据所确定的量化边界和目标量化位宽对所述多个子卷积核分别进行线性量化。
  17. 一种神经网络模型的量化装置，其特征在于，包括：
    至少一个处理器；以及，
    与至少一个处理器通信连接的存储器；其中，
    存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行：
    确定神经网络模型中系数分布异常的目标卷积核；
    对所述目标卷积核进行拆分，得到多个子卷积核；
    对所述多个子卷积核分别进行量化以得到多个量化卷积核，并利用所述多个量化卷积核替代所述目标卷积核。
  18. 一种计算机可读存储介质，所述计算机可读存储介质存储有程序，当所述程序被多核处理器执行时，使得所述多核处理器执行如权利要求1-8中任一项所述的方法。
PCT/CN2020/118673 2019-10-31 2020-09-29 一种神经网络模型的量化方法、装置及计算机可读存储介质 WO2021082851A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/771,969 US20220398413A1 (en) 2019-10-31 2020-09-29 Quantization method and device for neural network model, and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911062937.0A CN112749783A (zh) 2019-10-31 2019-10-31 一种神经网络模型的量化方法、装置及计算机可读存储介质
CN201911062937.0 2019-10-31

Publications (1)

Publication Number Publication Date
WO2021082851A1 true WO2021082851A1 (zh) 2021-05-06

Family

ID=75645007

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118673 WO2021082851A1 (zh) 2019-10-31 2020-09-29 一种神经网络模型的量化方法、装置及计算机可读存储介质

Country Status (3)

Country Link
US (1) US20220398413A1 (zh)
CN (1) CN112749783A (zh)
WO (1) WO2021082851A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326985A (zh) * 2016-08-18 2017-01-11 北京旷视科技有限公司 神经网络训练方法和装置及数据处理方法和装置
CN106557812A (zh) * 2016-11-21 2017-04-05 北京大学 基于dct变换的深度卷积神经网络压缩与加速方案
US20190095795A1 (en) * 2017-03-15 2019-03-28 Samsung Electronics Co., Ltd. System and method for designing efficient super resolution deep convolutional neural networks by cascade network training, cascade network trimming, and dilated convolutions
CN110059796A (zh) * 2018-01-19 2019-07-26 杭州海康威视数字技术股份有限公司 卷积神经网络的生成方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326985A (zh) * 2016-08-18 2017-01-11 北京旷视科技有限公司 神经网络训练方法和装置及数据处理方法和装置
CN106557812A (zh) * 2016-11-21 2017-04-05 北京大学 基于dct变换的深度卷积神经网络压缩与加速方案
US20190095795A1 (en) * 2017-03-15 2019-03-28 Samsung Electronics Co., Ltd. System and method for designing efficient super resolution deep convolutional neural networks by cascade network training, cascade network trimming, and dilated convolutions
CN110059796A (zh) * 2018-01-19 2019-07-26 杭州海康威视数字技术股份有限公司 卷积神经网络的生成方法及装置

Also Published As

Publication number Publication date
CN112749783A (zh) 2021-05-04
US20220398413A1 (en) 2022-12-15

Similar Documents

Publication Publication Date Title
CN106951962B (zh) 用于神经网络的复合运算单元、方法和电子设备
JP7146952B2 (ja) データ処理方法、装置、コンピュータデバイス、及び記憶媒体
TWI796286B (zh) 一種機器學習系統的訓練方法和訓練系統
US10540145B2 (en) Method and processing apparatus for performing arithmetic operation
US20200364552A1 (en) Quantization method of improving the model inference accuracy
EP3037980A2 (en) Computer-implemented system and method for efficient sparse matrix representation and processing
CN111401518B (zh) 一种神经网络量化方法、装置及计算机可读存储介质
CN112990454B (zh) 基于集成dpu多核异构的神经网络计算加速方法及装置
US20230196142A1 (en) Information processing method and information processing system
US20220230064A1 (en) Calibration of analog circuits for neural network computing
WO2020103883A1 (zh) 执行矩阵乘法运算的方法、电路及soc
CN114861907A (zh) 数据计算方法、装置、存储介质和设备
TW202001701A (zh) 影像的量化方法及神經網路的訓練方法
CN111123833A (zh) 一种机床的热误差补偿计算方法、装置、设备和存储介质
WO2021082851A1 (zh) 一种神经网络模型的量化方法、装置及计算机可读存储介质
US20210406690A1 (en) Efficient weight clipping for neural networks
CN111767980B (zh) 模型优化方法、装置及设备
CN117454948B (zh) 一种适用于国产硬件的fp32模型转换方法
US20220405594A1 (en) Composition conversion apparatus, composition conversion method, and computer readable medium
CN116157808B (zh) 用于神经网络的组平衡式稀疏激活和联合激活权重稀疏训练的系统和方法
US20230162010A1 (en) Synthesizing Zero-Loss Low-Power Approximate DNN Accelerators With Large-Scale Search
CN118410849A (zh) 确定量化截断值的方法、设备和介质
Huang et al. Quantization and Deployment Study of Classification Models for Embedded Platforms
CN116611930A (zh) 金融产品的推荐方法、装置、处理器及电子设备
Lee et al. Development of Parameter Bit Operation Framework for Deep Neural Networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20882985

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20882985

Country of ref document: EP

Kind code of ref document: A1