WO2022006919A1 - Procédé et système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif - Google Patents

Procédé et système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif Download PDF

Info

Publication number
WO2022006919A1
WO2022006919A1 PCT/CN2020/101550 CN2020101550W WO2022006919A1 WO 2022006919 A1 WO2022006919 A1 WO 2022006919A1 CN 2020101550 W CN2020101550 W CN 2020101550W WO 2022006919 A1 WO2022006919 A1 WO 2022006919A1
Authority
WO
WIPO (PCT)
Prior art keywords
quantization
weight
activation
point
fixed
Prior art date
Application number
PCT/CN2020/101550
Other languages
English (en)
Chinese (zh)
Inventor
王培松
程健
Original Assignee
中国科学院自动化研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院自动化研究所 filed Critical 中国科学院自动化研究所
Publication of WO2022006919A1 publication Critical patent/WO2022006919A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention belongs to the field of data processing, and particularly relates to a post-training quantization method and system of a convolutional neural network based on activation fixed-point fitting.
  • deep convolutional neural networks have made breakthroughs in many fields such as image processing, computer vision, and deep learning. In various fields such as image classification, target detection, and face recognition, they have approached or even surpassed human recognition accuracy. Therefore, deep convolutional neural networks have been widely used in assisted driving, e-commerce, video surveillance and other industries.
  • Low-ratio specific point quantization is one of the methods of deep convolutional neural network compression. Since fixed-point quantization can be applied to a variety of network structures and is very hardware-friendly, low-ratio specific point quantization has become one of the important methods in the field of network compression. However, at present, the quantization of the deep convolutional neural network at a low ratio to a specific point often requires the original training data. After a large number of network retraining, a relatively high accuracy can be obtained, that is, training-aware Quantization.
  • the present invention provides a convolutional neural network based on activation fixed-point fitting Post-training quantization method, the quantization method includes:
  • Step S10 network weight matrix fitting, obtaining the weight matrix of each layer of the original convolutional neural network, and performing low-to-specific point quantization of the weight matrix of each layer respectively, to obtain the fixed-point weight matrix and weight quantization scale factor of each layer of the network ;
  • Step S20 the network is fitted for the first activation matrix, a set of verification data is obtained, and an optimization objective function from input activation to output activation is constructed based on the fixed-point weight matrix and weight quantization scale factor of each layer of the network, and iterative Optimize the fixed-point weight matrix and the weight quantization scale factor to obtain the weight fixed-point quantization convolutional neural network;
  • Step S30 the network is fitted for the second activation matrix, based on the set of verification data and the weight fixed-point quantized convolutional neural network, the activation quantization scale factor is solved, and the quantized weight-activated fixed-point quantized convolutional neural network is obtained .
  • step S10 "respectively perform low-to-specific point quantization of the weight matrix of each layer to obtain the fixed-point weight matrix and weight quantization scale factor of each layer of the network", and the method is:
  • Step S11 dividing the weight matrix of the current layer into weight vectors by row, and dividing the maximum absolute value of the weight vector by the maximum value of the fixed-point weight quantization as the initial weight quantization scale factor of each row;
  • Step S12 constructing the weight quantization error function between the weight vector, the weight quantization fixed-point number and the initial weight quantization scale factor
  • Step S13 based on the weight quantization error function, iteratively perform the solution of the weight quantization fixed-point number and the weight quantization scale factor, until the weight quantization error function value is lower than the set threshold, and obtain the fixed-point weight of the current layer.
  • Matrix and weight quantization scale factor
  • step S14 the fixed-point weight matrix and the weight quantization scale factor of each layer of the network are obtained respectively by the method of step S11-step S13.
  • the weight quantization error function between the weight vector, the weight quantization fixed-point number and the initial weight quantization scale factor is:
  • W is a two-dimensional floating-point number matrix of C out ⁇ K
  • W i is the weight vector of the i-th row of the matrix
  • C out is the number of output channels of the current layer
  • K C in *K h *K w
  • K h and K w are the height and width of the convolution kernel respectively
  • K C in
  • C in is the number of input channels of the current layer
  • Q i represents the weight value Quantization fixed point number
  • ⁇ ii represents the initial weight quantization scale factor
  • step S20 includes:
  • Step S21 inputting the obtained set of verification data into the original convolutional neural network to obtain the input activation and output activation of the current layer;
  • Step S22 based on the input activation and output activation of the current layer, construct a linear least squares optimization objective function under the fixed-point constraint of the current layer;
  • Step S23 splitting the linear least squares optimization objective function according to the row of the fixed-point weight matrix, and iteratively carrying out the optimization of the weight quantization scale factor and the fixed-point weight, until the output activation quantization error is less than the set threshold, obtain: The optimized fixed-point weight matrix and weight quantization scale factor of the current layer;
  • step S24 the optimized fixed-point weight matrix and weight quantization scale factor of each layer of the network are obtained respectively by the methods of steps S21-S23, and the weight fixed-point quantization convolutional neural network is obtained.
  • the linear least squares optimization objective function is:
  • X and Y represent the input activation and output activation of the current layer, respectively
  • Q is the fixed-point weight matrix of the current layer
  • ⁇ is the weight quantization scale factor of the current layer
  • the set of verification data is a small amount of training data of the original convolutional neural network, or a small amount of other data whose distribution is similar to the training data of the original convolutional neural network, or artificially generated simulation data, or randomly generated random data.
  • step S30 includes:
  • Step S31 inputting the set of verification data into the weighted fixed-point convolutional neural network, and obtaining the output activations of each layer to form an output activation vector;
  • Step S32 divide the maximum absolute value of the output activation vector by the maximum value of the activation quantization fixed-point number as the initial activation quantization scale factor
  • Step S33 constructing the activation quantization error function between the output activation vector, the activation quantization function and the initial activation quantization scale factor;
  • Step S34 based on the activation quantization error function, iteratively carry out the solution of the fixed-point activation vector and the activation quantization scale factor, until the activation quantization error function value is lower than the set threshold, obtain the optimized fixed-point activation vector and activation quantization scale factor, Get quantized weights - activate a fixed-point quantized convolutional neural network.
  • the activation quantization function is:
  • q min and q max represent the minimum and maximum activation quantization fixed-point numbers, respectively, ⁇ represents the activation quantization scale factor, x i represents the output activation of the i-th layer, round(*) represents the rounding operation, and clip(*) represents the Threshold truncation operation.
  • the activation quantization error function is:
  • ⁇ represents the activation quantization scale factor
  • x represents the output activation vector
  • activation quantization function represents the activation quantization function
  • a post-training quantization system of convolutional neural network based on activation fixed-point fitting is proposed.
  • the quantization system includes network weights.
  • the network weight matrix fitting module is configured to obtain the weight matrices of each layer of the original convolutional neural network, and perform low-ratio specific point quantization of the weight matrix of each layer respectively, and obtain the fixed-point weight matrix and weight quantization of each layer of the network. scale factor;
  • the network activates the matrix fitting module for the first time, obtains a set of verification data, constructs an optimization objective function from input activation to output activation based on the fixed-point weight matrix and weight quantization scale factor of each layer of the network, and iterates Optimize the fixed-point weight matrix and the weight quantization scale factor to obtain the weight fixed-point quantization convolutional neural network;
  • the second activation matrix fitting of the network is configured to solve the activation quantization scale factor based on the set of verification data and the weight fixed-point quantized convolutional neural network, and obtain the quantized weight-activated fixed-point quantized convolutional neural network
  • the internet
  • the output module is configured to output the obtained quantized weight-activated fixed-point quantized convolutional neural network.
  • a storage device wherein a plurality of programs are stored, and the programs are adapted to be loaded and executed by a processor to realize the above-mentioned post-training quantization method of convolutional neural network based on activation fixed-point fitting .
  • a processing device including a processor and a storage device; the processor is suitable for executing various programs; the storage device is suitable for storing multiple programs; the program is suitable for Loaded and executed by a processor to implement the above-described post-training quantization method for a convolutional neural network based on activation fixed-point fitting.
  • the present invention is based on the post-training quantization method of the convolutional neural network of activation fixed-point fitting, by quantizing the weight parameter matrix and input activation of the convolutional layer of the convolutional neural network and the fully connected layer and the input activation, using fixed-point number storage Replacing the original floating-point number storage, and using fixed-point number operations to replace the original floating-point number operations, can achieve optimized acceleration and compression after deep convolutional neural network training.
  • the present invention is based on the post-training quantization method of the convolutional neural network that activates fixed-point fitting, the quantization process does not need to use data to retrain, the quantization process requires less computing resources, the use threshold is low, and the quantization speed is fast.
  • FIG. 1 is a schematic flowchart of a quantization method after training of a convolutional neural network based on activation fixed-point fitting of the present invention
  • FIG. 2 is a schematic diagram of the image classification process of the convolutional neural network according to an embodiment of the quantization method after the training of the convolutional neural network based on activation fixed-point fitting of the present invention
  • FIG. 3 is a schematic diagram of a convolution operation of a convolutional neural network in an image classification process according to an embodiment of the post-training quantization method of a convolutional neural network based on activation fixed-point fitting according to the present invention.
  • a post-training quantization method of a convolutional neural network based on activation fixed-point fitting of the present invention includes:
  • Step S10 network weight matrix fitting, obtaining the weight matrix of each layer of the original convolutional neural network, and performing low-to-specific point quantization of the weight matrix of each layer respectively, to obtain the fixed-point weight matrix and weight quantization scale factor of each layer of the network ;
  • Step S20 the network is fitted for the first activation matrix, a set of verification data is obtained, and an optimization objective function from input activation to output activation is constructed based on the fixed-point weight matrix and weight quantization scale factor of each layer of the network, and iterative Optimize the fixed-point weight matrix and the weight quantization scale factor to obtain the weight fixed-point quantization convolutional neural network;
  • Step S30 the network is fitted for the second activation matrix, based on the set of verification data and the weight fixed-point quantized convolutional neural network, the activation quantization scale factor is solved, and the quantized weight-activated fixed-point quantized convolutional neural network is obtained .
  • FIG. 2 it is a schematic diagram of an image classification process of a convolutional neural network according to an embodiment of the quantization method after training of a convolutional neural network based on activation fixed-point fitting, wherein the convolutional neural network includes a plurality of convolutional layers and multiple fully-connected layers, the input image is processed by convolutional layers and fully-connected layers to obtain classification results.
  • each convolutional layer is There is a group of convolution kernels, which together form the weight tensor of the layer.
  • the convolution kernel can be set to 3 ⁇ 3; the processing method of the convolution layer is to use the input of the convolution kernel to the layer
  • the feature map is subjected to convolution operation (that is, the multiplication and summation of each convolution kernel and the corresponding element of the convolution area of each position of the input feature map), and the output feature map of the corresponding layer is obtained.
  • the post-training quantization method for a convolutional neural network based on activation fixed-point fitting includes steps S10 to S30, and each step is described in detail as follows:
  • Step S10 network weight matrix fitting, obtaining the weight matrix of each layer of the original convolutional neural network, and performing low-to-specific point quantization of the weight matrix of each layer respectively, to obtain the fixed-point weight matrix and weight quantization scale factor of each layer of the network .
  • the quantization scale factor of the weight to be calculated is ⁇ , which is a floating-point diagonal matrix of C out ⁇ C out ; it is solved by minimizing the optimization problem of the quantization error function between W, Q and ⁇ , that is, solving function.
  • step S11 the weight matrix of the current layer is divided into weight vectors by row, and the maximum absolute value of the weight vector is divided by the maximum value of the fixed-point weight quantization as the initial weight quantization scale factor of each row, as shown in the formula: (1) shows:
  • ⁇ ii is the initial weight quantization scale factor
  • W i is the weight vector of the i-th row obtained by dividing the weight matrix of the current layer into rows
  • Q max is the maximum value of the fixed-point weight quantization
  • ) represents the maximum absolute value of W i.
  • Step S12 constructing a weight quantization error function among the weight vector, the weight quantization fixed-point number, and the initial weight quantization scale factor.
  • W is a two-dimensional floating-point number matrix of C out ⁇ K
  • W i is the weight vector of the i-th row of the matrix
  • C out is the number of output channels of the current layer
  • K C in *K h *K w
  • K h and K w are the height and width of the convolution kernel respectively
  • K C in
  • C in is the number of input channels of the current layer
  • Q i represents the weight value Quantization fixed point number
  • ⁇ ii represents the initial weight quantization scale factor
  • Step S13 based on the weight quantization error function, iteratively perform the solution of the weight quantization fixed-point number and the weight quantization scale factor, until the weight quantization error function value is lower than the set threshold, and obtain the fixed-point weight of the current layer. Matrices and weights quantize scale factors.
  • Q min and Q max represent the minimum and maximum value of the fixed-point weight quantization, respectively, round(*) represents the rounding operation, and clip(*) represents the threshold truncation operation.
  • step S14 the fixed-point weight matrix and the weight quantization scale factor of each layer of the network are obtained respectively by the method of step S11-step S13.
  • Step S20 the network is fitted for the first activation matrix, a set of verification data is obtained, and an optimization objective function from input activation to output activation is constructed based on the fixed-point weight matrix and weight quantization scale factor of each layer of the network, and iterative The fixed-point weight matrix and the weight quantization scale factor are optimized to obtain the weight fixed-point quantization convolutional neural network.
  • a fixed-point function fitting optimization objective from input activation to output activation is constructed, and the fixed-point weight matrix and weight quantization scale factor are further optimized to obtain a weighted fixed-point convolutional neural network.
  • Step S21 input the obtained set of verification data into the original convolutional neural network, and obtain the input activation X and output activation Y of the current layer.
  • Step S22 based on the input activation X and output activation Y of the current layer, construct a linear least squares optimization objective function under the fixed-point constraint of the current layer, as shown in formula (5):
  • X and Y represent the input activation and output activation of the current layer, respectively
  • Q is the fixed-point weight matrix of the current layer
  • ⁇ is the weight quantization scale factor of the current layer
  • Step S23 splitting the linear least squares optimization objective function according to the row of the fixed-point weight matrix, and iteratively carrying out the optimization of the weight quantization scale factor and the fixed-point weight, until the output activation quantization error is less than the set threshold, obtain: The optimized fixed-point weight matrix and weight quantization scale factor of the current layer.
  • step S10 The optimized fixed-point weight matrix and weight quantization scale factor obtained in step S10 are used as the initialization values of q and ⁇ , and q is fixed to solve ⁇ , as shown in formula (8):
  • the approximate solution of ⁇ and q can be obtained by iterating the above process of solving ⁇ and q.
  • step S24 the optimized fixed-point weight matrix and weight quantization scale factor of each layer of the network are obtained respectively by the methods of steps S21-S23, and the weight fixed-point quantization convolutional neural network is obtained.
  • Step S30 the network is fitted for the second activation matrix, based on the set of verification data and the weight fixed-point quantized convolutional neural network, the activation quantization scale factor is solved, and the quantized weight-activated fixed-point quantized convolutional neural network is obtained .
  • Step S31 Input the set of verification data into the weighted fixed-point convolutional neural network, and obtain the output activations of each layer to form an output activation vector.
  • Step S32 divide the maximum absolute value of the output activation vector by the maximum value of the activation quantization fixed-point number as the initial activation quantization scale factor, as shown in formula (11):
  • ⁇ is the initial activation quantization scale factor
  • x is the output activation vector
  • q max is the maximum value of the activation quantization fixed-point number
  • ) represents the maximum absolute value of x.
  • Step S33 constructing the activation quantization error function between the output activation vector, activation quantization function and initial activation quantization scale factor, as shown in formula (12):
  • ⁇ represents the activation quantization scale factor
  • x represents the output activation vector
  • activation quantization function represents the activation quantization function
  • the activation quantization function is shown in formula (13):
  • q min and q max represent the minimum and maximum activation quantization fixed-point numbers, respectively, ⁇ represents the activation quantization scale factor, x i represents the output activation of the i-th layer, round(*) represents the rounding operation, and clip(*) represents the Threshold truncation operation.
  • Step S34 based on the activation quantization error function, iteratively carry out the solution of the fixed-point activation vector and the activation quantization scale factor, until the activation quantization error function value is lower than the set threshold, obtain the optimized fixed-point activation vector and activation quantization scale factor, Get quantized weights - activate a fixed-point quantized convolutional neural network.
  • the verification data used in steps S20 and S30 is a small amount of training data of the original convolutional neural network, or a small amount of other data with a distribution similar to the training data of the original convolutional neural network, or artificial Generated simulation data, or randomly generated random data.
  • the floating-point matrix multiplication of the original convolutional neural network can be converted into a fixed-point matrix multiplication, and the parameter matrix can be replaced by a fixed-point matrix during storage, so it can significantly reduce the computational overhead and storage capacity. running speed.
  • the invention quantizes the weights and activations of the deep convolutional neural network after training, converts the weights and activations from 32-bit floating point numbers to low-bit integer values, and stores the weights in a low-to-specific point format to achieve compression.
  • the convolution operation is also converted from the original floating-point multiplication and addition operation to a low-ratio specific point operation, so as to achieve the purpose of accelerating the forward inference speed of the network.
  • the present invention is mainly oriented to post-training quantization, that is, it is not necessary to use training data to retrain or fine-tune the post-quantization network. Therefore, the present invention can be easily generalized to quantization in the training of the network.
  • the method provided by the present invention can realize the acceleration and compression of the deep convolutional neural network, which is different from the previous post-training quantization method in which the weights and activations are split, and the quantization of the weights and activations is determined separately through manually selected criteria.
  • Scale factor one of the advantages of the method provided by the present invention is to provide a post-training quantization method and calculation scheme based on the fitting of a specific point with a low activation ratio.
  • the present invention directly learns the low-bit mapping function from the input activation to the output activation, which can ensure that the output of the convolution is similar before and after the weight quantization. Therefore, the model accuracy of the quantization method provided by the present invention after training is much higher than previous training. Post-quantification scheme.
  • the selected weights and the number of activated quantization bits are both 4 bits, and the method of the present invention is used to quantize the ResNet18 deep convolutional neural network after training, and obtain the weight- Activates the fixed-point quantized ResNet18 deep convolutional neural network.
  • the storage space occupied by the ResNet18 deep convolutional neural network with the weight-activated fixed-point quantization obtained after being processed by the method of the present invention is reduced to at least 1/4 of the original, and the calculation is converted from the original 32-bit floating-point operation to 4-bit operation.
  • the test accuracy on ImageNet, a large-scale image classification task is also the highest among known post-training quantization networks.
  • the post-training quantization system of convolutional neural network based on activation fixed-point fitting is based on the above-mentioned post-training quantization method of convolutional neural network based on activation fixed-point fitting, the quantization system includes a network weight matrix fitting module , the first activation matrix fitting module of the network, the second activation matrix fitting and output module of the network;
  • the network weight matrix fitting module is configured to obtain the weight matrices of each layer of the original convolutional neural network, and perform low-ratio specific point quantization of the weight matrix of each layer respectively, and obtain the fixed-point weight matrix and weight quantization of each layer of the network. scale factor;
  • the network activates the matrix fitting module for the first time, obtains a set of verification data, constructs an optimization objective function from input activation to output activation based on the fixed-point weight matrix and weight quantization scale factor of each layer of the network, and iterates Optimize the fixed-point weight matrix and the weight quantization scale factor to obtain the weight fixed-point quantization convolutional neural network;
  • the second activation matrix fitting of the network is configured to solve the activation quantization scale factor based on the set of verification data and the weight fixed-point quantized convolutional neural network, and obtain the quantized weight-activated fixed-point quantized convolutional neural network
  • the internet
  • the output module is configured to output the obtained quantized weight-activated fixed-point quantized convolutional neural network.
  • the post-training quantization system of the convolutional neural network based on activation fixed-point fitting provided by the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be allocated as required. It is completed by different functional modules, that is, the modules or steps in the embodiments of the present invention are decomposed or combined. For example, the modules in the above embodiments can be combined into one module, and can also be further split into multiple sub-modules to complete the above description. all or part of the functions.
  • the names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and should not be regarded as an improper limitation of the present invention.
  • a storage device stores a plurality of programs, and the programs are adapted to be loaded and executed by a processor to implement the above-mentioned post-training quantization method of a convolutional neural network based on activation fixed-point fitting.
  • a processing device includes a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded and executed by the processor In order to realize the above-mentioned quantization method after training of convolutional neural network based on activation fixed-point fitting.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé et un système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif, qui vise à résoudre le problème de la technologie existante dans laquelle la quantification post-apprentissage d'un réseau neuronal convolutif ne peut pas être mise en œuvre au moyen d'un procédé de quantification plus efficace à débit binaire faible. Le procédé de quantification consiste à : effectuer une quantification de point fixe à débit binaire faible sur une matrice de pondération de chaque couche d'un réseau neuronal convolutif d'origine ; obtenir un groupe de données de vérification, construire une fonction cible optimisée d'une activation d'entrée vers une activation de sortie, optimiser de manière itérative une matrice de poids à point fixe et un facteur d'échelle de quantification de poids, puis obtenir un réseau neuronal convolutif de quantification à point fixe de poids ; et d'après les données de vérification et le réseau neuronal convolutif de quantification à point fixe de poids, résoudre un facteur d'échelle de quantification d'activation, puis obtenir un réseau neuronal convolutif à point fixe activé par le poids. L'apprentissage direct d'une fonction de mappage à faible débit binaire d'une activation d'entrée vers une activation de sortie garantit que la sortie de convolution avant et après la quantification de poids est similaire, que la précision d'un modèle quantifié est élevée et que le processus de quantification ne nécessite pas l'utilisation de données pour un apprentissage.
PCT/CN2020/101550 2020-07-10 2020-07-13 Procédé et système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif WO2022006919A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010660722.5 2020-07-10
CN202010660722.5A CN111783961A (zh) 2020-07-10 2020-07-10 基于激活定点拟合的卷积神经网络训练后量化方法及系统

Publications (1)

Publication Number Publication Date
WO2022006919A1 true WO2022006919A1 (fr) 2022-01-13

Family

ID=72767163

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/101550 WO2022006919A1 (fr) 2020-07-10 2020-07-13 Procédé et système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif

Country Status (2)

Country Link
CN (1) CN111783961A (fr)
WO (1) WO2022006919A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115859746A (zh) * 2023-02-13 2023-03-28 无锡祝融航空航天科技有限公司 一种基于深度学习的铜材料增材制造成形精度控制方法
CN116720563A (zh) * 2022-09-19 2023-09-08 荣耀终端有限公司 一种提升定点神经网络模型精度的方法、装置及电子设备

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112598020A (zh) * 2020-11-24 2021-04-02 深兰人工智能(深圳)有限公司 目标识别方法及系统
CN112766456B (zh) * 2020-12-31 2023-12-26 平安科技（深圳）有限公司 浮点型深度神经网络的量化方法、装置、设备及存储介质
CN112668702B (zh) * 2021-01-15 2023-09-19 北京格灵深瞳信息技术股份有限公司 定点化参数优化方法、系统、及终端、存储介质
CN113080988B (zh) * 2021-03-26 2024-01-16 京东方科技集团股份有限公司 基于注意力机制的12导联心电图整体分类方法和装置
WO2022216109A1 (fr) * 2021-04-09 2022-10-13 Samsung Electronics Co., Ltd. Procédé et dispositif électronique de quantification d'un modèle de réseau neuronal profond (rnp)
CN113887721B (zh) * 2021-09-29 2024-02-27 中国科学技术大学 一种语音识别任务中的训练后量化压缩方法及系统
CN113673532B (zh) * 2021-10-21 2022-04-22 北京科技大学 基于量化模型的目标检测方法及装置
WO2023137710A1 (fr) * 2022-01-21 2023-07-27 深圳市大疆创新科技有限公司 Procédé d'entraînement de réseau neuronal, procédé et dispositif de traitement d'images, système, et support

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107480770A (zh) * 2017-07-27 2017-12-15 中国科学院自动化研究所 可调节量化位宽的神经网络量化与压缩的方法及装置
CN108334945A (zh) * 2018-01-30 2018-07-27 中国科学院自动化研究所 深度神经网络的加速与压缩方法及装置
CN109409514A (zh) * 2018-11-02 2019-03-01 广州市百果园信息技术有限公司 卷积神经网络的定点运算方法、装置、设备和存储介质
US20190138882A1 (en) * 2017-11-07 2019-05-09 Samusung Electronics Co., Ltd. Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization
CN111126557A (zh) * 2018-10-31 2020-05-08 阿里巴巴集团控股有限公司 神经网络量化、应用方法、装置和计算设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107480770A (zh) * 2017-07-27 2017-12-15 中国科学院自动化研究所 可调节量化位宽的神经网络量化与压缩的方法及装置
US20190138882A1 (en) * 2017-11-07 2019-05-09 Samusung Electronics Co., Ltd. Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization
CN108334945A (zh) * 2018-01-30 2018-07-27 中国科学院自动化研究所 深度神经网络的加速与压缩方法及装置
CN111126557A (zh) * 2018-10-31 2020-05-08 阿里巴巴集团控股有限公司 神经网络量化、应用方法、装置和计算设备
CN109409514A (zh) * 2018-11-02 2019-03-01 广州市百果园信息技术有限公司 卷积神经网络的定点运算方法、装置、设备和存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116720563A (zh) * 2022-09-19 2023-09-08 荣耀终端有限公司 一种提升定点神经网络模型精度的方法、装置及电子设备
CN116720563B (zh) * 2022-09-19 2024-03-29 荣耀终端有限公司 一种提升定点神经网络模型精度的方法、装置及电子设备
CN115859746A (zh) * 2023-02-13 2023-03-28 无锡祝融航空航天科技有限公司 一种基于深度学习的铜材料增材制造成形精度控制方法
CN115859746B (zh) * 2023-02-13 2023-04-25 无锡祝融航空航天科技有限公司 一种基于深度学习的铜材料增材制造成形精度控制方法

Also Published As

Publication number Publication date
CN111783961A (zh) 2020-10-16

Similar Documents

Publication Publication Date Title
WO2022006919A1 (fr) Procédé et système basés sur un ajustement de point fixe d'activation pour la quantification post-apprentissage d'un réseau neuronal convolutif
CN109754066B (zh) 用于生成定点型神经网络的方法和装置
US11429862B2 (en) Dynamic adaptation of deep neural networks
JP7315748B2 (ja) データ識別器訓練方法、データ識別器訓練装置、プログラム及び訓練方法
EP3340129B1 (fr) Réseau neuronal artificiel élagage basé sur la classe
WO2019091020A1 (fr) Procédé de stockage de données de poids, et processeur de réseau neuronal basé sur le procédé
US20210019630A1 (en) Loss-error-aware quantization of a low-bit neural network
CN108334945B (zh) 深度神经网络的加速与压缩方法及装置
CN113610232B (zh) 网络模型量化方法、装置、计算机设备以及存储介质
KR102152374B1 (ko) 인공신경망의 비트 양자화 방법 및 시스템
WO2022105108A1 (fr) Procédé, appareil et dispositif de classification de données de réseau, et support de stockage lisible
US20220414432A1 (en) Method and system for splitting and bit-width assignment of deep learning models for inference on distributed systems
CN110647974A (zh) 深度神经网络中的网络层运算方法及装置
WO2021042857A1 (fr) Procédé de traitement et appareil de traitement pour modèle de segmentation d'image
KR20190130443A (ko) 뉴럴 네트워크의 양자화 방법 및 장치
CN115952832A (zh) 自适应模型量化方法及装置、存储介质及电子装置
CN112598062A (zh) 一种图像识别方法和装置
CN115496144A (zh) 配电网运行场景确定方法、装置、计算机设备和存储介质
JP6935868B2 (ja) 画像認識装置、画像認識方法、およびプログラム
US20210125066A1 (en) Quantized architecture search for machine learning models
CN116956997A (zh) 用于时序数据处理的lstm模型量化重训练方法、系统及设备
WO2021248544A1 (fr) Bloc de calcul à faibles ressources pour réseau de neurones entraîné
US20230306255A1 (en) Method and system for smooth training of a quantized neural network
CN110717402A (zh) 一种基于层级优化度量学习的行人再识别方法
Xu et al. Eq-net: Elastic quantization neural networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20943846

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20943846

Country of ref document: EP

Kind code of ref document: A1