CN112183742B - Neural network hybrid quantization method based on progressive quantization and Hessian information - Google Patents

Neural network hybrid quantization method based on progressive quantization and Hessian information Download PDF

Info

Publication number
CN112183742B
CN112183742B CN202010915248.6A CN202010915248A CN112183742B CN 112183742 B CN112183742 B CN 112183742B CN 202010915248 A CN202010915248 A CN 202010915248A CN 112183742 B CN112183742 B CN 112183742B
Authority
CN
China
Prior art keywords
quantization
neural network
precision
model
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010915248.6A
Other languages
Chinese (zh)
Other versions
CN112183742A (en
Inventor
王振宁
许金泉
王溢
蔡碧颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanqiang Zhishi Xiamen Technology Co ltd
Original Assignee
Nanqiang Zhishi Xiamen Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanqiang Zhishi Xiamen Technology Co ltd filed Critical Nanqiang Zhishi Xiamen Technology Co ltd
Priority to CN202010915248.6A priority Critical patent/CN112183742B/en
Publication of CN112183742A publication Critical patent/CN112183742A/en
Application granted granted Critical
Publication of CN112183742B publication Critical patent/CN112183742B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a neural network mixed quantization method based on progressive quantization and Hessian information, which comprises the following steps: dividing a sample set and a calibration set by a given image-label pair set; defining a quantization precision range selectable by each layer of the neural network; randomly selecting a quantization layer to perform bit-down quantization, and repeatedly sampling for n times to obtain n basic mixed precision models; taking the image-label pair in the calibration set as input, and carrying out a forward process on all candidate hybrid precision neural network models; calculating a Hessian approximation by using Adam second-order Momentum information; recalculating the performance evaluation index
Figure DDA0002664795920000011
N are to
Figure DDA0002664795920000012
The index is sequenced, a mixed precision strategy corresponding to the minimum value is selected as a mixed precision super-parameter combination with the best performance under the current step, the calculation cost of the model is calculated, and then the current mixed precision network model is trained; and iterating until the ending condition is met. The method can greatly reduce the optimized mixed precision search space in each step and improve the performance evaluation efficiency of the quantization model.

Description

Neural network hybrid quantization method based on progressive quantization and Hessian information
Technical Field
The invention belongs to the technical field of neural network compression and acceleration, in particular to a method for improving the calculation speed of a neural network model and reducing the memory capacity of the model by utilizing a quantization technology, which can be used in various application fields such as picture classification, target detection, super-resolution reconstruction and the like.
Background
In recent years, products of artificial intelligence gradually open the doors of thousands of households, intelligent robots, VR, unmanned vehicles and the like slowly enter eyes of people, but these devices all face the problems of low computing performance and low storage capacity, which makes many efficient algorithms not applicable to these devices.
With the development of neural network algorithms, network performance is stronger and stronger, but the network performance is brought with the consumption of computing resources and memory storage, in order to obtain a more efficient network structure and be deployed at a mobile terminal, the compression algorithm of the neural network becomes a research hotspot in recent years, five main network compression approaches are adopted, quantization, pruning, low-rank decomposition, teacher-student network and light-weight network design are adopted, quantization is to convert floating point storage and operation into fixed point storage and operation, and the floating point storage and operation which is expressed by 32 bits or 64 bits in the past is stored and operated in a form of occupying less memory space by 1bit and 2bit, so that the method is an effective technology with very good application prospect.
As described in the reference (Jacob B, kligys S, chen B, et al, quantization and training of neural networks for efficient integer-arithmetical-only reference.in Proceedings of the IEEE Conference on Computer Vision and Pattern recovery.2018.) the neural network quantization algorithm requires the determination of three main variables: quantization bit number, scale factor, and zero offset in non-uniform quantization algorithms.
Traditional quantization methods compress and accelerate by setting the same number of quantization bits for the entire network structure, despite significant advances, but still suffer from reduced network performance, especially on lower bit network structures. For example, binary network-based methods (Hubara I, coubariaux M, soudry D, et al, binary neural networks in Neural Information Processing Systems, 2016.), ternary quantization networks (Rastegari M, ordonez V, redmon J, et al, xnor-net: imagenet classification using binary convolutional neural networks in European Conference on Computer vision Springer, cham, 2016.) and low bit networks (Zhou S, wu Y, ni Z, et al, dorefa-net: training low bitwidth convolutional neural networks with low bitwidth grad. ArXiv pre arXiv:1606.06160,2016.) are limited by lower expression spaces, and neural network performance drops dramatically as the number of quantized bits decreases. Recently, neural network quantization algorithms based on reinforcement learning, evolutionary learning and derivable are sequentially proposed, and quantization network performance is improved through mixed bit precision expression. However, the performance estimation method for distinguishing the hybrid precision strategy is still based on a training mode, and in addition, the search space of the performance estimation method increases exponentially with the size of the candidate hybrid precision bit number set.
Disclosure of Invention
The invention aims to solve the problems of overlarge search space and unreasonable performance evaluation indexes generated in the processes of searching for optimal weight and activating quantized bit width in the actual deployment process of a low-bit quantized neural network model, and provides a hybrid bit quantization method based on progressive quantization and Hession information based on the idea of progressive optimization and the perception characteristics of Hession itself to noise disturbance of the neural network, which can greatly reduce the optimized hybrid precision search space in each step, and simultaneously improve the performance evaluation indexes of the hybrid bit neural network based on Hessian information and the performance evaluation efficiency of the quantized model. Compared with the traditional fixed precision, the quantization method achieves good effects on model precision and compression rate.
In order to achieve the above object, the solution of the present invention is:
a neural network hybrid quantization method based on progressive quantization and Hessian information comprises the following steps:
step 1, a marked image-label pair set is given, the set is divided into a training sample set, a test sample set and a verification sample set, and a part of the set is randomly sampled from the training sample set to serve as a calibration set; defining a selectable quantization precision range of each layer of the neural network, and giving a floating point network for training convergence and a required calculation cost constraint;
step 2, randomly selecting a quantization layer for sampling, performing bit-down quantization on all the selected quantization layers, and repeatedly sampling for n times, thereby obtaining n basic mixed precision models;
step 3, taking the image-label pairs in the calibration set as input, and mixing all candidates obtained by sampling in step 2Performing a forward process on the precision neural network model; calculation of Hessian approximation using Adam second order Momentum information
Figure BDA0002664795900000031
Based on this approximation, calculate +.>
Figure BDA0002664795900000032
Figure BDA0002664795900000033
wherein ,
Figure BDA0002664795900000034
representing calibration set data, +.>
Figure BDA0002664795900000035
Representing the parameters of the whole neural network model,/->
Figure BDA0002664795900000036
Representing Hessian information, w i Representing the trainable weight of the ith layer in the original given trained floating point model, wherein the trainable parameter of the corresponding network layer of the mixed bit model in the corresponding current step is +.>
Figure BDA0002664795900000037
Step 4, corresponding to all sample models
Figure BDA0002664795900000038
The performance indexes are ordered, and the value of the performance indexes is the smallest>
Figure BDA0002664795900000039
The corresponding mixing precision strategy is used as the mixing precision super-parameter combination with the best performance under the current step;
step 5, calculating the calculation cost of the model based on the mixed precision super-parameter combination with the best performance in the current step, and training the current mixed precision network model by using the training sample set in the step 1;
step 6, judging whether the ending condition is met, if yes, taking the current mixed precision network model as a final neural network model; otherwise, the model trained in the step 5 is used as an initialization, and the steps 2 to 5 are repeated until the ending condition is met.
In the step 2, performing bit-down quantization on the selected quantization layer refers to setting the number of quantization bits of the selected quantization layer as 1 less than the quantization bits of the corresponding quantization layer in the previous step.
In the step 2, the initial quantization bit number of each layer of the neural network is set to be the maximum value of the selectable quantization range of the layer.
In the above step 5, the calculation formula of the calculation cost of the model is:
Figure BDA00026647959000000310
wherein ,
Figure BDA00026647959000000311
and />
Figure BDA00026647959000000312
Respectively representing quantized parameter number and corresponding quantized bit number, P of ith layer in network W Representing the amount of parameters in the overall network.
In the above step 6, the ending condition means that the quantized bit number of all the quantized layers in the network is 2 or the calculation cost of the model reaches the given calculation cost.
After the scheme is adopted, compared with the prior art, the method has the following outstanding advantages:
firstly, the invention provides a progressive quantization scheme based on discrete optimization, so that the mixed bit quantization space in each iterative optimization process is greatly reduced, and the optimization efficiency is improved;
secondly, based on Hessian information, the invention leads out the performance influence index of the quantized bit on the whole neural network model, and the traditional training-based performance verification mode is abandoned by the calculation approximation of the Adam algorithm, so that the model performance verification efficiency is greatly improved. The problem that the performance of the mixed bit strategy model is difficult to measure is solved to a certain extent;
thirdly, the invention provides a method for simultaneously solving search space and performance verification in the mixed bit quantization problem for the first time. Through verification experiments on different network structures and data sets, it is found that even with a simpler random search strategy, an optimal result can be obtained.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is an algorithm flow pseudocode of the present invention;
FIG. 3 is a comparison of the present invention with a prior quantization algorithm;
FIG. 4 is a graph showing the results of the mixed bit strategy on CIFAR-10ResNet-50 of the present invention.
Detailed Description
The technical scheme and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the present invention provides a neural network hybrid quantization method based on progressive quantization and Hessian information, comprising the steps of:
step 1, a marked image-label pair set is given, the set is divided into a training sample set, a test sample set and a verification sample set, and a part of the set is randomly sampled from the training sample set to serve as a calibration set; defining the possible search space that requires a neural network, namely: each layer of the neural network can select a quantization precision range (such as 1-bit-8-bits); while given the floating point network of training convergence and the required computational cost constraints (described in detail below).
Step 2, randomly selecting a quantization layer for sampling, performing bit-down quantization on all the selected quantization layers, specifically, setting the quantization bit number of the selected quantization layer as the quantization bit number of the corresponding quantization layer in the previous step minus one, and repeatedly sampling 500 times, thereby obtaining 500 basic mixed precision models; in particular, the initial quantization bit number of each layer of the neural network is set to be the maximum value of the selectable quantization range of that layer, such as: in step 1, the quantization range is 1-8, and then initially, the quantization bit of each layer of the neural network is set to 8.
And 3, taking the image-label pair in the calibration set as input, and performing a forward process on all the candidate mixed precision neural network models obtained by sampling in the step 2. Calculation of Hessian approximation using Adam second order Momentum information
Figure BDA0002664795900000041
Based on this approximation, calculate +.>
Figure BDA0002664795900000042
Figure BDA0002664795900000051
wherein ,
Figure BDA0002664795900000052
representing calibration set data, +.>
Figure BDA0002664795900000053
Representing the parameters of the whole network model; wherein (1)>
Figure BDA0002664795900000054
The method is characterized in that Hessian information is second derivative information, adam is an algorithm for calculating second derivative information based on a batch of data and calculating a proper learning rate to perform network optimization, and Hessian is estimated and calculated directly by the aid of the second derivative information calculated in Adam, wherein the two information are approximately equal; in addition, w i Representing the trainable weight of a certain layer in the original given trained floating point model, wherein the trainable parameter of the corresponding network layer of the mixed bit model in the corresponding current step is +.>
Figure BDA0002664795900000055
Step 4, corresponding to all sample models
Figure BDA0002664795900000056
The performance indexes are ordered, and the value of the performance indexes is the smallest>
Figure BDA0002664795900000057
The corresponding mixing precision strategy is the mixing precision super-parameter combination with the best performance under the current step.
Step 5, calculating the calculation cost (cost) of the model based on the current mixed precision super-parameter combination with the best performance, wherein the specific calculation formula is as follows:
Figure BDA0002664795900000058
wherein ,
Figure BDA0002664795900000059
is the quantized parameter quantity (or floating point calculation quantity) of the ith layer in the network,/or->
Figure BDA00026647959000000510
Is the quantized bit number, P, corresponding to the ith layer in the network W Is the amount of parameters (or floating point operations) in the entire network. In order to compensate for the performance loss of the low-precision model, the training sample set in the step 1 is used for training the sample (the mixed-precision network model) corresponding to the minimum value obtained in the step 5.
And 6, if the quantized bit number of all the quantized layers in the network is 2 or the calculation cost of the model reaches the given calculation cost, terminating the algorithm. Otherwise, the trained model in the step 5 is used as an initialization, the steps 2 to 5 are repeated, and the mixing precision strategy in each iteration process and the corresponding network model trained in the step 5 are recorded.
The effects of the present invention are further described by the following simulation experiments.
1) Simulation conditions
The invention is developed on a VSCODE platform, and a deep learning framework developed is based on Pytorch. The language used mainly in the present invention is Python.
2) Emulation content
We simulated MNIST, CIFAR-10 and CIFAR-100 datasets, MNIST being a handwritten digital dataset, whose training dataset contained 60000, and test dataset contained 10000 samples. Each picture in the MNIST dataset is made up of 28 x 28 pixels, each represented by a gray value. Whereas the CIFAR-10 dataset consisted of 10 classes of 32 x 32 color pictures, containing 60000 pictures in total, each class containing 6000 pictures. Wherein 50000 pictures are used as training sets and 10000 pictures are used as test sets. The CIFAR-10 dataset was divided into 5 trained batches and 1 tested batch, each containing 10000 pictures. The pictures of the test set batch are composed of 1000 pictures randomly selected from each category, and the training set batch contains the remaining 50000 pictures in random order. The CIFAR-100 dataset is much like the CIFAR-10 dataset except that the former has 100 classes, each containing 600 images. There are 500 training images and 100 test images for each class.
In summary, the existing hybrid precision quantization algorithm only considers the design of the search algorithm, ignores the defects of overlarge search space and inaccurate performance evaluation standard of quantization precision, greatly reduces the search space of hybrid precision through progressive quantization design, and improves the accuracy of performance verification of a quantization model through staged retraining. Through experimentation, even if we use a random search algorithm, a better result can be obtained. Specifically, given a neural network structure and hardware requirements, after the algorithm processing provided by the invention, a hybrid precision neural network model meeting the given hardware requirements can be obtained, and after compiling processing, the hybrid precision neural network model can be directly applied to terminal equipment, so that the storage pressure of the terminal equipment is reduced, the calculation pressure of the terminal equipment is relieved, and meanwhile, a good result can be obtained.
Therefore, in the invention, based on the angle of discrete optimization, a hybrid precision progressive quantization method guided by Hessian information is provided, so that the search space and the performance verification time of a hybrid bit quantization model can be effectively reduced, the hybrid bit quantization model is applicable to various scale data sets and different network structures, and the final network model can obviously reduce the operation amount and the memory storage amount on the basis of keeping the precision because of the low bit expression, improve the performance of a neural network and is beneficial to deployment on a mobile terminal. In addition, the algorithm can be operated once to obtain the mixed bit model under the constraint of different hardware devices, and the applicability and the effectiveness of the algorithm are effectively improved. Specifically, with a progressive quantization scheme, our search space is reduced 19 times over conventional hybrid precision algorithms on ResNet-18. Meanwhile, under the condition that the Weight and the action are compressed 16 times, the precision of more than 90 can still be maintained.
The above embodiments are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereto, and any modification made on the basis of the technical scheme according to the technical idea of the present invention falls within the protection scope of the present invention.

Claims (5)

1. The neural network hybrid quantization method based on progressive quantization and Hessian information is characterized by comprising the following steps:
step 1, a marked image-label pair set is given, the set is divided into a training sample set, a test sample set and a verification sample set, and a part of the set is randomly sampled from the training sample set to serve as a calibration set; defining a selectable quantization precision range of each layer of the neural network, and giving a floating point network for training convergence and a required calculation cost constraint;
step 2, randomly selecting a quantization layer for sampling, performing bit-down quantization on all the selected quantization layers, and repeatedly sampling for n times, thereby obtaining n basic mixed precision models;
step 3, taking the image-label pair in the calibration set as input, and carrying out a forward process on all candidate hybrid precision neural network models obtained by sampling in the step 2; calculation of Hessian approximation using Adam second order Momentum information
Figure FDA0002664795890000011
Based on this approximation, calculate/>
Figure FDA0002664795890000012
Figure FDA0002664795890000013
wherein ,
Figure FDA0002664795890000014
representing calibration set data, +.>
Figure FDA0002664795890000015
Representing the model parameters of the whole neural network, w i Representing the trainable weight of the ith layer in the original given trained floating point model, wherein the trainable parameter of the corresponding network layer of the mixed bit model in the corresponding current step is +.>
Figure FDA0002664795890000016
Step 4, corresponding to all sample models
Figure FDA0002664795890000017
The performance indexes are ordered, and the value of the performance indexes is the smallest>
Figure FDA0002664795890000018
The corresponding mixing precision strategy is used as the mixing precision super-parameter combination with the best performance under the current step;
step 5, calculating the calculation cost of the model based on the mixed precision super-parameter combination with the best performance in the current step, and training the current mixed precision network model by using the training sample set in the step 1;
step 6, judging whether the ending condition is met, if yes, taking the current mixed precision network model as a final neural network model; otherwise, the model trained in the step 5 is used as an initialization, and the steps 2 to 5 are repeated until the ending condition is met.
2. The neural network hybrid quantization method based on progressive quantization and Hessian information as recited in claim 1, wherein: in the step 2, performing bit-down quantization on the selected quantization layer refers to setting the number of quantization bits of the selected quantization layer as 1 less than the quantization bits of the corresponding quantization layer in the previous step.
3. The neural network hybrid quantization method based on progressive quantization and Hessian information as recited in claim 1, wherein: in the step 2, the initial quantization bit number of each layer of the neural network is set as the maximum value of the selectable quantization range of the layer.
4. The neural network hybrid quantization method based on progressive quantization and Hessian information as recited in claim 1, wherein: in the step 5, the calculation formula of the calculation cost of the model is:
Figure FDA0002664795890000021
wherein ,
Figure FDA0002664795890000022
and />
Figure FDA0002664795890000023
Respectively representing quantized parameter number and corresponding quantized bit number, P of ith layer in network W Representing the amount of parameters in the overall network.
5. The neural network hybrid quantization method based on progressive quantization and Hessian information as recited in claim 1, wherein: in the step 6, the ending condition means that the quantized bit number of all the quantized layers in the network is 2 or the calculation cost of the model reaches the given calculation cost.
CN202010915248.6A 2020-09-03 2020-09-03 Neural network hybrid quantization method based on progressive quantization and Hessian information Active CN112183742B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010915248.6A CN112183742B (en) 2020-09-03 2020-09-03 Neural network hybrid quantization method based on progressive quantization and Hessian information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010915248.6A CN112183742B (en) 2020-09-03 2020-09-03 Neural network hybrid quantization method based on progressive quantization and Hessian information

Publications (2)

Publication Number Publication Date
CN112183742A CN112183742A (en) 2021-01-05
CN112183742B true CN112183742B (en) 2023-05-12

Family

ID=73925052

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010915248.6A Active CN112183742B (en) 2020-09-03 2020-09-03 Neural network hybrid quantization method based on progressive quantization and Hessian information

Country Status (1)

Country Link
CN (1) CN112183742B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112908446B (en) * 2021-03-20 2022-03-22 张磊 Automatic mixing control method for liquid medicine in endocrinology department
CN113222148B (en) * 2021-05-20 2022-01-11 浙江大学 Neural network reasoning acceleration method for material identification
CN113256657B (en) * 2021-06-03 2022-11-04 上海交通大学烟台信息技术研究院 Efficient medical image segmentation method and system, terminal and medium
CN113762403B (en) * 2021-09-14 2023-09-05 杭州海康威视数字技术股份有限公司 Image processing model quantization method, device, electronic equipment and storage medium
CN116611493B (en) * 2023-05-16 2024-06-07 上海交通大学 Hardware perception hybrid precision quantization method and system based on greedy search

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107967517A (en) * 2016-10-19 2018-04-27 三星电子株式会社 The method and apparatus quantified for neutral net
CN110674326A (en) * 2019-08-06 2020-01-10 厦门大学 Neural network structure retrieval method based on polynomial distribution learning
CN111062474A (en) * 2018-10-16 2020-04-24 北京大学 Neural network optimization method for solving and improving adjacent computer machines
CN111553193A (en) * 2020-04-01 2020-08-18 东南大学 Visual SLAM closed-loop detection method based on lightweight deep neural network

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11645493B2 (en) * 2018-05-04 2023-05-09 Microsoft Technology Licensing, Llc Flow for quantized neural networks

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107967517A (en) * 2016-10-19 2018-04-27 三星电子株式会社 The method and apparatus quantified for neutral net
CN111062474A (en) * 2018-10-16 2020-04-24 北京大学 Neural network optimization method for solving and improving adjacent computer machines
CN110674326A (en) * 2019-08-06 2020-01-10 厦门大学 Neural network structure retrieval method based on polynomial distribution learning
CN111553193A (en) * 2020-04-01 2020-08-18 东南大学 Visual SLAM closed-loop detection method based on lightweight deep neural network

Also Published As

Publication number Publication date
CN112183742A (en) 2021-01-05

Similar Documents

Publication Publication Date Title
CN112183742B (en) Neural network hybrid quantization method based on progressive quantization and Hessian information
CN109711413B (en) Image semantic segmentation method based on deep learning
CN108764471B (en) Neural network cross-layer pruning method based on feature redundancy analysis
CN108133188B (en) Behavior identification method based on motion history image and convolutional neural network
CN108900346B (en) Wireless network flow prediction method based on LSTM network
CN109711426B (en) Pathological image classification device and method based on GAN and transfer learning
CN112116030A (en) Image classification method based on vector standardization and knowledge distillation
CN111310672A (en) Video emotion recognition method, device and medium based on time sequence multi-model fusion modeling
CN111460936A (en) Remote sensing image building extraction method, system and electronic equipment based on U-Net network
CN115393396B (en) Unmanned aerial vehicle target tracking method based on mask pre-training
CN116503676B (en) Picture classification method and system based on knowledge distillation small sample increment learning
CN113591978B (en) Confidence penalty regularization-based self-knowledge distillation image classification method, device and storage medium
CN112686376A (en) Node representation method based on timing diagram neural network and incremental learning method
CN113988357B (en) Advanced learning-based high-rise building wind induced response prediction method and device
CN110647990A (en) Cutting method of deep convolutional neural network model based on grey correlation analysis
CN115358305A (en) Incremental learning robustness improving method based on iterative generation of boundary samples
CN114819143A (en) Model compression method suitable for communication network field maintenance
CN115902806A (en) Multi-mode-based radar echo extrapolation method
CN111325259A (en) Remote sensing image classification method based on deep learning and binary coding
CN114882278A (en) Tire pattern classification method and device based on attention mechanism and transfer learning
CN114004152A (en) Multi-wind-field wind speed space-time prediction method based on graph convolution and recurrent neural network
CN111783688B (en) Remote sensing image scene classification method based on convolutional neural network
CN117636183A (en) Small sample remote sensing image classification method based on self-supervision pre-training
CN113643303A (en) Three-dimensional image segmentation method based on two-way attention coding and decoding network
CN112132207A (en) Target detection neural network construction method based on multi-branch feature mapping

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant