CN112784964A - Image classification method based on bridging knowledge distillation convolution neural network - Google Patents

Image classification method based on bridging knowledge distillation convolution neural network Download PDF

Info

Publication number
CN112784964A
CN112784964A CN202110107120.1A CN202110107120A CN112784964A CN 112784964 A CN112784964 A CN 112784964A CN 202110107120 A CN202110107120 A CN 202110107120A CN 112784964 A CN112784964 A CN 112784964A
Authority
CN
China
Prior art keywords
network
layer
teacher
student
student network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110107120.1A
Other languages
Chinese (zh)
Inventor
杜兰
王震
宋佳伦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202110107120.1A priority Critical patent/CN112784964A/en
Publication of CN112784964A publication Critical patent/CN112784964A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image classification method based on a bridging knowledge distillation convolution neural network, which mainly solves the problem of low accuracy of student network image classification caused by information loss in the knowledge distillation process in the prior art, and comprises the following implementation steps: (1) constructing a teacher network and a student network; (2) generating a training set: (3) training a teacher network: (4) constructing a bridging structure: (5) training a student network: (6) and classifying the image to be classified. According to the invention, a bridge structure is constructed between the teacher network and the student network, and the student network is trained according to the KL divergence loss function and the cross entropy loss function, so that the student network has higher image classification accuracy and lower terminal deployment requirements at the same time, and can be used for classifying and identifying images on low-computing-power and low-storage terminal equipment.

Description

Image classification method based on bridging knowledge distillation convolution neural network
Technical Field
The invention belongs to the technical field of image processing, and further relates to an image classification method based on a bridging knowledge distillation convolution neural network in the technical field of image classification. The method can be used for classifying and identifying the images on low-computing-power and low-storage terminal equipment.
Background
The most classical network in the convolutional neural network is ResNet, and by introducing 'jump connection', ResNet effectively relieves the problem of gradient disappearance in neural network training and successfully trains hundreds or even thousands of layers of convolutional neural networks. Generally, the more the number of network layers, the larger the parameter quantity, the stronger the expression capability of the network, and the higher accuracy can be obtained in the image classification task. The large-scale convolutional neural network has long reasoning time and large storage cost, and in application scenes such as safe production, industrial quality inspection, intelligent hardware and the like, the memory capacity and the computing power of terminal equipment are limited, so that the deployment of the convolutional neural network is greatly hindered; although the direct design of a small-scale convolutional neural network is beneficial to terminal deployment, the problem of low classification accuracy exists. Therefore, the classification accuracy and efficiency of the convolutional neural network are difficult to obtain.
Knowledge distillation is a common model compression algorithm, and the method generally selects a network with large scale and high precision as a teacher network and selects a network with small scale and poor precision as a student network, and realizes the knowledge transfer from the teacher network to the student network by guiding the student network to simulate the output of the teacher network, so that the student network achieves the precision close to the teacher network.
An image classification method based on attention knowledge distillation is proposed In the published paper "Paying more annotation to annotation" by Zagoruyko (In International Conference on Learning Representations, 2017). The method converts the output characteristics of the network middle layers of the teacher network and the student network into the attention characteristics, enables the student network to learn the image area concerned by the teacher network middle layer by minimizing the difference between the attention characteristics of the teacher network and the student network, completes the knowledge transfer from the teacher network to the student network, and improves the classification accuracy of the student network images with small scale. The method has the following defects: the method converts the output characteristics of the network middle layers of the teacher network and the student network into attention characteristics, but the output of the middle layers contains two parts of information of space dimension and channel dimension.
Byeong Heo, In its published paper, "Knowledge transfer via partition of activation floors for used by high nodes" (In AAAI Conference on intellectual insight, 2019), proposed an image classification method that uses Knowledge distillation of neuron activation boundaries. The method judges whether hidden layer neurons of intermediate layers of a teacher network and a student network are activated or not by setting a threshold, if the neuron activation value is larger than the threshold, the neurons are considered to be in an activation state, if the neuron activation value is smaller than the activation value, the neurons are not activated, the method guides whether the neurons of the intermediate layers corresponding to the student network are activated or not, and the information is transmitted to the student network, so that the classification accuracy of the student network images with small scale is improved. The method has the following defects: the method judges whether the neurons of the teacher network middle layer are activated or not by setting a threshold value, and guides whether the corresponding neurons of the student network middle layer are activated or not, so that the method only teaches whether the corresponding neurons of the student network should be activated or not, does not teach amplitude information after the corresponding neurons of the student network are activated, does not transmit complete neuron activation information to the student network, and causes slow convergence of the student network.
Disclosure of Invention
The invention aims to provide an image classification method based on a bridging knowledge distillation convolution neural network aiming at overcoming the defects of the prior art and solving the problem of low accuracy of student network image classification caused by information loss in the knowledge distillation process in the prior art.
The technical idea for realizing the purpose of the invention is as follows: by establishing a bridging structure between the teacher network and the student network, the information of the middle layer of the student network is mapped into class probability characteristics by using the teacher network, so that the key information of the middle layer is extracted, and then the KL divergence loss function and the cross entropy loss function are used for training the student network to learn the knowledge in the teacher network with high image classification accuracy, so that the image classification accuracy similar to that of the teacher network is achieved.
The method comprises the following specific steps:
(1) constructing a teacher network and a student network:
(1a) build the teacher network of 14 layers and the student network of 14 layers that the structure is the same, its structure is in proper order: the device comprises an input layer, a first convolution layer, a first active layer, a first maximum pooling layer, a second convolution layer, a second active layer, a second maximum pooling layer, a third convolution layer, a third active layer, a third maximum pooling layer, a fourth convolution layer, a fourth active layer, a fifth convolution layer and an output layer;
(1b) the parameters of each layer of the teacher network are set as follows:
setting the numbers of the first to fifth convolution layer feature maps to 16, 32, 64, 128 and 10 respectively, and setting the sizes of convolution kernels to 5 × 5, 6 × 6, 5 × 5 and 3 × 3 respectively;
setting the pooling windows of the first to third largest pooling layers to be 2 multiplied by 2, and setting the step length to be 2;
setting the activation functions of the first to fourth activation layers as the ReLU activation functions;
(1c) the parameters of each layer of the student network are set as follows:
setting the numbers of the first to fifth convolution layer feature maps to 9, 10, 31, 8 and 10 respectively, and setting the sizes of convolution kernels to 5 × 5, 6 × 6, 5 × 5 and 3 × 3 respectively;
setting the pooling windows of the first to third largest pooling layers to be 2 multiplied by 2, and setting the step length to be 2;
setting the activation functions of the first to fourth activation layers as the ReLU activation functions;
(2) generating a training set:
selecting at least 2 types of images, and forming a training set by at least 200 images in each type;
(3) training a teacher network:
inputting the training set into a teacher network to obtain the prediction category probability of each training image, calculating the loss between the prediction category probability of each image and the category label corresponding to the image by using a cross entropy loss function, and iteratively updating teacher network parameters by using a back propagation algorithm until the cross entropy loss function is converged to obtain the trained teacher network;
(4) constructing a bridging structure:
connecting the trained fourth convolution layer of the teacher network with the trained fourth convolution layer of the student network to obtain a bridging structure;
(5) training a student network:
(5a) simultaneously inputting the training set into a student network and a trained teacher network to obtain the output of the student network, the output of the teacher network and the output of the bridging structure;
(5b) calculating a KL divergence loss value between the output of the teacher network and the output of the bridging structure by using the KL divergence loss function;
(5c) calculating a cross entropy loss value between the output of the student network and the class label of the training image by using a cross entropy loss function;
(5d) and taking the sum of the KL divergence loss value and the cross entropy loss value as a total loss value, and iteratively updating parameters of the student network through a back propagation algorithm until the total loss value is converged to obtain the trained student network.
(6) Classifying the images to be classified:
and inputting the images to be classified into the trained student network to obtain the prediction class probability of the student network for the images to be classified, and selecting the class corresponding to the probability with the highest median value of the prediction class probability as the classification result of the images.
Compared with the prior art, the invention has the following advantages:
firstly, a bridging structure is constructed between a teacher network and a student network, and the structure can simultaneously utilize two parts of information, namely space dimension and channel dimension, in an intermediate layer between the teacher network and the student network, so that the problem that channel dimension features are lost in intermediate layer features extracted in the prior art is solved, and the image classification accuracy of the student network is improved.
Secondly, the KL divergence loss function and the cross entropy loss function are used when the student network is trained, the loss function can transfer the intermediate layer knowledge of the teacher network to the student network, the problem of slow convergence of the student network caused by neglecting amplitude information of the intermediate layer of the teacher network in the prior art is solved, and the convergence efficiency of the student network is effectively accelerated.
Drawings
FIG. 1 is a flow chart of the present invention;
fig. 2 is a schematic diagram of a bridging structure in the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
With reference to fig. 1, the specific steps of the implementation of the present invention are described in detail.
Step 1, a teacher network and a student network are constructed.
Build the teacher network of 14 layers and the student network of 14 layers that the structure is the same, its structure is in proper order: the multilayer chip comprises an input layer, a first convolution layer, a first active layer, a first maximum pooling layer, a second convolution layer, a second active layer, a second maximum pooling layer, a third convolution layer, a third active layer, a third maximum pooling layer, a fourth convolution layer, a fourth active layer, a fifth convolution layer and an output layer.
The parameters of each layer of the teacher network are set as follows:
the numbers of the first to fifth convolution layer feature maps are set to 16, 32, 64, 128, 10, respectively, and the convolution kernel sizes are set to 5 × 5, 6 × 6, 5 × 5, 3 × 3, respectively.
The pooling windows of the first to third largest pooling layers are all set to 2 × 2, and the step sizes are all set to 2.
The activation functions of the first to fourth activation layers are all set as the ReLU activation functions.
The parameters of each layer of the student network are set as follows:
the numbers of the first to fifth convolution layer feature maps are set to 9, 10, 31, 8, 10, respectively, and the convolution kernel sizes are set to 5 × 5, 6 × 6, 5 × 5, 3 × 3, respectively.
The pooling windows of the first to third largest pooling layers are all set to 2 × 2, and the step sizes are all set to 2.
The activation functions of the first to fourth activation layers are all set as the ReLU activation functions.
And 2, generating a training set.
At least 2 categories of at least 200 images in each category are selected to form a training set.
And step 3, training a teacher network.
Inputting the training set into a teacher network to obtain the prediction category probability of each training image, calculating the loss between the prediction category probability of each image and the category label corresponding to the image by using a cross entropy loss function, and iteratively updating teacher network parameters by using a back propagation algorithm until the cross entropy loss function is converged to obtain the trained teacher network.
The cross entropy loss function is as follows:
Figure BDA0002917976340000051
where J represents the cross entropy loss function, N represents the total number of images in the training set, Sigma represents the summation operation, i represents the number of images in the training set, Y represents the number of images in the training setiRepresenting the class label corresponding to the ith image in the training set, log representing the base 2 logarithm operation, PiThe predicted class probability obtained by inputting the ith image in the training set into the teacher network is shown.
And 4, constructing a bridging structure.
And connecting the trained fourth convolution layer of the teacher network with the trained fourth convolution layer of the student network to obtain the bridging structure.
The bridging structure constructed in the present invention is further described with reference to fig. 2.
In fig. 2, the left side is a schematic structural diagram of a teacher network, the right side is a schematic structural diagram of a student network, and a bridge structure is obtained by connecting the fourth convolutional layer of the trained teacher network and the fourth convolutional layer of the student network, as shown in the middle part of fig. 2. As can be seen in fig. 2, the bridging structure consists of layers below the fourth convolutional layer of the student network and layers above the fourth convolutional layer of the teacher network.
And 5, training a student network.
And simultaneously inputting the training set into the student network and the trained teacher network to obtain the output P of the student network, the output Q of the teacher network and the output B of the bridging structure.
And calculating a KL divergence loss value between the output of the teacher network and the output of the bridging structure by using the KL divergence loss function.
The KL divergence loss function is as follows:
Figure BDA0002917976340000061
wherein the content of the first and second substances,
Figure BDA0002917976340000062
representing KL divergence loss function, QiRepresenting the probability of the predicted class obtained by inputting the ith image in the training set into the teacher's network, BiRepresenting the prediction class probability obtained by inputting the ith image in the training set into the bridge structure.
And calculating a cross entropy loss value between the output of the student network and the class label of the training image by using a cross entropy loss function.
The cross entropy loss function is as follows:
Figure BDA0002917976340000063
wherein J represents a cross entropy loss function, YiIndicates the class label, P, corresponding to the ith image in the training setiRepresenting the probability of the predicted class obtained by inputting the ith image in the training set into the student network.
And taking the sum of the KL divergence loss value and the cross entropy loss value as a total loss value, and iteratively updating parameters of the student network through a back propagation algorithm until the total loss value is converged to obtain the trained student network.
And 6, classifying the images to be classified.
And inputting the images to be classified into the trained student network to obtain the prediction class probability of the student network for the images to be classified, and selecting the class corresponding to the probability with the highest median value of the prediction class probability as the classification result of the images.

Claims (3)

1. An image classification method based on a bridging knowledge distillation convolution neural network is characterized in that a bridging structure is constructed between a teacher network and a student network, and the student network is trained according to a KL divergence loss function and a cross entropy loss function, and the method comprises the following steps:
(1) constructing a teacher network and a student network:
(1a) build the teacher network of 14 layers and the student network of 14 layers that the structure is the same, its structure is in proper order: the device comprises an input layer, a first convolution layer, a first active layer, a first maximum pooling layer, a second convolution layer, a second active layer, a second maximum pooling layer, a third convolution layer, a third active layer, a third maximum pooling layer, a fourth convolution layer, a fourth active layer, a fifth convolution layer and an output layer;
(1b) the parameters of each layer of the teacher network are set as follows:
setting the numbers of the first to fifth convolution layer feature maps to 16, 32, 64, 128 and 10 respectively, and setting the sizes of convolution kernels to 5 × 5, 6 × 6, 5 × 5 and 3 × 3 respectively;
setting the pooling windows of the first to third largest pooling layers to be 2 multiplied by 2, and setting the step length to be 2;
setting the activation functions of the first to fourth activation layers as the ReLU activation functions;
(1c) the parameters of each layer of the student network are set as follows:
setting the numbers of the first to fifth convolution layer feature maps to 9, 10, 31, 8 and 10 respectively, and setting the sizes of convolution kernels to 5 × 5, 6 × 6, 5 × 5 and 3 × 3 respectively;
setting the pooling windows of the first to third largest pooling layers to be 2 multiplied by 2, and setting the step length to be 2;
setting the activation functions of the first to fourth activation layers as the ReLU activation functions;
(2) generating a training set:
selecting at least 2 types of images, and forming a training set by at least 200 images in each type;
(3) training a teacher network:
inputting the training set into a teacher network to obtain the prediction category probability of each training image, calculating the loss between the prediction category probability of each image and the category label corresponding to the image by using a cross entropy loss function, and iteratively updating teacher network parameters by using a back propagation algorithm until the cross entropy loss function is converged to obtain the trained teacher network;
(4) constructing a bridging structure:
connecting the trained fourth convolution layer of the teacher network with the trained fourth convolution layer of the student network to obtain a bridging structure;
(5) training a student network:
(5a) simultaneously inputting the training set into a student network and a trained teacher network to obtain the output of the student network, the output of the teacher network and the output of the bridging structure;
(5b) calculating a KL divergence loss value between the output of the teacher network and the output of the bridging structure by using the KL divergence loss function;
(5c) calculating a cross entropy loss value between the output of the student network and the class label of the training image by using a cross entropy loss function;
(5d) taking the sum of the KL divergence loss value and the cross entropy loss value as a total loss value, and iteratively updating parameters of the student network through a back propagation algorithm until the total loss value is converged to obtain a trained student network;
(6) classifying the images to be classified:
and inputting the images to be classified into the trained student network to obtain the prediction class probability of the student network for the images to be classified, and selecting the class corresponding to the probability with the highest median value of the prediction class probability as the classification result of the images.
2. The image classification method based on the bridge knowledge distillation convolutional neural network of claim 1, wherein the cross entropy loss function in the steps (3) and (5c) is as follows:
Figure FDA0002917976330000021
where J represents the cross entropy loss function, N represents the total number of images in the training set, Sigma represents the summation operation, i represents the number of images in the training set, Y represents the number of images in the training setiRepresenting the class label corresponding to the ith image in the training set, log representing the base 2 logarithm operation, PiAnd the prediction class probability obtained by inputting the ith image in the training set into the network is shown.
3. The method for image classification based on the bridge knowledge distillation convolutional neural network of claim 2, wherein the KL divergence loss function in step (5b) is as follows:
Figure FDA0002917976330000022
wherein the content of the first and second substances,
Figure FDA0002917976330000031
representing KL divergence loss function, QiRepresenting the probability of the predicted class obtained by inputting the ith image in the training set into the teacher's network, BiRepresenting the prediction class probability obtained by inputting the ith image in the training set into the bridge structure.
CN202110107120.1A 2021-01-27 2021-01-27 Image classification method based on bridging knowledge distillation convolution neural network Pending CN112784964A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110107120.1A CN112784964A (en) 2021-01-27 2021-01-27 Image classification method based on bridging knowledge distillation convolution neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110107120.1A CN112784964A (en) 2021-01-27 2021-01-27 Image classification method based on bridging knowledge distillation convolution neural network

Publications (1)

Publication Number Publication Date
CN112784964A true CN112784964A (en) 2021-05-11

Family

ID=75757981

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110107120.1A Pending CN112784964A (en) 2021-01-27 2021-01-27 Image classification method based on bridging knowledge distillation convolution neural network

Country Status (1)

Country Link
CN (1) CN112784964A (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113222123A (en) * 2021-06-15 2021-08-06 深圳市商汤科技有限公司 Model training method, device, equipment and computer storage medium
CN113392938A (en) * 2021-07-30 2021-09-14 广东工业大学 Classification model training method, Alzheimer disease classification method and device
CN113421243A (en) * 2021-06-23 2021-09-21 深圳大学 Method and device for detecting type of fundus image based on knowledge distillation network
CN113449776A (en) * 2021-06-04 2021-09-28 中南民族大学 Chinese herbal medicine identification method and device based on deep learning and storage medium
CN113505719A (en) * 2021-07-21 2021-10-15 山东科技大学 Gait recognition model compression system and method based on local-integral joint knowledge distillation algorithm
CN113591978A (en) * 2021-07-30 2021-11-02 山东大学 Image classification method, device and storage medium based on confidence penalty regularization self-knowledge distillation
CN113610146A (en) * 2021-08-03 2021-11-05 江西鑫铂瑞科技有限公司 Method for realizing image classification based on knowledge distillation enhanced by interlayer feature extraction
CN113610173A (en) * 2021-08-13 2021-11-05 天津大学 Knowledge distillation-based multi-span domain few-sample classification method
CN113673591A (en) * 2021-08-13 2021-11-19 上海交通大学 Image classification method, device and medium for self-adjusting sampling optimization
CN113807214A (en) * 2021-08-31 2021-12-17 中国科学院上海微系统与信息技术研究所 Small target face recognition method based on deit attached network knowledge distillation
CN114095447A (en) * 2021-11-22 2022-02-25 成都中科微信息技术研究院有限公司 Communication network encrypted flow classification method based on knowledge distillation and self-distillation
CN114373133A (en) * 2022-01-10 2022-04-19 中国人民解放军国防科技大学 Missing modal terrain classification method based on dense feature group distillation
CN114936567A (en) * 2022-05-26 2022-08-23 清华大学 Unsupervised machine translation quality estimation method and device based on knowledge distillation
CN115908955A (en) * 2023-03-06 2023-04-04 之江实验室 Bird classification system, method and device for small-sample learning based on gradient distillation
CN115965964A (en) * 2023-01-29 2023-04-14 中国农业大学 Egg freshness identification method, system and equipment
CN116091849A (en) * 2023-04-11 2023-05-09 山东建筑大学 Tire pattern classification method, system, medium and equipment based on grouping decoder

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113449776A (en) * 2021-06-04 2021-09-28 中南民族大学 Chinese herbal medicine identification method and device based on deep learning and storage medium
CN113222123A (en) * 2021-06-15 2021-08-06 深圳市商汤科技有限公司 Model training method, device, equipment and computer storage medium
CN113421243A (en) * 2021-06-23 2021-09-21 深圳大学 Method and device for detecting type of fundus image based on knowledge distillation network
CN113505719A (en) * 2021-07-21 2021-10-15 山东科技大学 Gait recognition model compression system and method based on local-integral joint knowledge distillation algorithm
CN113505719B (en) * 2021-07-21 2023-11-24 山东科技大学 Gait recognition model compression system and method based on local-integral combined knowledge distillation algorithm
CN113392938A (en) * 2021-07-30 2021-09-14 广东工业大学 Classification model training method, Alzheimer disease classification method and device
CN113591978A (en) * 2021-07-30 2021-11-02 山东大学 Image classification method, device and storage medium based on confidence penalty regularization self-knowledge distillation
CN113591978B (en) * 2021-07-30 2023-10-20 山东大学 Confidence penalty regularization-based self-knowledge distillation image classification method, device and storage medium
CN113610146B (en) * 2021-08-03 2023-08-04 江西鑫铂瑞科技有限公司 Method for realizing image classification based on knowledge distillation with enhanced intermediate layer feature extraction
CN113610146A (en) * 2021-08-03 2021-11-05 江西鑫铂瑞科技有限公司 Method for realizing image classification based on knowledge distillation enhanced by interlayer feature extraction
CN113673591B (en) * 2021-08-13 2023-12-01 上海交通大学 Self-adjusting sampling optimization image classification method, device and medium
CN113673591A (en) * 2021-08-13 2021-11-19 上海交通大学 Image classification method, device and medium for self-adjusting sampling optimization
CN113610173B (en) * 2021-08-13 2022-10-04 天津大学 Knowledge distillation-based multi-span domain few-sample classification method
CN113610173A (en) * 2021-08-13 2021-11-05 天津大学 Knowledge distillation-based multi-span domain few-sample classification method
CN113807214B (en) * 2021-08-31 2024-01-05 中国科学院上海微系统与信息技术研究所 Small target face recognition method based on deit affiliated network knowledge distillation
CN113807214A (en) * 2021-08-31 2021-12-17 中国科学院上海微系统与信息技术研究所 Small target face recognition method based on deit attached network knowledge distillation
CN114095447A (en) * 2021-11-22 2022-02-25 成都中科微信息技术研究院有限公司 Communication network encrypted flow classification method based on knowledge distillation and self-distillation
CN114095447B (en) * 2021-11-22 2024-03-12 成都中科微信息技术研究院有限公司 Communication network encryption flow classification method based on knowledge distillation and self-distillation
CN114373133A (en) * 2022-01-10 2022-04-19 中国人民解放军国防科技大学 Missing modal terrain classification method based on dense feature group distillation
CN114936567A (en) * 2022-05-26 2022-08-23 清华大学 Unsupervised machine translation quality estimation method and device based on knowledge distillation
CN115965964A (en) * 2023-01-29 2023-04-14 中国农业大学 Egg freshness identification method, system and equipment
CN115965964B (en) * 2023-01-29 2024-01-23 中国农业大学 Egg freshness identification method, system and equipment
CN115908955B (en) * 2023-03-06 2023-06-20 之江实验室 Gradient distillation-based bird classification system, method and device with less sample learning
CN115908955A (en) * 2023-03-06 2023-04-04 之江实验室 Bird classification system, method and device for small-sample learning based on gradient distillation
CN116091849A (en) * 2023-04-11 2023-05-09 山东建筑大学 Tire pattern classification method, system, medium and equipment based on grouping decoder

Similar Documents

Publication Publication Date Title
CN112784964A (en) Image classification method based on bridging knowledge distillation convolution neural network
CN111291836B (en) Method for generating student network model
CN111554268B (en) Language identification method based on language model, text classification method and device
CN112508085B (en) Social network link prediction method based on perceptual neural network
CN109783666B (en) Image scene graph generation method based on iterative refinement
CN110222760B (en) Quick image processing method based on winograd algorithm
CN107392919A (en) Gray threshold acquisition methods, image partition method based on self-adapted genetic algorithm
CN105975457A (en) Information classification prediction system based on full-automatic learning
CN112381179A (en) Heterogeneous graph classification method based on double-layer attention mechanism
CN116541779B (en) Individualized public safety emergency detection model training method, detection method and device
CN114077659A (en) Knowledge graph question-answering method and system based on neighbor interaction network
CN114841318A (en) Intelligent contract vulnerability detection method based on cross-modal knowledge distillation
CN114780767A (en) Large-scale image retrieval method and system based on deep convolutional neural network
CN111160536B (en) Convolution embedding representation inference method based on fragmentation knowledge
CN112860977A (en) Link prediction method based on convolutional neural network
CN111985560A (en) Knowledge tracking model optimization method, system and computer storage medium
CN116797850A (en) Class increment image classification method based on knowledge distillation and consistency regularization
Parvin et al. A new approach to improve the vote-based classifier selection
CN114265954B (en) Graph representation learning method based on position and structure information
CN113343041B (en) Message reply relation judgment system based on graph model representation learning
CN115660882A (en) Method for predicting user-to-user relationship in social network and multi-head mixed aggregation graph convolutional network
CN107122472A (en) Extensive unstructured data extracting method, its system, DDM platform
CN113806488A (en) Heterogeneous graph conversion text mining method based on meta-structure learning
CN110991366A (en) Shipping monitoring event identification method and system based on three-dimensional residual error network
JPH0492955A (en) Learning system for neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210511