CN114821097A - Multi-scale feature image classification method based on transfer learning - Google Patents

Multi-scale feature image classification method based on transfer learning Download PDF

Info

Publication number
CN114821097A
CN114821097A CN202210365307.6A CN202210365307A CN114821097A CN 114821097 A CN114821097 A CN 114821097A CN 202210365307 A CN202210365307 A CN 202210365307A CN 114821097 A CN114821097 A CN 114821097A
Authority
CN
China
Prior art keywords
network
classification
convolution
data set
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210365307.6A
Other languages
Chinese (zh)
Other versions
CN114821097B (en
Inventor
龚勋
樊琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest Jiaotong University
Original Assignee
Southwest Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest Jiaotong University filed Critical Southwest Jiaotong University
Priority to CN202210365307.6A priority Critical patent/CN114821097B/en
Publication of CN114821097A publication Critical patent/CN114821097A/en
Application granted granted Critical
Publication of CN114821097B publication Critical patent/CN114821097B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of multi-scale feature image classification, in particular to a multi-scale feature image classification method based on transfer learning, which comprises the following steps: s1, constructing a data set; comprises a source domain and a target domain; s2, training a model on a source domain by using a multi-scale feature extraction network to extract features; s3, sending the extracted features into a classification network for classification training; and S4, carrying out classification training on the data sets of the same task and different domains under the condition of transfer learning. The invention can easily apply the relation obtained for a certain type of data in a model training task to different problems in the same field.

Description

Multi-scale feature image classification method based on transfer learning
Technical Field
The invention relates to the technical field of multi-scale feature image classification, in particular to a multi-scale feature image classification method based on transfer learning.
Background
At present, in the process of solving problems by using a deep learning technology, the most common obstacle is that a large amount of parameters of a model need to be trained, and therefore, massive training data is needed for support. When faced with a particular problem in a certain field, data of the size required for building a model may not be available in general.
Disclosure of Invention
The invention provides a multi-scale feature image classification method based on transfer learning, which can be easily applied to different problems in the same field.
The invention discloses a multi-scale feature image classification method based on transfer learning, which comprises the following steps of:
s1, constructing a data set; comprises a source domain and a target domain;
s2, training a model on a source domain by using a multi-scale feature extraction network to extract features;
s3, sending the extracted features into a classification network for classification training;
and S4, carrying out classification training on the data sets of the same task and different domains under the condition of transfer learning.
Preferably, in step S1, the MNIST dataset is used as the source domain; an MNIST-M data set is constructed to serve as a target domain, the MNIST-M data set is formed by mixing MNIST numbers and random color blocks in a BSDS500 data set, 10 samples are selected from each type, a small sample data set under MNIST-M is constructed, and then the source domain data set and the target domain data set are divided into a training set and a testing set.
Preferably, after step S1, image preprocessing is performed: rotating the source domain data set, and adjusting the size of the image to make the image conform to network input; adjusting preprocessing operations such as color saturation brightness and the like of the image; and adjusting the size of the target domain data image, and matching the size of the source domain data image.
Preferably, in step S2, the multi-scale feature extraction network is divided into three branches, where the first branch includes 12 convolution layers with convolution kernel size of 3 × 3, 12 normalization layers, and 12 active layers, where the 7 th to 10 th convolutions are hollow convolutions; the second branch comprises 12 convolution layers with convolution kernel size of 5 x 5, 12 normalization layers and 12 activation layers, wherein the 7 th convolution to the 10 th convolution is a hole convolution; the third branch comprises 12 convolution layers with convolution kernel size of 7 x 7, 12 normalization layers and 12 activation layers, wherein the 7 th convolution to the 10 th convolution is a hole convolution; the network training inputs MNIST images of a source domain and labels corresponding to the images, the input images firstly enter a feature extraction network, three branches simultaneously extract features of different scales of the images, then perform interpolation and fusion on the features of different scales, and finally obtain the features:
Figure BDA0003585622140000021
wherein F concatenate Representing the overall feature after fusion of the multi-scale features, F i*i (i-3, 5,7) represents the features extracted by the network under different scales,
Figure BDA0003585622140000022
representing the merging of different scale features in a first dimension.
Preferably, in step S3, the classification network is composed of three convolutional layers and three fully-connected layers; the network training input is F concatenate Training by combining the label of each picture; adopting Adam to carry out combined updating on parameters of the classification network and the feature extraction network; using a cross-entropy loss function as a classification loss function for the network:
Figure BDA0003585622140000023
wherein N represents the number of samples, y s,i Represents the label from the source domain sample i, with a positive class of 1, a negative class of 0, p s,i Is the probability that sample i is predicted to be positive;
the model is trained using source domain data until the loss function no longer falls, and the multi-scale feature extraction network and the classification network are tested using a source domain test set.
Preferably, in step S4, through step S2 and step S3, obtaining a multi-scale feature extraction and classification network trained on a source domain, then adopting a migration learning strategy, freezing a feature sub-network layer, and performing iterative training on a target domain data set to obtain a final small sample data set classification model, thereby implementing classification detection on an MNIST-M data set; using a cross-entropy loss function as a classification loss function for the network:
Figure BDA0003585622140000031
wherein N represents the number of samples, y t,i Represents the label from the target domain sample i, with a positive class of 1, a negative class of 0, p t,i Is the probability that sample i is predicted to be positive.
The method utilizes the source domain data with more data to assist in training the target domain data with less data and different fields, simultaneously constructs a multi-scale feature extraction network, and fuses the features of different receptive fields of the image by using convolution kernels and cavity convolutions with different sizes. The invention can easily apply the relation obtained for a certain type of data in a model training task to different problems in the same field.
Drawings
Fig. 1 is a flowchart of a multi-scale feature image classification method based on transfer learning in embodiment 1.
Detailed Description
For a further understanding of the invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples. It is to be understood that the examples are illustrative of the invention and not limiting.
Example 1
As shown in fig. 1, the present embodiment provides a multi-scale feature image classification method based on transfer learning, which is characterized in that: the method comprises the following steps:
s1, constructing a data set; the MNIST dataset is used as a source domain (source). And constructing an MNIST-M data set as a target domain (target), wherein the MNIST-M data set is formed by mixing MNIST numbers and random color blocks in the BSDS500 data set, 10 samples are selected from each class, and a small sample data set under MNIST-M is constructed. Dividing a source domain (source) data set and a target domain (target) data set into a training set and a testing set;
and S2, training a model on a source domain (source) by using a multi-scale feature extraction network to extract features. The multi-scale feature extraction network is divided into three branches. The first branch contains 12 convolution layers with convolution kernel size of 3 x 3, 12 normalization layers, 12 active layers, of which 7-10 convolutions are hollow convolution (scaled convolution); the second branch contains 12 convolution layers with convolution kernel size 5 x 5, 12 normalization layers, 12 active layers, of which 7-10 convolutions are hole convolutions (scaled convolution); the third branch contains 12 convolution layers with convolution kernel size 7 x 7, 12 normalization layers, 12 active layers, of which the 7 th to 10 th convolutions are hole convolutions (scaled convolution). The network training inputs MNIST images of a source domain (source) and labels corresponding to the images, the input images firstly enter a feature extraction network, three branches simultaneously extract features of different scales of the images, then interpolate and fuse the features of different scales, and finally the obtained features are as follows:
Figure BDA0003585622140000041
wherein F concatenate Representing the fused overall features of the multi-scale features, F i*i (i-3, 5,7) represents the features extracted by the network at different scales.
Figure BDA0003585622140000042
Representing the merging of different scale features in a first dimension.
And S3, sending the extracted features into a classification network for classification training. The classification network consists of three convolutional layers and three fully-connected layers. The network training input is F concatenate Training is performed in combination with the label of each picture. Adam is adopted to carry out combined updating on the parameters of the classification network and the feature extraction network. Using a Cross Entropy loss function (Cross Encopy) as a classification loss function for a networkNumber:
Figure BDA0003585622140000043
wherein N represents the number of samples, y s,i Represents a label from a source domain (source) sample i, with a positive class of 1, a negative class of 0, p s,i Is the probability that sample i is predicted to be positive.
The model is trained using source domain (source) data until the loss function no longer falls. And testing the multi-scale feature extraction network and the classification network using a source domain (source) test set.
S4, carrying out classification training on data sets of the same task and different domains under the condition of transfer learning: and S2 and S3 are carried out to obtain a multi-scale feature extraction and classification network trained on a source domain, then a transfer learning strategy is adopted, a feature sub-network layer is frozen, iterative training is carried out on a target domain data set (target), a final small sample data set classification model is obtained, and the MNIST-M data set is classified and detected. As a classification loss function of the network, a Cross Entropy loss function (Cross entry) is used:
Figure BDA0003585622140000044
wherein N represents the number of samples, y t,i A label representing a sample i from a target domain (target), a positive class of 1, a negative class of 0, p t,i Is the probability that sample i is predicted to be positive. The final test result on the target data set was 99%.
The present invention and its embodiments have been described above schematically, without limitation, and what is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Therefore, if the person skilled in the art receives the teaching, without departing from the spirit of the invention, the person skilled in the art shall not inventively design the similar structural modes and embodiments to the technical solution, but shall fall within the scope of the invention.

Claims (6)

1. A multi-scale feature image classification method based on transfer learning is characterized in that: the method comprises the following steps:
s1, constructing a data set; comprises a source domain and a target domain;
s2, training a model on a source domain by using a multi-scale feature extraction network to extract features;
s3, sending the extracted features into a classification network for classification training;
and S4, carrying out classification training on the data sets of the same task and different domains under the condition of transfer learning.
2. The multi-scale feature image classification method based on the transfer learning as claimed in claim 1, wherein: in step S1, the MNIST dataset is used as a source domain; an MNIST-M data set is constructed to serve as a target domain, the MNIST-M data set is formed by mixing MNIST numbers and random color blocks in a BSDS500 data set, 10 samples are selected from each type, a small sample data set under MNIST-M is constructed, and then the source domain data set and the target domain data set are divided into a training set and a testing set.
3. The multi-scale feature image classification method based on transfer learning according to claim 2, characterized in that: after step S1, image preprocessing is performed: rotating the source domain data set, and adjusting the size of the image to make the image conform to network input; adjusting preprocessing operations such as color saturation brightness and the like of the image; and adjusting the size of the target domain data image, and matching the size of the target domain data image with the size of the source domain.
4. The multi-scale feature image classification method based on the transfer learning according to claim 3, characterized in that: in step S2, the multi-scale feature extraction network is divided into three branches, where the first branch includes 12 convolution layers with convolution kernel size of 3 × 3, 12 normalization layers, and 12 active layers, where the 7 th convolution to 10 th convolution is a hole convolution; the second branch comprises 12 convolution layers with convolution kernel size of 5 x 5, 12 normalization layers and 12 activation layers, wherein the 7 th convolution to the 10 th convolution is a hole convolution; the third branch comprises 12 convolution layers with convolution kernel size of 7 x 7, 12 normalization layers and 12 activation layers, wherein the 7 th convolution to the 10 th convolution is a hole convolution; the network training inputs MNIST images of a source domain and labels corresponding to the images, the input images firstly enter a feature extraction network, three branches simultaneously extract features of different scales of the images, then perform interpolation and fusion on the features of different scales, and finally obtain the features:
Figure FDA0003585622130000011
wherein F concatenate Representing the overall feature after fusion of the multi-scale features, F i*i (i-3, 5,7) represents the features extracted by the network under different scales,
Figure FDA0003585622130000021
representing the merging of different scale features in a first dimension.
5. The multi-scale feature image classification method based on the transfer learning according to claim 4, characterized in that: in step S3, the classification network includes three convolutional layers and three full link layers; the network training input is F concatenate Training by combining the label of each picture; adopting Adam to carry out combined updating on parameters of the classification network and the feature extraction network; using a cross-entropy loss function as a classification loss function for the network:
Figure FDA0003585622130000022
wherein N represents the number of samples, y s,i Represents the label from the source domain sample i, with a positive class of 1, a negative class of 0, p s,i Is the probability that sample i is predicted to be positive;
the model is trained using source domain data until the loss function no longer falls, and the multi-scale feature extraction network and the classification network are tested using a source domain test set.
6. The multi-scale feature image classification method based on the transfer learning according to claim 5, characterized in that: in step S4, obtaining a multi-scale feature extraction and classification network trained on a source domain through step S2 and step S3, then adopting a transfer learning strategy, freezing a feature sub-network layer, and performing iterative training on a target domain data set to obtain a final small sample data set classification model, thereby realizing the classification detection of an MNIST-M data set; using a cross-entropy loss function as a classification loss function for the network:
Figure FDA0003585622130000023
wherein N represents the number of samples, y t,i Represents the label from the target domain sample i, with a positive class of 1, a negative class of 0, p t,i Is the probability that sample i is predicted to be positive.
CN202210365307.6A 2022-04-07 2022-04-07 Multi-scale feature image classification method based on transfer learning Active CN114821097B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210365307.6A CN114821097B (en) 2022-04-07 2022-04-07 Multi-scale feature image classification method based on transfer learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210365307.6A CN114821097B (en) 2022-04-07 2022-04-07 Multi-scale feature image classification method based on transfer learning

Publications (2)

Publication Number Publication Date
CN114821097A true CN114821097A (en) 2022-07-29
CN114821097B CN114821097B (en) 2023-04-18

Family

ID=82535449

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210365307.6A Active CN114821097B (en) 2022-04-07 2022-04-07 Multi-scale feature image classification method based on transfer learning

Country Status (1)

Country Link
CN (1) CN114821097B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630679A (en) * 2023-03-29 2023-08-22 南通大学 Osteoporosis identification method based on CT image and domain invariant feature
CN118520357A (en) * 2024-07-23 2024-08-20 湖南科技大学 Bearing fault diagnosis method, system and storage medium based on characterization guidance
CN118520357B (en) * 2024-07-23 2024-09-24 湖南科技大学 Bearing fault diagnosis method, system and storage medium based on characterization guidance

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110509109A (en) * 2019-07-16 2019-11-29 西安交通大学 Tool Wear Monitoring method based on multiple dimensioned depth convolution loop neural network
US20200226368A1 (en) * 2019-01-15 2020-07-16 International Business Machines Corporation Weakly and fully labeled mammogram classification and localization with a dual branch deep neural network
CN111814854A (en) * 2020-06-28 2020-10-23 北京交通大学 Target re-identification method adaptive to unsupervised domain
CN112906756A (en) * 2021-01-28 2021-06-04 昆山则立诚智能设备有限公司 High-image classification method and system for cross-channel quantity transfer learning
CN112926547A (en) * 2021-04-13 2021-06-08 北京航空航天大学 Small sample transfer learning method for classifying and identifying aircraft electric signals

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200226368A1 (en) * 2019-01-15 2020-07-16 International Business Machines Corporation Weakly and fully labeled mammogram classification and localization with a dual branch deep neural network
CN110509109A (en) * 2019-07-16 2019-11-29 西安交通大学 Tool Wear Monitoring method based on multiple dimensioned depth convolution loop neural network
CN111814854A (en) * 2020-06-28 2020-10-23 北京交通大学 Target re-identification method adaptive to unsupervised domain
CN112906756A (en) * 2021-01-28 2021-06-04 昆山则立诚智能设备有限公司 High-image classification method and system for cross-channel quantity transfer learning
CN112926547A (en) * 2021-04-13 2021-06-08 北京航空航天大学 Small sample transfer learning method for classifying and identifying aircraft electric signals

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"融合多尺度特征的多监督人脸活体检测算法" *
YAROSLAV GANIN等: "Unsupervised Domain Adaptation by Backpropagation", 《PROCEEDINGS OF THE 32ND INTERNATIONAL CONFERENCE ON MACHINE LEARNING》 *
YUN JIANG等: "Multi-Scale and Multi-Branch Convolutional Neural Network for Retinal Image Segmentation" *
袁彩艳: "深度卷积网络多尺度多通道特征学习及迁移的轴承故障诊断研究" *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630679A (en) * 2023-03-29 2023-08-22 南通大学 Osteoporosis identification method based on CT image and domain invariant feature
CN116630679B (en) * 2023-03-29 2024-06-04 南通大学 Osteoporosis identification method based on CT image and domain invariant feature
CN118520357A (en) * 2024-07-23 2024-08-20 湖南科技大学 Bearing fault diagnosis method, system and storage medium based on characterization guidance
CN118520357B (en) * 2024-07-23 2024-09-24 湖南科技大学 Bearing fault diagnosis method, system and storage medium based on characterization guidance

Also Published As

Publication number Publication date
CN114821097B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN108764292B (en) Deep learning image target mapping and positioning method based on weak supervision information
CN113486981B (en) RGB image classification method based on multi-scale feature attention fusion network
CN110232394B (en) Multi-scale image semantic segmentation method
CN109741341B (en) Image segmentation method based on super-pixel and long-and-short-term memory network
CN111160217B (en) Method and system for generating countermeasure sample of pedestrian re-recognition system
CN106951911A (en) A kind of quick multi-tag picture retrieval system and implementation method
CN105528638A (en) Method for grey correlation analysis method to determine number of hidden layer characteristic graphs of convolutional neural network
CN112529005B (en) Target detection method based on semantic feature consistency supervision pyramid network
CN112949929B (en) Knowledge tracking method and system based on collaborative embedded enhanced topic representation
CN114821097B (en) Multi-scale feature image classification method based on transfer learning
CN113128455B (en) Cell image reconstruction model training method and system
CN111832580B (en) SAR target recognition method combining less sample learning and target attribute characteristics
CN112131403B (en) Knowledge graph representation learning method in dynamic environment
CN108229557A (en) The acceleration training method and system of a kind of neural network with label
CN111768792B (en) Audio steganalysis method based on convolutional neural network and domain countermeasure learning
CN116129219A (en) SAR target class increment recognition method based on knowledge robust-rebalancing network
CN111753966A (en) Implementation method for implementing multi-label model training framework by using missing multi-label data
CN113378962B (en) Garment attribute identification method and system based on graph attention network
CN110991563A (en) Capsule network random routing algorithm based on feature fusion
CN111353583A (en) Deep learning network based on group convolution characteristic topological space and training method thereof
CN114511733A (en) Fine-grained image identification method and device based on weak supervised learning and readable medium
CN112132867B (en) Remote sensing image change detection method and device
CN110751061B (en) SAR image recognition method, device, equipment and storage medium based on SAR network
CN117058394A (en) Zero sample semantic segmentation method
CN116343111A (en) Abandoned land monitoring method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant