CN107451565B - Semi-supervised small sample deep learning image mode classification and identification method - Google Patents

Semi-supervised small sample deep learning image mode classification and identification method Download PDF

Info

Publication number
CN107451565B
CN107451565B CN201710647312.5A CN201710647312A CN107451565B CN 107451565 B CN107451565 B CN 107451565B CN 201710647312 A CN201710647312 A CN 201710647312A CN 107451565 B CN107451565 B CN 107451565B
Authority
CN
China
Prior art keywords
layer
sample
representing
deep learning
semi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710647312.5A
Other languages
Chinese (zh)
Other versions
CN107451565A (en
Inventor
周喜川
刘念
唐枋
胡盛东
林�智
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CN201710647312.5A priority Critical patent/CN107451565B/en
Publication of CN107451565A publication Critical patent/CN107451565A/en
Application granted granted Critical
Publication of CN107451565B publication Critical patent/CN107451565B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a semi-supervised small sample deep learning image mode classification and identification method, and belongs to the field of image identification. The method comprises the following steps: s1: preprocessing an image sample; s2: inputting the data obtained by preprocessing into a trained network, and extracting the characteristics of the network through a 3D convolutional layer to obtain a characteristic layer; s3: each convolution layer is followed by a pooling layer for reducing the size of the characteristic layer to reduce the number of parameters in the network; s4: connecting the features extracted by the multilayer convolution layer and the pooling layer with a full-connection layer to extract and rearrange the features to be classified; the layer introduces local neighbor preserving regularization operation; s5: and inputting a sample to be detected to obtain the classification accuracy. The invention utilizes the position correlation among a large number of collected label-free samples, and improves the applicability and accuracy of the algorithm under a small sample set.

Description

Semi-supervised small sample deep learning image mode classification and identification method
Technical Field
The invention belongs to the field of image recognition, and relates to a semi-supervised small sample deep learning image mode classification recognition method.
Background
Under the condition that the number of the label samples is enough, the deep learning method can extract the characteristics of the image in a self-adaptive manner by establishing a hierarchical model, and further the accuracy of pattern classification and identification is obviously improved. The accuracy of the deep Convolutional Neural Network (CNN) on a part of remote sensing image data sets reaches 95% -99%. An article published by Yushi Chen2016 on TGRS provides a feature extraction model based on a 3D convolutional neural network (3DCNN), and the classification accuracy rate is more than 98%. But because the model parameters are large in scale, the requirement on the number of label samples for training is high.
The deep learning model supervised by CNN (requiring label training samples) is large in parameter scale, and a large number of label samples are required for training to improve the classification accuracy. However, in some fields, due to the reasons of difficult sample collection, high label analysis cost and the like, the pattern classification and identification method based on deep learning faces the difficulty of label sample shortage. At present, the hyperspectral image processing based on the deep learning method generally uses 10 per category3～104The optimal performance can be achieved only by training the labels, which is far beyond the label sample size which can be provided in practical applications such as mineral identification.
Disclosure of Invention
In view of the above, the present invention provides a semi-supervised small sample deep learning image pattern classification and identification method, which combines an unsupervised (label-free training sample) pattern identification method, and utilizes a large amount of obtained unlabelled sample data on the basis of 3DCNN to reduce the dependency of the deep learning method on the labeled sample and improve the pattern classification and identification accuracy based on deep learning.
In order to achieve the purpose, the invention provides the following technical scheme:
a semi-supervised small sample deep learning image pattern classification and identification method comprises the following steps:
s1: preprocessing an image sample;
s2: inputting the data obtained by preprocessing into a trained network, and extracting the characteristics of the network through a 3D convolutional layer to obtain a characteristic layer;
s3: each convolution layer is followed by a pooling layer for reducing the size of the characteristic layer to reduce the number of parameters in the network;
s4: connecting the features extracted by the multilayer convolution layer and the pooling layer with a full-connection layer to extract and rearrange the features to be classified; the layer introduces local neighbor preserving regularization operation, reduces the characteristic difference of adjacent samples, and accordingly reduces the classification accuracy reduction caused by the lack of label samples;
s5: and inputting a sample to be detected to obtain the classification accuracy.
Further, the S1 specifically includes: selecting a number A of target pixel points and corresponding labels as training samples, and forming a neighborhood matrix including the target pixel points by the number A of pixel points around each target pixel point; and taking the labels and the neighborhood matrixes of all target pixel points as input data of a 3D Convolutional Neural Network (CNN).
Further, the specific algorithm of S2 is:
Figure BDA0001367124640000021
wherein
Figure BDA0001367124640000022
Representing the neuron output value positioned at (x, y, z) on the jth characteristic layer of the ith layer, m representing the index value of the characteristic layer connected with the i-1 layer and the jth characteristic layer, and Pi、Qi、RiRespectively representing the height, width and depth of the convolution kernel of the ith layer,
Figure BDA0001367124640000023
is the value on the mth characteristic layer and at the (p, q, r) point, bijFor the bias of the jth feature layer, g () is an activation function.
Further, the specific algorithm of S3 is:
Figure BDA0001367124640000024
where u (nxnxnxnxn) denotes a three-dimensional window, alpha, acting on the convolutional layer output featuresi,j,mRepresenting the maximum of the feature points in the neighborhood.
Further, the specific algorithm of S4 is:
Figure BDA0001367124640000025
Figure BDA0001367124640000026
hf (i)＝sig(Wfvf (i)+bf)
wherein s isijRepresents the distance, g, between the ith and jth training samples(i)And g(j)Coordinates of the ith and jth training samples, hf (i)And hf (j)Respectively representing the extracted features of the ith training sample and the jth training sample; r (W)f) Representing a regularization term; when training samples i are adjacent to j, sijThe larger the parameter Wfvf (i)And Wfvf (j)The smaller the difference is, the smaller the feature h isf (i)And hf (j)And also adjacent.
Further, R (W)f) Satisfies the following conditions:
Figure BDA0001367124640000031
wherein Vf＝[vf (1),...vf (L),...vf (N)]In the form of a matrix representing the input vector of the fully-connected layer, D is the diagonal matrix (D)ii＝∑jsij)，P＝D-S；
The regularization term is derived:
Figure BDA0001367124640000032
the invention has the beneficial effects that: the invention provides a deep learning model of small label samples on the basis of the existing 3DCNN algorithm, and improves the applicability and accuracy of the algorithm under a small sample set by utilizing the position correlation among a large number of collected label-free samples.
Drawings
In order to make the object, technical scheme and beneficial effect of the invention more clear, the invention provides the following drawings for explanation:
FIG. 1 is a block diagram of the present invention;
FIG. 2 is a schematic diagram of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
In a large number of image pattern recognition applications, the correlation of nearby samples is widely present. For example, in applications of scene recognition, remote sensing feature recognition, and medical image recognition, labels of nearby pixel samples tend to have correlation. Therefore, a local adjacent Convolutional Neural Network (local compressive Neural Network) based mode classification and identification method is provided, the position correlation among a large number of collected label-free samples is utilized, and the applicability and accuracy of the algorithm under a small sample set are improved. As shown in fig. 1, the specific steps are as follows:
1. data pre-processing
And selecting a certain number of target pixel points and corresponding labels as training samples. And forming a neighborhood matrix (including the target pixel points) by using the same number of pixel points around each target pixel point. And taking the labels and the neighborhood matrixes of all target pixel points as input data of the 3 DCNN.
2. Feature extraction
(1) Convolutional layer(s)
Inputting the obtained data into a trained network, performing feature extraction on the network through 3D convolution layers, wherein each layer comprises different numbers of 3D convolution kernels, and performing feature extraction on the input data to generate different feature layers.
The specific algorithm is as follows:
Figure BDA0001367124640000041
wherein
Figure BDA0001367124640000042
And representing the output value of the neuron positioned at (x, y, z) on the jth characteristic image layer of the ith layer. And m represents the index value of the characteristic layer connected with the i-1 layer and the jth characteristic layer. PiAnd QiRespectively representing the height and width R of the convolution kernel of the i-th layeriIndicating the depth of the i-th layer convolution kernel.
Figure BDA0001367124640000043
And the value of the point (p, q, r) on the mth feature layer. bijIs the bias of the jth characteristic layer. g () is an activation function.
(2) Pooling layer (multiple)
Each convolutional layer is followed by a pooling layer, which serves to reduce the size of the feature layer to reduce the number of parameters in the network. The specific algorithm for the most common pooling operation (max pooling operation) is as follows:
Figure BDA0001367124640000044
where u (nxnxnxnxn) denotes a three-dimensional window, alpha, acting on the convolutional layer output featuresi,j,mRepresenting the maximum of the feature points in the neighborhood.
(3) Fully connected layer with local neighbor preserving regularization
The features extracted by the multi-layer convolutional layer and the pooling layer are connected with a full connection layer to extract and rearrange the features to be classified. The layer introduces local neighbor preserving regularization operation, reduces the feature difference of adjacent samples, and accordingly reduces the classification accuracy reduction caused by the lack of label samples.
The specific algorithm is as follows:
Figure BDA0001367124640000045
Figure BDA0001367124640000046
hf (i)＝sig(Wfvf (i)+bf)
wherein s isijRepresents the distance, g, between the ith and jth training samples(i)And g(j)The coordinates of the ith and jth training samples are represented, respectively. h isf (i)And hf (j)Respectively representing the extracted features of the ith and jth training samples.
When training samples i are adjacent to j, sijThe larger the parameter Wfvf (i)And Wfvf (j)The smaller the difference is, the smaller the feature h isf (i)And hf (j)And also adjacent.
The regularization term in the above equation is expressed in matrix form as follows:
Figure BDA0001367124640000051
wherein Vf＝[vf (1),...vf (L),...vf (N)]In the form of a matrix representing the input vector of the fully-connected layer, D is the diagonal matrix (D)ii＝∑jsij)，P＝D-S。
The regularization term is derived:
Figure BDA0001367124640000052
thus, the regularization term may be reduced using a standard gradient descent method.
3. A classification layer
And inputting a sample to be detected, and outputting the type of the sample label with the width as the classification accuracy.
As shown in FIG. 2, assume A1、A2And B1、B2Respectively associated with different labels, and the output features of the classification extraction also keep the relevance in consideration of the relevance of adjacent samples.
Finally, it is noted that the above-mentioned preferred embodiments illustrate rather than limit the invention, and that, although the invention has been described in detail with reference to the above-mentioned preferred embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention as defined by the appended claims.

Claims (4)

1. A semi-supervised small sample deep learning image mode classification and identification method is characterized by comprising the following steps: the method comprises the following steps:
s1: preprocessing an image sample;
s2: inputting the data obtained by preprocessing into a trained network, and extracting the characteristics of the network through a 3D convolutional layer to obtain a characteristic layer;
s3: each convolution layer is followed by a pooling layer for reducing the size of the characteristic layer to reduce the number of parameters in the network;
s4: connecting the features extracted by the multilayer convolution layer and the pooling layer with a full-connection layer to extract and rearrange the features to be classified; the full-connection layer introduces local neighbor-preserving regularization operation, so that the characteristic difference of adjacent samples is reduced, and the classification accuracy reduction caused by the lack of label samples is reduced;
s5: inputting a sample to be detected to obtain classification accuracy;
the S1 specifically includes: selecting a number A of target pixel points and corresponding labels as training samples, and forming a neighborhood matrix including the target pixel points by the number A of pixel points around each target pixel point; taking the labels and the neighborhood matrixes of all target pixel points as input data of a 3D Convolutional Neural Network (CNN);
the specific algorithm of S2 is as follows:
Figure FDA0002740401570000011
wherein
Figure FDA0002740401570000012
Representing the neuron output value positioned at (x, y, z) on the jth characteristic layer of the ith layer, m representing the index value of the characteristic layer connected with the i-1 layer and the jth characteristic layer, and Pi、Qi、RiRespectively representing the height, width and depth of the convolution kernel of the ith layer,
Figure FDA0002740401570000013
is the value on the mth characteristic layer and at the (p, q, r) point, bijFor the bias of the jth feature layer, g () is an activation function.
2. The semi-supervised small-sample deep learning image pattern classification and identification method as claimed in claim 1, wherein: the specific algorithm of S3 is as follows:
Figure FDA0002740401570000014
where u (nxnxnxnxn) denotes a three-dimensional window, alpha, acting on the convolutional layer output featuresi,j,mRepresenting the maximum of the feature points in the neighborhood.
3. The semi-supervised small-sample deep learning image pattern classification and identification method as claimed in claim 1, wherein: the specific algorithm of S4 is as follows:
Figure FDA0002740401570000015
Figure FDA0002740401570000021
hf (i)＝sig(Wfvf (i)+bf)
wherein s isijRepresents the distance, g, between the ith and jth training samples(i)And g(j)Coordinates of the ith and jth training samples, hf (i)Representing the extracted features of the ith training sample; r (W)f) Representing a regularization term; when training samples i are adjacent to j, sijThe larger the parameter Wfvf (i)And Wfvf (j)The smaller the difference.
4. The semi-supervised small-sample deep learning image pattern classification and identification method as claimed in claim 3, wherein: the R (W)f) Satisfies the following conditions:
Figure FDA0002740401570000022
wherein Vf＝[vf (1),...vf (L),...vf (N)]In the form of a matrix representing the input vector of the fully-connected layer, D is the diagonal matrix (D)ii＝∑jsij)，P＝D-S；
The regularization term is derived:
Figure FDA0002740401570000023
CN201710647312.5A 2017-08-01 2017-08-01 Semi-supervised small sample deep learning image mode classification and identification method Active CN107451565B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710647312.5A CN107451565B (en) 2017-08-01 2017-08-01 Semi-supervised small sample deep learning image mode classification and identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710647312.5A CN107451565B (en) 2017-08-01 2017-08-01 Semi-supervised small sample deep learning image mode classification and identification method

Publications (2)

Publication Number Publication Date
CN107451565A CN107451565A (en) 2017-12-08
CN107451565B true CN107451565B (en) 2020-12-11

Family

ID=60490637

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710647312.5A Active CN107451565B (en) 2017-08-01 2017-08-01 Semi-supervised small sample deep learning image mode classification and identification method

Country Status (1)

Country Link
CN (1) CN107451565B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108009284A (en) * 2017-12-22 2018-05-08 重庆邮电大学 Using the Law Text sorting technique of semi-supervised convolutional neural networks
CN108171200B (en) * 2018-01-12 2022-05-17 西安电子科技大学 SAR image classification method based on SAR image statistical distribution and DBN
CN108537119B (en) * 2018-03-06 2020-07-10 北京大学 Small sample video identification method
CN109657697B (en) * 2018-11-16 2023-01-06 中山大学 Classification optimization method based on semi-supervised learning and fine-grained feature learning
CN109685135B (en) * 2018-12-21 2022-03-25 电子科技大学 Few-sample image classification method based on improved metric learning
CN110245714B (en) * 2019-06-20 2021-08-20 厦门美图之家科技有限公司 Image recognition method and device and electronic equipment
CN111024912A (en) * 2019-12-27 2020-04-17 重庆国环绿源科技有限公司 Ship wastewater pretreatment type detection device
CN111259366B (en) * 2020-01-22 2021-06-18 支付宝(杭州)信息技术有限公司 Verification code recognizer training method and device based on self-supervision learning
CN111353583B (en) * 2020-02-20 2023-04-07 南京工程学院 Deep learning network based on group convolution characteristic topological space and training method thereof
CN111639714B (en) * 2020-06-01 2021-07-23 贝壳找房(北京)科技有限公司 Method, device and equipment for determining attributes of users

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102346847A (en) * 2011-09-26 2012-02-08 青岛海信网络科技股份有限公司 License plate character recognizing method of support vector machine
CN106778494A (en) * 2016-11-21 2017-05-31 河海大学 A kind of target in hyperspectral remotely sensed image feature extracting method based on SIFT LPP

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104809426B (en) * 2014-01-27 2019-04-05 日本电气株式会社 Training method, target identification method and the device of convolutional neural networks
CN106204587B (en) * 2016-05-27 2019-01-08 浙江德尚韵兴图像科技有限公司 Multiple organ dividing method based on depth convolutional neural networks and region-competitive model

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102346847A (en) * 2011-09-26 2012-02-08 青岛海信网络科技股份有限公司 License plate character recognizing method of support vector machine
CN106778494A (en) * 2016-11-21 2017-05-31 河海大学 A kind of target in hyperspectral remotely sensed image feature extracting method based on SIFT LPP

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Locality Preserving Projections;Xiaofei He，Partha Niyogi;《Advances in Neural Information Processing Systems 16》;20051231;第3.1节 *
Reliable Crowdsourcing and Deep LocalityPreserving Learning for Unconstrained Facial Expression Recognition;Li, Shan , W. Deng , and J. P. Du;《2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IEEE》;20170701;第2.2节第2段，第2.3节第1段，第4节第2-3段，第6节第2段，表2 *
Wanjiang Xu;Can Luo;Aiming Ji.Coupled locality preserving projections for cross-view gait recognition.《Neurocomputing》.2017, *
基于QR分解的正则化邻域保持嵌入算法;翟冬灵, 王正群, 徐春林;《计算机应用》;20160610;第36卷(第6期);第1.2节第1段 *

Also Published As

Publication number Publication date
CN107451565A (en) 2017-12-08

Similar Documents

Publication Publication Date Title
CN107451565B (en) Semi-supervised small sample deep learning image mode classification and identification method
CN110443143B (en) Multi-branch convolutional neural network fused remote sensing image scene classification method
CN108182441B (en) Parallel multichannel convolutional neural network, construction method and image feature extraction method
CN106023065B (en) A kind of tensor type high spectrum image spectral-spatial dimension reduction method based on depth convolutional neural networks
CN109002755B (en) Age estimation model construction method and estimation method based on face image
CN110097060B (en) Open set identification method for trunk image
CN105160310A (en) 3D (three-dimensional) convolutional neural network based human body behavior recognition method
CN107767416B (en) Method for identifying pedestrian orientation in low-resolution image
Liu et al. Remote sensing image change detection based on information transmission and attention mechanism
CN108985252B (en) Improved image classification method of pulse depth neural network
CN109241995B (en) Image identification method based on improved ArcFace loss function
CN112308152B (en) Hyperspectral image ground object classification method based on spectrum segmentation and homogeneous region detection
CN104700100A (en) Feature extraction method for high spatial resolution remote sensing big data
CN111898621A (en) Outline shape recognition method
CN111178177A (en) Cucumber disease identification method based on convolutional neural network
CN113011386B (en) Expression recognition method and system based on equally divided characteristic graphs
CN112101467A (en) Hyperspectral image classification method based on deep learning
Su et al. LodgeNet: Improved rice lodging recognition using semantic segmentation of UAV high-resolution remote sensing images
CN109472733A (en) Image latent writing analysis method based on convolutional neural networks
CN111160389A (en) Lithology identification method based on fusion of VGG
CN107766810B (en) Cloud and shadow detection method
CN111274905A (en) AlexNet and SVM combined satellite remote sensing image land use change detection method
Barman et al. Smartphone assist deep neural network to detect the citrus diseases in agri-informatics
CN109886281A (en) One kind is transfinited learning machine color image recognition method based on quaternary number
CN109635726B (en) Landslide identification method based on combination of symmetric deep network and multi-scale pooling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant