CN115063624A - Small sample classification learning method based on graph neural network - Google Patents
Small sample classification learning method based on graph neural network Download PDFInfo
- Publication number
- CN115063624A CN115063624A CN202210479820.8A CN202210479820A CN115063624A CN 115063624 A CN115063624 A CN 115063624A CN 202210479820 A CN202210479820 A CN 202210479820A CN 115063624 A CN115063624 A CN 115063624A
- Authority
- CN
- China
- Prior art keywords
- sample
- class
- network
- classification
- learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The contact network defect detection based on the 4C image plays an important role in ensuring the safety of railway transportation and the stable operation of trains, and the deep learning-based method has the problems of more parts and defect types and less defect sample number. The application provides a small sample classification learning method based on a graph neural network, which comprises the following steps: 1) acquiring a real scene high-speed rail contact network picture, positioning and classifying parts of the high-speed rail contact network picture and constructing a 4C sample library; 2) inputting the pictures of the corresponding categories into a small sample classification network for training to obtain a model training result; 3) a single unmarked test picture can obtain a corresponding category name through the trained model; 4) the specific classification precision can be obtained by comparing the predicted value with the true value of the batch-input marked test pictures. The method can improve the classification precision of the small sample learning method, constructs a relatively rich and complete 4C sample library, and obtains a relatively good classification effect on both the small sample open source data set and the 4C sample library.
Description
Technical Field
The application belongs to the field of machine learning, and particularly relates to a small sample classification learning method based on a graph neural network.
Background
The contact net is used as the key equipment of the high-speed railway traction power supply system, and directly influences the safety and reliability of high-speed rail transportation. The overhead line system suspension system is composed of a plurality of different parts, is influenced by physical/mechanical impact and vibration caused by long-term running of a train, and easily causes the problems of looseness, breakage, even falling, loss and the like of the parts, so that potential safety hazards of high-speed railway operation can be caused. In order to ensure the reliable operation of a traction power supply system, the detection and monitoring of the suspension state of the contact network (4C detection for short), which is a process of recording images of the contact network by a camera on the roof of a detection vehicle along a line and carrying out analysis and analysis based on the images, mainly guides the operation and maintenance of the contact network in a mode of periodically inspecting the state of suspension parts of the contact network, is a key technology of the traction power supply system of the high-speed railway at present and is also an important guarantee for the safe and reliable railway transportation.
In the existing 4C detection system, tens of sets of high-speed and high-definition industrial cameras and a background operation control system which are arranged on a detection vehicle implement automatic snapshot imaging on geometric parameters and important parts of a contact network, regular monitoring and inspection are carried out, and then visual inspection and defect identification are carried out on a snapshot image manually. With the development of image processing technology, it is a current development trend to apply an advanced contact network component defect detection method based on vision to 4C image intelligent analysis to realize automatic detection. Automatic image recognition is a key technology, and such methods generally include two steps: the parts of the contact net supporting and hanging device are firstly positioned, and then possible defects of the parts are identified and classified. The positioning method of the parts is mainly divided into two types based on manual characteristics and deep learning, and the target detection method based on the deep learning has good effect in the aspect of positioning the parts. In addition to part localization, deep learning based methods have also been used in defect recognition classification problems, and although deep learning based methods have made a breakthrough in terms of accuracy, scalability, and robustness, such methods often require large amounts of labeled data. However, in practical application, the catenary suspension system has the problems of multiple types of rods, multiple parts of each type of rod, multiple defect types of each type of part, and very small number of defect samples, and is not enough to train a robust classifier, so that the deep learning method still has certain difficulty in catenary defect identification and classification. The recently proposed small sample learning method Few-shot learning (FSL) in the field of machine learning has the ability to learn and summarize from a small number of samples, independent of large-scale training samples, thus avoiding the high cost of preparing large amounts of data in some applications. The goal of small sample learning is to train a classification model based on known classes, enabling it to have good generalization performance on unknown classes with only a small amount of data.
The method based on model fine tuning is a relatively traditional small sample learning method, and the method is used on the premise that a target sample data set and a training data set are distributed similarly, but under the condition that the target sample data set and the training data set are not distributed similarly, a model has the problem of overfitting on the target sample data set; the method based on data enhancement does not need to adjust the model, only needs to perform data expansion or feature enhancement on the original target sample data set by means of auxiliary data or information, but possibly introduces noise data or features to generate negative influence on the classification effect; the methods based on migratory learning migrate and apply learned knowledge to new domains quickly in a way that old knowledge learns new knowledge, including metric learning, meta learning, Graph Neural Networks (GNNs), etc. The metric learning-based method is convenient for formulation, but in the case of a small number of samples, the similarity measurement simply by distance easily results in a decrease in accuracy. Recent research shows that the graph neural network has the advantages of good interpretability and strong data expression capability, but when the total number of samples is increased, the number of edges in the graph neural network is increased rapidly, so that the computational complexity is increased rapidly, and the conventional GNN method and other meta-learning methods only consider the relationship among the samples explicitly or implicitly, but ignore the relationship among the sample distributions.
Disclosure of Invention
1. Technical problem to be solved
At present, computer vision technology and machine learning methods are widely used for solving some engineering problems, and particularly deep learning methods have good effects. However, the deep learning-based method often requires a large number of data samples for training the network model, and the requirement of a large number of samples limits the application of the method in some fields. In the existing contact network 4C detection, the problem that the number of types of rods, parts of each type of rod and the number of types of defects of each part are large, but the number of samples of each type of defect is small, is insufficient to train a robust deep learning model to realize reliable defect classification, and thus, a general deep learning method still has great challenges in the aspect of contact network defect identification and classification. The application provides a small sample classification learning method combining strong data expression capability of an image neural network and a class center estimation method based on Wasserstein distance. And high detection precision is obtained under the condition that excessive defect samples are not needed.
2. Technical scheme
In order to achieve the above object, the present application provides a small sample classification learning method based on a graph neural network, the method comprising the following steps:
step 1: utilizing dozens of sets of high-speed and high-definition industrial cameras and a background operation control system which are arranged on a detection vehicle in a 4C detection system to automatically snapshot and image geometrical parameters and important parts of a contact net and acquire original picture data of the contact net in a real high-speed railway scene;
step 2: the contact net parts are roughly positioned and finely positioned by a target detection method based on deep learning, so that a more accurate and specific part picture of the contact net is obtained;
and step 3: processing the obtained contact net part picture, constructing a relatively complete 4C sample library, and preparing for small sample classification training and testing;
and 4, step 4: the method combines a double-graph neural network with class center estimation based on Wasserstein distance, inputs a contact net part picture, optimizes and enhances feature expression through the double-graph neural network, aligns the enhanced feature distribution with the distribution of the belonging class by using the Wasserstein distance optimization method, improves classification precision, and outputs the corresponding name of the normal or defect type of the final part, namely a classification result.
Another embodiment provided by the present application is: the contact network pictures collected in the step 1 come from a plurality of different high-speed railway scenes, and normal and defect image materials of common and key contact network equipment are collected, sorted and analyzed by using a high-resolution camera according to the characteristics of different high-speed railway power supply line equipment.
Another embodiment provided by the present application is: the coarse positioning and the fine positioning in the step 2 are two-step walking plans provided for realizing target detection, the specific position of the large component is positioned in the coarse positioning mode, and the specific position of the defective part in the large component is determined in the fine positioning mode, so that the detection and positioning accuracy is improved.
Another embodiment provided by the present application is: the contact network part processing in the step 3 comprises two parts, namely, after the result of the fine positioning defect part is obtained, the size and format of all normal or defect pictures are processed in a unified mode, and the pictures with unified size and format are classified in a manual and automatic combined mode to construct a 4C sample library.
Another embodiment provided by the present application is: the 4C sample library comprises three levels of catalogs, the first level of catalogs are English names of all parts of the contact network, the second level of catalogs are English names of all parts under the parts, the third level of catalogs are specific part defect types, different names represent different defects, and the complete and clear 4C sample library is constructed in sequence.
Another embodiment provided by the present application is: the step 4 comprises the following steps:
a. the method adopts a conduction type small sample classification learning method, wherein the small sample learning is composed of a plurality of learning tasks (tasks), and the sample of each learning task comprises a support set S and a query set Q. Given number of trainingsData set D train And a test data set D test And is and
at D train Including all the sample pairs (S) required by the learning task i ,Q i ) I is 1 … … n, and n is the task number.
Contains C classes, and K samples (C-way, K-shot) are extracted from each class, which can be expressed as S { (x) 1 ，y 1 )，(x 2 ，y 2 )，…，(x C×K ，y C×K )}；
Corresponding to S, K samples are each drawn from the C classes, which may be denoted as Q { (x) C×K+1 ，y C×K+1 )，…，(x C×K+CⅹK ，y C×K+C×K )}. Test data D test Also consisting of two parts (S, Q), but with D train And the method has no intersection and is mainly used for testing the precision of the training model. The tags of the support set S and the query set Q are mutually exclusive at this time.
b. Selecting a residual error network ResNet12 with strong feature extraction capability, and simultaneously using a data set D in the training stage of the feature extraction network train And D test Training the model, applying the corresponding network model to a test data set D after obtaining the corresponding network model test In (1) are respectively paired with test And carrying out feature extraction on the support set and the query set samples in the database. Due to the use of D in the training phase test And the data set is favorable for extracting the characteristics of the test sample.
c. In order to maximize the classification effect, a Dual Graph Neural Network (DGNN) for Feature enhancement is designed, the DGNN is composed of a Feature Graph Neural Network (FG) and a Relation Graph Neural Network (RG), the Relation between sample features and sample distribution is fully fused through the loop iteration between the FG and the RG, the labeled support set sample features and label information are propagated to the unlabeled query set samples by using the message propagation mechanism of the Graph Network, and the characterization enhancement of the extracted features of the backbone Network is realized.
The DGNN-based model comprises h generations, and each generation comprises a feature map (FG) G h f ＝(N h f ,E h f ) And a Relational Graph (RG) G h r ＝(N h r ,E h r ). FG describes the characteristics of the sample and RG describes the relationship between the sample characteristic distributions. The process can be briefly expressed as
N h f →E h f →N h r →E h r →N h+1 f
Wherein h represents the h generation.
d. And (3) adopting a sinkhorn algorithm based on entropy regularization, evaluating a mapping path from the enhanced feature vector distribution to the corresponding extracted sample by combining Wassertein distance, finding the maximum posterior probability estimation (MAP) of a class center in an updating iteration mode to obtain an optimal transmission solution, and after iteration of a certain step, taking the class with the maximum probability of the sample to be classified in the optimal mapping as a classification result.
The maximum posterior probability estimation comprises several steps, firstly, a mapping matrix W is optimized and estimated by using a Sinkhorn algorithm, secondly, a class center is updated by using the mapping matrix W, then, the mapping matrix W is updated according to the newly obtained class center, and the process is iterated.
The overall optimization estimation method model is as follows:
where 0< i ≦ CQ represents the index of the unlabeled sample,
c j class center representing class j, i.e., the most central feature vector in the spatial features of class j, 0<j≤C，
n i Representing the feature vector corresponding to the unlabeled exemplar,
l(n i ) Is a class corresponding to the unlabeled exemplarThe number of the labels is such that,
P(n i |j＝l(n i ) Represents the probability that an unlabeled exemplar belongs to class j,
The second line of the formula corresponds to the Wasserstein optimum transmission distance. In order to solve the optimal transmission from the distribution of the feature vectors of the enhanced samples to the balanced sampling of the samples, a Sinkhorn algorithm based on entropy regularization is adopted, and the Sinkhorn optimization estimation can be expressed as:
wherein W represents the probability that CQ samples to be classified belong to C classes respectively,
D∈R CQ×C representing the euclidean distance between the unlabeled exemplars and the class center,
h (W) represents the entropy of the mapping matrix W.
where p is the sum of each row of the matrix W, representing the distribution of the number of classes assigned per unlabeled exemplar,
q is the sum of each column of the matrix W, representing the distribution of the number of unlabeled samples assigned to each class.
The optimization iterative estimation method of the class center comprises the following steps:
the class center updating method comprises the following steps:
c j ←c j +α(v j -c j )
wherein α (0< α ≦ 1) represents the learning rate.
3. Advantageous effects
Compared with the prior art, the small sample classification learning method based on the graph neural network has the beneficial effects that:
the small sample classification learning method based on the graph neural network provided by the application aims at the problems that the existing 4C intelligent defect detection based on deep learning has multiple rod types, multiple parts of each rod and multiple defect types of each part and the quantity of samples of each defect is small from a data sample, and when a large amount of high-speed railway contact net part picture data are collected and processed in a manual and automatic mode, the defect pictures are expanded by utilizing various data enhancement methods to train a feature extraction network, so that training data are enriched, a relatively complete 4C sample library is constructed, and the classification accuracy and precision are improved to a certain extent.
According to the small sample classification learning method based on the graph neural network, compared with an inductive learning method, the conductive learning method is adopted, and clustering can be found by using label-free test sample information under the conditions that training samples are very few, test samples are very many, and a class discrimination model obtained by using the inductive learning method is poor in performance, so that more effective classification is realized. This is also not possible with inductive learning methods that use only training samples to derive the model. The method provided by the application trains a classification model by utilizing a training set of known labels and a testing set of unknown labels, so that the classification model still has good generalization performance on a label-free class data set with only a small number of samples.
The small sample classification learning method based on the graph neural network realizes feature extraction and enhancement in a mode of a residual error network + a Double Graph Neural Network (DGNN), and aligns the enhanced feature distribution with the distribution of the class by adopting a Wassertein distance optimization method. As the DGNN model further strengthens the characteristics of the sample, the characteristic expression of the sample is more accurate, as known prior knowledge, the prior probability is correspondingly improved in the process of optimizing the Maximum Posterior probability (MAP), the improvement of the Posterior probability is very helpful, the improvement is very suitable for the purpose of finally maximizing the Posterior probability of the MAP, and the combination of DGNN characteristic enhancement and the MAP algorithm plays a crucial role in optimizing the class center estimation and finally improving the classification precision.
According to the small sample classification learning method based on the graph neural network, a large number of experiments are carried out on four small sample open source data sets and a 4C sample library constructed by the method, and the classification accuracy of the method provided by the application is superior to that of the SOTA method of the same type under the condition that fewer defective samples exist.
Drawings
FIG. 1 is a schematic diagram of an implementation of a small sample classification learning method of the present application;
fig. 2 is a flowchart of the DGNN _ MAP algorithm of the present application.
Detailed Description
Hereinafter, specific embodiments of the present application will be described in detail with reference to the accompanying drawings, and it will be apparent to those skilled in the art from this detailed description that the present application can be practiced. Features from different embodiments may be combined to yield new embodiments, or certain features may be substituted for certain embodiments to yield yet further preferred embodiments, without departing from the principles of the present application.
Small sample Learning (Few-Shot Learning, FSL) aims to learn a valid model for a new class from a small number of samples, get the ability to learn and summarize from a small number of samples, and identify a new class.
Referring to fig. 1-2, the application provides a small sample classification learning method, which includes the following steps:
step 1: utilizing dozens of sets of high-speed and high-definition industrial cameras and a background operation control system which are arranged on a detection vehicle in a 4C detection system to automatically capture and image geometric parameters and important parts of a contact network and acquire original picture data of the contact network in a real high-speed railway scene;
step 2: the contact net parts are roughly positioned and finely positioned by a target detection method based on deep learning, so that a more accurate and specific part picture of the contact net is obtained;
and 3, step 3: processing the obtained contact net part picture, constructing a relatively complete 4C sample library, and preparing for small sample classification training and testing;
and 4, step 4: the method is characterized in that a method combining a double-graph neural network and class center estimation based on Wasserstein distance is adopted, a contact net part picture is input, feature expression is enhanced through double-graph neural network optimization, enhanced feature distribution is aligned with distribution of the class by using a Wassertein distance optimization method, classification precision is improved, and a corresponding name of a normal or defect type of a final part, namely a classification result, is output.
Further, the overhead line system pictures acquired in the step 1 come from a plurality of different high-speed railway scenes, and normal and defect image materials of common and key overhead line system equipment are collected, sorted and analyzed by using a high-resolution camera according to the characteristics of different high-speed railway power supply line equipment.
Further, the coarse positioning and the fine positioning in the step 2 are two-step planning provided for realizing target detection, the specific position of the large component is positioned in the coarse positioning mode, and the specific position of the defective part in the large component is determined in the fine positioning mode, so that the detection and positioning accuracy is improved.
Further, the processing of the parts of the overhead line system in the step 3 includes two parts, one is to perform unified processing of the size and format on all normal or defective pictures after the result of the fine positioning of the defective parts is obtained, and the other is to classify the pictures with unified size and format by using a manual and automatic combination mode to construct a 4C sample library.
Further, the 4C sample library comprises three levels of catalogs, the first level of catalogs are English names of all parts of the overhead line system, the second level of catalogs are English names of all parts under the parts, the third level of catalogs are specific part defect types, different names represent different defects, and the complete and clear 4C sample library is sequentially constructed.
Further, the step 4 comprises the following steps:
a. the method adopts a conduction type small sample classification learning method, wherein the small sample learning is composed of a plurality of learning tasks (tasks), and the sample of each learning task comprises a support set S and a query set Q. Given a training data set D train And a test data set D test And is and
at D train Including all the sample pairs (S) required by the learning task i ,Q i ) I is 1 … … n, and n is the task number.
Contains C classes, and K samples (C-way, K-shot) are extracted from each class, which can be expressed as S { (x) 1 ，y 1 )，(x 2 ，y 2 )，…，(x C×K ，y C×K )}；
Corresponding to S, K samples are each drawn from the C classes, which may be denoted as Q { (x) C×K+1 ，y C×K+1 )，…，(x C×K+CⅹK ，y C×K+C×K )}. Test data D test Also consisting of two parts (S, Q), but with D train And the method has no intersection and is mainly used for testing the precision of the training model. The tags of the support set S and the query set Q are mutually exclusive at this time.
b. Selecting a residual error network ResNet12 with strong feature extraction capability, and simultaneously using a data set D in the training stage of the feature extraction network train And D test Training the model, applying the model to a test data set D after obtaining a corresponding network model test In (1) are respectively paired with test Branch of Chinese herbal medicineAnd carrying out feature extraction on the support set and the query set samples. Due to the use of D in the training phase test And the data set is favorable for extracting the characteristics of the test sample.
c. In order to maximize the classification effect, a Dual Graph Neural Network (DGNN) for feature enhancement is designed, the DGNN consists of a feature graph neural network (FG) and a relation graph neural network (RG), the relation between sample features and sample distribution is fully fused through loop iteration between the FG and the RG, and the labeled support set sample features and label information are propagated to unlabeled query set samples by using a message propagation mechanism of the graph network, so that the characterization enhancement of the extracted features of a backbone network is realized.
The DGNN-based model comprises h generations, and each generation comprises a feature map (FG) G h f ＝(N h f ,E h f ) And a Relational Graph (RG) G h r ＝(N h r ,E h r ). FG describes the characteristics of the sample and RG describes the relationship between the sample characteristic distributions. The process can be briefly expressed as
N h f →E h f →N h r →E h r →N h+1 f
Wherein h represents the h generation.
d. And (3) adopting a sinkhorn algorithm based on entropy regularization, evaluating a mapping path from the enhanced feature vector distribution to the corresponding extracted sample by combining Wassertein distance, finding the maximum posterior probability estimation (MAP) of the class center in an updating iteration mode to obtain an optimal transmission solution, and after iteration of a certain step, taking the class with the maximum probability of the sample to be classified in the optimal mapping as a classification result.
The maximum posterior probability estimation comprises several steps, firstly, a mapping matrix W is optimized and estimated by using a Sinkhorn algorithm, secondly, a class center is updated by using the mapping matrix W, then, the mapping matrix W is updated according to the newly obtained class center, and the process is iterated.
The overall optimization estimation method model is as follows:
where 0< i ≦ CQ represents the index of the unlabeled sample,
c j class center representing class j, i.e., the most central feature vector in the spatial features of class j, 0<j≤C，
n i Representing the feature vector corresponding to the unlabeled exemplar,
l(n i ) Is a class label corresponding to the unlabeled exemplar,
P(n i |j＝l(n i ) Represents the probability that an unlabeled exemplar belongs to class j,
The second line of the formula corresponds to the Wasserstein optimum transmission distance. In order to solve the optimal transmission from the distribution of the feature vectors of the enhanced samples to the balanced sampling of the samples, a Sinkhorn algorithm based on entropy regularization is adopted, and the Sinkhorn optimization estimation can be expressed as:
wherein W represents the probability that CQ samples to be classified belong to C classes respectively,
D∈R CQ×C representing the euclidean distance between the unlabeled exemplars and the class center,
h (W) represents the entropy of the mapping matrix W.
where p is the sum of each row of the matrix W, representing the distribution of the number of classes assigned per unlabeled exemplar,
q is the sum of each column of the matrix W, representing the distribution of the number of unlabeled samples assigned to each class.
The optimization iterative estimation method of the class center comprises the following steps:
the class center updating method comprises the following steps:
c j ←c j +α(v j -c j )
wherein α (0< α ≦ 1) represents the learning rate.
Examples
Referring to fig. 1, the present application includes:
s1, for a contact network in a real high-speed railway scene, automatically capturing and imaging geometric parameters and important parts of the contact network by adopting dozens of sets of high-speed and high-definition industrial cameras and a background operation control system which are arranged on a detection vehicle in a 4C detection system to obtain original picture data of the contact network;
s2, aiming at the original contact network picture obtained in the S1, performing coarse positioning and fine positioning on contact network parts by using a target detection method based on deep learning to obtain a more accurate and specific normal or defective picture of the parts;
s3, uniformly processing the fine positioning part pictures obtained in the step S2, constructing a relatively complete 4C sample library, and preparing for small sample classification training and testing;
and S4, inputting the processed picture data set in S3 by adopting a method of combining a double-graph neural network with class center estimation based on Wasserstein distance, optimizing and enhancing feature expression through the double-graph neural network, aligning the enhanced feature distribution with the distribution of the belonged class by using a Wasserstein distance optimization-based method, improving classification precision, and outputting a corresponding name of a normal or defect type of the final part, namely a classification result.
The originally acquired contact network pictures are not limited to one high-speed railway but come from a plurality of different high-speed railway scenes, and the picture contents collected, sorted and analyzed by using the high-resolution camera are not limited to defect types, normal types and image materials from common and important contact network equipment.
The original contact net picture is large, and specific part images need to be positioned firstly, and then corresponding defect identification and classification are carried out. And in the target detection, the specific position of the large part is roughly positioned, and the specific position of the defective part in the large part is precisely positioned and determined. The 4C sample library constructed according to the method comprises three levels of catalogues, wherein the first level catalog is the names of all parts of the contact net in rough positioning, the second level catalog is the names of all parts under the parts obtained in fine positioning, and the third level catalog is a specific part defect type. Components without fine positioning results still use the third level directory, except that no specific part name is subdivided under the second level directory.
The rough positioning and the fine positioning are realized by using a target detection method based on deep learning, and the establishment of a third-level directory needs to classify pictures in a manual and automatic combined mode to establish a complete data set.
Unified processing requirements: the unified processing of size and format is performed on all the pictures of normal or defect types, and it is required to ensure that all the pictures are in the jpg format, and the size is 84 × 84.
Referring to fig. 2, in this embodiment, the specific steps of defining the problem and the data set using method in step S4 and training the feature extraction network by using the data set include:
s411, the small sample learning comprises a plurality of learning tasks (tasks), and each sample of the learning task is composed of a support set S and a query set Q. Given a training data set D train And a test data set D test And is and
at D train In which is containedHaving sample pairs (S) required for the learning task i ,Q i ) I is 1 … … n, and n is the number of tasks.
Contains C classes, and K samples (C-way, K-shot) are extracted from each class, which can be expressed as S { (x) 1 ，y 1 )，(x 2 ，y 2 )，…，(x C×K ，y C×K )}；
Corresponding to S, K samples are each drawn from the C classes, which may be denoted as Q { (x) C×K+1 ，y C×K+1 )，…，(x C×K+CⅹK ，y C×K+C×K )}. Test data D test Also consisting of two parts (S, Q), but with D train And the method has no intersection and is mainly used for testing the precision of the training model.
S412, selecting a residual error network ResNet12 with strong feature extraction capability, and simultaneously using a data set D in the feature extraction network training stage train And D test Training the model, applying the model to a test data set D after obtaining a corresponding network model test In (1) are respectively paired with test And carrying out feature extraction on the support set and the query set samples in the database.
In this embodiment, the specific steps of enhancing the extracted sample features and performing maximum posterior probability estimation to obtain a sample class decision in step S4 are as follows:
s421, in order to maximize the classification effect, a Dual Graph Neural Network (DGNN) for feature enhancement is designed, which is composed of a feature graph neural network (FG) and a relation graph neural network (RG), the relation between the sample features and the sample distribution is fully fused through the loop iteration between the FG and the RG, the labeled support set sample features and the label information are propagated to the unlabeled query set samples by using the message propagation mechanism of the graph network, and the characterization enhancement of the extracted features of the backbone network is realized.
The DGNN-based model comprises h generations, and each generation comprises a feature map (FG) G h f ＝(N h f ,E h f ) And a Relational Graph (RG) G h r ＝(N h r ,E h r ). FG describes the characteristics of the sample and RG describes the relationship between the sample characteristic distributions. The process can be briefly expressed as
N h f →E h f →N h r →E h r →N h+1 f
Wherein h represents the h generation.
S422, a sinkhorn algorithm based on entropy regularization is adopted, a mapping path from the enhanced feature vector distribution to the corresponding extracted sample is evaluated in combination with Wasserstein distance, the maximum posterior probability estimation (MAP) of a class center is found in an updating iteration mode to obtain an optimal transmission solution, and after iteration of a certain step, the class with the maximum probability of the sample to be classified in the optimal mapping is used as a classification result.
The maximum posterior probability estimation comprises several steps, firstly, a mapping matrix W is optimized and estimated by using a Sinkhorn algorithm, secondly, a class center is updated by using the mapping matrix W, then, the mapping matrix W is updated according to the newly obtained class center, and the process is iterated.
The overall optimization estimation method model is as follows:
where 0< i ≦ CQ represents the index of the unlabeled sample,
c j class center representing class j, i.e., the most central feature vector in the spatial features of class j, 0<j≤C，
n i Representing the feature vector corresponding to the unlabeled exemplar,
l(n i ) Is a class label corresponding to the unlabeled exemplar,
P(n i |j＝l(n i ) Represents the probability that an unlabeled exemplar belongs to class j,
The second line of the formula corresponds to the Wasserstein optimum transmission distance. In order to solve the optimal transmission from the distribution of the feature vectors of the enhanced samples to the balanced sampling of the samples, a Sinkhorn algorithm based on entropy regularization is adopted, and the Sinkhorn optimization estimation can be expressed as:
wherein W represents the probability that CQ samples to be classified belong to C classes respectively,
D∈R CQ×C representing the euclidean distance between the unlabeled exemplars and the class center,
h (W) represents the entropy of the mapping matrix W.
where p is the sum of each row of the matrix W, representing the distribution of the number of classes assigned per unlabeled exemplar,
q is the sum of each column of the matrix W, representing the distribution of the number of unlabeled samples assigned to each class.
The optimization iterative estimation method of the class center comprises the following steps:
the class center updating method comprises the following steps:
c j ←c j +α(v j -c j )
wherein α (0< α ≦ 1) represents the learning rate.
The small sample classification learning method provided by the application constructs a relatively complete 4C sample library, and aims at the problems that the existing 4C intelligent defect detection based on deep learning has more types of rods, more parts of each rod and more types of defects of each part, but the number of samples of each defect is small, a conductive learning method is adopted to ensure that the method still has good generalization performance on a label-free class data set with only a small number of samples, the feature extraction and enhancement are realized in a residual error network and dual-graph neural network (DGNN) mode, the enhanced feature distribution is aligned with the distribution of the belonged classes by adopting a Wassertein distance optimization method, and the classification accuracy and precision are improved to a certain extent.
Although the present application has been described above with reference to specific embodiments, those skilled in the art will recognize that many changes may be made in the configuration and details of the present application within the principles and scope of the present application. The scope of protection of the application is determined by the appended claims, and all changes that come within the meaning and range of equivalency of the technical features are intended to be embraced therein.
Claims (6)
1. A small sample classification learning method based on a graph neural network is characterized by comprising the following steps: the method comprises the following steps:
step 1: utilizing dozens of sets of high-speed and high-definition industrial cameras and a background operation control system which are arranged on a detection vehicle in a 4C detection system to automatically capture and image geometric parameters and important parts of a contact network and acquire original picture data of the contact network in a real high-speed railway scene;
step 2: the contact net parts are roughly positioned and finely positioned by a target detection method based on deep learning, so that a more accurate and specific part picture of the contact net is obtained;
and step 3: processing the obtained contact net part picture, constructing a relatively complete 4C sample library, and preparing for small sample classification training and testing;
and 4, step 4: the method combines a double-graph neural network with class center estimation based on Wasserstein distance, inputs a contact net part picture, optimizes and enhances feature expression through the double-graph neural network, aligns the enhanced feature distribution with the distribution of the belonging class by using the Wasserstein distance optimization method, improves classification precision, and outputs the corresponding name of the normal or defect type of the final part, namely a classification result.
2. The small sample classification learning method of claim 1, characterized in that: the contact network pictures collected in the step 1 come from a plurality of different high-speed railway scenes, and normal and defect image materials of common and key contact network equipment are collected, sorted and analyzed by using a high-resolution camera according to the characteristics of different high-speed railway power supply line equipment.
3. The small sample classification learning method of claim 1, characterized in that: the coarse positioning and the fine positioning in the step 2 are two-step walking plans provided for realizing target detection, the specific position of the large component is positioned in the coarse positioning mode, and the specific position of the defective part in the large component is determined in the fine positioning mode, so that the detection and positioning accuracy is improved.
4. The small sample classification learning method of claim 1, characterized in that: the contact network part processing in the step 3 comprises two parts, namely, after the result of the fine positioning defect part is obtained, the size and format of all normal or defect pictures are processed in a unified mode, and the pictures with unified size and format are classified in a manual and automatic combined mode to construct a 4C sample library.
5. The small sample classification learning method as claimed in any one of claims 1 to 4, characterized in that: the 4C sample library comprises three levels of catalogs, the first level catalog is the English name of each component of the contact net, the second level catalog is the English name of each part under the component, the third level catalog is a specific part defect type, different names represent different defects, and the complete and clear 4C sample library is constructed in sequence.
6. The small sample classification learning method of claim 1, characterized in that: the step 4 comprises the following steps:
a. the small sample learning is composed of a plurality of learning tasks (tasks), the sample of each learning task comprises a support set S and a query set Q, and a training data set D is given train And a test data set D test And is and
at D train Including all the sample pairs (S) required by the learning task i ,Q i ) I is 1 … … n, n is the task number,
the method comprises C classes, wherein K samples (C-way, K-shot) are extracted from each class, and can be expressed as S { (x { (X) 1 ，y 1 )，(x 2 ，y 2 )，…，(x C×K ，y C×K )}；
Corresponding to S, K samples are each drawn from the C classes, which may be denoted as Q { (x) C×K+1 ，y C×K+1 )，…，(x C×K+CⅹK ，y C×K+C×K ) Test data D test Also consisting of two parts (S, Q), but with D train The method is free of intersection and is mainly used for testing the precision of the training model, and the labels of the support set S and the query set Q are mutually exclusive at the moment;
b. selecting a residual error network ResNet12 with strong feature extraction capability, and simultaneously using a data set D in the training stage of the feature extraction network train And D test Training the model, applying the model to a test data set D after obtaining a corresponding network model test In (1) are respectively paired with test The support set and the query set samples in (1) are subjected to feature extraction, and D is used in the training stage test The data set is beneficial to extracting the characteristics of the test sample;
c. in order to maximize the classification effect, a Dual Graph Neural Network (DGNN) for feature enhancement is designed, the DGNN consists of a feature graph neural network (FG) and a relational graph neural network (RG), the relationship between sample features and sample distribution is fully fused through loop iteration between the FG and the RG, labeled support set sample features and label information are propagated to unlabeled query set samples by using a message propagation mechanism of the graph network, the characterization enhancement of extracted features of a backbone network is realized, the DGNN-based model comprises h generations in total, and each generation comprises a Feature Graph (FG) G h f ＝(N h f ,E h f ) And a Relational Graph (RG) G h r ＝(N h r ,E h r ). FG describes the characteristics of the sample and RG describes the relationship between the sample characteristic distributions. The process can be briefly denoted as N h f →E h f →N h r →E h r →N h+1 f ,
Wherein h represents the h generation;
d. the method comprises the following steps of firstly optimizing and estimating a mapping matrix W by using a Sinkhorn algorithm based on entropy regularization and combining Wassertein distance evaluation to obtain a mapping path from the enhanced feature vector distribution to a corresponding extracted sample, finding a maximum posterior probability estimation (MAP) of a class center in an updating iteration mode to obtain an optimal transmission solution, and after iteration of certain steps, taking the class with the maximum probability of the sample to be classified in the optimal mapping as a classification result, wherein the maximum posterior probability estimation comprises the following steps of firstly optimizing and estimating the mapping matrix W by using the Sinkhorn algorithm, secondly updating the class center by using the mapping matrix W, then updating the mapping matrix W according to the newly obtained class center, and iterating the process, and the overall optimization estimation method comprises the following models:
where 0< i ≦ CQ represents the index of the unlabeled sample,
c j class center representing class j, i.e., the most central feature vector in the spatial features of class j, 0<j≤C，
n i Representing the feature vector corresponding to the unlabeled exemplar,
l(n i ) Is a class label corresponding to the unlabeled exemplar,
P(n i |j＝l(n i ) Represents the probability that an unlabeled exemplar belongs to class j,
the second row of the formula corresponds to the Wasserstein optimal transmission distance, in order to solve the optimal transmission from the enhanced sample feature vector distribution to the sample balanced sampling, a Sinkhorn algorithm based on entropy regularization is adopted, and the Sinkhorn optimization estimation can be expressed as:
wherein W represents the probability that CQ samples to be classified belong to C classes respectively,
D∈R CQ×C representing the euclidean distance between the unlabeled exemplars and the class center,
h (W) denotes the entropy of the mapping matrix W,
where p is the sum of each row of the matrix W, representing the distribution of the number of classes assigned per unlabeled exemplar,
q is the sum of each column of the matrix W, representing the distribution of the number of unlabeled samples assigned to each class,
the optimization iterative estimation method of the class center comprises the following steps:
the class center updating method comprises the following steps:
c j ←c j +α(v j -c j )，
wherein α (0< α ≦ 1) represents the learning rate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210479820.8A CN115063624A (en) | 2022-05-05 | 2022-05-05 | Small sample classification learning method based on graph neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210479820.8A CN115063624A (en) | 2022-05-05 | 2022-05-05 | Small sample classification learning method based on graph neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115063624A true CN115063624A (en) | 2022-09-16 |
Family
ID=83196919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210479820.8A Pending CN115063624A (en) | 2022-05-05 | 2022-05-05 | Small sample classification learning method based on graph neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115063624A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115525786A (en) * | 2022-10-11 | 2022-12-27 | 中国传媒大学 | Method for constructing ionospheric frequency high-graph classification sample library |
-
2022
- 2022-05-05 CN CN202210479820.8A patent/CN115063624A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115525786A (en) * | 2022-10-11 | 2022-12-27 | 中国传媒大学 | Method for constructing ionospheric frequency high-graph classification sample library |
| CN115525786B (en) * | 2022-10-11 | 2024-02-20 | 中国传媒大学 | Method for constructing ionospheric frequency high-graph classification sample library |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414368B (en) | Unsupervised pedestrian re-identification method based on knowledge distillation | |
| CN106096561B (en) | Infrared pedestrian detection method based on image block deep learning features | |
| CN112069940B (en) | Cross-domain pedestrian re-identification method based on staged feature learning | |
| CN112819065B (en) | Unsupervised pedestrian sample mining method and unsupervised pedestrian sample mining system based on multi-clustering information | |
| CN111524164B (en) | Target tracking method and device and electronic equipment | |
| CN112633382A (en) | Mutual-neighbor-based few-sample image classification method and system | |
| CN111507413B (en) | Urban management case image recognition method based on dictionary learning | |
| CN107833211B (en) | Infrared image-based zero value insulator automatic detection method and device | |
| CN114299324B (en) | Pathological image classification method and system based on multiscale domain countermeasure network | |
| CN113283282A (en) | Weak supervision time sequence action detection method based on time domain semantic features | |
| CN111476307A (en) | Lithium battery surface defect detection method based on depth field adaptation | |
| CN116051479A (en) | Textile defect identification method integrating cross-domain migration and anomaly detection | |
| CN115063624A (en) | Small sample classification learning method based on graph neural network | |
| CN115546553A (en) | Zero sample classification method based on dynamic feature extraction and attribute correction | |
| CN115019294A (en) | Pointer instrument reading identification method and system | |
| CN114579794A (en) | Multi-scale fusion landmark image retrieval method and system based on feature consistency suggestion | |
| CN113435329B (en) | Unsupervised pedestrian re-identification method based on video track feature association learning | |
| CN111612803B (en) | Vehicle image semantic segmentation method based on image definition | |
| CN112927783A (en) | Image retrieval method and device | |
| CN113128410A (en) | Weak supervision pedestrian re-identification method based on track association learning | |
| CN109784244B (en) | Low-resolution face accurate identification method for specified target | |
| CN110045691B (en) | Multi-task processing fault monitoring method for multi-source heterogeneous big data | |
| CN111696070A (en) | Multispectral image fusion power internet of things fault point detection method based on deep learning | |
| Guo et al. | Udtiri: An open-source road pothole detection benchmark suite | |
| CN109993172B (en) | License plate recognition method for road traffic flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |