CN114972299A - Railway track defect detection method based on deep migration learning - Google Patents

Railway track defect detection method based on deep migration learning Download PDF

Info

Publication number
CN114972299A
CN114972299A CN202210678529.3A CN202210678529A CN114972299A CN 114972299 A CN114972299 A CN 114972299A CN 202210678529 A CN202210678529 A CN 202210678529A CN 114972299 A CN114972299 A CN 114972299A
Authority
CN
China
Prior art keywords
feature
data
models
training
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210678529.3A
Other languages
Chinese (zh)
Other versions
CN114972299B (en
Inventor
段勇
封皓元
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenyang University of Technology
Original Assignee
Shenyang University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenyang University of Technology filed Critical Shenyang University of Technology
Priority to CN202210678529.3A priority Critical patent/CN114972299B/en
Publication of CN114972299A publication Critical patent/CN114972299A/en
Application granted granted Critical
Publication of CN114972299B publication Critical patent/CN114972299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30116Casting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a railway track defect detection method based on deep migration learning, which belongs to the field of artificial intelligence and is characterized in that: the detection method comprises the steps of obtaining a data set for training and testing through preprocessing original picture data, designing three pre-training models, putting the data into the pre-training models for training to obtain three migration models, putting the data into three improved feature extraction models based on the migration models respectively for three times to realize extraction and abstraction of image features, and screening three groups of feature representations with the highest accuracy by utilizing distinguishing joint distribution self-adaptive similarity features and taking the accuracy of the second classification of a multilayer perceptron as a standard; and finally, fusing the features in a feature splicing mode, training a multi-layer perceptron classifier by using the fused features, and finally obtaining the analysis result of the second classification by the multi-layer perceptron classifier. The invention aims to effectively reduce the difference between features, enhance the feature extraction capability, enhance the generalization capability and improve the accuracy.

Description

Railway track defect detection method based on deep migration learning
Technical Field
The invention belongs to the field of artificial intelligence, and particularly relates to a railway track defect detection method based on deep migration learning.
Background
Rail transit has become an indispensable role in modern economic life as an important branch of the transportation field. Whether the rail fails or not is judged only through the working experience of people or according to the abnormal change of the current, so that more human resources and physical resources are consumed, and the current analysis of the rail failure is one of the research hotspots of rail transit.
At present, the intelligent detection method of the rail fault has the problems that the fault detection is carried out by specially processing data and establishing a mathematical model by using the data, the railway rail defect detection is realized by designing different solving algorithms through a model of a support vector machine, and various identification methods such as a principal component analysis method, a long-term and short-term memory network and the like have certain effects. However, the methods have the problems of limited feature extraction capability of a single network, weak generalization capability, inaccurate prediction and the like.
Disclosure of Invention
1. Objects of the invention
The invention aims to solve the defects in the prior art, and provides a rail fault detection and identification method which comprises the steps of preprocessing original image data, constructing ResNeXt, Xconcentration and SENet network models by using model parameters of an ImageNet data set, extracting image feature representation, adopting a discrimination joint distribution adaptive algorithm to similarity feature vectors to finish high-quality feature representation, fusing model features by taking the result as a criterion, and training the fused features through a multilayer perceptron to realize high accuracy. The feature extraction capability is enhanced, the generalization capability is enhanced, and the accuracy is improved.
2. Technical scheme
A railway track defect detection method based on deep migration learning is characterized in that: the detection method comprises the following steps:
step (1): acquiring original picture data of a defective track and a non-defective track;
step (2): dividing the original picture data in the step (1) into source domain data and target domain data, and then preprocessing the divided data to obtain a data set for training and testing;
and (3): constructing three deep learning models of ResNeXt, Xception and SENet, and respectively carrying out fine tuning treatment on the three models, namely converting original thousand classification results of each deep learning model into binary classification results while keeping original weight coefficients to obtain three pre-training models;
and (4): dividing the source domain data in the step (2) into a training set and a verification set according to a proportion, putting the training set and the verification set into the three pre-training models in the step (3) for three times for training, and obtaining three migration models for creating a feature extraction model;
and (5): based on the structure of the migration model in the step (4), removing the full connection layer of the last layer, creating three feature extraction models to extract features of the image data, namely, taking the data obtained by the source domain and the target domain as input in the step (2), respectively putting the input data into the three layers for three times, based on the structure of the migration model, removing the three feature extraction models improved by the full connection layer of the last layer, realizing extraction and abstraction of the image features, obtaining three groups of feature representations, wherein each group of feature representation is divided into source domain data and target domain data and represents the features extracted by three different feature extraction models with single deep learning;
and (6): applying the three groups of feature representations in the step (5) to the discrimination joint distribution self-adaptation of the field self-adaptation, realizing the feature transformation of reducing dimensionality and simulating each group of features, simultaneously establishing a multi-layer perceptron classifier, training the classifier by using the feature representation of each group of feature transformation in an iteration mode to obtain the accuracy of two classifications, and screening out the three groups of feature representations with the highest accuracy by taking the accuracy result as the standard;
and (7): and (4) fusing the three groups of feature representations obtained in the step (6) in a feature splicing mode to obtain fused features, training a multi-layer perceptron classifier by using the fused features, and finally obtaining an analysis result of the second classification by using the multi-layer perceptron classifier.
Further, the preprocessing of the picture in step (2) refers to: and performing data enhancement processing on the divided data of the source domain and the target domain to prevent overfitting generated by a training model, wherein:
and performing random cutting, horizontal turning, random scaling and standardization treatment on the source domain data, and finally dividing the data into a training set and a verification set. And carrying out random cutting, random scaling and standardization treatment on the target domain data to obtain a data set for training and testing.
Further, three deep learning models of ResNeXt, Xprediction and SENet are built in the step (3), fine tuning processing is carried out on the three models respectively, the weight coefficient trained in the ImageNet data set is reserved, and meanwhile, the structure of the model output layer is modified.
Further, in the step (4), the source domain data are divided into a training set and a verification set according to a proportion, the training set and the verification set are placed into three pre-training models for learning at different depths for three times, momentum, learning rate and training batch number parameters are adjusted in the training process of the models, and then training is carried out for multiple times until the models are converged to obtain three migration models with the highest accuracy rate results for creating the feature extraction model.
Further, in the step (5), based on the structure of the migration model, the parameters and the weights of the original three migration models are reserved, the full connection layer of the last layer of each migration model is deleted, feature extraction models corresponding to three different-deep learning models are created, three groups of different image data features are extracted, data obtained from a source domain and a target domain are used as input and are respectively placed into the three feature extraction models for three times, extraction and abstraction of the image features are realized, and three groups of feature representations are obtained; wherein each set of features represents features extracted by a feature extraction model divided into source domain and target domain data representing three different single deep learning.
Further, step (6) uses a domain adaptive discrimination joint distribution adaptive algorithm to realize the feature transformation of reducing dimensionality and similarity each group of features, specifically:
the error optimization of the target domain is realized by estimating the difference between the edge distribution and the condition distribution; in the calculation process, the algorithm minimizes the joint probability distribution difference of the same class among different domains, maximizes the joint probability distribution difference among different classes in different domains, and obtains a feature representation applied to screening the optimal similarity features;
the formula is as follows:
(X(R min -μR max )X T +λI)A=ηXHX T A
s.t.A T XHX T A=I
wherein X is a source domain feature X s And target Domain characteristics X t Of the combined matrix, X T Is the transpose of X matrix, R is the joint probability matrix, R min Is a matrix generated by measuring transferability of the same kind between different domains, R max Is a matrix generated by measuring the differentiability between different classes in different domains, mu is a trade-off parameter, lambda is a regularization parameter, eta is the kernel bandwidth of an RBF kernel, H is a central matrix, A is a feature transformation matrix, A is a T Is the transposition of A matrix, s.t. represents constraint condition, I is the condition that the formula needs to satisfy;
the specific iteration steps of the algorithm are as follows:
s1: constructing a joint probability matrix R min And R max
S2: solving a generalized characteristic decomposition problem in a formula, and selecting a designated subspace dimension to construct a characteristic transformation matrix A;
s3: transpose A with feature transformation matrix T And source domain feature X s And source domain label Y s Will (A) T X s ,Y s ) Putting the obtained object into a classifier for training;
s4: transpose by feature transform matrix A T And target domain feature X t A is T X t Putting the obtained product on a classifier to obtain a result;
s5: and repeating all the steps until reaching the specified iteration number.
Further, after the feature change is realized by judging the joint distribution self-adaptive processing, a multilayer perceptron classifier is established, the classifier is trained for a plurality of times by using the iterative mode through the feature representation of each group of feature transformation, the accuracy of a plurality of secondary classifications is obtained in the iterative process, and three groups of feature representations corresponding to the classifier with the highest accuracy are screened out by taking the result of the two classification accuracy as the standard.
Further, the three groups of feature representations with the best effect are fused in the step (7); the fusion mode fuses three groups of features in a feature splicing mode, source domain data of the three groups of features are fused in a feature splicing mode, target domain data of the three groups of features are fused in a feature splicing mode, fusion data of a group of source domains and a group of target domains are obtained, and then the multi-layer perceptron classifier is trained through the obtained fusion data.
Furthermore, the source domain data of the fusion features are used as input data, the data comprise feature representation and labels of whether the track is defective or not, the labels are used for training the multi-layer perceptron classifier, the multi-layer perceptron classifier is tested by using the feature representation of the target domain data, and finally, the classifier is used for obtaining a binary classification result of whether the track is defective or not.
3. The advantages and effects are as follows:
the invention has the beneficial effects that: by fusing the characteristics of various network models, the final accuracy is effectively improved. In addition, feature transformation is carried out through a discriminant joint distribution self-adaptive algorithm, so that the method is beneficial to minimizing the joint probability distribution difference of the same class among different domains and maximizing the joint probability distribution difference among different classes in different domains. The method and the device improve the accuracy result while improving the stability of feature extraction, and can be used for the problem of rail fault detection in the field of rail transit.
Description of the drawings:
FIG. 1 is a flow chart of track fault detection identification based on deep migration learning domain adaptation;
FIG. 2 is a process diagram of a fusion feature;
FIG. 3 is a graph of accuracy results of different models for different methods.
Detailed Description
The invention provides a self-adaptive track fault detection and identification method based on the field of deep migration learning, which utilizes a multi-layer perceptron classifier to make analysis by judging a joint distribution self-adaptive algorithm and fusing multiple model feature representations, thereby strengthening the feature extraction capability and simultaneously improving the accuracy.
As shown in fig. 1, a method for detecting defects of a railway track based on deep migration learning includes the following steps:
step (1): acquiring original picture data of a defective track and a non-defective track;
step (2): dividing the original picture data in the step (1) into source domain data and target domain data, and then preprocessing the divided data to obtain a data set for training and testing;
and (3): constructing three deep learning models of ResNeXt, Xprediction and SENet, and respectively carrying out fine adjustment processing on the three models, namely converting original thousand classification results of each deep learning model into two classification results, so that the model can complete the analysis on whether the track is defective or not, and obtaining three pre-training models;
and (4): dividing the source domain data in the step (2) into a training set and a verification set according to a proportion, and putting the training set and the verification set into the three pre-training models in the step (3) for training three times to obtain three migration models for creating a feature extraction model;
and (5): based on the structure of the migration model in the step (4), removing the last full-connection layer, creating three feature extraction models to extract features of the image data, taking the data obtained from the source domain and the target domain in the step (2) as input, and putting the data into the three feature extraction models respectively for three times to realize extraction and abstraction of the image features and obtain three groups of feature representations, wherein each group of feature representation is divided into source domain data and target domain data to represent the features extracted by the three different feature extraction models for single deep learning;
and (6): applying the three groups of feature representations in the step (5) to the discrimination joint distribution self-adaptation of the field self-adaptation, realizing the feature transformation of reducing dimensionality and simulating each group of features, simultaneously establishing a multi-layer perceptron classifier, training the classifier by using the feature representation of each group of feature transformation in an iteration mode to obtain the accuracy of two classifications, and screening out the three groups of feature representations with the highest accuracy by taking the accuracy result as the standard;
and (7): and (4) fusing the three groups of feature representations obtained in the step (6) in a feature splicing mode to obtain fused features, training a multi-layer perceptron classifier by using the fused features, and finally obtaining an analysis result of the second classification by using the multi-layer perceptron classifier.
The preprocessing of the picture in the step (2) refers to: in order to improve the robustness of the model, the divided data of the source domain and the target domain are subjected to data enhancement processing, so that the problem of overfitting generated by a training model is prevented, wherein:
and performing random cutting, horizontal turning, random scaling and standardization processing on the source domain data, and finally dividing the data into a training set and a verification set. And carrying out random cutting, random scaling and standardization treatment on the target domain data to obtain a data set for training and testing.
And (3) constructing three deep learning models of ResNeXt, Xchoice and SEnet and respectively carrying out fine adjustment processing on the three models, on one hand, keeping the weight coefficient trained in ImageNet to accelerate the convergence speed of the models, and on the other hand, modifying the structure of the last layer of full connection layer according to the classification result of whether the railway track is defective, so that the model can realize the task of analyzing whether the track is defective, and obtaining three pre-training models.
And (4) dividing the source domain data into a training set and a verification set according to a proportion, putting the training set and the verification set into three pre-training models for learning at different depths for three times, adjusting parameters such as momentum, learning rate, training batch number and the like in the training process of the models to improve the performance of the models, and then training the models for multiple times until the models are converged to obtain three migration models with the highest accuracy rate results for creating a feature extraction model.
And (5) based on the structure of the migration model, keeping the parameters and the weights of the original three migration models, deleting the full connection layer of the last layer of each migration model, creating feature extraction models corresponding to three different-deep learning models to extract three groups of different image data features, preventing the relatively low accuracy result obtained by directly obtaining a single deep learning network, taking the data obtained from the source domain and the target domain as input, and putting the data into the three feature extraction models for three times respectively to realize the extraction and abstraction of the image features and obtain three groups of feature representations. Wherein each set of features represents features extracted by a feature extraction model that is divided into source domain and target domain data, representing three different single deep learning.
As shown in fig. 2, in step (6), a domain-adaptive discrimination joint distribution adaptive algorithm is used to implement feature transformation for reducing dimensionality and for similarity of each group of features, and the feature transformation is characterized as follows:
and the error optimization of the target domain is realized by estimating the difference between the edge distribution and the condition distribution. In the calculation process, the algorithm minimizes the joint probability distribution difference of the same class among different domains, maximizes the joint probability distribution difference among different classes in different domains, and obtains the feature representation applied to screening the optimal similarity features.
The formula is as follows:
(X(R min -μR max )X T +λI)A=ηXHX T A
s.t.A T XHX T A=I
wherein X is a source domain feature X s And target Domain characteristics X t Of the combined matrix, X T Is the transpose of the X matrix, R is the joint probability matrix, R min Is a matrix generated by measuring transferability of the same kind between different domains, R max Is a matrix generated by measuring the differentiability between different classes in different domains, mu is a trade-off parameter, lambda is a regularization parameter, eta is the kernel bandwidth of an RBF kernel, H is a central matrix, A is a feature transformation matrix, A is a T Is the transpose of the A matrix, s.t. represents the constraint condition, and I is the condition that the formula needs to satisfy.
The specific iteration steps of the algorithm are as follows:
s1: constructing a joint probability matrix R min And R max
S2: solving the generalized characteristic decomposition problem in the formula, selecting the dimension of the appointed subspace to construct a characteristic transformation matrix A
S3: transpose A with feature transformation matrix T And source domain feature X s And source domain label Y s Will (A) T X s ,Y s ) Putting the data into a classifier for training.
S4: transpose by feature transform matrix A T And target domain feature X t A is T X t And putting the obtained product into a classifier to obtain a result.
S5: and repeating all the steps until reaching the specified iteration number.
After the feature change is realized by judging the joint distribution self-adaptive processing, a multilayer perceptron classifier is established, the classifier is trained for a plurality of times by using the iterative mode through the feature representation of each group of feature transformation, the accuracy of a plurality of secondary classifications is obtained in the iterative process, and three groups of feature representations corresponding to the classifier with the highest accuracy are screened out by taking the result of the two classification accuracy as the standard.
And (7) fusing the three groups of feature representations with the best effect, and combining the image features with different styles to improve the upper limit of the model identification accuracy. The fusion mode fuses three groups of features in a feature splicing mode, source domain data of the three groups of features are fused in a feature splicing mode, target domain data of the three groups of features are fused in a feature splicing mode, fusion data of a group of source domains and a group of target domains are obtained, and then the multi-layer perceptron classifier is trained through the obtained fusion data.
And (8) taking the source domain data of the fusion features as input data, wherein the data comprises feature representation and a label of whether the track is defective or not, and is used for training the multi-layer perceptron classifier, testing the multi-layer perceptron classifier by using the feature representation of the target domain data, and finally obtaining a binary classification result of whether the track is defective or not by using the classifier.
As shown in fig. 3, a graph of accuracy results of different models for different methods is shown as a comparison of the present invention with other methods. Wherein, the oblique lines, the cross lines and the dots respectively represent the use of ResNeXt, Xconcentration and SENet network models, and the stars represent three fused models. The horizontal axes respectively represent the results obtained using only a single network model, the results obtained using the discriminative joint distribution adaptation method, and the results of the present invention. In fig. 3, if the last fully-connected layer is not removed, the highest accuracy can only reach 0.77 at the leftmost side of fig. 3, and if the last fully-connected layer is removed and the data re-fusion feature is adaptively processed by using the discriminant joint distribution, the highest accuracy can reach 0.95 at the rightmost side of fig. 3.
Example 1
As shown in fig. 1, the method for detecting defects of a railway track based on deep migration learning comprises the following steps:
step 1: raw picture data of defective tracks and non-defective tracks are acquired.
Step 2: the method comprises the steps of dividing original picture data into source domain data and target domain data according to a proportion, then preprocessing the picture, for preprocessing the source domain data, firstly randomly dividing the source domain data into a training set and a verification set according to the proportion, then carrying out data enhancement processing of random cutting, horizontal turning and random scaling and standardization processing on the picture, and directly carrying out random cutting, random scaling and standardization processing on the target domain data to serve as a test set.
And step 3: the method comprises the steps of creating ResNeXt, Xchoice and SEnet models pre-trained by ImageNet1000, finely adjusting the three models according to rail fault detection classification results, and converting original thousand classification results of each deep learning model into two classification results of whether a railway rail is defective or not so that the model can realize a task of analyzing whether the rail is defective or not to obtain the three pre-trained models because the created models are all trained by ImageNet1000 data sets and are currently classified into 1000 types.
And 4, step 4: and (3) taking the source domain data processed in the step (2) as input, putting the source domain data into three pre-training models for training at different depths, adjusting parameters such as momentum, learning rate, training batch number and the like in the training process of the models, and then training for multiple times until the models are converged to obtain three migration models with the highest accuracy results.
And 5: the method comprises the steps of improving three migration model network structures, extracting and abstracting data features of a source domain and a target domain, wherein the improved network model is based on the migration model network structure, creating feature extraction models corresponding to learning models of three different depths by reserving parameters and weights of the original three migration models and deleting a full connection layer of the last layer of each migration model, and extracting three groups of feature representations through the feature extraction models, wherein each group of feature representation is divided into data of the source domain and the target domain and represents features extracted by the three feature extraction models. Each piece of data is composed of a 2048-dimensional feature representation and a label of whether the 1-dimensional track is faulty or not.
Step 6: and respectively carrying out feature transformation on each group of source domain and target domain data by the three groups of features through a discriminant joint distribution adaptive algorithm so as to improve the similarity between the source domain and the target domain data, and simultaneously reducing the dimension of feature representation to 100 dimensions, wherein the obtained feature representation is applied to screening out the optimal similarity feature.
And 7: and (3) establishing a multi-layer perceptron classifier in an iteration mode, putting the data obtained in the step (6) into the classifier for training, obtaining the accuracy of the second classification of whether the track is in fault or not for many times in the iteration process, screening three groups of characteristic representations corresponding to the classifier with the highest accuracy by taking the result of the accuracy of the second classification as a standard, and meanwhile, carrying out persistent storage on the three groups of characteristic representations.
And 8: and (4) fusing the three groups of feature representations obtained in the step (7) in a feature splicing mode, taking the source domain data of the fused features as a training set, taking the target domain data as a test set, putting the target domain data into a multi-layer perceptron classifier for training and testing, and finally obtaining a detection result of whether the track is in fault.

Claims (9)

1. A railway track defect detection method based on deep migration learning is characterized in that: the detection method comprises the following steps:
step (1): acquiring original picture data of a defective track and a non-defective track;
step (2): dividing the original picture data in the step (1) into source domain data and target domain data, and then preprocessing the divided data to obtain a data set for training and testing;
and (3): constructing three deep learning models of ResNeXt, Xception and SENet, and respectively carrying out fine tuning treatment on the three models, namely converting original thousand classification results of each deep learning model into binary classification results while keeping original weight coefficients to obtain three pre-training models;
and (4): dividing the source domain data in the step (2) into a training set and a verification set according to a proportion, putting the training set and the verification set into the three pre-training models in the step (3) for three times for training, and obtaining three migration models for creating a feature extraction model;
and (5): based on the structure of the migration model in the step (4), removing the full connection layer of the last layer, creating three feature extraction models to extract features of the image data, namely, taking the data obtained by the source domain and the target domain as input in the step (2), respectively putting the input data into the three layers for three times, based on the structure of the migration model, removing the three feature extraction models improved by the full connection layer of the last layer, realizing extraction and abstraction of the image features, obtaining three groups of feature representations, wherein each group of feature representation is divided into source domain data and target domain data and represents the features extracted by three different feature extraction models with single deep learning;
and (6): applying the three groups of feature representations in the step (5) to the discrimination joint distribution self-adaptation of the field self-adaptation, realizing the feature transformation of reducing dimensionality and simulating each group of features, simultaneously establishing a multi-layer perceptron classifier, training the classifier by using the feature representation of each group of feature transformation in an iteration mode to obtain the accuracy of two classifications, and screening out the three groups of feature representations with the highest accuracy by taking the accuracy result as the standard;
and (7): and (4) fusing the three groups of feature representations obtained in the step (6) in a feature splicing mode to obtain fused features, training a multi-layer perceptron classifier by using the fused features, and finally obtaining an analysis result of the second classification by using the multi-layer perceptron classifier.
2. The railway track defect detection method based on deep migration learning of claim 1, wherein: the preprocessing of the picture in the step (2) refers to: and performing data enhancement processing on the divided data of the source domain and the target domain to prevent overfitting generated by the training model, wherein:
and performing random cutting, horizontal turning, random scaling and standardization treatment on the source domain data, and finally dividing the data into a training set and a verification set. And carrying out random cutting, random scaling and standardization treatment on the target domain data to obtain a data set for training and testing.
3. The railway track defect detection method based on deep migration learning of claim 1, wherein: and (3) constructing three deep learning models of ResNeXt, Xprediction and SENet, respectively carrying out fine tuning treatment on the three models, keeping the weight coefficient trained in the ImageNet data set, and modifying the structure of the model output layer.
4. The deep migration learning-based railway track defect detection method according to claim 1, wherein: and (4) dividing the source domain data into a training set and a verification set according to a proportion, training the training set and the verification set in three pre-training models for learning at different depths, adjusting parameters of momentum, learning rate and training batch number in the training process of the models, and then training the models for multiple times until the models are converged to obtain three migration models with the highest accuracy rate results for creating a feature extraction model.
5. The deep migration learning-based railway track defect detection method according to claim 1, wherein: based on the structure of the migration model, parameters and weights of the original three migration models are reserved, a full connection layer of the last layer of each migration model is deleted, feature extraction models corresponding to three learning models with different depths are created, three groups of different image data features are extracted, data obtained from a source domain and a target domain are used as input and are respectively placed into the three feature extraction models in three times, extraction and abstraction of the image features are achieved, and three groups of feature representations are obtained; wherein each set of features represents features extracted by a feature extraction model divided into source domain and target domain data representing three different single deep learning.
6. The railway track defect detection method based on deep migration learning of claim 1, wherein: step (6) uses a domain adaptive discrimination joint distribution adaptive algorithm to realize dimension reduction and similarity feature transformation of each group of features, and specifically comprises the following steps:
the error optimization of the target domain is realized by estimating the difference between the edge distribution and the condition distribution; in the calculation process, the algorithm minimizes the joint probability distribution difference of the same class among different domains, maximizes the joint probability distribution difference among different classes in different domains, and obtains a feature representation applied to screening the optimal similarity features;
the formula is as follows:
(X(R min -μR max 0X T +λI)A=ηXHX T A
s.t.A T XHX T A=I
wherein X is a source domain feature X s And target Domain characteristics X t Of the combined matrix, X T Is the transpose of X matrix, R is the joint probability matrix, R min Is a matrix generated by measuring transferability of the same kind between different domains, R max Is a matrix generated by measuring the differentiability between different classes in different domains, mu is a trade-off parameter, lambda is a regularization parameter, eta is the kernel bandwidth of an RBF kernel, H is a central matrix, A is a feature transformation matrix, A is a T Is the transposition of A matrix, s.t. represents constraint condition, I is the condition that the formula needs to satisfy;
the specific iteration steps of the algorithm are as follows:
s1: constructing a joint probability matrix R min And R max
S2: solving a generalized characteristic decomposition problem in a formula, and selecting a designated subspace dimension to construct a characteristic transformation matrix A;
s3: transpose A with feature transformation matrix T And source domain feature X s And source domain label Y s Will (A) T X s ,Y s ) Putting the obtained object into a classifier for training;
s4: transpose by feature transform matrix A T And target domain feature X t A is T X t Putting the obtained product into a classifier to obtain a result;
s5: and repeating all the steps until reaching the specified iteration number.
7. The railway track defect detection method based on deep migration learning of claim 6, wherein: after the feature change is realized by judging the joint distribution self-adaptive processing, a multilayer perceptron classifier is established, the classifier is trained for a plurality of times by using the iterative mode through the feature representation of each group of feature transformation, the accuracy of a plurality of secondary classifications is obtained in the iterative process, and three groups of feature representations corresponding to the classifier with the highest accuracy are screened out by taking the result of the two classification accuracy as the standard.
8. The deep migration learning-based railway track defect detection method according to claim 1, wherein: step (7) fusing the three groups of feature representations with the best effect; the fusion mode fuses three groups of features in a feature splicing mode, source domain data of the three groups of features are fused in a feature splicing mode, target domain data of the three groups of features are fused in a feature splicing mode, fusion data of a group of source domains and a group of target domains are obtained, and then the multi-layer perceptron classifier is trained through the obtained fusion data.
9. The railway track defect detection method based on deep migration learning of claim 8, wherein: and taking the source domain data fused with the characteristics as input data, wherein the data comprises characteristic representation and a label for judging whether the track is defective or not, and is used for training the multi-layer perceptron classifier, testing the multi-layer perceptron classifier by using the characteristic representation of the target domain data, and finally obtaining a classification result whether the track is defective or not by using the classifier.
CN202210678529.3A 2022-06-16 2022-06-16 Railway track defect detection method based on deep migration learning Active CN114972299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210678529.3A CN114972299B (en) 2022-06-16 2022-06-16 Railway track defect detection method based on deep migration learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210678529.3A CN114972299B (en) 2022-06-16 2022-06-16 Railway track defect detection method based on deep migration learning

Publications (2)

Publication Number Publication Date
CN114972299A true CN114972299A (en) 2022-08-30
CN114972299B CN114972299B (en) 2024-03-26

Family

ID=82963289

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210678529.3A Active CN114972299B (en) 2022-06-16 2022-06-16 Railway track defect detection method based on deep migration learning

Country Status (1)

Country Link
CN (1) CN114972299B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117994753A (en) * 2024-04-03 2024-05-07 浙江浙能数字科技有限公司 Vision-based device and method for detecting abnormality of entrance track of car dumper

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112613410A (en) * 2020-12-24 2021-04-06 江苏大学 Parasite egg identification method based on transfer learning
CN113010013A (en) * 2021-03-11 2021-06-22 华南理工大学 Wasserstein distance-based motor imagery electroencephalogram migration learning method
US20220092420A1 (en) * 2020-09-21 2022-03-24 Intelligent Fusion Technology, Inc. Method, device, and storage medium for deep learning based domain adaptation with data fusion for aerial image data analysis
CN114334139A (en) * 2022-01-25 2022-04-12 山东工商学院 Epileptic seizure detection system based on EEG feature distribution adaptation transfer learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220092420A1 (en) * 2020-09-21 2022-03-24 Intelligent Fusion Technology, Inc. Method, device, and storage medium for deep learning based domain adaptation with data fusion for aerial image data analysis
CN112613410A (en) * 2020-12-24 2021-04-06 江苏大学 Parasite egg identification method based on transfer learning
CN113010013A (en) * 2021-03-11 2021-06-22 华南理工大学 Wasserstein distance-based motor imagery electroencephalogram migration learning method
CN114334139A (en) * 2022-01-25 2022-04-12 山东工商学院 Epileptic seizure detection system based on EEG feature distribution adaptation transfer learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
杜超;刘桂华;: "改进的VGG网络的二极管玻壳图像缺陷检测", 图学学报, no. 06, 15 December 2019 (2019-12-15) *
闫美阳;李原;: "多源域混淆的双流深度迁移学习", 中国图象图形学报, no. 12, 16 December 2019 (2019-12-16) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117994753A (en) * 2024-04-03 2024-05-07 浙江浙能数字科技有限公司 Vision-based device and method for detecting abnormality of entrance track of car dumper
CN117994753B (en) * 2024-04-03 2024-06-07 浙江浙能数字科技有限公司 Vision-based device and method for detecting abnormality of entrance track of car dumper

Also Published As

Publication number Publication date
CN114972299B (en) 2024-03-26

Similar Documents

Publication Publication Date Title
CN113378632B (en) Pseudo-label optimization-based unsupervised domain adaptive pedestrian re-identification method
EP3767536A1 (en) Latent code for unsupervised domain adaptation
CN109359608B (en) Face recognition method based on deep learning model
CN111738315A (en) Image classification method based on countermeasure fusion multi-source transfer learning
CN106650789A (en) Image description generation method based on depth LSTM network
CN111353373B (en) Related alignment domain adaptive fault diagnosis method
CN113807420A (en) Domain self-adaptive target detection method and system considering category semantic matching
CN114842267A (en) Image classification method and system based on label noise domain self-adaption
CN111275175A (en) Neural network training method, neural network training device, image classification method, image classification equipment and medium
CN111783841A (en) Garbage classification method, system and medium based on transfer learning and model fusion
CN112766386A (en) Generalized zero sample learning method based on multi-input multi-output fusion network
CN111652264B (en) Negative migration sample screening method based on maximum mean value difference
CN114998602A (en) Domain adaptive learning method and system based on low confidence sample contrast loss
CN110110845A (en) Learning method based on parallel multi-level width neural network
Lien et al. Product surface defect detection based on deep learning
CN112668698A (en) Neural network training method and system
CN114067314B (en) Neural network-based peanut mildew identification method and system
CN116192500A (en) Malicious flow detection device and method for resisting tag noise
CN109101984B (en) Image identification method and device based on convolutional neural network
CN112949344B (en) Characteristic autoregression method for anomaly detection
CN114972299A (en) Railway track defect detection method based on deep migration learning
CN105787045B (en) A kind of precision Enhancement Method for visual media semantic indexing
CN117058716A (en) Cross-domain behavior recognition method and device based on image pre-fusion
CN116883316A (en) Method for constructing chip quality detection model and terminal
CN114495114B (en) Text sequence recognition model calibration method based on CTC decoder

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant