CN110414383A - Convolutional Neural Network Adversarial Transfer Learning Method Based on Wasserstein Distance and Its Application - Google Patents

Abstract

The invention relates to a convolutional neural network confrontation migration learning method based on Wasserstein distance and its application, comprising: using a convolutional neural network to be migrated to obtain a source domain feature set and a source domain fault judgment set of a source domain marked sample set; The target feature set of the target domain sample set; with the goal of maximizing the Wasserstein distance between the source domain feature set and the target feature set and minimizing the sum of the Wasserstein distance and the judgment loss value of the source domain fault judgment set, based on the convergence criterion , which implements adversarial transfer learning for convolutional neural networks. The invention introduces the Wasserstein distance in the migration learning of the convolutional neural network, takes the maximum Wasserstein distance as the goal, improves the discrimination sensitivity of the features extracted from the two sample sets, and then uses the Wasserstein distance and the loss value of the source domain fault judgment set. In order to improve the judgment accuracy of the convolutional neural network, it has low requirements on the sample data and network structure while ensuring the fault diagnosis ability. It is suitable for migration between multiple working conditions and has strong practical applicability.

Description

Convolutional neural network adversarial transfer learning method based on Wasserstein distance and its application

technical field

The invention belongs to the field of industrial process fault diagnosis, and more particularly, relates to a convolutional neural network confrontation transfer learning method based on Wasserstein distance and its application,

Background technique

Fault diagnosis aims to isolate faults on the system by monitoring and analyzing machine status using acquired measurement data and other information. This requires highly skilled and experienced experts, which increases the use of artificial intelligence techniques. The need for troubleshooting decisions. The deployment of a real-time troubleshooting framework allows maintenance teams to take early action to replace or repair affected components, increasing productivity and maintaining operational safety. Therefore, accurate diagnosis of bearing faults is of critical significance to the reliability and safety of mechanical manufacturing systems.

Many advanced signal processing and machine learning techniques have been used for fault diagnosis. In the past few years, deep learning models (e.g., deep belief networks, sparse autoencoders, and especially convolutional neural networks), have shown better performance than rule-based and model-based methods in fault diagnosis tasks Combined learning capability, which aims to leverage knowledge from the relevant source domain (with sufficient labeled data) to learn in the target domain (insufficient or no labeled data), saving the need to rebuild a new fault diagnosis model from scratch considerable time and computational cost.

However, the above deep learning models still suffer from two difficulties: 1) Most methods need to be based on the IID assumption, that is, the sample sets of the source and target domain tasks require the same distribution. Therefore, the adaptability of pre-trained networks is limited when faced with new diagnostic tasks, where different operating conditions and physical properties of the new task may result in a new sample set (target sample set) and original sample set (source sample set) The difference in distribution between them leads to the following: For new fault diagnosis tasks, deep learning models usually need to be trained from scratch, which leads to wasted computing resources and training time; 2) Insufficient labeled or unlabeled data in the target domain is another common problem. In practical industrial situations, for new diagnostic tasks, it is extremely difficult to collect enough typical samples to construct large-scale and high-quality datasets to train networks. And because it is difficult to install enough sensors in the packaging equipment, at the same time, industrial marking often requires expensive labor. Therefore, the challenge of domain adaptation is that labeled data cannot be collected in the target domain or only a small amount of labeled data can be collected.

Based on the above challenges, the current deep transfer learning algorithm has made some improvements. One is the transfer network based on the collected data, which is mainly to transfer the selected data in the source domain to the data collected in the source domain by assigning appropriate weight values. As a data supplement in the target domain, this transfer learning method does not depend on the model and requires high data similarity. The second is the transfer network based on network structure, which refers to the transfer of part of the network pre-trained in the source domain, including its network structure and connection parameters, to convert it into a part of the deep neural network used in the target domain, but this transfer The learning method only has good results for some network structures, such as: LeNet, AlexNet, VGG, Inception, ResNet. The third is the transfer network based on domain fusion, in which the maximum average difference method is the most common, but this maximum average difference method, the computational cost will increase quadratically with the increase of the number of samples, which limits MMD in many applications with Applicability to practical applications of large datasets.

Therefore, developing a deep transfer learning algorithm with strong industrial practicability and high fault diagnosis ability is an urgent technical problem to be solved in the field of target industrial process fault diagnosis.

SUMMARY OF THE INVENTION

The invention provides a convolutional neural network confrontation transfer learning method based on Wasserstein distance and its application, which are used to solve the requirements of the existing deep transfer learning method for actual sample data and/or neural network structure while ensuring the fault diagnosis capability of industrial processes. It is high and leads to technical problems that are difficult to apply to actual industrial processes.

The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a Convolutional Neural Network confrontation transfer learning method based on Wasserstein distance, comprising:

Step 1. From the source domain labeled dataset and the target domain dataset, determine the source domain labeled sample set and the target domain sample set;

Step 2, using the convolutional neural network to be transferred and learned, to obtain the source domain feature set and the source domain fault judgment set of the source domain marked sample set and the target feature set of the target domain sample set;

Step 3. With the goal of maximizing the Wasserstein distance between the source domain feature set and the target feature set and minimizing the sum of the Wasserstein distance and the judgment loss value of the source domain fault judgment set, adjust all parameters. The parameters of the convolutional neural network described above, based on the convergence criterion, repeat step 1, or, complete the adversarial transfer learning of the convolutional neural network.

The beneficial effects of the present invention are as follows: the present invention introduces the Wasserstein distance in the migration learning of the convolutional neural network, takes the maximum Wasserstein distance as the target, improves the discrimination sensitivity of the features extracted from the two sample sets, and then uses the Wasserstein distance and the source The goal is to minimize the sum of the judgment loss values of the domain fault judgment set to improve the judgment accuracy of the convolutional neural network. Based on the above goals, the parameters of the convolutional neural network are optimized. During the training process, the two goals form a confrontation to achieve deep adversarial transfer learning. Therefore, the convolutional neural network trained by the present invention can minimize the distribution difference between the source domain and the target domain, and using unlabeled target domain data for unsupervised transfer learning is sufficient to obtain a high-precision fault diagnosis capability. To a certain extent, it solves the technical problems of the waste of computing resources and training time caused by the need to rebuild the deep learning model from scratch in the face of new fault diagnosis tasks in the existing technology and the lack of sufficient labeled data in the target domain, and it is in the industry to ensure In addition to the process fault diagnosis ability, it does not have particularly high requirements on sample data and network structure, and is suitable for fault diagnosis of actual industrial processes.

On the basis of the above technical solutions, the present invention can also be improved as follows.

Further, the Wasserstein distance is:

By using the domain similarity evaluation neural network, the source domain feature set is mapped to the source domain real number set, the target feature set is mapped to the target real number set, and the real number average value of the source domain real number set is subtracted from all real numbers. Calculate the average value of the real numbers of the target real number set.

The further beneficial effects of the present invention are as follows: the present invention introduces a domain similarity evaluation neural network, respectively maps the source domain judgment set and the target judgment set to real numbers, calculates the Wasserstein distance based on the real numbers, takes the maximum Wasserstein distance as the target, and trains the training based on the domain similarity. The evaluation neural network improves the discrimination sensitivity of the domain similarity evaluation neural network to the features extracted from the two sample sets. Among them, the Wasserstein distance calculation is performed based on the average value of real numbers, which is simple and highly reliable.

Further, the step 3 includes:

Step 3.1, with the goal of increasing the Wasserstein distance between the source domain feature set and the target feature set, optimize the parameters of the domain similarity evaluation neural network, and obtain a new domain similarity corresponding to the preset number of iterations Evaluate neural networks;

Step 3.2. Evaluate the neural network based on the source domain labeled sample set, the target domain sample set and the new domain similarity to reduce the addition of the Wasserstein distance and the judgment loss value of the convolutional neural network. and as the goal, optimize the parameters of the convolutional neural network to obtain a new convolutional neural network corresponding to the preset number of iterations;

Step 3.3: Evaluate the convergence criterion of the neural network and the convolutional neural network based on the domain similarity, stop training, and complete the confrontation transfer learning of the convolutional neural network, or repeat step 1.

The further beneficial effects of the invention are: aiming at maximizing the Wasserstein distance, the domain similarity evaluation neural network is trained, and the discrimination sensitivity of the domain similarity evaluation neural network to the features extracted from the two sample sets is improved. After that, the parameters of the neural network are evaluated by the fixed domain similarity, and the parameters of the convolutional neural network are adjusted. On the one hand, the judgment loss value of the source domain sample set is obtained based on the convolutional neural network. The domain feature set and the target domain feature set are input into the above trained domain similarity evaluation neural network, and the Wasserstein distance is obtained, and the fault judgment convolutional neural network is trained with the goal of minimizing the sum of the Wasserstein distance and the judgment loss value. Therefore, based on the method of first training the domain similarity evaluation neural network, then training the convolutional neural network, and then training the domain similarity evaluation neural network and then training the convolutional neural network, iterates many times until the two neural networks converge, and the volume is completed. The product neural network is trained, so that the product neural network has a higher fault judgment ability.

Further, in the step 3.1, the Wasserstein distance is the difference between the real average value of the source domain real number set and the real number average value of the target real number set based on gradient penalty;

In the step 3.2, the Wasserstein distance is the difference between the real average value of the source domain real number set and the real number average value of the target real number set.

Further beneficial effects of the present invention are: the present invention introduces gradient penalty to limit the domain similarity evaluation neural network, prevent the parameters from being too complicated, reduce the computational complexity, and improve the convergence speed of the domain similarity evaluation neural network and the convolutional neural network. In addition, when optimizing the parameters of the convolutional neural network, the gradient penalty is ignored, and the gradient penalty does not affect the learning process of the feature representation, which avoids the fault diagnosis ability of the convolutional neural network by the gradient penalty.

Further, the Wasserstein distance expresses:

The real average value of the sub-real number set in the source domain minus the real average value of the target sub-real number set;

Wherein, the source domain sub-real number set is obtained from the source domain real number set through the Lipschitz constraint, and the target sub-real number set is obtained from the target real number set through the Lipschitz constraint.

The further beneficial effects of the present invention are: based on the Lipschitz constraint, the real number set required for calculating the Wasserstein distance is obtained, the data dimension is reduced, the fault diagnosis capability of the convolutional neural network is guaranteed, and the domain similarity evaluation neural network and the neural network are improved. Convergence speed of convolutional neural networks.

Further, the Lipschitz constraint is specifically a first-order Lipschitz constraint.

The further beneficial effects of the present invention are: adopting the first-order Lipschitz constraint to calculate the Wasserstein distance to ensure the continuity and derivability of the convolutional neural network, while ensuring the fault diagnosis capability of the convolutional neural network, improving the Domain similarity evaluates the convergence speed of neural networks and convolutional neural networks.

Further, when the target domain sample set includes some marked samples, the step 2 is:

Using the convolutional neural network to be transferred and learned, obtain the source domain feature set and source domain fault judgment set of the source domain marked sample set and the target feature set and target fault judgment set of the target domain sample set;

The step 3 is:

To maximize the Wasserstein distance between the source domain feature set and the target feature set and minimize the Wasserstein distance, the judgment loss value of the source domain fault judgment set and the judgment loss value of the target fault judgment set The sum of the convolutional neural network is adjusted as the goal, and based on the convergence criterion, step 1 is repeated, or, the adversarial transfer learning of the convolutional neural network is completed.

A further beneficial effect of the present invention is that the deep adversarial transfer learning method of the present invention, even in the face of transfer tasks between tasks that are related but not sufficiently similar (such as transfer learning between different sensor positions), is incompatible with a large number of unlabeled samples. Compared with the unsupervised case of , only adding a small number of labeled target domain samples greatly improves the accuracy of transfer learning. Therefore, the present invention is not only applicable to the case where the target domain sample set is unmarked, but also applicable to the partially marked sample set, and can be applied flexibly. Moreover, when a partially labeled target domain sample set is used, the accuracy of judging industrial process failures by the convolutional neural network obtained by adversarial transfer training can be further improved, and the practicability is strong.

The present invention also provides a convolutional neural network for fault diagnosis in an industrial process, which is obtained by training using any of the above-mentioned Wasserstein distance-based convolutional neural network confrontation transfer learning methods.

The beneficial effects of the present invention are: based on the correlation between tasks, any one of the above-mentioned Wasserstein distance-based convolutional neural networks can be used to confront the convolutional neural network trained by the migration learning method, and the convolutional neural network corresponding to the source domain can be Deep adversarial transfer learning is a high-precision convolutional neural network suitable for source and target domains. The waste of training time and the lack of sufficient sample data in the target domain are technical problems.

The present invention also provides an industrial process fault diagnosis method, based on any of the above-mentioned industrial process fault diagnosis convolutional neural networks, when receiving any of the above-mentioned Wasserstein distance-based convolutional neural networks against transfer learning When a new sample of the target domain is used in the method, the industrial process fault judgment result corresponding to the new sample is obtained.

The beneficial effects of the present invention are as follows: using any of the above-mentioned Wasserstein distance-based convolutional neural networks against the convolutional neural network trained by the migration learning method to carry out fault diagnosis in industrial processes, under the premise of ensuring safety and high efficiency, the accuracy of fault diagnosis is improved. higher.

The present invention also provides a storage medium, the storage medium stores instructions, when the computer reads the instructions, the computer is made to execute any of the above-mentioned Wasserstein distance-based convolutional neural network adversarial transfer learning methods and/or Or any of the industrial process fault diagnosis methods described above.

Description of drawings

1 is a flowchart of a Wasserstein distance-based convolutional neural network confrontation transfer learning method provided by an embodiment of the present invention;

2 is a flowchart of a Wasserstein distance-based convolutional neural network adversarial transfer learning method provided by an embodiment of the present invention;

3 is a schematic diagram of the visualization of the convolutional neural network output of the migration task US(C)→US(A) provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram of a convolutional neural network output visualization of a migration task US(E)→US(F) provided by an embodiment of the present invention;

FIG. 5 is a graph showing the variation curve of the diagnostic accuracy of the task (a) US(E)→US(F) and the task (b) S(E)→S(F) with the number of samples provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Example 1

A Convolutional Neural Network Adversarial Transfer Learning Method 100 Based on Wasserstein Distance, as shown in Figure 1, includes:

Step 110, from the source domain labeled dataset and the target domain dataset, determine the source domain labeled sample set and the target domain sample set;

Step 120, using the convolutional neural network to be transferred and learned, to obtain the source domain feature set and the source domain fault judgment set of the source domain marked sample set and the target feature set of the target domain sample set;

Step 130, with the goal of maximizing the Wasserstein distance between the source domain feature set and the target feature set and minimizing the sum of the Wasserstein distance and the judgment loss value of the source domain fault judgment set, adjust the parameters of the convolutional neural network, based on convergence Criterion, repeat step 110, or complete the adversarial transfer learning of the convolutional neural network.

It should be noted that, before the present invention, a convolutional neural network needs to be obtained by training based on the source domain data set, and the convolutional neural network is the convolutional neural network to be transferred and learned.

Specifically, first, the convolutional neural network model is pre-trained using the labeled dataset Xs of the source domain: the convolutional layer contains a filter of size ^k and bias are used to compute features. An output feature v _i is obtained by filter w and a nonlinear activation function Γ, and the specific expression is: υ _i =Γ(w*u _j +b), where, is the input data representing the jth sub-vector of the source domain dataset X ^s , and "*" represents the convolution operation. Modified Linear Units (ReLUs) with nonlinear activation functions to reduce the risk of vanishing gradients affecting the deep learning optimization process. Therefore, define the feature map as v=[υ ₁ , υ ₂ , . . . , υ _L ], where L=(pN-s)/I _cv +1 is the number of features, p represents the size of the padding, and N represents the input dimension, s is the filter size, is the stride of the convolution operation.

Then, a max pooling layer is applied to the feature map to extract the largest feature value within a stride range The filter size corresponding to max pooling is β and the step size is I _pl .

By alternately stacking multiple convolutional layers and max-pooling layers (with variable filter size), a multi-layer structure for describing features is constructed. The output features of the multilayer structure are flattened and passed to a fully connected layer for classification, resulting in the final output of a probability distribution over the labels. The pretrained convolutional neural network in the source domain uses the Softmax function to obtain the final classification result. To compute predicted labels in the source domain and labels for real data The difference between , the loss _lc is calculated using cross entropy, namely:

The transferable features of the target domain with unlabeled data are passed through the feature extractor of the above-trained convolutional neural network (that is, the network structure before the fully connected layer, including the convolutional layer and the maximum pooling layer. structure) directly obtained.

The next step is to measure the difference in feature distribution between the source and target datasets. Specifically, adversarial training can be performed based on a common latent space between two feature distributions using Wasserstein distance to learn to extract invariant features. Specifically, a feature extractor from a pretrained convolutional neural network model is used to learn features from both domains. For n < N ^s , N ^t , given two mini-batch instances from X ^s and X ^t and Both instances pass through a feature extractor with parameter θ _f The source domain features h ^s =r _f (x ^s ) and the target domain features h ^t =r _f (x ^t ) are directly generated. Assume and are the distributions of h ^s and h ^t , respectively. Among them, ( ) ^s and ( ) ^t represent the source domain and target domain information, respectively.

The purpose of domain adaptation via Wasserstein distance is to optimize the parameter _θf to reduce the distribution and The distance between, i.e. learning invariant features of the two domains.

This embodiment adopts the idea of back-propagation theory to optimize the model. The Wasserstein distance is introduced in the transfer learning of the convolutional neural network, and the maximum Wasserstein distance is taken as the goal to improve the discrimination sensitivity of the features extracted from the two sample sets. The minimum summation is the goal to improve the judgment accuracy of the convolutional neural network. Based on the above goals, the parameters of the convolutional neural network are changed. During the training process, the two goals form a confrontation to achieve deep adversarial transfer learning. Therefore, the convolutional neural network trained in this embodiment can minimize the distribution difference between the source domain and the target domain, and using unlabeled target domain data for unsupervised transfer learning is sufficient to obtain a high-precision fault diagnosis capability. To a certain extent, it solves the technical problems of the waste of computing resources and training time caused by the need to rebuild the deep learning model from scratch in the face of new fault diagnosis tasks in the existing technology and the lack of sufficient sample data in the target domain.

Preferably, the Wasserstein distance is:

By using the domain similarity evaluation neural network, the source domain feature set is mapped to the source domain real number set, the target feature set is mapped to the target real number set, and the real number average value of the source domain real number set is subtracted from the real number average value of the target real number set. Calculated.

In this embodiment, a domain similarity evaluation neural network is introduced, the source domain judgment set and the target judgment set are respectively mapped to real numbers, the Wasserstein distance is calculated based on the real numbers, and the maximum Wasserstein distance is taken as the goal, and the domain similarity evaluation neural network is trained to improve the domain similarity Evaluate the discriminative sensitivity of the neural network to the features extracted from the two sample sets. Among them, the Wasserstein distance calculation is performed based on the average value of real numbers, which is simple and highly reliable.

Preferably, step 130 includes:

Step 131, aiming to increase the Wasserstein distance between the source domain feature set and the target feature set, adjust the parameters of the domain similarity evaluation neural network, and obtain a new domain similarity evaluation neural network corresponding to the preset number of iterations;

Step 132: Evaluate the neural network based on the source domain labeled sample set, the target domain sample set and the new domain similarity, and change the value of the convolutional neural network to reduce the sum of the Wasserstein distance and the judgment loss value of the convolutional neural network. parameters to obtain a new convolutional neural network corresponding to the preset number of iterations;

Step 133 , evaluate the convergence criterion of the neural network and the convolutional neural network based on the domain similarity, stop training, and complete the confrontation transfer learning of the convolutional neural network, or repeat step 110 .

Aiming at the maximum Wasserstein distance, the domain similarity evaluation neural network is trained to improve the discrimination sensitivity of the domain similarity evaluation neural network to the features extracted from the two sample sets. After that, the parameters of the neural network are evaluated by the fixed domain similarity, and the parameters of the convolutional neural network are adjusted. On the one hand, the judgment loss value of the source domain sample set is obtained based on the convolutional neural network. The domain feature set and the target domain feature set are input into the above trained domain similarity evaluation neural network, and the Wasserstein distance is obtained, and the fault judgment convolutional neural network is trained with the goal of minimizing the sum of the Wasserstein distance and the judgment loss value. Therefore, based on the method of first training the domain similarity evaluation neural network, then training the convolutional neural network, and then training the domain similarity evaluation neural network and then training the convolutional neural network, iterates many times until the two neural networks converge, and the volume is completed. The product neural network is trained, so that the product neural network has a higher fault judgment ability.

Preferably, the Wasserstein distance represents: the real average value of the sub-real number set in the source domain minus the real average value of the target sub-real number set; wherein, the sub-real number set in the source domain is obtained from the real number set in the source domain through the Lipschitz constraint, and the sub-real number set in the target domain is obtained from the real number set in the target domain. The set of real numbers is obtained from the target set of real numbers by Lipschitz constraints.

Based on the Lipschitz constraint, the real number set required to calculate the Wasserstein distance is obtained, and the data dimension is reduced. While ensuring the fault diagnosis ability of the convolutional neural network, the convergence speed of the domain similarity evaluation neural network and the convolutional neural network is improved.

Preferably, the Lipschitz constraint is specifically a first-order Lipschitz constraint.

The first-order Lipschitz constraint is used to calculate the Wasserstein distance, which reduces the computational complexity. While ensuring the fault diagnosis capability of the convolutional neural network, the convergence speed of the domain similarity evaluation neural network and the convolutional neural network is improved.

Preferably, the Wasserstein distance is a Wasserstein distance based on gradient penalty.

Gradient penalty is introduced to limit the neural network, prevent the parameters from being too complex, reduce the computational complexity, and improve the convergence speed of the domain similarity evaluation neural network and convolutional neural network.

Specifically, as shown in FIG. 2 , a neural network for evaluating domain similarity (referred to as DomainCritic, abbreviated as DC) is introduced in step 130 to learn the mapping whose parameter is θ _c The features of the source and target domains can be mapped to real numbers.

And the Wasserstein distance can be obtained by to calculate where the upper bound exceeds all first-order Lipschitz functions And the empirical Wasserstein distance can be approximately calculated as follows: where _lwd represents the DC loss between the source domain data ^Xs and the target domain data ^Xt (i.e. named empirical Wasserstein-1 distance). Now find the maximum value of _lwd when it is constrained by Lipschitz. It has been proved that it can be combined with gradient penalty. The parameters θ _c to train the DC, where the feature representation h is generated by the source and target domain features (i.e. h ^s and h ^t ) and a randomly selected point hr ^along the line between h ^s and h ^t pair composition. Since the Wasserstein-1 distance is differentiable and continuous almost everywhere, the DC is trained by solving the following optimization problem: where ρ is the balance coefficient.

The above-mentioned unlabeled target domain sample set is a kind of unsupervised feature learning for domain adaptation, but the features learned in the two domains may not be separated enough. The ultimate goal of this embodiment is to target the domain Developing an accurate deep transfer learning classifier requires incorporating supervised learning of labeled source domain (and target domain, if available) data into the problem of invariant feature learning. The discriminator (with two fully connected layers) is then used to further reduce the distance between the source and target domain feature distributions. In this step, the parameter θ _c of DC is the parameter trained in the above, and the parameter θ _f is updated to optimize the minimization operator.

The final objective function can be expressed in terms of the discriminator's cross-entropy loss l _c and the above empirical Wasserstein distance l _wd related to the domain difference, namely: where _θd denotes the parameters of the discriminator and λ is a hyperparameter that determines the degree of domain confusion. When optimizing the above minimization operator, the gradient penalty _lgrad (i.e. setting ρ equal to 0) is ignored since it should not affect the learning process of the representation.

Preferably, when the target domain sample set includes some marked samples, then step 120 is:

Using the convolutional neural network to be transferred and learned, the source domain feature set and source domain fault judgment set of the source domain marked sample set and the target feature set and target fault judgment set of the target domain sample set are obtained; Step 130 is: to maximize the source domain Take the Wasserstein distance between the domain feature set and the target feature set and minimize the sum of the Wasserstein distance, the judgment loss value of the source domain fault judgment set and the judgment loss value of the target fault judgment set as the goal, and optimize the parameters of the convolutional neural network, Based on the convergence criterion, step 110 is repeated, or, the adversarial transfer learning of the convolutional neural network is completed.

The deep adversarial transfer learning method of this embodiment, even when faced with transfer tasks between tasks that are related but not sufficiently similar (such as transfer learning between different sensor locations), compared to the unsupervised case of a large number of unlabeled samples, Just adding a small number of labeled target domain samples greatly improves the accuracy of transfer learning. Therefore, the present invention is not only applicable to the case where the target domain sample set is unmarked, but also applicable to the partially marked sample set, and can be applied flexibly. Moreover, when a partially labeled target domain sample set is used, the diagnostic accuracy of the convolutional neural network obtained by adversarial transfer training for industrial process faults can be further improved, and the practicability is strong.

Embodiment 2

A convolutional neural network for fault diagnosis in an industrial process is obtained by using any of the Wasserstein distance-based convolutional neural network confrontation transfer learning methods described in the first embodiment.

Based on the correlation between tasks, the convolutional neural network trained by any of the above-mentioned Wasserstein distance-based convolutional neural network adversarial transfer learning methods can be used to learn the deep adversarial transfer learning of the convolutional neural network corresponding to the source domain as suitable for The high-precision convolutional neural network in the source and target domains, to a certain extent, solves the waste of computing resources and training time caused by the need to rebuild the deep learning model from scratch in the face of new fault diagnosis tasks in the existing technology. Technical issues with sufficient sample data in the target domain.

Embodiment 3

An industrial process fault diagnosis method, based on any of the industrial process fault diagnosis convolutional neural networks as described in the second embodiment, when receiving any of the Wasserstein distance-based convolutional neural network confrontations as described in the first embodiment When migrating a new sample of the target domain in the learning method, the industrial process fault judgment result corresponding to the new sample is obtained.

The convolutional neural network trained by any of the above-mentioned Wasserstein distance-based convolutional neural network adversarial transfer learning methods is used to diagnose industrial process faults. On the premise of ensuring safety and efficiency, the fault diagnosis accuracy is higher.

In order to verify the effectiveness of the above-mentioned deep adversarial transfer learning on the fault diagnosis problem, this example introduces the benchmark bearing fault data set obtained by the Case Western Reserve University data center. Four types of bearing conditions (ie, normal, inner ring fault, outer ring fault, and roller fault) are monitored, and the digital signal is sampled at a frequency of 12 kHz. The drive-end bearing fault data is also collected at a sampling rate of 48 kHz. At the same time, each fault type operates with different fault severities (0.007-inch, 0.014-inch, and 0.021-inch fault diameters). Each type of faulty bearing was equipped with a test motor, which was run at four different motor speeds (ie 1797rpm, 1772rpm, 1750rpm and 1730rpm). The vibration signal of each experiment was recorded for fault diagnosis.

Data Preprocessing: Apply simple data preprocessing techniques to the bearing dataset: 1) Split the samples to keep each sample in the and There are 2000 measurement points in each sample; 2) the frequency-domain power spectrum of each sample is calculated using the Fast Fourier Transform (FFT); 3) the left part of the symmetrical power spectrum calculated by the FFT is used as the input of the deep transfer learning model. Therefore, each input sample has 1000 measurements.

The following is verified based on two unsupervised scenarios and one supervised scenario (see Table 1). The scenarios include:

(1) Unsupervised migration between motor speeds (US-Speed): For this case, the data obtained at the motor drive side is tested and the severity of the fault is ignored. Four classification tasks are constructed here (i.e. normal, and three fault conditions for inner ring, outer ring and roller failure), spanning 4 domains of different motor speeds: 1797rpm (US(A)), 1772rpm (US(B)) , 1750rpm (US(C)) and 1730rpm (US(D)).

(2) Unsupervised transfer between two sensor locations (US-Location): For this case, focus on domain adaptation between different sensor locations, but ignore the difference in fault severity and motor speed. Likewise, four classification (normal and three faults) tasks are constructed here for the source and target domains, where the vibration acceleration data is composed of two samples located at the drive end (US(E)) and the fan end (US(F)) of the motor housing, respectively. sensor acquisition.

(3) Supervised transfer between datasets of two sensor locations (S-Location): This scenario uses the same settings as the previous scenario US-Location, but adds a small amount of labeled data from the target domain in the source domain (approx. 0.5%), aiming to improve the classification performance.

Table 1

For comparison, other methods were also tested on the same dataset, including:

(1) Convolutional Neural Network (CNN): This model is the pre-training network described in the first embodiment, the network is trained based on the labeled source domain data, and is directly used to test the classification results on the target domain.

(2) Deep Adaptation Network (DAN): Transferable features are learned through MK-MMD in a deep neural network. The MMD metric is an integral probability metric that measures the distance between two probability distributions by mapping samples to a Regenerated Kernel Hilbert Space (RKHS).

In addition, compared with using traditional statistical features, this embodiment also evaluates the feature extraction ability of convolutional neural networks, and compares the results of traditional transfer learning methods using statistical (hand-made) features, including transfer component analysis (TCA), joint Distribution Adaptation (JDA) and Correlation Alignment (CORAL).

The specific implementation details are as follows:

Using TensorFlow as the software framework for the experiments, the models are all trained using Adam. Each method was tested five times in 5000 iterations and the best result for each test was recorded. Means and 95% confidence intervals for classification accuracy were used here for comparison. The sample sizes for the motor speed tasks (A), (B), (C) and (D) were 1026, 1145, 1390 and 1149, respectively. The sample sizes for tasks (E) and (F) at different sensor locations are 3790 and 4710, respectively. For all experiments, the sample batch size n for each training or diagnosis is fixed to 32.

Convolutional Neural Network (CNN): The Convolutional Neural Network architecture consists of two convolutional layers (Conv1-Conv2), two max-pooling layers (Pool1-Pool2) and two fully connected layers (FC1-FC2). The activation function in the output layer is Softmax, while ReLU is used in the convolutional layer. The number of neurons in FC1 and FC2 are 128 and 4, respectively. The number of filters, kernel size and step size of each layer can be seen in Table 2. Before migration, the CNN model is fine-tuned in order to achieve the best validation accuracy for all transfer scenarios.

Deep Adaptive Network (DAN): The convolutional layers (Conv1-Conv2) of the convolutional neural network are used as feature extractors. Then, in order to minimize the domain distance between the source and target domains, FC1 is used as an adapted hidden layer. The final representations of hidden layers in both domains are embedded in RKHS to reduce the MK-MMD distance. The final objective function is a combination of MK-MMD loss and classification loss.

The Wasserstein-based Deep Adversarial Transfer Learning Model (WD-DTL) of this application: Similar to deep adaptive networks, convolutional layers (Conv1-Conv2) are used to extract features. The nodes of the hidden layer in the DC network are set to 128 and 1, respectively. The number of training times C for each batch is set to 10. The learning rates of the discriminator and DC are α ₁ =10 ⁻³ and α ₂ =2×10 ⁻⁴ , respectively. The gradient loss ρ is set to 10. The balance coefficients λ used to optimize the minimization operator are 0.1 and 0.8 for motor speed migration and sensor position migration, respectively.

For the traditional transfer learning methods TCA, JDA and CORAL, the regularization term λ is selected from {0.001 0.01 0.11.0 10 100}. Classification at TCA and CORAL using SVM.

Table 2

Floor filter nuclear size step size Conv1/2 8/16 1x20 2 Pool1/2 - 1x2 2

Based on the above implementation details, the following results are obtained:

The migration task results of WD-DTL and the other two methods are shown in Table 3. For transfer tasks with unlabeled data in the target domain (i.e. US-Speed and US-Location), it can be observed that deep transfer learning models significantly outperform convolutional neural networks, and the average accuracy is transferred on motor speed and sensor location, respectively There are roughly 13.6% and 25% increases in tasks. Furthermore, the transfer accuracy of WD-DTL outperforms most results of DAN (with an average increase of 5%), except in the transfer task US(D)US(A), which is less than 1% accurate for DAN.

table 3

TCA JDA CORAL CNN DAN WD-DTL Us(A)→Us(B) 26.55 65.07(±7.55) 59.18 82.75(±6.77) 92.97(±3.88) 97.52(±3.09) Us(A)→Us(C) 46.80 51.31(±1.56) 62.14 78.65(±4.54) 85.32(±5.26) 94.43(±2.99) Us(A)→Us(D) 26.57 57.70(±8.59) 49.83 82.99(±5.89) 89.39(±4.37) 95.05(±2.12) Us(B)→Us(A) 26.63 71.19(±1.21) 53.57 84.14(±6.63) 94.43(±2.95) 96.80(±1.10) Us(B)→Us(C) 26.60 69.80(±5.67) 57.28 85.41(±9.44) 90.43(±4.62) 99.69(±0.59) Us(B)→Us(D) 26.57 88.50(±1.96) 60.53 86.09(±4.63) 87.37(±5.42) 95.51(±2.52) Us(C)→Us(A) 26.63 56.42(±2.52) 54.03 76.50(±3.76) 89.88(±1.57) 92.16(±2.61) Us(C)→Us(B) 26.66 69.18(±1.90) 76.66 82.75(±5.51) 92.93(±1.57) 96.03(±6.27) Us(C)→Us(D) 46.75 77.45(±0.83) 70.34 87.04(±6.81) 90.66(±5.24) 97.56(±3.31) Us(D)→Us(A) 46.74 61.72(±5.48) 59.78 79.23(±6.96) 90.88(±1.82) 89.82(±2.41) Us(D)→Us(B) 46.79 74.03(±0.86) 59.73 79.73(±5.49) 87.91(±2.42) 95.16(±3.67) Us(D)→Us(C) 26.60 65.24(±4.18) 63.02 80.64(±4.23) 92.94(±3.96) 99.62(±0.80) average 33.32 67.35(±3.53) 56.01 82.10(±5.89) 90.42(±3.59) 95.75(±2.62) Us(E)→Us(F) 19.05 57.35(±0.47) 47.97 39.07(±2.22) 56.89(±2.73) 64.17(±7.16) Us(F)→Us(E) 20.45 66.34(±4.47) 39.87 39.95(±3.84) 55.97(±3.17) 64.24(±3.87) average 19.75 61.85(±2.47) 43.92 39.51(±3.03) 56.43(±2.95) 64.20(±5.52) s(E)→s(F) 20.43 65.48(±0.57) 51.77 54.04(±7.67) 59.68(±4.61) 65.69(±3.74) s(F)→s(E) 19.02 59.07(±0.56) 47.88 50.47(±5.74) 58.78(±5.67) 64.15(±5.52) average 19.73 62.28(±0.57) 49.83 52.26(±6.71) 59.23(±5.14) 64.92(±4.63)

To sum up, the following observations can be drawn: 1) WD-DTL achieves the best transfer accuracy with an average score of 95.75%; 2) in the absence of domain adaptation, the convolutional neural network approach due to its excellent feature detection ability , already possessed the ability to achieve good classification performance for the motor speed transfer task; 3) the WD-DTL method proposed in the present invention shows a good ability to solve the supervision problem with a small amount of labeled data. The supervised transfer tasks S(E)→S(F) and S(F)→S(E) are performed using only 0.5% of the sample size of the unsupervised case, but achieves as good as the unsupervised case using 100% unlabeled samples performance.

Based on the above results, the experimental analysis is as follows:

Feature Visualization: Non-linear dimensionality reduction for network visualization using T-distributed Stochastic Neighbor Embedding (t-SNE). For the transfer task between motor speeds, i.e., US-Speed, the task US(C)→US(A) is randomly selected to visualize the learned feature representations at different motor speeds. Figure 3 shows the comparison results. It can be observed that the clusters in Fig. 3(c) formed by the proposed WD-DTL of the present invention are better separated than the CNN network results in Fig. 3(a) and the results of DAN domain adaptation in Fig. 3(b) . More importantly, a clear improvement in domain adaptation can be observed in Fig. 3(c), since the source and target domain features are almost in the same cluster.

For the migration tasks between different sensor locations, i.e. US-Location and S-Location, the t-SNE of the transfer task US(E)→US(F), the results are shown in Fig. 4. It can be seen that although fault types 1, 2 and 3 are difficult to be clearly divided into individual clusters, WD-DTL shows better clustering results than CNN and DAN. It must be emphasized that the above results are performed by using 100% (4710) sample size in the target domain, and even in this case the performance is not sufficient. This raises the question of how to enhance transfer learning performance when the signals in the source and target domains are correlated but not similar enough.

Effect of sample size on unsupervised and supervised accuracy: Figure 5 shows the performance of WD-DTL for tasks US(E)→US(F) and S(E)→S(F) relative to US-Location and S-Location Accuracy curve where the number of samples is increased from 10 to 2500. When the number of samples is greater than 2500, the diagnostic accuracy will saturate around the fixed value, so only results from 10 to 2500 are displayed. In Fig. 5(a), it can be observed that the accuracy of WD-DTL increases from 59.47%, and the final test accuracy is limited to around 64%. When the number of samples increases, the fault diagnosis accuracy of WD-DTL method is higher than that of DAN and CNN. This analysis shows that for this unsupervised case, increasing the number of samples can improve the accuracy of transfer learning, however, the improvement is limited (less than 5%) even when the number of samples in the target domain is 100%. To solve this problem, in Fig. 5(b), a small amount of labeled data is adopted to improve the accuracy of fault diagnosis, which corresponds to the limited labeled data in practical industrial applications. The figure shows that when the labeled sample size is greater than 20 (total number of samples 4710), the transfer learning accuracy of WD-DTL will exceed the case with 100% sample size in Fig. 5(a) (Fig. 5(a) blue area). More specifically, 80% transfer learning accuracy can be achieved using only 100 labeled samples (equivalent to 25 for each fault classification), indicating that the proposed WD-DTL is also an excellent framework for supervised transfer tasks.

Embodiment 4

A storage medium, storing instructions in the storage medium, when a computer reads the instructions, the computer is made to execute any one of the Wasserstein distance-based convolutional neural network adversarial transfer learning methods and/or as above Any of the industrial process fault diagnosis methods described in the third embodiment.

First build a convolutional neural network architecture, extract features and introduce Wasserstein distance to learn domain-invariant feature representation. Through the adversarial training process, the domain variance is significantly reduced. The related technical solutions are the same as above, and will not be repeated here.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (10)

1. a convolutional neural network confrontation transfer learning method based on Wasserstein distance, is characterized in that, comprises: Step 1. From the source domain labeled dataset and the target domain dataset, determine the source domain labeled sample set and the target domain sample set; Step 2, using the convolutional neural network to be migrated to obtain the source domain feature set and the source domain fault judgment set of the source domain marked sample set and the target feature set of the target domain sample set; Step 3, with the goal of maximizing the Wasserstein distance between the source domain feature set and the target feature set and minimizing the sum of the Wasserstein distance and the judgment loss value of the source domain fault judgment set, optimize all The parameters of the convolutional neural network described above, based on the convergence criterion, repeat step 1, or, complete the adversarial transfer learning of the convolutional neural network. 2. a kind of convolutional neural network confrontation transfer learning method based on Wasserstein distance according to claim 1, is characterized in that, described Wasserstein distance is: By using the domain similarity evaluation neural network, the source domain feature set is mapped to the source domain real number set, the target feature set is mapped to the target real number set, and the real number average value of the source domain real number set is subtracted from all real numbers. Calculate the average value of the real numbers of the target real number set. 3. a kind of convolutional neural network confrontation transfer learning method based on Wasserstein distance according to claim 2, is characterized in that, described step 3 comprises: Step 3.1, with the goal of increasing the Wasserstein distance between the source domain feature set and the target feature set, optimize the parameters of the domain similarity evaluation neural network, and obtain a new domain similarity corresponding to the preset number of iterations Evaluate neural networks; Step 3.2. Evaluate the neural network based on the source domain labeled sample set, the target domain sample set and the new domain similarity to reduce the addition of the Wasserstein distance and the judgment loss value of the convolutional neural network. and as the goal, optimize the parameters of the convolutional neural network to obtain a new convolutional neural network corresponding to the preset number of iterations; Step 3.3: Evaluate the convergence criterion of the neural network and the convolutional neural network based on the domain similarity, stop training, and complete the confrontation transfer learning of the convolutional neural network, or repeat step 1. 4. a kind of convolutional neural network confrontation transfer learning method based on Wasserstein distance according to any one of claims 3, is characterized in that, in described step 3.1, described Wasserstein distance is described source domain based on gradient penalty the difference between the real average value of the real number set and the real number average value of the target real number set; In the step 3.2, the Wasserstein distance is the difference between the real average value of the source domain real number set and the real number average value of the target real number set. 5. a kind of convolutional neural network confrontation transfer learning method based on Wasserstein distance according to claim 2, is characterized in that, described Wasserstein distance represents: The real average value of the sub-real number set in the source domain minus the real average value of the target sub-real number set; Wherein, the source domain sub-real number set is obtained from the source domain real number set through the Lipschitz constraint, and the target sub-real number set is obtained from the target real number set through the Lipschitz constraint. 6 . The Wasserstein distance-based convolutional neural network confrontation transfer learning method according to claim 5 , wherein the Lipschitz constraint is specifically a first-order Lipschitz constraint. 7 . 7. The Convolutional Neural Network confrontation transfer learning method based on Wasserstein distance according to any one of claims 1 to 6, wherein when the target domain sample set includes some labeled samples, the step 2 is: Using the convolutional neural network to be transferred and learned, obtain the source domain feature set and source domain fault judgment set of the source domain marked sample set and the target feature set and target fault judgment set of the target domain sample set; The step 3 is: To maximize the Wasserstein distance between the source domain feature set and the target feature set and minimize the Wasserstein distance, the judgment loss value of the source domain fault judgment set and the judgment loss value of the target fault judgment set The sum of the convolutional neural network is optimized as the goal, and based on the convergence criterion, step 1 is repeated, or, the adversarial transfer learning of the convolutional neural network is completed. 8 . A convolutional neural network for fault diagnosis in an industrial process, characterized in that it is obtained by training with a Wasserstein distance-based convolutional neural network confrontation transfer learning method according to any one of claims 1 to 7 . 9. An industrial process fault diagnosis method, characterized in that, based on a convolutional neural network for industrial process fault diagnosis as claimed in claim 8, when receiving a method as claimed in any one of claims 1 to 7 When the convolutional neural network based on Wasserstein distance confronts a new sample in the target domain described in the transfer learning method, the industrial process fault judgment result corresponding to the new sample is obtained. 10. A storage medium, characterized in that, the storage medium stores instructions, and when a computer reads the instructions, the computer is made to execute the above-mentioned one based on any one of claims 1 to 7. Convolutional neural network adversarial transfer learning method with Wasserstein distance and/or an industrial process fault diagnosis method as claimed in claim 9.