CN114120010B - Multi-view multi-layer industrial robot migration fault diagnosis method - Google Patents
Multi-view multi-layer industrial robot migration fault diagnosis method Download PDFInfo
- Publication number
- CN114120010B CN114120010B CN202111389808.XA CN202111389808A CN114120010B CN 114120010 B CN114120010 B CN 114120010B CN 202111389808 A CN202111389808 A CN 202111389808A CN 114120010 B CN114120010 B CN 114120010B
- Authority
- CN
- China
- Prior art keywords
- domain
- data
- sample
- layer
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013508 migration Methods 0.000 title claims abstract description 51
- 230000005012 migration Effects 0.000 title claims abstract description 51
- 238000003745 diagnosis Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000010586 diagram Methods 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims description 56
- 230000006978 adaptation Effects 0.000 claims description 40
- 238000004364 calculation method Methods 0.000 claims description 35
- 230000006870 function Effects 0.000 claims description 31
- 238000011176 pooling Methods 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 239000003638 chemical reducing agent Substances 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005299 abrasion Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000010921 in-depth analysis Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Testing And Monitoring For Control Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a multi-view multi-layer industrial robot migration fault diagnosis method, which comprises the following steps: extracting a characteristic diagram of a query set formed by target domain data of the industrial robot and a support set formed by a small sample of source domain data, wherein the characteristic diagram comprises proprietary characteristics and cross-data-domain common characteristics; splicing the feature images of the support set and the feature images of the target data domain query set to obtain multi-view features; and calculating the matching similarity of the support set sample and the target data domain query set sample by utilizing the multi-view features, classifying the target data domain sample according to the matching score, and realizing the fault diagnosis of the industrial robot parts. The method can distinguish the characteristics of the source data domain, distinguish the special characteristics of the data domain and the common characteristics of the cross-data domain, adaptively migrate the characteristics of the source data domain with good migration, inhibit the characteristics unsuitable for migration and improve the classification performance of the network model in the target domain.
Description
Technical Field
The invention belongs to the technical field of industrial robot fault diagnosis, and particularly relates to a multi-view multi-layer industrial robot migration fault diagnosis method.
Background
With the rapid development of modern industrial intellectualization, more and more industrial robots are applied to intelligent factories, society has fully started to enter the intelligent factory era, and a large number of industrial robots are in service. In the operation process of the industrial robot, various unexpected faults are inevitably generated, the economic loss is caused by light weight, and the economic loss is caused by heavy weight, so that personal injury is caused. Therefore, the industrial robot fault diagnosis has important practical application value and is an important research hotspot in the field of mechanical fault diagnosis.
However, in the service process of the industrial robot, the industrial robot is in a normal running state for a long time, the occurrence probability of the fault state is small, the fault types are large, the fault data collection is relatively difficult, the marking of the fault data is also labor-consuming, and a large amount of effective fault data is difficult to obtain and is used for training the intelligent industrial robot fault diagnosis method. In practical application, how to fully and effectively utilize all available fault data and train a more excellent intelligent fault diagnosis model is an important problem faced by industrial robot fault diagnosis algorithm research. The concept of transfer learning is adopted, and the fault characteristics or knowledge learned in one data set is transferred to another data set to be utilized, so that a fault diagnosis model with more excellent performance is learned, and the method is a solution for solving the problem.
The main migration mechanisms of the existing migration learning solutions mainly comprise three types: the first type is that a compromise feature space is learned, and samples of a source data field and a target data field are mapped to the compromise feature space for representation, so that data of the source data field can be utilized, and an intelligent fault diagnosis model of the target data field is trained; the second class realizes the migration of data by aligning the conditional probability distribution or joint probability distribution of the data of the source data domain and the target data domain; the third class measures the distribution difference between the source data domain and the target data domain based on a certain data distribution difference, and realizes the data characteristic migration by minimizing the distribution difference. The third type of method is commonly used for a deep migration solution, namely a fault diagnosis depth model with knowledge migration capability is realized by combining a migration learning method and a deep learning model. The third class of methods obtains better migration diagnosis performance under the assistance of the depth model. However, various existing deep migration fault diagnosis techniques and methods do not treat all data and all features differently, migrate, and do not perform in-depth analysis and evaluation on the data and feature portability. Due to the difference between the source data domain and the target data domain, certain sample features in the source data domain are unsuitable for migration. These features have an important impact on the sample classification performance in the field, but have no effect on sample classification in the target data domain, and even lead to a degradation in classification performance. Therefore, the consistency of all the source domain data features is treated indiscriminately, and the optimal classification performance of the target data domain caused by feature migration obviously cannot be ensured.
Disclosure of Invention
In order to solve the problems in the prior art, the invention aims to provide a multi-view multi-layer industrial robot migration fault diagnosis method which can distinguish source data domain characteristics, distinguish data domain special characteristics and cross-data domain common characteristics, adaptively migrate source data domain characteristics with good migration performance, inhibit characteristics unsuitable for migration and improve the classification performance of a network model in a target domain.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a multi-view multi-layer industrial robot migration fault diagnosis method comprises the following steps:
extracting a characteristic diagram of a query set formed by target domain data of the industrial robot and a support set formed by a small sample of source domain data, wherein the characteristic diagram comprises proprietary characteristics and cross-data-domain common characteristics;
splicing the feature images of the support set and the feature images of the target data domain query set to obtain multi-view features; and calculating the matching similarity of the support set sample and the target data domain query set sample by utilizing the multi-view features, classifying the target data domain sample according to the matching score, and realizing the fault diagnosis of the industrial robot parts.
The invention also provides a multi-view multi-layer industrial robot migration fault diagnosis system, which comprises:
and the feature extraction module is used for: a feature map for extracting a query set of an industrial robot data domain and a support set composed of small samples of source domain data, the feature map comprising proprietary features and cross-data domain common features;
and a matching calculation module: the method comprises the steps of splicing a feature map of a support set and a feature map of a query set to obtain multi-view features; and calculating the matching similarity of the support set sample and the query set sample by utilizing the multi-view features, and classifying the query set sample according to the matching score.
Preferably, the feature extraction module comprises a source domain branch and a target domain branch, and the weights of the source domain branch and the target domain branch are shared;
the source domain branch and the target domain branch have the same structure and comprise two feature extraction channels, the two feature extraction channels have the same structure, and the weight is shared.
Preferably, each feature extraction channel comprises 3 convolution layers and 3 average pooling layers, wherein each convolution layer is followed by one pooling layer, and a domain adaptation loss is added between a third convolution layer of one feature extraction channel of the source domain branch and a third convolution layer of one feature extraction channel of the target domain branch.
Preferably, the matching calculation module includes:
and the characteristic splicing module is used for: the method comprises the steps of splicing a feature map of a support set and a feature map of a query set to obtain multi-view features;
similarity calculation module: and the method is used for calculating the matching similarity of the support set sample and the query set sample by utilizing the multi-view features, and classifying the query set sample according to the matching score.
Preferably, the feature stitching module comprises a vector flattening layer and a full-connection output layer, and the full-connection output layer is connected with the field classification loss.
Preferably, the similarity calculation module adopts a single-channel module, and comprises 2 convolution layers, 2 average pooling layers, 1 vector flattening layer and 2 full connection layers, wherein each convolution layer is next to one pooling layer; and adding domain adaptation loss based on the multi-core maximum mean difference on 2 full-connection layers.
Preferably: the multi-view multi-layer industrial robot migration fault diagnosis system is trained in a mode based on small-batch rapid training, and optimal super parameters for training are selected in a super parameter grouping alternating grid selection mode.
The invention also provides an electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the multi-view multi-level industrial robot migration fault diagnosis method as described above.
The invention also provides a storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, realizes the multi-view multi-level industrial robot migration fault diagnosis method.
The invention has the following beneficial effects:
according to the multi-view multi-level industrial robot migration fault diagnosis method, the source data domain features are treated differently, the feature map of the support set formed by the query set of the industrial robot data domain and the small sample of the source domain data is extracted, the feature map comprises the special features and the cross-data domain common features, the special features and the cross-data domain common features form the multi-view features together, the source data domain features with good migration adaptability can be adaptively migrated, the characteristics unsuitable for migration are restrained, and excellent migration fault diagnosis effects can be obtained when the multi-view multi-level industrial robot migration fault diagnosis method is applied to industrial robot migration fault diagnosis.
Drawings
FIG. 1 is a diagram of a multi-view and multi-layer network model in accordance with the present invention;
FIG. 2 (a) is a specific block diagram of a feature extraction module in a multi-view and multi-layer network model according to an embodiment of the present invention;
FIG. 2 (b) is a specific block diagram of a domain classification module in a multi-view and multi-layer network model according to an embodiment of the present invention;
fig. 2 (c) is a specific structure diagram of a matching calculation module in a multi-view multi-layer network model according to an embodiment of the present invention.
Detailed Description
Specific embodiments of the present invention are described in further detail below with reference to the drawings and examples. The following examples are given for the purpose of illustration and are not intended to limit the scope of the invention.
The method can be applied to the field of industrial robot fault diagnosis, particularly RV reducer fault diagnosis, and can be used for timely detecting and diagnosing the industrial robot fault at the early stage of occurrence.
Referring to fig. 1, the multi-view multi-layer industrial robot migration fault diagnosis method of the present invention includes the steps of:
s1: collecting data of RV speed reducers under different working conditions of the same industrial robot as different data fields; specifically, an industrial robot RV reducer planetary gear is used as a data acquisition object, different working conditions are set according to different rotation directions, rotation speeds and loads, the rotation directions comprise unidirectional rotation and reciprocating swing, the rotation speeds and loads can be set within limit, vibration data in four states including tooth surface abrasion, tooth surface pitting, tooth root fracture and normal are acquired, and the data acquired under each working condition setting condition is used as a data field.
S2: constructing a multi-view fault feature extraction module, wherein the feature extraction module adopts a dual-channel mechanism to respectively extract the special features of the data domain and the common features of the cross-data domain; specifically, features to be learned by a multi-view multi-layer network model are distinguished, the features learned by a hidden network layer are divided into two types of special features of a data domain and common features of a cross-data domain, a feature extraction module with two feature channels is constructed for extracting the two types of features, two feature extraction channels are respectively constructed for respectively extracting special features of the data domain and common features of the cross-data domain, each feature extraction channel consists of 3 convolution layers and 3 average pooling layers, one pooling layer is arranged next to each convolution layer, the convolution layers adopt 1 multiplied by 3 small-size convolution kernels, and the step length of the average pooling layers is 1 multiplied by 4;
s3: constructing a multi-level classification structure, and adopting two levels of classification of data domain classification and fault type classification to realize data domain special feature extraction; specifically, the data domain-specific feature extraction channel is connected to an independent domain classifier, and both channels are connected to subsequent fault classifiers, so that features specific to each data domain can be extracted through a combination of domain classification and fault classification, wherein the domain classification is realized through a vector flattening layer and a 1×2 fully-connected output layer.
S4: constructing a matching calculation module, introducing field adaptation calculation based on multi-core maximum mean value difference into the feature extraction module and the matching calculation module, and realizing cross-data-domain common feature extraction by combining fault classification; specifically, the constructed matching calculation module is a single-channel module, the module consists of 2 convolution layers, 2 average pooling layers, 1 vector flattening layer and 2 full-connection layers, each convolution layer is next to one pooling layer, the convolution layers adopt convolution kernels with small sizes of 1×3, the step size of the average pooling layer is 1×4, the sizes of the vector flattening layer are 1× 1280,2 full-connection layers and are 1280×512 and 512×256 respectively, the field adaptation calculation based on the multi-core maximum mean difference is introduced into the feature extraction module and the matching calculation module, the field adaptation based on the multi-core maximum mean difference is added onto the 3 rd convolution layer in the feature extraction module, the field adaptation based on the multi-core maximum mean difference is added onto the 2 full-connection layers in the matching calculation module, and the field adaptation is combined with fault classification to realize the cross-data-domain common feature extraction.
S5: constructing a double-branch structure, respectively processing a source data domain input sample and a target domain input sample, wherein the source domain data is provided with label supervised input for completing classification, and the target domain data is provided with non-label unsupervised input for completing field self-adaption; the specific process is as follows: the method comprises the steps of constructing a double-branch structure for a source data domain and a target data domain, respectively processing a source data domain input sample and a target domain input sample, wherein each branch comprises two characteristic channels in a characteristic extraction module, respectively extracting data domain special characteristics and cross-data domain common characteristics, the source domain data is provided with label supervised input for completing classification, and the target domain data is provided with label-free unsupervised input for completing field self-adaption.
S6: the three losses of the field classification loss, the field adaptation loss and the fault classification loss are combined to form a loss function of the multi-view multi-layer industrial robot migration fault diagnosis network model, and the loss function is used for training the model; specifically, the overall loss function can be expressed as:
wherein ,representing failure classification loss, ++> and />For 3 field adaptation losses, +.> and />Lambda for domain classification loss from source and target data domains, respectively 1 、λ 2 、λ 3 and λ4 Is a hyper-parameter used to balance the losses. Failure classification loss->The cross entropy loss function is used to represent:
wherein nbs Representing the number of source data field samples in the training set for each batch,representing the ith query sample (i.e. sample to be classified) from the source data field,/i>The domain classification loss adopts cross entropy loss, +.>Is sample->And a corresponding fault class label. The 3 field adaptation losses are all loss functions based on the multi-core maximum mean value difference, and are expressed as:
wherein Query samples from the source data field and the target data field, respectively, < >>And the multi-core maximum mean difference estimation function is adopted. The domain classification loss uses a cross entropy loss function and computes domain classification losses for samples from the source data domain and the target data domain, respectively, expressed as:
and
wherein nbs and nbt Representing the number of samples from the source data field and the target data field respectively, and />Sample from source data field and target data field, respectively,/-> and />Respectively-> and />Corresponding domain label, when the sample is from the source data domain, the domain label is +.>When the sample comes from the target data domain, the domain label is as follows
S7: training the network by adopting a mode based on small-batch rapid training, and selecting optimal super parameters by adopting a mode of selecting super parameter grouping alternative grids for training the constructed multi-view multi-layer network; specifically, a mode based on small batch rapid training is adopted to train the network, only 5 samples are adopted to form a training set for each type of faults in each small batch training, so that sample distribution difference can be quickly adapted, training efficiency is enhanced, and an optimal super-parameter combination is selected by adopting a super-parameter grouping alternating grid selection mode. The invention adopts 5 total losses, wherein the fault classification losses do not adopt balance super parameters, and the 3 field adaptation losses adopt 3 balance super parameters lambda 1 、λ 2 and λ3 2 field classification losses share 1 superparameter lambda 4 . Lambda is set to 1 、λ 2 and λ3 Divided into a first group lambda 4 And independently forming a second group, adopting a grouping alternating mode, firstly selecting a first group of optimal super parameters, then fixing the first group of super parameters, and selecting a second group of optimal super parameters. When two groups of super parameters are selected, a grid search mode is adopted. And training the multi-view multi-layer network model by using the selected optimal super parameters and adopting an Adam method. .
The multi-view multi-level network model of the present invention is further described below, in which a support set is generated from a source data domain as a template sample, two query sets are generated from the source data domain and a target data domain, wherein the query sets generated from the source data domain are used for fault classification and domain classification, and the query sets generated from the target data domain are used for domain adaptation and domain classification between the source domain and the target domain. Inputting the support set sample into a network model, and extracting the template characteristics of the support set; and respectively inputting a pair of source domain query set samples and target domain query set samples into a source domain branch and a target domain branch of the network, and extracting the respective characteristics of the source domain branch and the target domain branch. And carrying out field classification on the sample characteristics of the source domain query set and the sample characteristics of the target domain query set, and calculating field classification loss. The multi-core maximum mean value difference is used for measuring the data distribution difference in the high-dimensional space, calculating the multi-core maximum mean value difference of the source domain query set sample characteristics and the target domain query set sample characteristics, and calculating the field adaptation loss. And performing feature splicing on the source domain query set sample features and the support set template feature graphs in the channel dimension, inputting the feature splicing to a matching calculation module, outputting matching scores of different connection features, representing the similarity between the support set templates and the query set samples, wherein the higher the similarity is, the higher the matching score is, calculating the fault category of the support set templates and the query set samples, and obtaining fault classification loss.
Examples
As shown in fig. 1, the multi-view multi-layer industrial robot migration fault diagnosis method of the embodiment includes the following steps:
step 1: the planetary gear of the RV reducer of the industrial robot is used as a data acquisition object, a vibration sensor is additionally arranged to acquire vibration signals of the planetary gear, and the vibration signals are acquired under different working conditions. Different working conditions are set according to different rotation directions, rotation speeds and loads, and data collected under each working condition form a data set. Two different loads are set respectively: 0 load and 24kg load; two different rotational speeds: 45 °/s and 90 °/s; two different turns: unidirectional rotation and reciprocating oscillation. Under the setting of the working conditions, 8 working conditions are formed, 8 vibration data sets are collected in total, and multi-view and multi-layer network model migration training is performed based on the 8 data sets. In each data set, vibration data including tooth surface abrasion, tooth surface pitting, tooth root fracture and normal four states are collected, each sample is composed of a sequence of 1000 time points, a four-classification data set is formed, data under each working condition is used as a data field, and the data can be used as a source data field or a target data field in different experimental tasks.
Step 2: the method comprises the steps of distinguishing the features learned by the multi-view multi-level network model, dividing the features learned by the hidden network layer into two types of special features of a data domain and common features of a cross-data domain, and constructing a feature extraction module with two feature channels for extracting the two types of features, respectively constructing two feature extraction channels, and respectively extracting the special features of the data domain and the common features of the cross-data domain. As shown in fig. 1, the feature extraction module is composed of two branches of a source domain branch and a target domain branch, each of which is composed of two feature channels. Each feature extraction channel has a structure shown in fig. 2 (a), and consists of 3 convolution layers and 3 average pooling layers, wherein each convolution layer is followed by one pooling layer, the convolution layers adopt 1×3 small-size convolution kernels, the step size of the average pooling layers is 1×4, and the activation functions of the convolution layers all adopt ReLU functions. The data domain specific feature channel and the cross-data domain common feature channel are identical in structure and are arranged according to the specific structure of the feature extraction module shown in fig. 2 (a).
Step 3: the data domain-specific feature extraction channel is connected with an independent domain classifier, and the specific structure of the domain classifier is shown in the domain classification module structure in fig. 2 (b). The two feature channels are both connected to subsequent fault classifiers, so that features specific to each data domain that are valid can be extracted by a combination of domain classification and fault classification. Since the domain classification aims at distinguishing two data domains, namely a source data domain and a target data domain, the domain classification is a two-classification task. The domain classification is implemented by a vector flattening layer of dimension 5120 and a 1 x 2 fully connected output layer.
Step 4: the constructed fault matching calculation module is a single channel module, and is also divided into two branches according to whether the input comes from a source data domain or a target data domain, as shown in fig. 1. The structure of the matching calculation module in each branch is shown as a specific structure of the matching calculation module in fig. 2 (c), and is composed of 2 convolution layers, 2 average pooling layers, 1 vector flattening layer and 2 full-connection layers, wherein each convolution layer is next to one pooling layer, the convolution layers adopt convolution kernels with small sizes of 1×3, the step size of the average pooling layer is 1×4, the size of the vector flattening layer is 1× 1280,2 full-connection layers, the sizes of the vector flattening layer are 1280×512 and 512×256 respectively, and domain adaptation calculation based on the multi-core maximum mean difference is introduced into the feature extraction module and the matching calculation module, as shown in fig. 1, in the feature extraction module, domain adaptation based on the multi-core maximum mean difference is added on the 3 rd convolution layer, in the matching calculation module, domain adaptation based on the multi-core maximum mean difference is added on the 2 full-connection layers, and the domain adaptation and fault classification are combined to realize the feature extraction across data domains.
Step 5: a dual-branch structure is constructed for a source data domain and a target data domain, as shown in fig. 1, a source data domain input sample and a target domain input sample are respectively processed, each branch comprises two characteristic channels in a characteristic extraction module, data domain special characteristics and data domain cross-common characteristics are respectively extracted, the source domain data is provided with label supervised input for completing classification, and the target domain data is provided with label-free unsupervised input for completing field self-adaption.
Step 6: the three kinds of losses are integrated, namely field classification loss, field adaptation loss and fault classification loss, a loss function of a multi-view multi-layer network industrial robot fault diagnosis network model is formed, and the total loss function can be expressed as:
wherein ,representing failure classification loss, ++> and />For 3 field adaptation losses, +.> and />Lambda for domain classification loss from source and target data domains, respectively 1 、λ 2 、λ 3 and λ4 Is a hyper-parameter used to balance the losses. Failure classification loss->The cross entropy loss function is used to represent:
wherein nbs Representing the number of source data field samples in the training set for each batch,representing the ith query sample (i.e. sample to be classified) from the source data field,/i>The domain classification loss adopts cross entropy loss, +.>Is sample->And a corresponding fault class label. The 3 field adaptation losses are all loss functions based on the multi-core maximum mean value difference, and are expressed as:
wherein Query samples from the source data field and the target data field, respectively, < >>And the multi-core maximum mean difference estimation function is adopted. The domain classification loss uses a cross entropy loss function and computes domain classification losses for samples from the source data domain and the target data domain, respectively, expressed as:
and
wherein nbs and nbt Representing the number of samples from the source data field and the target data field respectively, and />Sample from source data field and target data field, respectively,/-> and />Respectively-> and />Corresponding domain label, when the sample is from the source data domain, the domain label is +.>When the sample comes from the target data domain, the domain label is as follows
Step 7: training the network by adopting a mode based on small batch rapid training, forming a training set by adopting only 5 samples for each type of faults in each small batch training, thereby being capable of rapidly adapting to sample distribution difference, enhancing training efficiency, selecting optimal super-parameter combination by adopting a mode of selecting super-parameter groups by using alternative grids, and specifically comprising the following steps of:
the invention adopts 5 total losses, wherein the fault classification losses do not adopt balance super parameters, and the 3 field adaptation losses adopt 3 balance super parameters lambda 1 、λ 2 and λ3 2 field classification losses share 1 superparameter lambda 4 . Lambda is set to 1 、λ 2 and λ3 Divided into a first group lambda 4 And independently forming a second group, adopting a grouping alternative optimizing mode, applying grid search, and selecting two groups of optimal super parameters. First a first set of optimal superparameters is selected, each parameter being selected in the range 0.1 to 5, each step increased by 0.05. And training a network model by applying each group of parameter settings, and generating a special verification set to test the performance of the model. After traversing all the combinations of the first three parameters, selecting a group of super parameters capable of obtaining the optimal model performance as optimal parameters, fixing the first group of super parameters, and selecting a second group of optimal super parameters. Lambda in the second set of parameters 4 A selection is made from the set 0.001,0.01,0.1,1,10,100 and after traversal through the set of parameters, the hyper-parameters that can achieve optimal model performance are selected as optimal parameter values.
The optimal super parameters selected by the optimal application are obtained, and the Adam method is adopted to train the multi-view multi-layer network model, and the specific training steps are as follows: and initializing a network model by adopting a random initialization mode, and training the model by using a small sample set. A support set is generated from the source data domain, two query sets are generated from the source data domain and the target data domain, wherein the query sets generated from the source data domain are used for fault classification and domain classification, and the query sets generated from the target data domain are used for domain adaptation and domain classification between the source domain and the target domain. Inputting the support set sample into a network model, and extracting the template characteristics of the support set; a pair of source domain query set samples and target domain query set samples are input into a network model, and their respective features are extracted. And carrying out field classification on the sample characteristics of the source domain query set and the sample characteristics of the target domain query set, and calculating field classification loss. And calculating the multi-core maximum mean difference of the source domain query set sample characteristics and the target domain query set sample characteristics, and calculating the field adaptation loss. And carrying out matching calculation on the sample characteristics of the source domain query set and the template characteristics of the support set, and calculating the fault category of the sample characteristics to obtain the fault classification loss. And (3) minimizing the sum of all obtained losses by adopting an Adam method, and performing error back propagation to complete multi-view multi-layer network model training.
According to the scheme, the multi-view characteristics and multi-level classification are combined, the characteristics of migration of the non-beneficial target domain classification and the characteristics of migration of the beneficial target domain classification are fully mined from the mobility of the source data domain characteristics, and a novel and effective multi-view multi-level network adapting to the mobility of the characteristics is constructed. By combining the classification of the two levels of field classification and fault classification, the characteristics with resolution only for the classification of the source data field sample can be extracted, so that the special characteristics of the data field can be obtained; by combining fault classification and field adaptation based on the multi-core maximum average difference, features with resolution on both source data domain samples and target data domain samples can be extracted, so that cross-data-domain common features are obtained. The special characteristics of the data fields and the common characteristics of the cross-data fields form multi-view characteristics, so that the multi-view characteristics are applied to the migration fault diagnosis of the industrial robot, excellent migration fault diagnosis effect is obtained, and the migration fault diagnosis accuracy rate in the target data fields can reach more than 99%.
Claims (10)
1. The multi-view multi-layer industrial robot migration fault diagnosis method is characterized by comprising the following steps of:
extracting a characteristic diagram of a query set formed by target domain data of the industrial robot and a support set formed by a small sample of source domain data, wherein the characteristic diagram comprises proprietary characteristics and cross-data-domain common characteristics;
splicing the feature images of the support set and the feature images of the target data domain query set to obtain multi-view features; calculating the matching similarity of the support set sample and the target data domain query set sample by utilizing the multi-view features, classifying the target data domain sample according to the matching score, and realizing the fault diagnosis of the industrial robot parts;
the process of extracting the feature map of the support set formed by the small sample of the target domain data of the industrial robot is as follows:
distinguishing the characteristics learned by the multi-view multi-layer network model, and dividing the characteristics learned by the network hidden layer into two types of data domain proprietary characteristics and cross-data domain common characteristics;
constructing a feature extraction module with two feature channels, respectively constructing the two feature extraction channels, and respectively extracting special features of a data domain and common features across the data domain; the feature extraction module is composed of two branches of a source domain branch and a target domain branch, and each branch is composed of two feature channels; each feature extraction channel structure consists of 3 convolution layers and 3 average pooling layers, wherein each convolution layer is followed by one pooling layer, and the convolution layers adoptIs a step size of +.>The activation functions of the convolution layers all adopt ReLU functions; the special characteristic channel of the data domain is consistent with the cross-data-domain common characteristic channel;
the special feature extraction channel of the data domain is connected with an independent domain classifier, two feature channels in the domain classifier are connected with a subsequent fault classifier, the domain classification aims at distinguishing two data domains of a source data domain and a target data domain, and the domain classification is one-to-twoClassification tasks, domain classification through a vector flattening layer of dimension 5120 and aIs realized by the full connection output layer;
calculating the matching similarity of the support set sample and the target data domain query set sample by utilizing the multi-view features, and classifying the target data domain sample according to the matching score as follows:
the constructed fault matching calculation module is a single-channel module, the fault matching calculation module is divided into two branches according to whether input comes from a source data domain or a target data domain, the matching calculation module structure in each branch consists of 2 convolution layers, 2 average pooling layers, 1 vector leveling layer and 2 full connection layers, each convolution layer is next to one pooling layer, and the convolution layers adopt the following steps ofIs a step size of +.>Vector flattening layer size is +.>The dimensions of the 2 fully connected layers are +.> and 512/>The method comprises the steps that field adaptation calculation based on the multi-core maximum mean value difference is introduced into a feature extraction module and a matching calculation module, in the feature extraction module, the field adaptation based on the multi-core maximum mean value difference is added to a 3 rd convolution layer, in the matching calculation module, the field adaptation based on the multi-core maximum mean value difference is added to 2 full connection layers, and the field adaptation is combined with fault classification to achieve cross-data-domain common feature extraction;
constructing a double-branch structure facing to a source data domain and a target data domain, respectively processing a source data domain input sample and a target domain input sample, wherein each branch comprises two characteristic channels in a characteristic extraction module, respectively extracting data domain special characteristics and cross-data domain common characteristics, the source domain data is provided with label supervised input for finishing classification, and the target domain data is provided with label-free unsupervised input for finishing field self-adaption;
the three kinds of losses are integrated, namely field classification loss, field adaptation loss and fault classification loss, a loss function of a multi-view multi-layer network industrial robot fault diagnosis network model is formed, and the total loss function can be expressed as:
，
wherein ,representing failure classification loss, ++>、/> and />For the 3 fields of adaptation loss, and />For domain classification loss from source data domain and target data domain, respectively, < >>、/>、/> and />Is a hyper-parameter for balancing losses; failure classification loss->The cross entropy loss function is used to represent:
wherein Representing the number of source data field samples in each batch of training set, +.>Representing +.>Sample of queries->The domain classification loss adopts cross entropy loss, +.>Is sample->A corresponding fault class label; the 3 field adaptation losses are all loss functions based on the multi-core maximum mean value difference, and are expressed as:
wherein ，/>Query samples from the source data domain and the target data domain, respectively, < >>A multi-core maximum mean difference estimation function; the domain classification loss uses a cross entropy loss function and computes domain classification losses for samples from the source data domain and the target data domain, respectively, expressed as:
and
wherein and />Representing the number of samples from the source data field and the target data field, respectively,/-> and />Sample from source data field and target data field, respectively,/-> and />Respectively-> and />Corresponding domain label, when the sample is from the source data domain, the domain label is +.>When the sample comes from the target data domain, its domain label is +.>。
2. A multi-view and multi-layer industrial robot migration fault diagnosis system, comprising:
and the feature extraction module is used for: a feature map for extracting a query set of an industrial robot data domain and a support set composed of small samples of source domain data, the feature map comprising proprietary features and cross-data domain common features;
and a matching calculation module: the method comprises the steps of splicing a feature map of a support set and a feature map of a query set to obtain multi-view features; calculating matching similarity of the support set sample and the query set sample by utilizing the multi-view features, and classifying the query set sample according to the matching score;
the process of extracting the feature map of the support set formed by the small sample of the target domain data of the industrial robot is as follows:
distinguishing the characteristics learned by the multi-view multi-layer network model, and dividing the characteristics learned by the network hidden layer into two types of data domain proprietary characteristics and cross-data domain common characteristics;
constructing a feature extraction module with two feature channels, respectively constructing the two feature extraction channels, and respectively extracting special features of a data domain and common features across the data domain; the feature extraction module is composed of two branches of a source domain branch and a target domain branch, and each branch is composed of two feature channels; each feature extraction channel structure consists of 3 convolution layers and 3 average pooling layers, wherein each convolution layer is followed by one pooling layer, and the convolution layers adoptIs a step size of +.>The activation functions of the convolution layers all adopt ReLU functions; the special characteristic channel of the data domain is consistent with the cross-data-domain common characteristic channel;
the special feature extraction channel of the data domain is connected with an independent domain classifier, two feature channels in the domain classifier are connected with a subsequent fault classifier, the domain classification aims at distinguishing two data domains of a source data domain and a target data domain, the domain classification is a two-classification task, and the domain classification passes through a vector flattening layer of dimension 5120 and a vector flattening layer of dimension 5120Is realized by the full connection output layer;
calculating the matching similarity of the support set sample and the target data domain query set sample by utilizing the multi-view features, and classifying the target data domain sample according to the matching score as follows:
the constructed fault matching calculation module is a single-channel module and is divided into two branches according to whether input comes from a source data domain or a target data domain, and each branch is divided into two branchesThe matching calculation module structure in each branch consists of 2 convolution layers, 2 average pooling layers, 1 vector flattening layer and 2 full connection layers, wherein each convolution layer is next to one pooling layer, and the convolution layers adoptIs a step size of +.>Vector flattening layer size is +.>The dimensions of the 2 fully connected layers are +.> and 512/>The method comprises the steps that field adaptation calculation based on the multi-core maximum mean value difference is introduced into a feature extraction module and a matching calculation module, in the feature extraction module, the field adaptation based on the multi-core maximum mean value difference is added to a 3 rd convolution layer, in the matching calculation module, the field adaptation based on the multi-core maximum mean value difference is added to 2 full connection layers, and the field adaptation is combined with fault classification to achieve cross-data-domain common feature extraction;
constructing a double-branch structure facing to a source data domain and a target data domain, respectively processing a source data domain input sample and a target domain input sample, wherein each branch comprises two characteristic channels in a characteristic extraction module, respectively extracting data domain special characteristics and cross-data domain common characteristics, the source domain data is provided with label supervised input for finishing classification, and the target domain data is provided with label-free unsupervised input for finishing field self-adaption;
the three kinds of losses are integrated, namely field classification loss, field adaptation loss and fault classification loss, a loss function of a multi-view multi-layer network industrial robot fault diagnosis network model is formed, and the total loss function can be expressed as:
，
wherein ,representing failure classification loss, ++>、/> and />For the 3 fields of adaptation loss, and />For domain classification loss from source data domain and target data domain, respectively, < >>、/>、/> and />Is a hyper-parameter for balancing losses; failure classification loss->The cross entropy loss function is used to represent:
wherein Representing the number of source data field samples in each batch of training set, +.>Representing +.>Sample of queries->The domain classification loss adopts cross entropy loss, +.>Is sample->A corresponding fault class label; the 3 field adaptation losses are all loss functions based on the multi-core maximum mean value difference, and are expressed as:
wherein ，/>Query samples from the source data domain and the target data domain, respectively, < >>Is the most multi-coreA large-mean difference estimation function; the domain classification loss uses a cross entropy loss function and computes domain classification losses for samples from the source data domain and the target data domain, respectively, expressed as:
and
wherein and />Representing the number of samples from the source data field and the target data field, respectively,/-> and />Sample from source data field and target data field, respectively,/-> and />Respectively-> and />Corresponding domain label, when the sample is from the source data domain, the domain label is +.>When the sample comes from the target data domain, its domain label is +.>。
3. The multi-view multi-level industrial robot migration fault diagnosis system according to claim 2, wherein the feature extraction module comprises a source domain branch and a target domain branch, and the source domain branch and the target domain branch share weights;
the source domain branch and the target domain branch have the same structure and comprise two feature extraction channels, the two feature extraction channels have the same structure, and the weight is shared.
4. A multi-view multi-level industrial robot migration fault diagnosis system according to claim 3, wherein each feature extraction channel comprises 3 convolution layers and 3 average pooling layers, wherein each convolution layer is followed by a pooling layer, and a domain adaptation loss is added between the third convolution layer of one feature extraction channel of the source domain branch and the third convolution layer of one feature extraction channel of the target domain branch.
5. The multi-view multi-level industrial robot migration fault diagnosis system according to claim 2, wherein the matching calculation module comprises:
and the characteristic splicing module is used for: the method comprises the steps of splicing a feature map of a support set sample and a feature map of a query set sample to obtain multi-view features;
similarity calculation module: and the method is used for calculating the matching similarity of the support set sample and the query set sample by utilizing the multi-view features, and classifying the samples of the query set according to the matching score.
6. The multi-view multi-layer industrial robot migration fault diagnosis system of claim 5, wherein the feature stitching module comprises a vector flattening layer and a fully-connected output layer, wherein the fully-connected output layer is connected with field classification loss.
7. The multi-view multi-layer industrial robot migration fault diagnosis system according to claim 5, wherein the similarity calculation module adopts a single channel module, and comprises 2 convolution layers, 2 average pooling layers, 1 vector flattening layer and 2 full connection layers, wherein each convolution layer is next to one pooling layer; and adding domain adaptation loss based on the multi-core maximum mean difference on 2 full-connection layers.
8. The multi-view multi-level industrial robot migration fault diagnosis system according to any one of claims 2-7, wherein:
the multi-view multi-layer industrial robot migration fault diagnosis system is trained in a mode based on small-batch rapid training, and optimal super parameters for training are selected in a super parameter grouping alternating grid selection mode.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the multi-view multi-level industrial robot migration fault diagnosis method of claim 1.
10. A storage medium having a computer program stored thereon, wherein the computer program when executed by a processor implements the multi-view multi-level industrial robot migration fault diagnosis method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111389808.XA CN114120010B (en) | 2021-11-22 | 2021-11-22 | Multi-view multi-layer industrial robot migration fault diagnosis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111389808.XA CN114120010B (en) | 2021-11-22 | 2021-11-22 | Multi-view multi-layer industrial robot migration fault diagnosis method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114120010A CN114120010A (en) | 2022-03-01 |
| CN114120010B true CN114120010B (en) | 2023-11-03 |
Family
ID=80439700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111389808.XA Active CN114120010B (en) | 2021-11-22 | 2021-11-22 | Multi-view multi-layer industrial robot migration fault diagnosis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120010B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709448A (en) * | 2020-05-20 | 2020-09-25 | 西安交通大学 | Mechanical fault diagnosis method based on migration relation network |
| CN111898095A (en) * | 2020-07-10 | 2020-11-06 | 佛山科学技术学院 | Deep migration learning intelligent fault diagnosis method and device, storage medium and equipment |
| CN112434602A (en) * | 2020-11-23 | 2021-03-02 | 西安交通大学 | Fault diagnosis method based on migratable common feature space mining |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109710636B (en) * | 2018-11-13 | 2022-10-21 | 广东工业大学 | Unsupervised industrial system anomaly detection method based on deep transfer learning |
| CN110780146B (en) * | 2019-12-10 | 2021-04-27 | 武汉大学 | Transformer fault identification and positioning diagnosis method based on multi-stage transfer learning |
-
2021
- 2021-11-22 CN CN202111389808.XA patent/CN114120010B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709448A (en) * | 2020-05-20 | 2020-09-25 | 西安交通大学 | Mechanical fault diagnosis method based on migration relation network |
| CN111898095A (en) * | 2020-07-10 | 2020-11-06 | 佛山科学技术学院 | Deep migration learning intelligent fault diagnosis method and device, storage medium and equipment |
| CN112434602A (en) * | 2020-11-23 | 2021-03-02 | 西安交通大学 | Fault diagnosis method based on migratable common feature space mining |
Non-Patent Citations (2)
| Title |
|---|
| 金余丰 ; 刘晓锋 ; 姚美常 ; 黄凤良 ; .基于域对抗迁移的变工况滚动轴承故障诊断模型.自动化仪表.2019,(第12期),全文. * |
| 陈丹敏 ; 周福娜 ; 王清贤 ; .基于多源异构信息迁移学习的融合故障诊断方法.信息工程大学学报.2020,(第02期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114120010A (en) | 2022-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiao et al. | Deep coupled dense convolutional network with complementary data for intelligent fault diagnosis | |
| Hu et al. | Data-driven fault diagnosis method based on compressed sensing and improved multiscale network | |
| Sun et al. | Convolutional discriminative feature learning for induction motor fault diagnosis | |
| CN110751207B (en) | Fault diagnosis method for anti-migration learning based on deep convolution domain | |
| Li et al. | A deep transfer nonnegativity-constraint sparse autoencoder for rolling bearing fault diagnosis with few labeled data | |
| CN110657984B (en) | Planetary gearbox fault diagnosis method based on reinforced capsule network | |
| Ye et al. | AKSNet: A novel convolutional neural network with adaptive kernel width and sparse regularization for machinery fault diagnosis | |
| CN108256636A (en) | A kind of convolutional neural networks algorithm design implementation method based on Heterogeneous Computing | |
| Tu et al. | Efficient monocular depth estimation for edge devices in internet of things | |
| CN114972213A (en) | Two-stage mainboard image defect detection and positioning method based on machine vision | |
| Wang et al. | Intelligent fault diagnosis for planetary gearbox using transferable deep q network under variable conditions with small training data | |
| CN113065581B (en) | Vibration fault migration diagnosis method for reactance domain self-adaptive network based on parameter sharing | |
| CN112132102B (en) | Intelligent fault diagnosis method combining deep neural network with artificial bee colony optimization | |
| Sun et al. | An adaptive anti-noise gear fault diagnosis method based on attention residual prototypical network under limited samples | |
| CN113642486A (en) | Unmanned aerial vehicle distribution network inspection method with airborne front-end identification model | |
| CN114491828A (en) | Compound fault diagnosis method based on multi-gating hybrid expert multi-task learning | |
| Shahid et al. | Performance comparison of 1d and 2d convolutional neural networks for real-time classification of time series sensor data | |
| Yang et al. | Few-shot learning for rolling bearing fault diagnosis via siamese two-dimensional convolutional neural network | |
| Wang et al. | A fault diagnosis method using improved prototypical network and weighting similarity-Manhattan distance with insufficient noisy data | |
| Liu et al. | Multi-Scale Fusion Attention Convolutional Neural Network for Fault Diagnosis of Aero-Engine Rolling Bearing | |
| Dong et al. | An intelligent bearing fault diagnosis framework: one-dimensional improved self-attention-enhanced CNN and empirical wavelet transform | |
| CN114120010B (en) | Multi-view multi-layer industrial robot migration fault diagnosis method | |
| Fan et al. | A novel metric-based model with the ability of zero-shot learning for intelligent fault diagnosis | |
| Bi et al. | Critical direction projection networks for few-shot learning | |
| CN116244640A (en) | Unbalanced bearing fault diagnosis method and system based on federal learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |