CN117909854A - Zero sample composite fault diagnosis method based on multi-mode contrast embedding - Google Patents

Abstract

The invention belongs to the technical field of intelligent automobile fault diagnosis and discloses a zero sample compound fault diagnosis method based on multi-mode contrast embedding. The method comprehensively considers the roles of the high-dimensional features and the low-dimensional features in fault diagnosis, solves the problems that the composite fault features are complex to stack and the fault semantic information is insufficient in feature description, and can obtain better generalization performance without professional knowledge. The invention introduces a contrast learning idea in the field of computer vision in the composite fault diagnosis, provides a zero sample composite fault diagnosis method based on contrast learning, and solves the problems of difficult diagnosis of composite faults, difficult distinction of signal samples and poor prediction accuracy balance. Based on generalized zero sample learning, a composite fault increment learning framework based on an iterative idea is provided, and the problems that a composite fault sample is difficult to reproduce and has high complexity are solved. The invention can avoid the known fault sample from being wrongly diagnosed as the unknown composite fault on the premise of ensuring the accuracy rate.

Description

Zero sample composite fault diagnosis method based on multi-mode contrast embedding

Technical Field

The invention relates to the technical field of intelligent automobile fault diagnosis, in particular to a zero sample compound fault diagnosis method based on multi-mode contrast embedding.

Background

Automobile fault diagnosis is a critical area in the modern automotive industry that involves the use of various techniques and methods to detect and determine possible faults in an automobile. In recent years, with the innovation and improvement of artificial intelligence and machine learning technologies, automobile fault diagnosis has begun to utilize these technologies to improve the accuracy and efficiency of diagnosis. By collecting and analyzing a large amount of vehicle operating data, an algorithmic model may be trained to predict and identify specific failure modes. Furthermore, with active learning and data enhancement, the model can be continually improved to handle the newly emerging fault types.

Composite faults generally refer to multiple faults occurring simultaneously in a system or device, and are characterized by multiple processes, burstiness, and multiple fault couplings. Since the occurrence of faults is not manually controlled in the task of fault diagnosis, which leads to the situation of lack of priori knowledge and unbalance under the actual working condition, the newly-appearing unknown faults often face the challenge of lack of fault samples, and the problem is particularly obvious in the aspect of composite fault diagnosis. In addition, the complex faults are not considered enough in the existing research on the complex faults, and in the test set which simultaneously comprises the double-fault coupling complex faults and the three-complex fault coupling complex, the diagnosis precision is obviously reduced compared with the test set which is only used for the double-fault coupling complex faults. While diagnosis of known faults can reach 100% generally, improving comprehensive model accuracy depending on diagnosis accuracy of known faults has not been able to meet requirements ."S. Xing, Y. Lei, S. Wang, N. Lu and N. Li, "A label description space embedded model for zero-shot intelligent diagnosis of mechanical compound faults", Mech. Syst. Signal Process., vol. 162, Jan. 2022." a tag description space embedded model for intelligent diagnosis of mechanical composite fault zero samples has been proposed, in which a tag description space is established to represent semantic relationships between different fault modes. However, the independent fault semantic information is insufficient in feature description, the defect that the composite fault diagnosis of the bearing based on zero sample learning, which is proposed in the high-dimensional depth feature ."J. Xu, L. Zhou, W. Zhao, Y. Fan and X. Ding, "Zero-shot learning compound fault diagnosis of bearings," 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, 2021, pp. 1-8, doi: 10.1109/IJCNN52387.2021.9534279.", effectively improves the defect of excessive required data quantity, a novel fault semantic vector definition method is created to express the semantics of single faults and composite faults, the unknown composite faults are identified by combining vibration data of the single faults, but the fault semantics are set to be a binary threshold, so that part of detail features are ignored, and the fault semantic vectors are relatively similar in feature space, so that the preference of the known fault types is easily generated. Therefore, considering the current mainstream classical fault diagnosis algorithm, the requirements of both precision and balance cannot be met.

Disclosure of Invention

Aiming at the problems, the invention provides a zero sample compound fault diagnosis method based on multi-mode contrast embedding. In order to solve the problems of complex stacking of composite fault characteristics and insufficient description of fault semantic information on the characteristics, the invention provides a multi-mode joint embedding method based on SDP transformation, which mainly comprises an SDP image transformation module, a fault semantic construction module and a self-encoder. When the vibration signal is converted from low-dimensional information to high-dimensional information, a conversion mode based on SDP dense dot diagram conversion is adopted, and compared with traditional methods such as wavelet conversion and the like, the SDP conversion has the defects of requiring professional knowledge and poor generalization performance, and can better acquire global characteristics of amplitude and phase visually; meanwhile, in the semantic construction stage, multiple sections of processing are creatively selected according to the vibration amplitude threshold value, and compared with a fuzzy binary threshold value semantic definition method, the low-dimensional detail characteristic can be better embodied; the high-dimensional features from the image and the low-dimensional features from the semantics are embedded jointly through the self-encoder, and the image features are supplemented by the semantic detail features, so that the model convergence speed and the diagnosis accuracy can be considered.

The technical scheme of the invention is as follows: a zero sample compound fault diagnosis method based on multi-mode contrast embedding is divided into a training stage and a testing stage;

the training phase comprises the following steps:

Step 1: bearing vibration signals are collected and are respectively input to an SDP conversion module and a semantic construction module after being preprocessed; the SDP image data output by the SDP conversion module is input into a fault diagnosis model based on contrast embedding; the semantic construction module performs threshold segmentation on the bearing vibration signal to obtain a fault initial semantic vector;

The fault diagnosis model based on the contrast embedding comprises a feature extraction module, a contrast learning module and a self-encoder; the training feature extraction module is used for extracting visual features from the SDP image data;

Step 2: training a self-encoder according to the visual features, carrying out feature reconstruction on the fault initial semantic vector through the self-encoder, and outputting the reconstructed fault semantic features; the self-encoder includes an encoder and a decoder;

In particular to the visual characteristics Input into encoder, visual features/>Dimension reduction is converted into an output vector; the decoder reconstructs the output vector as a visual feature vector/>; Through visual characteristics/>, in training processAnd visual feature vector/>MSE loss and cross entropy loss/>Reconstruction loss function/>, for self-encoderOptimizing;

during the test phase, the initial semantic vector matrix of the single type of fault sample And an initial semantic vector matrix/>, of the composite fault samplesInputting the reconstructed semantic feature vector into an optimized self-encoder, and outputting the reconstructed semantic feature vector/>Reconstructing semantic feature vectorsInputting the fault samples into a comparison learning module to classify the fault samples; training a contrast learning module, inputting the classified unknown compound faults into an iteration-based diagnosis increment frame, and diagnosing again on a generalized zero sample test set;

After the classification of the unknown composite fault class is completed, inputting the classified samples into a training set in a training stage, randomly selecting the samples, adding the samples into a comparison learning pool, and training the samples as known samples.

The preprocessing specifically adopts a sliding window-based slicing method to divide fault signal samples on bearing vibration signals; intercepting bearing vibration signals with fixed length from a starting position as a fault sample source, sampling fault samples from each fault data file with different types in an open source data set, dividing the fault samples into different data sets according to different load working conditions, and using the data sets of single type fault samples in a training stage and a testing stage; the same mode is adopted to divide the composite fault signal file, and a composite fault signal sample is obtained as a composite fault data set and used in a test stage.

The specific conversion process of the SDP image conversion module is as follows:

1) For single type faults, an initial semantic vector matrix of single type fault samples Represented as，/>Representing initial semantic vector at/>Taking the value of dimension; selecting/> -of original bearing vibration signalData segments and set threshold/>So that/>， />The value of (2) is 3, and the sampling points are respectively 4 values according to the amplitude;

2) Normalizing the original bearing vibration signal according to the normalized value Decision/>Of (2), whereinFor the threshold number of copies, represent partition/>The number of the values of (a); obtaining/>, single type of faultDimension initial semantic vector/>; The specific conversion method is shown as the following formula:

3) For compound faults, an initial semantic vector matrix of compound fault samples is used for Expressed as/>，The number of initial semantic vectors for the composite fault; obtaining initial semantic vectors of the composite faults by performing element-by-element maximization operation on the initial semantic vectors of the single faults forming the composite faults, wherein/>Representing an element-by-element maximization operation.

Constructing a comparison sample cell in the iteration-based diagnostic incremental frameworkExtracting a fixed quantity of visual feature vectors from each known fault type, adding the visual feature vectors into a comparison sample pool, normalizing the visual feature vectors of the known fault types, forming an example subspace of a class plane on an hypersphere feature space, and calculating the probability that one query feature vector belongs to a certain type through projection distance and cosine similarity; test stage query feature vector is/>; Under the learning scenario of a generalized zero sample test set, the first unknown fault sample does not have positive sample pairs in a comparison sample pool, and the construction process of the comparison loss function is as follows:

1) Generating a mask matrix, wherein the sample pair entries with the same label are 1, and the sample pair entries with different labels are 0, so as to distinguish positive and negative sample pairs, as shown in the following formula:

Wherein the method comprises the steps of 、/>Labels representing two comparative samples in a comparative sample cell;

2) The basis for judging whether the query feature vector belongs to a certain category is a projection distance, and the projection distance calculating method is as follows:

Representing query feature vectors/> The distance of the projection on the instance subspace, n, represents the number of samples that make up the real instance space;

Projection distance Expressed as normal vectors orthogonal to all vectors in the example subspace, by calculating the dot product between eigenvectors and by the temperature parameter/>Scaling, setting a temperature parameter/>, in a fault diagnosis task0.7, The formula is as follows:

Wherein the method comprises the steps of Representing query feature vectors,/>A set of total vectors representing an instance subspace;

Then subtracting the maximum value of each row from the dot product to obtain ；

3) UsingLog probability mean/>, for each positive sample pairThe log probability of samples without positive sample pairs is not calculated, as follows:

4) Final contrast loss function The mean value of the logarithmic probability negative values for N samples is given by:

5) The contrast learning module overall loss is expressed as:

For cross entropy loss to account for total loss weight,/> Is a cross entropy loss.

The characteristic extraction module is used for acquiring visual characteristics in the SDP image through a multi-scale convolutional neural network based on channel weights; the SDP image output by the SDP image conversion module and the corresponding label form a setAs input to the feature extraction module; /(I)，/>For/>Number of SDP image samples,/>For/>Corresponding labels of the SDP images;

Extracting an equal-length-width-size square region containing all images from the center of the SDP image as a characteristic block through image transformation operation; inputting the characteristic blocks into a multi-scale convolutional neural network based on channel weight for characteristic extraction, and performing multi-scale convolutional layer, batch processing layer, Linear rectifying layer, max pooling layer and/>Depth visual features/>, obtained after manipulation; The depth visual features are weighted by the attention weights of the channels after being compressed;

Wherein the method comprises the steps of For compressed depth visual features,/>Table full connectivity layer operation,/>Is the weight of the full connection layer,/>And/>Weights of two full connection layers,/>, respectivelyRepresentation/>The activation function outputs the weighting factor/>, for each channelIs a collection of (3); /(I)Representation/>Activating a function to per channel/>, of a depth visual featureMultiplying the corresponding weight coefficient/>Weighting the feature map is realized;

Weighted feature map The visual characteristics/>, are obtained by sequentially inputting the visual characteristics/>, the visual characteristics and the visual characteristics to the full-connection layer and the MLP classifier of 1 x 1024And an image classification result; in the training stage, an equal number of fault samples are extracted from the visual characteristics to construct a visual contrast learning poolVisual characteristics/>, of each known fault classAfter normalization, a plane-like example subspace is formed over an hyperspherical feature space, using visual features/>Calculating the probability that a query feature vector belongs to a certain category according to the projection distance and cosine similarity of the example subspace; the training phase inquiry feature vector is visual feature/>And calculating a contrast loss optimization feature extraction model, wherein the calculation mode of the contrast loss function is the same as that of a contrast sample pool.

The test stage, the output reconstructed semantic feature vectorInputting the fault samples into a comparison learning module to classify the fault samples;

Reconstructing semantic feature vectors using training sets Constructing a semantic comparison sample pool/>The reconstructed semantic feature vector/>, obtained in the test setInputting the vector as a query feature vector into an hypersphere feature space; by comparing sample cells/>The reconstructed semantic feature vector in the model (C) constructs an example space on the hypersphere feature space, and uses the semantic feature vectorAnd calculating the probability that one query feature vector belongs to a certain category according to the projection distance and cosine similarity of the example subspace, and performing first diagnosis classification diagnosis to obtain a classification result.

The iteration-based diagnosis increment framework modifies the comparison sample pool after obtaining the primary classification diagnosis resultTemperature parameter taken for diagnosis of unknown composite fault samples/>The samples belong to a specific class probability threshold; The method comprises the following specific steps:

adding the reconstructed semantic feature vector of the detected unknown composite fault sample into a comparison sample pool As a forward sample of the unknown class; when the unknown composite fault sample of the primary diagnosis is less than a fixed quantity, linear transformation data enhancement is carried out on the unknown composite fault sample until the quantity of the forward samples is sufficient; utilize new comparative sample cell/>Constructing example subspaces including unknown composite samples, calculating cosine similarity in each example subspace, and performing iterative detection on unknown fault samples which are not detected in the test set; and continuing to iterate the comparison sample pool until the diagnosis precision and recall rate of the fault reach the optimal value, and stopping iterating.

The invention has the beneficial effects that: the method aims at solving the problems that in a real road condition scene, the composite fault data acquisition difficulty is high, the labeling is complex, the characteristics are stacked, the fault characteristics are inconsistent with semantic dimensions and the like. The method provided by the invention can extract the combination of the high-dimensional visual characteristics and the low-dimensional semantic characteristics, improves the robustness of the composite fault diagnosis, effectively relieves the attenuation of the diagnosis precision of the composite fault under the generalized zero sample scene, and improves the balance of the multi-fault coupling composite fault diagnosis.

Drawings

FIG. 1 is a general flow chart of a zero sample composite fault diagnosis method based on multi-modal contrast embedding;

FIG. 2 is a schematic illustration of a sliding window based slicing method;

FIG. 3 is a diagram of a contrast learning module and feature space architecture;

FIG. 4 is a diagnostically incremental framework block diagram of a testing phase;

FIG. 5 is a diagnostically incremental framework structure diagram of a training phase;

FIG. 6 (a) is the accuracy in a composite fault diagnosis task;

FIG. 6 (b) is a confusion matrix in a composite fault diagnosis task;

FIG. 7 is a graph of diagnostic accuracy and recall obtained in terms of iteration number in a single unknown fault diagnosis task;

FIG. 8 is a comparative experimental diagram in conventional fault diagnosis;

FIG. 9 is a flow chart of a feature extraction module;

fig. 10 is a diagram of a visual feature extraction module convolutional block structure.

Detailed Description

The following describes the present invention in detail.

The invention discloses a zero sample composite fault diagnosis method based on multi-mode contrast embedding, and a specific implementation flow is shown in figure 1. The method can be divided into a multi-mode joint embedding based on SDP transformation, zero sample composite fault diagnosis based on contrast learning and composite fault diagnosis incremental frame based on iterative ideas.

In order to solve the problems of difficult diagnosis of composite faults, difficult distinction of signal samples and poor prediction accuracy balance, the invention provides a zero sample composite fault diagnosis method based on multi-mode comparison and embedding, which mainly comprises a multi-scale convolutional neural network model based on channel weight for feature extraction, a comparison and embedding space without pre-training and a composite fault classifier based on MLP. In the contrast embedding space, the falling points of the normalized feature vector are distributed on the spherical surface, and the contrast loss is obtained by calculating the cosine similarity of the positive sample pair and the negative sample pair which are of the same or different structures according to the category of other samples, so that better clustering can be completed in the feature space, the model convergence speed is improved, and better accuracy rate can be obtained on the composite fault diagnosis of the signal samples which are difficult to distinguish.

In order to solve the problems that a composite fault sample is difficult to reproduce and has high complexity, and simultaneously, the prediction accuracy balance is further improved, the invention provides a diagnosis increment learning framework based on an iterative idea based on a composite fault diagnosis task under generalized zero sample learning, and the diagnosis increment learning framework mainly comprises an increment framework aiming at a comparison sample pool in a test stage and an increment framework for iteratively diagnosing unknown composite fault samples of different categories in a training stage. In consideration of the problem that the effect of the data enhancement method on the composite fault samples is not obvious, a plurality of batches of small batches of composite fault samples are innovatively used for iteration, the complexity of the obtained samples can be enhanced, the iteration effect is improved, and the method can be well represented in composite fault tasks with higher complexity.

Step 1: vibration signal samples of each category are read from a given dataset, taking CWRU datasets as an example, vibration signals consisting of the first 120000 sampling points are taken, and are preprocessed as shown in fig. 2: a total of 4000 signal samples of length 4096 are sliced in each file. The original signal samples are respectively converted into SDP images and fault initial semantic vectors. After training a self-encoder by using the visual feature vector extracted from the SDP image, inputting the fault initial semantic vector into a decoder for feature reconstruction to obtain the reconstructed fault semantic feature.

Step 1.1: the method comprises the steps of extracting signal samples with the length of 1 x 120000 from original vibration signal files, designing a fault sample cutting method based on a sliding window, slicing the original signal samples with the length of 1 x 4096, extracting 4000 samples from each file, and storing the 4000 samples as an original signal data set.

Step 1.2: the original signal data set is input into an SDP image transformation module, and the time series signals are visually represented in the form of symmetrical scatter diagrams under polar coordinates through an SDP image transformation method. The differences in the edge shape and width of the lobes of the SDP graph reflect the changes in the frequency and amplitude of the original signal. The specific flow is as follows:

1) For a one-dimensional time series signal Let/>Is the signal/>(1 /)Sampling points/>Is the post-adjacent time interval signal/>(1 /)And sampling points.

2) Transforming a signal point of a time domain into a scattered point SDP diagram expressed in a polar coordinate space, wherein the formula is as follows:

(1)

(2)

(3)

3) Wherein the method comprises the steps of Is the radius of the scattered point on the polar coordinate graph, and the value is equal to the absolute value of normalized signals,/>AndThe angles at which the petals rotate along the original line in a clockwise and counterclockwise direction, respectively. /(I)Is the rotation angle of the mirror symmetry plane， />Is the number of mirror symmetry planes, typically 6,/>Magnification showing the drawing angle/>， />Is a time interval factor/>. Through preliminary experiments, at/>，/>，/>A distinct difference in the different fault class scatter plots can be observed as shown in fig. 2.

4) And (3) adjusting the image specification, scaling the SDP picture obtained in the previous step into 224 x 224 images, and dividing the images into image data sets according to the category to which the original signal sample belongs.

Step 1.3: and converting the original fault vibration signal data into a fault initial semantic vector through a semantic construction module. The module aims at solving the problem that the conventional fault semantics are too close to each other in feature space, so that the problem is difficult to distinguish, and the vibration features of the original signals are taken as the semantic features of the fault attributes. The specific flow is as follows:

1) For single type fault semantics, the initial semantic vector matrix of single type fault samples is used for Represented asSelecting/>, of the original vibration signalData points, and set threshold/>So that/>In our setup/>The value of (2) is 3, and the sampling points are respectively 4 values according to the amplitude.

2) Normalizing the raw vibration signal if the data points are normalizedGreater thanWill/>The value of the dimension is set to/>Obtain the/>, of single faultDimension semantic vector, wherein/>For the threshold number of copies, represent partition/>Is the number of the values of (a). Representing initial semantic vector at/>Taking the value of dimension; in this study set/>. The specific conversion method is shown as the following formula:

(4)

For compound faults, an initial semantic vector matrix of compound fault samples is used for Expressed as/>，The number of the composite fault semantics is the number of faults which form the composite fault. Obtaining the semantic vector of the composite fault by maximizing the semantic vector of the single fault constituting the composite fault element by element, wherein/>Representing an element-by-element maximization operation.

Step 1.4: and extracting visual features from the SDP image data set through a multi-scale convolutional neural network based on channel weights, and taking a vector with the output size of 1 x 1024 of the previous layer of the output result, namely the full-connection layer, as a visual feature vector of the sample. Then inputting to Encoder in the self-encoder to obtain the coded characteristics, inputting to the decoder to reconstruct the characteristics, and training the self-encoder through MSE loss. The flow is as follows:

1) The SDP image obtains visual characteristics through a characteristic extraction module after input training Will/>Input to self-encoderThe number of neurons in each layer is 1024, 256, 128 in turn, and use/>Activating the function to obtain the coded characteristic/>。

2) Will beInput to/>The number of neurons per layer is 128, 256, 1024 in order, and use/>Activating the function, and acquiring the reconstructed visual feature vector/>。

3) The reconstruction loss of the self-encoder is calculated through the MSE loss function, and the self-encoder is trained. The loss function is as follows:

(5)

To address the problem of poor balance of composite fault diagnosis accuracy that often occurs in composite faults, a binary cross entropy loss is introduced to balance the difference between the information differences between the reconstructed data and the original data. The loss function is as follows:

(6)

the macroscopic loss of the part can be obtained through weighted fusion The formula of (2) is as follows:

(7)

representing the weight taken up by the reconstruction loss.

Step 2: the comparative learning module proposed in this study is shown in fig. 3. In a multi-scale convolutional neural network based on channel weights, we introduce multi-scale convolutional kernels, from 5*5, 3*3 to 1*1, to extract non-smooth fault features in the SDP image; an SEB module based on a channel attention mechanism is introduced to selectively enhance the useful feature map on the convolution channels to reduce redundancy features. In a multi-layer perceptron-based MLP classifier, the linear layer with output size 4096 is followed by a batch normalization layer,The layers are then linear layers with an output size of 1024. In the classification module, combine/>And optimizing the model by the cross entropy loss function and the contrast loss function to realize the classification of fault signals.

Step 2.1: during the training phase we construct a sample feature space. The image dataset 224 from the SDP image conversion module in the previous step is used as the input of the feature extraction module, and all sample features are represented as vectors in the feature space, so as to obtain a sample training set. The training set contains 4000 annotated samples from one condition, 400 annotations for each category. The expression is as follows:

(8)

Is a training dataset in which the known samples are expressed as/> Noted as/>。

1) In forward propagation, we input training sets into a fault diagnosis model framework based on contrast embedding for training, for each sample pictureSDP image input as size 224×224, visual feature output as 1×1024/>。

2) In the back propagation process, through visual featuresConstruction of comparative sample cell/>Diagnosis, method and contrast loss/>, by contrast learning module; We use/>As an optimizer, cross entropy loss/>, is employedAnd comparative loss/>As a joint loss function. The learning rate is set to/>In order to more finely adjust the weight in the later stage of training and avoid excessive oscillation, the attenuation rate is set as/> 。

3) By passing throughThe classifier predicts the sample to obtain a predicted label/>This section contains a linear layer of neuron numbers 1024, 256 in turn. The cross entropy loss is calculated by combining the original label by the following formula:

(9)

Step 2.2: in the contrast learning module, visual characteristic vectors with the length of 1 x 1024 are extracted from the output of the multi-layer perceptron As a unit of comparison. For each training packet, input from the output in the multi-layer perceptron will input feature vectors/>Normalization.

1) A mask matrix is generated in which entries for the same tag are 1 and entries for different tags are 0. For distinguishing positive and negative sample pairs, as shown in the following formula:

(10)

2) At the position of In contrast learning, we need to set positive and negative pairs for the vectors we query. The positive pair comprises a vector to be queried and a vector belonging to a subspace, and the negative pair comprises a group of sum query vectors/>Vectors from different subspaces. The basis for judging whether the query belongs to a certain class is the projection distance. The calculation method is as follows:

(11)

Projection distance Represented as normal vectors where all vectors are orthogonal within the instance subspace. Since the vector length is constant as a unit length, minimizing the projection distance is equal to maximizing the projection length thereof. While the projection length can be seen as the cosine similarity between the query vector and the instance subspace, i.e. by computing the dot product between the feature vectors and by the temperature parameter/>Scaling, temperature parameter/>, in a conventional fault diagnosis taskSet to 0.7, the adjustment of the parameters will be described later. The formula is as follows:

(12)

the maximum value for each row is then subtracted from the dot product to increase the numerical stability.

(13)

3) UsingThe log probability average for each positive sample pair is calculated, and the log probabilities for those samples without positive sample pairs are ignored, as follows:

(14)

4) The final loss is the average of the logarithmic probability negative values for all samples, as shown below:

(15)

5) The overall loss of this part is expressed as:

(16)

Step 2.3: in the test stage, since the dimension of the semantic vector is inconsistent with the fault feature extracted by the feature extraction module, if the detail feature ignored by the SDP image is to be supplemented by semantic information, the semantic vector closest to the fault feature needs to be found by measuring the distance between the fault feature and the semantic vector set in the feature space, so that the model can identify the fault category. We use the trained self-encoder to reconstruct semantic features, the reconstructed features being used for comparative classification in the test stage to distinguish between unknown faults and composite faults. The method comprises the following specific steps:

1) Inputting the test set image into a feature extraction module, extracting visual features from the output of the MLP 。

2) Will beInput to the self-encoder, the self-encoder network is trained by reconstruction loss.

3) Vector semanticsInput to decoder to obtain reconstructed eigenvector/>。

4) From training sets by random sampling methodsSelecting a comparison sample pool, and performing the operation to obtain a reconstructed feature vector set/>。

5) Reconstructing semantic vectors for a test setCalculation/>And/>Positive/negative samples in each categoryCosine similarity of pair/>Classification is performed. /(I)Is a temperature parameter; the formula is as follows:

(17)

Wherein c is the sample category, v is the query vector, N is the number of samples in the test set, m is a natural number between 1 and N, and N is the number of positive samples of the reconstructed semantic features. I x i ₂ represents an L2 norm; if it is Not belonging to any class, which is identified as an unknown fault. If/>Meanwhile, multiple classes have higher and similar similarity, and can be identified as compound faults.

Step 3: the composite fault increment learning diagnosis framework for the generalized zero sample learning condition is shown in fig. 4 and 5. On the basis of the composite fault diagnosis model, we propose two incremental frameworks. Firstly, constructing a comparison sample pool consisting of known fault samples, and improving a clustering effect by taking the detected unknown fault samples as positive faces by an incremental framework in a prediction stage; the incremental framework of the training stage is to add the existing unknown fault sample into the comparison learning pool when new unknown faults are added into the test set, so as to iteratively classify the unknown faults of different categories.

Step 3.1: feature vectors obtained by training after model input from training set by random samplingFor training a self-encoder, training set semantic vectors/>After reconstruction by decoder, 20 reconstructed semantic features/>, 1024 in length, are extracted for each classAdd control sample cell/>As positive samples of the category to which they originally belong and negative samples of other categories. Experiments prove that different sampling modes have small influence on the accuracy and can be ignored.

Step 3.2: inputting samples of the test set into a fault diagnosis model based on contrast embedding, and outputting feature vectors of the multi-layer perceptron MLPFor training a self-encoder, test set semantic vector/>Reconstructed semantic features/>, obtained after reconstruction by a decoderIncremental framework input to prediction phase/>As query vector/>Constructing real example space/>, on hyperspheric feature space by comparing samples in a sample pool. Clustering is completed at the second iteration by computing the distance between the query vector and the instance subspace.

1) An instance subspace is constructed for each class from the comparison sample pool as follows:

(18)

Wherein the method comprises the steps of Representing the class to which the feature vector belongs within the instance subspace.

2) The first iteration, based on the calculations of equations (10) - (12) and the cosine similarity within each instance subspace, extracts unknown fault samples that cannot be assigned to any one of the known fault categories. In the first iteration, temperature parametersSet to 0.9, evaluate probability parameter/>Set to 0.75. Under the generalized zero sample learning condition, the test set includes not only the known fault samples but also the unknown fault samples, so that the unknown fault samples need to be diagnosed as much as possible on the premise of not incorrectly diagnosing the known fault as the unknown fault.

3) And sampling an unknown fault sample obtained by the primary detection and adding the sample into a comparison sample pool to serve as a forward sample of the unknown faults. If the number of unknown fault samples for initial diagnosis is less than 20, the linear transformation data is enhanced until the number of forward samples is sufficient.

4) And (3) performing iteration for the second time, constructing example subspaces including unknown faults by using a new comparison sample pool, and performing iterative detection on unknown fault samples which are not detected in the test set and are not obvious enough based on the formulas (10) - (12) and cosine similarity in each example subspace.

5) And continuing to iterate the sample pool until the diagnosis precision and recall rate of the fault reach the optimal values, and stopping iterating. Usually, after the second iteration, the diagnosis accuracy of the unknown fault can reach the optimal solution.

Step 3.3: after the iteration of one test stage is completed, the diagnosed unknown fault sample is added into a training set to be used as a known fault for training, and a new model for learning the unknown fault is obtained. The identification can be completed when the unknown fault sample of the unknown class appears again in the test set, and when the unknown fault sample of the new class appears, the unknown faults of different classes can be identified iteratively. Unlike active learning, contrast learning does not require manual labeling of the learned unknown faults, and only a small number of samples are needed to construct an instance subspace to distinguish different unknown faults.

And setting a multi-dimensional multi-scene contrast experiment to verify the detection performance of the algorithm. The data set is used primarily to verify the performance of the algorithm training and testing. In order to better verify the detection performance of the model proposed herein, the number of target categories in the dataset, the universality, the stability and other factors are comprehensively considered, and the current experiment is the same as most of the existing zero sample learning fault diagnosis methods, and a CWRU dataset and a PU dataset are selected as verification datasets. And competitive results are obtained on the composite fault diagnosis task, the single unknown fault diagnosis task and the conventional fault diagnosis task.

The pseudo code of the algorithm 1 bearing vibration signal preprocessing method is as follows:

Input:/bearing_ datasets/. Mat// labeling of mat files within the dataset

Output SDP data setSemantic vector dataset/>Tag set/>

1 For mat File do in directory

2, Reading the original vibration signal data

3 Dividing the then into sample sets with the size of 1 x 4096 by adopting a fault clipping method based on a sliding window

4, The then obtains SDP images corresponding to each sample through an SDP image conversion module

5, Then obtaining the semantic vector corresponding to each sample through the semantic vector conversion module

6, Then labeling according to the category of the directory file

7: end for

8: Storing the image set, the semantic feature set and the label as SDP data sets respectivelySemantic vector dataset/>Tag set/>

9: The semantic vector and the label are output to the folder and stored in the format of pt, and the images are sequentially output to the image folder for storage.

The pseudo code of the fault diagnosis method based on the comparison embedding of the algorithm 2 of the invention is as follows:

input SDP data set Semantic vector dataset/>Tag set/>

Output: test set sample class and class classification accuracy

1, Training phase:

2: partitioning 75% from SDP dataset as training set 25% As test set/>

3: For training iteration step = 32 do

4 Reading SDP image with 128 samples as batch_size in training set

5 Forward propagation phase: inputting the image to a feature extraction module, extracting visual features from the output of the MLP. Will/>Input to/>Classifier calculates cross entropy loss/>, based on network output and actual labels; Will/>Input into a contrast embedded classifier to construct a contrast sample pool/>Obtain comparative loss/>。

Back propagation phase: calculating gradients relative to network parameters according to the loss function toAs an optimizer, the learning rate was set to 0.001, and model parameters were optimized by a gradient descent algorithm.

7: end for

And 6, testing:

7 reading SDP image with 128 samples as batch_size in test set

8: The then inputs the image to a feature extraction module, extracts visual features from the output of the MLP。

9: Then will beInput to the self-encoder, the self-encoder network is trained by reconstruction loss.

10: Then will semantic vectorInput to decoder to obtain reconstructed eigenvector/>。

11: Then from the training set by random sampling methodSelecting a comparison sample, and performing the operation to obtain a reconstructed feature vector set/>Adding a comparative sample cell/>Is a kind of medium.

12: For reconstructing feature vectors do：

13: Calculation ofAnd positive/negative samples in each category/>Cosine similarity of pair/>Classification is performed.

14: If it isThe similarity to each category is low, identifying it as an unknown fault.

15: Meanwhile, ifMeanwhile, the classes have relatively high and similar similarity, and can be identified as unknown composite faults.

16: And outputting the identification precision of each type of fault.

The pseudo code of the incremental diagnostic framework based on iterative ideas of algorithm 3 of the present invention is as follows:

input unknown fault sample first detected in Algorithm 2

Output: iteration detected remaining unknown fault samples

1, Testing:

2: LOOP1:

Training the self-encoder by using the unknown fault samples detected for the first time, and matching the corresponding semantic vectors Input into decoder to obtain reconstructed eigenvector/>。

4, Will beAdd control sample cell/>。

5:Then toAnd (4) searching positive samples of the vector for the test set, and clustering by taking other samples as negative samples.

6 The then detects other similar unknown fault samples which have not been detected in the initial detection

7: When the diagnostic accuracy is no longer improved: end Loop1

8: Training phase

9： LOOP2:

And 10, adding the sample of the detected unknown faults into a training set, taking the sample as a sample which is already identified, and training the model so that the model can identify the unknown faults.

And 11, identifying new unknown fault samples in the test set.

12 Then LOOP1

13: When an unknown fault is no longer diagnosed: end Loop2

The method of the present embodiment is as follows: the operating system is windows 11, and the deep learning framework is PyTorch.

The present invention uses CWRU datasets and PU datasets to evaluate the proposed method. The present invention selects 75% from the dataset as training data, the remaining 25% as test data. The characteristics of different fault image samples are extracted by using a multi-scale convolutional neural network based on channel weights, and the effectiveness of the method is measured by indexes such as accuracy, precision, recall rate and the like.

In order to evaluate the effectiveness of the method, the experiment selects more advanced algorithms in the corresponding fields for comparison in three aspects:

In terms of composite fault diagnosis, we achieved optimal results for a three-fault coupled composite fault. Compared with the zero sample composite fault diagnosis method with best results based on semantic learning and distinguishing characteristics, the method has the advantages in balance, and the recognition precision difference between the double-fault-coupling composite fault and the three-fault-coupling composite fault is reduced to 1%.

In the aspect of unknown fault diagnosis of generalized zero sample learning, the iterative framework also achieves the effect of improving the diagnosis accuracy and recall rate, and the lowest false alarm rate is achieved on a test set comprising known faults and unknown faults at the same time, and the accuracy is improved to 94%.

The zero sample composite fault diagnosis method based on multi-mode contrast embedding carries out contrast analysis from different indexes, and results show that in the complex and changeable generalized zero sample learning scene, the fault diagnosis method provided by the invention can be used for carrying out multi-classification on known fault categories and identifying the unknown faults and the composite faults in the diagnosis of the unknown composite faults, the unknown single faults and the conventional faults. The method has the advantages of improving the precision, recall rate and balance of fault detection, and reducing the false alarm rate of model detection, namely, the method provided by the invention has better performance.

The present invention is not limited to the above-described embodiments, and any modifications, variations, substitutions, etc. which are within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (7)

1. A zero sample compound fault diagnosis method based on multi-mode contrast embedding is characterized by comprising a training stage and a testing stage;

the training phase comprises the following steps:

Step 1: bearing vibration signals are collected and are respectively input to an SDP conversion module and a semantic construction module after being preprocessed; the SDP image data output by the SDP conversion module is input into a fault diagnosis model based on contrast embedding; the semantic construction module performs threshold segmentation on the bearing vibration signal to obtain a fault initial semantic vector;

The fault diagnosis model based on the contrast embedding comprises a feature extraction module, a contrast learning module and a self-encoder; the training feature extraction module is used for extracting visual features from the SDP image data;

Step 2: training a self-encoder according to the visual features, carrying out feature reconstruction on the fault initial semantic vector through the self-encoder, and outputting the reconstructed fault semantic features; the self-encoder includes an encoder and a decoder;

In particular to the visual characteristics Input into encoder, visual features/>Dimension reduction is converted into an output vector; the decoder reconstructs the output vector as a visual feature vector/>; Through visual characteristics/>, in training processAnd visual feature vector/>MSE loss and cross entropy loss/>Reconstruction loss function/>, for self-encoderOptimizing;

during the test phase, the initial semantic vector matrix of the single type of fault sample And an initial semantic vector matrix/>, of the composite fault samplesInputting the reconstructed semantic feature vector into an optimized self-encoder, and outputting the reconstructed semantic feature vector/>Reconstructing semantic feature vector/>Inputting the fault samples into a comparison learning module to classify the fault samples; training a contrast learning module, inputting the classified unknown compound faults into an iteration-based diagnosis increment frame, and diagnosing again on a generalized zero sample test set;

After the classification of the unknown composite fault class is completed, inputting the classified samples into a training set in a training stage, randomly selecting the samples, adding the samples into a comparison learning pool, and training the samples as known samples.

2. The zero sample composite fault diagnosis method based on multi-mode contrast embedding of claim 1, wherein the preprocessing specifically adopts a sliding window-based slicing method to divide fault signal samples on bearing vibration signals; intercepting bearing vibration signals with fixed length from a starting position as a fault sample source, sampling fault samples from each fault data file with different types in an open source data set, dividing the fault samples into different data sets according to different load working conditions, and using the data sets of single type fault samples in a training stage and a testing stage; the same mode is adopted to divide the composite fault signal file, and a composite fault signal sample is obtained as a composite fault data set and used in a test stage.

3. The method for diagnosing zero-sample composite fault based on multi-mode contrast embedding of claim 1, wherein the specific conversion process of the SDP image conversion module is as follows:

1) For single type faults, an initial semantic vector matrix of single type fault samples Represented as，/>Representing initial semantic vector at/>Taking the value of dimension; selecting/> -of original bearing vibration signalData segments and set threshold/>So that/>， />The value of (2) is 3, and the sampling points are respectively 4 values according to the amplitude;

2) Normalizing the original bearing vibration signal according to the normalized value Decision/>Of (3), wherein/>For the threshold number of copies, represent partition/>The number of the values of (a); obtaining/>, single type of faultDimension initial semantic vector/>; The specific conversion method is shown as the following formula:

3) For compound faults, an initial semantic vector matrix of compound fault samples is used for Expressed as/>，The number of initial semantic vectors for the composite fault; obtaining initial semantic vectors of the composite faults by performing element-by-element maximization operation on the initial semantic vectors of the single faults forming the composite faults, wherein/>Representing an element-by-element maximization operation.

4. A method of zero sample composite fault diagnosis based on multi-modal contrast embedding as claimed in claim 2 or 3, wherein the contrast sample cell is constructed in the iterative based diagnostic incremental frameworkExtracting a fixed quantity of visual feature vectors from each known fault class, adding the visual feature vectors into a comparison sample pool, normalizing the visual features of each known fault class, forming an example subspace of a class plane on an hypersphere feature space, and calculating the probability that one query feature vector belongs to a certain class through projection distance and cosine similarity; under the learning scenario of a generalized zero sample test set, the first unknown fault sample does not have positive sample pairs in a comparison sample pool, and the construction process of the comparison loss function is as follows:

1) Generating a mask matrix, wherein the sample pair entries with the same label are 1, and the sample pair entries with different labels are 0, so as to distinguish positive and negative sample pairs, as shown in the following formula:

Wherein the method comprises the steps of 、/>Labels representing two comparative samples in a comparative sample cell;

2) The basis for judging whether the query feature vector belongs to a certain category is a projection distance, and the projection distance calculating method is as follows:

Representing query feature vectors/> The distance of the projection on the instance subspace, n, represents the number of samples that make up the real instance space;

Projection distance Expressed as normal vectors orthogonal to all vectors in the example subspace, by calculating the dot product between eigenvectors and by the temperature parameter/>Scaling, setting a temperature parameter/>, in a fault diagnosis task0.7, The formula is as follows:

Wherein the method comprises the steps of Representing query feature vectors,/>A set of total vectors representing an instance subspace;

Then subtracting the maximum value of each row from the dot product to obtain ；

3) UsingLog probability mean/>, for each positive sample pairThe log probability of samples without positive sample pairs is not calculated, as follows:

4) Final contrast loss function The mean value of the logarithmic probability negative values for N samples is given by:

5) The contrast learning module overall loss is expressed as:

For cross entropy loss to account for total loss weight,/> Is a cross entropy loss.

5. The zero-sample composite fault diagnosis method based on multi-modal contrast embedding of claim 4, wherein the feature extraction module specifically acquires visual features in the SDP image through a multi-scale convolutional neural network based on channel weights; the SDP image output by the SDP image conversion module and the corresponding label form a setAs input to the feature extraction module; /(I)， />For/>Number of SDP image samples,/>For/>Corresponding labels of the SDP images;

Extracting an equal-length-width-size square region containing all images from the center of the SDP image as a characteristic block through image transformation operation; inputting the characteristic blocks into a multi-scale convolutional neural network based on channel weight for characteristic extraction, and performing multi-scale convolutional layer, batch processing layer, Linear rectifying layer, max pooling layer and/>Depth visual features/>, obtained after manipulation; The depth visual features are weighted by the attention weights of the channels after being compressed;

Wherein the method comprises the steps of For compressed depth visual features,/>Table full connectivity layer operation,/>Is the weight of the full connection layer,/>And/>Weights of two full connection layers,/>, respectivelyRepresentation/>The activation function outputs the weighting factor/>, for each channelIs a collection of (3); /(I)Representation/>Activating a function to per channel/>, of a depth visual featureMultiplying the corresponding weight coefficient/>Weighting the feature map is realized;

Weighted feature map The visual characteristics/>, are obtained by sequentially inputting the visual characteristics/>, the visual characteristics and the visual characteristics to the full-connection layer and the MLP classifier of 1 x 1024And an image classification result; in the training stage, an equal number of fault samples are extracted from the visual characteristics to construct a visual contrast learning poolVisual characteristics/>, of each known fault classAfter normalization, a plane-like example subspace is formed over an hyperspherical feature space, using visual features/>Calculating the probability that a query feature vector belongs to a certain category according to the projection distance and cosine similarity of the example subspace; the training phase inquiry feature vector is visual feature/>And calculating a contrast loss optimization feature extraction model, wherein the calculation mode of the contrast loss function is the same as that of a contrast sample pool.

6. The method for zero sample composite fault diagnosis based on multi-modal contrast embedding of claim 5, wherein the testing stage outputs a reconstructed semantic feature vectorInputting the fault samples into a comparison learning module to classify the fault samples;

Reconstructing semantic feature vectors using training sets Constructing a semantic comparison sample pool/>The reconstructed semantic feature vector/>, obtained in the test setInputting the vector as a query feature vector into an hypersphere feature space; by comparing sample cellsThe reconstructed semantic feature vector in (1) constructs an example space on the hypersphere feature space, and uses the semantic feature vector/>And calculating the probability that one query feature vector belongs to a certain category according to the projection distance and cosine similarity of the example subspace, and performing first diagnosis classification diagnosis to obtain a classification result.

7. The zero-sample composite fault diagnosis method based on multi-modal contrast embedding of claim 6, wherein the iteration-based diagnosis increment framework modifies the contrast sample pool after obtaining the primary classification diagnosis resultTemperature parameter taken for diagnosis of unknown composite fault samples/>The samples belong to a specific class probability threshold; The method comprises the following specific steps:

adding the reconstructed semantic feature vector of the detected unknown composite fault sample into a comparison sample pool As a forward sample of the unknown class; when the unknown composite fault sample of the primary diagnosis is less than a fixed quantity, linear transformation data enhancement is carried out on the unknown composite fault sample until the quantity of the forward samples is sufficient; utilize new comparative sample cell/>Constructing example subspaces including unknown composite samples, calculating cosine similarity in each example subspace, and performing iterative detection on unknown fault samples which are not detected in the test set; and continuing to iterate the comparison sample pool until the diagnosis precision and recall rate of the fault reach the optimal value, and stopping iterating.