CN118035866A - Fault diagnosis method for aluminum extrusion equipment based on expert network - Google Patents

Abstract

The invention provides a fault diagnosis method of deep reinforcement learning aluminum extrusion equipment based on an expert network, which is implemented according to the following steps: step 1, preprocessing collected aluminum extrusion equipment data, processing the data into input signals which can be identified by a classifier, and determining the number of samples to be identified and the characteristic number of each type of sample, namely sample dimension; step 2, selecting a convolutional neural network as a classifier model, inputting the preprocessed aluminum extrusion equipment data into a classifier, and outputting probability distribution matrixes of the classifier on samples of different categories; step 3, constructing an expert network, and providing labels for data samples input into the expert network based on the similarity between samples; step 4, constructing a deep reinforcement learning DQN algorithm frame; step 5, training a deep reinforcement learning DQN algorithm, and outputting an optimal action; and 6, inputting the data sample of the aluminum extrusion equipment to be identified into the trained network model, and calculating the identification accuracy.

Description

Fault diagnosis method for aluminum extrusion equipment based on expert network

Technical Field

The invention belongs to the technical field of power equipment monitoring, and particularly relates to a fault diagnosis method for deep reinforcement learning aluminum extrusion equipment based on an expert network.

Background

The industrial Internet is used as the industrial and application ecology of the comprehensive deep fusion of a new generation of information technology and an industrial system, and is the key foundation for realizing intelligent operation and maintenance of equipment. The modern manufacturing industry tends to develop intelligent, lean and personalized customization, wherein the importance of aluminum extrusion production is particularly outstanding, and the aluminum extrusion production is widely applied to the industries of aerospace, transportation building and the like. In recent years, the Chinese aluminum processing industry combines the demands of market and scientific development, so that the traditional aluminum extruded profile has gradually completed the conversion into the modern aluminum extruded profile, and the technical level and the production process of part of enterprises have reached the international leading level. Representative of these are Liaoning Zhong groups which are mainly engaged in the development of aluminum processed products, including industrial aluminum extrusion business, deep processing business, aluminum calendaring business, and the like. Because the aluminum processing factory equipment is large in quantity and complex in structure, faults possibly occur in the long-time use process, once the equipment fails, if the fault state of the equipment cannot be diagnosed in time and corresponding decisions are made, serious problems such as production stagnation and the like are caused, and loss which is difficult to measure is caused. In practice, it is unavoidable that the equipment fails, so it is important to accurately determine whether the equipment fails in time.

Disclosure of Invention

The invention aims to provide a fault diagnosis method for deep reinforcement learning aluminum extrusion equipment based on an expert network, which solves the problem of low accuracy in identifying novel fault data in the prior art and is more suitable for the complex and changeable actual environment at present.

The technical scheme adopted by the invention is that the fault diagnosis method of the deep reinforcement learning aluminum extrusion equipment based on the expert network is implemented according to the following steps:

Step 1, preprocessing collected aluminum extrusion equipment data, processing the data into input signals which can be identified by a classifier, and determining the number of samples to be identified and the characteristic number of each type of sample, namely sample dimension;

step 2, selecting a convolutional neural network as a classifier model, inputting the preprocessed aluminum extrusion equipment data into a classifier, and outputting probability distribution matrixes of the classifier on samples of different categories;

step 3, constructing an expert network, and providing labels for data samples input into the expert network based on the similarity between samples;

step 4, constructing a deep reinforcement learning DQN algorithm frame;

step 5, training a deep reinforcement learning DQN algorithm, and outputting an optimal action;

and 6, inputting the data sample of the aluminum extrusion equipment to be identified into the trained network model, and calculating the identification rate.

The invention also has the following optional features.

In step 2, the classifier model is determined to be a convolutional neural network, the output layer activation function is set to softmax, and the classifier is trained.

In step 2, setting super parameters of the classifier, establishing an objective function, and optimizing the parameters by adopting a gradient descent method.

In step 4, an action space is defined, a state space is defined, and a reward function is defined.

In step 5, the Q network is set up, then the memory bank is set up, and then the DQN algorithm is trained.

In step 6, the recognition accuracy

And Num is the total number of samples to be identified, and b is the samples to be identified with correct classification.

The method has the beneficial effects that by processing and analyzing the data of the aluminum extrusion equipment, due to the high dimensional characteristic of the processed data, the convolutional neural network is used as a classifier of the data, and the classification effect is superior to that of the traditional neural network (BP) and the traditional classification method (SVM and the like); meanwhile, in the stage of training the classifier, all types of data samples are difficult to train, and the performance of the classifier is possibly reduced due to the new type of samples, so that an expert network and a deep reinforcement learning DQN algorithm are introduced, data with abnormal classification results are processed by the expert network, the final recognition precision is improved, and the loss caused by diagnosis errors is minimized as much as possible. Specific advantages include the following:

1) The method has strong intelligence, can judge whether the classification result of the classifier is abnormal or not, and adopts corresponding measures to ensure that the final classification precision is optimal.

2) The method has the advantages of high practicability, better recognition precision aiming at the problem of fault recognition of the aluminum extrusion equipment, and simple realization process.

3) The method has stronger universality, can be suitable for other mode identification problems in the aspect of fault diagnosis of aluminum extrusion equipment, and can still obtain ideal identification performance.

Drawings

Fig. 1 is a general flow chart of the present method.

Detailed Description

The invention will be described in detail below with reference to the drawings and the detailed description.

Referring to fig. 1, the fault diagnosis method of the aluminum extrusion apparatus of the present invention is implemented as follows:

step 1, preprocessing collected aluminum extrusion equipment data, processing the collected aluminum extrusion equipment data into input signals which can be identified by a network, and determining the number of categories of samples to be identified and sample dimensions;

The processed aluminum extrusion equipment data sample set is { (x ⁽¹⁾,y⁽¹⁾),...,(x^(m),y^(m)) }, wherein m is the number of training samples, x ⁽ⁱ⁾ is the ith training sample, y ⁽ⁱ⁾ epsilon {1, 2..the k } is the label of the ith training sample, k is the number of classes to be identified in the aluminum extrusion equipment data, the data is divided into training samples and test samples, the training samples are divided into classifier training samples and new class samples according to class labels, and the class of the classifier samples and the class of the new class samples are not overlapped with each other.

Step 2, selecting a convolutional neural network as a classifier model

Determining classifier models as convolutional neural networks

Because the neural network has strong self-learning capability and fault tolerance capability, compared with the traditional machine learning algorithm, the neural network has obvious advantages in solving the problem of pattern recognition; in the classification problem, the accurate extraction of the representative features of the data is critical, and the convolutional neural network is adopted as a classifier model because the convolutional neural network has unique advantages when the features of the data are extracted and has the characteristic of weight sharing, so that the model convergence speed can be accelerated;

The output layer activation function is set to softmax.

The output layer activation function of this step is set to softmax, and when the training sample set is { (x ⁽¹⁾,y⁽¹⁾),...,(x^(m),y^(m)) }, the softmax function classifies the extracted data feature to be identified according to the following formula:

Wherein, each element p of the vector h _θ(x⁽ⁱ⁾) is assumed (y ⁽ⁱ⁾＝j|x⁽ⁱ⁾; θ) represents the probability that the sample object feature x ⁽ⁱ⁾ belongs to the j-th class (the sum of the elements of the vector h _θ(x⁽ⁱ⁾) is 1, and the larger the probability, the larger the probability that the feature x ⁽ⁱ⁾ to be identified belongs to the j-th class, wherein θ ₁,θ₂,...,θ_k is the parameter vector of the model.

And training a classifier.

Setting super parameters of classifier

The convolutional neural network uses a LeNet-5 network, the number of nodes of an input layer is set to be the characteristic number of a sample to be identified, a convolutional layer filling mode (padding) is set to be all zero filling, the moving step length (stride) of a convolutional kernel is set to be 1, an activation function (activation) is set to be relu functions, a batch standardization (BN) layer is set in front of the convolutional layer for reducing the calculated amount, and a classifier is used for training the sample.

An objective function is established.

The objective function, loss function Loss, is defined as the mean square error of the classifier:

wherein y _i represents the true class label of the ith sample, A predictive category label representing the fault diagnosis network for the ith sample. The smaller the value of Loss, the better the performance of the classifier.

Optimizing parameters by gradient descent method

And calculating gradient variables by adopting a back propagation mode, and optimizing weight parameters in the convolutional neural network to minimize an objective function.

Step 3, constructing expert network

As shown in table 1, an expert knowledge base is constructed, and tags are provided for data samples input into the expert network based on expert knowledge.

TABLE 1

And 4, constructing a deep reinforcement learning DQN algorithm frame.

In the method, the fault diagnosis process of the aluminum extrusion equipment is constructed as a Markov decision process, a Markov decision problem model is based on a four-element group { S, A, T, R }, wherein a state space S is a finite set containing all states, an action space A is a finite set containing all actions, T is a probability transition matrix, which means the probability of transition from a current moment state S _t to a next moment state S _t+1, the transition probability is defined as 1 in the method, a reward R is defined as a feedback value given by an environment in the interaction process of an agent and the environment, and a corresponding value is obtained according to the advantages and disadvantages of the actions. In the fault diagnosis problem, because the action space is discrete, a deep reinforcement learning DQN algorithm is selected to carry out modeling solution on the problem, and the correlation definition of the model is as follows:

an action space is defined.

The action space a is defined as the set of actions that the agent has all the possible choices, as shown in (2),

A＝{A_c,A_e} (2)

The classification action a _c＝[a₁,a₂,...,a_k],a_i refers to classifying the current sample into the i-th category, requesting the label action a _e＝[a_e, and indicating that the agent decides to request the expert network, obtains the label probability distribution given by the expert network, and then completes the classification operation.

A state space is defined.

The state should be sufficiently related to all the possibilities in the diagnostic process, reflecting the uncertainty in practice, and therefore the state space S is defined as a matrix formed by stitching together the probability distribution matrix P _c output by the classifier for all training samples and the probability distribution matrix P _e output by the expert network, as shown in (3),

S＝[P_c,P_e] (3)

If the action executed by the agent on the classifier output p _c of the ith sample is the classification action a _c, i.e. the action is directly selected according to the output probability distribution of the classifier without depending on the knowledge of the expert network, in order to ensure the consistency of the state dimension, at this time, p _e in the state s is filled by using the all-zero matrix.

A bonus function is defined.

The environmental rewards R for agents are defined in terms of the actions a selected by the agents, the rewards function being shown in (4),

If the intelligent agent selects action a _c according to the input state and classifies correctly, the corresponding obtained rewarding value is +1, if the classification is wrong, the obtained rewarding value is-2, if the selected action is a _e, the obtained rewarding value is-1, and the setting mode is more beneficial to the intelligent agent to have intelligent autonomous decision capability in interaction with the environment.

Step 5, training the deep reinforcement learning DQN algorithm

First set up Q network

The Q network is set as a long-short-term memory network (LSTM) and is used for selecting actions according to strategies in different states, the strategies are defined as weight parameters of the neural network, and in order to enable the algorithm to have stronger robustness, the method adopts two Q networks with the same structure and different parameters: the parameters of the training network and the target network are theta' and theta respectively.

And a memory bank is set.

Because the action space is discrete, the method solves the problem of fault diagnosis by using the DQN algorithm in deep reinforcement learning, and in order to solve the problem of high correlation between adjacent states, a memory library D with the capacity of N is set up and set up in the method for storing the exploration result in the process of interacting with the environment, and the correlation between the states is reduced by randomly extracting a fixed number of samples in the training process.

The storage form is shown as (5),

{s_t,a_t,q,r_t}(5)

Where s _t represents the environmental state observed by the agent at time t, a _t represents the action taken by the agent for that state, q represents the return value of each action selected in state s _t, and r _t represents the instant prize value obtained after action a _t is performed in that state.

The DQN algorithm is retrained.

A loss function is established, a mean square error function is used as the loss function in the DQN training process, as shown in a formula (6),

Wherein,Representing the target network parameter as/>Value obtained when action a is performed on input state s,/>Representing training network parameters as/>For the value obtained when the input state s 'executes the action a', an epsilon-greedy algorithm is used as an action selection strategy, gamma is an attenuation factor, the value range is [0,1], and the larger the value of gamma is, the larger the influence of the Q value of the next state on the Q value of the current state is.

Optimizing parameters using gradient descent method, calculating loss functionFor parameter/>As shown in (7),

Training the agent by continuously extracting small batches of samples from the memory bank during the training process, and fixing network parameters when the loss function converges to a certain constant valueAnd applies it to the actual aluminum extrusion fault diagnosis process.

Step 6, inputting the data sample of the aluminum extrusion equipment to be identified into the trained network model, calculating the identification rate,

In order to more intuitively express the classification effect of the network on the sample to be identified, the accuracy rate of identifying the fault data of the aluminum extrusion equipment by the network is calculated by using the formula (8):

And Num is the total number of samples to be identified, and b is the samples to be identified with correct classification.

In a word, the fault diagnosis method of the aluminum extrusion equipment provided by the invention adopts the expert knowledge and deep reinforcement learning mode to carry out secondary identification on the data samples, so that the interference effect of novel samples on the classifier can be effectively reduced, the identification capability of a fault diagnosis system on various samples can be effectively improved, and the loss caused by the fault of the aluminum extrusion equipment on actual production and life is greatly reduced.

The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims. The components and structures not specifically described in this embodiment are well known in the art and are not described in detail herein.

Claims (6)

1. The fault diagnosis method of the deep reinforcement learning aluminum extrusion equipment based on the expert network is characterized by comprising the following steps of:

Step 1, preprocessing collected aluminum extrusion equipment data, processing the data into input signals which can be identified by a classifier, and determining the number of samples to be identified and the characteristic number of each type of sample, namely sample dimension;

step 2, selecting a convolutional neural network as a classifier model, inputting the preprocessed aluminum extrusion equipment data into a classifier, and outputting probability distribution matrixes of the classifier on samples of different categories;

step 3, constructing an expert network, and providing labels for data samples input into the expert network based on the similarity between samples;

step 4, constructing a deep reinforcement learning DQN algorithm frame;

step 5, training a deep reinforcement learning DQN algorithm, and outputting an optimal action;

And 6, inputting the data sample of the aluminum extrusion equipment to be identified into the trained network model, and calculating the identification accuracy.

2. The expert network-based fault diagnosis method for deep reinforcement learning aluminum extrusion equipment according to claim 1, wherein in step 2, a classifier model is determined as a convolutional neural network, an output layer activation function is set as softmax, and a classifier is trained.

3. The fault diagnosis method for deep reinforcement learning aluminum extrusion equipment based on expert network as claimed in claim 2, wherein in step 2, classifier super parameters are set, objective functions are established, and the parameters are optimized by adopting a gradient descent method.

4. The expert network-based fault diagnosis method for deep reinforcement learning aluminum extrusion equipment according to claim 1, wherein in step 4, an action space is defined, a state space is defined, and a reward function is defined.

5. The fault diagnosis method for deep reinforcement learning aluminum extrusion equipment based on expert network according to claim 1, wherein in step 5, the Q network is set first, then the memory bank is set, and then the DQN algorithm is trained.

6. The expert network-based fault diagnosis method for deep reinforcement learning aluminum extrusion equipment of claim 1, wherein in step 6, the recognition accuracy is high

And Num is the total number of samples to be identified, and b is the samples to be identified with correct classification.