CN112699244A - Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment - Google Patents

Abstract

The present invention provides a method and system for classifying power transmission and transformation equipment defect texts based on deep learning. The method includes the following steps: S1: preprocessing the acquired power transmission and transformation equipment defect text, and then performing word embedding to obtain a first text with power semantic features. Word vector; S2: Obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long-term memory network, and output the hidden layer state vector; S3: Use the self-attention mechanism to weight the hidden layer state vector to obtain the deep layer. Semantic features are obtained to obtain the final sentence vector to be classified; S4: the vector to be classified is output to the Softmax classifier through the fully connected layer to obtain the text classification result of power transmission and transformation equipment defects. The method can solve the technical problems that the existing text classification of defects in the electric power field has high labor costs, the classification results are easily affected by the experience of classification technicians, and the traditional text classification methods are not suitable for the electric power field.

Description

Text classification method and system for power transmission and transformation equipment defects based on deep learning

technical field

The invention belongs to the technical field of natural language processing, and in particular relates to a method and system for classifying defective texts of power transmission and transformation equipment based on deep learning.

Background technique

With the continuous development of the smart grid, a large amount of defect text data will be generated during the daily operation and maintenance of the power grid; and the analysis and summary of the defect text data is the original basis for the processing and analysis of the faults of power grid equipment. At present, the text analysis of power transmission and transformation equipment defects in the power grid is mainly done manually, which is costly and inefficient, and is easily affected by differences in manual experience, resulting in biased classification results. The development of artificial intelligence and natural language processing technology provides the possibility of text mining of electrical equipment defects. Existing text classification technologies include Naive Bayes, Support Vector Machines, Decision Trees, etc., but traditional text classifiers based on machine learning related algorithms are difficult to mine the deep features of text, which is not conducive to further analysis and application of text data. At the same time, the text in the field of electric power contains a large number of professional terms and special symbols, which is highly professional, and it is difficult for the general classification model in deep learning to be directly transferred and applied, and the current power text mining is still in its infancy.

SUMMARY OF THE INVENTION

In view of this, one of the objectives of the present invention is to provide a text classification method for power transmission and transformation equipment defects based on deep learning, which can be applied to the classification of defect texts in the power field.

In order to achieve the above purpose, the technical scheme of the present invention is: a deep learning-based method for classifying defective texts of power transmission and transformation equipment, comprising the following steps:

S1: Preprocess the acquired power transmission and transformation equipment defect text, and then perform word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features;

S2: Obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long-short-term memory network, and output the hidden layer state vector;

S3: Use the self-attention mechanism to perform weighted transformation on the hidden layer state vector, obtain deep semantic features, and obtain the final sentence vector to be classified;

S4: Output the to-be-classified sentence vector to the Softmax classifier through a fully connected layer to obtain a text classification result of power transmission and transformation equipment defects.

Further, the preprocessing includes word segmentation, stop word removal and unified terminology processing on the power transmission and transformation equipment defect text.

Further, in the step S1, the step of performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain the first word vector with power semantic features specifically includes:

Read the preprocessed power transmission and transformation equipment defect text, and count the word frequency information;

Build a dictionary, initialize the Huffman tree and randomly initialize the word vector;

Train the model in row units and get an input sample in the current row;

Accumulate the value of each dimension in the context word vector and average it to get the projection layer vector;

Traverse each intermediate node from the current word to the root node;

Calculate the gradient g*learning rate corresponding to the intermediate node, refresh the error vector of the projection layer to the intermediate node, refresh the intermediate node vector, and refresh the context word vector.

Further, the step S2 specifically includes the following steps:

Define the forward LSTM structure and the backward LSTM structure, use the dynamic RNN unit to splicing the results of the network output, and then input them to the next layer of bidirectional long and short-term memory network, and divide the output results of the last layer of Bi-LSTM into split methods. forward and backward output;

Add the forward and backward outputs to get the final hidden layer state.

Further, each time state in the LSTM structure is updated in the following way:

为LSTM状态和LSTM内部状态的激励函数，设置为双曲正切函数tanh， b为偏置常量，下标中i、f、o分别表示输入门、遗忘门和输出门；g为随时间步更新的控制门 单元，即以sigmiod函数为激励函数的前馈神经网络，

为当前t时间状态，

为前一 时间状态，

为当前时刻的输入，

为权重值，

为输入门权重值，

为输出门权重 值，

为遗忘门权重值，

为当前时刻的抽象化信息，

为前一时间步的抽象化信息，

为权重系数。 in,

is the excitation function of the LSTM state and the LSTM internal state, set to the hyperbolic tangent function tanh, b is the bias constant, i, f, o in the subscript represent the input gate, forget gate and output gate respectively; g is updated with time steps The control gate unit of , that is, the feedforward neural network with the sigmiod function as the excitation function,

is the current state at time t,

is the state of the previous time,

is the input at the current moment,

is the weight value,

is the input gate weight value,

is the output gate weight value,

is the forget gate weight value,

is the abstract information of the current moment,

is the abstracted information of the previous time step,

is the weight coefficient.

Further, the deep semantic features in the step S3 are obtained in the following ways:

；

;

为词向量维度，T为 矩阵转置。 Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose.

The second purpose of the present invention is to provide a text classification system for power transmission and transformation equipment defects based on deep learning, which can be applied to the classification of defect texts in the field of electric power.

In order to achieve the above purpose, the technical solution of the present invention is: a deep learning-based power transmission and transformation equipment defect text classification system, comprising:

The text processing module is used to preprocess the acquired power transmission and transformation equipment defect text, and then perform word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features;

The semantic feature extraction module is connected to the text processing module, and is used to obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long and short-term memory network, output the hidden layer state vector, and use it to use the self-attention mechanism. The hidden layer state vector is weighted and transformed to obtain deep semantic features, and the final sentence vector to be classified is obtained;

The text classification module is used for inputting the received sentence vector to be classified into the fully connected layer and outputting it to the Softmax classifier, so as to obtain the text classification result of the defect of the power transmission and transformation equipment.

Further, the text processing module preprocessing the power transmission and transformation equipment defect text includes word segmentation, stop word removal, and unified terminology processing on the power transmission and transformation equipment defect text.

Further, the deep semantic features are obtained in the following ways:

；

;

为词向量维度，T 为矩阵转置。 Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose.

Further, the semantic feature extraction module is also used to define the forward LSTM structure and the backward LSTM structure, and use the dynamic RNN unit to splicing the results output by the network, and then input it to the next layer of bidirectional long-term memory network, and the last The output of the layer Bi-LSTM is split into forward and backward outputs by the split method, and the forward and backward outputs are added to obtain the final hidden layer state.

Compared with the prior art, the present invention has the following advantages:

1. Reduce the labor cost of the actual production of the power grid, avoid the failure and defect text classification results from being affected by the experience of different personnel, greatly improve the efficiency of power transmission and transformation equipment defect text classification, and improve the power transmission and transformation equipment defect text classification accuracy rate.

2. In the actual production of the power industry, the automatic classification of power transmission and transformation equipment defect texts can provide an objective and efficient fault classification reference, and at the same time, it can be combined with structured data to perform comprehensive state estimation of the health status of power transmission and transformation equipment;

3. Power transmission and transformation equipment defect text classification is part of the research content of power text mining, which lays the foundation for mining and analyzing power equipment defect text data and other power text data.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a structural diagram of a deep learning-based power transmission and transformation equipment defect text classification system of the present invention;

2 is a flowchart of a deep learning-based method for classifying defective texts of power transmission and transformation equipment according to the present invention;

FIG. 3 is a schematic diagram of a 100-dimensional word vector after training in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The examples are given to better illustrate the present invention, but the content of the present invention is not limited to the examples. Therefore, those skilled in the art make non-essential improvements and adjustments to the embodiments according to the above-mentioned contents of the invention, which still belong to the protection scope of the present invention.

Example 1

1 is a structural diagram of a text classification system for power transmission and transformation equipment defects based on deep learning of the present invention. Specifically, the system includes:

The text processing module 1 is used for preprocessing the acquired power transmission and transformation equipment defect text, and then performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features;

In this embodiment, the text processing module performs preprocessing on the defect text of the power transmission and transformation equipment, including word segmentation, removal of stop words, and unified terminology processing on the defect text of the power transmission and transformation equipment.

The semantic feature extraction module 2 is connected to the text processing module 1, and is used to obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long-short-term memory network, output the hidden layer state vector, and use it to use the self-attention mechanism. The hidden layer state vector is weighted and transformed to obtain deep semantic features, and the final sentence vector to be classified is obtained;

In the semantic feature extraction module 2, the forward LSTM structure and the backward LSTM structure are defined, and the dynamic RNN unit is used to splicing the results of the network output, and then input to the next layer of bidirectional long and short-term memory network, and the last layer of Bi-LSTM The output results are divided into forward and backward outputs by the split method, and the forward and backward outputs are added to obtain the final hidden layer state.

Preferably, the deep semantic features are obtained by:

；

;

为词向量维度，T为 矩阵转置。 Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose.

The text classification module 3 is connected to the semantic feature extraction module 2, and is used for inputting the received sentence vector to be classified into the fully connected layer and outputting it to the Softmax classifier, so as to obtain the text classification result of power transmission and transformation equipment defects.

Example 2

Based on the system of Embodiment 1, this embodiment provides a deep learning-based text classification method for power transmission and transformation equipment defects. Referring to FIG. 2 , the method includes the following steps:

S1: Preprocess the acquired power transmission and transformation equipment defect text, and then perform word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features;

In this step, the preprocessing of the power transmission and transformation equipment defect text includes word segmentation, stop word removal and unified terminology processing for the power transmission and transformation equipment defect text;

Further, the step of performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features specifically includes:

Read the preprocessed power transmission and transformation equipment defect text, and count the word frequency information;

Build a dictionary, initialize the Huffman tree and randomly initialize the word vector, and the word vector dimension is 100;

Train the model in row units and get an input sample in the current row;

Accumulate the value of each dimension in the context word vector and average it to get the projection layer vector;

Traverse each intermediate node from the current word to the root node;

Calculate the gradient g*learning rate corresponding to the intermediate node, refresh the error vector of the projection layer to the intermediate node, refresh the intermediate node vector, and refresh the context word vector.

In a specific embodiment, the grid equipment defect record data of a provincial power company from 2018 to 2019 is selected, because the power transmission and transformation equipment is the equipment type with the largest number of defect records, while the number of other equipment types is small, and does not have the ability to constitute deep learning. Data set conditions, so the defect records of power transmission and transformation equipment are selected as the original data. Preprocessing the text of power transmission and transformation equipment defects, including word segmentation, removing stop words and unified terms; using the 5-fold verification method, all power transmission and transformation equipment defect texts are randomly arranged and divided into 5 parts on average, and 4 of them are selected in turn. As a training set, 1 is used as a test set for verification, assuming that one of the test data sets is "35kv Wanlipo substation 1# main transformer oil conservator oil level is too low", "35kv day thermal hydropower plant main transformer tap Excessive oil sealing of the switch respirator"; after preprocessing the above data set, they are "35kv", "main transformer", "main body", "oil conservator", "oil level", "too low" and "35kv" "main transformer" "Tap changer" "Breather" "Oil seal" "Too much".

As in a specific embodiment, it is assumed that the word segmentation result of one of the test data sets is "insulator", and its 100-dimensional word vector representation after training is shown in Figure 3;

S2: Obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long-short-term memory network, and output the hidden layer state vector;

In this embodiment, the forward long short-term memory network (LSTM) structure and the backward long short-term memory network (LSTM) structure are first defined, and the dynamic RNN unit is used to output the results of the network (fw and bw, fw is the forward LSTM output result , bw is the reverse LSTM output result) for splicing, and then input to the next layer of bidirectional long and short-term memory network, and the result of the last layer of Bi-LSTM output is divided into forward and backward through the split method (split function) in python Output;

Add the forward and backward outputs to get the final hidden layer state.

Further, each time (step) state in the LSTM structure is updated by the following formula:

为系统状态和内部状态的激励函数，设置为双曲正切函数tanh，b为 偏置常量，下标中i、f、o分别表示输入门、遗忘门和输出门；g为随时间步更新的控制门单 元，即以sigmiod函数为激励函数的前馈神经网络，

为当前时刻t的状态，

为前一时间 隐藏层状态，

为当前时刻的输入，

为权重值，

为输入门权重值，

为输出门权重 值，

为遗忘门权重值，

为当前时刻的抽象化信息，

为前一时间步的抽象化信息，

为权重系数。 in,

is the excitation function of the system state and internal state, set as the hyperbolic tangent function tanh, b is the bias constant, i, f, o in the subscript represent the input gate, forget gate and output gate respectively; g is updated with time steps The control gate unit, that is, the feedforward neural network with the sigmiod function as the excitation function,

is the state at the current time t,

is the state of the hidden layer at the previous time,

is the input at the current moment,

is the weight value,

is the input gate weight value,

is the output gate weight value,

is the forget gate weight value,

is the abstract information of the current moment,

is the abstracted information of the previous time step,

is the weight coefficient.

S3: Use the self-attention mechanism to perform weighted transformation on the hidden layer state vector, obtain deep semantic features, and obtain the final sentence vector to be classified;

In this step, the state of the hidden layer is introduced into the self-attention mechanism; the attention weight assigned to each input is calculated, and the calculated weight is applied to the feature vector to obtain the final output feature vector. The specific calculation formula is as follows:

；

;

为词向量维度，T 为矩阵转置。 Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose.

S4: The to-be-classified sentence vector is output to the Softmax classifier through the fully connected layer, and the text classification result of the defect of the power transmission and transformation equipment is obtained.

In this embodiment, the sentence vector to be classified is output to the softmax classifier through the fully connected layer to obtain the probability of the corresponding class, and the class with the highest probability is selected as the final class.

Preferably, the construction process of the softmax classifier in this embodiment is as follows: acquiring the power transmission and transformation equipment defect texts of known defect types, performing data preprocessing through the defect text preprocessing unit, and then using the defect text feature representation unit to obtain text features with text features The word vector is obtained through the defect text feature extraction unit to obtain the semantic feature sentence vector to be classified, and the training data is obtained. The model parameters of the softmax classifier are trained by the stochastic gradient descent method, and the training ends when the loss function is minimized, and the softmax classifier is obtained. .

The embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of the present invention, without departing from the scope of protection of the present invention and the claims, many forms can be made, which all belong to the protection of the present invention.

Claims (10)

1. a power transmission and transformation equipment defect text classification method based on deep learning, is characterized in that, comprises the following steps: S1: Preprocess the acquired power transmission and transformation equipment defect text, and then perform word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features; S2: Obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long-short-term memory network, and output the hidden layer state vector; S3: Use the self-attention mechanism to perform weighted transformation on the hidden layer state vector, obtain deep semantic features, and obtain the final sentence vector to be classified; S4: Output the to-be-classified sentence vector to the Softmax classifier through a fully connected layer to obtain a text classification result of power transmission and transformation equipment defects. 2 . The method according to claim 1 , wherein the preprocessing includes word segmentation, stop word removal, and unified terminology processing on the power transmission and transformation equipment defect text. 3 . 3. The method according to claim 1, wherein in the step S1, the step of performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain the first word vector with power semantic features specifically comprises: Read the preprocessed power transmission and transformation equipment defect text, and count the word frequency information; Build a dictionary, initialize the Huffman tree and randomly initialize the word vector; Train the model in row units and get an input sample in the current row; Accumulate the value of each dimension in the context word vector and average it to get the projection layer vector; Traverse each intermediate node from the current word to the root node; Calculate the gradient g*learning rate corresponding to the intermediate node, refresh the error vector of the projection layer to the intermediate node, refresh the intermediate node vector, and refresh the context word vector. 4. The method according to claim 1, wherein the step S2 specifically comprises the following steps: Define the forward LSTM structure and the backward LSTM structure, use the dynamic RNN unit to splicing the results of the network output, and then input them to the next layer of bidirectional long and short-term memory network, and divide the output results of the last layer of Bi-LSTM into split methods. forward and backward output; Add the forward and backward outputs to get the final hidden layer state. 5. The method according to claim 4, wherein each time state in the LSTM structure is updated in the following manner:

in,

is the excitation function of the LSTM state and the internal state of the LSTM, set to the hyperbolic tangent function tanh, b is the bias constant, i, f, o in the subscript represent the input gate, forget gate and output gate respectively; g is updated with time steps the control gate unit,

is the current state at time t,

is the state of the previous time,

is the input at the current moment,

is the weight value,

is the input gate weight value,

is the output gate weight value,

is the forget gate weight value,

is the abstract information of the current moment,

is the abstracted information of the previous time step,

is the weight coefficient. 6. The method according to claim 1, wherein the deep semantic feature in the step S3 is obtained by the following methods:

; Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose. 7. A power transmission and transformation equipment defect text classification system based on deep learning is characterized in that, comprising: The text processing module is used to preprocess the acquired power transmission and transformation equipment defect text, and then perform word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with power semantic features; The semantic feature extraction module is connected to the text processing module, and is used to obtain the forward and backward feature information of the defect text of the power transmission and transformation equipment through the bidirectional long and short-term memory network, output the hidden layer state vector, and use it to use the self-attention mechanism. The hidden layer state vector is weighted and transformed to obtain deep semantic features, and the final sentence vector to be classified is obtained; The text classification module is used for inputting the received sentence vector to be classified into the fully connected layer and outputting it to the Softmax classifier, so as to obtain the text classification result of the defect of the power transmission and transformation equipment. 8 . The system according to claim 7 , wherein the preprocessing of the power transmission and transformation equipment defect text by the text processing module comprises word segmentation, stop word removal and unification on the power transmission and transformation equipment defect text. 9 . word processing. 9. The system of claim 7, wherein the deep semantic features are obtained by:

; Where Q=W ^q H, K= W ^k H, V=W ^v H, W ^q , W ^k , W ^v are initialization weight matrices, H is the hidden layer state matrix, Self-attention is the self-attention value, Q is the quary value, K is the key value, V is the value value,

is the word vector dimension, and T is the matrix transpose. 10. The system according to claim 7, wherein the semantic feature extraction module is also used to define a forward LSTM structure and a backward LSTM structure, and uses a dynamic RNN unit to splicing the results output by the network, and then input to The next layer of bidirectional long-short-term memory network divides the output of the last layer of Bi-LSTM into forward and backward outputs through the split method, and adds the forward and backward outputs to obtain the final hidden layer state.

Images