CN112699244A - Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment - Google Patents
Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment Download PDFInfo
- Publication number
- CN112699244A CN112699244A CN202110279537.6A CN202110279537A CN112699244A CN 112699244 A CN112699244 A CN 112699244A CN 202110279537 A CN202110279537 A CN 202110279537A CN 112699244 A CN112699244 A CN 112699244A
- Authority
- CN
- China
- Prior art keywords
- vector
- transformation equipment
- text
- power transmission
- word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 75
- 230000009466 transformation Effects 0.000 title claims abstract description 74
- 230000005540 biological transmission Effects 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 239000013598 vector Substances 0.000 claims abstract description 73
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 230000002457 bidirectional effect Effects 0.000 claims abstract description 13
- 230000007246 mechanism Effects 0.000 claims abstract description 8
- 230000002950 deficient Effects 0.000 claims abstract description 4
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000005284 excitation Effects 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000005065 mining Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 239000012212 insulator Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The invention provides a deep learning-based method and a deep learning-based system for classifying defect texts of power transmission and transformation equipment, wherein the method comprises the following steps: s1: preprocessing the acquired defective text of the power transmission and transformation equipment, and then embedding words to obtain a first word vector with electric power semantic features; s2: acquiring forward and backward characteristic information of a defect text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, and outputting a hidden layer state vector; s3: carrying out weighted transformation on the hidden layer state vector by using a self-attention mechanism to obtain deep semantic features and obtain a final sentence vector to be classified; s4: and outputting the vectors to be classified to a Softmax classifier through a full connection layer to obtain a classification result of the defect texts of the power transmission and transformation equipment. The method can solve the technical problems that the labor cost of the existing electric power field defect text classification is high, the classification result is easily influenced by the experience of classification technicians, and the traditional text classification method is not suitable for the electric power field.
Description
Technical Field
The invention belongs to the technical field of natural language processing, and particularly relates to a deep learning-based method and system for classifying a defect text of power transmission and transformation equipment.
Background
With the continuous development of the smart power grid, a large amount of defect text data can be generated in the daily operation and maintenance process of the power grid; and the analysis of the defect text data is always the original basis for processing and analyzing the defect fault of the power grid equipment. At present, the defect text analysis of the power grid power transmission and transformation equipment is mainly completed manually, the cost is high, the efficiency is low, and the classification result is prone to being influenced by manual experience difference, so that the classification result has deviation. The development of artificial intelligence and natural language processing technology provides possibility for text mining of defects of electric power equipment. The existing text classification technology has naive Bayes, a support vector machine, a decision tree and the like, but a traditional text classifier based on a machine learning related algorithm is difficult to dig out deep features of a text and is not beneficial to further analysis, research and application of text data, meanwhile, the text in the electric power field contains a large amount of professional terms and special symbols, the specialization is strong, a general classification model in deep learning is difficult to obtain direct migration and application, and the existing electric power text mining is still in a starting stage.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method for classifying a defect text of a power transmission and transformation device based on deep learning, which is applicable to classification of defect texts in the power field.
In order to achieve the purpose, the technical scheme of the invention is as follows: a deep learning-based power transmission and transformation equipment defect text classification method comprises the following steps:
s1: preprocessing the acquired power transmission and transformation equipment defect text, and then performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
s2: acquiring forward and backward characteristic information of a defect text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, and outputting a hidden layer state vector;
s3: carrying out weighted transformation on the hidden layer state vector by using a self-attention mechanism to obtain deep semantic features and obtain a final sentence vector to be classified;
s4: and outputting the sentence vectors to be classified to a Softmax classifier through a full connection layer to obtain a classification result of the defect texts of the power transmission and transformation equipment.
Further, the preprocessing comprises word segmentation, stop word removal and unified expression processing on the power transmission and transformation equipment defect text.
Further, in step S1, the step of performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with electric semantic features specifically includes:
reading the preprocessed power transmission and transformation equipment defect text, and counting word frequency information;
constructing a dictionary, and initializing a Huffman tree and a random initialization word vector;
training a model by using a row unit to obtain an input sample in a current row;
accumulating the value of each dimension in the context word vector and averaging to obtain a projection layer vector;
traversing each intermediate node from the current word to the root node;
calculating the corresponding gradient g of the intermediate node, refreshing the error vector from the projection layer to the intermediate node, refreshing the vector of the intermediate node, and refreshing the vector of the context word.
Further, the step S2 specifically includes the following steps:
defining a forward LSTM structure and a backward LSTM structure, splicing results output by a network by adopting a dynamic RNN unit, inputting the results into a next layer of bidirectional long-time memory network, and dividing the results output by the last layer of Bi-LSTM into forward and backward outputs by a split method;
the outputs in the forward and backward directions are added to obtain the final hidden layer state.
Further, each time state in the LSTM structure is updated by the following method:
wherein,
setting hyperbolic tangent function tanh as excitation functions of an LSTM state and an LSTM internal state, setting b as a bias constant, and respectively representing an input gate, a forgetting gate and an output gate by i, f and o in subscripts; g is a control gate unit updated along with time steps, namely a feedforward neural network taking a sigmiod function as an excitation function,
for the current state of time t to be,
the state of the previous time is the state of the previous time,
is an input for the current time of day,
is a weight value of the weight value,
in order to enter the gate weight value,
in order to output the weight value of the gate,
in order to forget the weight value of the door,
for the purpose of abstracting the information at the current moment,
is the abstracted information of the previous time step,
are weight coefficients.
Further, the deep semantic features in step S3 are obtained by:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
The invention also aims to provide a power transmission and transformation equipment defect text classification system based on deep learning, which can be applied to defect text classification in the electric power field.
In order to achieve the purpose, the technical scheme of the invention is as follows: a deep learning-based electric transmission and transformation equipment defect text classification system comprises:
the text processing module is used for preprocessing the acquired electric transmission and transformation equipment defect text and then performing word embedding on the preprocessed electric transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
the semantic feature extraction module is connected with the text processing module and used for acquiring forward and backward feature information of the defective text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, outputting a hidden layer state vector, performing weighted transformation on the hidden layer state vector by using a self-attention mechanism, acquiring deep semantic features and obtaining a final sentence vector to be classified;
and the text classification module is used for inputting the received sentence vector to be classified into the full connection layer and outputting the sentence vector to be classified into the Softmax classifier to obtain a classification result of the defect text of the power transmission and transformation equipment.
Further, the text processing module is used for preprocessing the electric transmission and transformation equipment defect text and comprises the steps of performing word segmentation, stop word removal and unified expression processing on the electric transmission and transformation equipment defect text.
Further, the deep semantic features are obtained by:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
Furthermore, the semantic feature extraction module is further configured to define a forward LSTM structure and a backward LSTM structure, splice results output by the network by using a dynamic RNN unit, input the results into a next layer of bidirectional long-and-short-term memory network, segment results output by the last layer of Bi-LSTM into forward and backward outputs by a split method, and add the forward and backward outputs to obtain a final hidden layer state.
Compared with the prior art, the invention has the following advantages:
1. the labor cost of actual production of the power grid is reduced, the fault defect text classification result is prevented from being influenced by experience of different personnel, the fault text classification efficiency of the power transmission and transformation equipment is greatly improved, and the fault text classification accuracy of the power transmission and transformation equipment is improved.
2. In the actual production of the power industry, the automatic classification of the defect texts of the power transmission and transformation equipment can provide objective and efficient fault division reference, and meanwhile, the comprehensive state estimation can be performed on the health state of the power transmission and transformation equipment by combining with structured data;
3. the classification of the defect texts of the power transmission and transformation equipment is part of research content of electric power text mining, and lays a foundation for mining and analyzing the defect text data of the electric power equipment and other electric power text data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive exercise.
FIG. 1 is a structural diagram of a deep learning-based electric transmission and transformation equipment defect text classification system according to the present invention;
FIG. 2 is a flowchart of a deep learning-based method for classifying defect texts of electric transmission and transformation equipment according to the present invention;
FIG. 3 is a diagram illustrating a 100-dimensional word vector after training according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The examples are given for the purpose of better illustration of the invention, but the invention is not limited to the examples. Therefore, those skilled in the art should make insubstantial modifications and adaptations to the embodiments of the present invention in light of the above teachings and remain within the scope of the invention.
Example 1
Referring to fig. 1, a diagram of a text classification system for defects of power transmission and transformation equipment based on deep learning according to the present invention is shown, and specifically, the system includes:
the text processing module 1 is used for preprocessing the acquired electric transmission and transformation equipment defect text and then performing word embedding on the preprocessed electric transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
in this embodiment, the preprocessing the electric transmission and transformation equipment defect text by the text processing module includes performing word segmentation, stop word removal and unified phrase processing on the electric transmission and transformation equipment defect text.
The semantic feature extraction module 2 is connected with the text processing module 1 and used for acquiring forward and backward feature information of the defective text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, outputting a hidden layer state vector, performing weighted transformation on the hidden layer state vector by using a self-attention mechanism, acquiring deep semantic features and obtaining a final sentence vector to be classified;
in the semantic feature extraction module 2, a forward LSTM structure and a backward LSTM structure are defined, a dynamic RNN unit is adopted to splice the results output by the network, then the results are input into a next layer of bidirectional long-time memory network, the results output by the last layer of Bi-LSTM are divided into forward and backward outputs by a split method, and the forward and backward outputs are added to obtain a final hidden layer state.
Preferably, the deep semantic features are obtained by:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
And the text classification module 3 is connected with the semantic feature extraction module 2 and is used for inputting the received sentence vectors to be classified into the full connection layer and outputting the sentence vectors to the Softmax classifier to obtain a classification result of the defect texts of the power transmission and transformation equipment.
Example 2
Based on the system of embodiment 1, the present embodiment provides a deep learning-based method for classifying a defect text of an electric transmission and transformation device, and referring to fig. 2, the method includes the following steps:
s1: preprocessing the acquired power transmission and transformation equipment defect text, and then performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
in the step, the preprocessing of the electric transmission and transformation equipment defect text comprises the steps of performing word segmentation, stop word removal and unified phrase processing on the electric transmission and transformation equipment defect text;
further, the step of performing word embedding on the preprocessed electric transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features specifically includes:
reading the preprocessed power transmission and transformation equipment defect text, and counting word frequency information;
constructing a dictionary, and initializing a Huffman tree and a random initialization word vector, wherein the dimension of the word vector is 100;
training a model by using a row unit to obtain an input sample in a current row;
accumulating the value of each dimension in the context word vector and averaging to obtain a projection layer vector;
traversing each intermediate node from the current word to the root node;
calculating the corresponding gradient g of the intermediate node, refreshing the error vector from the projection layer to the intermediate node, refreshing the vector of the intermediate node, and refreshing the vector of the context word.
In a specific embodiment, the defect record data of the power grid equipment of 2018 and 2019 of a certain power saving company is selected, and the defect record of the power transmission and transformation equipment is selected as the original data because the power transmission and transformation equipment is the equipment type with the largest defect record number, and the other equipment types are fewer and do not have the condition of forming a deep learning data set. Preprocessing the defect text of the power transmission and transformation equipment, including word segmentation, and removing stop words and unified expressions; adopting a 5-fold verification method, randomly arranging defect texts of all power transmission and transformation equipment, then averagely dividing the defect texts into 5 parts, selecting 4 parts as a training set in turn, and verifying 1 part as a test set, wherein one part of the test data set is assumed to be that the oil level of a main transformer body oil conservator of a No. 1 main transformer of a 35kv kali-slope transformer substation is too low, and the oil seal of a main transformer tap changer breather of a 35kv hot water power plant is too much; after the data set is preprocessed, the data set is respectively ' 35kv ', ' main transformer ', ' body ', ' oil storage cabinet ', ' oil level ', ' too low ', ' 35kv ', ' main transformer ', ' tap switch ', ' respirator ', ' oil seal ' and too much '.
For example, in one embodiment, assuming the word segmentation result of one of the test data sets is "insulator", the 100-dimensional word vector after training is shown in fig. 3;
s2: acquiring forward and backward characteristic information of a defect text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, and outputting a hidden layer state vector;
in the embodiment, a forward long-short time memory network (LSTM) structure and a backward long-short time memory network (LSTM) structure are defined, the results (fw and bw, fw is a forward LSTM output result, and bw is a backward LSTM output result) output by the network are spliced by using a dynamic RNN unit, then the results are input to the next layer of bidirectional long-short time memory network, and the result output by the last layer of Bi-LSTM is divided into forward and backward outputs by a split method (split function) in python;
the outputs in the forward and backward directions are added to obtain the final hidden layer state.
Further, each time (step) state in the LSTM structure is updated by the following method:
wherein,
setting hyperbolic tangent function tanh as excitation functions of a system state and an internal state, setting b as a bias constant, and respectively representing an input gate, a forgetting gate and an output gate by i, f and o in subscripts; g is a control gate unit updated along with time steps, namely a feedforward neural network taking a sigmiod function as an excitation function,
in the state of the present time t,
the layer state is hidden for the previous time,
is an input for the current time of day,
is a weight value of the weight value,
in order to enter the gate weight value,
in order to output the weight value of the gate,
in order to forget the weight value of the door,
for the purpose of abstracting the information at the current moment,
is the abstracted information of the previous time step,
are weight coefficients.
S3: carrying out weighted transformation on the hidden layer state vector by using a self-attention mechanism to obtain deep semantic features and obtain a final sentence vector to be classified;
in this step, the hidden layer state is introduced into a self-attention mechanism; and calculating attention weights assigned to the various inputs, and applying the weights obtained by calculation to the feature vector to obtain a final output feature vector. The specific calculation formula is as follows:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
S4: and (4) outputting the sentence vectors to be classified to a Softmax classifier through a full connection layer to obtain a classification result of the defect texts of the power transmission and transformation equipment.
In this embodiment, the sentence vectors to be classified are output to the softmax classifier through the full connection layer, so as to obtain the probability of the corresponding class, and the class with the highest probability is selected as the final class.
Preferably, in the present embodiment, the construction process of the softmax classifier is as follows: the method comprises the steps of obtaining a defect text of the power transmission and transformation equipment with a known defect type, preprocessing the data through a defect text preprocessing unit, obtaining word vectors with text features through a defect text feature representation unit, obtaining semantic feature sentence vectors to be classified through a defect text feature extraction unit, obtaining training data, training model parameters of a softmax classifier by adopting a random gradient descent method, finishing training when a loss function is minimized, and obtaining the softmax classifier.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A deep learning-based power transmission and transformation equipment defect text classification method is characterized by comprising the following steps:
s1: preprocessing the acquired power transmission and transformation equipment defect text, and then performing word embedding on the preprocessed power transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
s2: acquiring forward and backward characteristic information of a defect text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, and outputting a hidden layer state vector;
s3: carrying out weighted transformation on the hidden layer state vector by using a self-attention mechanism to obtain deep semantic features and obtain a final sentence vector to be classified;
s4: and outputting the sentence vectors to be classified to a Softmax classifier through a full connection layer to obtain a classification result of the defect texts of the power transmission and transformation equipment.
2. The method of claim 1, wherein the pre-processing comprises performing word segmentation, stop word removal, and unionization wording on the electric transmission and transformation equipment defect text.
3. The method according to claim 1, wherein the step of performing word embedding on the preprocessed transmission and transformation equipment defect text in step S1 to obtain the first word vector with the electric power semantic features specifically comprises:
reading the preprocessed power transmission and transformation equipment defect text, and counting word frequency information;
constructing a dictionary, and initializing a Huffman tree and a random initialization word vector;
training a model by using a row unit to obtain an input sample in a current row;
accumulating the value of each dimension in the context word vector and averaging to obtain a projection layer vector;
traversing each intermediate node from the current word to the root node;
calculating the corresponding gradient g of the intermediate node, refreshing the error vector from the projection layer to the intermediate node, refreshing the vector of the intermediate node, and refreshing the vector of the context word.
4. The method according to claim 1, wherein the step S2 specifically comprises the steps of:
defining a forward LSTM structure and a backward LSTM structure, splicing results output by a network by adopting a dynamic RNN unit, inputting the results into a next layer of bidirectional long-time memory network, and dividing the results output by the last layer of Bi-LSTM into forward and backward outputs by a split method;
the outputs in the forward and backward directions are added to obtain the final hidden layer state.
5. The method of claim 4, wherein each time state in the LSTM structure is updated by:
wherein,
setting hyperbolic tangent function tanh as excitation functions of an LSTM state and an LSTM internal state, setting b as a bias constant, and respectively representing an input gate, a forgetting gate and an output gate by i, f and o in subscripts; g is a control gate unit updated with time steps,
in the state of the current time t,
the state of the previous time is the state of the previous time,
is an input for the current time of day,
is a weight value of the weight value,
in order to enter the gate weight value,
in order to output the weight value of the gate,
in order to forget the weight value of the door,
for the purpose of abstracting the information at the current moment,
is the abstracted information of the previous time step,
are weight coefficients.
6. The method according to claim 1, wherein the deep semantic features in step S3 are obtained by:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
7. The utility model provides a power transmission and transformation equipment defect text classification system based on deep learning which characterized in that includes:
the text processing module is used for preprocessing the acquired electric transmission and transformation equipment defect text and then performing word embedding on the preprocessed electric transmission and transformation equipment defect text to obtain a first word vector with electric power semantic features;
the semantic feature extraction module is connected with the text processing module and used for acquiring forward and backward feature information of the defective text of the power transmission and transformation equipment through a bidirectional long-time and short-time memory network, outputting a hidden layer state vector, performing weighted transformation on the hidden layer state vector by using a self-attention mechanism, acquiring deep semantic features and obtaining a final sentence vector to be classified;
and the text classification module is used for inputting the received sentence vector to be classified into the full connection layer and outputting the sentence vector to be classified into the Softmax classifier to obtain a classification result of the defect text of the power transmission and transformation equipment.
8. The system of claim 7, wherein the text processing module pre-processes the electrical transmission and transformation equipment defect text including word segmentation, stop word removal, and unionization.
9. The system of claim 7, wherein the deep semantic features are obtained by:
wherein Q = WqH，K= WkH，V=WvH，Wq、Wk、WvAll are initialized weight matrix, H is hidden layer state matrix, Self-attention value, Q is square value, K is key value, V is value,
is the word vector dimension and T is the matrix transpose.
10. The system of claim 7, wherein the semantic feature extraction module is further configured to define a forward LSTM structure and a backward LSTM structure, splice the results output by the network using a dynamic RNN unit, input the results into a next layer of bidirectional long-and-short-term memory network, divide the results output by the last layer of Bi-LSTM into forward and backward outputs by a split method, and add the forward and backward outputs to obtain a final hidden layer state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110279537.6A CN112699244A (en) | 2021-03-16 | 2021-03-16 | Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110279537.6A CN112699244A (en) | 2021-03-16 | 2021-03-16 | Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112699244A true CN112699244A (en) | 2021-04-23 |
Family
ID=75515268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110279537.6A Pending CN112699244A (en) | 2021-03-16 | 2021-03-16 | Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699244A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935338A (en) * | 2021-10-13 | 2022-01-14 | 北京有竹居网络技术有限公司 | Method, apparatus, device and medium for language translation based on layer prediction |
| CN114328797A (en) * | 2021-11-09 | 2022-04-12 | 腾讯科技（深圳）有限公司 | Content search method, content search device, electronic device, storage medium, and program product |
| CN115563528A (en) * | 2022-11-25 | 2023-01-03 | 四川飞宇电力股份有限公司 | Transformer maintenance equipment control method, electronic equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108363753A (en) * | 2018-01-30 | 2018-08-03 | 南京邮电大学 | Comment text sentiment classification model is trained and sensibility classification method, device and equipment |
| CN110135551A (en) * | 2019-05-15 | 2019-08-16 | 西南交通大学 | A kind of robot chat method of word-based vector sum Recognition with Recurrent Neural Network |
| CN110457562A (en) * | 2019-08-15 | 2019-11-15 | 中国农业大学 | A kind of food safety affair classification method and device based on neural network model |
| US20200380213A1 (en) * | 2018-02-09 | 2020-12-03 | Salesforce.Com, Inc. | Multitask Learning As Question Answering |
-
2021
- 2021-03-16 CN CN202110279537.6A patent/CN112699244A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108363753A (en) * | 2018-01-30 | 2018-08-03 | 南京邮电大学 | Comment text sentiment classification model is trained and sensibility classification method, device and equipment |
| US20200380213A1 (en) * | 2018-02-09 | 2020-12-03 | Salesforce.Com, Inc. | Multitask Learning As Question Answering |
| CN110135551A (en) * | 2019-05-15 | 2019-08-16 | 西南交通大学 | A kind of robot chat method of word-based vector sum Recognition with Recurrent Neural Network |
| CN110457562A (en) * | 2019-08-15 | 2019-11-15 | 中国农业大学 | A kind of food safety affair classification method and device based on neural network model |
Non-Patent Citations (5)
| Title |
|---|
| 冯斌 等: "基于BiLSTM-Attention神经网络的电力设备缺陷文本挖掘", 《中国电机工程学报》 * |
| 廖伟涵: "基于机器学习与自然语言处理的变压器故障诊断与状态评价研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 张振: "基于车辆通行大数据的高速公路路况预测的研究与应用", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 杨鹏: "基于领域词典与机器学习的中文评论情感分析", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 类库: ""双向LSTM模型的tensorflow实现"", 《HTTPS://WWW.136.LA/SHIDA/SHOW-97712.HTML》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935338A (en) * | 2021-10-13 | 2022-01-14 | 北京有竹居网络技术有限公司 | Method, apparatus, device and medium for language translation based on layer prediction |
| CN114328797A (en) * | 2021-11-09 | 2022-04-12 | 腾讯科技（深圳）有限公司 | Content search method, content search device, electronic device, storage medium, and program product |
| CN114328797B (en) * | 2021-11-09 | 2024-03-19 | 腾讯科技（深圳）有限公司 | Content search method, device, electronic apparatus, storage medium, and program product |
| CN115563528A (en) * | 2022-11-25 | 2023-01-03 | 四川飞宇电力股份有限公司 | Transformer maintenance equipment control method, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112699244A (en) | Deep learning-based method and system for classifying defect texts of power transmission and transformation equipment | |
| WO2018218705A1 (en) | Method for recognizing network text named entity based on neural network probability disambiguation | |
| CN112905804B (en) | Dynamic updating method and device for power grid dispatching knowledge graph | |
| CN109933792B (en) | Viewpoint type problem reading and understanding method based on multilayer bidirectional LSTM and verification model | |
| CN111767397A (en) | Electric power system secondary equipment fault short text data classification method | |
| CN112395417A (en) | Network public opinion evolution simulation method and system based on deep learning | |
| CN114661905B (en) | BERT-based power grid fault diagnosis method | |
| CN112837175B (en) | Frequent power failure work order information extraction method and system based on information extraction technology | |
| CN114926150A (en) | Digital intelligent auditing method and device for transformer technology conformance assessment | |
| CN112419096A (en) | Automatic user power demand worksheet transferring method based on NLP information extraction and few-sample self-learning | |
| CN111274814A (en) | Novel semi-supervised text entity information extraction method | |
| CN116842337A (en) | Transformer fault diagnosis method based on LightGBM (gallium nitride based) optimal characteristics and COA-CNN (chip on board) model | |
| CN113723083A (en) | Weighted negative supervision text emotion analysis method based on BERT model | |
| CN112559741B (en) | Nuclear power equipment defect record text classification method, system, medium and electronic equipment | |
| CN115292490A (en) | Analysis algorithm for policy interpretation semantics | |
| CN113051886B (en) | Test question duplicate checking method, device, storage medium and equipment | |
| CN117932295A (en) | Multi-source data fusion power grid monitoring operation characteristic information extraction method and system | |
| Lin et al. | The study on classification and prediction for data mining | |
| CN117033961A (en) | Multi-mode image-text classification method for context awareness | |
| CN116029295A (en) | Electric power text entity extraction method, defect positioning method and fault diagnosis method | |
| CN114936667A (en) | Power grid day-ahead scheduling plan optimization decision method based on knowledge graph | |
| Qu et al. | Improvement of power equipment defect text quality based on improved BI-LSTM | |
| CN114153975A (en) | Text classification method and device, electronic equipment and storage medium | |
| CN114818651A (en) | Text similarity determination method and device, storage medium and electronic device | |
| Tan et al. | Application of self-supervised learning in non-intrusive load monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210423 |
|
| RJ01 | Rejection of invention patent application after publication |