CN111767398A - Secondary equipment fault short text data classification method based on convolutional neural network - Google Patents
Secondary equipment fault short text data classification method based on convolutional neural network Download PDFInfo
- Publication number
- CN111767398A CN111767398A CN202010610878.2A CN202010610878A CN111767398A CN 111767398 A CN111767398 A CN 111767398A CN 202010610878 A CN202010610878 A CN 202010610878A CN 111767398 A CN111767398 A CN 111767398A
- Authority
- CN
- China
- Prior art keywords
- layer
- data
- secondary equipment
- vector
- short text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 29
- 239000013598 vector Substances 0.000 claims abstract description 67
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000013507 mapping Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 238000012360 testing method Methods 0.000 claims abstract description 6
- 238000012216 screening Methods 0.000 claims abstract description 5
- 238000012795 verification Methods 0.000 claims abstract description 4
- 239000011159 matrix material Substances 0.000 claims description 26
- 238000011176 pooling Methods 0.000 claims description 25
- 230000007547 defect Effects 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 10
- 208000037408 Device failure Diseases 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 7
- 238000013145 classification model Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 4
- 238000005065 mining Methods 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims description 2
- 238000003058 natural language processing Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000011089 mechanical engineering Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/237—Lexical tools
- G06F40/242—Dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Business, Economics & Management (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Economics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for classifying secondary equipment fault short text data based on a convolutional neural network, which comprises the steps of firstly, acquiring text information data of secondary equipment faults, determining a data set and obtaining a training sample set; preprocessing the short text data of the secondary equipment fault, and mapping word data and word vectors one by one to obtain text vector data; then training the convolutional neural network model by using the training sample to obtain a trained convolutional neural network model; and then, according to the verification set data, inputting test input data into the trained convolutional neural network model, wherein the output value of the model is the classification result of the secondary equipment fault short text information to be classified. The secondary equipment fault short text data classification method based on the convolutional neural network utilizes the good characteristic screening and analyzing capability of the convolutional neural network model, improves the prediction precision and strengthens the generalization capability of the model.
Description
Technical Field
The invention belongs to a Chinese natural language processing technology, and particularly relates to a secondary equipment fault short text data classification method based on a convolutional neural network.
Background
In the process of construction and operation of an intelligent power grid, the occurrence of electric power big data explosions can be roughly divided into two types according to Chinese electric power big data development white papers compiled by the information special committee of the Chinese electro-mechanical engineering society in 2013, wherein one type is structured data represented by output power, equipment and environment temperature and humidity thereof, light intensity of an optical module and the like in a time sequence mode, and the other type is semi-structured and non-structured data represented by texts, images, audios and the like and difficult to express by using a relational database. Structured data mining work is mature, however, the problem of low data value density brought by the fact that normal data are always generated in the running process of a power grid and only a very small amount of fault data exist restricts the mining of unstructured data. The short text fault information occurring in the operation process of the secondary equipment is also the information which is focused in the construction process of the internet of things.
In the operation process of the secondary equipment, a plurality of fault defect short text data are accumulated, the data are often manually recorded by a transportation and inspection person to finish the classification work of the defect levels, the accurate classification is difficult to achieve due to the fact that the subjectivity and the experience of the transportation and inspection person are different, and the fault data are more, a large amount of manpower is needed to participate, and the efficiency is difficult to guarantee. With the development of Chinese text classification technology, automatic classification of a large amount of fault short text information in a power grid production management system becomes possible by means of machine learning.
At present, for short text classification, foreign countries already research emotion classification on hotel evaluation through natural language processing, but the natural language processing of English is difficult to use in Chinese due to different Chinese and English structures, such as characters of capitalization of proper nouns in English, space connection between words and the like, and is difficult to use in Chinese due to different industries, and a lot of proper nouns exist in the field of secondary equipment text classification, so that the research on the improvement of classification models is lacked, and most of the researches are based on the field of traditional machine learning. And because the reason that personnel of recording record by hand, the spoken language is recorded more, the text message is shorter, there is no method disclosed to the short text message classification of the secondary equipment at present.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a method for classifying secondary equipment fault short text data based on a convolutional neural network, and aims to solve the problem that fault short text information recorded by secondary equipment is insufficient in identification and classification in the operation process of a smart grid.
The technical scheme is as follows: a secondary equipment fault short text data classification method based on a convolutional neural network comprises the following steps:
(1) establishing a sample data set, collecting fault short text data generated by secondary equipment in an electric power system in the operation process, marking the fault short text data according to different defect grades, and dividing the fault short text data into a training set, a verification set and a test set;
(2) text preprocessing, namely constructing a stop word dictionary and a secondary equipment special dictionary, wherein the stop word dictionary is used for filtering and removing noise in secondary equipment fault short text information, and the noise comprises words and punctuations which have no actual physical significance in secondary equipment fault short text data; the secondary equipment special dictionary is used for identifying attribute data of the secondary equipment, and the attribute data comprises the name, the model, the station name and the route name of the secondary equipment;
(3) word vector modeling, namely establishing a word2vec model for the preprocessed text data, acquiring sample data set secondary equipment fault short text data, performing word vector training, mining context semantic relations among words, and establishing a word and word vector mapping relation;
(4) constructing a multi-size convolutional neural network secondary equipment fault short text data classification model, taking the word vectors trained in the step (3) as an input matrix of the convolutional neural network model, carrying out convolutional operation on the secondary equipment fault short text data matrix according to different word quantity combination modes by adopting a multi-size convolutional kernel to obtain a plurality of phrase sequences, outputting the phrase sequences to a pooling layer, carrying out maximum pooling extraction on features through the pooling layer, reducing feature dimensions, and screening the phrase sequences with the highest weight values;
(5) predicting a classification result, setting a layer of full-connection layer after the structures of the convolution layer and the pooling layer, performing feature extraction and combination on vectors output by different pooling layers, and outputting the combined vectors to a subsequent SoftMax layer for classification and judgment; and obtaining the probability belonging to each fault grade type, obtaining the maximum probability in the probabilities, and outputting the text type corresponding to the maximum probability as the type of the text to be classified.
Further, the text data of the fault short messages in the step (1) are divided into three types of fault defects of 'serious defect', 'critical defect' and 'general defect' according to the evaluation guide of the relay protection state of the national power grid company, corresponding fault feature labels are arranged on the fault short messages, and the fault short messages are randomly combined according to the following steps of 7: 2: 1 into a training set, a validation set and a test set.
Further, the text preprocessing in the step (2) includes filtering noise, extracting stem features, restoring part of speech, and recognizing entity nouns and proper nouns which appear in the short text data of the secondary equipment failure; the method comprises the steps of constructing a stop word dictionary and a secondary equipment special dictionary by performing feature sequencing on an original language database text, traversing a language database according to dictionary contents, filtering data noise and extracting data features.
Further, the construction process of the Word2vec Word vector model in the step (3) is as follows:
(31) expressing the Chinese text into a structured word vector according to the context relationship between words, and adopting WordEmbedding to construct vector mapping so that the semantic information of the word context is not lost by the word vector generated after mapping, and the dimension of the text vector is reduced;
(32) constructing three-layer structures which are an input layer, a hidden layer and an output layer respectively, entering discrete numerical values obtained by the input layer into a linear unit of the hidden layer for training, and finally using SoftMax regression on the output layer;
(33) two neural network model structures, namely a CBOW structure and a Skip-gram structure, are constructed according to the step (31) and the step (32), and are specifically as follows:
(A) CBOW structure model
The CBOW structure model predicts the final output result through the context relation of words in an input layer, supposing that the dimension of the words is V, the context of the input context is C, the size of a dictionary is D, inputting One-hot coding data into a hidden layer structure, and multiplying all discrete data by an input weight matrix WV×NN is the number of the set hidden layer structures, and the obtained vector is multiplied by an output weight matrix W'V×NObtaining the probability distribution with one-dimensional vector, and taking the maximum probability as the predicted intermediate words;
continuously selecting words, training an input weight matrix and an output weight matrix by adopting a gradient descent algorithm, wherein the smaller the error is, the better the error is;
the conditional probability of CBOW for a target word is calculated as follows:
the specific maximization formula of the objective function of CBOW is as follows:
∑(ω,c)∈DlogP(ω,c)
(B) skip-gram structure model
The Skip-gram structure model constructs words of an input layer to make predicted word vectors form a context form, and an objective function of the Skip-gram model is as follows:
further, the convolutional neural network secondary device failure short text information classification model in step (4) includes an input layer, a convolutional layer, a pooling layer, and an output layer, and specifically includes the following steps:
the first layer is an input layer, selects text data to be classified, completes vectorization of the text data according to the step (3), and outputs a matrix I ∈ Rm×nM is the number of words of the text, i.e., the number of rows of the input layer, n is the number of dimensions of the text vector, i.e., the number of columns of the input layer, and each word data is divided into word vectors of equal dimensions according to the division in step (3) so that the number of columns of the input layer is equal, thereby forming a matrix I ∈ Rm×nIn the training process, the word vector is adjusted by a random gradient descent method;
selecting convolution kernels of different sizes, wherein each convolution kernel has multiple convolution kernels, and inputting matrix I ∈ R to input layer respectivelym×nPerforming convolution operation to extract the matrix characteristics of the input layer to obtain a convolution result vector ri(i ═ 1,2,3,4, 5, 6, L), the formula is as follows:
ri＝W·Ii:i+h-1
wherein the matrix W represents a weight coefficient, "·" represents a dot product operation;
activating the convolution result through an activation function ReLU, and performing nonlinear processing to obtain a result ciThe formula is as follows:
ci＝ReLU(ri+b)
c is toiThe convolutional layer vector c ∈ R is obtained from left to right, in the order from top to bottoms-h+1The formula is as follows:
c＝[c1,c2,K,cs-h+1]
the third layer is a pooling layer, which is pooled by maximum pooling method according to the result vector c ∈ R extracted from the convolutional layers -h+1The element with the largest value in the sequence is extracted as the characteristic value pj(j is 1,2,3,4, 5, 6, L, n) and all the characteristic values pjSequentially spliced into a vector p ∈ Rn×1Inputting the data into a fourth output layer, wherein a vector p represents a vector of global features of the text data;
the fourth layer is an output layer: fully connecting the pooling layer with the output layer, taking a vector p of the pooling layer as input, classifying the vector p by adopting a Softmax classifier, and outputting a final classification result; the probability of Softmax classification calculation is as follows:
wherein the function L (p)j) And expressing the probability of belonging to the secondary equipment category, selecting the result with the maximum probability, and outputting the defect level of the secondary equipment fault.
Has the advantages that: compared with the prior art, the method has the advantages that the stop word dictionary and the special dictionary in the secondary equipment field are constructed, and the text convolution neural network has good feature screening capability on the short text, so that the feature model structure is simplified, and the classification precision and generalization capability are improved. And a secondary equipment failure short text classification model based on deep learning is established, the prediction precision is obviously superior to that of the traditional neural network model, and the accuracy of secondary equipment failure short text classification is greatly improved.
Drawings
FIG. 1 is a flow chart of an embodiment of the method of the present invention.
Detailed Description
For the purpose of illustrating the technical solutions disclosed in the present invention in detail, the following description is further provided with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, the method for classifying short text data of secondary device failure based on convolutional neural network provided by the present invention includes the following steps:
s1: determining a data set;
collecting short text data of faults generated in the operation process of secondary equipment, dividing the fault types into 'serious defects', 'critical defects' and 'general defects' according to the requirements of relevant guide rules, and dividing a text data set into: training set, verification set and test set.
S2: preprocessing a text;
constructing a stop word dictionary, filtering and removing noise in the short text information of the secondary equipment fault, and keeping words with specific practical meanings such as nouns, verbs, quantifiers and the like; and constructing a secondary equipment professional dictionary, and identifying low-frequency words such as equipment names and equipment models and special nouns such as plant stations and lines where the equipment is located.
S3: constructing a word vector;
word vector modeling, namely establishing a word2vec model for the preprocessed text data, acquiring sample data set secondary equipment fault short text data, performing word vector training, mining context semantic relations among words, and establishing a word and word vector mapping relation;
s4: constructing a convolutional neural network;
setting four layers of convolution neural network
(1) The first layer is an input layer;
selecting text data to be classified, completing vectorization of the text data according to the step (S3), and outputting a matrix I ∈ Rm×nM is the number of words of the text, i.e., the number of rows of the input layer, n is the number of dimensions of the text vector, i.e., the number of columns of the input layer, and each word data is divided into word vectors of equal dimensions according to the division in step (3) so that the number of columns of the input layer is equal, thereby forming a matrix I ∈ Rm ×nIn the training process, the word vector is adjusted by a random gradient descent method;
(2) the second layer is a convolution layer;
selecting convolution kernels of different sizes, wherein each convolution kernel of different size has a plurality of matrixes I ∈ R respectively input to input layersm×nPerforming convolution operation to extract the matrix characteristics of the input layer to obtain a convolution result vector ri(i ═ 1,2,3,4, 5, 6, L), the formula is as follows:
ri＝W·Ii:i+h-1
wherein the matrix W represents a weight coefficient, "·" represents a dot product operation;
activating the convolution result through an activation function ReLU, and performing nonlinear processing to obtain a result ciThe formula is as follows:
ci＝ReLU(ri+b)
c is toiThe convolutional layer vector c ∈ R is obtained from left to right, in the order from top to bottoms-h+1The formula is as follows:
c＝[c1,c2,K,cs-h+1]
(3) the third layer is a pooling layer;
pooling by maximum pooling based on the result vector c ∈ R extracted from the convolutional layers-h+1The element with the largest value in the sequence is extracted as the characteristic value pj(j is 1,2,3,4, 5, 6, L, n) and all the characteristic values pjSequentially spliced into a vector p ∈ Rn×1Inputting the data into a fourth output layer, wherein a vector p represents a vector of global features of the text data;
(4) the fourth layer is an output layer;
and fully connecting the pooling layer with the output layer, taking the vector p of the pooling layer as input, classifying the vector p by adopting a Softmax classifier, and outputting a final classification result. The probability of Softmax classification calculation is as follows:
wherein the function L (p)j) Representing the probability of belonging to a secondary device class.
The following description will take the classification of the secondary device failure short text message as an example.
2000 data of a Xinjiang power grid relay protection action statistical table from 2015 to 2019 are analyzed, and 1471 data of repeated invalid data with too large noise are removed. According to the following steps: 2: 1, dividing a data set, and performing word segmentation on each piece of data, wherein the word segmentation is performed on short text data, such as '220 kv/river park line/b/sleeve/protection/fiber channel/fault'. And training the text convolutional neural network, and bringing the training result into the trained convolutional neural network model, wherein the classification result is correct.
Three criteria are generally used to evaluate classification performance: accuracy, recall, F1 value:
the prediction results are shown in the following table, and as can be seen from table 1, the method of the invention has a good effect on short text information of secondary equipment failure. In conclusion, the invention can realize the classification of the faults and can be used for practical engineering application.
TABLE 1 text convolution neural model classification result evaluation index
The method provided by the invention eliminates the subjectivity of the traditional classification method for classifying the fault data depending on the experience of operation and maintenance maintainers to a certain extent. And a large amount of fault short text data generated in a production management system is operated around secondary equipment, and related training and analysis of automatic text classification based on a convolutional neural network are carried out. The secondary equipment fault short text data classification method based on the convolutional neural network utilizes the good characteristic screening and analyzing capability of the convolutional neural network model, improves the prediction precision and strengthens the generalization capability of the model.
Claims (5)
1. A secondary equipment fault short text data classification method based on a convolutional neural network is characterized by comprising the following steps: the method comprises the following steps:
(1) establishing a sample data set, collecting fault short text data generated by secondary equipment in an electric power system in the operation process, marking the fault short text data according to different defect grades, and dividing the fault short text data into a training set, a verification set and a test set;
(2) text preprocessing, namely constructing a stop word dictionary and a secondary equipment special dictionary, wherein the stop word dictionary is used for filtering and removing noise in secondary equipment fault short text information, and the noise comprises words and punctuations which have no actual physical significance in secondary equipment fault short text data; the secondary equipment special dictionary is used for identifying attribute data of the secondary equipment, and the attribute data comprises the name, the model, the station name and the route name of the secondary equipment;
(3) word vector modeling, namely establishing a word2vec model for the preprocessed text data, acquiring sample data set secondary equipment fault short text data, performing word vector training, mining context semantic relations among words, and establishing a word and word vector mapping relation;
(4) constructing a multi-size convolutional neural network secondary equipment fault short text data classification model, taking the word vectors trained in the step (3) as an input matrix of the convolutional neural network model, carrying out convolutional operation on the secondary equipment fault short text data matrix according to different word quantity combination modes by adopting a multi-size convolutional kernel to obtain a plurality of phrase sequences, outputting the phrase sequences to a pooling layer, carrying out maximum pooling extraction on features through the pooling layer, reducing feature dimensions, and screening the phrase sequences with the highest weight values;
(5) predicting a classification result, setting a layer of full-connection layer after the structures of the convolution layer and the pooling layer, performing feature extraction and combination on vectors output by different pooling layers, and outputting the combined vectors to a subsequent SoftMax layer for classification and judgment; and obtaining the probability belonging to each fault grade type, obtaining the maximum probability in the probabilities, and outputting the text type corresponding to the maximum probability as the type of the text to be classified.
2. The convolutional neural network-based secondary device failure short text data classification method according to claim 1, characterized in that: dividing the fault short text data in the step (1) into three fault defect characteristic labels of 'serious defect', 'critical defect' and 'general defect' according to the relay protection state evaluation guide rule of the national power grid company, and randomly combining the fault short text data according to the following steps of 7: 2: 1 into a training set, a validation set and a test set.
3. The convolutional neural network-based secondary device failure short text data classification method according to claim 1, characterized in that: the text preprocessing in the step (2) comprises noise filtering, word stem feature extraction, part of speech reduction, entity noun and proper noun identification of short text data of secondary equipment faults; the method comprises the steps of constructing a stop word dictionary and a secondary equipment special dictionary by performing feature sequencing on an original language database text, traversing a language database according to dictionary contents, filtering data noise and extracting data features.
4. The convolutional neural network-based secondary device failure short text data classification method according to claim 1, characterized in that: the construction process of the Word2vec Word vector model in the step (3) is as follows:
(31) expressing the Chinese text into a structured word vector according to the context relationship between words, and adopting WordEmbedding to construct vector mapping so that the semantic information of the word context is not lost by the word vector generated after mapping, and the dimension of the text vector is reduced;
(32) constructing three-layer structures which are an input layer, a hidden layer and an output layer respectively, entering discrete numerical values obtained by the input layer into a linear unit of the hidden layer for training, and finally using SoftMax regression on the output layer;
(33) two neural network model structures, namely a CBOW structure and a Skip-gram structure, are constructed according to the step (31) and the step (32), and are specifically as follows:
(A) CBOW structure model
The CBOW structure model predicts the final output result through the context relation of words in an input layer, supposing that the dimension of the words is V, the context of the input context is C, the size of a dictionary is D, inputting One-hot coding data into a hidden layer structure, and multiplying all discrete data by an input weight matrix WV×NN is the number of the set hidden layer structures, and the obtained vector is multiplied by an output weight matrix W'V×NObtaining the probability distribution with one-dimensional vector, and taking the maximum probability as the predicted intermediate words; continuously selecting words, and training an input weight matrix and an output weight matrix by adopting a gradient descent algorithm so as to reduce errors;
the conditional probability of CBOW for a target word is calculated as follows:
the specific maximization formula of the objective function of CBOW is as follows:
∑(ω,c)∈DlogP(ω,c)
(B) skip-gram structure model
The Skip-gram structure model constructs words of an input layer to make predicted word vectors form a context form, and an objective function of the Skip-gram model is as follows:
5. the convolutional neural network-based secondary device failure short text data classification method according to claim 1, characterized in that: the convolutional neural network secondary equipment fault short text information classification model in the step (4) comprises an input layer, a convolutional layer, a pooling layer and an output layer, and specifically comprises the following steps:
the first layer is an input layer: selecting text data to be classified, and finishing the text according to the step (3)Vectorization of data, output matrix I ∈ Rm×nM is the number of words of the text, i.e., the number of rows of the input layer, n is the number of dimensions of the text vector, i.e., the number of columns of the input layer, and each word data is divided into word vectors of equal dimensions according to the division in step (3) so that the number of columns of the input layer is equal, thereby forming a matrix I ∈ Rm×nIn the training process, the word vector is adjusted by a random gradient descent method;
selecting convolution kernels of different sizes, wherein each convolution kernel has multiple convolution kernels, and inputting matrix I ∈ R to input layer respectivelym×nPerforming convolution operation to extract the matrix characteristics of the input layer to obtain a convolution result vector ri(i ═ 1,2,3,4, 5, 6, L), the formula is as follows:
ri＝W·Ii:i+h-1
wherein the matrix W represents a weight coefficient, "·" represents a dot product operation;
activating the convolution result through an activation function ReLU, and performing nonlinear processing to obtain a result ciThe formula is as follows:
ci＝ReLU(ri+b)
c is toiThe convolutional layer vector c ∈ R is obtained from left to right, in the order from top to bottoms-h+1The formula is as follows:
c＝[c1,c2,K,cs-h+1]
the third layer is a pooling layer, which is pooled by maximum pooling method according to the result vector c ∈ R extracted from the convolutional layers-h+1The element with the largest value in the sequence is extracted as the characteristic value pj(j is 1,2,3,4, 5, 6, L, n) and all the characteristic values pjSequentially spliced into a vector p ∈ Rn×1Inputting the data into a fourth output layer, wherein a vector p represents a vector of global features of the text data;
the fourth layer is an output layer: fully connecting the pooling layer with the output layer, taking a vector p of the pooling layer as input, classifying the vector p by adopting a Softmax classifier, and outputting a final classification result; the probability of Softmax classification calculation is as follows:
wherein the function L (p)j) And expressing the probability of belonging to the secondary equipment category, selecting the result with the maximum probability, and outputting the defect level of the secondary equipment fault.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010610878.2A CN111767398A (en) | 2020-06-30 | 2020-06-30 | Secondary equipment fault short text data classification method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010610878.2A CN111767398A (en) | 2020-06-30 | 2020-06-30 | Secondary equipment fault short text data classification method based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111767398A true CN111767398A (en) | 2020-10-13 |
Family
ID=72724137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010610878.2A Pending CN111767398A (en) | 2020-06-30 | 2020-06-30 | Secondary equipment fault short text data classification method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767398A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182249A (en) * | 2020-10-23 | 2021-01-05 | 四川大学 | Automatic classification method and device for aviation safety report |
| CN112465052A (en) * | 2020-12-07 | 2021-03-09 | 重庆忽米网络科技有限公司 | Equipment fault diagnosis report generation method and system based on convolutional neural network |
| CN113111183A (en) * | 2021-04-20 | 2021-07-13 | 通号（长沙）轨道交通控制技术有限公司 | Traction power supply equipment defect grade classification method |
| CN113434667A (en) * | 2021-04-20 | 2021-09-24 | 国网浙江省电力有限公司杭州供电公司 | Text classification method based on distribution network automation terminal text classification model |
| CN114638558A (en) * | 2022-05-19 | 2022-06-17 | 天津市普迅电力信息技术有限公司 | Data set classification method for operation accident analysis of comprehensive energy system |
| CN116738323A (en) * | 2023-08-08 | 2023-09-12 | 北京全路通信信号研究设计院集团有限公司 | Fault diagnosis method, device, equipment and medium for railway signal equipment |
| CN117270482A (en) * | 2023-11-22 | 2023-12-22 | 博世汽车部件(苏州)有限公司 | Automobile factory control system based on digital twin |
| CN117495338A (en) * | 2023-09-30 | 2024-02-02 | 国网江苏省电力有限公司信息通信分公司 | System fault diagnosis and repair method based on automatic operation and maintenance |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107885883A (en) * | 2017-12-01 | 2018-04-06 | 北京国信宏数科技有限公司 | A kind of macroeconomy field sentiment analysis method and system based on Social Media |
| CN108399230A (en) * | 2018-02-13 | 2018-08-14 | 上海大学 | A kind of Chinese financial and economic news file classification method based on convolutional neural networks |
| CN108536801A (en) * | 2018-04-03 | 2018-09-14 | 中国民航大学 | A kind of civil aviaton's microblogging security public sentiment sentiment analysis method based on deep learning |
| CN108596470A (en) * | 2018-04-19 | 2018-09-28 | 浙江大学 | A kind of power equipments defect text handling method based on TensorFlow frames |
| CN109977224A (en) * | 2019-03-11 | 2019-07-05 | 南京邮电大学 | The news category method combined based on convolutional neural networks and term vector |
| CN110134786A (en) * | 2019-05-14 | 2019-08-16 | 南京大学 | A kind of short text classification method based on theme term vector and convolutional neural networks |
| CN110232188A (en) * | 2019-06-04 | 2019-09-13 | 上海电力学院 | The Automatic document classification method of power grid user troublshooting work order |
-
2020
- 2020-06-30 CN CN202010610878.2A patent/CN111767398A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107885883A (en) * | 2017-12-01 | 2018-04-06 | 北京国信宏数科技有限公司 | A kind of macroeconomy field sentiment analysis method and system based on Social Media |
| CN108399230A (en) * | 2018-02-13 | 2018-08-14 | 上海大学 | A kind of Chinese financial and economic news file classification method based on convolutional neural networks |
| CN108536801A (en) * | 2018-04-03 | 2018-09-14 | 中国民航大学 | A kind of civil aviaton's microblogging security public sentiment sentiment analysis method based on deep learning |
| CN108596470A (en) * | 2018-04-19 | 2018-09-28 | 浙江大学 | A kind of power equipments defect text handling method based on TensorFlow frames |
| CN109977224A (en) * | 2019-03-11 | 2019-07-05 | 南京邮电大学 | The news category method combined based on convolutional neural networks and term vector |
| CN110134786A (en) * | 2019-05-14 | 2019-08-16 | 南京大学 | A kind of short text classification method based on theme term vector and convolutional neural networks |
| CN110232188A (en) * | 2019-06-04 | 2019-09-13 | 上海电力学院 | The Automatic document classification method of power grid user troublshooting work order |
Non-Patent Citations (2)
| Title |
|---|
| JANG B 等: "Word2vec convolutional neural networks for classification of news articles and tweets", 《PLOS ONE》 * |
| 黄鑫 等: "卷积神经网络在机械设备故障诊断领域应用与挑战", 《制造技术与机床》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182249A (en) * | 2020-10-23 | 2021-01-05 | 四川大学 | Automatic classification method and device for aviation safety report |
| CN112182249B (en) * | 2020-10-23 | 2022-12-13 | 四川大学 | Automatic classification method and device for aviation safety report |
| CN112465052A (en) * | 2020-12-07 | 2021-03-09 | 重庆忽米网络科技有限公司 | Equipment fault diagnosis report generation method and system based on convolutional neural network |
| CN112465052B (en) * | 2020-12-07 | 2023-04-07 | 重庆忽米网络科技有限公司 | Equipment fault diagnosis report generation method and system based on convolutional neural network |
| CN113111183A (en) * | 2021-04-20 | 2021-07-13 | 通号（长沙）轨道交通控制技术有限公司 | Traction power supply equipment defect grade classification method |
| CN113434667A (en) * | 2021-04-20 | 2021-09-24 | 国网浙江省电力有限公司杭州供电公司 | Text classification method based on distribution network automation terminal text classification model |
| CN113434667B (en) * | 2021-04-20 | 2024-01-23 | 国网浙江省电力有限公司杭州供电公司 | Text classification method based on distribution network automation terminal text classification model |
| CN114638558A (en) * | 2022-05-19 | 2022-06-17 | 天津市普迅电力信息技术有限公司 | Data set classification method for operation accident analysis of comprehensive energy system |
| CN116738323A (en) * | 2023-08-08 | 2023-09-12 | 北京全路通信信号研究设计院集团有限公司 | Fault diagnosis method, device, equipment and medium for railway signal equipment |
| CN116738323B (en) * | 2023-08-08 | 2023-10-27 | 北京全路通信信号研究设计院集团有限公司 | Fault diagnosis method, device, equipment and medium for railway signal equipment |
| CN117495338A (en) * | 2023-09-30 | 2024-02-02 | 国网江苏省电力有限公司信息通信分公司 | System fault diagnosis and repair method based on automatic operation and maintenance |
| CN117270482A (en) * | 2023-11-22 | 2023-12-22 | 博世汽车部件(苏州)有限公司 | Automobile factory control system based on digital twin |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111767398A (en) | Secondary equipment fault short text data classification method based on convolutional neural network | |
| CN110597735B (en) | Software defect prediction method for open-source software defect feature deep learning | |
| CN111767397A (en) | Electric power system secondary equipment fault short text data classification method | |
| CN111966812B (en) | Automatic question answering method based on dynamic word vector and storage medium | |
| CN108549658A (en) | A kind of deep learning video answering method and system based on the upper attention mechanism of syntactic analysis tree | |
| CN110895565A (en) | Method and system for classifying fault defect texts of power equipment | |
| CN111709244B (en) | Deep learning method for identifying cause and effect relationship of contradictory dispute | |
| CN111126057B (en) | Case scenario accurate sentencing system of hierarchical neural network | |
| CN108052504A (en) | Mathematics subjective item answers the structure analysis method and system of result | |
| CN113065356B (en) | IT equipment operation and maintenance fault suggestion processing method based on semantic analysis algorithm | |
| CN112836809A (en) | Device characteristic extraction method and fault prediction method of convolutional neural network based on differential feature fusion | |
| CN112883286A (en) | BERT-based method, equipment and medium for analyzing microblog emotion of new coronary pneumonia epidemic situation | |
| CN112199496A (en) | Power grid equipment defect text classification method based on multi-head attention mechanism and RCNN (Rich coupled neural network) | |
| CN109740164A (en) | Based on the matched electric power defect rank recognition methods of deep semantic | |
| CN113420548A (en) | Entity extraction sampling method based on knowledge distillation and PU learning | |
| CN114036993A (en) | Multi-modal ironic object detection method based on multi-scale trans-modal neural network | |
| CN113204967A (en) | Resume named entity identification method and system | |
| CN109063983A (en) | A kind of natural calamity loss real time evaluating method based on social media data | |
| CN113378024B (en) | Deep learning-oriented public inspection method field-based related event identification method | |
| CN113868422A (en) | Multi-label inspection work order problem traceability identification method and device | |
| CN112559741B (en) | Nuclear power equipment defect record text classification method, system, medium and electronic equipment | |
| CN113111183A (en) | Traction power supply equipment defect grade classification method | |
| CN116029295A (en) | Electric power text entity extraction method, defect positioning method and fault diagnosis method | |
| CN114020901A (en) | Financial public opinion analysis method combining topic mining and emotion analysis | |
| CN114547294A (en) | Rumor detection method and system based on comprehensive information of propagation process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201013 |
|
| RJ01 | Rejection of invention patent application after publication |