CN113642862A - Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model - Google Patents

Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model Download PDF

Info

Publication number
CN113642862A
CN113642862A CN202110864643.0A CN202110864643A CN113642862A CN 113642862 A CN113642862 A CN 113642862A CN 202110864643 A CN202110864643 A CN 202110864643A CN 113642862 A CN113642862 A CN 113642862A
Authority
CN
China
Prior art keywords
power grid
mbigru
grid dispatching
model
dispatching instruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110864643.0A
Other languages
Chinese (zh)
Inventor
杨梓俊
荆江平
孙昕杰
张刘冬
吴海洋
王黎明
杨明
申张亮
邓晨
赵帅
蒋雪冬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Jiangsu Electric Power Co Ltd
Original Assignee
State Grid Jiangsu Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Jiangsu Electric Power Co Ltd filed Critical State Grid Jiangsu Electric Power Co Ltd
Priority to CN202110864643.0A priority Critical patent/CN113642862A/en
Publication of CN113642862A publication Critical patent/CN113642862A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Human Resources & Organizations (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Strategic Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Tourism & Hospitality (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • General Business, Economics & Management (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Game Theory and Decision Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Public Health (AREA)
  • Water Supply & Treatment (AREA)
  • Primary Health Care (AREA)
  • Machine Translation (AREA)

Abstract

The invention discloses a method and a system for identifying a named entity of a power grid dispatching instruction based on a BERT-MBIGRU-CRF model, wherein the system comprises the following steps: the system comprises a sample preprocessing module, a BERT pre-training module, an MBIGRU feature extraction module, a CRF training module and an output recognition result module. The method comprises the following steps: preprocessing a power grid dispatching instruction before training, building a BERT-MBIGRU-CRF model, training and modeling a historical power grid dispatching instruction training set by adopting the model, generating a neural network model with power grid characteristics, predicting a prediction sample by a completely trained named entity recognition system, and finally obtaining a named entity recognition result. The invention utilizes a multilayer bidirectional neural network Model (MBIGRU) to deeply depict important characteristics of the scheduling instruction, extracts context information in a bidirectional way and effectively improves the accuracy of named entity identification of the power grid scheduling instruction.

Description

Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model
Technical Field
The invention relates to the field of power grid dispatching instruction identification, in particular to a power grid dispatching instruction named entity identification method and system based on a BERT-MBIGRU-CRF model.
Background
Electric power systems are increasingly intelligent, and in order to be matched with an intelligent control system introduced into a power grid, scheduling instructions of scheduling personnel are intelligently identified in the daily scheduling process of a distribution network, and the correctness, the normalization and the scheduling instructions of the command receiving and sending process between the scheduling personnel and field personnel are ensured to follow the scheduling safety standard. The acceleration of power grid intellectualization drives artificial intelligence to replace manual work, and the identification of dispatching instruction receiving and sending contents in the power grid dispatching process is very important.
At present, many language recognition methods in natural language processing exist, such as Hidden Markov Models (HMMs), Conditional Random Fields (CRFs), long and short term memory neural networks (LSTM), and the like, and the named entity recognition methods have the defects that contexts cannot be labeled or data features need to be manually extracted, and cannot accurately recognize data sets. With the continuous acceleration of the intelligent speed of the power grid, the increase of the data volume and the fluctuation, randomness and the like of the data make the traditional semantic identification method more and more unable to meet the requirements of practical application.
Disclosure of Invention
In order to solve the defects in the prior art, the invention aims to provide a method and a system for identifying a named entity of a power grid dispatching instruction based on a BERT-MBIGRU-CRF model.
In order to realize the purpose of the invention, the technical scheme adopted by the invention is as follows:
a method for identifying a named entity of a power grid dispatching instruction based on a BERT-MBIGRU-CRF model comprises the following steps:
(1) preprocessing the power grid dispatching instruction before training to obtain a labeled power grid dispatching instruction named entity recognition training set;
(2) building a BERT-MBIGRU-CRF power grid dispatching instruction identification model and completing training;
(3) and identifying the current power grid dispatching instruction by using the well-trained BERT-MBIGRU-CRF power grid dispatching instruction identification model, and outputting an identification result.
Further, the step (1) specifically includes:
(1.1) eliminating punctuation marks and special characters in a power grid dispatching instruction sample, correcting wrongly written or mispronounced characters, and normalizing;
and (1.2) marking each single-sentence power grid dispatching instruction by adopting a named entity marking method.
Further, the step (2) specifically includes:
(2.1) pre-training by using a BERT model to obtain a power grid dispatching instruction vector;
(2.2) extracting features by using the MBIGRU model to generate feature vectors;
and (2.3) marking constraint on the extracted feature vectors by using a CRF model, calculating a loss function, and stopping iteration when a loss value reaches a threshold value to finish training.
Further, in the step (2.1),
the BERT model comprises an embedding layer, a bidirectional Transformer encoder and a pooling layer; and respectively capturing word-level and sentence-level expressions through a masking language model and a next sentence prediction task, and performing combined training.
Further, the step (2.1) specifically includes:
(2.1.1) converting the preprocessed power grid dispatching instruction named entity recognition training set into word vectors, text vectors and position vectors which are used as BERT model input, and entering an embedding layer;
(2.1.2) the embedding layer divides the power grid dispatching instruction into distributed expression vectors; in the embedding process, a [ CLS ] is used for marking a first mark on the power grid dispatching instruction, and an [ SEP ] is used for marking a second mark on the power grid dispatching instruction;
(2.1.3) carrying out nonlinear representation on each power grid dispatching instruction by using a bidirectional transformer coder, and bidirectionally connecting context information by using a position coding memory word vector sequence;
and (2.1.4) outputting a power grid dispatching instruction vector and an integral sequence representation by the BERT model through a pooling layer.
Further, in the step (2.2), the MBIGRU model includes two gating structures, namely, an update gate and a reset gate, and the reset gate and the update gate together determine the output of the hidden state.
Further, the updated gate in the MBIGRU controls the weight of the previous time information to be substituted into the current time, and the updated gate state value ztThe formula is as follows:
zt＝f(Wzxt+Uzht-1)
where f denotes the sigmod function, xtRepresenting the input vector at time t, ht-1Represents the hidden layer state at time t-1, WzRepresenting an updated gate weight matrix, UzRepresenting an updated gate bias matrix;
reset gate in MBIGRU controls whether previous time information was discarded or entered into a candidate state
Figure BDA0003187115210000021
Reset gate state value rtThe formula is as follows:
rt＝f(Wrxt+Urht-1)
wherein, WrRepresenting a reset gate weight matrix, UrRepresenting a reset gate bias matrix;
the output value of MBIGRU is formed by the series value of the previous time step and the current time step and rtMultiplication, output state htThe formula is as follows:
Figure BDA0003187115210000022
Figure BDA0003187115210000023
where g represents the activation function and the output state value depends on the update gate state and the reset gate state.
Further, the MBIGRU model stacks forward and reverse GRU models on a single structure, whose model structure is represented as follows:
ht＝f(Uxt+Wht-1)
ht＝f(U′xt+W′h′t-1)
ot＝g(Vht+V′h′t)
wherein f and g represent activation functions, U, W, V represents weights in forward operation, U ', W ' and V ' represent weights in reverse operation, and otDenotes an output value, h'tRepresenting the hidden layer state at time t in the reverse state.
Further, the MBIGRU model is superposed on the forward and reverse GRU models for multiple times, and the model structure is expressed as follows:
Figure BDA0003187115210000031
Figure BDA0003187115210000032
……
Figure BDA0003187115210000033
Figure BDA0003187115210000034
Figure BDA0003187115210000035
wherein, OtAn output value representing a multi-layer forward and backward result,
Figure BDA0003187115210000036
indicating the hidden layer state at the instant of the i-th layer reverse state t.
A power grid dispatching instruction named entity recognition system based on a BERT-MBIGRU-CRF model comprises a sample preprocessing module, a BERT pre-training module, an MBIGRU feature extraction module, a CRF training module and an output recognition result module;
the sample preprocessing module is used for preprocessing the power grid dispatching instruction before training to obtain a labeled power grid dispatching instruction named entity recognition training set;
the BERT pre-training module is used for inputting a preprocessed power grid dispatching instruction named entity recognition training set as a BERT model, respectively capturing word-level and sentence-level expressions through a masking language model and a next sentence prediction task, performing combined training and outputting a power grid dispatching instruction vector;
the MBIGRU feature extraction module is used for extracting the features of the power grid dispatching instruction vector which is completely trained by using the MBIGRU model to generate a feature vector;
the CRF training module is used for labeling and constraining the extracted feature vectors by using a CRF model, calculating a loss function, and stopping iteration when a loss value reaches a threshold value to finish training;
and the output recognition result module is used for recognizing the current power grid dispatching instruction by using the well-trained BERT-MBIGRU-CRF model and outputting a recognition result.
The invention has the advantages that compared with the prior art,
compared with most of traditional natural language processing models, the BERT model can acquire more language intrinsic information more accurately, and the accuracy of named entity recognition is improved.
The deep learning model based on the MBGRU deepens the dependency of context information through a multilayer bidirectional structure, deeply describes the internal characteristics of a language, and can better extract the characteristics of a power grid dispatching instruction.
The method for recognizing the named entity based on the BERT-MBGRU-CRF starts from a power grid dispatching instruction data set, trains a neural network model meeting power grid dispatching specifications, and greatly improves the accuracy of dispatching instruction recognition when forecasting sentences.
Drawings
FIG. 1 is a flow chart of a method for identifying a named entity of a power grid dispatching instruction based on a BERT-MBIGRU-CRF model according to an embodiment of the invention;
FIG. 2 is a block diagram of a BERT model according to an embodiment of the present invention;
FIG. 3 is a block diagram of a BERT-MBIGRU-CRF model according to an embodiment of the present invention;
fig. 4 is a diagram of the MBGRU model structure according to the embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present application is not limited thereby.
The invention relates to a power grid dispatching instruction named entity recognition system based on a BERT-MBIGRU-CRF model, which comprises a sample preprocessing module, a BERT pre-training module, an MBIGRU feature extraction module, a CRF training module and an output recognition result module.
And the sample preprocessing module is used for preprocessing the large-scale original power grid dispatching instruction before training to obtain the marked power grid dispatching instruction named entity recognition training set.
And the BERT pre-training module is used for converting the preprocessed power grid scheduling instruction named entity recognition training set into word vectors, text vectors and position vectors which are used as input of a BERT model, respectively capturing word-level and sentence-level expressions through two tasks of a masked language model and a next sense prediction by using the BERT model, and performing combined training. The BERT model outputs vector and integral sequence representation after the dispatching instruction is fused corresponding to the words in the language.
And the MBIGRU feature extraction module is used for extracting the data features of the data set which is completely trained by using the MBIGRU neural network to generate feature vectors.
The CRF training module takes the feature vectors output by the neural network as input, and the CRF layer marks and constrains the word vectors after the neural network features are extracted, so that reasonable dependency among the word vectors is increased; a loss function is calculated and when the loss value reaches a certain level, the iteration is stopped.
And the output recognition result module is used for recognizing the current scheduling instruction by utilizing the well-trained BERT-MBIGRU-CRF neural network model and outputting a recognition result.
As shown in fig. 1, the method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model of the invention comprises the following steps:
(1) preprocessing a large-scale original power grid dispatching instruction before training to obtain a labeled power grid dispatching instruction named entity recognition training set;
the pretreatment specifically comprises the steps of:
(1.1) eliminating punctuation marks and special characters in the sample acquisition instruction, correcting wrongly written characters and standardizing;
and (1.2) marking by adopting a named entity marking method, namely a BIOSE marking method, wherein the power grid dispatching instruction is generally a single sentence, and marking can be carried out on each sentence of the single sentence.
(2) Building a BERT-MBIGRU-CRF power grid dispatching instruction identification model, and completing training, as shown in FIG. 3;
(2.1) pre-training by using a BERT model to obtain a power grid dispatching instruction vector;
as shown in FIG. 2, the BERT model includes an embedding layer, a bi-directional Transformer encoder, and a pooling layer. Word-level and sentence-level representations are captured and jointly trained by two tasks, a masked language model and a next sense prediction.
And converting the preprocessed power grid dispatching instruction named entity recognition training set into a word vector, a text vector and a position vector to be used as BERT model input, and entering an embedding layer. The embedded layer divides the power grid dispatching instruction into distributed expression vectors; during embedding, the [ CLS ] is used for marking the scheduling instruction firstly, and the [ SEP ] is used for marking the scheduling instruction secondly. The bidirectional transformer encoder performs nonlinear representation on each sentence scheduling instruction, and uses position coding memory word vector sequence to bidirectionally connect context information. And after passing through the pooling layer, the BERT model outputs vector and integral sequence representation after the dispatching instruction is fused and corresponding to the words in the language.
(2.2) extracting features by using the MBIGRU model to generate feature vectors;
and inputting the power grid dispatching instruction vector output by the BERT model into the MBIGRU model, and extracting the training set data characteristics with complete training by using the multi-layer bidirectional cyclic neural network MBIGRU to generate the characteristic vector.
(2.3) completing training by using a CRF model;
inputting the feature vectors output by the MBIGRU model into a CRF model, wherein the CRF model labels and constrains the feature vectors after the neural network features are extracted, and reasonable dependence among the feature vectors is increased; a loss function is calculated and when the loss value reaches a certain level, the iteration is stopped.
(3) And identifying the current power grid dispatching instruction by using the well-trained BERT-MBIGRU-CRF neural network model, and outputting an identification result.
As shown in fig. 4, the MBIGRU neural network model is improved on the gating structure of LSTM, and the model structure only includes two gating structures, namely, an update gate and a reset gate, which together determine the output of the hidden state.
The update gate in the MBIGRU controls the weight of the previous time information brought into the current time, and updates the state value z of the gatetThe formula is as follows:
zt＝f(Wzxt+Uzht-1)
where f denotes the sigmod function, xtRepresenting the input vector at time t, ht-1Represents the hidden layer state at time t-1, WzRepresenting an updated gate weight matrix, UzIndicating that the gate bias matrix is updated.
Reset gate in MBIGRU controls whether previous time information was discarded or entered into a candidate state
Figure BDA0003187115210000061
Reset gate state value rtThe formula is as follows:
rt＝f(Wrxt+Urht-1)
wherein, WrRepresenting a reset gate weight matrix, UrRepresenting a reset gate bias matrix.
The output value of MBIGRU is formed by the series value of the previous time step and the current time step and rtMultiplication, output state ht：
Figure BDA0003187115210000062
Figure BDA0003187115210000063
Where g represents the activation function and the output state value depends on the update gate state and the reset gate state.
The MBIGRU model structure stacks forward and reverse GRU models on a single GRU structure, whose model structure is represented as follows:
ht＝f(Uxt+Wht-1)
ht′＝f(U′xt+W′h′t-1)
ot＝g(Vht+V′h′t)
wherein f and g represent activation functions, U, W, V represents weights in forward operation, U ', W ' and V ' represent weights in reverse operation, and otDenotes an output value, h'tRepresenting the hidden layer state at time t in the reverse state.
The MBIGRU model structure is superposed for many times on the basis of forward and reverse GRU models, and the model structure is expressed as follows:
Figure BDA0003187115210000064
Figure BDA0003187115210000065
……
Figure BDA0003187115210000066
Figure BDA0003187115210000067
Figure BDA0003187115210000068
wherein, OtAn output value representing a multi-layer forward and backward result,
Figure BDA0003187115210000069
indicating the hidden layer state at the instant of the i-th layer reverse state t.
The invention has the advantages that compared with the prior art,
compared with most of traditional natural language processing models, the BERT model can acquire more language intrinsic information more accurately, and the accuracy of named entity recognition is improved.
The deep learning model based on the MBGRU deepens the dependency of context information through a multilayer bidirectional structure, deeply describes the internal characteristics of a language, and can better extract the characteristics of a power grid dispatching instruction.
The method for recognizing the named entity based on the BERT-MBGRU-CRF starts from a power grid dispatching instruction data set, trains a neural network model meeting power grid dispatching specifications, and greatly improves the accuracy of dispatching instruction recognition when forecasting sentences.
The present applicant has described and illustrated embodiments of the present invention in detail with reference to the accompanying drawings, but it should be understood by those skilled in the art that the above embodiments are merely preferred embodiments of the present invention, and the detailed description is only for the purpose of helping the reader to better understand the spirit of the present invention, and not for limiting the scope of the present invention, and on the contrary, any improvement or modification made based on the spirit of the present invention should fall within the scope of the present invention.

Claims (10)

1. A method for identifying a named entity of a power grid dispatching instruction based on a BERT-MBIGRU-CRF model is characterized by comprising the following steps:
(1) preprocessing the power grid dispatching instruction before training to obtain a labeled power grid dispatching instruction named entity recognition training set;
(2) building a BERT-MBIGRU-CRF power grid dispatching instruction identification model and completing training;
(3) and identifying the current power grid dispatching instruction by using the well-trained BERT-MBIGRU-CRF power grid dispatching instruction identification model, and outputting an identification result.
2. The method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model as claimed in claim 1, wherein the step (1) specifically comprises:
(1.1) eliminating punctuation marks and special characters in a power grid dispatching instruction sample, correcting wrongly written or mispronounced characters, and normalizing;
and (1.2) marking each single-sentence power grid dispatching instruction by adopting a named entity marking method.
3. The method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model as claimed in claim 1, wherein the step (2) specifically comprises:
(2.1) pre-training by using a BERT model to obtain a power grid dispatching instruction vector;
(2.2) extracting features by using the MBIGRU model to generate feature vectors;
and (2.3) marking constraint on the extracted feature vectors by using a CRF model, calculating a loss function, and stopping iteration when a loss value reaches a threshold value to finish training.
4. The BERT-MBIGRU-CRF model-based power grid dispatching instruction named entity identification method of claim 3, wherein in the step (2.1),
the BERT model comprises an embedding layer, a bidirectional Transformer encoder and a pooling layer; and respectively capturing word-level and sentence-level expressions through a masking language model and a next sentence prediction task, and performing combined training.
5. The method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model as claimed in claim 4, wherein the step (2.1) specifically comprises the following steps:
(2.1.1) converting the preprocessed power grid dispatching instruction named entity recognition training set into word vectors, text vectors and position vectors which are used as BERT model input, and entering an embedding layer;
(2.1.2) the embedding layer divides the power grid dispatching instruction into distributed expression vectors; in the embedding process, a [ CLS ] is used for marking a first mark on the power grid dispatching instruction, and an [ SEP ] is used for marking a second mark on the power grid dispatching instruction;
(2.1.3) carrying out nonlinear representation on each power grid dispatching instruction by using a bidirectional transformer coder, and bidirectionally connecting context information by using a position coding memory word vector sequence;
and (2.1.4) outputting a power grid dispatching instruction vector and an integral sequence representation by the BERT model through a pooling layer.
6. The method according to claim 3, wherein in step (2.2), the MBIGRU model comprises two gating structures, namely an update gate and a reset gate, and the reset gate and the update gate together determine the output of the hidden state.
7. The BERT-MBIGRU-CRF model-based power grid dispatching instruction named entity identification method of claim 6, wherein,
the update gate in the MBIGRU controls the weight of the previous time information brought into the current time, and updates the state value z of the gatetThe formula is as follows:
zt＝f(Wzxt+Uzht-1)
where f denotes the sigmod function, xtRepresenting the input vector at time t, ht-1Represents the hidden layer state at time t-1, WzRepresenting an updated gate weight matrix, UzRepresenting an updated gate bias matrix;
reset gate in MBIGRU controls whether previous time information was discarded or entered into a candidate state
Figure FDA0003187115200000023
Reset gate state value rtThe formula is as follows:
rt＝f(Wrxt+Urht-1)
wherein, WrRepresenting a reset gate weight matrix, UrRepresenting a reset gate bias matrix;
the output value of MBIGRU is formed by the series value of the previous time step and the current time step and rtMultiplication, output state htThe formula is as follows:
Figure FDA0003187115200000021
Figure FDA0003187115200000022
where g represents the activation function and the output state value depends on the update gate state and the reset gate state.
8. The method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model as claimed in claim 7, wherein the MBIGRU model is a multi-layer stack of forward and reverse GRU models on a single structure, and the model structure is represented as follows:
ht＝f(Uxt+Wht-1)
h′t＝f(U′xt+W′h′t-1)
ot＝g(Vht+V′h′t)
wherein f and g represent activation functions, U, W, V represents weights in forward operation, U ', W ' and V ' represent weights in reverse operation, and otDenotes an output value, h'tRepresenting the hidden layer state at time t in the reverse state.
9. The method for identifying the named entity of the power grid dispatching instruction based on the BERT-MBIGRU-CRF model as claimed in claim 8, wherein the MBIGRU model is superimposed many times on the basis of the forward and reverse GRU models, and the model structure is represented as follows:
Figure FDA0003187115200000031
wherein, OtAn output value representing a multi-layer forward and backward result,
Figure FDA0003187115200000032
indicating the i-th layer reversalHidden layer state at time t.
10. A power grid dispatching instruction named entity recognition system based on a BERT-MBIGRU-CRF model is characterized by comprising a sample preprocessing module, a BERT pre-training module, an MBIGRU feature extraction module, a CRF training module and an output recognition result module;
the sample preprocessing module is used for preprocessing the power grid dispatching instruction before training to obtain a labeled power grid dispatching instruction named entity recognition training set;
the BERT pre-training module is used for inputting a preprocessed power grid dispatching instruction named entity recognition training set as a BERT model, respectively capturing word-level and sentence-level expressions through a masking language model and a next sentence prediction task, performing combined training and outputting a power grid dispatching instruction vector;
the MBIGRU feature extraction module is used for extracting the features of the power grid dispatching instruction vector which is completely trained by using the MBIGRU model to generate a feature vector;
the CRF training module is used for labeling and constraining the extracted feature vectors by using a CRF model, calculating a loss function, and stopping iteration when a loss value reaches a threshold value to finish training;
and the output recognition result module is used for recognizing the current power grid dispatching instruction by using the well-trained BERT-MBIGRU-CRF model and outputting a recognition result.
CN202110864643.0A 2021-07-29 2021-07-29 Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model Pending CN113642862A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110864643.0A CN113642862A (en) 2021-07-29 2021-07-29 Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110864643.0A CN113642862A (en) 2021-07-29 2021-07-29 Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model

Publications (1)

Publication Number Publication Date
CN113642862A true CN113642862A (en) 2021-11-12

Family

ID=78418987

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110864643.0A Pending CN113642862A (en) 2021-07-29 2021-07-29 Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model

Country Status (1)

Country Link
CN (1) CN113642862A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115099606A (en) * 2022-06-21 2022-09-23 厦门亿力吉奥信息科技有限公司 Training method and terminal for power grid dispatching model
CN117012185A (en) * 2023-06-20 2023-11-07 国网山东省电力公司泗水县供电公司 Power grid dispatching method and system based on knowledge graph
CN115099606B (en) * 2022-06-21 2024-06-07 厦门亿力吉奥信息科技有限公司 Training method and terminal of power grid dispatching model

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110083831A (en) * 2019-04-16 2019-08-02 武汉大学 A kind of Chinese name entity recognition method based on BERT-BiGRU-CRF
CN112115238A (en) * 2020-10-29 2020-12-22 电子科技大学 Question-answering method and system based on BERT and knowledge base
CN112733541A (en) * 2021-01-06 2021-04-30 重庆邮电大学 Named entity identification method of BERT-BiGRU-IDCNN-CRF based on attention mechanism

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110083831A (en) * 2019-04-16 2019-08-02 武汉大学 A kind of Chinese name entity recognition method based on BERT-BiGRU-CRF
CN112115238A (en) * 2020-10-29 2020-12-22 电子科技大学 Question-answering method and system based on BERT and knowledge base
CN112733541A (en) * 2021-01-06 2021-04-30 重庆邮电大学 Named entity identification method of BERT-BiGRU-IDCNN-CRF based on attention mechanism

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115099606A (en) * 2022-06-21 2022-09-23 厦门亿力吉奥信息科技有限公司 Training method and terminal for power grid dispatching model
CN115099606B (en) * 2022-06-21 2024-06-07 厦门亿力吉奥信息科技有限公司 Training method and terminal of power grid dispatching model
CN117012185A (en) * 2023-06-20 2023-11-07 国网山东省电力公司泗水县供电公司 Power grid dispatching method and system based on knowledge graph

Similar Documents

Publication Publication Date Title
CN110609891B (en) Visual dialog generation method based on context awareness graph neural network
CN110083831B (en) Chinese named entity identification method based on BERT-BiGRU-CRF
CN107729309B (en) Deep learning-based Chinese semantic analysis method and device
CN110232114A (en) Sentence intension recognizing method, device and computer readable storage medium
CN111897908A (en) Event extraction method and system fusing dependency information and pre-training language model
CN110647612A (en) Visual conversation generation method based on double-visual attention network
CN111339750B (en) Spoken language text processing method for removing stop words and predicting sentence boundaries
CN107797987B (en) Bi-LSTM-CNN-based mixed corpus named entity identification method
CN109918681B (en) Chinese character-pinyin-based fusion problem semantic matching method
CN112183064B (en) Text emotion reason recognition system based on multi-task joint learning
CN110909736A (en) Image description method based on long-short term memory model and target detection algorithm
CN112101044B (en) Intention identification method and device and electronic equipment
CN115292463B (en) Information extraction-based method for joint multi-intention detection and overlapping slot filling
CN111274804A (en) Case information extraction method based on named entity recognition
CN114239574A (en) Miner violation knowledge extraction method based on entity and relationship joint learning
CN111597342B (en) Multitasking intention classification method, device, equipment and storage medium
CN112182191A (en) Structured memory map network model for multi-round-mouth linguistic understanding
CN114443813B (en) Intelligent on-line teaching resource knowledge point concept entity linking method
CN112037773A (en) N-optimal spoken language semantic recognition method and device and electronic equipment
CN113223509A (en) Fuzzy statement identification method and system applied to multi-person mixed scene
CN107797988A (en) A kind of mixing language material name entity recognition method based on Bi LSTM
CN111597816A (en) Self-attention named entity recognition method, device, equipment and storage medium
CN113642862A (en) Method and system for identifying named entities of power grid dispatching instructions based on BERT-MBIGRU-CRF model
CN116484848B (en) Text entity identification method based on NLP
CN112257432A (en) Self-adaptive intention identification method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20211112