CN113312500B - Method for constructing event map for safe operation of dam - Google Patents
Method for constructing event map for safe operation of dam Download PDFInfo
- Publication number
- CN113312500B CN113312500B CN202110702542.3A CN202110702542A CN113312500B CN 113312500 B CN113312500 B CN 113312500B CN 202110702542 A CN202110702542 A CN 202110702542A CN 113312500 B CN113312500 B CN 113312500B
- Authority
- CN
- China
- Prior art keywords
- event
- vector
- attention
- sentence
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 17
- 239000013598 vector Substances 0.000 claims abstract description 107
- 238000002372 labelling Methods 0.000 claims abstract description 23
- 230000004927 fusion Effects 0.000 claims abstract description 17
- 238000010276 construction Methods 0.000 claims abstract description 15
- 230000001364 causal effect Effects 0.000 claims abstract description 9
- 239000011159 matrix material Substances 0.000 claims description 48
- 230000006870 function Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 10
- 239000004576 sand Substances 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 239000000047 product Substances 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 239000013604 expression vector Substances 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 5
- 239000013589 supplement Substances 0.000 claims description 5
- 230000001502 supplementing effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 239000000945 filler Substances 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a construction method of an event map for safe operation of a dam, which comprises the following steps: (1) embedding enhanced semantic information using word vectors; (2) local attention is introduced, global attention captures keywords and document information is mined; (3) solving the sample imbalance problem and extracting event types by using Focalloss; (4) adding event type codes to form a new sentence code in a serial connection manner, and processing an embedded vector after the serial connection by using BilSTM; (5) converting an attention network and an attention network according to a proportional fusion graph, extracting effective characteristics, and capturing corresponding event arguments; (6) filling the network with arguments to fill in the document-level arguments missing in the event; (7) and converting the attention network and the attention network labeling sequence by the weighted fusion graph, acquiring the causal relationship between the events and constructing an event graph.
Description
Technical Field
The invention relates to an event map construction method for dam safe operation, which extracts dam operation condition events and event arguments thereof in text data through an event detection and extraction model based on double attention to form an event knowledge map. The method aims to automatically extract events and arguments thereof from a large number of dam operation record texts and generate a dam event knowledge graph.
Background
The concept of knowledge graph was proposed by Google in 2012, and was first used by search engines for entity-based searches instead of string-based searches, thereby improving the user search quality and experience. In the big data age, the knowledge graph expresses the information of the internet in a structured form to a form closer to the human cognitive world, and provides the capability of better organizing, managing and understanding the mass information of the internet.
In the big data era, manual labor cannot meet the construction requirement of the knowledge graph. Many enterprises begin to actively explore and try automatic construction technologies, and extract data from different sources and structures by using machines to form knowledge which is stored in a knowledge graph. In industrial practice, knowledge graph construction through extracting knowledge from unstructured data such as text information and the like faces many technical challenges.
Event relation extraction is compared with entity relation extraction, the relation between two events needs to be judged, the description of the events in the text is usually complex and may be a sentence or a plurality of sentences, and therefore the difficulty of establishing a knowledge graph with the events as the center is larger than that of establishing the entity knowledge graph.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems in the prior art, the invention provides an event map construction method facing dam safe operation, which is used for constructing a map with dam safe operation events as a center, and the map with the dam safe operation events as the center can better reflect the cause and time sequence relation between the events, thereby improving the capability of dam managers for dealing with future emergency events.
The technical scheme is as follows: an event map construction method for safe operation of a dam comprises the following steps:
(1) and converting sentences and documents containing the information of the dam safe operation events into feature vectors by using an ALBERT embedding layer, enhancing the semantic information of Chinese, and processing the feature vectors converted by each word by using BilSTM.
(2) Introducing a local attention simulation event trigger, distributing corresponding weight of each word according to the importance degree, taking the word with the highest weight value as a hidden event trigger, introducing keywords and document context information in the global attention learning sentence, obtaining the unique meaning of the trigger in the scene, and assisting in judging the event type of the sentence.
(3) Training a trigger-free event detection model, adopting Focal loss as a loss function in the training process of the trigger-free event detection model, solving the problem of sample imbalance, simultaneously strengthening the influence of a positive sample and a difficultly-divided sample on the model, and outputting a predicted event type.
(4) And connecting event type coding vectors in series behind the feature vectors to form new sentence codes, and capturing context information by processing the embedded vectors after connection through the BilSTM.
(5) According to the sentence structure generated by dependency syntax analysis and the semantic vector generated by BilSTM, the attention mode is introduced to convert the network layer and the characteristics extracted by the attention network layer according to the weight fusion graph, a new expression vector is generated, and the event argument is extracted in a BIO sequence labeling mode.
(6) And judging key events in the key sentences by adopting the TextCNN, and supplementing missing event roles by using filling words in adjacent sentences to realize missing argument extraction of the events and supplement an event knowledge graph.
(7) And converting the attention network and the attention network by the weighted fusion graph to label sequences, acquiring causal relationships among events and constructing an event graph.
Further, in the step (1), sentences and documents containing the information of the safe operation events of the dam are converted into feature vectors by using an ALBERT embedding layer, semantic information of Chinese is enhanced, and the specific steps of processing the feature vectors converted by each word by using BilSTM are as follows:
(1.1) taking the operation records of equipment under the daily working condition and the emergency working condition of the dam as a fine tuning training corpus, then performing fine tuning training on a pre-trained ALBERT model, converting sentences into feature vectors W in a mathematical form, dynamically learning context information of a document, training different feature vectors of the same-name vocabulary according to the document information, and avoiding the problem of word ambiguity;
(1.2) processing the sentence feature vector W using the BilSTM network, outputting two hidden states
And
synthesis of
Sentence context information is denoted by h.
Further, the step (2) introduces a local attention simulation event trigger, assigns a corresponding weight value to each word according to the importance degree, takes the word with the highest weight value as a hidden event trigger, introduces keywords and document context information in the global attention learning sentence, obtains the unique meaning of the trigger in the scene, and assists in judging the event type of the sentence as follows:
(2.1) introduce a local attention mechanism to output LSTM output vector h, event type feature toQuantity t1As input, use the formula
Obtaining a local attention vector alphasWherein h iskIs the kth part of the output vector h,
is the local attention vector alphasThe kth part of (1);
(2.2) introducing a global attention mechanism, and embedding the output vector h and the event type into the vector t2And a document level embedded vector d is used as input, and a formula is used
Computing a global attention embedding vector αdWherein h iskIs the kth part of the output vector h,
is the global attention vector alphadThe kth part of (1);
(2.3) for αsAnd t1Using dot product operations, generating vsEvent trigger to capture local features and simulate concealment, for alphadAnd t2Using dot product operations, generating vdGlobal feature and context information is captured, followed by the Sigmoid function o- σ (λ · v)s+(1-λ)·vd) Processing the weighted-averaged dual attention vector vsAnd vd，λ∈[0,1]Is at vsAnd vdA hyper-parameter to make a trade-off between.
Further, in the step (3), Focal loss is adopted as a loss function in the model training process, the influence of the positive sample and the difficult sample on the model is strengthened while the problem of sample imbalance is solved, and the specific steps of outputting the predicted event type are as follows:
(3.1) during the model training process, the Focal loss is used as a loss function J (theta), and the formula is as follows:
where x is composed of sentences and target event types, y ∈ {0,1}, o (x)(i)) Is the predicted value of the model, | theta | | Y luminance2Is the sum of squares of each element in the model, δ > 0 is the weight of the L2 normalization term, β is the parameter of the proportion of positive and negative weights of the balanced samples, and γ is the parameter of the proportion of hard-to-classify and easy-to-classify weights of the balanced samples. Adding L2 regularization to the loss function prevents overfitting of the model, after which the model is trained;
and (3.2) testing on a dam operation condition data set by using the trained model, and outputting most possible event type numbers contained in each sentence according to preset possible event types.
Further, the step (4) concatenates the event type encoding vectors after the feature vectors to form a new sentence code, and captures the context information by processing the concatenated embedded vectors through BilSTM as follows:
and (3.2) adding the obtained sentence prediction event type vector into the sentence coding, and connecting the sentence prediction event type vector with word embedding, entity type embedding and part of speech tagging in series to form a 312-dimensional vector as an embedded vector which is used as the input of a BilSTM model to obtain a hidden vector sequence as a text semantic structure.
Further, the step (5) introduces the attention mode according to the characteristics extracted by the weight fusion graph conversion network layer and the attention network layer according to the sentence structure generated by the dependency syntax analysis and the semantic vector generated by the BilSTM, generates a new expression vector, and extracts the event argument by the BIO sequence labeling mode, which comprises the following specific steps:
(5.1) use of a binary matrix A of NdAs a syntactic structure, when a word wiAnd wjWith links in the dependency tree, then Ad(i, j) is set to 1, otherwise 0;
(5.2) if wiAnd wjThere is a dependency edge between and the dependency label is r, and a is initialized by using the embedded vector of r embedded in the lookup tabledl(i, j), otherwise using p-dimensional all zerosVector initialization Adl(i, j), then using the formula
Will depend on the label matrix AdlConversion to dependency tag score matrix
Wherein U is a trainable weight matrix;
(5.3) calculating a hidden vector hiAnd hjScore between them to obtain semantic score matrix AsThe calculation formula is as follows:
ki＝Ukhi,qi＝Uqhi,
wherein U iskAnd UqIs a trainable weight matrix;
(5.4) Adjacent dependency Tree to matrix AdDependent tag matrix AdlAnd semantic score matrix AsObtaining a dependency graph matrix by cascading
(5.5) it proposes graph transformation attention networks GTANs, which adopt 1 x 1 convolution to the adjacent matrix A set and soft select two intermediate adjacent matrixes Q1And Q2Generating a new meta-path graph A by matrix multiplicationlCompare path graph AlEach channel employs a GAT network and represents a plurality of nodes in series as Z, the formula is as follows:
where, | | is the join operator, C represents the number of channels,
is AlThe contiguous matrix of the ith channel of (a),
is that
V is a trainable weight matrix shared across channels, and X is a feature matrix;
(5.6) introducing an attention mechanism, and calculating an attention network layer weight matrix
And then, using sigmoid function to activate weighted fusion graph to convert characteristics of attention network layer and attention network layer
(5.7) using sequence labeling to label the event, labeling the beginning part of a key argument B, labeling the middle part of the key argument I, labeling other words except the key argument in a sentence O, and then using a conditional random field CRF to process a feature fusion vector
And outputs a predicted argument tag for each character in the specified event.
Further, the step (6) adopts the TextCNN to judge the key events in the key sentences, and then uses the filler words in the adjacent sentences to supplement the missing event roles to realize the missing argument extraction, and the specific steps of supplementing the event knowledge graph are as follows:
(6.1) filling arguments missed in the event, connecting four embedded vectors of argument labels, entity types, sentences and documents in series to form 880-dimensional new vectors, setting 128 vectors input by processing with convolution kernel sizes of 3, 4 and 5, projecting to 128 dimensions, and then judging whether the sentences contain key events through pooling and full connection layers;
and (6.2) for sentences containing key events, calculating the similarity between the rest sentences in the same document and the key sentences by using a MalSTM model, sequencing, and searching for argument roles in adjacent sentences which have the key events corresponding to the missing arguments and the highest similarity and filling the argument roles.
Further, the step (7) of converting the attention network and the attention network labeling sequence by the weighted fusion graph to obtain the causal relationship between the events comprises the following specific steps of:
constructing a frame comprising an embedding layer, a bidirectional long-short memory layer, a feature extraction layer, a fusion gate layer and a conditional random field layer, and carrying out BIO and CE labeling on the sequence, wherein B represents the beginning of an event argument, I represents the inside of the event argument, C is a reason, E is a result, and O is other words. B-C and I-C sequences are causal events, B-E and I-E sequences are effect events, so that the causal relationship among the events is constructed, and the knowledge graph is constructed.
Has the advantages that: compared with the prior art, the dam-oriented safe operation event map construction method based on the double attention mechanism avoids the situations that an event trigger has multiple meanings and words are not matched with the trigger. By means of local attention, semantic information is fully mined, an event trigger word is extracted by replacing a trigger with the importance degree, and the problem that the word is not matched with the trigger is solved; by means of global attention, the intermediate keywords and the document context information are learned, the unique meaning of the trigger in the scene is obtained, and the trigger ambiguity problem is solved. And finally, a Focal loss function training model is used for learning the safe operation record text data of the dam, and an event knowledge graph taking the event as the center is automatically constructed, so that the labor cost is saved, meanwhile, a knowledge base is established for the dam, the knowledge is stored in a structured form, and a foundation is provided for establishing event-driven application of the dam in the future.
Drawings
FIG. 1 is a flow diagram of a method of an embodiment;
FIG. 2 is a partial result graph of an event graph for a specific embodiment.
Detailed Description
The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention, which is to be given the full breadth of the claims appended hereto.
As shown in fig. 1, a method for constructing an event graph for safe operation of a dam includes the following steps:
(1) and converting sentences and documents containing the information of the dam safe operation events into feature vectors by using an ALBERT embedding layer, enhancing the semantic information of Chinese, and processing the feature vectors converted by each word by using BilSTM.
(1.1) taking the operation records of equipment under the daily working condition and the emergency working condition of the dam as a fine tuning training corpus, then performing fine tuning training on the pre-trained ALBERT model, converting sentences into feature vectors W in a mathematical form, and dynamically learning document context information;
(1.2) processing the sentence feature vector W using the BilSTM network, outputting two hidden states
And
synthesizing into
Sentence context information is denoted by h.
(2) Introducing a local attention simulation event trigger, distributing corresponding weight of each word according to the importance degree, taking the word with the highest weight value as a hidden event trigger, introducing keywords and document context information in the global attention learning sentence, obtaining the unique meaning of the trigger in the scene, and assisting in judging the event type of the sentence.
(2.1) introducing a local attention mechanism, and outputting an LSTM output vector h and an event type feature vector t1As input, use the formula
To gain local attentionQuantity alphasWherein h iskIs the kth part of the output vector h,
is the local attention vector alphasThe kth part of (1);
(2.2) introducing a global attention mechanism, and embedding the output vector h and the event type into the vector t2And a document level embedded vector d is used as input, and a formula is used
Computing a global attention embedding vector αdWherein h iskIs the kth part of the output vector h,
is the global attention vector alphadThe kth part of (1);
(2.3) for αsAnd t1Using dot product operations, generating vsEvent trigger to capture local features and simulate concealment, for alphadAnd t2Using dot product operations, generating vdGlobal feature and context information is captured, followed by the Sigmoid function o- σ (λ · v)s+(1-λ)·vd) Processing the weighted-averaged dual attention vector vsAnd vd，λ∈[0,1]Is at vsAnd vdA hyper-parameter to make a trade-off between.
(3) And in the model training process, the Focal loss is used as a loss function, the influence of the positive sample and the difficultly-divided sample on the model is enhanced while the problem of sample imbalance is solved, and the type of a predicted event is output.
(3.1) during the model training process, the Focal loss is used as a loss function J (theta), and the formula is as follows:
where x is composed of sentences and target event types, y ∈ {0,1}, o (x)(i)) Is a model forecastMeasuring value, | θ | | non-conducting phosphor2Is the sum of squares of each element in the model, δ > 0 is the weight of the L2 normalization term, β is the parameter of the proportion of positive and negative weights of the balanced samples, and γ is the parameter of the proportion of hard-to-classify and easy-to-classify weights of the balanced samples. Adding L2 regularization to the loss function prevents overfitting of the model, after which the model is trained;
and (3.2) testing on the dam operation condition data set by using the trained model, and outputting the most likely event type number contained in each sentence according to the possible event types set in advance.
(4) And connecting event type coding vectors in series behind the feature vectors to form new sentence codes, and capturing context information by processing the embedded vectors after connection through the BilSTM.
And (3.2) adding the obtained sentence prediction event type vector into the sentence coding, and connecting the sentence prediction event type vector with word embedding, entity type embedding and part of speech tagging in series to form a 312-dimensional vector as an embedded vector which is used as the input of a BilSTM model to obtain a hidden vector sequence as a text semantic structure.
(5) According to the sentence structure generated by dependency syntax analysis and the semantic vector generated by BilSTM, the attention mode is introduced to convert the network layer and the characteristics extracted by the attention network layer according to the weight fusion graph, a new expression vector is generated, and the event argument is extracted in a BIO sequence labeling mode.
(5.1) use of a binary matrix A of NdAs a syntactic structure, when a word wiAnd wjWith links in the dependency tree, then Ad(i, j) is set to 1, otherwise 0;
(5.2) if wiAnd wjWith a dependency edge and a dependency label of r, initializing A by using an embedded vector of r embedded in a lookup tabledl(i, j), otherwise initializing A using a p-dimensional all-zero vectordl(i, j), then using the formula
Will depend on the label matrix AdlConversion to dependency tag score matrix
Wherein U is a trainable weight matrix;
(5.3) calculating a hidden vector hiAnd hjScore between them to obtain semantic score matrix AsThe calculation formula is as follows:
ki＝Ukhi,qi＝Uqhi,
wherein U iskAnd UqIs a trainable weight matrix;
(5.4) Adjacent dependency Tree to matrix AdDependent tag matrix AdlAnd semantic score matrix AsObtaining a dependency graph matrix by cascading
(5.5) it proposes graph transformation attention networks GTANs, which adopt 1 x 1 convolution to the adjacent matrix A set and soft select two intermediate adjacent matrixes Q1And Q2Generating a new meta-path graph A by matrix multiplicationlCompare path graph AlEach channel employs a GAT network and represents the nodes in series as Z, with the following equation:
where, | | is the join operator, C represents the number of channels,
is AlThe contiguous matrix of the ith channel of (a),
is that
V is a trainable weight matrix shared across channels, and X is a feature matrix;
(5.6) introducing an attention mechanism, and calculating an attention network layer weight matrix
And then, using sigmoid function to activate weighted fusion graph to convert characteristics of attention network layer and attention network layer
(5.7) using sequence labeling to label the event, labeling the beginning part of a key argument B, labeling the middle part of the key argument I, labeling other words except the key argument in a sentence O, and then using a conditional random field CRF to process a feature fusion vector
And outputs a predicted argument tag for each character in the specified event.
(6) And judging key events in the key sentences by adopting the TextCNN, and supplementing missing event roles by using filling words in adjacent sentences to realize missing argument extraction of the events and supplement an event knowledge graph.
(6.1) filling arguments missed in the event, connecting four embedded vectors of argument labels, entity types, sentences and documents in series to form 880-dimensional new vectors, setting 128 vectors input by processing with convolution kernel sizes of 3, 4 and 5, projecting to 128 dimensions, and then judging whether the sentences contain key events through pooling and full connection layers;
and (6.2) for sentences containing key events, calculating the similarity between the rest sentences in the same document and the key sentences by using a MalSTM model, sequencing, and searching for argument roles in adjacent sentences which have the key events corresponding to the missing arguments and the highest similarity and filling the argument roles.
(7) Constructing a frame comprising an embedding layer, a bidirectional long-short memory layer, a feature extraction layer, a fusion gate layer and a conditional random field layer, and carrying out BIO and CE labeling on the sequence, wherein B represents the beginning of an event argument, I represents the inside of the event argument, C is a reason, E is a result, and O is other words. B-C and I-C sequences are causal events, B-E and I-E sequences are effect events, so that the causal relationship among the events is constructed, and finally the knowledge graph is constructed.
Claims (8)
1. An event map construction method for safe operation of a dam is characterized by comprising the following steps:
(1) converting sentences and documents containing dam safe operation event information into feature vectors by using an ALBERT embedding layer, enhancing Chinese semantic information, and processing the feature vectors converted from each word by using BilSTM;
(2) introducing a local attention simulation event trigger, distributing corresponding weight of each word according to importance degree, taking the word with the highest weight value as a hidden event trigger, introducing keywords and document context information in a global attention learning sentence, obtaining the unique meaning of the trigger in the current scene, and assisting in judging the event type of the sentence;
(3) during the model training process, Focal loss is used as a loss function, the influence of a positive sample and a difficultly-divided sample on the model is enhanced while the problem of sample imbalance is solved, and a prediction event type is output;
(4) connecting event type coding vectors in series behind the feature vectors to form new sentence codes, processing the embedded vectors after connection in series through BilSTM, and capturing context information;
(5) according to a sentence structure generated by dependency syntactic analysis and a semantic vector generated by BilSTM, introducing the characteristics extracted by an attention mode conversion network layer and an attention network layer according to a weight fusion graph to generate a new expression vector, and extracting event arguments in a BIO sequence labeling mode;
(6) judging key events in the key sentences by using the TextCNN, and supplementing missing event roles by using filling words in adjacent sentences to realize missing argument extraction of the events and supplement an event knowledge graph;
(7) and converting the attention network and the attention network by the weighted fusion graph to label sequences, acquiring causal relationships among events and constructing an event graph.
2. The dam safety operation oriented event map construction method according to claim 1, wherein in the step (1), sentences and documents containing dam safety operation event information are converted into feature vectors by using an ALBERT embedding layer, Chinese semantic information is enhanced, and the specific steps of processing the feature vectors converted by each word by using BilSTM are as follows:
(1.1) taking the operation records of equipment under the daily working condition and the emergency working condition of the dam as a fine tuning training corpus, then performing fine tuning training on the pre-trained ALBERT model, and converting sentences into feature vectors W in a mathematical form;
(1.2) processing the sentence feature vector W using the BilSTM network, outputting two hidden states
And
synthesis of
The sentence context information is represented by the LSTM output vector h.
3. The dam safety operation-oriented event map construction method according to claim 2, wherein a local attention simulation event trigger is introduced in the step (2), a corresponding weight value of each word is distributed according to the importance degree, the word with the highest weight value is taken as a hidden event trigger, keywords and document context information in a global attention learning sentence are introduced, the unique meaning of the trigger in the scene is obtained, and the specific steps of assisting in judging the event type of the sentence are as follows:
(2.1) introducing a local attention mechanism, and outputting an LSTM output vector h and an event type feature vector t1As input, use the formula
Obtaining a local attention vector alphasWherein h iskIs the kth part of the output vector h,
is the local attention vector alphasThe kth part of (1);
(2.2) introducing a global attention mechanism, and embedding the output vector h and the event type into the vector t2And a document level embedded vector d is used as input, and a formula is used
Computing a global attention embedding vector αdWherein h iskIs the kth part of the output vector h,
is the global attention vector alphadThe kth part of (1);
(2.3) for αsAnd t1Using dot product operations, generating vsEvent trigger to capture local features and simulate concealment, for alphadAnd t2Using dot product operations, generating vdGlobal feature and context information is captured, followed by the Sigmoid function o- σ (λ · v)s+(1-λ)·vd) Processing the weighted-averaged dual attention vector vsAnd vd，λ∈[0,1]Is at vsAnd vdA hyper-parameter to make a trade-off between.
4. The dam safety operation oriented event graph construction method according to claim 1, wherein in the step (3), Focal local is adopted as a loss function in a model training process, influence of a positive sample and a hard sample on the model is strengthened while the problem of sample unbalance is solved, and specific steps of outputting a predicted event type are as follows:
(3.1) during the model training process, the Focal loss is used as a loss function J (theta), and the formula is as follows:
where x is composed of sentences and target event types, y ∈ {0,1}, o (x)(i)) Is the predicted value of the model, | theta | | Y luminance2The sum of squares of all elements in the model, delta & gt 0 is the weight of an L2 normalization term, beta is a parameter of the proportion of positive and negative weights of a balance sample, gamma is a parameter of the proportion of hard-to-classify and easy-to-classify weights of the balance sample, L2 regularization is added in a loss function to prevent the model from being over-fitted, and then the model is trained;
and (3.2) testing on the dam operation condition data set by using the trained model, and outputting the most likely event type number contained in each sentence according to the possible event types set in advance.
5. The method for constructing an event graph for safe operation of a dam according to claim 4, wherein in the step (4), the event type coding vector is connected in series after the feature vector to form a new sentence code, and the embedded vector after connection is processed by BilSTM, and the specific steps of capturing context information are as follows:
and (3) adding the sentence prediction event type obtained in the step (3) into the sentence coding, and connecting the sentence prediction event type with word embedding, entity type embedding and part of speech tagging in series to form a 312-dimensional vector as an embedded vector which is used as the input of a BilSTM model to obtain a hidden vector sequence as a text semantic structure.
6. The dam safety operation oriented event map construction method according to claim 1, wherein the step (5) introduces the attention mode according to the sentence structure generated by the dependency syntax analysis and the semantic vector generated by BilSTM, integrates the features extracted by the graph conversion network layer and the attention network layer according to the weight, generates a new expression vector, and extracts the event argument through the BIO sequence labeling mode specifically comprises the following steps:
(5.1) use of a dependency tree adjacency matrix A of NdAs a syntax structure, whenWord wiAnd wjWith links in the dependency tree, then Ad(i, j) is set to 1, otherwise 0;
(5.2) if wiAnd wjThere is a dependency edge between and the dependency label is r, and a is initialized by using the embedded vector of r embedded in the lookup tabledl(i, j), otherwise initializing A using a p-dimensional all-zero vectordl(i, j), then using the formula
Will depend on the label matrix AdlConversion to dependency tag score matrix
Wherein U is a trainable weight matrix;
(5.3) calculating a hidden vector hiAnd hjScore between them to obtain semantic score matrix AsThe calculation formula is as follows:
ki＝Ukhi,qi＝Uqhi,
wherein U iskAnd UqIs a trainable weight matrix;
(5.4) Adjacent dependency Tree to matrix AdDependent tag matrix AdlAnd semantic score matrix AsObtaining a dependency graph matrix by cascading
(5.5) providing graph transformation attention networks GTANs, adopting 1 x 1 convolution to the dependency graph matrix A set, and selecting two intermediate adjacent matrixes Q1And Q2Generating a new meta-path graph A by matrix multiplicationlCompare path graph AlEach channel employs a GAT network and represents the nodes in series as Z, with the following equation:
where | is the join operator, C denotes the number of channels,
is AlThe contiguous matrix of the ith channel of (a),
is that
V is a trainable weight matrix shared across channels, and X is a feature matrix;
(5.6) introducing an attention mechanism, and calculating an attention network layer weight matrix
And then, using sigmoid function to activate weighted fusion graph to convert characteristics of attention network layer and attention network layer
(5.7) using sequence labeling to label the event, labeling the beginning part of a key argument B, labeling the middle part of the key argument I, labeling other words except the key argument in a sentence O, and then using a conditional random field CRF to process a feature fusion vector
And outputs a predicted argument tag for each character in the specified event.
7. The dam safety operation-oriented event map construction method according to claim 1, wherein in the step (6), TextCNN is adopted to judge key events in key sentences, and then filler words in adjacent sentences are used to supplement missing event roles to realize missing argument extraction, and the specific steps of supplementing an event knowledge map are as follows:
(6.1) filling the argument missed by the event, connecting four embedded vectors of argument labels, entity types, sentences and documents in series to form 880-dimensional new vectors, setting 128 vectors with convolution kernel sizes of 3, 4 and 5 for processing input, projecting to 128 dimensions, and then judging whether the sentences contain key events through a pooling and full-connection layer;
and (6.2) for sentences containing key events, calculating the similarity between the rest sentences in the same document and the key sentences by using a MaLSTM model, sequencing, searching for argument roles in adjacent sentences of which the key events correspond to the missing arguments and have the highest similarity, and filling.
8. The dam safety operation oriented event map construction method according to claim 1, wherein the step (7) comprises the steps of:
and (3) labeling the events by using sequence labeling, extracting reason and result arguments, and defining the events to which the reason and result arguments belong as reason events and result events, thereby establishing a causal relationship between the events and constructing an event map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702542.3A CN113312500B (en) | 2021-06-24 | 2021-06-24 | Method for constructing event map for safe operation of dam |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702542.3A CN113312500B (en) | 2021-06-24 | 2021-06-24 | Method for constructing event map for safe operation of dam |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113312500A CN113312500A (en) | 2021-08-27 |
| CN113312500B true CN113312500B (en) | 2022-05-03 |
Family
ID=77380046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110702542.3A Active CN113312500B (en) | 2021-06-24 | 2021-06-24 | Method for constructing event map for safe operation of dam |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312500B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935502B (en) * | 2021-10-15 | 2022-04-22 | 河海大学 | Dam-oriented emergency condition event extraction method based on double attention mechanism |
| CN113901815B (en) * | 2021-10-15 | 2023-05-05 | 华能澜沧江水电股份有限公司 | Emergency working condition event detection method based on dam operation log |
| CN113761941B (en) * | 2021-11-09 | 2022-02-08 | 华南师范大学 | Text emotion analysis method |
| CN114201970A (en) * | 2021-11-23 | 2022-03-18 | 国家电网有限公司华东分部 | Method and device for capturing power grid scheduling event detection based on semantic features |
| CN114706992B (en) * | 2022-02-17 | 2022-09-30 | 中科雨辰科技有限公司 | Event information processing system based on knowledge graph |
| CN114756681B (en) * | 2022-04-28 | 2024-04-02 | 西安交通大学 | Evaluation and education text fine granularity suggestion mining method based on multi-attention fusion |
| CN114880584B (en) * | 2022-05-16 | 2024-05-28 | 华能澜沧江水电股份有限公司 | Generator set fault analysis method based on community discovery |
| CN115080716B (en) * | 2022-05-31 | 2024-09-17 | 华南理工大学 | Event argument extraction method, system and medium based on graph convolution and machine reading understanding |
| CN114781553B (en) * | 2022-06-20 | 2023-04-07 | 浙江大学滨江研究院 | Unsupervised patent clustering method based on parallel multi-graph convolution neural network |
| CN114996414B (en) * | 2022-08-05 | 2022-09-30 | 中科雨辰科技有限公司 | Data processing system for determining similar events |
| CN115577678B (en) * | 2022-09-21 | 2024-04-02 | 中国人民解放军海军工程大学 | Method, system, medium, equipment and terminal for identifying causal relationship of document-level event |
| CN115599899B (en) * | 2022-11-08 | 2023-04-07 | 中国空气动力研究与发展中心计算空气动力研究所 | Intelligent question-answering method, system, equipment and medium based on aircraft knowledge graph |
| CN116049345B (en) * | 2023-03-31 | 2023-10-10 | 江西财经大学 | Document-level event joint extraction method and system based on bidirectional event complete graph |
| CN116303996B (en) * | 2023-05-25 | 2023-08-04 | 江西财经大学 | Theme event extraction method based on multifocal graph neural network |
| CN116738366B (en) * | 2023-06-16 | 2024-07-16 | 河海大学 | Method and system for identifying causal relationship of dam emergency event based on feature fusion |
| CN117786092B (en) * | 2024-02-27 | 2024-05-14 | 成都晓多科技有限公司 | Commodity comment key phrase extraction method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015103899A1 (en) * | 2014-01-09 | 2015-07-16 | 百度在线网络技术（北京）有限公司 | Construction method and device for event repository |
| CN109005069A (en) * | 2018-08-29 | 2018-12-14 | 中国人民解放军国防科技大学 | Network security knowledge graph association analysis method based on heaven-earth integrated network |
| CN111522961A (en) * | 2020-04-09 | 2020-08-11 | 武汉理工大学 | Attention mechanism and entity description based industrial map construction method |
| CN111581396A (en) * | 2020-05-06 | 2020-08-25 | 西安交通大学 | Event graph construction system and method based on multi-dimensional feature fusion and dependency syntax |
| CN111832922A (en) * | 2020-06-30 | 2020-10-27 | 北方工业大学 | Food safety event risk studying and judging method and device based on knowledge graph reasoning |
| CN111897974A (en) * | 2020-08-12 | 2020-11-06 | 吉林大学 | Heterogeneous knowledge graph learning method based on multilayer attention mechanism |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109726293B (en) * | 2018-11-14 | 2020-12-01 | 数据地平线（广州）科技有限公司 | Causal event map construction method, system, device and storage medium |
| CN109710919A (en) * | 2018-11-27 | 2019-05-03 | 杭州电子科技大学 | A kind of neural network event extraction method merging attention mechanism |
| CN111143576A (en) * | 2019-12-18 | 2020-05-12 | 中科院计算技术研究所大数据研究院 | Event-oriented dynamic knowledge graph construction method and device |
| CN112307740B (en) * | 2020-12-30 | 2021-03-26 | 中国人民解放军国防科技大学 | Event detection method and device based on hybrid attention network |
-
2021
- 2021-06-24 CN CN202110702542.3A patent/CN113312500B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015103899A1 (en) * | 2014-01-09 | 2015-07-16 | 百度在线网络技术（北京）有限公司 | Construction method and device for event repository |
| CN109005069A (en) * | 2018-08-29 | 2018-12-14 | 中国人民解放军国防科技大学 | Network security knowledge graph association analysis method based on heaven-earth integrated network |
| CN111522961A (en) * | 2020-04-09 | 2020-08-11 | 武汉理工大学 | Attention mechanism and entity description based industrial map construction method |
| CN111581396A (en) * | 2020-05-06 | 2020-08-25 | 西安交通大学 | Event graph construction system and method based on multi-dimensional feature fusion and dependency syntax |
| CN111832922A (en) * | 2020-06-30 | 2020-10-27 | 北方工业大学 | Food safety event risk studying and judging method and device based on knowledge graph reasoning |
| CN111897974A (en) * | 2020-08-12 | 2020-11-06 | 吉林大学 | Heterogeneous knowledge graph learning method based on multilayer attention mechanism |
Non-Patent Citations (5)
| Title |
|---|
| Jointly embedding the local and global relations of heterogeneous graph for rumor detection;Chunyuan Yuan 等;《2019 IEEE International Conference on Data Mining (ICDM)》;20200130;1-10 * |
| 事件图谱的构建、推理与应用;胡志磊 等;《大数据》;20210506;第7卷(第03期);80-96 * |
| 事件的触发词与论元联合抽取方法研究;纪宇泽;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20210315(第(2021)03期);I138-834 * |
| 基于事理图谱的航空安全事故因果关系研究;祝寒;《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅰ辑》;20200215(第(2020)02期);B026-114 * |
| 面向流域防洪安全的态势图谱构建及可视化方法;陈华 等;《河海大学学报(自然科学版)》;20201125;第48卷(第6期);498-505 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113312500A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113312500B (en) | Method for constructing event map for safe operation of dam | |
| CN113095415B (en) | Cross-modal hashing method and system based on multi-modal attention mechanism | |
| CN111797241B (en) | Event Argument Extraction Method and Device Based on Reinforcement Learning | |
| CN110083729B (en) | Image searching method and system | |
| CN113282713B (en) | Event trigger detection method based on difference neural representation model | |
| CN108628935A (en) | A kind of answering method based on end-to-end memory network | |
| CN113656563B (en) | Neural network searching method and related equipment | |
| CN111476038A (en) | Long text generation method and device, computer equipment and storage medium | |
| JP7290861B2 (en) | Answer classifier and expression generator for question answering system and computer program for training the expression generator | |
| CN116204674B (en) | Image description method based on visual concept word association structural modeling | |
| CN110991149A (en) | Multi-mode entity linking method and entity linking system | |
| CN113657115A (en) | Multi-modal Mongolian emotion analysis method based on ironic recognition and fine-grained feature fusion | |
| CN114282528A (en) | Keyword extraction method, device, equipment and storage medium | |
| CN115455171A (en) | Method, device, equipment and medium for mutual retrieval and model training of text videos | |
| CN114548099A (en) | Method for jointly extracting and detecting aspect words and aspect categories based on multitask framework | |
| CN110659392B (en) | Retrieval method and device, and storage medium | |
| CN110852066A (en) | Multi-language entity relation extraction method and system based on confrontation training mechanism | |
| CN113392929B (en) | Biological sequence feature extraction method based on word embedding and self-encoder fusion | |
| CN113240033B (en) | Visual relation detection method and device based on scene graph high-order semantic structure | |
| CN111445545B (en) | Text transfer mapping method and device, storage medium and electronic equipment | |
| CN109446518B (en) | Decoding method and decoder for language model | |
| CN115774782A (en) | Multilingual text classification method, device, equipment and medium | |
| CN113821610A (en) | Information matching method, device, equipment and storage medium | |
| CN117828072B (en) | Dialogue classification method and system based on heterogeneous graph neural network | |
| Rodoshi et al. | Automated image caption generator in Bangla using multimodal learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |