CN117408247B - Intelligent manufacturing triplet extraction method based on relational pointer network - Google Patents
Intelligent manufacturing triplet extraction method based on relational pointer network Download PDFInfo
- Publication number
- CN117408247B CN117408247B CN202311726555.XA CN202311726555A CN117408247B CN 117408247 B CN117408247 B CN 117408247B CN 202311726555 A CN202311726555 A CN 202311726555A CN 117408247 B CN117408247 B CN 117408247B
- Authority
- CN
- China
- Prior art keywords
- entity
- relation
- head
- text
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 42
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 36
- 239000013598 vector Substances 0.000 claims description 49
- 238000012549 training Methods 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 5
- 230000003213 activating effect Effects 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000000034 method Methods 0.000 description 10
- 238000002372 labelling Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000000306 recurrent effect Effects 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/288—Entity relationship models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention belongs to the field of intelligent manufacturing triplet extraction, and discloses an intelligent manufacturing triplet extraction method based on a relation pointer network.
Description
Technical Field
The invention belongs to the field of intelligent manufacturing triplet extraction, and particularly relates to an intelligent manufacturing triplet extraction method based on a relation pointer network.
Background
With the rapid development of new generation information technology, the amount and complexity of information has increased dramatically. There is a need for a more efficient way to organize, retrieve and understand large amounts of information, and conventional relational databases and search engines are limited in this regard. The knowledge graph is a semantic knowledge base of a directed graph structure, and can extract useful information from redundant data and knowledge texts to effectively express internal relations between the data. Knowledge maps allow knowledge from different domains to be integrated into a unified structure. This is very effective for complex problem solving and knowledge discovery involving multiple fields. In the process of constructing a large-scale domain knowledge graph, triad extraction is one of key technologies used for extracting entities and relations from massive texts.
In the knowledge graph, the triplet is a data structure shaped as (head entity, relation, tail entity), and at present, the triplet extraction is divided into pipeline extraction and joint extraction, and the pipeline extraction divides the triplet extraction into two independent tasks: namely, entity identification is carried out firstly, then relationship classification is carried out, and the two tasks are not interacted, so that errors generated by the entity identification are transmitted to the relationship classification; the joint extraction is used for uniformly modeling the entity identification and relation extraction sub-models, and compared with pipeline extraction, the inherent relation and the dependent relation in two tasks are further utilized, so that the defect of error accumulation is relieved to a certain extent.
The Chinese patent application number CN2021111821736 discloses a Chinese triplet extraction method based on the bidirectional coding representation of convertors, which can fully describe the characteristic relationship among character level, word level and sentence level. However, the technology firstly identifies the entity and then identifies the relation through pipeline type extraction, the inherent relation and the dependent relation between the two tasks cannot be fully utilized, and the entity pair without the relation is identified to promote the error rate, waste the calculated amount, bring additional redundant information and cannot correspondingly process complex scenes such as the entity with overlapping relation and shared head.
Chinese patent application No. CN202111658767X provides a method for extracting relation triples based on cascade binary annotation framework, which models the relation as mapping head entity to tail entity in sentence, namely f Relationship of The (head entity) =tail entity solves the problem of relation overlapping, and the discrete labels of the entity pairs are allocated to the relation by the single label labeling model, the Start and End positions of the entities are labeled by the multi-label binary labeling framework, and the problem of sample imbalance is solved. However, the technology has the problems of relation redundancy, waste of calculation amount, extra redundant information, and incapability of correspondingly processing complex scenes such as overlapping relation, entities sharing the head and the like. For example, there are hundreds of relations in the relation set, but the density of the relations is very low, and one relation appears in each sentence of text on average, so that for each head entity, the method needs to predict tail entity information corresponding to hundreds of relations, and a plurality of redundant relation judgment is generated.
In summary, there are a problem of overlapping of one pair of entities corresponding to a plurality of relationships, namely, "one-to-many" entity pair, a problem of overlapping of one entity corresponding to a plurality of relationships, namely, "many-to-one" single entity pair, and a nesting situation of the entities themselves in the text, so the task of extracting triples faces a great challenge.
Disclosure of Invention
Aiming at the technical problems and the demands, the invention provides an intelligent manufacturing triplet extraction method based on a relation pointer network, which predicts potential relations in data by utilizing an average pooling network and a fully-connected neural network, solves the problem of intelligent manufacturing relation calculation redundancy in the subsequent entity identification process, calculates entity positions by utilizing two span-based double-layer pointer networks, solves the problem of entity nesting and relation overlapping, improves the accuracy of intelligent manufacturing entity labeling, and finally matches entity pairs based on the potential relations by utilizing the relation pointer network, thereby correctly extracting intelligent manufacturing triples and improving the efficiency and quality of triplet extraction tasks.
In order to achieve the above purpose, the invention is realized by the following technical scheme:
the invention relates to an intelligent manufacturing triplet extraction method based on a relation pointer network, which comprises a context pre-training model coding module, a potential relation prediction module, a head entity decoding module and a head and tail entity alignment module, wherein the context pre-training model coding module is used for acquiring semantic feature representation of intelligent text data, and learning context information of each word in a text by using a context pre-training model to obtain semantic vectors of the text; the potential relation prediction module is used for constructing an intelligent manufacturing relation candidate set and predicting potential relation existing in the intelligent manufacturing text; the header entity decoding module is used for realizing entity span extraction and determining the starting and ending positions of the entity; the head-tail entity alignment module is used for realizing the matching of intelligent manufacturing triples, traversing potential relations in the relation candidate set for each head entity, checking whether tail entities related to the head entity exist, and outputting { head entity, relation, tail entity } triples if the tail entities exist.
The specific triplet extraction method comprises the following steps:
step 1, preparing text data and defining a relation set existing in the text dataWherein the input is a text sentence;
step 2, adopting a context pre-training model coding module to code the text data in the step 1;
step 3, predicting a candidate relation set in the text data by utilizing a potential relation prediction module，/>For the relation set defined in step 1 +.>Is to get->；
Step 4, setting a relation threshold valueWhen +.>Is greater than->When the relationship is considered to be contained in the triplet, the corresponding relationship is marked as 1, the rest is marked as 0, and the relationship is recorded as a candidate relationship set +.>Candidate relation set->Less than relation set->；
Step 5, predicting the starting position and the ending position of the head entity by using a head entity decoding module, and determining the head existing in the text according to the natural continuity of the span of the head entityEntityInput sequence +.>The token is a header entity +.>The start and end positions of (1) are denoted +.>And->；
Step 6, if obtained in step 5、/>When the value of (2) is larger than a preset threshold value, marking the corresponding token as 1, and regarding the corresponding token as the starting or ending position of the head entity;
step 7, traversing the relation candidate set by using the head-tail entity alignment moduleIn (2) calculating whether there is +.>Related tail entity->First->Probability of a person token as a start position of a header entity +.>Probability of ending position->Respectively->，；
Step 8, if obtained in step 7When the value is larger than the preset value, the corresponding token is marked as 1 and is regarded as the starting position of the tail entity, if +.>When the value of (2) is larger than the preset value, the corresponding token is marked as 1 and is regarded as the end position of the tail entity, and the { head entity, relation and tail entity } triples are successfully output by matching.
The invention further improves that: the step 2 specifically comprises the following steps:
step 2.1, firstly, inputting text sentencesEach sub-word of (a) is converted into a word vector and a position vectorThe word vector and the position vector are input into 12 double-layer converter blocks to extract the features,
step 2.2, the transformation blocks learn information through a multi-head self-attention mechanism, each transformation block transmits the learned information through a layer of full-connection network, and finally semantic vectors are output：，/>Representing the dimension of the last hidden layer of the BERT model.
The invention further improves that: the step 3 is specifically as follows:
step 3.1, the potential relation prediction module encodes the semantic vector of the text output by the context pre-training model encoding modulePushing into a global average pooling layer;
step 3.2, inputting a fully-connected neural network;
step 3.3, finally calculating the probability of each relation by activating the function to obtain。
The invention further improves that: the step 5 specifically comprises the following steps:
step 5.1, the head entity decoding module outputs the semantic vector from the context pre-training model encoding moduleInputting GRU to obtain->；
Step 5.2, inputting two identical double-layer pointer networks for marking the starting position and the ending position of the entity, wherein the nonlinearity of the network is enhanced by using a ReLU activation function in the middle of the two-layer pointer networks;
step 5.3, calculating to obtain probability:，，/>and->Respectively represent the +.>The start and end positions of the respective token as header entity,/->Is a weight that can be learned, +.>Indicating bias(s)>Representing an activation function.
The invention further improves that: the model is trained jointly by adopting a mode of sharing parameters during model training, and the objective function of the combination is optimized during training, and the total loss is divided into three parts:，，the total loss is the sum of the three parts: />。
The beneficial effects of the invention are as follows:
(1) The invention utilizes the relation pointer-based network to realize the triplet extraction model, improves the triplet extraction quality, provides new kinetic energy for the comprehensive landing of the knowledge graph, and accelerates the enterprise to realize resource aggregation and optimization.
(2) According to the method, the potential relation in the text data is predicted by utilizing the neural network, and the entity pair matching is performed by the pointer network based on the relation, so that the problem of relation redundancy is effectively solved, and the calculated amount in the head-to-tail entity alignment process is greatly reduced by constructing the candidate relation set.
(3) According to the invention, the GRU is matched with the double-layer pointer network to calculate the position of the intelligent manufacturing entity, the accuracy of marking the intelligent manufacturing entity is improved, the characteristics are combined through the operation of dimension increasing and dimension decreasing of the two layers of fully connected neural network layers, the resolution capability of a model is improved, the combined characteristics with low distinguishing degree are removed, and the characterization of the entity is effectively learned.
Drawings
FIG. 1 is a flow chart of the intelligent manufacturing triplet extraction method of the present invention based on a potential relationship and pointer network.
FIG. 2 is a block diagram of an intelligent manufacturing triplet extraction method based on a potential relationship and pointer network of the present invention.
Detailed Description
Embodiments of the invention are disclosed in the drawings, and for purposes of explanation, numerous practical details are set forth in the following description. However, it should be understood that these practical details are not to be taken as limiting the invention. That is, in some embodiments of the invention, these practical details are unnecessary.
The invention relates to an intelligent manufacturing triplet extraction method based on a relation pointer network, which uses a BERT pre-training model to replace the existing word2vec (Word Representations in Vector Space) word vector generation model, thereby overcoming the defects of the traditional triplet extraction method, and the word vector obtained by the BERT model has stronger generalization capability, can fully describe the characteristic relation among character level, word level and sentence level, uses Bi-directional Gated recurrent units and Bi-GRU as network structures, combines an attention mechanism to perform relation extraction, not only improves the accuracy of relation extraction, but also expands the application range of the BERT pre-training model. The invention also processes the extracted sentences through the BERT pre-training model to obtain semantic feature representation coding vectors in the sentences; decoding the output coded vector, and identifying Start and End position labels of the head entities, so as to obtain feature vector matrixes of all possible head entities and corresponding Token in the sentence; averaging vectors corresponding to Token of the feature vector matrix to obtain a head entity feature vector, and fusing the output decoding vectors to obtain a fused vector. According to the fused vector->And combining a group of specific relation sets, identifying Start and End position labels of tail entities of the corresponding relation, identifying all relations and tail entities related to the head entity, and finally extracting relation triples.
Aiming at the problems of entity overlapping and relation redundancy in text data, a relation pointer network-based triplet extraction method is introduced, and the method comprises a context pre-training model coding module, a potential relation prediction module, a head entity decoding module and a head and tail entity alignment module, wherein the whole framework is shown in figure 1.
The context pre-training model coding module is used for obtaining semantic feature representation of intelligent text data, and learning context information of each word in the text by using the context pre-training model to obtain semantic vectors of the text. And mapping the words in the text into corresponding word vectors, and inputting the corresponding word vectors into the BERT pre-training model. The embedded vector obtained through the BERT pre-training model has stronger generalization capability, can fully describe the characteristic relation in the text, has better global expression effect, and the text semantic vector output by the pre-training model coding module is used as the input of the next module.
The potential relation prediction module is used for constructing an intelligent manufacturing relation candidate set and predicting potential relations existing in the intelligent manufacturing text. The number of potential relations is far smaller than the number of relations in the predefined relation set, so that only the relations in the relation candidate set are subjected to entity alignment in the head-to-tail entity alignment module, all the relations are not calculated, redundant relation judgment can be reduced, and the redundancy of relation calculation is reduced. Text semantic vector output by context pre-training model coding module，/>Input global average pooling layer (Global Average Pooling, GAP) to remove redundant messagesAnd the characteristics are compressed at the same time, so that the number of parameters and the calculated amount are reduced, and the overfitting is effectively restrained. Then the probability +.sub.f was obtained using the softmax two classifier>When->When the relation is larger than a preset threshold value, the relation is considered to be contained, the corresponding relation is marked as 1, the relation is put into the intelligent manufacturing relation candidate set, and the rest is marked as 0.
The header entity decoding module is used for realizing entity span extraction and determining the starting and ending positions of the entity. Semantic vector of text output by pre-training model coding module，/>Input gating of the recurrent neural unit (Gate Recurrent Unit, GRU), by resetting the short-term dependence in the gate capture sequence, by updating the long-term dependence in the gate capture sequence, obtaining +.>The start and end positions of the entity are then calculated using a span-based two-layer pointer network: />，，/>And->Respectively represent the +.>The start of a token as a header entityAnd end position if->When the value of (2) is larger than a preset value, marking the corresponding token as 1, and regarding the corresponding token as the starting position of the head entity; if->When the value of (2) is larger than the preset value, marking the corresponding token as 1, and regarding the corresponding token as the end position of the head entity.
The head-tail entity alignment module is used for realizing the matching of intelligent manufacturing triples, traversing potential relations in the relation candidate set for each head entity, checking whether tail entities related to the head entity exist, outputting { head entity, relation, tail entity } triples if the tail entities exist, namely searching for the tail entities according to the head entities and the relation, and outputting the head entity, the relation and the tail entity triples if the tail entities exist. Alignment of head and tail entities based on potential relationships using a relationship pointer network, not only considering GRU processing when decoding and identifying tail entities for further use of global information of context in textThe characteristics of the head entity obtained in the head entity decoding module also need to be considered, so that the average +.f of all word vectors of the head entity is added when the tail entity corresponding to the head entity is calculated>The position of each tail entity is calculated using the fused feature vector. Calculating to obtain probability，，/>Andrespectively represent input sequencesMiddle->The start and end positions of the token as header entity, if->When the value of (2) is larger than a preset value, marking the corresponding token as 1, and regarding the token as the starting position of the tail entity; if->When the value of (2) is larger than the preset value, marking the corresponding token as 1, and regarding the corresponding token as the end position of the tail entity.
Specifically, the intelligent manufacturing triplet extraction method based on the relation pointer network comprises the following steps:
step 1, preparing enough text data and defining the relation set existing in the text dataWherein the input is a text sentence, wherein there may be multiple triples, and there may be overlapping situations of entities and relationships.
And 2, encoding the text data in the step 1 by adopting a context pre-training model encoding module. First, input sentencesIs converted into a word vector and a position vector +.>The method comprises the steps of inputting the characteristics into 12 double-layer convertors, learning information by the convertors through a multi-head self-attention mechanism, transmitting the learned information by each convertors through a layer of full-connection network, and finally outputting semantic vectors->：，/>Representing the dimension of the last hidden layer of the BERT model.
Step 3, predicting a candidate relation set in the text by utilizing a potential relation prediction module，/>Is the relation set defined in step one +.>Is a subset of the set of (c). The module encodes the semantic vector +.>Pushing into global average pooling layer, inputting fully connected neural network, and calculating probability of each relation by activating function to obtain ∈>。
Step 4, setting a relation threshold valueWhen +.>Is greater than->When the relation is considered to be contained, the corresponding relation is marked as 1, the rest is marked as 0, and the relation is recorded as a candidate relation set +.>Candidate relation set->Less than relation set->；
Step 5, predicting the starting position and the ending position of the head entity by using a head entity decoding module, and determining the head entity existing in the text according to the natural continuity of the span of the head entityThe module encodes the semantic vector outputted by the module of the context pre-training model +.>Inputting GRU to obtain->Then, two identical double-layer pointer networks are input for marking the starting position and the ending position of the entity, wherein the nonlinearity of the network is enhanced by using a ReLU activation function in the middle of the two-layer pointer networks, and finally, the probability is calculated: />，，/>And->Respectively represent the +.>The start and end positions of the respective token as header entity,/->Is a weight that can be learned, +.>Indicating bias(s)>Representing an activation function.
Step 6, if obtained in step 5、/>When the value of (2) is larger than a preset threshold value, marking the corresponding token as 1, and regarding the corresponding token as the starting or ending position of the head entity;
step 7, traversing the relation candidate set by using the head-tail entity alignment moduleIn (2) calculating whether there is +.>Related tail entity->. Matching entity pairs by using a relationship-based pointer network, and simultaneously considering the output +.f through GRU when decoding and identifying tail entities for further using global information in text>And the average +.about.of all word vectors of the head entity>. First->Probability of a person token as a start position of a header entity +.>Probability of ending position->Respectively->，
；
Step 8, if obtained in step 7When the value is larger than the preset value, the corresponding token is marked as 1 and is regarded as the starting position of the tail entity, if +.>When the value of (2) is larger than the preset value, the corresponding token is marked as 1 and is regarded as the end position of the tail entity, and the { head entity, relation and tail entity } triples are successfully output by matching.
Examples
The invention relates to an intelligent manufacturing triplet extraction method based on a relation pointer network, which comprises the following specific flow: character vector characterization of intelligent manufacturing text using pre-trained language model BERT, intelligent manufacturing text as character sequenceInputting BERT, using self-attention mechanism, residual and layer normalization in a transducer encoder, feedforward neural network for each character +.>Character vector representation is performed:wherein { Q, K, V } is the input matrix, { I }>For the dimension of the input vector, the output vector is finally obtained>。
Will beInputting an average pooling layer, and calculating each relation by using the fully connected neural network and the activation function sigmoidProbability of->. And predicting potential relations existing in the sentence, counting a candidate relation set, and setting three potential relations contained in the sentence, wherein when entity matching is performed, only entity pairs corresponding to the three potential relations are calculated instead of calculating tail entities corresponding to all relations in the relation set.
Using a header entity decoding module toDecoding, namely outputting semantic vectors of intelligent manufacturing text by the pre-training model coding module>，/>Input GRU,/->And predicting the starting position and the ending position of the head entity by using a double-layer labeling mode to form a candidate head entity set. Let the starting position decoding vector of the sentence output be [1,0,0,0,0,0,1,0,1,0 ]]The end position decoding vector is [0,1,1,0,0,0,0,1,0,1 ]]The head entity is +.>。
And traversing the relation in the relation candidate set for each entity in the candidate head entity set, similarly, calculating tail entities related to the head entities by combining GRU with a double-layer pointer network labeling mode, determining the tail entities under the corresponding relation, and outputting the { head entity, relation and tail entity } triples if the matching is successful.
The model is trained jointly by adopting a mode of sharing parameters during model training, and the objective function of the combination is optimized during training, so that the total loss can be divided into three parts:，，the total loss is the sum of the three parts:here, equal weights are assigned.
In order to verify the effect of the present invention, experiments were performed to evaluate models using Precision (P), recall (R), and F1 values. The calculation methods of the accuracy, recall and F1 values are shown below.
。
The experiment used a WebNLG dataset containing 246 relationships, with the training set containing 5019 triples and the test set containing 703 triples. The data set has rich relationship types and complex entity and relationship matching scenes. The WebNLG dataset triplet distribution statistics are shown in table 1 and the experimental results are shown in table 2.
TABLE 1
TABLE 2
Experimental results show that the triad extraction accuracy rate of the invention on the WebNLG data set is 90.5%, the recall rate is 92.2%, and the F1 value is 91.4%, and compared with other three comparison models, the triad extraction accuracy rate is improved to a certain extent.
The foregoing description is only illustrative of the invention and is not to be construed as limiting the invention. Various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of the present invention, should be included in the scope of the claims of the present invention.
Claims (3)
1. An intelligent manufacturing triplet extraction method based on a relation pointer network is characterized by comprising the following steps of: the intelligent manufacturing triplet extraction method specifically comprises the following steps:
step 1, preparing text data, and defining a relation set R existing in the text data, wherein the text data is input into a text sentence;
step 2, adopting a context pre-training model coding module to code the text data in the step 1;
step 3, predicting candidate relation sets R 'and R' in text data to obtain p, wherein the candidate relation sets R 'and R' are subsets of the relation sets R defined in the step 1 by utilizing a potential relation prediction module rel ；
Step 4, setting a relation threshold p' rel When p in step 3 rel Greater than p' rel When the relationship is considered to be contained in the triples, the corresponding relationship is marked as 1, the rest is marked as 0, the relationship is recorded into a candidate relationship set R ', and the candidate relationship set R' is smaller than the relationship set R;
step 5, predicting the start position and the end position of the head entity by using the head entity decoding module, determining the head entity s existing in the text according to the natural continuity of the span of the head entity, and representing the start position and the end position of the ith token in the input sequence as the head entity s as followsAnd->
Step 6, if obtained in step 5When the value of (2) is larger than a preset threshold value, marking the corresponding token as 1, and regarding the corresponding token as the starting or ending position of the head entity;
step 7, using the head-tail entity alignment module to traverse the relationship in the relationship candidate set R', and calculating whether there is a tail entity o related to the head entity s, wherein the ith token is used as the probability of the beginning position of the tail entityProbability of ending position->Respectively is
h avg Averaging all word vectors for the head entity;
step 8, if obtained in step 7When the value is larger than the preset value, the corresponding token is marked as 1 and is regarded as the starting position of the tail entity, if +.>When the value of the token is larger than a preset value, marking the corresponding token as 1, and regarding the token as the end position of the tail entity, and at the moment, successfully outputting the triples by matching;
the step 5 specifically includes the following steps:
step 5.1, the head entity decoding module inputs the semantic vector h output by the context pre-training model encoding module into the GRU to obtain h' =GRU (h);
step 5.2, inputting two identical double-layer pointer networks for marking the starting position and the ending position of the entity, wherein the nonlinearity of the network is enhanced by using a ReLU activation function in the middle of the two-layer pointer networks;
step 5.3, calculating to obtain probability: and->Representing the start and end positions of the ith token in the input sequence as the head entity, W (·) Is a weight capable of learning, b (·) Representing bias, σ represents activation function;
the step 2 specifically comprises the following steps:
step 2.1, firstly, converting each sub word of the input text sentence S into a word vector and a position vector E= { E 1 ,E 2 ，...，E n Inputting word vectors and position vectors into 12 double-layer converterler blocks to extract features;
step 2.2, the transformers learn information through a multi-head self-attention mechanism, each Transformer transmits the learned information through a layer of fully-connected network, and finally semantic vectors h are output: h= { h 0 ,h 1 ,h 2 ,...,h m ,h m+1 |h i ∈R d×1 D represents the dimension of the last hidden layer of the BERT model;
the step 3 specifically comprises the following steps:
step 3.1, the potential relation prediction module pushes a semantic vector h of the text output by the context pre-training model coding module into a global average pooling layer;
step 3.2, inputting a fully-connected neural network;
step 3.3, finally calculating the probability of each relation by activating the function to obtain p rel ＝σ(w(Pooling(h))+b)。
2. The intelligent manufacturing triplet extraction method based on the relation pointer network according to claim 1, wherein: the model is trained jointly by adopting a mode of sharing parameters during model training, and the objective function of the combination is optimized during training, and the total loss is divided into three parts: ，the total loss is the sum of the three parts: l (L) total ＝αL rel +βL s +γL o 。
3. The intelligent manufacturing triplet extraction method based on the relation pointer network according to claim 1, wherein: the extraction method comprises a context pre-training model coding module, a potential relation prediction module, a head entity decoding module and a head and tail entity alignment module,
the context pre-training model coding module is used for acquiring semantic feature representation of intelligent manufacturing text data, and learning context information of each word in the text by using the context pre-training model to obtain semantic vectors of the text;
the potential relation prediction module is used for constructing an intelligent manufacturing relation candidate set and predicting potential relation existing in the intelligent manufacturing text;
the header entity decoding module is used for realizing entity span extraction and determining the starting and ending positions of the entity;
the head-tail entity alignment module is used for realizing the matching of intelligent manufacturing triples, traversing potential relations in the relation candidate set for each head entity, checking whether tail entities related to the head entity exist or not, and outputting the triples if the tail entities exist.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311726555.XA CN117408247B (en) | 2023-12-15 | 2023-12-15 | Intelligent manufacturing triplet extraction method based on relational pointer network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311726555.XA CN117408247B (en) | 2023-12-15 | 2023-12-15 | Intelligent manufacturing triplet extraction method based on relational pointer network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117408247A CN117408247A (en) | 2024-01-16 |
| CN117408247B true CN117408247B (en) | 2024-03-29 |
Family
ID=89498385
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311726555.XA Active CN117408247B (en) | 2023-12-15 | 2023-12-15 | Intelligent manufacturing triplet extraction method based on relational pointer network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117408247B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078889A (en) * | 2019-12-20 | 2020-04-28 | 大连理工大学 | Method for extracting relationships among medicines based on attention of various entities and improved pre-training language model |
| KR20210018571A (en) * | 2019-08-05 | 2021-02-18 | 강원대학교산학협력단 | An apparatus extracting a relation among multiple entities by using a dual pointer network and a method thereof |
| KR20210057983A (en) * | 2019-11-13 | 2021-05-24 | 창원대학교 산학협력단 | Triple Extraction method using Pointer Network and the extraction apparatus |
| CN113051929A (en) * | 2021-03-23 | 2021-06-29 | 电子科技大学 | Entity relationship extraction method based on fine-grained semantic information enhancement |
| CN113553850A (en) * | 2021-03-30 | 2021-10-26 | 电子科技大学 | Entity relation extraction method based on ordered structure encoding pointer network decoding |
| CN113988075A (en) * | 2021-10-28 | 2022-01-28 | 广东工业大学 | Network security field text data entity relation extraction method based on multi-task learning |
| CN114417839A (en) * | 2022-01-19 | 2022-04-29 | 北京工业大学 | Entity relation joint extraction method based on global pointer network |
| CN114547230A (en) * | 2022-02-24 | 2022-05-27 | 山东大学 | Intelligent administrative law enforcement case information extraction and case law identification method |
| CN114691895A (en) * | 2022-05-31 | 2022-07-01 | 南京航天数智科技有限公司 | Criminal case entity relationship joint extraction method based on pointer network |
| CN114722820A (en) * | 2022-03-21 | 2022-07-08 | 河海大学 | Chinese entity relation extraction method based on gating mechanism and graph attention network |
| CN114764566A (en) * | 2022-04-11 | 2022-07-19 | 中国航空综合技术研究所 | Knowledge element extraction method for aviation field |
| CN115659986A (en) * | 2022-12-13 | 2023-01-31 | 南京邮电大学 | Entity relation extraction method for diabetes text |
| CN116090468A (en) * | 2023-01-09 | 2023-05-09 | 河南科技大学 | Entity relationship joint extraction method and system based on stacked pointer network |
| CN116186237A (en) * | 2023-02-28 | 2023-05-30 | 北京石油化工学院 | Entity relationship joint extraction method based on event cause and effect inference |
| CN116595195A (en) * | 2023-05-30 | 2023-08-15 | 苏州浪潮智能科技有限公司 | Knowledge graph construction method, device and medium |
| CN116663539A (en) * | 2023-06-09 | 2023-08-29 | 广西大学 | Chinese entity and relationship joint extraction method and system based on Roberta and pointer network |
-
2023
- 2023-12-15 CN CN202311726555.XA patent/CN117408247B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20210018571A (en) * | 2019-08-05 | 2021-02-18 | 강원대학교산학협력단 | An apparatus extracting a relation among multiple entities by using a dual pointer network and a method thereof |
| KR20210057983A (en) * | 2019-11-13 | 2021-05-24 | 창원대학교 산학협력단 | Triple Extraction method using Pointer Network and the extraction apparatus |
| CN111078889A (en) * | 2019-12-20 | 2020-04-28 | 大连理工大学 | Method for extracting relationships among medicines based on attention of various entities and improved pre-training language model |
| CN113051929A (en) * | 2021-03-23 | 2021-06-29 | 电子科技大学 | Entity relationship extraction method based on fine-grained semantic information enhancement |
| CN113553850A (en) * | 2021-03-30 | 2021-10-26 | 电子科技大学 | Entity relation extraction method based on ordered structure encoding pointer network decoding |
| CN113988075A (en) * | 2021-10-28 | 2022-01-28 | 广东工业大学 | Network security field text data entity relation extraction method based on multi-task learning |
| CN114417839A (en) * | 2022-01-19 | 2022-04-29 | 北京工业大学 | Entity relation joint extraction method based on global pointer network |
| CN114547230A (en) * | 2022-02-24 | 2022-05-27 | 山东大学 | Intelligent administrative law enforcement case information extraction and case law identification method |
| CN114722820A (en) * | 2022-03-21 | 2022-07-08 | 河海大学 | Chinese entity relation extraction method based on gating mechanism and graph attention network |
| CN114764566A (en) * | 2022-04-11 | 2022-07-19 | 中国航空综合技术研究所 | Knowledge element extraction method for aviation field |
| CN114691895A (en) * | 2022-05-31 | 2022-07-01 | 南京航天数智科技有限公司 | Criminal case entity relationship joint extraction method based on pointer network |
| CN115659986A (en) * | 2022-12-13 | 2023-01-31 | 南京邮电大学 | Entity relation extraction method for diabetes text |
| CN116090468A (en) * | 2023-01-09 | 2023-05-09 | 河南科技大学 | Entity relationship joint extraction method and system based on stacked pointer network |
| CN116186237A (en) * | 2023-02-28 | 2023-05-30 | 北京石油化工学院 | Entity relationship joint extraction method based on event cause and effect inference |
| CN116595195A (en) * | 2023-05-30 | 2023-08-15 | 苏州浪潮智能科技有限公司 | Knowledge graph construction method, device and medium |
| CN116663539A (en) * | 2023-06-09 | 2023-08-29 | 广西大学 | Chinese entity and relationship joint extraction method and system based on Roberta and pointer network |
Non-Patent Citations (7)
| Title |
|---|
| BGPNRE:一种基于BERT的全局指针网络实体关系联合抽取方法;邓亮 等;计算机科学;20230315;第50卷(第3期);42-48 * |
| Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction;Tapas Nayak et al;Proceedings of the AAAI Conference on Artificial Intelligence;20200403;第34卷(第5期);8528-8535 * |
| Relation Extraction Model Based on Global Pointer and Potential Relation Embedding;Han Yang et al;IIHMSP 2022：Advances in Intelligent Information Hiding and Multimedia Signal Processing;20230524;205-216 * |
| 中文信息抽取及在煤矿安全领域应用研究;冯琳;中国优秀硕士学位论文全文数据库工程科技Ⅰ辑;20230215(第2期);正文第37-46页 * |
| 基于对抗学习与全局指针生成网络的实体关系抽取研究;胡佳;中国优秀硕士学位论文全文数据库信息科技辑;20221115(第11期);I138-307 * |
| 旅游领域嵌套实体和重叠关系联合抽取模型BPNRel;陈赟 等;东北师大学报(自然科学版);20230914;第55卷(第3期);64-74 * |
| 面向复杂文本场景的关系抽取方法研究;刘欢;中国优秀硕士学位论文全文数据库信息科技辑;20220615(第6期);I138-669 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117408247A (en) | 2024-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11941522B2 (en) | Address information feature extraction method based on deep neural network model | |
| CN113177132B (en) | Image retrieval method based on depth cross-modal hash of joint semantic matrix | |
| CN110647904A (en) | Cross-modal retrieval method and system based on unmarked data migration | |
| CN112818676A (en) | Medical entity relationship joint extraction method | |
| CN110888980A (en) | Implicit discourse relation identification method based on knowledge-enhanced attention neural network | |
| CN113255366B (en) | Aspect-level text emotion analysis method based on heterogeneous graph neural network | |
| Shi et al. | GAEN: graph attention evolving networks | |
| CN115526236A (en) | Text network graph classification method based on multi-modal comparative learning | |
| CN115687609A (en) | Zero sample relation extraction method based on Prompt multi-template fusion | |
| CN114564563A (en) | End-to-end entity relationship joint extraction method and system based on relationship decomposition | |
| CN114564950A (en) | Electric Chinese named entity recognition method combining word sequence | |
| CN117273134A (en) | Zero-sample knowledge graph completion method based on pre-training language model | |
| CN116245107B (en) | Electric power audit text entity identification method, device, equipment and storage medium | |
| CN117407532A (en) | Method for enhancing data by using large model and collaborative training | |
| CN115600602B (en) | Method, system and terminal device for extracting key elements of long text | |
| CN117408247B (en) | Intelligent manufacturing triplet extraction method based on relational pointer network | |
| CN111444316A (en) | Knowledge graph question-answer oriented composite question analysis method | |
| Xue et al. | Hard sample mining for the improved retraining of automatic speech recognition | |
| CN111259106A (en) | Relation extraction method combining neural network and feature calculation | |
| CN116680407A (en) | Knowledge graph construction method and device | |
| CN115408536A (en) | Knowledge graph complementing method based on context information fusion | |
| CN114298052A (en) | Entity joint labeling relation extraction method and system based on probability graph | |
| Wen et al. | Few-shot named entity recognition with joint token and sentence awareness | |
| CN114996407B (en) | Remote supervision relation extraction method and system based on packet reconstruction | |
| Cai et al. | Transformer-Based BiLSTM for Aspect-Level Sentiment Classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |