CN111428481A - Entity relation extraction method based on deep learning - Google Patents
Entity relation extraction method based on deep learning Download PDFInfo
- Publication number
- CN111428481A CN111428481A CN202010222471.2A CN202010222471A CN111428481A CN 111428481 A CN111428481 A CN 111428481A CN 202010222471 A CN202010222471 A CN 202010222471A CN 111428481 A CN111428481 A CN 111428481A
- Authority
- CN
- China
- Prior art keywords
- neural network
- stm
- learning
- candidate
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 45
- 238000013135 deep learning Methods 0.000 title claims abstract description 16
- 238000013528 artificial neural network Methods 0.000 claims abstract description 42
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 32
- 239000013598 vector Substances 0.000 claims abstract description 5
- 230000002457 bidirectional effect Effects 0.000 claims description 4
- 230000001537 neural effect Effects 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 125000004122 cyclic group Chemical group 0.000 abstract description 7
- 239000000284 extract Substances 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 16
- 238000010801 machine learning Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000013106 supervised machine learning method Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an entity relation extraction method based on deep learning, which comprises the steps of extracting entity relations by using a convolutional neural network and a cyclic neural network, adding word vector characteristics, position characteristics, local characteristics, sequence characteristics and the like in texts on CNN and L STM neural networks, combining the convolutional neural network and the cyclic neural network together, respectively combining the combined models in series and parallel to extract the entity relations, learning characteristics from different angles to obtain more complete learning capability, combining the convolutional neural network and the cyclic neural network together on the basis of neural networks with various different structures, respectively using the modes of series and parallel to automatically extract samples, finally combining a plurality of models based on deep learning, combining the models together to extract the entity relations by using a plurality of previously designed relation extraction models, and further selecting the entity relations with better samples.
Description
Technical Field
The invention relates to the field of entity relationship extraction, in particular to an entity relationship extraction method based on deep learning.
Background
The task of entity relation extraction is to perform semantic recognition on entity pairs in a text, and judge whether a relation exists between the entity pairs or what type of relation the entity pairs belong to according to meanings of the entity pairs in the text and sentences. Early entity relationship extraction mainly adopted a pattern matching method. Rule-based methods analyze some implicit features in the text, rules and patterns are manually defined by participating professionals, and relationships are discovered and matched by using relationship patterns. Rule-based methods require a person with specialized knowledge in the professional domain to manually write the rules, requiring a large amount of human input. When the extraction rule is applied to other fields, the extraction rule is limited and has strong professionalism and limitation. This early rule-based approach, which relied primarily, has achieved good results in a number of areas and areas of expertise.
With the development and application of machine learning, entity relationship extraction has a new research method, and a research method for extracting entity relationship by using machine learning is widely concerned. The machine learning method is widely applied to the extraction of the entity relationship, unsupervised, weakly supervised and supervised machine learning methods are correspondingly researched in the extraction of the entity relationship, the unsupervised and weakly supervised learning method does not need too many manually marked sample data, the dependence on the sample data label is reduced in training and learning, but the training process is easily interfered by noise, and the extraction performance of the entity relationship is reduced. Besides unsupervised and weakly supervised entity relation extraction methods, the extraction method based on supervised learning is researched and applied in multiple fields, and a better extraction effect is achieved. The entity relationship extraction method with supervised learning relatively needs more energy input, the extraction model depends on more labeled data sets, the training process of the entity relationship extraction model can be smoothly carried out only by sufficient labeled data, and the labeled data is obtained with time and labor waste, so that the general premise with supervised learning is to have enough labor input.
Most of entity relation extraction methods based on machine learning need to extract features from texts, perform text vocabulary analysis, grammar analysis and the like, and need to research automatic extraction of entity relations by means of language knowledge and natural language processing tools. The professional terms, noun analysis and annotation data of many professional fields also need the participation of persons with rich professional knowledge, and some machine learning methods based on language features cannot be simply applied to other fields.
Disclosure of Invention
The invention aims to provide an entity relation extraction method based on deep learning aiming at the defects in the prior art.
In order to achieve the above object, the present invention provides an entity relationship extraction method based on deep learning, which includes:
extracting features from the input sample by adopting a CNN neural network and learning to obtain a first candidate relation type in the mode;
extracting features from the input samples by adopting a B L STM neural network and learning to obtain a second candidate relation type in the mode;
respectively extracting features from input samples by adopting a CNN neural network and a B L STM neural network, learning, and sequentially and respectively inputting the learned information to the B L STM neural network and the CNN neural network for further learning so as to respectively obtain a third candidate relationship type and a fourth candidate relationship type;
extracting and learning features from the input sample by adopting a CNN neural network and a B L STM neural network, and splicing the learned features to obtain a fifth candidate relationship type;
and evaluating the first candidate relationship type, the second candidate relationship type, the third candidate relationship type, the fourth candidate relationship type and the fifth candidate relationship type, and selecting the candidate relationship type with the highest score as a final relationship classification result of the sample.
Further, if there is more than one candidate relationship type with the highest score, one type is randomly selected from all the candidate relationship types with the highest score as the classification result.
Furthermore, the CNN neural network extracts local features by adopting sliding windows of multiple sizes, performs maximum pooling sampling on the learned features, and trains an entity relationship extraction model through calculation processes such as full connection layer, softmax operation and back propagation.
Further, the input sample is input in a bidirectional L STM statement and is in an entity relationIn the system extraction model, sample data is input into L STM neural units according to the original sequence of words in sentences, and the state h of each timetDepending on the state of the previous time and the state of the current time, htThe STM is formed by splicing the outputs of a forward sequence and a reverse sequence at the time t of B L STM, and the specific steps are as follows:
wherein the content of the first and second substances,
for sentences input in forward order to the output of B L STM at time t,
the output in reverse order, to let the B L STM node feature learn the forward and backward sequences simultaneously.
Further, the feature extraction from the input sample includes a word vector feature, a position feature, a local feature and a sequence feature.
Has the advantages that: the method utilizes the connection between the characteristics of locality, sequence and the like of text sentences and the text relation categories, exerts the characteristics of two types of neural network structures by combining a convolutional neural network and a cyclic neural network, and combines a plurality of models to extract entity relations so as to strengthen the autonomous learning capability of the models on the characteristics of all aspects of texts; according to the characteristics of the two types of neural networks, a plurality of combined models which connect the two types of neural networks in parallel and in series are designed, the network structure forms of a single neural network model and the combined models which connect the two types of neural networks in series and in parallel are different, the learning ability may be emphasized on a certain aspect, the learned characteristics of the single model are single, the neural network models with different structures are combined, the learning characteristics are learned from different angles, and more full and comprehensive learning ability is obtained. Except that the convolutional neural network and the cyclic neural network are adopted to extract the entity relationship respectively, the convolutional neural network and the cyclic neural network are combined together on the basis of neural networks with various different structures, and the samples are extracted automatically in a series connection mode and a parallel connection mode respectively. And finally combining a plurality of models based on deep learning, combining a plurality of previously designed relationship extraction models together for entity relationship extraction by using the models, and further selecting an entity relationship with a better sample. The neural unit is adopted to simulate the process of human brain learning, and the automatic extraction of the relationship is completed by constructing structures such as a convolutional neural network, a cyclic neural network and the like. Compared with the traditional machine learning-based method, the entity relation extraction method based on deep learning does not need manual feature extraction, and the deep learning algorithm has the capability of automatic learning algorithm.
Drawings
FIG. 1 is a schematic diagram of a multi-model federated entity relationship extraction process;
FIG. 2 is a schematic diagram of a tandem combinatorial model of CNN tandem B L STM;
FIG. 3 is a schematic diagram of a tandem combinatorial model of B L STM tandem CNN;
fig. 4 is a schematic diagram of a parallel combinatorial model of CNN tandem B L STM.
Detailed Description
The present invention will be further illustrated with reference to the accompanying drawings and specific examples, which are carried out on the premise of the technical solution of the present invention, and it should be understood that these examples are only for illustrating the present invention and are not intended to limit the scope of the present invention.
As shown in fig. 1 to 4, an embodiment of the present invention provides an entity relationship extraction method based on deep learning, including:
and extracting features from the input sample by adopting a CNN neural network and learning to obtain a first candidate relation type in the mode. Consider a CNN neural network with multiple windows and entity pair locations, with location features taking the distance of individual words from the entity pair e1 and e2, and sample input vectors of sentence word features and location features stitched together. The input layer is input into the model, the rolling layer adopts a plurality of sliding windows to extract local features, the learned features are sampled in a largest pooling mode, and then the entity relation extraction model is trained through calculation processes of a full connection layer, softmax operation, back propagation and the like.
Using Bidirectional L STM (Bidirectional L ong Short-Term Memory, B L STM) to input sample sentences into the model, in the entity relation extraction model, sample data is input into L STM neural unit according to the original sequence of words in the sentences, and the state h of each momenttDepending on the state of the previous time and the state of the current time, the following is specific:
wherein the content of the first and second substances,
for sentences input in forward order to the output of B L STM at time t,
output in reverse order, htThe method is characterized in that the method is formed by splicing the outputs of a forward sequence and a reverse sequence at the time t of a B L STM, and a B L STM node simultaneously performs feature learning on a forward sequence and a backward sequence.
And respectively extracting features from the input samples by adopting a CNN neural network and a B L STM neural network, learning, and sequentially and respectively inputting the learned information to the B L STM neural network and the CNN neural network for further learning so as to respectively obtain a third candidate relationship type and a fourth candidate relationship type.
And respectively extracting and learning features from the input samples by adopting a CNN neural network and a B L STM neural network, and splicing the learned features to obtain a fifth candidate relationship type.
For the first candidate relationship type, the second candidate relationship type, the third candidate relationship type, the fourth candidate relationship type and the fifth candidate relationship typeThe method comprises the steps of evaluating candidate relation types, selecting the candidate relation type with the highest score as a final relation classification result of a sample, specifically, counting the prediction results of CNN, B L STM, CNN-S-B L STM (CNN is connected with B L STM in series), B L STM-S-CNN (B L STM is connected with CNN in series) and CNN-P-B L STM (CNN is connected with B L STM in parallel), adding 1 score to the relation type with the highest probability predicted by each model, and finally selecting the relation type with the highest score as the final relation classification result of the sample2、R2、R1、R3、R2Then select R2Score highest, R2As the final result of the relationship classification.
If there is more than one candidate relationship type with the highest score in the present embodiment, one type is randomly selected from all the candidate relationship types with the highest score as the result of classification. If the model prediction types are respectively as follows: r2、R2、R3、R1、R3Wherein R is2And R3The relationship types score the same and most, then R is chosen randomly2、R3One of which is the result of a classification of the relationship.
The features extracted from the input samples by the CNN neural network and the B L STM neural network respectively comprise word vector features, position features, local features, sequence features and the like, and different influences of the features on the performance of the entity relationship extraction model are analyzed.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that other parts not specifically described are within the prior art or common general knowledge to those of ordinary skill in the art. Without departing from the principle of the invention, several improvements and modifications can be made, and these improvements and modifications should also be construed as the scope of the invention.
Claims (5)
1. An entity relationship extraction method based on deep learning is characterized by comprising the following steps:
extracting features from the input sample by adopting a CNN neural network and learning to obtain a first candidate relation type in the mode;
extracting features from the input samples by adopting a B L STM neural network and learning to obtain a second candidate relation type in the mode;
respectively extracting features from input samples by adopting a CNN neural network and a B L STM neural network, learning, and sequentially and respectively inputting the learned information to the B L STM neural network and the CNN neural network for further learning so as to respectively obtain a third candidate relationship type and a fourth candidate relationship type;
extracting and learning features from the input sample by adopting a CNN neural network and a B L STM neural network, and splicing the learned features to obtain a fifth candidate relationship type;
and evaluating the first candidate relationship type, the second candidate relationship type, the third candidate relationship type, the fourth candidate relationship type and the fifth candidate relationship type, and selecting the candidate relationship type with the highest score as a final relationship classification result of the sample.
2. The deep learning-based entity relationship extraction method according to claim 1, wherein if there is more than one type of the highest-score candidate relationship types, one type is randomly selected from all the highest-score candidate relationship types as the classification result.
3. The entity relationship extraction method based on deep learning of claim 1, wherein the CNN neural network adopts sliding windows of multiple sizes to extract local features, performs maximum pooling sampling on the learned features, and then trains the entity relationship extraction model through computation flows such as full connectivity, softmax operation, and back propagation.
4. The entity relationship extraction method based on deep learning of claim 1, wherein the input samples are inputted in a bidirectional L STM sentence, and in the entity relationship extraction model, the sample data are inputted into L STM neural units according to the original sequence of words in the sentence, and the state h at each time istDependent on previous timeThe state of the moment and the state of the current moment, htThe STM is formed by splicing the outputs of a forward sequence and a reverse sequence at the time t of B L STM, and the specific steps are as follows:
wherein the content of the first and second substances,
for sentences input in forward order to the output of B L STM at time t,
the output in reverse order, to let the B L STM node feature learn the forward and backward sequences simultaneously.
5. The entity relationship extraction method based on deep learning of claim 1, wherein the feature extraction from the input sample comprises word vector feature, position feature, local feature and sequence feature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010222471.2A CN111428481A (en) | 2020-03-26 | 2020-03-26 | Entity relation extraction method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010222471.2A CN111428481A (en) | 2020-03-26 | 2020-03-26 | Entity relation extraction method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111428481A true CN111428481A (en) | 2020-07-17 |
Family
ID=71548865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010222471.2A Pending CN111428481A (en) | 2020-03-26 | 2020-03-26 | Entity relation extraction method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111428481A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111898364A (en) * | 2020-07-30 | 2020-11-06 | 平安科技（深圳）有限公司 | Neural network relation extraction method, computer device and readable storage medium |
| CN111931719A (en) * | 2020-09-22 | 2020-11-13 | 苏州科达科技股份有限公司 | High-altitude parabolic detection method and device |
| CN111985245A (en) * | 2020-08-21 | 2020-11-24 | 江南大学 | Attention cycle gating graph convolution network-based relation extraction method and system |
| CN114842982A (en) * | 2022-07-06 | 2022-08-02 | 广东省科技基础条件平台中心 | Knowledge expression method, device and system for medical information system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108733792A (en) * | 2018-05-14 | 2018-11-02 | 北京大学深圳研究生院 | A kind of entity relation extraction method |
| CN110704890A (en) * | 2019-08-12 | 2020-01-17 | 上海大学 | Automatic text causal relationship extraction method fusing convolutional neural network and cyclic neural network |
-
2020
- 2020-03-26 CN CN202010222471.2A patent/CN111428481A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108733792A (en) * | 2018-05-14 | 2018-11-02 | 北京大学深圳研究生院 | A kind of entity relation extraction method |
| CN110704890A (en) * | 2019-08-12 | 2020-01-17 | 上海大学 | Automatic text causal relationship extraction method fusing convolutional neural network and cyclic neural network |
Non-Patent Citations (1)
| Title |
|---|
| 张晓斌 等: "基于CNN 和双向LSTM 融合的实体关系抽取" * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111898364A (en) * | 2020-07-30 | 2020-11-06 | 平安科技（深圳）有限公司 | Neural network relation extraction method, computer device and readable storage medium |
| CN111898364B (en) * | 2020-07-30 | 2023-09-26 | 平安科技（深圳）有限公司 | Neural network relation extraction method, computer equipment and readable storage medium |
| CN111985245A (en) * | 2020-08-21 | 2020-11-24 | 江南大学 | Attention cycle gating graph convolution network-based relation extraction method and system |
| CN111985245B (en) * | 2020-08-21 | 2024-03-12 | 江南大学 | Relationship extraction method and system based on attention cycle gating graph convolution network |
| CN111931719A (en) * | 2020-09-22 | 2020-11-13 | 苏州科达科技股份有限公司 | High-altitude parabolic detection method and device |
| CN114842982A (en) * | 2022-07-06 | 2022-08-02 | 广东省科技基础条件平台中心 | Knowledge expression method, device and system for medical information system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110188343B (en) | Multi-mode emotion recognition method based on fusion attention network | |
| CN108763284B (en) | Question-answering system implementation method based on deep learning and topic model | |
| CN110083833B (en) | Method for analyzing emotion by jointly embedding Chinese word vector and aspect word vector | |
| CN106156003B (en) | A kind of question sentence understanding method in question answering system | |
| CN107943784B (en) | Relationship extraction method based on generation of countermeasure network | |
| CN111428481A (en) | Entity relation extraction method based on deep learning | |
| CN111414461B (en) | Intelligent question-answering method and system fusing knowledge base and user modeling | |
| CN110609891A (en) | Visual dialog generation method based on context awareness graph neural network | |
| CN110134946B (en) | Machine reading understanding method for complex data | |
| CN111651593A (en) | Text emotion analysis method based on word vector and word vector mixed model | |
| CN111144448A (en) | Video barrage emotion analysis method based on multi-scale attention convolutional coding network | |
| CN110765775A (en) | Self-adaptive method for named entity recognition field fusing semantics and label differences | |
| CN110598005A (en) | Public safety event-oriented multi-source heterogeneous data knowledge graph construction method | |
| CN110222163A (en) | A kind of intelligent answer method and system merging CNN and two-way LSTM | |
| CN110750635B (en) | French recommendation method based on joint deep learning model | |
| CN108121702A (en) | Mathematics subjective item reads and appraises method and system | |
| CN110415071A (en) | A kind of competing product control methods of automobile based on opining mining analysis | |
| CN110717330A (en) | Word-sentence level short text classification method based on deep learning | |
| CN113221571A (en) | Entity relation joint extraction method based on entity correlation attention mechanism | |
| CN114417851A (en) | Emotion analysis method based on keyword weighted information | |
| CN112732921A (en) | False user comment detection method and system | |
| CN116028604A (en) | Answer selection method and system based on knowledge enhancement graph convolution network | |
| CN114021584B (en) | Knowledge representation learning method based on graph convolution network and translation model | |
| CN112560440A (en) | Deep learning-based syntax dependence method for aspect-level emotion analysis | |
| CN111984790A (en) | Entity relation extraction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |
|
| RJ01 | Rejection of invention patent application after publication |