CN112487203A - Relation extraction system integrated with dynamic word vectors - Google Patents
Relation extraction system integrated with dynamic word vectors Download PDFInfo
- Publication number
- CN112487203A CN112487203A CN202011387516.8A CN202011387516A CN112487203A CN 112487203 A CN112487203 A CN 112487203A CN 202011387516 A CN202011387516 A CN 202011387516A CN 112487203 A CN112487203 A CN 112487203A
- Authority
- CN
- China
- Prior art keywords
- sentence
- vector
- vectors
- relation
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000013598 vector Substances 0.000 title claims abstract description 135
- 238000000605 extraction Methods 0.000 title claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 238000003062 neural network model Methods 0.000 claims description 12
- 230000007246 mechanism Effects 0.000 claims description 11
- 239000000284 extract Substances 0.000 claims description 10
- 230000002457 bidirectional effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims 1
- 238000002372 labelling Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 235000002595 Solanum tuberosum Nutrition 0.000 description 1
- 244000061456 Solanum tuberosum Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 235000012015 potatoes Nutrition 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The invention provides an entity relation extraction method and system integrated with a dynamic word vector technology. The system utilizes a remote supervision method to correspond the existing knowledge base to rich unstructured data so as to generate a large amount of training data, thereby relieving the problem of insufficient artificial labeling linguistic data, reducing the dependency on the labeling data and effectively reducing the labor cost. In order to acquire characteristic information among entities as much as possible, a basic framework of the model adopts a segmented convolutional neural network; and the semantic information of the example sentence is further extracted by integrating the dynamic word vector technology.
Description
Technical Field
The invention relates to the field of information extraction, in particular to a method for mining semantic relations between entities.
Background
The information extraction aims at extracting structured information from large-scale unstructured or semi-structured natural language texts, and the main tasks include entity extraction, relation extraction and event extraction. The main content of the Relation Extraction (RE) research is to extract semantic relations between entities from text contents, and extract deep Relation structures between entities by using a Relation Extraction technology, so that the RE Relation Extraction method has profound theoretical significance and great research value, and is also the basic work for optimizing a search engine, establishing a knowledge graph and developing an intelligent question-answering system.
Practice proves that the supervised learning method can extract more effective features, the accuracy rate and the recall rate are higher, but the supervised learning method seriously depends on natural language processing labels such as part of speech labels, syntactic analysis and the like to provide classification features. The natural language processing and labeling tool often has a large number of errors, and the errors are continuously propagated and amplified in the relationship extraction system, so that the effect of relationship extraction is finally influenced. With the rapid development of deep learning, the neural network model can automatically learn sentence characteristics without relying on complex characteristic engineering. Much research work has tended to use neural network models to solve the problem of relationship extraction.
Using neural network models faces two major problems: (1) without enough labeling data, the coverage rate of the training data set on the entities and entity relationships is low, and the universality is poor. And a great deal of time and energy are needed for manually marking the training data; (2) because word usage is semantically and grammatically complex and variable, existing models are "static" using pre-trained word vectors and cannot change as the language environment changes. Thus, its characterization capability has certain limitations.
Disclosure of Invention
In view of the above, the present invention provides a relation extraction model and system merged into a dynamic word vector, which automatically construct a large amount of training data by aligning a knowledge base with an unstructured text by using a remote supervision method, reduce dependence of the model on manual labeling data, and enhance the adaptability of the model across fields. And an attention mechanism is introduced to eliminate the influence of noise data generated by remote supervision. And finally, a dynamic word vector technology is used, so that the accuracy rate of relation extraction is improved. Thereby overcoming, at least to some extent, one or more of the problems due to limitations and insufficient corpus numbers of the related art.
In order to achieve the above object, according to one aspect of the present invention, the present invention provides the following technical solutions: a method for extracting a relation fused into a dynamic word vector comprises the following steps:
part one: obtaining a dynamic word vector: firstly, pre-training a deep bidirectional Language Model (LM) on a large text corpus, and then using a function learned according to the internal state of the model as a word vector; such a word vector is not invariant, but varies over time depending on the context. The part uses an existing ELMO model or BERT model as a generation model of the word vector.
The ELMO model, based on a bi-directional language model, represents one way of word vectors using linear combinations between layers.
The BERT model is represented by a transform's bidirectional encoder. Unlike other language representation models in the near future, BERT aims to pre-train the deep bi-directional representation by jointly adjusting the context in all layers. Therefore, the pre-trained BERT representation can be fine-tuned through an additional output layer, and is suitable for building the most advanced model of a wide range of tasks.
And part two: and training by adopting a segmented neural network model introducing an attention mechanism, respectively calculating weights corresponding to the word vectors according to the obtained word vectors, and splicing to obtain sentence vector representation. The same entity pair may represent different relationships in different statements. Therefore, when using remote supervision, extracting statements by entity pair inevitably introduces noise data. This section is to greatly reduce the effect of noise on the relationship extraction.
More specifically: the input of the segmented neural network model is a training sentence containing an entity pair; the training sentence is divided into three sections by the entity, and the three sections are mapped into corresponding three sections of word vectors; using a convolutional neural network to respectively extract the features of the three segments of word vectors to obtain three segments of feature vectors, and respectively calculating the weights and relationship vectors of the three segments of feature vectors; specifically, the weight and the relation vector of the three sections of feature vectors are calculated by adopting the following formula:
bi＝Conv(vectorsentence_i),i＝1,2,3 (1)
wi＝wa T(tanh[bi；vrelation])+ba (2)
vrelation＝vent1-vent2 (4)
wherein, biThe feature vector, representing the i-th part of the sentence, extracted by the convolutional neural networksentence_iRepresenting the i-th partial word vector of the sentence sensor; w is aiRepresenting the fusion relation vector of the ith part of feature vectors to obtain a new feature vector wa、baIs a model parameter; alpha is alphaiIs a feature vector weight, vrelationIs a relation vector; v. ofent1And vent2Respectively representing a first entity vector and a second entity vector;
multiplying the three sections of feature vectors by corresponding weights according to the obtained weights of the feature vectors, and splicing to obtain a final sentence vector representation, wherein the sentence vector is classified by softmax after passing through a full connection layer, and the method specifically comprises the following steps:
s＝contact[b1·α1；b2·α2；b3·α3] (5)
c＝softmax(w·s+b) (6)
wherein s represents a new vector spliced after each w is subjected to weighted summation; c represents a vector of categories.
According to another aspect of the present invention, the present invention further provides a system for extracting relationships merged into a dynamic word vector, the system comprising:
and the corpus collection module extracts an entity pair from the manually constructed triple knowledge base in a remote supervision mode, and extracts sentences containing the entity pair from the network text by taking the entity pair as a keyword to serve as training corpuses under the relationship. The module is essentially a web crawler module, automatically selects entity pairs according to the principle of language material category balance, and uses keywords thereof to crawl corresponding language materials as training language materials of the relationship; in addition, the module also has a data cleaning task, filters invalid non-text data and stores the data into a database according to the text length of a sentence;
the dynamic word vector production module converts the sentence characters into vectors, and the dynamic word vectors are generated according to different sentences, so that a sentence length threshold value is set, if the sentence length threshold value is larger than the threshold value, the sentence length threshold value is intercepted, and if the sentence length threshold value is smaller than the threshold value, the sentence length threshold value is supplemented; inputting the aligned sentences into a word vector generation module, outputting word vectors of each word in the sentences, splicing to obtain sentence vectors of the sentences, and sending the sentence vectors into a relation receiving module for training;
and the relation extraction module extracts the relation of the entity pair in the sentence, takes the sentence vector made by the module II as the input of the module and finally outputs the category of the sentence.
The relation extraction module adopts a segmented neural network model, and the input of the segmented neural network model is a training sentence containing an entity pair;
the training sentence is divided into three sections by the entity, and the three sections are mapped into corresponding three sections of word vectors;
and respectively extracting the features of the three sections of word vectors by using a convolutional neural network to obtain three sections of feature vectors, and respectively calculating the weights and relationship vectors of the three sections of feature vectors.
Compared with the prior art, the embodiment of the invention has the beneficial effects that: the method utilizes a remote supervision method to correspond the existing knowledge base to rich unstructured data so as to generate a large amount of training data, thereby relieving the problem of insufficient linguistic data of manual labeling. In order to obtain characteristic information among entities as much as possible, a segmented Convolutional Neural network (PIecewise Convolutional Neural Networks) is adopted as a basic framework of the model; and the semantic information of the example sentence is further extracted by integrating the dynamic word vector technology. Finally, an Attention mechanism is also used to reduce the effects of noisy data. The invention adopts a remote supervision mode to automatically acquire the training corpus, combines an attention mechanism, eliminates error labels generated when the corpus is automatically acquired, and finally integrates a dynamic word vector technology, thereby further improving the accuracy and recall rate of entity relationship extraction.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a schematic diagram of a segmented neural network model incorporating the attention mechanism of the present invention.
FIG. 2 is a schematic diagram of a dynamic word vector production process according to the present invention.
FIG. 3 is a schematic diagram of a data flow of a system for extracting relationships incorporated into dynamic word vectors according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, and "a" and "an" generally include at least two, but do not exclude at least one, unless the context clearly dictates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
It should be understood that although the terms first, second, third, etc. may be used to describe … … in embodiments of the present invention, these … … should not be limited to these terms. These terms are used only to distinguish … …. For example, the first … … can also be referred to as the second … … and similarly the second … … can also be referred to as the first … … without departing from the scope of embodiments of the present invention.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.
In addition, the sequence of steps in each method embodiment described below is only an example and is not strictly limited. Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the segmented neural network model with attention mechanism introduced in the present invention mainly includes:
the input to the model is a training statement that contains some pair of entities, for example: monday, Youka and potatoes are combined to form a first large video website.
The main body of the neural network is a convolutional neural network, and the sentence is divided into three segments by two entities in the sentence and is mapped into corresponding word vectors. And (4) respectively extracting the features of the three segments of vectors by using a convolutional neural network. When the attention mechanism is adopted, the weight of the three sections of feature vectors and the relation vector is calculated by the following formula:
bi＝Conv(vectorsentence_i),i＝1,2,3 (1)
wi＝wa T(tanh[bi；vrelation])+ba (2)
vrelation＝vent1-vent2 (4)
wherein, b in the formula (1)iThe feature vector, representing the i-th part of the sentence, extracted by the convolutional neural networksentence_iRepresenting the i-th partial word vector of the sentence sensor; formula (2) fuses the relationship vectors so that the feature vectors carry more information, wiRepresenting the fusion relation vector of the ith part of feature vectors to obtain a new feature vector wa、baIs a model parameter; equation (3) calculates the corresponding weight, alpha, of the feature vector at the attribute leveliIs a feature vector weight, vrelationIs a relation vector; formula (4) calculates a relationship vector by subtracting the corresponding first entity vector and second entity vector, vent1And vent2Respectively representing a first entity vector and a second entity vector, randomly initializing the entity vectors, changing with model training, initializing a matrix in advance, wherein each row of the matrix represents an entityAnd vector, each entity is marked with a label, and the corresponding entity vector is found according to the label.
Multiplying the three sections of feature vectors by corresponding weights according to the obtained weights of the feature vectors, and splicing to obtain a final sentence vector representation, wherein the sentence vector is classified by softmax after passing through a full connection layer, and the method specifically comprises the following steps:
s＝contact[b1·α1；b2·α2；b3·α3] (5)
c＝softmax(w·s+b) (6)
wherein s represents a new vector spliced after each w is subjected to weighted summation; c represents a vector of categories.
Fig. 2, which shows exemplarily the generation process of a dynamic word vector.
The core part of the dynamic word vector generation module is an ELMO/BERT model, the training corpus takes 100 sentences as a batch of training batches, and the sentence length is set to 200 words. If the word number of the sentence is larger than the threshold, the part larger than 200 is truncated to only retain the content of the first 200 words, and if the word number of the sentence is smaller than the threshold, the sentence is filled with the character "< pad >". The output of the model is the word vector corresponding to the training corpus.
FIG. 3 illustrates a relationship extraction system that incorporates dynamic word vectors.
The system consists of a corpus collection module, a dynamic word vector generation module and a relation extraction algorithm module.
The core of the corpus collection module is a remote supervision module which randomly extracts relation triples from a knowledge base (the knowledge base is constructed by domain experts and is stored by triples in the form of (entity 1, entity 2, relation)). And using the entity pair in the triple as a keyword, and using a crawler system to crawl sentences containing the entity pair from massive network texts, wherein the sentences are used as training corpora and are stored in a database. The domain experts update the knowledge base at regular time, relation triples with prior knowledge are continuously added, the acquisition module continuously operates in a 24 x 7 mode, corpora of corresponding entity pairs are automatically extracted from massive unstructured texts, and the corpora are stored in the database.
The dynamic word vector generation module is a module for converting the sentence characters into vectors. When the model is trained, firstly, sentences are extracted from the database, sent to the ELMO/BERT model, output corresponding dynamic word vectors and sent to the relation extraction module for training.
The relation extraction module is a core part of the system, and the model used by the module is a segmented neural network model introducing an attention mechanism. The model is trained over a large number of training sentences and then saved. Used to predict the entity relationship category of the new text statement.
The system utilizes a remote supervision method to correspond the existing knowledge base to rich unstructured data so as to generate a large amount of training data, thereby relieving the problem of insufficient artificial labeling linguistic data, reducing the dependency on the labeling data and effectively reducing the labor cost. In order to obtain characteristic information among entities as much as possible, a segmented Convolutional Neural network (PIecewise Convolutional Neural Networks) is adopted as a basic framework of the model; and the semantic information of the example sentences is further extracted by integrating the dynamic word vector technology popular in the academic world at present. Finally, an Attention mechanism is also used to reduce the effects of noisy data. The invention adopts a remote supervision mode to automatically acquire the training corpus, combines an attention mechanism, eliminates error labels generated when the corpus is automatically acquired, and finally integrates a dynamic word vector technology, thereby further improving the accuracy and recall rate of entity relationship extraction.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (5)
1. A system for extracting relationships fused into dynamic word vectors, the system comprising:
the corpus collection module randomly extracts the relation triplets from the knowledge base in a remote supervision mode, crawls sentences containing entity pairs from massive network texts by using a crawler system by taking the entity pairs in the triplets as keywords, and stores the sentences as training corpuses in a database;
the dynamic word vector production module is used for converting the sentence characters into vectors, firstly extracting the sentence linguistic data from the database, sending the sentence linguistic data into a deep bidirectional language model, outputting corresponding dynamic word vectors, further calculating to obtain sentence vectors, and sending the sentence vectors into the relation extraction module for training;
the relation extraction module uses a model which is a segmented neural network model introducing an attention mechanism, trains the model through a large number of training sentences, and then stores the model for predicting entity relation categories of new text sentences;
inputting a training sentence containing an entity pair into the segmented neural network model;
the training sentence is divided into three sections by the entity, and the three sections are mapped into corresponding three sections of word vectors;
using a convolutional neural network to respectively extract the features of the three segments of word vectors to obtain three segments of feature vectors, and respectively calculating the weights and relationship vectors of the three segments of feature vectors; specifically, the weight and the relation vector of the three sections of feature vectors are calculated by adopting the following formula:
bi＝Conv(vectorsentence_i),i＝1,2,3 (1)
wi＝wa T(tanh[bi；vrelation])+ba (2)
vrelation＝vent1-vent2 (4)
wherein, biThe feature vector, representing the i-th part of the sentence, extracted by the convolutional neural networksentence_iRepresenting the i-th partial word vector of the sentence sensor; w is aiRepresenting the fusion relation vector of the ith part of feature vectors to obtain a new feature vector wa、baIs a model parameter; alpha is alphaiIs a feature vector weight, vrelationIs a relation vector; v. ofent1And vent2Respectively representing a first entity vector and a second entity vector;
multiplying the three sections of feature vectors by corresponding weights according to the obtained weights of the feature vectors, and splicing to obtain a final sentence vector representation, wherein the sentence vector is classified by softmax after passing through a full connection layer, and the method specifically comprises the following steps:
s＝contact[b1·α1；b2·α2；b3·α3] (5)
c＝softmax(w·s+b) (6)
wherein s represents a new vector spliced after each w is subjected to weighted summation; c represents a vector of categories; w represents a new obtained feature vector after the feature vector is fused with the relation vector; b represents the feature vector of the sentence sensor extracted by the convolutional neural network.
2. The system according to claim 1, wherein said corpus collection module further comprises a data cleansing function for filtering invalid non-textual data and storing crawled sentences in a database according to sentence text length.
3. The system of claim 2, wherein the dynamic word vectors are generated from different sentences, and a sentence length threshold is set, wherein the sentence length threshold is greater than the threshold and is truncated, and the sentence length threshold is smaller than the threshold and is filled; and inputting the aligned sentences into a dynamic word vector generation module, outputting word vectors of each word in the sentences, and splicing to obtain the sentence vectors of the sentences.
4. The system of claim 3, wherein the relationship extraction module takes the sentence vector generated by the dynamic word vector generation module as input, extracts the relationship of the entity pair in the sentence, and outputs the category to which the sentence belongs.
5. The system of claim 4, wherein the deep bi-directional language model is an ELMO model or a BERT model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011387516.8A CN112487203B (en) | 2019-01-25 | 2019-01-25 | Relation extraction system integrated with dynamic word vector |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910071228.2A CN109871451B (en) | 2019-01-25 | 2019-01-25 | Method and system for extracting relation of dynamic word vectors |
| CN202011387516.8A CN112487203B (en) | 2019-01-25 | 2019-01-25 | Relation extraction system integrated with dynamic word vector |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910071228.2A Division CN109871451B (en) | 2019-01-25 | 2019-01-25 | Method and system for extracting relation of dynamic word vectors |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112487203A true CN112487203A (en) | 2021-03-12 |
| CN112487203B CN112487203B (en) | 2024-01-16 |
Family
ID=66918012
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011387516.8A Active CN112487203B (en) | 2019-01-25 | 2019-01-25 | Relation extraction system integrated with dynamic word vector |
| CN201910071228.2A Active CN109871451B (en) | 2019-01-25 | 2019-01-25 | Method and system for extracting relation of dynamic word vectors |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910071228.2A Active CN109871451B (en) | 2019-01-25 | 2019-01-25 | Method and system for extracting relation of dynamic word vectors |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN112487203B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800777A (en) * | 2021-04-14 | 2021-05-14 | 北京育学园健康管理中心有限公司 | Semantic determination method |
| CN113051356A (en) * | 2021-04-21 | 2021-06-29 | 深圳壹账通智能科技有限公司 | Open relationship extraction method and device, electronic equipment and storage medium |
| CN113326371A (en) * | 2021-04-30 | 2021-08-31 | 南京大学 | Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information |
| CN113807518A (en) * | 2021-08-16 | 2021-12-17 | 中央财经大学 | Relationship extraction system based on remote supervision |
| CN114373554A (en) * | 2021-12-28 | 2022-04-19 | 大连海事大学 | Drug interaction relation extraction method using drug knowledge and syntactic dependency relation |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110210037B (en) * | 2019-06-12 | 2020-04-07 | 四川大学 | Syndrome-oriented medical field category detection method |
| CN110287337A (en) * | 2019-06-19 | 2019-09-27 | 上海交通大学 | The system and method for medicine synonym is obtained based on deep learning and knowledge mapping |
| CN110458397A (en) * | 2019-07-05 | 2019-11-15 | 苏州热工研究院有限公司 | A kind of nuclear material military service performance information extracting method |
| CN110347835B (en) * | 2019-07-11 | 2021-08-24 | 招商局金融科技有限公司 | Text clustering method, electronic device and storage medium |
| CN110489521B (en) * | 2019-07-15 | 2021-03-12 | 北京三快在线科技有限公司 | Text type detection method and device, electronic equipment and computer readable medium |
| CN112396201A (en) * | 2019-07-30 | 2021-02-23 | 北京国双科技有限公司 | Criminal name prediction method and system |
| CN110390110B (en) * | 2019-07-30 | 2023-06-27 | 创新先进技术有限公司 | Method and apparatus for pre-training generation of sentence vectors for semantic matching |
| CN110598000B (en) * | 2019-08-01 | 2023-06-09 | 达而观信息科技（上海）有限公司 | Relation extraction and knowledge graph construction method based on deep learning model |
| CN110414008B (en) * | 2019-08-09 | 2023-06-20 | 深巨科技(北京)有限公司 | Relation extraction system and method based on deep learning |
| CN110516239B (en) * | 2019-08-26 | 2022-12-09 | 贵州大学 | Segmentation pooling relation extraction method based on convolutional neural network |
| CN111639152B (en) * | 2019-08-29 | 2021-04-13 | 上海卓繁信息技术股份有限公司 | Intention recognition method |
| CN110688407B (en) * | 2019-09-09 | 2022-05-17 | 创新奇智(南京)科技有限公司 | Social relationship mining method |
| CN110599999A (en) * | 2019-09-17 | 2019-12-20 | 寇晓宇 | Data interaction method and device and robot |
| CN110781312B (en) * | 2019-09-19 | 2022-07-15 | 平安科技（深圳）有限公司 | Text classification method and device based on semantic representation model and computer equipment |
| CN110728153A (en) * | 2019-10-15 | 2020-01-24 | 天津理工大学 | Multi-category emotion classification method based on model fusion |
| CN110909541A (en) * | 2019-11-08 | 2020-03-24 | 杭州依图医疗技术有限公司 | Instruction generation method, system, device and medium |
| CN111222338A (en) * | 2020-01-08 | 2020-06-02 | 大连理工大学 | Biomedical relation extraction method based on pre-training model and self-attention mechanism |
| CN111241303A (en) * | 2020-01-16 | 2020-06-05 | 东方红卫星移动通信有限公司 | Remote supervision relation extraction method for large-scale unstructured text data |
| CN111274394B (en) * | 2020-01-16 | 2022-10-25 | 重庆邮电大学 | Method, device and equipment for extracting entity relationship and storage medium |
| CN111506696A (en) * | 2020-03-03 | 2020-08-07 | 平安科技（深圳）有限公司 | Information extraction method and device based on small number of training samples |
| CN111597812B (en) * | 2020-05-09 | 2021-09-17 | 北京合众鼎成科技有限公司 | Financial field multiple relation extraction method based on mask language model |
| CN111813896B (en) * | 2020-07-13 | 2022-12-02 | 重庆紫光华山智安科技有限公司 | Text triple relation identification method and device, training method and electronic equipment |
| CN112199508B (en) * | 2020-08-10 | 2024-01-19 | 淮阴工学院 | Parameter self-adaptive agricultural knowledge graph recommendation method based on remote supervision |
| CN112380328B (en) * | 2020-11-11 | 2024-02-06 | 广州知图科技有限公司 | Interaction method and system for safety emergency response robot |
| CN112651224A (en) * | 2020-12-24 | 2021-04-13 | 天津大学 | Intelligent search method and device for engineering construction safety management document text |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170200066A1 (en) * | 2016-01-13 | 2017-07-13 | Adobe Systems Incorporated | Semantic Natural Language Vector Space |
| US20180189269A1 (en) * | 2016-12-30 | 2018-07-05 | Microsoft Technology Licensing, Llc | Graph long short term memory for syntactic relationship discovery |
| CN108733792A (en) * | 2018-05-14 | 2018-11-02 | 北京大学深圳研究生院 | A kind of entity relation extraction method |
| CN108829722A (en) * | 2018-05-08 | 2018-11-16 | 国家计算机网络与信息安全管理中心 | A kind of Dual-Attention relationship classification method and system of remote supervisory |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6801655B2 (en) * | 2001-05-10 | 2004-10-05 | The United States Of America As Represented By The Secretary Of The Navy | Spatial image processor |
| US10733380B2 (en) * | 2017-05-15 | 2020-08-04 | Thomson Reuters Enterprise Center Gmbh | Neural paraphrase generator |
| CN108280058A (en) * | 2018-01-02 | 2018-07-13 | 中国科学院自动化研究所 | Relation extraction method and apparatus based on intensified learning |
| CN108763284B (en) * | 2018-04-13 | 2021-07-20 | 华南理工大学 | Question-answering system implementation method based on deep learning and topic model |
-
2019
- 2019-01-25 CN CN202011387516.8A patent/CN112487203B/en active Active
- 2019-01-25 CN CN201910071228.2A patent/CN109871451B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170200066A1 (en) * | 2016-01-13 | 2017-07-13 | Adobe Systems Incorporated | Semantic Natural Language Vector Space |
| US20180189269A1 (en) * | 2016-12-30 | 2018-07-05 | Microsoft Technology Licensing, Llc | Graph long short term memory for syntactic relationship discovery |
| CN108829722A (en) * | 2018-05-08 | 2018-11-16 | 国家计算机网络与信息安全管理中心 | A kind of Dual-Attention relationship classification method and system of remote supervisory |
| CN108733792A (en) * | 2018-05-14 | 2018-11-02 | 北京大学深圳研究生院 | A kind of entity relation extraction method |
Non-Patent Citations (1)
| Title |
|---|
| 张兰霞;胡文心;: "基于双向GRU神经网络和双层注意力机制的中文文本中人物关系抽取研究", 计算机应用与软件, no. 11 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800777A (en) * | 2021-04-14 | 2021-05-14 | 北京育学园健康管理中心有限公司 | Semantic determination method |
| CN113051356A (en) * | 2021-04-21 | 2021-06-29 | 深圳壹账通智能科技有限公司 | Open relationship extraction method and device, electronic equipment and storage medium |
| CN113326371A (en) * | 2021-04-30 | 2021-08-31 | 南京大学 | Event extraction method fusing pre-training language model and anti-noise interference remote monitoring information |
| CN113326371B (en) * | 2021-04-30 | 2023-12-29 | 南京大学 | Event extraction method integrating pre-training language model and anti-noise interference remote supervision information |
| CN113807518A (en) * | 2021-08-16 | 2021-12-17 | 中央财经大学 | Relationship extraction system based on remote supervision |
| CN113807518B (en) * | 2021-08-16 | 2024-04-05 | 中央财经大学 | Relation extraction system based on remote supervision |
| CN114373554A (en) * | 2021-12-28 | 2022-04-19 | 大连海事大学 | Drug interaction relation extraction method using drug knowledge and syntactic dependency relation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112487203B (en) | 2024-01-16 |
| CN109871451A (en) | 2019-06-11 |
| CN109871451B (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109871451B (en) | Method and system for extracting relation of dynamic word vectors | |
| CN112528672B (en) | Aspect-level emotion analysis method and device based on graph convolution neural network | |
| WO2018028077A1 (en) | Deep learning based method and device for chinese semantics analysis | |
| CN110321563B (en) | Text emotion analysis method based on hybrid supervision model | |
| CN114064918B (en) | Multi-modal event knowledge graph construction method | |
| CN111143569B (en) | Data processing method, device and computer readable storage medium | |
| CN109214006B (en) | Natural language reasoning method for image enhanced hierarchical semantic representation | |
| CN113392209B (en) | Text clustering method based on artificial intelligence, related equipment and storage medium | |
| CN106372061A (en) | Short text similarity calculation method based on semantics | |
| WO2023159767A1 (en) | Target word detection method and apparatus, electronic device and storage medium | |
| CN113255320A (en) | Entity relation extraction method and device based on syntax tree and graph attention machine mechanism | |
| CN111125367A (en) | Multi-character relation extraction method based on multi-level attention mechanism | |
| CN114462409A (en) | Audit field named entity recognition method based on countermeasure training | |
| Barbella et al. | Analogical word sense disambiguation | |
| Lee et al. | Detecting suicidality with a contextual graph neural network | |
| CN112183060B (en) | Reference resolution method of multi-round dialogue system | |
| Gao et al. | Chinese causal event extraction using causality‐associated graph neural network | |
| Behere et al. | Text summarization and classification of conversation data between service chatbot and customer | |
| Veisi | Central Kurdish Sentiment Analysis Using Deep Learning. | |
| CN116595023A (en) | Address information updating method and device, electronic equipment and storage medium | |
| CN114638222B (en) | Natural disaster data classification method and model training method and device thereof | |
| Han et al. | Unsupervised Word Sense Disambiguation based on Word Embedding and Collocation. | |
| CN112131879A (en) | Relationship extraction system, method and device | |
| CN115906816A (en) | Text emotion analysis method of two-channel Attention model based on Bert | |
| CN115906824A (en) | Text fine-grained emotion analysis method, system, medium and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |