CN111222338A - Biomedical relation extraction method based on pre-training model and self-attention mechanism - Google Patents

Biomedical relation extraction method based on pre-training model and self-attention mechanism Download PDF

Info

Publication number
CN111222338A
CN111222338A CN202010017867.3A CN202010017867A CN111222338A CN 111222338 A CN111222338 A CN 111222338A CN 202010017867 A CN202010017867 A CN 202010017867A CN 111222338 A CN111222338 A CN 111222338A
Authority
CN
China
Prior art keywords
biomedical
text
sentence
self
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010017867.3A
Other languages
Chinese (zh)
Inventor
张益嘉
于洪海
田方正
刘雨
张雨琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN202010017867.3A priority Critical patent/CN111222338A/en
Publication of CN111222338A publication Critical patent/CN111222338A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)

Abstract

The invention belongs to the technical field of natural language processing, and relates to a biomedical relation extraction method based on a pre-training model and a self-attention mechanism. According to the invention, the ELMO pre-training model can be used for extracting more complex information in the biomedical sentences, so that the expression effect on sentence characteristics of the biomedical texts is improved, and the relationship between biomedical entities is better extracted; the position features are added to learn the position relationship between the internal structure of the sentence and the biomedical entity, and the internal correlation of the data and the features in the sentence is captured better by using a self-attention mechanism, so that the task of extracting the biomedical relationship is completed better. The method solves the problem that the current biomedical relation extraction mostly only focuses on the simple semantics of the sentence sequence, can enhance the relation extraction effect of the regular biomedical text, and has better effect on the relation extraction of the irregular biomedical text on the social media.

Description

Biomedical relation extraction method based on pre-training model and self-attention mechanism
Technical Field
The invention belongs to the technical field of natural language processing, and relates to a biomedical relation extraction method based on a pre-training model and a self-attention mechanism.
Background
With the rapid development of the internet, the network information is explosively increased, 2000 to 4000 biomedical documents are published every day in the biomedical field, and the documents contain massive biomedical entity relationships, such as drug interaction relationships, protein interaction relationships and the like, and are important resources for biomedical research.
The use of manual labeling methods to label entity relationships from biomedical literature is time consuming, labor intensive, and requires support by knowledge in the biomedical field of expertise. The automatic and efficient extraction of entity relationships hidden in biomedical documents can effectively save manpower and resources.
The traditional extraction method based on the template and the rule has low recall rate and needs to manually construct the template. As deep learning theories and methods mature, researchers have begun to use deep learning neural network models to automatically extract biomedical entity relationships from biomedical literature. However, most of these methods only focus on the simple semantics of the sentence sequence, and actually, the dependency relationship between two entities in the sentence depends on more complex semantic information.
Disclosure of Invention
The invention aims to provide a natural language relationship extraction method based on a pre-training model and a self-attention mechanism for solving the defects of the prior art, the ELMO pre-training model can be used for extracting more complex information in a biomedical sentence, including grammar, semantics and even word ambiguity, so that the expression effect of the sentence characteristics of the biomedical text is improved, and biomedical entities such as: the relationship between proteins, diseases and drugs, drugs and side effects. The position features are added to learn the position relationship between the internal structure of the sentence and the biomedical entity, and the internal correlation of the data and the features in the sentence is captured better by using a self-attention mechanism, so that the task of extracting the biomedical relationship is completed better.
The technical scheme of the invention is as follows:
a biomedical relation extraction method based on a pre-training model and a self-attention mechanism comprises the following specific steps:
s1) preprocessing the labeled biomedical corpus: writing a data preprocessing program, converting the original corpus into input which can be accepted by a deep learning network model, constructing the position characteristics of the biomedical entities, and providing information for the relationship among the biomedical entities; biomedical entities include diseases, drugs and side effects.
S2) constructing a sentence vector of the biomedical text: inputting the corpus preprocessed in the step S1) into an ELMO pre-training model, extracting the characteristics of the whole biomedical text sentence, and outputting a vector; meanwhile, processing the position characteristics of the biomedical entity generated in the step S1) by adopting a word embedding mode to form a position vector of the biomedical entity; all the vectors obtained are then concatenated to form a long vector to represent the sentence in the biomedical text.
S3) inputting the BILSTM neural network model to extract features: and (4) inputting the long vector obtained in the step (S2) into a BILSTM neural network after passing through a dropout layer, learning the context information of the biomedicine, understanding a single sentence of the biomedicine text from two directions, and outputting a feature vector by the BILSTM.
S4) extracting key features of the biomedical text using a multi-headed self-attentiveness mechanism: after the BILSTM neural network outputs the feature vectors, a multi-head self-attention mechanism is used for capturing data in the biomedical text and the internal correlation of the features, and then key features are extracted through a full connection layer.
S5) biomedical entity relationship prediction: after key features are extracted through the full-connection layer, the results are input into the full-connection neural network again, the relation of the two biomedical entities in sentences is predicted, and finally probability distribution of the biomedical relation is obtained, so that the biomedical relation is extracted.
The invention has the technical characteristics that:
(1) representing the word vector by using a pre-training model ELMO;
(2) adding location features of the entity;
(3) a multi-head self-attention mechanism is added behind the BILSTM neural network model.
(4) The ELMO pre-training model can be used for learning complex features of words, including grammar, semantics and the like, and learning word ambiguity under different contexts, the position features of the entities can provide information for a following multi-head self-attention mechanism, the context information of the words is acquired through the BILSTM, and then the multi-head attention mechanism is used for processing biomedical sentences output by an upper layer in a plurality of attention modes from a plurality of representation spaces, so that the relationship among the entities is better extracted.
Compared with the prior art, the invention has the beneficial effects that:
1) word context information can be considered through an ELMO pre-training model, so that word vector representation which is more similar to actual semantics is carried out on words, and the problem of word ambiguity of the words in the biomedical text is solved.
2) The position feature and the BILSTM are used to cooperate with a multi-head self-attention mechanism to understand a sentence from multiple angles, and more complex semantics among biomedical entities are obtained.
The invention solves the problem that the current biomedical relation extraction mostly only focuses on the simple semantics of sentence sequences, can enhance the relation extraction effect of regular biomedical texts, and has better effect on the relation extraction of the non-regular biomedical texts on social media due to better excavation of the text characteristics of the non-regular biomedical texts.
Drawings
FIG. 1 is a schematic flow chart of the present invention.
FIG. 2 is a schematic diagram of a BILSTM neural network.
FIG. 3 is a schematic diagram of the ELMO pre-training model.
Detailed Description
The following further describes a specific embodiment of the present invention with reference to the drawings and technical solutions.
The invention relates to a biomedical relation extraction method based on a pre-training model and a self-attention mechanism, which comprises the following steps of firstly preprocessing a labeled corpus and constructing two biomedical entities: such as position characteristics among proteins, diseases, side effects and the like, and then converting the original linguistic data into input which can be accepted by a deep learning network model; then, generating word vectors by using an ELMO pre-training model and word embedding, and then connecting the word vectors to generate long vectors to represent numerous sentences of the biomedical texts; inputting the long vector into a BILSTM neural network after passing through a dropout layer, and learning the context information of the biomedical text; and realizing a multi-head Self-attention mechanism through a Self-attention layer to understand the biomedical text from a plurality of angles by a plurality of side points, extracting a full-connection layer of key features through a layer, and finally outputting the probability distribution of the relation through an output layer. The flow is shown in FIG. 1, and the steps are further described below.
S1) preprocessing the labeled biomedical text corpus, and converting the original corpus into an input which can be accepted by the deep learning network model, namely, firstly, constructing a dictionary through a corpus, wherein each word corresponds to a sequence number, so that each sentence is represented as a sequence of a plurality of numbers. And (3) constructing the position characteristics of the biomedical entities, namely constructing the position sequences of the two entities according to the positions of the two biomedical entities in the sentence.
S2) constructing sentence vectors of the biomedical texts, inputting the digitalized sentence sequences processed in the step S1) and dictionaries, namely mapping of serial numbers and words, into an ELMO pre-training model, extracting the characteristics of the whole biomedical text sentences, and learning the biomedical texts through a multi-layer neural network in the ELMO, wherein the state of the high-layer neural network can capture the characteristics of the biomedical entities and the aspects related to the contexts, so that semantic disambiguation can be performed, and the characteristics of the grammatical aspects can be found by the low-layer neural network, so that part-of-speech tagging can be performed, and the grammatical characteristics of the biomedical entities-relation-entities can be obtained. Combining them together and extracting the relationship between them will be advantageous. And outputting the generated word vector, and processing the position characteristics of the biomedical entity generated in the step S1) by adopting a word embedding mode to form a position vector. After that, the position vector and the word vector generated by the ELMO pre-training model are connected together by using a concatenate function in the keras to form a long vector, so that a large number of key features are extracted from the biomedical text, and the biomedical text is better represented.
The ELMO pre-training model is a dynamic model, the internal structure of which is shown in FIG. 2, and can update Word Embedding of words, essentially, a language model is firstly learned out Word Embedding of words on a large corpus, parameters in the model are finely adjusted by biomedical text data, which is also called a fine-tuning pre-trained ELMO model, so that characteristics of the biomedical text, such as grammatical semantics and the like, can be better represented by means of huge corpus information.
The ELMO pre-training model is a bidirectional LSTM language model and is composed of forward and backward language models, and the objective function is the maximum likelihood of the language models in the two directions. The forward model formula and the reverse model formula are respectively:
Figure BDA0002359587750000051
Figure BDA0002359587750000052
the log-likelihood function is expressed as:
Figure BDA0002359587750000053
wherein, t1,t2,…,tNRepresenting each word in a biomedical text sentence, p (t)1,t2,…,tN) Representing the probability of occurrence of biomedical text consisting of these words; Θ x represents the word embedding, and Θ s represents the parameters of the softmax layer.
The ELMO model differs from the previous models in that it uses only the last layer's output values as the values for WordEmaddressing, whereas ELMO represents the values for WordEmaddressing in a linear combination of all the layers' output values, and for each token in a sentence, the BILSTM of one L layer has to compute 2L +1 tokens:
Figure BDA0002359587750000061
wherein the content of the first and second substances,
Figure BDA0002359587750000062
the word representing the kth position, at each position k, a corresponding token is output on each layer LSTM
Figure BDA0002359587750000063
j represents the number of layers of the neural network.
Then R will be set in the downstream taskkCompressed into one vector:
Figure BDA0002359587750000064
wherein the content of the first and second substances,
Figure BDA0002359587750000065
is the softmax normalized weight, gammataskIs a scaling factor, which may have the effect of scaling the entire ELMO vector. The learning range of the ELMO pre-training model is wide, and compared with Word2vec, the ELMO pre-training model has the advantages of wide learning rangeThe method comprises the steps of learning the complex characteristics of word usage and the changes of the complex usage in different contexts, and directly calling the complex usage in a specific network model, wherein the embodiment is that a pre-trained ELMO language model is directly called from https:// tfhub.
And then changing the position features of the biomedical text into position vectors by calling an embedding layer in Keras, and adding the position features to learn the position relationship between the internal structure of the biomedical text and the biomedical entities.
S3) inputting the BILSTM network extraction features, preventing overfitting through a dropout layer, and then inputting the features into the BILSTM network. BILSTM is chosen because BILSTM is more powerful than RNN and LSTM, able to learn context information and solve problems such as gradient explosion and gradient disappearance.
The BILSTM essentially combines forward LSTM and backward LSTM, learning both forward and backward information. While LSTM is a variation of RNN that combines short-term memory with long-term memory through subtle gating controls, if RNN is used, if the sentence of the biomedical text is too long, the gradient of the neural network disappears and the extraction of the relationship between the biomedical entities cannot be performed efficiently. The use of BILSTM effectively solves this problem.
LSTM has the ability to remove or add information to the cell state through three gate structures, input, forgetting, and output gates, respectively. The gate is essentially a method for selectively passing information, and comprises a sigmoid neural network layer and a pointwise multiplication operation, wherein the sigmoid outputs a value between 0 and 1, which represents how much of each part can pass through. The principle of LSTM is shown in fig. 3, which contains corresponding parameters, and the formula for implementing LSTM function is as follows:
ft＝σ(Wf·[ht-1,xt])
it＝σ(Wi·[ht-1,xt])
Figure BDA0002359587750000071
Figure BDA0002359587750000072
ot＝σ(Wo·[ht-1,xt])
ht＝ot*tanh(Ct)
the first step of LSTM is to decide what information will be discarded from the cell state, which is done by the forgoing forgetting gate. Forget the door will read ht-1And xtThen, a value between 0 and 1 is output as each cell state C by calculationt-1The number 1 indicates complete retention, and 0 indicates complete rejection. By this value, the cell can be made aware of what the forgotten proportion is, and the next step is to determine what new information is accepted by the cell deposit. Implemented by a sigmoid layer, which decides what value is to be updated, and a tanh layer, which creates a new vector of candidate values
Figure BDA0002359587750000073
This state is added to the state and the update of the state is to be performed next. Old state and ftBy multiplying, discarding of information can be achieved, adding
Figure BDA0002359587750000074
A new state is obtained, i.e. the contents of forgetting and remembering are determined by the sigmoid function to select updates, and candidate values are obtained by the tanh function. And finally, through an output gate, determining that part of the cell is to be output by the sigmoid function, multiplying the state of the cell by the output of the sigmoid through tanh function processing, and outputting the determined part of the output.
While the output at the t time step is obtained by the BILSTM
Figure BDA0002359587750000081
Representing the output of the forward LSTM,
Figure BDA0002359587750000082
representing the output of the backward LSTM. The new vector H (H) finally generated1,h2,...,hn) T can reflect the semantic meaning of a sentence better and more deeply at a higher level. And H ∈ RnX 2d, where n represents the length of the sentence sequence and d represents the dimension of LSTM.
S4) extracting the characteristics of the biomedical text by using a multi-head self-attention mechanism, and after the BILSTM is output, better capturing the internal correlation of the data and the characteristics in the biomedical sentence by using the multi-head self-attention mechanism.
Inputting a sentence in the biomedical field, and performing attribute calculation on each word in the sentence and all words in the sentence, wherein the self-attribute mechanism is adopted to better learn the dependency relationship and the internal structure of the sentence in the biomedical sentence.
As mentioned above, the invention introduces the position feature, namely, the dependency relationship of the relative distance in the sentence is learned through self-attribute mechanism, the attention value of each word in the sentence can be calculated, and the internal structure of a sentence can be further learned.
While Multi-Head-Self-Attention is not only calculated once, it refers to capturing relevant information on different subspaces in different dimensions by calculating many times. The multi-head self-attention mechanism can provide a plurality of different expression subspaces for the biomedical text, and a plurality of groups of Q, K and V weight matrixes exist, so that the capability of the model for processing different positions is enhanced.
The calculation process is as follows:
Figure BDA0002359587750000083
headi＝Attention(QWi Q,KWi K,VWi V)
MultiHead(Q,K,V)＝Concat(head1,…,headh)Wo
wherein d iskIn the above formula, Q, K, and V are all long vectors described in step S2), and three of them are equal due to the self-attention mechanism; wi Q,Wi K,Wi V,WoRepresenting weight matrices, heads, within neural networksiRepresenting a certain head of the calculation, different heads being understood as different expressions, Concat (head), to the biomedical statement1,…,headh) Representing the vectors resulting from concatenating the different headers.
And finally extracting key features through a full connection layer.
S5) biomedical entity relationship prediction
After key features are extracted through the full-connection layer, the results are input into the full-connection neural network again, the relation of the two biomedical entities in the sentence is predicted, and the final relation prediction result is calculated by utilizing a Softmax function.

Claims (2)

1. A biomedical relation extraction method based on a pre-training model and a self-attention mechanism is characterized by comprising the following specific steps:
s1) preprocessing the labeled biomedical corpus: writing a data preprocessing program, converting the original corpus into input which can be accepted by a deep learning network model, constructing the position characteristics of the biomedical entities, and providing information for the relationship among the biomedical entities;
s2) constructing a sentence vector of the biomedical text: inputting the corpus preprocessed in the step S1) into an ELMO pre-training model, extracting the characteristics of the whole biomedical text sentence, and outputting a vector; meanwhile, processing the position characteristics of the biomedical entity generated in the step S1) by adopting a word embedding mode to form a position vector of the biomedical entity; then all the obtained vectors are connected to form a long vector to represent sentences in the biomedical text;
s3) inputting the BILSTM neural network model to extract features: after the long vector obtained in the step S2) passes through a dropout layer, the long vector is input into a BILSTM neural network to learn the context information of the biomedicine, so that a single statement of the biomedicine text is understood from two directions, and the BILSTM outputs a feature vector;
s4) extracting key features of the biomedical text using a multi-headed self-attentiveness mechanism: after the BILSTM neural network outputs the feature vectors, capturing data in the biomedical text and the internal correlation of the features by using a multi-head self-attention mechanism, and extracting key features through a full connection layer;
s5) biomedical entity relationship prediction: after key features are extracted through the full-connection layer, the results are input into the full-connection neural network again, the relation of the two biomedical entities in sentences is predicted, and finally probability distribution of the biomedical relation is obtained, so that the biomedical relation is extracted.
2. The method of claim 1, wherein the biomedical entities comprise diseases, drugs and side effects.
CN202010017867.3A 2020-01-08 2020-01-08 Biomedical relation extraction method based on pre-training model and self-attention mechanism Pending CN111222338A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010017867.3A CN111222338A (en) 2020-01-08 2020-01-08 Biomedical relation extraction method based on pre-training model and self-attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010017867.3A CN111222338A (en) 2020-01-08 2020-01-08 Biomedical relation extraction method based on pre-training model and self-attention mechanism

Publications (1)

Publication Number Publication Date
CN111222338A true CN111222338A (en) 2020-06-02

Family

ID=70829371

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010017867.3A Pending CN111222338A (en) 2020-01-08 2020-01-08 Biomedical relation extraction method based on pre-training model and self-attention mechanism

Country Status (1)

Country Link
CN (1) CN111222338A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814460A (en) * 2020-07-06 2020-10-23 四川大学 External knowledge-based drug interaction relation extraction method and system
CN111859912A (en) * 2020-07-28 2020-10-30 广西师范大学 PCNN model-based remote supervision relationship extraction method with entity perception
CN112435720A (en) * 2020-12-04 2021-03-02 上海蠡图信息科技有限公司 Prediction method based on self-attention mechanism and multi-drug characteristic combination
CN112597366A (en) * 2020-11-25 2021-04-02 中国电子科技网络信息安全有限公司 Encoder-Decoder-based event extraction method
CN112883732A (en) * 2020-11-26 2021-06-01 中国电子科技网络信息安全有限公司 Method and device for identifying Chinese fine-grained named entities based on associative memory network
CN113468872A (en) * 2021-06-09 2021-10-01 大连理工大学 Biomedical relation extraction method and system based on sentence level graph convolution
CN113868374A (en) * 2021-09-15 2021-12-31 西安交通大学 Graph convolution network biomedical information extraction method based on multi-head attention mechanism
CN114373512A (en) * 2021-12-28 2022-04-19 大连海事大学 Protein interaction relation extraction method based on Gaussian enhancement and auxiliary task
CN114881038A (en) * 2022-07-12 2022-08-09 之江实验室 Chinese entity and relation extraction method and device based on span and attention mechanism

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108519890A (en) * 2018-04-08 2018-09-11 武汉大学 A kind of robustness code abstraction generating method based on from attention mechanism
CN108536754A (en) * 2018-03-14 2018-09-14 四川大学 Electronic health record entity relation extraction method based on BLSTM and attention mechanism
CN108875809A (en) * 2018-06-01 2018-11-23 大连理工大学 The biomedical entity relationship classification method of joint attention mechanism and neural network
CN109492227A (en) * 2018-11-16 2019-03-19 大连理工大学 It is a kind of that understanding method is read based on the machine of bull attention mechanism and Dynamic iterations
CN109558576A (en) * 2018-11-05 2019-04-02 中山大学 A kind of punctuation mark prediction technique based on from attention mechanism
CN109783618A (en) * 2018-12-11 2019-05-21 北京大学 Pharmaceutical entities Relation extraction method and system based on attention mechanism neural network
CN109871451A (en) * 2019-01-25 2019-06-11 中译语通科技股份有限公司 A kind of Relation extraction method and system incorporating dynamic term vector
CN109992783A (en) * 2019-04-03 2019-07-09 同济大学 Chinese term vector modeling method
CN110134771A (en) * 2019-04-09 2019-08-16 广东工业大学 A kind of implementation method based on more attention mechanism converged network question answering systems
CN110147551A (en) * 2019-05-14 2019-08-20 腾讯科技（深圳）有限公司 Multi-class entity recognition model training, entity recognition method, server and terminal
CN110321566A (en) * 2019-07-10 2019-10-11 北京邮电大学 Chinese name entity recognition method, device, computer equipment and storage medium
CN110334196A (en) * 2019-06-28 2019-10-15 同济大学 Neural network Chinese charater problem based on stroke and from attention mechanism generates system
CN110597970A (en) * 2019-08-19 2019-12-20 华东理工大学 Multi-granularity medical entity joint identification method and device

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108536754A (en) * 2018-03-14 2018-09-14 四川大学 Electronic health record entity relation extraction method based on BLSTM and attention mechanism
CN108519890A (en) * 2018-04-08 2018-09-11 武汉大学 A kind of robustness code abstraction generating method based on from attention mechanism
CN108875809A (en) * 2018-06-01 2018-11-23 大连理工大学 The biomedical entity relationship classification method of joint attention mechanism and neural network
CN109558576A (en) * 2018-11-05 2019-04-02 中山大学 A kind of punctuation mark prediction technique based on from attention mechanism
CN109492227A (en) * 2018-11-16 2019-03-19 大连理工大学 It is a kind of that understanding method is read based on the machine of bull attention mechanism and Dynamic iterations
CN109783618A (en) * 2018-12-11 2019-05-21 北京大学 Pharmaceutical entities Relation extraction method and system based on attention mechanism neural network
CN109871451A (en) * 2019-01-25 2019-06-11 中译语通科技股份有限公司 A kind of Relation extraction method and system incorporating dynamic term vector
CN109992783A (en) * 2019-04-03 2019-07-09 同济大学 Chinese term vector modeling method
CN110134771A (en) * 2019-04-09 2019-08-16 广东工业大学 A kind of implementation method based on more attention mechanism converged network question answering systems
CN110147551A (en) * 2019-05-14 2019-08-20 腾讯科技（深圳）有限公司 Multi-class entity recognition model training, entity recognition method, server and terminal
CN110334196A (en) * 2019-06-28 2019-10-15 同济大学 Neural network Chinese charater problem based on stroke and from attention mechanism generates system
CN110321566A (en) * 2019-07-10 2019-10-11 北京邮电大学 Chinese name entity recognition method, device, computer equipment and storage medium
CN110597970A (en) * 2019-08-19 2019-12-20 华东理工大学 Multi-granularity medical entity joint identification method and device

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814460A (en) * 2020-07-06 2020-10-23 四川大学 External knowledge-based drug interaction relation extraction method and system
CN111859912A (en) * 2020-07-28 2020-10-30 广西师范大学 PCNN model-based remote supervision relationship extraction method with entity perception
CN112597366B (en) * 2020-11-25 2022-03-18 中国电子科技网络信息安全有限公司 Encoder-Decoder-based event extraction method
CN112597366A (en) * 2020-11-25 2021-04-02 中国电子科技网络信息安全有限公司 Encoder-Decoder-based event extraction method
CN112883732A (en) * 2020-11-26 2021-06-01 中国电子科技网络信息安全有限公司 Method and device for identifying Chinese fine-grained named entities based on associative memory network
CN112435720B (en) * 2020-12-04 2021-10-26 上海蠡图信息科技有限公司 Prediction method based on self-attention mechanism and multi-drug characteristic combination
CN112435720A (en) * 2020-12-04 2021-03-02 上海蠡图信息科技有限公司 Prediction method based on self-attention mechanism and multi-drug characteristic combination
CN113468872A (en) * 2021-06-09 2021-10-01 大连理工大学 Biomedical relation extraction method and system based on sentence level graph convolution
CN113468872B (en) * 2021-06-09 2024-04-16 大连理工大学 Biomedical relation extraction method and system based on sentence level graph convolution
CN113868374A (en) * 2021-09-15 2021-12-31 西安交通大学 Graph convolution network biomedical information extraction method based on multi-head attention mechanism
CN113868374B (en) * 2021-09-15 2024-04-12 西安交通大学 Graph convolution network biomedical information extraction method based on multi-head attention mechanism
CN114373512A (en) * 2021-12-28 2022-04-19 大连海事大学 Protein interaction relation extraction method based on Gaussian enhancement and auxiliary task
CN114881038A (en) * 2022-07-12 2022-08-09 之江实验室 Chinese entity and relation extraction method and device based on span and attention mechanism
CN114881038B (en) * 2022-07-12 2022-11-11 之江实验室 Chinese entity and relation extraction method and device based on span and attention mechanism

Similar Documents

Publication Publication Date Title
CN111222338A (en) Biomedical relation extraction method based on pre-training model and self-attention mechanism
CN107832400B (en) A kind of method that location-based LSTM and CNN conjunctive model carries out relationship classification
KR102008845B1 (en) Automatic classification method of unstructured data
CN111914085B (en) Text fine granularity emotion classification method, system, device and storage medium
Boopathi Deep Learning Techniques Applied for Automatic Sentence Generation
CN110502753A (en) A kind of deep learning sentiment analysis model and its analysis method based on semantically enhancement
CN110750635B (en) French recommendation method based on joint deep learning model
CN111966812B (en) Automatic question answering method based on dynamic word vector and storage medium
CN110598005A (en) Public safety event-oriented multi-source heterogeneous data knowledge graph construction method
CN110362819A (en) Text emotion analysis method based on convolutional neural networks
CN112784041B (en) Chinese short text sentiment orientation analysis method
CN112199503B (en) Feature-enhanced unbalanced Bi-LSTM-based Chinese text classification method
CN112561718A (en) Case microblog evaluation object emotion tendency analysis method based on BilSTM weight sharing
CN114462420A (en) False news detection method based on feature fusion model
CN110232127A (en) File classification method and device
CN112287106A (en) Online comment emotion classification method based on dual-channel hybrid neural network
CN113157919A (en) Sentence text aspect level emotion classification method and system
CN115906816A (en) Text emotion analysis method of two-channel Attention model based on Bert
Khan et al. A deep neural framework for image caption generation using gru-based attention mechanism
CN115759119A (en) Financial text emotion analysis method, system, medium and equipment
CN114547303A (en) Text multi-feature classification method and device based on Bert-LSTM
CN114416969A (en) LSTM-CNN online comment sentiment classification method and system based on background enhancement
Gao et al. Attention-based BiLSTM network with lexical feature for emotion classification
CN116843175A (en) Contract term risk checking method, system, equipment and storage medium
Ermatita et al. Sentiment Analysis of COVID-19 using Multimodal Fusion Neural Networks.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200602