CN112860904B - External knowledge-integrated biomedical relation extraction method - Google Patents

External knowledge-integrated biomedical relation extraction method Download PDF

Info

Publication number
CN112860904B
CN112860904B CN202110367973.9A CN202110367973A CN112860904B CN 112860904 B CN112860904 B CN 112860904B CN 202110367973 A CN202110367973 A CN 202110367973A CN 112860904 B CN112860904 B CN 112860904B
Authority
CN
China
Prior art keywords
sentence
vector
entity
head
representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110367973.9A
Other languages
Chinese (zh)
Other versions
CN112860904A (en
Inventor
王春宇
张�浩
梁天铭
刘晓燕
刘国军
郭茂祖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN202110367973.9A priority Critical patent/CN112860904B/en
Publication of CN112860904A publication Critical patent/CN112860904A/en
Application granted granted Critical
Publication of CN112860904B publication Critical patent/CN112860904B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis

Abstract

A biomedical relation extraction method integrated with external knowledge relates to the technical field of natural language processing, and aims at the problem of noise interference in training data of a remote supervision technology.

Description

External knowledge-integrated biomedical relation extraction method
Technical Field
The invention relates to the technical field of natural language processing, in particular to a biomedical relation extraction method integrated with external knowledge.
Background
With the rapid development of society, the society has entered the information explosion era at present, and in the field of biomedical science, thousands of biomedical literatures are published every day, and these literatures contain massive biomedical entity relationships, so that the needs of doctors and experts for effective information screening and induction are increasingly urgent, and how to extract effective information from massive data becomes a difficult problem.
At present, a relation extraction technology based on supervised learning is often adopted in the relation extraction technology, and the technology needs a large amount of manually labeled training data, so that the time and labor are consumed. Therefore, the automatic and efficient extraction of the biomedical entity relationships hidden in the literature can effectively save manpower and resources.
With the development of deep learning theory, researchers began to use neural network models to automatically extract entity relationships from biomedical literature. Among them, the most common method is to automatically generate a large amount of training data for human use by using remote supervision, but one of the serious problems of the remote supervision is that the generated training data has a large amount of noise, which is more prominent in biomedical data. The neural network model has no effective method for processing the noise generated by the remote monitoring biomedical data, so that how to use the neural network method for processing the remote monitoring biomedical data noise is a significant research direction
Disclosure of Invention
The purpose of the invention is: aiming at the problem of noise interference in training data of a remote supervision technology, a biomedical relation extraction method integrated with external knowledge is provided.
The technical scheme adopted by the invention to solve the technical problems is as follows:
a biomedical relation extraction method integrated with external knowledge comprises the following steps:
performing word embedding and position embedding operation on each word in each sentence in the biomedical data set to obtain a word vector and a position vector, splicing the obtained word vector and position vector to obtain vector representation of each word, and finally splicing the vector representations of all words in the sentences to obtain matrix representation of each sentence;
inputting the matrix representation of each sentence obtained in the step one into a PCNN neural network to obtain the vector representation of each sentence in the biomedical data set;
acquiring a head entity and a tail entity of each sentence in the biomedical data set, respectively extracting an entity related to the head entity and an entity related to the tail entity from an external knowledge graph to obtain a relation graph taking the head entity as a center and a relation graph taking the tail entity as a center, and inputting the two obtained relation graphs into a graph encoder to construct comprehensive vector representation of the external knowledge graph of the head entity and the tail entity;
combining the vector representation of each sentence in the biomedical data set with the comprehensive vector of the external knowledge map of the head and tail entities to obtain a sentence vector containing external information;
and step five, for each entity pair, selecting all sentences containing the entity pair to form a set, calculating the attention weight of each sentence in the set by adopting a sentence-level attention mechanism according to the sentence vector representation containing the external information, then taking the sum of the attention weights represented by all sentences containing the external information in the set as the vector representation of the set, and predicting the vector representation of the set to obtain the predicted relationship of the entity pair.
Further, the graph encoder in step three adopts KG-Transformer.
Further, the KG-Transformer encoding process is as follows:
KG-Transformer represents X ═ X in the vector of the node sequence in the two input relational graphs1,x2,...,xNIs inputted to the Muti-head attachment Layer and Add&Norm Layer：
The Muti-head attachment Layer is calculated as follows:
Figure BDA0003008193450000021
Figure BDA0003008193450000022
Figure BDA0003008193450000023
wherein the content of the first and second substances,
Figure BDA0003008193450000024
denotes the concatenation of the H attention heads of this layer, x'iRepresenting the node representation of the output, A is the adjacency matrix, i represents the ith row, j represents the jth column, d is the dimension of node embedding,
Figure BDA0003008193450000025
are all weight matrices, N represents the length of the node sequence, Masking (X, A) represents the length in terms of momentsMasking the value of the corresponding position of the matrix X at the position of which the value in the matrix A is 1;
the Add & Norm Layer was calculated as follows:
O＝LayerNorm(X+X')
wherein X ═ { X ═ X1,x2,......,xNX is a vector representation of a sequence of nodes, X '═ X'1,x'2,......x'NX' is the output calculated by the Muti-head attachment Layer, LayerNorm (-) is a Layer normalization function, and the output result O is used as the input of the next Layer of the Muti-head attachment Layer;
repeating the calculation process for L times, wherein L is any integer, obtaining the vector representation of all nodes, and finally summing the vector representations of all nodes of the relational graph of the head entity and the tail entity respectively to obtain the comprehensive vector representation of the head entity and the tail entity.
Further, L is 8, 12, 16 or 24.
Further, in the second step, the PCNN neural network obtains a vector representation of each sentence in the biomedical data set through convolution, pooling and nonlinear operations.
Further, the PCNN neural network in the second step specifically executes the following steps:
firstly, extracting local features from the matrix representation of the sentence by adopting a convolution kernel with the sliding window size of 3, and then combining all the local features through maximum pool operation to obtain a vector of the matrix representation of the sentence;
then dividing the vector represented by the matrix of the obtained sentence into three segments according to the positions of the head entity and the tail entity, and then pooling each segment to obtain a three-dimensional vector:
and splicing the three-dimensional vectors corresponding to all the convolution kernels, and performing non-linearization by using an activation function to obtain the final vector representation of the sentence.
Further, the activation function is a tanh function.
Further, the vector of the node sequence in the relational graph is represented as:
(e,r1,e1,r2,e2,...,rn,en)
wherein, entity e and entity e1,e2,...,enAssociating entity e with entity e1,e2,...,enRespectively is r1,r2,...,rn。
Further, the fourth step is to combine the vector representation of each sentence in the biomedical data set and the comprehensive vector of the external knowledge map of the head and tail entities to represent:
Figure BDA0003008193450000031
MutiHead(Q,K,V)＝Concat(head1,...,headh)Wo
in the above formula, Q represents a matrix representation of each sentence, K, V represents a vector representation of each sentence, and K and V are equal;
Figure BDA0003008193450000032
Worepresenting weight matrices, heads, within neural networksiRepresenting a certain head of the calculation, a different head representing a different expression, Concat (head), of the biomedical statement1,...,headh) Representing the vectors resulting from concatenating the different headers.
Further, the concrete steps of the fifth step are as follows:
the attention weight of each sentence in the set of sentences is calculated using a sentence-level attention mechanism, with the sum of the attention weights of all the sentence representations as a vector representation of the set of sentences, i.e.
Figure BDA0003008193450000041
Figure BDA0003008193450000042
ei＝xiAr
αiIs a sentence vector xiWeight of (1), xiIs a vector representation of the ith sentence, a is a diagonal weight matrix, r is a vector representation of the relationship r, s represents a vector representation of the set of sentences;
and finally, according to the vector representation s of the sentence set, calculating the probability that the sentence set belongs to the relation r by using a softmax classifier as follows:
P(r|S；θ)＝softmax(Ws+b)
wherein W represents a weight matrix, S represents a sentence vector, b represents a bias term, theta is a model parameter, and S represents the sentence set.
The invention has the beneficial effects that:
according to the method for extracting the relation of the biomedical entities, abundant semantic information and structural information among sentences and in sentences and information of the biomedical entities in an external knowledge base are utilized more fully, noise interference existing in data sets is reduced, a model is more stable, and more accurate relation prediction can be obtained.
Drawings
FIG. 1 is an overall flow chart of the present application;
fig. 2 is a schematic diagram of a model for extracting a relation of a single sentence packet according to the present application.
Detailed Description
It should be noted that, in the present invention, the embodiments disclosed in the present application may be combined with each other without conflict.
The first embodiment is as follows: specifically, the present embodiment is described with reference to fig. 1, and the biomedical relationship extraction method incorporating external knowledge according to the present embodiment includes the steps of:
performing word embedding and position embedding operation on each word in each sentence in the biomedical data set to obtain a word vector and a position vector, splicing the obtained word vector and position vector to obtain vector representation of each word, and finally splicing the vector representations of all words in the sentences to obtain matrix representation of each sentence;
inputting the matrix representation of each sentence obtained in the step one into a PCNN neural network to obtain the vector representation of each sentence in the biomedical data set;
acquiring a head entity and a tail entity of each sentence in the biomedical data set, respectively extracting an entity related to the head entity and an entity related to the tail entity from an external knowledge graph to obtain a relation graph taking the head entity as a center and a relation graph taking the tail entity as a center, and inputting the two obtained relation graphs into a graph encoder to construct comprehensive vector representation of the external knowledge graph of the head entity and the tail entity;
combining the vector representation of each sentence in the biomedical data set with the comprehensive vector of the external knowledge map of the head and tail entities to obtain a sentence vector containing external information;
and step five, for each entity pair, selecting all sentences containing the entity pair to form a set, calculating the attention weight of each sentence in the set by adopting a sentence-level attention mechanism according to the sentence vector representation containing the external information, then taking the sum of the attention weights represented by all sentences containing the external information in the set as the vector representation of the set, and predicting the vector representation of the set to obtain the predicted relationship of the entity pair.
The second embodiment is as follows: this embodiment is further described with respect to the first embodiment, and the difference between this embodiment and the first embodiment is that the graph encoder in step three employs a KG-Transformer.
The third concrete implementation mode: this embodiment is further described with respect to the second embodiment, and the difference between this embodiment and the second embodiment is that the KG-Transformer encoding process is as follows:
KG-Transformer represents X ═ X in the vector of the node sequence in the two input relational graphs1,x2,...,xNIs inputted to the Muti-head attachment Layer and Add&Norm Layer：
The Muti-head attachment Layer is calculated as follows:
Figure BDA0003008193450000051
Figure BDA0003008193450000052
Figure BDA0003008193450000053
wherein the content of the first and second substances,
Figure BDA0003008193450000054
denotes the concatenation of the H attention heads of this layer, x'iRepresenting the node representation of the output, A is the adjacency matrix, i represents the ith row, j represents the jth column, d is the dimension of node embedding,
Figure BDA0003008193450000055
all the data are weight matrixes, N represents the length of a node sequence, Masking (X, A) represents that the value of the corresponding position of the matrix X is subjected to Masking operation according to the position of the median value of the matrix A being 1;
the Add & Norm Layer was calculated as follows:
O＝LayerNorm(X+X')
wherein X ═ { X ═ X1,x2,......,xNX is a vector representation of a sequence of nodes, X '═ X'1,x'2,......x'NX' is the output calculated by the Muti-head attachment Layer, LayerNorm (-) is a Layer normalization function, and the output result O is used as the input of the next Layer of the Muti-head attachment Layer;
repeating the calculation process for L times, wherein L is a hyperparameter, L is any integer, and is generally 8, 12, 16 and 24, obtaining vector representations of all nodes, and finally summing the vector representations of all nodes of the relational graph of the head entity and the tail entity respectively to obtain the comprehensive vector representation of the head entity and the tail entity.
The fourth concrete implementation mode: this embodiment mode is a further description of the first embodiment mode, and the difference between this embodiment mode and the first embodiment mode is that L is 8, 12, 16, or 24.
The fifth concrete implementation mode: the present embodiment is a further description of the first embodiment, and the difference between the present embodiment and the first embodiment is that the PCNN neural network in the second step obtains a vector representation of each sentence in the biomedical data set through convolution, pooling and nonlinear operations.
The sixth specific implementation mode: the present embodiment is further described with respect to the first embodiment, and the difference between the present embodiment and the first embodiment is that the PCNN neural network in the second step specifically executes the following steps:
firstly, extracting local features from the matrix representation of the sentence by adopting a convolution kernel with the sliding window size of 3, and then combining all the local features through maximum pool operation to obtain a vector of the matrix representation of the sentence;
then dividing the vector represented by the matrix of the obtained sentence into three segments according to the positions of the head entity and the tail entity, and then pooling each segment to obtain a three-dimensional vector:
and splicing the three-dimensional vectors corresponding to all the convolution kernels, and performing non-linearization by using an activation function to obtain the final vector representation of the sentence.
The seventh embodiment: this embodiment mode is a further description of a sixth embodiment mode, and is different from the sixth embodiment mode in that the activation function is a tanh function.
The specific implementation mode is eight: this embodiment is a further description of a sixth embodiment, and a difference between this embodiment and the sixth embodiment is that a vector of a node sequence in the relational graph is represented as:
(e,r1,e1,r2,e2,...,rn,en)
wherein, entity e and entity e1,e2,...,enAssociating entity e with entity e1,e2,...,enRespectively is r1,r2,...,rn。
The specific implementation method nine: the present embodiment is further described with respect to the eighth embodiment, and the difference between the present embodiment and the eighth embodiment is that, in the fourth step, the vector representation of each sentence in the biomedical data set and the integrated vector of the external knowledge base of the head-tail entity are represented by combining:
Figure BDA0003008193450000071
MutiHead(Q,K,V)＝Concat(head1,...,headh)Wo
in the above formula, Q represents a matrix representation of each sentence, K, V represents a vector representation of each sentence, and K and V are equal;
Figure BDA0003008193450000072
Worepresenting weight matrices, heads, within neural networksiRepresenting a certain head of the calculation, a different head representing a different expression, Concat (head), of the biomedical statement1,...,headh) Representing the vectors resulting from concatenating the different headers.
The detailed implementation mode is ten: this embodiment is a further description of a ninth embodiment, and the difference between this embodiment and the ninth embodiment is that the specific step of the fifth step is:
the attention weight of each sentence in the set of sentences is calculated using a sentence-level attention mechanism, with the sum of the attention weights of all the sentence representations as a vector representation of the set of sentences, i.e.
Figure BDA0003008193450000073
Figure BDA0003008193450000074
ei＝xiAr
αiIs a sentence vector xiWeight of (1), xiIs a vector representation of the ith sentence, a is a diagonal weight matrix, r is a vector representation of the relationship r, s represents a vector representation of the set of sentences;
and finally, according to the vector representation s of the sentence set, calculating the probability that the sentence set belongs to the relation r by using a softmax classifier as follows:
P(r|S；θ)＝softmax(Ws+b)
wherein W represents a weight matrix, S represents a sentence vector, b represents a bias term, theta is a model parameter, and S represents the sentence set.
Example (b): a biomedical relationship extraction method incorporating external knowledge, comprising:
step one, embedding words of each sentence in a sentence sub-packet, splicing word embedding vectors and position embedding vectors of each word to obtain vector representation corresponding to each word, and then splicing all word vectors in each sentence to obtain matrix representation of the sentence.
Inputting the matrix representation of the sentences into a CNN layer and a piece-Max _ posing layer, and obtaining the vector representation of each sentence through convolution, pooling and nonlinear operation.
The CNN layer is a convolutional layer, which first extracts a local feature with a sliding window length of 3 above the sentence. It then combines all local features by max-pool operation to obtain a fixed size vector for the input sentence. Here, convolution is defined as the operation between a sequence of vectors W and a convolution matrix W,
Figure BDA0003008193450000081
dcis the embedding dimension of the sentence, the convolution operation can extract the local features through a sliding window with length l.
In the invention, considering the positions of two entities, the pooling operation can be further improved into segmented pooling, and each dimension characteristic p obtained by convolution is taken asiFrom the head entity and the tail entity into three segments (p)i1,pi2,pi3) Then pooling is performed separately for each segment:
[xij]＝max(pij)
then [ x ]i]Is defined as [ x ]ij]And (4) splicing.
At the end of this step, the vector x is subjected to a non-linearization, such as a tanh function, resulting in a final vector representation of the sentence.
Step two, each sentence in the biomedical data set comprises a head entity and a tail entity of the sentence, the head entity and the tail entity have relations with other entities in an external knowledge base, and the relations are expressed in a form of triples, namely the triples<Entity 1, relationship, entity 2>. And representing a plurality of triples in a mode of a graph, wherein corresponding relation nodes exist between two entity nodes in the graph and are connected with the entity nodes, so that the head entity and the tail entity are respectively associated with the entities in a plurality of external knowledge bases. The entity nodes and the relation nodes in the graph are converted into a sequence form (e, r1, e)1,r2,e2,...,rn,en) Wherein entity e and entity e1,e2,...,enAre related and have a corresponding relationship of r1,r2,...,rn。
The obtained head-tail entity relation sequence
Figure BDA0003008193450000082
And
Figure BDA0003008193450000083
and embedding the nodes, inputting the nodes into a KG-Transformer model for feature extraction, and obtaining KG expressions of the head entity and the tail entity.
Wherein the KG-Transformer model represents X ═ X as the synthetic vector of the input node sequence1,x2,...,xnIs inputted to the Muti-head attachment Layer and Add&Norm Layer：
Figure BDA0003008193450000084
Figure BDA0003008193450000085
Figure BDA0003008193450000086
Wherein the content of the first and second substances,
Figure BDA0003008193450000091
the connections of the H attention heads of this layer are shown,
Figure BDA0003008193450000092
and
Figure BDA0003008193450000093
respectively represent the h-th attention head node xjAnd node embedding XjWeight of the linear transformation of (1).
The transform blocks are stacked L times, and finally KG representation of the head and tail entities is obtained.
And step three, performing Knowledge-Attention operation on the results obtained in the two steps, capturing the internal correlation of the data and the features in the biomedical text by using a multi-head Attention mechanism, and fusing the obtained feature vectors with external Knowledge. The calculation process is as follows:
Figure BDA0003008193450000094
MutiHead(Q,K,V)＝Concat(head1,...,headh)Wo
in the above formula, Q is the comprehensive vector representation obtained in the first step, K and V are the comprehensive vector representation described in the second step, and K and V are equal;
Figure BDA0003008193450000095
Worepresenting weight matrices, heads, within neural networksiRepresenting a certain head of the calculation, different heads being understood as different expressions, Concat (head), to the biomedical statement1,...,headh) Representing the vectors resulting from concatenating the different headers.
Step four, defining the weight represented by each sentence vector by adopting a set sentence level attention mechanism;
in this step, a query-based function is used to measure the vector representation x of each sentenceiThe degree of association with the relationship r of the entity pair to be predicted finally.
Since the information of the relationship r between the pair of entities to be predicted finally is taken into consideration, with the sentence level attention mechanism set, the influence of noise is reduced by assigning a smaller weight to a noise sentence.
Finally, given the set of all sentences and the pair of entities, the probability of defining the predicted relationship r is:
Figure BDA0003008193450000096
wherein n isrThe number of all relation types, o is the input of the final neural network, and o is Ms+ d, where d is the offset vector and M is the all-relationship vector representation matrix.
It should be noted that the detailed description is only for explaining and explaining the technical solution of the present invention, and the scope of protection of the claims is not limited thereby. It is intended that all such modifications and variations be included within the scope of the invention as defined in the following claims and the description.

Claims (10)

1. A biomedical relation extraction method integrated with external knowledge is characterized by comprising the following steps:
performing word embedding and position embedding operation on each word in each sentence in the biomedical data set to obtain a word vector and a position vector, splicing the obtained word vector and position vector to obtain vector representation of each word, and finally splicing the vector representations of all words in the sentences to obtain matrix representation of each sentence;
inputting the matrix representation of each sentence obtained in the step one into a PCNN neural network to obtain the vector representation of each sentence in the biomedical data set;
acquiring a head entity and a tail entity of each sentence in the biomedical data set, respectively extracting an entity related to the head entity and an entity related to the tail entity from an external knowledge graph to obtain a relation graph taking the head entity as a center and a relation graph taking the tail entity as a center, and inputting the two obtained relation graphs into a graph encoder to construct comprehensive vector representation of the external knowledge graph of the head entity and the tail entity;
combining the vector representation of each sentence in the biomedical data set with the comprehensive vector of the external knowledge map of the head and tail entities to obtain a sentence vector containing external information;
and step five, for each entity pair, selecting all sentences containing the entity pair to form a set, calculating the attention weight of each sentence in the set by adopting a sentence-level attention mechanism according to the sentence vector representation containing the external information, then taking the sum of the attention weights represented by all sentences containing the external information in the set as the vector representation of the set, and predicting the vector representation of the set to obtain the predicted relationship of the entity pair.
2. The method of claim 1, wherein the histogram coder employs KG-Transformer.
3. The method for extracting biomedical relations fused with external knowledge according to claim 2, wherein the KG-Transformer is encoded by:
KG-Transformer represents X ═ X in the vector of the node sequence in the two input relational graphs1,x2,...,xNIs inputted to the Muti-head attachment Layer and Add&Norm Layer：
The Muti-head attachment Layer is calculated as follows:
Figure FDA0003354503050000011
Figure FDA0003354503050000012
Figure FDA0003354503050000013
wherein the content of the first and second substances,
Figure FDA0003354503050000021
denotes the concatenation of the H attention heads of this layer, x'iRepresenting the node representation of the output, A is the adjacency matrix, i represents the ith row, j represents the jth column, d is the dimension of node embedding,
Figure FDA0003354503050000022
all are weight matrixes, N represents the length of a node sequence, Masking (X, A) represents that the value of the corresponding position of the matrix X is masked according to the position of the value of 1 in the matrix A,
Figure FDA0003354503050000023
indicating the h-th head of attention with respect to node xjH denotes the h-th attention head,
Figure FDA0003354503050000024
to represent
Figure FDA0003354503050000025
Summing all row elements of (a), I being an identity matrix;
the Add & Norm Layer was calculated as follows:
O＝LayerNorm(X+X')
wherein X ═ { X ═ X1,x2,......,xNX is a vector of a node sequenceDenotes, X '═ X'1,x'2,......x'NX' is the output calculated by the Muti-head attachment Layer, LayerNorm (-) is a Layer normalization function, and the output result O is used as the input of the next Layer of the Muti-head attachment Layer;
repeating the calculation process for L times, wherein L is any integer, obtaining the vector representation of all nodes, and finally summing the vector representations of all nodes of the relational graph of the head entity and the tail entity respectively to obtain the comprehensive vector representation of the head entity and the tail entity.
4. The method of claim 3, wherein L is 8, 12, 16 or 24.
5. The method of claim 1, wherein the PCNN neural network in the second step obtains the vector representation of each sentence in the biomedical data set through convolution, pooling and nonlinear operations.
6. The method as claimed in claim 1, wherein the PCNN neural network in the second step specifically performs the following steps:
firstly, extracting local features from the matrix representation of the sentence by adopting a convolution kernel with the sliding window size of 3, and then combining all the local features through maximum pool operation to obtain a vector of the matrix representation of the sentence;
then dividing the vector represented by the matrix of the obtained sentence into three segments according to the positions of the head entity and the tail entity, and then pooling each segment to obtain a three-dimensional vector:
and splicing the three-dimensional vectors corresponding to all the convolution kernels, and performing non-linearization by using an activation function to obtain the final vector representation of the sentence.
7. The method of claim 6, wherein the activation function is a tanh function.
8. The method of claim 6, wherein the vectors of the node sequences in the relational graph are represented as follows:
(e,r1,e1,r2,e2,...,rn,en)
wherein, entity e and entity e1,e2,...,enAssociating entity e with entity e1,e2,...,enRespectively is r1,r2,...,rn。
9. The method for extracting biomedical relations fused with external knowledge according to claim 8, wherein the step four combines the vector representation of each sentence in the biomedical data set and the comprehensive vector of the external knowledge-graph of the head and tail entities into a representation:
Figure FDA0003354503050000031
MutiHead(Q,K,V)＝Concat(head1,...,headh)Wo
in the above formula, Q represents a matrix representation of each sentence, K, V represents a vector representation of each sentence, and K and V are equal;
Figure FDA0003354503050000032
Worepresenting weight matrices, heads, within neural networksiRepresenting a certain head of the calculation, a different head representing a different expression, Concat (head), of the biomedical statement1,...,headh) Representing the vectors resulting from the concatenation of the different heads, h denotes the h-th attention head.
10. The method for extracting biomedical relations integrated with external knowledge as claimed in claim 9, wherein the step five comprises the following specific steps:
the attention weight of each sentence in the set of sentences is calculated using a sentence-level attention mechanism, with the sum of the attention weights of all the sentence representations as a vector representation of the set of sentences, i.e.
Figure FDA0003354503050000033
Figure FDA0003354503050000034
ei＝xiAr
αiIs a sentence vector xiWeight of (1), xiIs a vector representation of the ith sentence, a is a diagonal weight matrix, r is a vector representation of the relationship r, s represents a vector representation of the set of sentences;
and finally, according to the vector representation s of the sentence set, calculating the probability that the sentence set belongs to the relation r by using a softmax classifier as follows:
P(r|S；θ)＝softmax(Ws+b)
wherein W represents a weight matrix, S represents a sentence vector, b represents a bias term, theta is a model parameter, S represents the sentence set, eiRepresenting the ith sentence xiScore of degree of match with its relation r, ekRepresenting the k-th sentence xiThe degree of match with its relationship r is scored.
CN202110367973.9A 2021-04-06 2021-04-06 External knowledge-integrated biomedical relation extraction method Active CN112860904B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110367973.9A CN112860904B (en) 2021-04-06 2021-04-06 External knowledge-integrated biomedical relation extraction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110367973.9A CN112860904B (en) 2021-04-06 2021-04-06 External knowledge-integrated biomedical relation extraction method

Publications (2)

Publication Number Publication Date
CN112860904A CN112860904A (en) 2021-05-28
CN112860904B true CN112860904B (en) 2022-02-22

Family

ID=75992228

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110367973.9A Active CN112860904B (en) 2021-04-06 2021-04-06 External knowledge-integrated biomedical relation extraction method

Country Status (1)

Country Link
CN (1) CN112860904B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113536795B (en) * 2021-07-05 2022-02-15 杭州远传新业科技有限公司 Method, system, electronic device and storage medium for entity relation extraction
CN114064938B (en) * 2022-01-17 2022-04-22 中国人民解放军总医院 Medical literature relation extraction method and device, electronic equipment and storage medium
CN114579755A (en) * 2022-01-26 2022-06-03 北京博瑞彤芸科技股份有限公司 Method and device for constructing traditional Chinese medicine knowledge map

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108959328B (en) * 2017-05-27 2021-12-21 株式会社理光 Knowledge graph processing method and device and electronic equipment
CN107391623B (en) * 2017-07-07 2020-03-31 中国人民大学 Knowledge graph embedding method fusing multi-background knowledge
CN108984745B (en) * 2018-07-16 2021-11-02 福州大学 Neural network text classification method fusing multiple knowledge maps
CN109635124B (en) * 2018-11-30 2021-04-23 北京大学 Remote supervision relation extraction method combined with background knowledge
CN109710932A (en) * 2018-12-22 2019-05-03 北京工业大学 A kind of medical bodies Relation extraction method based on Fusion Features
CN109902171B (en) * 2019-01-30 2020-12-25 中国地质大学（武汉） Text relation extraction method and system based on hierarchical knowledge graph attention model
CA3076638A1 (en) * 2019-03-22 2020-09-22 Royal Bank Of Canada Systems and methods for learning user representations for open vocabulary data sets
CN111291139B (en) * 2020-03-17 2023-08-22 中国科学院自动化研究所 Knowledge graph long-tail relation completion method based on attention mechanism
CN111260064A (en) * 2020-04-15 2020-06-09 中国人民解放军国防科技大学 Knowledge inference method, system and medium based on knowledge graph of meta knowledge
CN111931506B (en) * 2020-05-22 2023-01-10 北京理工大学 Entity relationship extraction method based on graph information enhancement

Also Published As

Publication number Publication date
CN112860904A (en) 2021-05-28

Similar Documents

Publication Publication Date Title
CN112860904B (en) External knowledge-integrated biomedical relation extraction method
Zhang et al. Multi-scale attention with dense encoder for handwritten mathematical expression recognition
CN106886543B (en) Knowledge graph representation learning method and system combined with entity description
CN110134946B (en) Machine reading understanding method for complex data
CN110287323B (en) Target-oriented emotion classification method
CN111681718A (en) Medicine relocation method based on deep learning multi-source heterogeneous network
CN112232087A (en) Transformer-based specific aspect emotion analysis method of multi-granularity attention model
CN112100212A (en) Case scenario extraction method based on machine learning and rule matching
CN113761893A (en) Relation extraction method based on mode pre-training
CN115687609A (en) Zero sample relation extraction method based on Prompt multi-template fusion
CN110889505A (en) Cross-media comprehensive reasoning method and system for matching image-text sequences
CN112559741B (en) Nuclear power equipment defect record text classification method, system, medium and electronic equipment
CN113076391B (en) Remote supervision relation extraction method based on multi-layer attention mechanism
CN115188440A (en) Intelligent matching method for similar medical records
CN113705242A (en) Intelligent semantic matching method and device for education consultation service
CN111783688B (en) Remote sensing image scene classification method based on convolutional neural network
CN110288002B (en) Image classification method based on sparse orthogonal neural network
CN114997360B (en) Evolution parameter optimization method, system and storage medium of neural architecture search algorithm
CN111259106A (en) Relation extraction method combining neural network and feature calculation
CN108762523A (en) Output characters through input method prediction technique based on capsule networks
CN115204171A (en) Document-level event extraction method and system based on hypergraph neural network
CN114169447A (en) Event detection method based on self-attention convolution bidirectional gating cyclic unit network
CN113901758A (en) Relation extraction method for knowledge graph automatic construction system
CN111858682A (en) Judgment document logic evaluation method and system based on deep learning
Liu et al. Sentiment analysis of MOOC reviews based on capsule network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant