CN113946665A - Knowledge base question-answering method for providing background information based on text - Google Patents

Knowledge base question-answering method for providing background information based on text Download PDF

Info

Publication number
CN113946665A
CN113946665A CN202111066945.XA CN202111066945A CN113946665A CN 113946665 A CN113946665 A CN 113946665A CN 202111066945 A CN202111066945 A CN 202111066945A CN 113946665 A CN113946665 A CN 113946665A
Authority
CN
China
Prior art keywords
entity
representation
document
question
knowledge base
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111066945.XA
Other languages
Chinese (zh)
Other versions
CN113946665B (en
Inventor
冯程程
姜琳颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeastern University China
Original Assignee
Northeastern University China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeastern University China filed Critical Northeastern University China
Priority to CN202111066945.XA priority Critical patent/CN113946665B/en
Publication of CN113946665A publication Critical patent/CN113946665A/en
Application granted granted Critical
Publication of CN113946665B publication Critical patent/CN113946665B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention belongs to the technical field of deep learning and intelligent question answering, and relates to a knowledge base question answering method for providing background information based on texts, wherein the representation of an entity is learned from a retrieved knowledge base subgraph through an entity representation module, and the problem representation is updated through fusing seed entities; learning, by a document characterization module, a correct representation of a document associated with a given problem, and updating the problem representation by fusing document information; in the answer prediction module, obtaining a final entity representation, a document representation and a question representation through an entity representation module and a document representation module; and splicing the entity representations and the corresponding document representations, and calculating a dot product with the problem to obtain a matching score. The invention creatively integrates the knowledge base information and the text information into the question, realizes the purpose of providing background knowledge by utilizing the knowledge base and the text information, improves the performance of the question-answering system and has strong expansibility.

Description

Knowledge base question-answering method for providing background information based on text
Technical Field
The invention belongs to the technical field of deep learning and intelligent question answering, relates to a knowledge base question answering method for providing background information based on texts, and particularly relates to an intelligent question answering method using two data sources of texts and a knowledge base.
Background
The Knowledge Base (KB) is considered as an essential resource for answering factual questions. However, a KB that is accurately built using complex patterns requires a significant amount of manpower, which inevitably limits the coverage of the KB. In fact, KB is often incomplete, not enough to cover all the evidence needed for the problem. On the other hand, the vast amount of unstructured text on the internet can easily cover a wide range of evolving knowledge that can be commonly used in question-and-answer systems. Therefore, in order to improve the coverage of the KB, researchers believe that the KB can be extended directly using text data. Currently, there are many possible ways to use text to enhance the performance of a knowledge base question-and-answer system. One method is to split the knowledge base tuple into entities and relations, split the text into entities and remove the remaining part of the entities (called patten), encode the entities by using a Universal Schema, and use the encoded text as the input of a memory network input module, and then obtain the predicted answer through a generalization module, an output module and a response module. The document is regarded as a heterogeneous node, the heterogeneous node and an entity in a knowledge base are combined into a unified graph, and the representation of each node is obtained in a graph convolution mode and is used for performing classification and predicting answers. Recently, researchers have pioneered the use of hypergraph convolutional networks to exploit the higher-order relationships contained in text.
These methods only consider the problem of incomplete answer coverage when the knowledge base of answer relief contained in the text is used as the only information source. Another information source may serve to provide clues to answers (background knowledge) when not considered in retrieving answers in the knowledge base or text. Xiong et al believe that when searching for answers in text, the information in the knowledge base may provide background knowledge to distinguish between relevant and irrelevant information (see FIG. 1(a)) (Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaooxino Guo, and William Yang Wang Wang.Impropering query in computer kbs with knowledge-aware reader. in Annual Meeting of the Association for computerized linkage, 2019.). Their approach is better at answering a first type of question, but it may encounter difficulties in answering a second type of question (see fig. 1(b)) because it is difficult to return an accurate answer when there are some constraints in the question that cannot be found directly in KB. The relationships and entities corresponding to the constraint "debug with Carolina Panthers" cannot be found directly in the subgraph of problem 2. When the answer is retrieved in the sub-graph, the algorithm does not know what the constraint "debug with Carolina Panthers" mentioned in the question refers to in the sub-graph, nor what the time mentioned in the question refers to. Thus, the system is likely to return to newtons when to do other things. From the foregoing analysis, we consider that in addition to the situation in fig. 1(a), when the answer is in a sub-graph, it is also necessary to use textual information to provide background knowledge (see fig. 1 (b)).
The invention utilizes the end-to-end neural network to improve the traditional information retrieval-based method, and combines a knowledge base and an entity link text as an information source. In addition to text data that may increase answer coverage of the knowledge base, the present invention recognizes that the knowledge base and text may complement each other by providing background knowledge to each other (see two examples in FIG. 1). The method fuses the knowledge base and the text information into the problem representation, and achieves the purposes of providing background knowledge and enhancing the problem representation. And enhances the interaction between the knowledge base, text and questions.
Disclosure of Invention
The Knowledge Base (KB) is widely used in the question-answering (QA) task to provide appropriate answers to a given question, i.e., KBQA questions. However, the knowledge base itself may be incomplete (e.g., incomplete answer coverage and incomplete context constraints in the knowledge base), which limits the overall performance of existing KBQA models. To solve this problem, the present invention proposes a new model TB-model, which enhances knowledge base coverage by text supplementation with additional answers and uses text to provide background information to enhance the representation of the problem with complex constraints, thereby improving the accuracy of answer retrieval.
The invention provides an intelligent question-answering method using two information sources of a text and a knowledge base, which comprises the following steps: and calculating the similarity of the relationship between the problem and each entity in the knowledge base subgraph to obtain a problem relationship matching score. And judging whether the neighbor entity is the subject entity, and generating the matching score of the problem neighbor according to the obtained problem relation matching score and the binary index of judging whether the neighbor entity is the subject entity. And fusing the neighbor information of each entity according to the matching score of the problem neighbor to obtain the entity representation containing the neighbor information. And integrating the seed entity characteristics into the problem characteristics, and updating the problem characteristics for the first time. The resulting entities are used to characterize the representation of each word-segmentation in the augmented document. And performing attention calculation between the question and the document to obtain a document representation containing the attention of the question. And fusing the document representation corresponding to the subject entity into the problem representation, and updating the problem representation for the second time. And splicing the entity representation with the document representation containing the entity, and matching the entity representation with the problem. And outputting the entity with the highest matching score as the answer.
The technical scheme of the invention is as follows:
a knowledge base question-answering method for providing background information based on texts is characterized by comprising the following steps:
s1, learning the representation of the entity from the retrieved knowledge base subgraph through an entity representation module, and updating the problem representation through fusing seed entities;
s2, learning the correct representation of the document related to the given question through a document representation module, and updating the question representation through fusing document information;
s3, in the answer prediction module, obtaining the final entity representation, the document representation and the question representation through the entity representation module in the step S1 and the document representation module in the step S2; and splicing the entity representations and the corresponding document representations, and calculating a dot product with the problem to obtain a matching score.
Further, step S1 specifically includes the following steps:
s101, after bi-LSTM coding, the hidden state of each word of the question and the relation is respectively
Figure BDA0003258803570000031
And
Figure BDA0003258803570000032
wherein d ishIs the hidden layer dimension; lqAnd lrLength of question and length of relation, respectively; r is a real number set; h isqHidden state for each word of the question; h isrHidden states for each word of the relationship; a representation of the self-attention mechanism relationship is then obtained:
Figure BDA0003258803570000033
wherein the content of the first and second substances,
Figure BDA0003258803570000034
is hrThe number of the ith row of (a),
Figure BDA0003258803570000035
for the trained vector, αiIs a correlation coefficient;
Figure BDA0003258803570000036
a relational representation generated for a self-attention mechanism;
use of
Figure BDA0003258803570000037
Calculating the degree of correlation with each participle of the question, taking the obtained degree of correlation of each participle as weight, calculating the sum of the weights, and matching the question and the relation by dot productrModeling:
Figure BDA0003258803570000038
wherein, betaiIs a correlation coefficient;
Figure BDA0003258803570000039
j line, s, of hqrMatching scores for questions and relationships; the ith neighbor to entity e (r)iEi) final match score s (r)iEi) is defined as:
Figure BDA00032588035700000310
wherein, I is a binary indicator and represents an entity eiWhether or not it is a seed entity, epsilon0Is a set of seed entities that is,
Figure BDA0003258803570000041
a matching score for the question and the ith relationship;
s102, obtaining a final matching score S (r) between the question and the neighbor according to the step S101i,ei) And controlling information propagation of the neighbors by using a linear gate function to obtain the representation of each entity containing the neighbor information:
Figure BDA0003258803570000042
Figure BDA0003258803570000043
g(x,y)＝sigmoid(W[x；y])
wherein
Figure BDA0003258803570000044
A representation of the entity containing the neighbor information,
Figure BDA0003258803570000045
and
Figure BDA0003258803570000046
for pre-trained tuples (e, r)i,ei) Head entities e and e iniIs embedded into the Glove knowledge graph of (1),
Figure BDA0003258803570000047
for a trainable transformation matrix, σ (-) is the activation function, Ne is the set of all neighbors of entity e, g (x, y) is the defined function, γeIs a compromise parameter for the calculation of the linear gate function;
s103, updating problem representation by fusing knowledge of a knowledge base of the seed entity after the representation that each entity contains neighbor information is obtained in the step S102:
Figure BDA0003258803570000048
Figure BDA0003258803570000049
Figure BDA00032588035700000410
a characterization of the problem obtained from the attention mechanism,
Figure BDA00032588035700000411
is the average value of the seed entity characterization, γ q is the compromise parameter of the gate function calculation,
Figure BDA00032588035700000412
updating the problem representation by fusing knowledge of the knowledge base of the seed entity; w is aq,
Figure BDA00032588035700000413
Are parameters for learning.
Further, step S2 specifically includes the following steps:
s201, fusing the text word segmentation features and the entity features corresponding to the text word segmentation features by using a conditional gate function to obtain knowledge-enhanced document word segmentation features:
Figure BDA00032588035700000414
Figure BDA0003258803570000051
wherein the content of the first and second substances,
Figure BDA0003258803570000052
for the ith word-segmentation of the document,
Figure BDA0003258803570000053
is a feature of the ith word-segmentation,
Figure BDA0003258803570000054
is a representation of the entity to which the ith participle is linked, and γ d is a compromise parameter for the gate function calculation,
Figure BDA0003258803570000055
segmenting word features for the knowledge-enhanced document;
s202, the knowledge-enhanced document word segmentation characteristics obtained in the step S201
Figure BDA0003258803570000056
Using Bi-LSTM encoding as input, the hidden state at the participle level is output
Figure BDA0003258803570000057
Wherein d ishIs the hidden layer dimension, ldFor document length:
hd＝Bi-LSTM(fd)
wherein f isdIs composed of
Figure BDA0003258803570000058
The matrix of the composition is composed of a plurality of matrixes,
Figure BDA0003258803570000059
is fdRow i of (1);
s203, using an altering co-annotation mechanism to solve the problem and the hidden state h of each participle of the documentq，hdAs input, a final document representation is obtained
Figure BDA00032588035700000510
Figure BDA00032588035700000511
Figure BDA00032588035700000512
H＝tanh(WxX+(Wgg)1T)
Wherein the content of the first and second substances,
Figure BDA00032588035700000513
is a vector with all elements 1, Wx,
Figure BDA00032588035700000514
And
Figure BDA00032588035700000515
is a parameter of learning, X is the input document feature, g is the lead of attention derived from the question, H is the intermediate state obtained, axIs the attention weight of feature X;
s204, fusing the question and the document: obtaining a representation of a document
Figure BDA00032588035700000516
Then, the information of the document containing the entity e is gathered by taking an average as follows:
Figure BDA00032588035700000517
the problem of fusing document information is characterized as follows:
Figure BDA0003258803570000061
Figure BDA0003258803570000062
Figure BDA0003258803570000063
wherein the content of the first and second substances,
Figure BDA0003258803570000064
is the average of all document tokens that contain the subject entity,
Figure BDA0003258803570000068
is a compromise parameter for the calculation of the linear gate function.
Further, step S203 specifically includes the following steps:
first step input X ═ hd，g＝0；
The second step X ═ hqG is the document characteristics output in the first step;
the third step X ═ hdG is the problem characteristic output in the second step; the output of the third step is used as the final document characterization
Figure BDA0003258803570000065
Further, step S3 specifically includes the following steps: the entity tokens and the corresponding document tokens are spliced and the dot product is calculated with the problem to obtain the matching scores of the entity tokens and the corresponding document tokens, as follows:
Figure BDA0003258803570000066
wherein s iseTo be the matching score of the question and candidate entity e,
Figure BDA0003258803570000067
is a parameter matrix of the transform dimension.
The invention has the advantages that: the invention creatively integrates knowledge base information and text information into the problem, and realizes the purpose of mutually providing background knowledge by utilizing the knowledge base and the text information. The performance of the question answering system is improved. The expansibility is strong.
Drawings
FIGS. 1(a) and 1(b) are explanatory views of how the present invention works; wherein FIG. 1(a) is the answer in the document; FIG. 1(b) shows the answer in the knowledge base.
FIG. 2 is a general model diagram of the present invention.
Detailed Description
The following further describes a specific embodiment of the present invention with reference to the drawings and technical solutions.
Examples
Task definition:
specifically, the QA task considered in embodiments of the present invention requires answering the question by reading the knowledge base tuple K { (es, r, eo) } and retrieving the wikipedia document set D. To build a scalable system, the present invention considers only one sub-graph per problem. The sub-graph is retrieved by the topic (seed) entity (the entity mentioned in the problem: E0 ═ E ∈ Q }). The document set D is retrieved by an existing document retriever and further sorted by Lucene index. Entities in the document are also annotated and linked to the knowledge base entities. For each question, the TB-model attempts to retrieve the answer entity from a candidate set that includes all knowledge base entities and document set entities.
The knowledge base question-answering method based on the text provides background information and is based on a TB-model, wherein the TB-model comprises an entity representation module, a document representation module and an answer prediction module; the entity representation module is used for learning the representation of the entity from the retrieved knowledge base subgraph and updating the problem representation by fusing the seed entity; the document representation module is used for learning the correct representation of the document related to the given problem and updating the problem representation again by fusing the document information; and the answer prediction module performs answer prediction according to the information of the knowledge base, the updated questions and the documents. The overall model diagram is shown in fig. 2. The method for providing background information based on the text comprises the following specific steps:
s1, in the entity characterization module, learning the representation of the entity from the retrieved knowledge base sub-graph, and updating the problem representation by fusing the seed entity.
Specifically, step S1 includes the steps of:
s101, before representing the entity, first obtain a matching score between a neighbor and the question by using a graph attention mechanism (in this embodiment, a 1-hop entity and a relationship connected to the candidate entity are regarded as neighbors), and the process includes matching the relationship between the question and the neighbor and determining whether the question includes the entity in the neighbor.
To match the problem and neighbor relations in the isomorphic potential space, a shared bi-LSTM coding problem and participle relation is employed. After bi-LSTM encoding, the hidden state of each word of the question and the relation is respectively
Figure BDA0003258803570000071
And
Figure BDA0003258803570000072
wherein d ishIs the hidden layer dimension; lqAnd lrLength of question and length of relation, respectively; r is a real number set; h isqHidden state for each word of the question; h isrAs a hidden state of each word of the relationship. According to
Figure BDA0003258803570000073
A representation of the self-attention mechanism relationship is obtained:
Figure BDA0003258803570000081
wherein the content of the first and second substances,
Figure BDA0003258803570000082
is hrThe number of the ith row of (a),
Figure BDA0003258803570000083
for the trained vector, αiIs a correlation coefficient;
Figure BDA0003258803570000084
is a self-attention mechanism.
Then, use
Figure BDA0003258803570000085
Calculating the degree of correlation with each word segmentation of the question, taking the obtained degree of correlation of each word segmentation as a weight, calculating the sum of the weights, and then performing dot product on the questionMatching score s of sum relationrModeling, as follows:
Figure BDA0003258803570000086
wherein, betaiIs a correlation coefficient;
Figure BDA0003258803570000087
is hqJ (th) row srMatching scores for questions and relationships;
in addition to the question-relationship similarity, if one neighbor entity is the subject entity in the question, then the corresponding tuple (e, r)i,ei) May be more likely to be relevant to the answer to the question than other non-subject neighbors, so a determination needs to be made as to whether an entity in a neighbor is a subject entity. The ith neighbor to entity e (r)i,ei) S (r) ofi,ei) Is defined as:
Figure BDA0003258803570000088
wherein, I is a binary indicator and represents an entity eiWhether or not it is a seed entity, epsilon0Is a set of seed entities that is,
Figure BDA0003258803570000089
is the matching score of the question with the ith relationship.
S102, obtaining a final matching score S (r) between the question and the neighbor according to the step S101i,ei) And controlling the information propagation of the neighbors by using a linear gate function, and obtaining the representation of each entity containing the neighbor information as follows:
Figure BDA00032588035700000810
Figure BDA00032588035700000811
g(x,y)＝sigmoid(W[x；y])
wherein
Figure BDA00032588035700000812
And
Figure BDA00032588035700000813
for pre-trained tuples (e, r)i,ei) Head entity e and tail entity e iniIs embedded into the Glove knowledge graph of (1),
Figure BDA00032588035700000814
σ (-) is the activation function for the trainable transformation matrix; ne is the set of all neighbors of entity e and g (x, y) is a defined function. In addition, γeIs a compromise parameter for the calculation of the linear gate function that controls how much of the original entity information should be retained.
And S103, updating the problem representation by fusing knowledge of the knowledge base of the seed entity after the representation that each entity contains the neighbor information is obtained in the step S102.
Figure BDA0003258803570000091
Figure BDA0003258803570000092
Figure BDA0003258803570000093
A characterization of the problem obtained from the attention mechanism,
Figure BDA0003258803570000094
is the mean value, gamma, of the seed entity characterizationqIs a compromise parameter of the gate function calculation,
Figure BDA0003258803570000095
to update the problem representation by fusing knowledge of the knowledge base of seed entities, wq,
Figure BDA0003258803570000096
Are parameters for learning.
S2, in the document characterization module, the correct representation of the document associated with the given question is learned and the question representation is further updated by fusing the document information.
Specifically, step S2 includes the steps of:
s201, using the character of document word segmentation designed by predecessor
Figure BDA0003258803570000097
(Danqi Chen, Adam Fisch, Jason Weston, and Autoine Bordes.2017.reading with knowledge to answer optional information questions. in Proceedings of the 55th environmental Meeting of the Association for the computerized Linear constraints, ACL 2017, Vancouver, Canada, July 30-August 4, Volume 1: Long Papers, pages 1870-:
Figure BDA0003258803570000098
Figure BDA0003258803570000099
wherein the content of the first and second substances,
Figure BDA00032588035700000910
for the ith word-segmentation of the document,
Figure BDA00032588035700000911
is a feature of the ith word segmentation.
Figure BDA00032588035700000912
Is the ith minuteCharacterization of the entity to which the word is linked, learned by the entity characterization Module, gammadIs a compromise parameter of the gate function calculation,
Figure BDA00032588035700000913
word segmentation features for knowledge-enhanced documents.
S202, the knowledge-enhanced document word segmentation characteristics obtained in the step S201
Figure BDA0003258803570000101
Using Bi-LSTM encoding as input, the hidden state at the participle level is output
Figure BDA0003258803570000102
Wherein d ishIs the hidden layer dimension, ldFor document length:
hd＝Bi-LSTM(fd)
wherein f isdIs composed of
Figure BDA0003258803570000103
The matrix of the composition is composed of a plurality of matrixes,
Figure BDA0003258803570000104
is fdRow i of (2).
S203, using an altering co-annotation mechanism to solve the problem and the hidden state h of each participle of the documentq，hdAs input, a final document representation is obtained
Figure BDA0003258803570000105
Figure BDA0003258803570000106
Figure BDA0003258803570000107
H＝tanh(WxX+(Wgg)1T)
Wherein 1 ∈ Rdh is a vector with all elements 1. Wx,
Figure BDA0003258803570000108
And
Figure BDA0003258803570000109
are parameters of learning. X is the input document (question) feature, g is the lead of attention derived from the question (document), H is the intermediate state obtained, axIs the attention weight of feature X.
First step input X ═ hdG is 0; the second step X ═ hqG is the document characteristics output in the first step; the third step X ═ hdAnd g is the problem characteristic output by the second step. The output of the third step is used as the final document characterization
Figure BDA00032588035700001016
S204, fusing the question and the document: obtaining a representation of a document
Figure BDA00032588035700001010
Then, we aggregate the information of the document containing entity e by taking the average as follows:
Figure BDA00032588035700001011
the problem of fusing document information is characterized as follows:
Figure BDA00032588035700001012
Figure BDA00032588035700001013
Figure BDA00032588035700001014
wherein the content of the first and second substances,
Figure BDA00032588035700001015
is the average of all document tokens that contain the subject entity,
Figure BDA00032588035700001017
is a compromise parameter for the calculation of the linear gate function.
S3, in the answer prediction module, obtaining the final entity representation through the entity representation module and the document representation module
Figure BDA0003258803570000111
Document characterization
Figure BDA0003258803570000112
Problem characterization
Figure BDA0003258803570000113
Then; the entity tokens and the corresponding document tokens are spliced together and the dot product is calculated with the problem to obtain the matching scores of the entity tokens and the corresponding document tokens, as follows:
Figure BDA0003258803570000114
wherein s iseTo be the matching score of the question and candidate entity e,
Figure BDA0003258803570000115
is a parameter matrix of the transform dimension.
The present invention trains the model using a binary cross entropy loss function. Excellent performance results were obtained on the webquestinsp dataset.
Therefore, the method and the device fuse the document information related to the question into the question representation through the gating function, so as to explain the constraint in the question, enhance the representation of the query and achieve the purpose of providing background information for answer retrieval.
The above description of exemplary embodiments has been presented only to illustrate the technical solution of the invention and is not intended to be exhaustive or to limit the invention to the precise form described. Obviously, many modifications and variations are possible in light of the above teaching to those skilled in the art. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to thereby enable others skilled in the art to understand, implement and utilize the invention in various exemplary embodiments and with various alternatives and modifications. It is intended that the scope of the invention be defined by the following claims and their equivalents.

Claims (6)

1. A knowledge base question-answering method for providing background information based on texts is characterized by comprising the following steps:
s1, learning the representation of the entity from the retrieved knowledge base subgraph through the entity representation module, and updating the problem representation through fusing seed entities;
s2, learning the correct representation of the document related to the given question through a document representation module, and updating the question representation through fusing document information;
s3, obtaining a final entity representation, a document representation and a question representation in the answer prediction module through the entity representation module in the step S1 and the document representation module in the step S2; and splicing the entity representations and the corresponding document representations, and calculating a dot product with the problem to obtain a matching score.
2. The knowledge base question-answering method for providing background information based on texts as claimed in claim 1, wherein the step S1 specifically comprises the following steps:
s101, after bi-LSTM coding, the hidden state of each word of the question and the relation is respectively
Figure FDA0003258803560000011
And
Figure FDA0003258803560000012
wherein d ishIs the hidden layer dimension; lqAnd lrLength of question and length of relation, respectively; r is a real number set; h isqHidden state for each word of the question; h isrHidden states for each word of the relationship; a representation of the self-attention mechanism relationship is then obtained:
Figure FDA0003258803560000013
wherein the content of the first and second substances,
Figure FDA0003258803560000014
is hrThe number of the ith row of (a),
Figure FDA0003258803560000015
for the trained vector, αiIs a correlation coefficient;
Figure FDA0003258803560000016
a relational representation generated for a self-attention mechanism;
use of
Figure FDA0003258803560000017
Calculating the degree of correlation with each participle of the question, taking the obtained degree of correlation of each participle as weight, calculating the sum of the weights, and matching the question and the relation by dot productrModeling:
Figure FDA0003258803560000018
wherein, betaiIs a correlation coefficient;
Figure FDA0003258803560000019
is hqJ (th) row srMatching scores for questions and relationships;
for instance, a pair of fruitsIth neighbor of body e (r)i,ei) S (r) ofi,ei) Is defined as:
Figure FDA00032588035600000110
wherein, I is a binary indicator and represents an entity eiWhether or not it is a seed entity, epsilon0Is a set of seed entities that is,
Figure FDA0003258803560000021
a matching score for the question and the ith relationship;
s102, obtaining a final matching score S (r) between the question and the neighbor according to the step S101i,ei) And controlling information propagation of the neighbors by using a linear gate function to obtain the representation of each entity containing the neighbor information:
Figure FDA0003258803560000022
Figure FDA0003258803560000023
g(x,y)＝sigmoid(W[x；y])
wherein
Figure FDA0003258803560000024
A representation of the entity containing the neighbor information,
Figure FDA0003258803560000025
and
Figure FDA0003258803560000026
for pre-trained tuples (e, r)i,ei) Head entities e and e iniIs embedded into the Glove knowledge graph of (1),
Figure FDA0003258803560000027
for a trainable transformation matrix, σ (-) is the activation function, Ne is the set of all neighbors of entity e, g (x, y) is the defined function, γeIs a compromise parameter for the calculation of the linear gate function;
s103, updating problem representation by fusing knowledge of a knowledge base of the seed entity after the representation that each entity contains neighbor information is obtained in the step S102:
Figure FDA0003258803560000028
Figure FDA0003258803560000029
Figure FDA00032588035600000210
a characterization of the problem obtained from the attention mechanism,
Figure FDA00032588035600000211
is the mean value, gamma, of the seed entity characterizationqIs a compromise parameter of the gate function calculation,
Figure FDA00032588035600000212
updating the problem representation by fusing knowledge of the knowledge base of the seed entity;
Figure FDA00032588035600000213
are parameters for learning.
3. The knowledge base question-answering method for providing background information based on texts according to claim 1 or 2, wherein the step S2 specifically comprises the following steps:
s201, fusing the text word segmentation features and the entity features corresponding to the text word segmentation features by using a conditional gate function to obtain knowledge-enhanced document word segmentation features:
Figure FDA0003258803560000031
Figure FDA0003258803560000032
wherein the content of the first and second substances,
Figure FDA0003258803560000033
for the ith word-segmentation of the document,
Figure FDA0003258803560000034
is a feature of the ith word-segmentation,
Figure FDA0003258803560000035
is a representation of the entity to which the ith participle is linked, γdIs a compromise parameter of the gate function calculation,
Figure FDA0003258803560000036
segmenting word features for the knowledge-enhanced document;
s202, the knowledge-enhanced document word segmentation characteristics obtained in the step S201
Figure FDA0003258803560000037
Using Bi-LSTM encoding as input, the hidden state at the participle level is output
Figure FDA0003258803560000038
Wherein d ishIs the hidden layer dimension, ldFor document length:
hd＝Bi-LSTM(fd)
wherein f isdIs composed of
Figure FDA0003258803560000039
The matrix of the composition is composed of a plurality of matrixes,
Figure FDA00032588035600000310
is fdRow i of (1);
s203, using an altering co-annotation mechanism to solve the problem and the hidden state h of each participle of the documentq，hdAs input, a final document representation is obtained
Figure FDA00032588035600000311
Figure FDA00032588035600000312
Figure FDA00032588035600000313
H＝tanh(WxX+(Wgg)1T)
Wherein the content of the first and second substances,
Figure FDA00032588035600000314
is a vector in which all elements are 1,
Figure FDA00032588035600000315
and
Figure FDA00032588035600000316
is a parameter of learning, X is the input document feature, g is the lead of attention derived from the question, H is the intermediate state obtained, axIs the attention weight of feature X;
s204, fusing the question and the document: obtaining a representation of a document
Figure FDA00032588035600000317
Then go toThe information for aggregating documents containing entity e by means of averaging is as follows:
Figure FDA0003258803560000041
the problem of fusing document information is characterized as follows:
Figure FDA0003258803560000042
Figure FDA0003258803560000043
Figure FDA0003258803560000044
wherein the content of the first and second substances,
Figure FDA0003258803560000045
is the average of all document tokens that contain the subject entity,
Figure FDA0003258803560000046
is a compromise parameter for the calculation of the linear gate function.
4. The knowledge base question-answering method for providing background information based on texts according to claim 3, wherein the step S203 specifically comprises the following steps:
first step input X ═ hd，g＝0；
The second step X ═ hqG is the document characteristics output in the first step;
the third step X ═ hdG is the problem characteristic output in the second step; the output of the third step is used as the final document characterization
Figure FDA0003258803560000047
5. The knowledge base question-answering method for providing background information based on texts according to claim 1, 2 or 4, wherein the step S3 specifically comprises the following steps: the entity tokens and the corresponding document tokens are spliced and the dot product is calculated with the problem to obtain the matching scores of the entity tokens and the corresponding document tokens, as follows:
Figure FDA0003258803560000048
wherein s iseTo be the matching score of the question and candidate entity e,
Figure FDA0003258803560000049
is a parameter matrix of the transform dimension.
6. The knowledge base question-answering method for providing background information based on texts according to claim 3, wherein the step S3 specifically comprises the following steps: the entity tokens and the corresponding document tokens are spliced and the dot product is calculated with the problem to obtain the matching scores of the entity tokens and the corresponding document tokens, as follows:
Figure FDA0003258803560000051
wherein s iseTo be the matching score of the question and candidate entity e,
Figure FDA0003258803560000052
is a parameter matrix of the transform dimension.
CN202111066945.XA 2021-09-13 2021-09-13 Knowledge base question-answering method for providing background information based on text Active CN113946665B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111066945.XA CN113946665B (en) 2021-09-13 2021-09-13 Knowledge base question-answering method for providing background information based on text

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111066945.XA CN113946665B (en) 2021-09-13 2021-09-13 Knowledge base question-answering method for providing background information based on text

Publications (2)

Publication Number Publication Date
CN113946665A true CN113946665A (en) 2022-01-18
CN113946665B CN113946665B (en) 2024-05-10

Family

ID=79328337

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111066945.XA Active CN113946665B (en) 2021-09-13 2021-09-13 Knowledge base question-answering method for providing background information based on text

Country Status (1)

Country Link
CN (1) CN113946665B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111414461A (en) * 2020-01-20 2020-07-14 福州大学 Intelligent question-answering method and system fusing knowledge base and user modeling
KR102194837B1 (en) * 2020-06-30 2020-12-23 건국대학교 산학협력단 Method and apparatus for answering knowledge-based question
CN112749265A (en) * 2021-01-08 2021-05-04 哈尔滨工业大学 Intelligent question-answering system based on multiple information sources
CN112800203A (en) * 2021-02-05 2021-05-14 江苏实达迪美数据处理有限公司 Question-answer matching method and system fusing text representation and knowledge representation
CN113010693A (en) * 2021-04-09 2021-06-22 大连民族大学 Intelligent knowledge graph question-answering method fusing pointer to generate network
CN113157886A (en) * 2021-04-19 2021-07-23 西安交通大学深圳研究院 Automatic question and answer generating method, system, terminal and readable storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111414461A (en) * 2020-01-20 2020-07-14 福州大学 Intelligent question-answering method and system fusing knowledge base and user modeling
KR102194837B1 (en) * 2020-06-30 2020-12-23 건국대학교 산학협력단 Method and apparatus for answering knowledge-based question
CN112749265A (en) * 2021-01-08 2021-05-04 哈尔滨工业大学 Intelligent question-answering system based on multiple information sources
CN112800203A (en) * 2021-02-05 2021-05-14 江苏实达迪美数据处理有限公司 Question-answer matching method and system fusing text representation and knowledge representation
CN113010693A (en) * 2021-04-09 2021-06-22 大连民族大学 Intelligent knowledge graph question-answering method fusing pointer to generate network
CN113157886A (en) * 2021-04-19 2021-07-23 西安交通大学深圳研究院 Automatic question and answer generating method, system, terminal and readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘莹;刘国奇;任介夫;姜琳颖;张斌;: "基于Web服务复杂网络的服务社区构建方法", 东南大学学报(自然科学版), no. 06, 20 November 2013 (2013-11-20) *

Also Published As

Publication number Publication date
CN113946665B (en) 2024-05-10

Similar Documents

Publication Publication Date Title
WO2023065545A1 (en) Risk prediction method and apparatus, and device and storage medium
US11436487B2 (en) Joint embedding of corpus pairs for domain mapping
CN113535984B (en) Knowledge graph relation prediction method and device based on attention mechanism
CN113392209B (en) Text clustering method based on artificial intelligence, related equipment and storage medium
CN105393265A (en) Active featuring in computer-human interactive learning
US20180052924A1 (en) Joint embedding of corpus pairs for domain mapping
Garreta et al. Scikit-learn: machine learning simplified: implement scikit-learn into every step of the data science pipeline
Lu et al. Research on classification and similarity of patent citation based on deep learning
CN112163099A (en) Text recognition method and device based on knowledge graph, storage medium and server
CN113569001A (en) Text processing method and device, computer equipment and computer readable storage medium
CN116383399A (en) Event public opinion risk prediction method and system
US20180052857A1 (en) Joint embedding of corpus pairs for domain mapping
CN113011172A (en) Text processing method and device, computer equipment and storage medium
CN116975615A (en) Task prediction method and device based on video multi-mode information
CN108804544A (en) Internet video display multi-source data fusion method and device
Feng et al. Ontology semantic integration based on convolutional neural network
Wan et al. A knowledge-enhanced interactive graph convolutional network for aspect-based sentiment analysis
CN114282528A (en) Keyword extraction method, device, equipment and storage medium
CN117112794A (en) Knowledge enhancement-based multi-granularity government service item recommendation method
CN113792144B (en) Text classification method of graph convolution neural network based on semi-supervision
CN115511082A (en) Fact verification method based on graph neural network and reinforcement learning
CN111782964B (en) Recommendation method of community posts
CN113946665A (en) Knowledge base question-answering method for providing background information based on text
CN111914201B (en) Processing method and device of network page
CN116431788B (en) Cross-modal data-oriented semantic retrieval method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant