CN111782769A - A Knowledge Graph Intelligent Question Answering Method Based on Relation Prediction - Google Patents

Abstract

The invention relates to a knowledge graph intelligent question answering method based on relationship prediction, and belongs to the field of natural language processing. The method includes steps: S1: input question Q, and preprocess the question; S2: use entity recognition technology to identify the entity e _question in the question, and map the entity e _question to the corresponding entity e KGs in the _KGs ; S3: query the KGs The category c of the entity e _KGs in the question Q is replaced with the category c of the entity e _question in the question Q, marked as Q _c ; S4: The relationship r is mapped from Q _c ; S5: In KGs, if the entity e _KGs and the relationship r are between S6: Learning new vector representations of central entities e _KGs ; S7: Inferring hidden relationships in KGs based on existing related triples; S8: Entities and relationships-based knowledge graph reasoning to obtain answer A. The invention can find the corresponding relationship of "question sentence entity--knowledge graph entity", and the corresponding relationship of "question sentence natural language description-knowledge graph semantic relationship".

Description

A Knowledge Graph Intelligent Question Answering Method Based on Relation Prediction

technical field

The invention belongs to the field of natural language processing, and relates to a knowledge graph intelligent question answering method based on relationship prediction.

Background technique

The keyword-based search method of traditional search engines lacks semantic analysis and semantic understanding of natural language, making it increasingly difficult to meet people's needs. For users, the best interaction method is in line with human natural language expression. When the question answering system exhibits sufficient intelligence, it can meet the user's needs for this interaction method. In 2012, Google proposed the concept of Knowledge Graphs (KGs), which further promoted the question answering system in the direction of intelligence. With the development of knowledge graph technology, intelligent question answering system has shown new development prospects. The rise of social networking sites has provided a large number of real question and answer corpora in different fields for the research of intelligent question answering systems, and it has provided great convenience for machines to understand natural language questions from the data level. The vigorous development of large-scale knowledge graphs such as Freebase and DBpedia has provided high-quality structured knowledge sources for intelligent question answering systems. The field of intelligent question answering based on knowledge graph is an important direction in the field of question answering research at this stage. It has great application prospects in the era of artificial intelligence, and also provides technical and theoretical support for the mobile Internet and the entrance of human social information.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a knowledge graph intelligent question answering method based on relationship prediction, by constructing an attention-based graph embedding method to infer the hidden relationship in KGs, so as to complete the missing relationship in KGs, Improve the accuracy of Question Answering over Knowledge Graphs (KGs-QA) based on knowledge graphs.

To achieve the above object, the present invention provides the following technical solutions:

A knowledge graph intelligent question answering method based on relationship prediction, the method includes the following steps:

S1: Input the question Q, and preprocess the question;

S2: Use entity recognition technology to identify the entity e _question in the question, and map the entity e _question to the corresponding entity e KGs in the _KGs ;

S3: Query the category c of the entity e _KGs in the KGs, replace the entity e _question in the question Q with the category c, and mark it as Q _c ;

S4: Map the relation r from Q _c ;

S5: In KGs, if there is a lack of connection between entity e _KGs and relation r;

S6: Learning new vector representations of central entities e _KGs ;

S7: Infer hidden relationships in KGs based on existing related triples;

S8: Knowledge graph reasoning based on entities and relationships, get answer A.

Optionally, the step S1 is specifically: dividing the text into words or phrases by using the CRF parser and the maximum entropy dependency parser in HanLP and Stanford parser, and obtaining part of speech, word order, keywords and quantitative descriptions of dependencies. .

Optionally, the step S2 is specifically: using a bidirectional long short-term memory network Bi-LSTM model to predict whether each word in the question is an entity;

The input sequence is processed by two LSTM units, forward and backward, and the final output vector is the concatenation of the two LSTM output vectors;

The output vector of the model is y=(y ₁ , y ₂ ,...,y _n ), where n is the length of the input sequence, the length of the output vector of the model is consistent with the input sequence, and y _i corresponds to the input question. The annotation information of the i-th word, if it is "1", it represents the entity to be found, otherwise it is not.

Optionally, the step S3 is specifically: using a latent Dirichlet topic model to conceptualize the entity in the question, so as to facilitate the understanding of the entity and increase its interpretability;

We develop a corpus-based framework for context-sensitive conceptualization by combining topic model latent Dirichlet assignments with a large-scale probabilistic KGs that capture semantic relationships between words.

Optionally, the step S4 is specifically: introducing a convolutional neural network CNNs model in the relationship linking task; extracting semantic information about the relationship in the question through the deep neural network model, and using the same for all relationships of candidate entities. The model is processed, and the obtained question attribute vector and the knowledge graph attribute vector are matched for similarity, and the correct relationship of the final link is obtained.

Optionally, the step S5 is specifically: based on the entity identified in the step S2 and the relationship linked by the S4, simplify the knowledge graph reasoning based on the entity and the relationship into a subgraph matching problem;

In the knowledge graph, if no match is found, that is, there is a lack of connection between the entity e _KGs and the relationship r, the next task of relationship prediction is performed.

Optionally, the step S6 is specifically: in order to solve the problem that information in the field hidden around the triplet cannot be obtained, optimize the vector learning model, and propose an attention-based feature embedding method, which captures any given entity. Entity features and relational features within the neighborhood;

Relation clusters and multi-hop relations are encapsulated in the model; to obtain a new vector representation of a central entity, a linear transformation is used to learn feature vectors for each set of relevant facts present in the central entity's neighborhood.

Optionally, the step S7 is specifically: infer the initially hidden relationship by identifying the existing relevant fact triples (h, r, t), where h represents the head semantic entity, r represents the semantic relationship, and t represents the tail. Semantic entities; that is, learning multi-hop entities and multi-hop relationships in the neighborhood of the central entity, and introducing an auxiliary edge between n-hop domains to realize the task of relationship prediction.

Optionally, the step S8 is specifically: KGs-QA supports binary fact question answering by learning 27 million templates of 2782 intents.

The beneficial effects of the present invention are as follows: the present invention adopts the method of knowledge graph to infer the entities and relationships in the natural language question, so as to find the corresponding relationship of "question entity-knowledge graph entity", and "question natural language description" ——The corresponding relationship of knowledge graph semantic relationship, template-based knowledge graph reasoning, get the corresponding answer, so that our natural language understanding function not only has the ability to understand the literal meaning, but also has the ability to logically reason and understand the deep meaning.

Other advantages, objects, and features of the present invention will be set forth in the description that follows, and will be apparent to those skilled in the art based on a study of the following, to the extent that is taught in the practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the following description.

Description of drawings

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be preferably described in detail below with reference to the accompanying drawings, wherein:

Fig. 1 is the flow chart of the intelligent question answering system construction based on knowledge graph of the present invention;

Fig. 2 is the schematic diagram of the entity recognition model based on Bi-LSTM of the present invention;

3 is a schematic diagram of a relational link model based on CNNs of the present invention;

4 is a schematic diagram of entity identification and relationship linking based on knowledge graph of the present invention;

5 is a schematic diagram of a subgraph of the knowledge graph of the present invention;

FIG. 6 is a schematic diagram of the knowledge graph reasoning based on entities and relationships according to the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic idea of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

Among them, the accompanying drawings are only used for exemplary description, and represent only schematic diagrams, not physical drawings, and should not be construed as limitations of the present invention; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings will be omitted, The enlargement or reduction does not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the accompanying drawings may be omitted.

The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms “upper”, “lower”, “left” and “right” , "front", "rear" and other indicated orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must be It has a specific orientation, is constructed and operated in a specific orientation, so the terms describing the positional relationship in the accompanying drawings are only used for exemplary illustration, and should not be construed as a limitation of the present invention. situation to understand the specific meaning of the above terms.

FIG. 1 is a flowchart of the construction of an intelligent question answering system based on a knowledge graph according to the present invention. Take Yahoo! Answer's natural language question and answer corpus is a semantic knowledge resource, and a knowledge graph is a semantic representation method. Through a data representation method of knowledge graph, this paper uses the knowledge graph to describe the entities of natural language and complete the question and answer task, so that the natural language understanding function of the model not only has the ability to understand the literal meaning, but also has the ability to logically reason and understand the deep meaning. ability. An embodiment of an intelligent question answering system using a knowledge graph is given below in conjunction with the accompanying drawings to further illustrate the present invention.

As shown in Figure 1, the specific implementation details of each part of the present invention are as follows:

1. Enter the question Q and preprocess the question. Divide the text into words or phrases through the CRF parser and maximum entropy dependency parser in HanLP and Stanford parser, and obtain quantitative descriptions of parts of speech, word order, keywords and dependencies.

2. Use entity recognition technology to identify the entity e _question in the question, and map the entity e _question to the corresponding entity e KGs in the _KGs . The Bi-directional Long Short-Term Memory (Bi-LSTM) model is used to predict whether each word in the question is an entity, as shown in Figure 2. The input sequence is processed by two LSTM units, forward and backward, and the final output vector is the concatenation of the two LSTM output vectors. Compared with the LSTM model, the Bi-LSTM model retains its advantages and takes into account the context information by separately training the forward and backward sequences, which can better extract deep semantic information. The output vector of the model is y=(y ₁ , y ₂ ,...,y _n ), where n is the length of the input sequence. It can be seen that the length of the output vector of the model is consistent with the input sequence, and y _i corresponds to the input The labeling information of the i-th word in the question sentence, if it is "1", it represents the entity to be found, otherwise it is not.

3. Query the category c of the entity e _KGs , and replace the entity e _question in the question Q with the category c, labeled Q _c . The latent Dirichlet topic model is used to conceptualize the entity in the question, which facilitates the understanding of the entity and increases its interpretability. We develop a corpus-based context-sensitive conceptualization model by combining topic model latent Dirichlet allocation with a large-scale KGs that capture semantic relationships between words. The model automatically disambiguates the input information (for example, the term "apple" in the question "Where is Apple's headquarters?" would be conceptualized as "company" rather than "fruit"). The conceptualization mechanism itself is based on a large semantic network of millions of concepts, so we have enough granularity to represent a wide variety of problems.

4. Map the relation r from _Qc . The Convolutional Neural Networks (CNNs) model is introduced in the relational linking task, as shown in Figure 3. Through the deep neural network model, the semantic information about the relationship in the question sentence is extracted, and all the relations of the candidate entities are processed with the same model, and the similarity between the obtained question sentence attribute vector and the knowledge map attribute vector is performed to obtain the final link. correct relationship.

5. In KGs, if there is a missing link between entity e _KGs and relation r. The knowledge reasoning based on entities and relationships is simplified to the subgraph matching problem. In the knowledge graph, if no match is found, that is, there is a lack of connection between the entity e _KGs and the relationship r, as shown in Figure 4, proceed to the next step Relationship prediction task.

6. Learning new vector representations of central entities e _KGs . In order to solve the problem that traditional methods cannot obtain information hidden in the domain around triples, a vector learning model is optimized, and an attention-based feature embedding method is proposed, which can capture entity features and relational features in the neighborhood of any given entity . In addition, we also encapsulate relational clusters and multi-hop relations in the model. To obtain a new vector representation of a central entity, a linear transformation is used to learn the eigenvectors of each relevant fact set existing in the central entity's neighborhood.

7. Infer hidden relationships in KGs based on existing related triples. The initially hidden relationship is inferred by discovering the existing relevant fact triples (h, r, t), where h represents the head semantic entity, r represents the semantic relationship, and t represents the tail semantic entity. More specifically, multi-hop entities and multi-hop relationships within the neighborhood of the central entity are learned, and an auxiliary edge is introduced between n-hop domains to realize the task of relationship prediction, as shown in Figure 5.

8. Knowledge graph reasoning based on entities and relationships, get answer A, as shown in Figure 6. Based on large-scale KGs and a large amount of question answering corpus, we design a new question representation method: templates. Learn 27 million templates for 2782 intents. Based on these templates, KGs-QA effectively supports binary fact question answering.

FIG. 2 is a schematic diagram of the entity recognition model based on Bi-LSTM of the present invention. The invention regards the entity recognition process as a sequence labeling problem, and uses the Bi-LSTM model to predict whether each word in the question is an entity. For example, for the question "Where is the capital of China", the participle result is "China/的/capital/is/where", and the entity is "China", then its labeling sequence is (1,0,0,0,0). "1" in the sequence represents the largest relationship between "China" and the entity in the question. In fact, this process is to first perform data processing such as word segmentation and dictionary construction for natural language questions, and put all possible entities as candidates into the candidate entity set.

其中，

为前向序列的输出，

为后向序列的输出。Bi-LSTM得到的输出还会送入sigmoid层进行处理，即

模型的输出向量为y＝(y₁,y₂,...,y_n)，其中n为输入序列的长度，可以看到该模型输出向量长度与输入序列是保持一致的，y_i对应输入问句中第i个单词的标注信息，如果为“1”则代表为寻找的实体，反之则不是。在本发明中，使用均方误差作为模型的损失函数，即

其中，ω为权重，b为偏差，y_i为模型的预测值，z_i为目标值，λ为控制正规化的超参数，

为L2正规化。The forward and backward LSTM process the input vectors (x ₁ , x ₂ ,...,x _t-1 , x _t ) respectively to obtain the output vector h _t , namely

in,

is the output of the forward sequence,

is the output of the backward sequence. The output obtained by Bi-LSTM will also be sent to the sigmoid layer for processing, namely

The output vector of the model is y=(y ₁ , y ₂ ,...,y _n ), where n is the length of the input sequence. It can be seen that the length of the output vector of the model is consistent with the input sequence, and y _i corresponds to the input The labeling information of the i-th word in the question sentence, if it is "1", it represents the entity to be found, otherwise it is not. In the present invention, the mean square error is used as the loss function of the model, namely

where ω is the weight, b is the bias, _yi is the predicted value of the model, _zi is the target value, λ is the hyperparameter that controls the regularization,

Normalize for L2.

其中，θ是向量Q和向量R之间的夹角，Q_i是问题语义向量的第i个元素，R_j是候选关系的第j个元素。余弦距离计算两个向量之间的余弦角。在相同的向量空间中，角度越小，两者之间的余弦距离越近。FIG. 3 is a schematic diagram of a relational link model based on CNNs of the present invention. In essence, the process of relational linking is to measure the relevance between the relevant relation of the question and the candidate relation. Based on this idea, this paper adopts the CNNs model shown in Figure 3. It is worth noting that the part of the question vector in the input, the candidate entity part is replaced by the "concept", which can avoid the influence of the named entity on the mapping result. As can be seen from the figure, this paper uses CNNs to convolve the question sentence vector and the candidate relationship vector respectively, and obtain the semantic vector corresponding to the question sentence and the relationship. Finally, the correlation calculation is performed on the obtained question sentence and the relational semantic vector to obtain the link result. After being processed by the CNNs model, the semantic vector of the question and the semantic vector of the candidate relationship can be obtained, and the semantic similarity is calculated by calculating the cosine similarity, that is,

where θ is the angle between vector Q and vector R, Q _i is the i-th element of the question semantic vector, and R _j is the j-th element of the candidate relation. Cosine distance calculates the cosine angle between two vectors. In the same vector space, the smaller the angle, the closer the cosine distance between the two.

FIG. 5 is a schematic diagram of a subgraph of the knowledge graph of the present invention. The solid line in the figure represents the existing relationship, and the dashed line represents the introduced auxiliary relationship (hidden relationship). For example, where was Bob born? In this example, the analysis process includes an entity recognition process and a relationship linking process, as shown in Figure 5. In the above example, after finding entities and relationships in KGs, we noticed that KGs are incomplete (i.e., relationships between entities are missing), as shown in Figure 5. The relation prediction task is achieved by assigning different attention to nearby entities, and the attention is propagated through the layers in an iterative manner, considering that the contribution of the entities becomes smaller and smaller as the number of iterations increases. A promising solution to the above problem is relation composition, which is achieved by introducing auxiliary edges (dashed lines) between n-hop neighbors, in this case n = 2, as shown in Figure 5. Based on the study, we noticed that <Bob, brother, Andy> + <Andy, born in, Washington> are more important. In our model, an auxiliary relation (dashed line) of 2-hop neighbors between two entities is introduced. The vector representation of an auxiliary relationship is the sum of the vector representations of all dependent relationships (solid lines). In this example, the auxiliary relationship (dotted line) can be understood as <Bob, brother, Andy> plus <Andy, born in Washington State>.

FIG. 6 is a schematic diagram of the knowledge graph reasoning based on entities and relationships according to the present invention. The concept of word embedding is introduced to convert the acquired knowledge graph training samples into low-dimensional space vectors, so that knowledge reasoning is transformed into the problem of processing natural language questions by constructing templates, so as to find the correspondence between "question entities -- knowledge graph entities" , and the corresponding relationship of "Question Natural Language Description-Knowledge Graph Semantic Relationship". Research the entity recognition technology based on deep learning, and use Bi-LSTM to make full use of the context information to locate the candidate entity position of the question. Research the relationship linking technology based on deep learning, use CNNs to extract deep semantic features, integrate into the attention mechanism, so as to obtain the semantic vector of the relevant relationship in the question, and use the parameter sharing mechanism to input the candidate relationship in the triplet into the same model. Get the semantic vector of candidate relations. Finally, the cosine similarity is used to calculate and select the most correct candidate relationship. In KGs, if there is a lack of connection between entity e _KGs and relation r, as shown in Figure 4, proceed to the next step of relation prediction task. First, new vector representations for central entities e _KGs are learned. In order to solve the problem that traditional methods cannot obtain information hidden in the domain around triples, a vector learning model is optimized, and an attention-based feature embedding method is proposed, which can capture entity features and relational features in the neighborhood of any given entity . In addition, we also encapsulate relational clusters and multi-hop relations in the model. To obtain a new vector representation of a central entity, a linear transformation is used to learn the eigenvectors of each relevant fact set existing in the central entity's neighborhood. Then, the hidden relations in the KGs are inferred based on the existing related triples. The initially hidden relationship is inferred by discovering the existing relevant fact triples (h, r, t), where h represents the head semantic entity, r represents the semantic relationship, and t represents the tail semantic entity. More specifically, multi-hop entities and multi-hop relationships within the neighborhood of the central entity are learned, and an auxiliary edge is introduced between n-hop domains to realize the task of relationship prediction, as shown in Figure 5. Finally, implement knowledge graph reasoning based on entities and relationships, and obtain answer A by querying the knowledge graph.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements, without departing from the spirit and scope of the technical solution, should all be included in the scope of the claims of the present invention.

Claims (9)

1. A knowledge graph intelligent question answering method based on relationship prediction, characterized in that: the method comprises the following steps: S1: Input the question Q, and preprocess the question; S2: Use entity recognition technology to identify the entity e _question in the question, and map the entity e _question to the corresponding entity e KGs in the _KGs ; S3: Query the category c of the entity e _KGs in the KGs, replace the entity e _question in the question Q with the category c, and mark it as Q _c ; S4: Map the relation r from Q _c ; S5: In KGs, if there is a lack of connection between entity e _KGs and relation r; S6: Learning new vector representations of central entities e _KGs ; S7: Infer hidden relationships in KGs based on existing related triples; S8: Knowledge graph reasoning based on entities and relationships, get answer A. 2. the knowledge graph intelligent question answering method based on relationship prediction according to claim 1, is characterized in that: described step S1 is specifically: by the CRF syntactic analyzer in HanLP and Stanfordparser and the maximum entropy dependent syntactic analyzer, the text is divided For words or phrases, and obtain quantitative descriptions of parts of speech, word order, keywords and dependencies. 3. the knowledge graph intelligent question answering method based on relationship prediction according to claim 1, is characterized in that: described step S2 is specifically: utilize bidirectional long short term memory network Bi-LSTM model to whether each word in question is an entity make predictions; The input sequence is processed by two LSTM units, forward and backward, and the final output vector is the concatenation of the two LSTM output vectors; The output vector of the model is y=(y ₁ , y ₂ ,...,y _n ), where n is the length of the input sequence, the length of the output vector of the model is consistent with the input sequence, and y _i corresponds to the input question. The annotation information of the i-th word, if it is "1", it represents the entity to be found, otherwise it is not. 4. The knowledge graph intelligent question answering method based on relationship prediction according to claim 1, characterized in that: the step S3 is specifically: utilize the potential Dirichlet topic model to conceptualize the entity in the question, so as to facilitate the identification of the entity. understanding, increasing its interpretability; We develop a corpus-based framework for context-sensitive conceptualization by combining topic model latent Dirichlet assignments with a large-scale probabilistic KGs that capture semantic relationships between words. 5. The knowledge graph intelligent question and answer method based on relationship prediction according to claim 1, characterized in that: the step S4 is specifically: introducing a convolutional neural network CNNs model in the relationship link task; The semantic information about the relationship in the question sentence, and at the same time, all the relations of the candidate entities are processed with the same model, and the obtained question sentence attribute vector and the knowledge graph attribute vector are matched for similarity, and the correct relationship of the final link is obtained. 6. The knowledge graph intelligent question answering method based on relationship prediction according to claim 1, wherein the step S5 is specifically: based on the entity identified in step S2 and the relationship linked by S4, Knowledge graph reasoning is simplified to subgraph matching problem; In the knowledge graph, if no match is found, that is, there is a lack of connection between the entity e _KGs and the relationship r, the next task of relationship prediction is performed. 7. The knowledge graph intelligent question answering method based on relationship prediction according to claim 1, wherein the step S6 is specifically: in order to solve the problem that information in the field hidden around the triplet cannot be obtained, optimizing the vector learning model, and propose an attention-based feature embedding method that captures entity features and relational features within the neighborhood of any given entity; Relation clusters and multi-hop relations are encapsulated in the model; to obtain a new vector representation of a central entity, a linear transformation is used to learn feature vectors for each set of relevant facts present in the central entity's neighborhood. 8. The knowledge graph intelligent question answering method based on relationship prediction according to claim 1, characterized in that: the step S7 is specifically: inferring the initial hidden by identifying existing related fact triples (h, r, t) relationship, where h represents the head semantic entity, r represents the semantic relationship, and t represents the tail semantic entity; that is, the multi-hop entity and multi-hop relationship in the neighborhood of the learning center entity, and an auxiliary edge is introduced between the n-hop fields to achieve Relationship prediction task. 9. The knowledge graph intelligent question answering method based on relationship prediction according to claim 1, wherein the step S8 is specifically: KGs-QA supports binary fact question answering by learning 27 million templates of 2782 intentions .

Images