CN115129834A

CN115129834A - Question answering method and system

Info

Publication number: CN115129834A
Application number: CN202210549895.9A
Authority: CN
Inventors: 郑杰文; 陈泽
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2022-05-20
Filing date: 2022-05-20
Publication date: 2022-09-30

Abstract

The invention discloses a question answering method, which comprises the following steps: identifying an entity in an input problem, and acquiring an entity pair list; acquiring the alternative relation of each entity pair in the entity pair list to form an alternative relation set corresponding to the entity pair list; acquiring probability distribution of the input problems on the alternative relation set according to the input problems and the alternative relation set; obtaining the probability distribution of the known problem of any entity pair in the entity pair list on the alternative relation set; and calculating the similarity of the probability distribution of the input question and the known question on the candidate relationship set, and determining an output answer corresponding to the input question according to the similarity of the probability distribution. The method is a question answering method based on probability distribution and probability distribution similarity, solves the technical problem that the multi-entity problem and the missing relation problem cannot be answered in the prior art, and improves the comprehensiveness of the question answering technology.

Description

Question answering method and system

Technical Field

The present invention relates to the field of artificial intelligence technologies, and in particular, to a question answering method, a question answering system, an electronic device, and a computer-readable storage medium.

Background

A Question Answering System (QA) is an advanced form of an information retrieval System, and with the rapid development of artificial intelligence technology, the Question Answering System can answer questions posed by users in natural language with more accurate and concise natural language. The question-answering system is widely applied to various fields such as the field of games, the technical field of navigation, the field of on-line teaching and the like.

A Knowledge Graph Based Question-Answering system (KGQA) is a mainstream Question-Answering system at present, and the following problems exist in the practical application process of the Knowledge Graph Based Question-Answering system (KGQA): first, a question containing only one entity can be answered, and a multi-entity question cannot be answered. Secondly, only questions that contain entities and have relations in the knowledge graph can be answered, and the questions that lack relations cannot be answered.

Disclosure of Invention

The invention provides a question answering method and a question answering system, which are used for solving the technical problem that the existing question answering method cannot answer multi-entity problems and missing relation problems.

The embodiment of the invention provides a question answering method, which comprises the following steps:

identifying entities in the input problem and acquiring an entity pair list;

acquiring the alternative relation of each entity pair in the entity pair list to form an alternative relation set corresponding to the entity pair list;

acquiring probability distribution of the input problems on the alternative relation set according to the input problems and the alternative relation set;

obtaining the probability distribution of the known problem of any entity pair in the entity pair list on the alternative relation set;

and calculating the similarity of the probability distribution of the input question and the known question on the candidate relationship set, and determining an output answer corresponding to the input question according to the similarity of the probability distribution.

Optionally, the identifying an entity in the input question and obtaining an entity pair list include:

identifying entities in the input question through an entity identification model;

and performing pairwise formation on the entities in the input question to form the entity pair list.

Optionally, the method for obtaining the entity recognition model includes:

taking entities existing in the knowledge graph as training data, providing the training data to an initial entity recognition model, and training the initial entity recognition model;

and taking the entity recognition model which reaches the preset standard after training as the entity recognition model which is actually used.

Optionally, the obtaining of the alternative relationship of each entity pair in the entity pair list includes: and taking the relation existing in the knowledge graph of the entity pair as an alternative relation.

Optionally, the obtaining the alternative relationship of each entity pair in the entity pair list further includes: and taking the relation of the type pair corresponding to the entity pair existing in the knowledge graph as an alternative relation.

Optionally, the obtaining, according to the input question and the candidate relationship set, a probability distribution of the input question on the candidate relationship set includes:

obtaining the score of the input question on each alternative relation in the alternative relation set through a relation extraction model;

and forming the probability distribution of the input question on the candidate relation set according to the score of the input question on each candidate relation in the candidate relation set.

Optionally, the method for obtaining the relationship extraction model includes:

taking known questions in a question-answering library and the corresponding relation description of the entity pair in the known questions as training data, providing the training data to an initial relation extraction model, and training the initial relation extraction model;

and taking the relation extraction model which reaches the preset standard after training as the actually used relation extraction model.

Optionally, the obtaining, by the relationship extraction model, a score of the input question on each candidate relationship in the candidate relationship set includes:

mapping each alternative relation in the alternative relation set into a relation description;

obtaining the correct probability of the input question on the relation description of each alternative relation through the relation extraction model;

and taking the probability of correctness of the input question on the relationship description of each alternative relationship as the score of the input question on each alternative relationship.

Optionally, the mapping each alternative relationship in the alternative relationship set into a relationship description includes: and correspondingly acquiring the relation description of each alternative relation in the alternative relation set according to the relation and relation description list.

Optionally, the obtaining a probability distribution of a known problem including any entity pair in the entity pair list on the candidate relationship set includes:

acquiring all known questions containing any entity pair in the entity pair list from a question-answering library;

and obtaining the probability distribution of each known problem on the candidate relation set through a relation extraction model.

Optionally, the calculating a similarity degree of the probability distribution of the input question and the known question on the candidate relationship set, and determining an output answer corresponding to the input question according to the similarity degree includes:

calculating JS divergence values of the probability distribution of the input problem on the alternative relation set and the probability distribution of the known problem on the alternative relation set;

judging the similarity between the input problem and the known problem according to the JS divergence value;

and determining an output answer corresponding to the input question according to the similarity between the input question and the known question.

Optionally, the judging of JS divergence value according to the similarity of the input question with the known question includes:

comparing whether the JS divergence value is smaller than a preset similarity threshold value or not;

judging whether the input question is similar to the known question according to the comparison result, comprising: if so, the input question is similar to the known question; if not, the input problem is dissimilar to the known problem.

Optionally, the judging of JS divergence value according to the similarity of the input question with the known question still includes:

comparing the magnitudes of the JS divergence values;

according to the comparison result, judging the known problem with the highest similarity to the input problem, wherein the judging comprises the following steps: the smaller the JS divergence value, the higher the similarity of the input question to the known question.

Optionally, the determining an output answer corresponding to the input question according to the similarity between the input question and the known question includes: and taking the answer of the known question with the highest similarity with the input question as the output answer corresponding to the input question.

The embodiment of the invention also provides a question-answering system, which comprises: the system comprises an entity identification unit, an alternative relation acquisition unit, an input problem probability distribution acquisition unit, a known problem probability distribution acquisition unit and a similarity calculation unit;

the entity identification unit is used for identifying the entity in the input problem and acquiring an entity pair list;

the candidate relationship obtaining unit is configured to obtain a candidate relationship of each entity pair in the entity pair list to form a candidate relationship set corresponding to the entity pair list;

the input problem probability distribution obtaining unit is used for obtaining the probability distribution of the input problems on the alternative relation set according to the input problems and the alternative relation set;

the known problem probability distribution obtaining unit is configured to obtain a probability distribution of a known problem including any one entity pair in the entity pair list on the candidate relationship set;

the similarity calculation unit is configured to calculate a similarity of probability distributions of the input question and the known question on the candidate relationship set, and determine an output response corresponding to the input question according to the similarity of the probability distributions.

An embodiment of the present invention further provides an electronic device, including: a memory, a processor;

the memory to store one or more computer instructions;

the processor is configured to execute the one or more computer instructions to implement the above-described method.

Embodiments of the present invention also provide a computer-readable storage medium having one or more computer instructions stored thereon, which, when executed by a processor, perform the above-described method.

Compared with the prior art, the question answering method provided by the invention comprises the following steps: identifying entities in the input problem and acquiring an entity pair list; acquiring the alternative relation of each entity pair in the entity pair list to form an alternative relation set corresponding to the entity pair list; acquiring probability distribution of the input problems on the alternative relation set according to the input problems and the alternative relation set; obtaining the probability distribution of the known problem of any entity pair in the entity pair list on the alternative relation set; and calculating the similarity of the probability distribution of the input question and the known question on the candidate relationship set, and determining an output answer corresponding to the input question according to the similarity of the probability distribution. According to the method, an entity pair list of an input question and an alternative relation set corresponding to the entity pair list are obtained, probability distribution of the input question on the alternative relation set and probability distribution of a known question on the alternative relation set are obtained, similarity between the probability distribution of the input question and the probability distribution of the known question is further calculated, and an output answer of the input question is determined according to the similarity. The question answering method is a question answering method based on probability distribution and probability distribution similarity calculation, and can obtain output answers corresponding to input questions according to the similarity of the probability distribution of the input questions and the known questions on the alternative relation set. The technical problem that the multi-entity problem and the missing relation problem cannot be solved in the prior art is solved, and the comprehensiveness of the question-answering technology is improved.

Drawings

Fig. 1 is a diagram of an application system of a question answering method according to an embodiment of the present invention;

fig. 2 is a diagram of an application system of another question answering method according to an embodiment of the present invention;

fig. 3 is a flowchart of a question answering method according to a first embodiment of the present invention;

fig. 4 is a flowchart of a probability distribution obtaining method according to a first embodiment of the invention;

FIG. 5 is a flowchart for obtaining a probability distribution of a known problem over a set of candidate relationships according to a first embodiment of the present invention;

FIG. 6 is a flowchart for determining output responses according to probability distribution similarity provided by the first embodiment of the present invention;

fig. 7 is a schematic structural diagram of a question answering system according to a second embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather construed as limited to the embodiments set forth herein.

The following describes terms involved in the embodiments of the present invention:

the question answering is a technical process of analyzing and answering questions asked by a user in natural language by using an artificial intelligence technology and outputting answers to the questions in the natural language.

The Question Answering System (QA) is a System applied to Question Answering, is a high-level form of an information retrieval System, and can reply questions posed by users in natural language with accurate and concise natural language. Are widely used in various fields, for example: in the field of games, different players have different familiarity with games, so that a lot of questions about game playing methods are generated in the process that the players are familiar with the games, and the players can quickly and accurately obtain answers to the questions through a question-answering system, which is of great significance for improving game retention.

The Knowledge Graph (KG) is a form of Knowledge representation, is a network formed by connecting Knowledge, is a Graph for describing a Knowledge development process and a structural relationship by using a visualization technology, generally stores the Knowledge in triples, and models the interrelation between the entities in a head entity-relationship-tail entity manner, wherein one triplet represents one piece of Knowledge.

The Knowledge Graph Question-Answering system (KGQA) is a Question-Answering system for Answering user questions Based on structured information of a Knowledge Graph, and when a user inputs a Question, related information of the Question can be obtained in the Knowledge Graph through a reasoning method and answered.

The Artificial Intelligence (AI), a branch of computer science, can generate intelligent machines that approach human Intelligence. Research in this field mainly relates to techniques such as robots, language recognition, image recognition, and the like. The artificial intelligence is to analyze and simulate the consciousness and thinking process of people so as to design an artificial intelligence robot capable of behaving like a human. On single intelligence, such as computing, it may even exceed human intelligence.

The knowledge graph-based question-answering system is the existing mainstream question-answering method, and utilizes the structural information of the knowledge graph to reason and acquire the relevant information and answers of the questions. A knowledge-graph based question-answering method generally comprises the steps of:

first, entity recognition, recognizing entities from input questions, and determining the question type data in a knowledge graph for several hops.

And secondly, identifying the relation, namely identifying the candidate relation of the entity based on the structural information of the knowledge graph, sequencing the candidate relation and selecting the relation with the highest possibility.

Thirdly, sub-graph recall, and all sub-graphs of the entity and the relation are output according to the identified entity and the identified relation.

Fourthly, sorting the answers, sorting according to the candidate scores of all the subgraphs, selecting the subgraph with the highest score, and taking the corresponding answer as the answer to the question.

Therefore, the knowledge-graph-based question-answering method actually aims at input questions, firstly predicts the head entity s and then predicts the relation r in a triple (head entity s-relation r-tail entity o) of the knowledge graph, and if the head entity s and the relation r exist in the knowledge graph, then predicts the tail entity o according to a subgraph output by the head entity s and the relation r, and takes a prediction result as an answer of the question.

The question-answering method based on the knowledge graph can better predict answers to questions, but has two problems:

first, the existing knowledge graph-based question-answering method can only deal with the situation that one entity appears in the questions, and cannot predict the problem of multiple entities.

Secondly, the existing knowledge graph-based question-answering method can only process the problem that the entity has the relation in the knowledge graph, and can not reason the answer for the problem of relation loss.

However, in practical situations, a plurality of entities often appear in a problem, and since the knowledge graph itself is also gradually improved, a situation that the entity relationship is missing exists, and for such a situation, the question-answering system mostly automatically replies: "your problem is really good, i will go back to good research and not bring effective information to the user, can not solve the user's problem. Particularly, in the field of games, when a new game is released, a game player presents many questions for the new game, and as long as multiple entities exist in the questions or the entity relationship is lost, the existing question-answering method cannot effectively answer the questions presented by the game player, is not favorable for the game player to understand the new game, and can cause the loss of the game player.

In order to solve the problems existing in the conventional question answering method, the question answering method is provided, and the similarity of the probability distribution of the input question and the probability distribution of the known question is calculated by taking the probability distribution of the question in the relation space as a starting point, so that the output answer corresponding to the input question is obtained. The question answering method and the question answering device can effectively solve the problem that a plurality of entities and entity relations are lost in the question, and improve comprehensiveness of the question answering technology. The question answering method provided by the invention is suitable for any field needing a question answering system, in particular to a question answering system in the field of games.

The question answering method, system, electronic device and computer readable storage medium according to the present invention will be described in detail with reference to the following embodiments and accompanying drawings.

Fig. 1 is a diagram of an application system of a question answering method according to an embodiment of the present invention. As shown in fig. 1, the system includes a client 101 and a server 102. The user terminal 101 and the server terminal 102 are in communication connection through a network. The user terminal 101 may be a touch terminal, such as a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or other devices; the number of the devices can be one or more than one, and the devices can also be a computer terminal, such as a notebook computer, a desktop computer and the like. The server 102 is used for deploying the question answering system provided by the invention. The user inputs a question through the user side 101, the question is transmitted to the server side 102 through the network, the server side 102 analyzes and answers the question, the answer of the question is transmitted back to the user side 101 through the network, and the user receives the answer to the question through the user side 101. The server 102 may be a question answering device for the user 101, the user 101 may be merged with the server 102, the user directly inputs a question through the server 102, the server 102 analyzes and answers the question, and the user receives an answer to the question through the server 102.

Fig. 2 is a diagram of an application system of another question answering method according to an embodiment of the present invention. As shown in fig. 2, the application system includes a user terminal 201 and a server 202. The user terminal 201 may be a touch terminal, such as a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or other devices; the terminal may also be a computer terminal, such as a notebook computer, a desktop computer, or other devices with a voice transmission function or a text transmission function, which may be one or more. The server 202 is used for deploying the question answering method provided by the invention. The user inputs a question through the user terminal 201, the question is transmitted to the server 202 through the network, the server 202 analyzes and answers the question, the answer of the question is transmitted back to the user terminal 201 through the network, and the user receives the answer to the question through the user terminal 201. The server 202 may be an independent server that deploys the method provided by the present invention, or may be a server cluster composed of multiple servers, where each server deploys one module of the method provided by the present invention. Such as: identification servers, computing servers, etc. Of course, the server 202 may also be a cloud server, and the question answering method provided by the present invention is deployed on the cloud server. The server 202 provides question responses for all users.

A first embodiment of the present invention provides a question-answering method.

Fig. 3 is a flowchart of the question answering method provided in this embodiment. The question answering method provided in this embodiment is described in detail below with reference to fig. 3. The following description relates to examples for explaining the technical aspects of the present invention, and is not intended to limit the practical use.

As shown in fig. 3, the question answering method provided in this embodiment includes the following steps:

step S301, identifying the entity in the input question and acquiring an entity pair list.

The purpose of this step is to identify a number of entities in the user's input question and to compose the identified entities into a list of entity pair forms.

An input question may include one entity or may include a plurality of entities. Entities in the input question may be identified by an entity identification model.

The entity recognition model is a neural network model capable of recognizing an entity in an input question. Neural Networks (NNs), which are composed of a number of neurons and their parameters, are a system that performs tasks by "learning" through a large number of examples, and are typically not programmed with task-specific rules. For example, in image recognition, the neural network may learn features of cats by analyzing example images labeled as "cats" or "not cats" and use the learning results to identify whether other images contain cats. In the learning of the neural network, the features of the cat are not directly input into the neural network, but the example image marked as the cat is input, and the neural network automatically generates feature information representing the cat according to the example image through iterative learning.

The entity recognition model provided by the embodiment takes entities existing in the knowledge graph as training data, and provides the training data to the initial entity recognition model for training; and taking the entity recognition model which reaches the preset standard after training as the entity recognition model which is actually used.

That is to say, acquiring the entity recognition model requires learning and training with entities existing in the knowledge graph as training data, and also requires detecting the recognition capability of the trained entity recognition model, and only the entity recognition model meeting the predetermined standard can be used as the entity recognition model used in the question-answering method provided in this embodiment.

The predetermined criteria provided by the present embodiment is the ability to obtain valid entities in the input question, i.e., entities present in the knowledge-graph.

After the entities in the input question are identified, the entities in the input question need to be grouped pairwise to form the entity pair list. Such as: the input problem comprises three entities (an entity a, an entity b and an entity c), and pairwise formation is carried out on the entities a, b and c, so that an entity pair ab, an entity pair ac and an entity pair bc can be obtained. The entity pair ab, the entity pair ac and the entity pair bc can form an entity pair list corresponding to the input problem.

If the relationships of these entity pairs in the knowledge-graph are directional, then the entity pair list includes entity pairs that are: entity pair ab, entity pair ba, entity pair ac, entity pair ca, entity pair bc, and entity pair cb.

Step S302, obtaining the candidate relationship of each entity pair in the entity pair list, and forming a candidate relationship set corresponding to the entity pair list.

The purpose of this step is to obtain the candidate relationship or the candidate relationship group corresponding to each entity pair in the entity pair list, and combine the candidate relationship or the candidate relationship group corresponding to each entity pair to form the candidate relationship set corresponding to the entity pair list.

The method for acquiring the alternative relationship or the alternative relationship group corresponding to the entity pair provided by the embodiment includes: and taking the relation existing in the knowledge graph of the entity pair as an alternative relation.

When an entity pair has a relationship in the knowledge-graph, the relationship is used as an alternative relationship of the entity pair. Such as: when a user plays a certain game, an entity pair of a game character and a big dog exists in an input problem, and in a knowledge graph of the game, the relation between the game character and the big dog is an example, so that the entity pair of the game character and the big dog exists in the knowledge graph, and the example is used as an alternative relation of the entity pair.

The method for acquiring the candidate relationship or the candidate relationship group corresponding to the entity pair provided by the embodiment further includes: and taking the relation of the type pair corresponding to the entity pair existing in the knowledge graph as an alternative relation.

And when the entity pair does not have a relation in the knowledge graph, expanding the entity pair into a corresponding type pair, and taking a relation group of the type pair existing in the knowledge graph as an alternative relation group of the entity pair. Such as: when a certain user carries out a certain game, entity pairs of 'fawn men' and 'fortune cats' exist in the input problem, and the relation between the 'fawn men' and the 'fortune cats' does not exist in the knowledge graph of the game, so that the entity pairs of 'fawn men-fortune cats' are expanded into type pairs of 'role-prop' (wherein, the 'fawn men' belongs to the role type, and the 'fortune cats' belongs to the prop type). The relationship of the type pair 'role-prop' in the knowledge graph is as follows: the "suitable for", "suitable for carrying", and "recommending soul", the alternative relationship group corresponding to the "deer man-cat bringing in wealth" by the entity is: the method is suitable for carrying and recommending souls.

The entity pair is expanded into the type pair, and the alternative relation group corresponding to the type pair is used as the alternative relation group corresponding to the entity pair, so that the technical problem that the problem can not be solved when the entity lacks the relation is solved in the mode.

One or more alternative relations corresponding to each entity pair in the entity pair list are obtained through the method, and the alternative relations are combined to form an alternative relation set corresponding to the entity pair list.

Step S303, according to the input question and the alternative relation set, obtaining the probability distribution of the input question on the alternative relation set.

The purpose of this step is to obtain the probability distribution of the input problem over the set of alternative relations obtained above.

The probability distribution is a correct probability distribution condition composed of correct probabilities of the problems on each candidate relation in the candidate relation set. Firstly, a score of the input question on each alternative relationship in the alternative relationship set needs to be obtained through a relationship extraction model, and secondly, the scores of the input question on each alternative relationship in the alternative relationship set are combined to form a probability distribution of the input question on the alternative relationship set.

The relationship extraction model is a neural network model capable of scoring the correct probability of the input problem on each alternative relationship in the alternative relationship set.

The relationship extraction model provided in this embodiment is provided for the initial relationship extraction model by using known questions in the question-and-answer library and the relationship description corresponding to the entity pair in the known questions as training data, and training the initial relationship extraction model; and taking the relation extraction model which reaches the preset standard after training as the actually used relation extraction model.

The question-answer library is a database for collecting known questions and answers, wherein the known questions and the corresponding relation description of entity pairs in the questions are included.

The relationship description is a language for describing the relationship of the entity pair, and the entity pair relationship description have a one-to-one correspondence relationship.

The relationship extraction model provided in this embodiment actually describes the relationship between the known questions in the question-and-answer library and the entity pairs in the known questions, forms training data in an input format of the [ CLS ] question [ SEP ] relationship description [ SEP ] ", trains the initial relationship extraction model, and obtains a neural network model capable of evaluating and scoring the correct probability of the question and the relationship description. Therefore, in the practical use of the relationship extraction model, the data to be evaluated also needs to be the input format of the "[ CLS ] problem [ SEP ] relationship description [ SEP ]".

Fig. 4 is a flowchart of a probability distribution obtaining method provided in the present embodiment.

As shown in fig. 4, the probability distribution obtaining method provided in this embodiment includes the following steps:

step S303-1, mapping each alternative relation in the alternative relation set into a relation description.

The candidate relationship set is a set formed by combining candidate relationships or candidate relationship groups corresponding to all entity pairs in the input question. Before the correct probability evaluation is performed, each candidate relationship in the candidate relationship set needs to be mapped to a relationship description.

An optional implementation manner provided by this embodiment is as follows: and correspondingly acquiring the relationship description of each alternative relationship in the alternative relationship set according to the relationship and relationship description list.

The relationship and relationship description list is a data set for collecting the relationship and relationship description correspondingly, and the relationship can be mapped into the relationship description from the relationship and relationship description list.

Such as: when a user plays a game, he inputs "a deer man may not bring a cat bringing in wealth? "there is an entity pair in the input question, i.e., the entity pair" fawn man-cat bringing in wealth ", as can be seen from step S301. As can be seen from step S302, the candidate relationship group corresponding to the entity pair is: the method is suitable for carrying and recommending souls. Since the input question only includes one entity pair, the set of alternative relationships corresponding to the entity pair list of the input question is: the method is suitable for carrying and recommending souls. And mapping each alternative relation in the alternative relation set into a corresponding relation description by adopting a relation and relation description list. The set of mapped alternative relationships is shown in table 1:

table 1 alternative relation set table after mapping

Step S303-2, obtaining the correct probability of the input question on the relation description of each alternative relation through the relation extraction model.

Combining the input questions and each relationship description in table 1 into the input data format of the relationship extraction model, as shown in table 2:

table 2 input data format table of relational extraction model

The relation descriptions corresponding to the input questions are input into the relation extraction model in the data format shown in table 2, and the relation extraction model evaluates the correct probability of the input questions on the relation descriptions of each alternative relation according to the result of learning and training. The results of the correct probability evaluation are shown in table 3:

TABLE 3 correct probability evaluation results Table

As can be seen from Table 3: input question "may a young deer man not have a wealth bringing cat? The correct probability of the prop/equipment/article and the like suitable for a certain role or scene in the relation description is 68.9 percent, and the correct degree is evaluated to be medium; input question "may a young deer man not have a wealth bringing cat? The correct probability of the relation description 'equipment/props/articles and the like which are suitable for being carried by the character' is 88.7%, and the correct degree is evaluated to be equal; input question "may a young deer man not have a wealth bringing cat? The probability of correctness on the relationship description 'imperial soul of recommended collocation' is 12.5%, the degree of correctness is evaluated as lower, and the like. That is, the input question "may a young deer man not have a wealth bringing cat? The description of the relationship of "there is a false of the recommended collocation" is basically impossible.

And step S303-3, taking the probability of correctness of the input question on the relationship description of each alternative relationship as the score of the input question on each alternative relationship.

In the mapped candidate relationship set, the candidate relationships and the relationship descriptions are in one-to-one correspondence, so that the probability that the input question is correct in the relationship description of each candidate relationship can be used as the score of the input question in each candidate relationship. That is, the correct probability of the input question over the relationship description is mapped to the score of the input question over the alternative relationship. The scoring results of the input questions on the alternative relationships are shown in table 4:

TABLE 4 input score results table for question on alternative relations

Step S303-4, according to the score of the input question on each alternative relation in the alternative relation set, forming the probability distribution of the input question on the alternative relation set.

The set of candidate relations comprises a plurality of candidate relations, and the score of the input question on each candidate relation constitutes the probability distribution of the input question on the set of candidate relations. According to the scoring result of the input question on the alternative relations shown in table 4, the following results can be obtained: input question "may a young deer man not have a wealth bringing cat? "probability distribution over the set of alternative relationships is [0.689, 0.887, 0.125 ].

An alternative set of relationships can be thought of as a relationship space, with the input question "may a doe man not have a wealth bringing cat? "there are three alternative relations in the corresponding alternative relation set, then the input question" may not be a lucky cat for a deer man? The corresponding relationship space is a three-dimensional relationship space. The relationship space may be multidimensional, depending on the set of candidate relationships to which the input question corresponds.

The probability distribution of the input question on the candidate relationship set is described in detail by taking the input question including two entities as an example. The method provided by the embodiment is also applicable to the input problem comprising more entities. Such as: the input question Q comprises an entity a, an entity b and an entity c, the probability distribution of the input question Q on the alternative relation set is obtained, and the method comprises the following steps:

first, an entity pair list is obtained: ab. ac, bc.

Secondly, acquiring a candidate relation set: the candidate relationship of the entity pair ab is alpha, the candidate relationship of the entity pair ac is beta, gamma and delta, and the candidate relationship of the entity pair bc is eta and theta. Then the set of candidate relations corresponding to the list of entity pairs of the input question is α, β, γ, δ, η, θ.

Thirdly, acquiring a mapped alternative relation set: mapping each alternative relation in the alternative relation set into a relation description, wherein the relation description of the alternative relations is alpha ', beta', gamma ', delta', eta 'and theta'.

Fourth, the correct probability of the input question on the relationship description of each alternative relationship is obtained: and (4) forming an input data format of a relation extraction model by the input question Q and each relation description, inputting the input data format into the relation extraction model, and evaluating the correct probability of the input question on each relation description. Such as: the correct probability distribution of the input problem Q on the relational description alpha ', beta', gamma ', delta', eta 'and theta' is u%, v%, w%, x%, y% and z%.

Fifthly, obtaining the probability distribution of the input problem on the candidate relation set: and taking the correct probability of the input question on the relation description of the alternative relation as the score of the input question on the alternative relation, and forming the probability distribution of the input question on the alternative relation set according to the score. Such as: the probability distribution of the input question Q on the candidate relationship set is [ u%, v%, w%, x%, y%, z% ].

Step S304, obtaining a probability distribution of the known problem including any entity pair in the entity pair list on the candidate relationship set.

The purpose of this step is to obtain the probability distribution of the known problem over the set of alternative relationships obtained above.

The known questions are derived from a question and answer library, and answers corresponding to the known questions and the questions are collected in the question and answer library.

Fig. 5 is a flowchart for acquiring a probability distribution of a known problem on a set of candidate relationships according to the present embodiment. As shown in fig. 5, the method for obtaining the probability distribution of the known problem on the candidate relationship set provided by this embodiment includes:

and step S304-1, all known questions containing any entity pair in the entity pair list are obtained from the question-answering library.

And according to the entity pair list obtained in the step S301, screening all known questions containing any entity pair in the entity pair list from the question and answer library. Such as: the entity pair list comprises: ab. ac and bc, then the known questions screened from the question-answer library are the known questions comprising entity pair ab, entity pair ac or entity pair bc.

With the above input question, "may a young deer man not bring a cat bringing in wealth? "to illustrate, input a question" deer men may not have a wealth bringing cat? The entity pair list comprises an entity pair of 'fawn male-wealth bringing cat', and all known questions comprising the entity pair of 'fawn male-wealth bringing cat' are screened from the question and answer library, wherein the entity pair list comprises the following steps: the first known problem "is a wealth bringing cat fitted with a patterned spirit without a deer man? "the second known problem" how to match a wealth bringing cat with 4 items of girl men? "

And S304-2, acquiring the probability distribution of each known problem on the candidate relation set through a relation extraction model.

Combining the known problem and the relationship description of each alternative relationship in the alternative relationship set acquired in step S302 into an input data format of a relationship extraction model, inputting the relationship extraction model, acquiring a score of the known problem on each alternative focus in the alternative relationship set, and further acquiring a distribution probability of the known problem on the alternative relationship set.

With the above input question, "may a deer man not bring a wealth bringing cat? "illustratively, a first known question and a second known question are obtained from a question and answer library. Obtaining the distribution probability of the first known problem and the second known problem on the alternative relation set through the relation extraction model distribution, wherein the obtaining steps are as follows:

first, the relationship descriptions of each candidate relationship in the first known problem and second known problem distribution and candidate relationship set are combined into an input data format of the relationship extraction model. As shown in table 5:

table 5 input data format table of relational extraction model

Second, the relational description of the known question and the candidate relations is input into the relational extraction model in the data format of table 5, and the relational extraction model evaluates the probability that the input question is correct in the relational description of each candidate relation. The results of the correct probability evaluation are shown in table 6:

TABLE 6 correct probability evaluation results table

Thirdly, the correct probability of the known question on the relationship description of each alternative relationship is used as the score of the known question on each alternative relationship. The results of the scores of the known questions on the alternative relations are shown in table 7:

TABLE 7 scoring result table of known problems on alternative relations

Fourthly, according to the score of each known problem on each candidate relation, forming the probability distribution of each known problem on the candidate relation set.

From the scoring results of the known questions on the candidate relationships shown in table 7, it is possible to obtain: the first known problem "is a wealth bringing cat fitted with a patterned spirit without a deer man? "probability distribution over alternative set of relationships is [0.589, 0.789, 0.305 ]; the second known problem, "how to match a deer man 4-piece suit? "probability distribution over the set of alternative relationships is [0.653, 0.485, 0.323 ].

Step S305, calculating a similarity of probability distributions of the input question and the known question on the candidate relationship set, and determining an output answer corresponding to the input question according to the similarity of the probability distributions.

The purpose of this step is to calculate the similarity of the probability distribution of the input question and the known question, determine the known question most similar to the input question, and output the answer of the known question as the answer of the input question.

The implementation provides a method for measuring the similarity of two probability distributions, that is, the LS divergence value is used to measure the similarity of the probability distributions of the input question and the known question on the candidate relationship set.

The JS divergence (Jensen-Shannon divergence) is a method for measuring the similarity between two probability distributions, the problem that the KL divergence is asymmetric is solved, the JS divergence is symmetric, the value is between 0 and 1, the two probability distributions are 0 if the two probability distributions are the same, and are 1 if the two probability distributions are opposite, and the similarity between the two probabilities can be represented by the intermediate value between 0 and 1.

Fig. 6 is a flowchart for determining an output response according to the similarity of probability distributions according to the present embodiment.

As shown in fig. 6, the method for calculating the similarity degree of the probability distribution of the input question and the known question on the candidate relationship set and determining the output answer corresponding to the input question according to the similarity degree provided by the present embodiment includes the following steps:

and step S305-1, calculating JS divergence values of the probability distribution of the input problem on the alternative relation set and the probability distribution of the known problem on the alternative relation set.

Calculating a JS divergence value of the probability distribution of the input problem and the known problem, firstly, carrying out normalization processing on the probability distribution, specifically, adopting a Softmax formula (normalization index function) to process, and then adopting a JS formula to calculate the similarity of the probability distribution after the normalization processing.

The Softmax formula is:

wherein z is _i Representing the ith data in the probability distribution, z _C Represents the C-th data in the probability distribution, and C represents the total number of data in the probability distribution.

The JS formula is:

where p represents the first probability distribution, q represents the second probability distribution, p (x) represents the xth data in the first probability distribution, and q (x) represents the xth data in the second probability distribution.

With the above input question, "may a deer man not bring a wealth bringing cat? "by way of example, through steps S303 and S304, an input question" may a young deer man not take a cat bringing in wealth? "probability distribution over set of alternative relationships [0.689, 0.887, 0.125]The first known problem, "is the deer man fit for a wealth bringing cat? "probability distribution over set of alternative relationships [0.589, 0.789, 0.305]The second known problem, "how to match a deer cat with a male 4-piece set? "probability distribution over set of alternative relationships [0.653, 0.485, 0.323]. Respectively with P ₀ 、P ₁ 、P ₂ Representing a probability distribution of the input question, the first known question, the second known question over the alternative relations. The probability distributions of the input question, the first known question and the second known question on the alternative relations are summarized as shown in table 8:

TABLE 8 summary of probability distributions

Representing symbol	Problem(s)	Probability distribution
			P ₀	May a young deer man bring a wealth bringing cat?	[0.689，0.887，0.125]
P ₁	Is there a deer man in the right spirit of a cat bringing in wealth?	[0.589，0.789，0.305]
			P ₂	How can a wealth bringing cat match 4 items of deer men?	[0.653，0.485，0.323]

Calculating the JS divergence values of the probability distribution of the input problems on the alternative relation set and the probability distribution of the known problems on the alternative relation set, and specifically comprising the following steps:

firstly, the probability distribution is normalized to obtain normalized probability distribution.

With P ₀ [0.689，0.887，0.125]For example, the calculation process is as follows:

after normalization processing, the probability distribution [0.689, 0.887, 0.125] of the input problem on the candidate relations is converted into [0.36, 0.44, 0.20], and the sum of all numerical values in the normalized probability distribution is 1. Thus, the normalized probability distribution can be considered as the probability distribution of the problem in different dimensions, such as: the normalized probability distribution [0.36, 0.44, 0.20] can be viewed as the probability distribution of the input problem in three dimensions.

The probability distributions of the first known problem and the second known problem on the candidate relationship are normalized by the same calculation method, and the normalization results are shown in table 9:

TABLE 9 normalized probability distribution results Table

Representing symbol	Problem(s)	Probability distribution	Normalized probability distribution
				P ₀	May a young deer man not have a wealth bringing cat?	[0.689，0.887，0.125]	[0.36，0.44，0.20]
P ₁	Is there a deer man in the right spirit of a cat bringing in wealth?	[0.589，0.789，0.305]	[0.34，0.41，0.25]
				P ₂	How can a wealth bringing cat match 4 items of deer men?	[0.653，0.485，0.323]	[0.39，0.33，0.28]

Second, the JS divergence values of the probability distribution of the input problem and the known problem over the set of alternative relationships are computed.

Calculating JS divergence value by adopting normalized probability distribution to input probability distribution P of problem on alternative relation set ₀ Probability distribution P over a set of alternative relations to a first known problem ₁ The JS divergence value of (1) is calculated as an example, and the calculation process is as follows:

JS(P ₀ ||P ₁ )＝0.5×0.002664+0.5×0.002528＝0.002596

thus, the JS divergence value of the probability distribution of the input problem and the first known problem over the set of candidate relationships is 0.002528.

The probability distribution P of the input problem on the alternative relation set is calculated in the same way ₀ Probability distribution P over a set of alternative relations to a second known problem ₂ The JS divergence value is calculated as follows:

JS(P ₀ ||P ₂ )＝0.01095

therefore, the JS divergence value of the probability distribution of the input problem and the second known problem over the set of candidate relationships is 0.01095.

And S305-2, judging the similarity between the input question and the known question according to the JS divergence value.

The JS divergence value measures the similarity of two probability distributions, the value range is [0,1], the two are equal to each other, and are 0, and the two are opposite to each other, and are 1. The similarity between the input question and the known question can be judged according to the JS divergence value of the probability distribution of the input question and the known question on the alternative relation set.

The basis that this embodiment provided judge according to JS divergence value the input problem with the similarity of known problem includes: comparing whether the JS divergence value is smaller than a preset similarity threshold value or not; judging whether the input question is similar to the known question according to the comparison result, comprising: if so, the input question is similar to the known question; if not, the input problem is dissimilar to the known problem.

The preset similarity threshold value is a preset JS divergence threshold value, if the JS divergence values of the input problems and the known problems in the probability distribution on the candidate relationship set, which are obtained through calculation, are smaller than the preset similarity threshold value, the input problems are similar to the known problems, and if the JS divergence values of the input problems and the known problems in the probability distribution on the candidate relationship set, which are obtained through calculation, are larger than or equal to the preset similarity threshold value, the input problems are not similar to the known problems. The preset similarity threshold may be adjusted according to the specific application scenario and practical accumulation.

With the above input question, "may a young deer man not bring a cat bringing in wealth? "to illustrate, the preset similarity threshold is set to 0.01, and as can be seen from the calculation result in step S305-1, the input question" may the young deer man not bring a cat bringing in wealth? "does there exist a deer man fit to a wealthy cat? "JS scatter value of probability distribution over candidate relationship set is 0.002528. Because 0.002528<0.01, it can be judged that there is an input question "deer man may not bring a cat bringing in wealth? "does there exist a deer man in a manner suitable for a wealthy cat? "similar.

Also, as can be seen from the calculation result in step S305-1, the input question "may the young deer man not bring a cat bringing in wealth? "with the second known problem" how to match a deer cat with 4 men? "the JS divergence value of the probability distribution over the candidate relationship set is 0.01095. Because 0.01095>0.01, it can be judged that there is an input question "deer man may not bring a cat bringing in wealth? "with the second known problem" how to match a deer cat with 4 men? "dissimilar".

The basis that this embodiment provided judge with JS divergence value the input problem with the similarity of known problem still includes: comparing the magnitudes of the JS divergence values; according to the comparison result, judging the known problem with the highest similarity to the input problem, wherein the judging comprises the following steps: the smaller the JS divergence value, the higher the similarity of the input question to the known question.

For the case that the input question corresponds to a plurality of known questions, JS divergence values of probability distributions of the input question and each known question on the candidate relationship set can be compared, and the smaller the JS divergence value, the higher the similarity between the input question and the known question is.

With the above input question, "may a young deer man not bring a cat bringing in wealth? "for example, as can be seen from the calculation result in step S305-1, the input question" deer man may not bring a wealth bringing cat? "does there exist a deer man fit to a wealthy cat? "JS divergence value of probability distribution on candidate relationship set is 0.002528, input question" may doe man not take cat bringing in wealth? "with the second known problem" how to match a deer cat with 4 men? "the JS divergence value of the probability distribution over the candidate relationship set is 0.01095. Because 0.002528<0.01095, it can be judged that there is an input question "deer man may not bring a cat bringing in wealth? "does there exist a deer man fit to a wealthy cat? "has the highest similarity.

And step S305-3, determining an output answer corresponding to the input question according to the similarity between the input question and the known question.

The determining an output answer corresponding to the input question according to the similarity between the input question and the known question provided by this embodiment includes: and taking the answer of the known question similar to the input question as the output answer corresponding to the input question.

With the above input question, "may a young deer man not bring a cat bringing in wealth? "to illustrate, if the preset similarity threshold is set to 0.01, as can be seen from step S305-1 and step S305-2, the input question" may a deer man not bring a wealth bringing cat? "does there exist a deer man fit to a wealthy cat? "similar, but input question" deer man may not have a wealth bringing cat? "with the second known problem" how to match a deer cat with 4 men? "dissimilar". Thus, can one first known question "do not have deer men in a suitable fashion for a wealthy cat? "answer to" do the deer man may not have a wealth bringing cat? "to output a response.

The determining an output answer corresponding to the input question according to the similarity between the input question and the known question provided by this embodiment further includes: and taking the answer of the known question with the highest similarity with the input question as the output answer corresponding to the input question.

For a case where the input question corresponds to a plurality of known questions, and JS divergence values of distribution probabilities of the input question and the plurality of known questions on the candidate relationship set are both smaller than the preset similarity threshold, the JS divergence value may be the smallest, that is, the answer of the known question most similar to the input question may be used as the output response corresponding to the input question.

With the above input question, "may a deer man not bring a wealth bringing cat? "for example, as shown in step S305-1 and step S305-2, the input question" deer man may not bring a cat bringing in wealth? "does there exist a deer man fit to a wealthy cat? "has the highest similarity. Thus, can one first known question "do not have deer men in a suitable fashion for a wealthy cat? "the answer to" may a deer man not have a wealth bringing cat? "to output a response.

The above first embodiment describes in detail the question-answering method provided by the present invention in an optional implementation manner, and the question-answering method provided by the present invention includes, but is not limited to, the implementation manner given in the first embodiment.

A second embodiment of the present invention provides a question-answering system. Fig. 7 is a schematic structural diagram of the question-answering system provided in this embodiment.

As shown in fig. 7, the question answering system provided in this embodiment includes: an entity identifying unit 701, an alternative relationship acquiring unit 702, an input problem probability distribution acquiring unit 703, a known problem probability distribution acquiring unit 704, and a similarity calculating unit 705.

The entity identifying unit 701 is configured to identify an entity in an input question, and obtain an entity pair list.

Optionally, the method for obtaining the entity recognition model includes:

The candidate relationship obtaining unit 702 is configured to obtain a candidate relationship of each entity pair in the entity pair list, so as to form a candidate relationship set corresponding to the entity pair list.

Optionally, the obtaining the alternative relationship of each entity pair in the entity pair list includes: and taking the relation existing in the knowledge graph of the entity pair as an alternative relation.

The input problem probability distribution obtaining unit 703 is configured to obtain, according to the input problem and the candidate relationship set, a probability distribution of the input problem on the candidate relationship set.

Optionally, the obtaining, according to the input question and the candidate relationship set, the probability distribution of the input question on the candidate relationship set includes:

and forming a probability distribution of the input question on the candidate relationship set according to the score of the input question on each candidate relationship in the candidate relationship set.

Optionally, the mapping each alternative relationship in the alternative relationship set into a relationship description includes: and correspondingly acquiring the relationship description of each alternative relationship in the alternative relationship set according to the relationship and relationship description list.

The known problem probability distribution obtaining unit 704 is configured to obtain a probability distribution of a known problem that includes any entity pair in the entity pair list over the candidate relationship set.

acquiring all known questions containing any entity pair in the entity pair list from a question-answer library;

The similarity calculation unit 705 is configured to calculate a similarity of probability distributions of the input question and the known question on the candidate relationship set, and determine an output answer corresponding to the input question according to the similarity of the probability distributions.

comparing the magnitudes of the JS divergence values;

A third embodiment of the present invention provides an electronic apparatus. Fig. 8 is a schematic structural diagram of the electronic device provided in this embodiment.

As shown in fig. 8, the electronic device provided in this embodiment includes: a memory 801 and a processor 802.

The memory 801 is used for storing computer instructions for executing the question answering method.

The processor 802, configured to execute the computer instructions stored in the memory 801, performs the following operations:

identifying entities in the input problem and acquiring an entity pair list;

and performing pairwise formation on the entities in the input problem to form the entity pair list.

Optionally, the method for obtaining the entity recognition model includes:

Optionally, the obtaining the alternative relationship of each entity pair in the entity pair list includes: and taking the relation of the entity pair existing in the knowledge graph as an alternative relation.

comparing the magnitudes of the JS divergence values;

Optionally, the determining, according to the similarity between the input question and the known question, an output answer corresponding to the input question includes: and taking the answer of the known question with the highest similarity with the input question as the output answer corresponding to the input question.

A fourth embodiment of the present invention provides a computer-readable storage medium, which includes computer instructions, and the computer instructions, when executed by a processor, are used to implement the technical solution described in the first embodiment of the present invention.

It is noted that the terms "first," "second," and the like herein are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprising," "having," "including," and "comprising," and other similar forms of words are intended to be inclusive and open-ended with respect to any item or items following any item of the preceding description, and no term or items from any preceding description is intended to be exhaustive or limited to any listed item or items.

As used herein, unless otherwise expressly specified, the term "or" includes all possible combinations unless not feasible. For example, if it is expressed that a database may include a or B, the database may include a, or B, or both a and B, unless specifically stated or not otherwise possible. As a second example, if expressed as a certain database may include A, B or C, the database may include databases a, or B, or C, or a and B, or a and C, or B and C, or a and B and C, unless specifically stated or not feasible otherwise.

It is to be noted that the above-described embodiments may be realized by hardware or software (program code), or a combination of hardware and software. If implemented in software, it may be stored in the computer-readable medium described above. The software, when executed by a processor, may perform the above disclosed methods. The computing unit and other functional units described in this disclosure may be implemented by hardware or software, or a combination of hardware and software. It will also be understood by those skilled in the art that the modules/units may be combined into one module/unit, and each module/unit may be further divided into a plurality of sub-modules/sub-units.

In the foregoing detailed description, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments may occur. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims. The sequence of steps shown in the figures is also for illustrative purposes only and is not meant to be limited to any particular sequence of steps. Thus, those skilled in the art will appreciate that the steps may be performed in a different order while performing the same method.

In the drawings and detailed description of the present application, exemplary embodiments are disclosed. However, many variations and modifications may be made to these embodiments. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. A question-answering method, comprising:

identifying an entity in an input problem, and acquiring an entity pair list;

acquiring the alternative relationship of each entity pair in the entity pair list to form an alternative relationship set corresponding to the entity pair list;

acquiring probability distribution of known problems of any entity pair in the entity pair list on the alternative relation set;

2. The method of claim 1, wherein identifying the entity in the input question, obtaining a list of entity pairs, comprises:

3. The method of claim 2, wherein obtaining the entity recognition model comprises:

4. The method of claim 1, wherein the obtaining the candidate relationship of each entity pair in the entity pair list comprises: and taking the relation existing in the knowledge graph of the entity pair as an alternative relation.

5. The method of claim 1, wherein obtaining the alternative relationship for each entity pair in the list of entity pairs further comprises: and taking the relation of the type pair corresponding to the entity pair existing in the knowledge graph as an alternative relation.

6. The method according to claim 1, wherein the obtaining a probability distribution of the input question over the set of candidate relationships according to the input question and the set of candidate relationships comprises:

7. The method of claim 6, wherein obtaining the relational extraction model comprises:

8. The method of claim 6, wherein obtaining the score of the input question on each candidate relationship in the set of candidate relationships through a relationship extraction model comprises:

9. The method according to claim 8, wherein the mapping each alternative relation in the alternative relation set to a relation description comprises: and correspondingly acquiring the relation description of each alternative relation in the alternative relation set according to the relation and relation description list.

10. The method of claim 1, wherein obtaining a probability distribution over the set of candidate relationships of a known problem involving any one of the entity pairs in the list of entity pairs comprises:

11. The method according to claim 1, wherein the calculating a degree of similarity of the input question to the probability distribution of the known questions over the set of candidate relationships and determining an output answer corresponding to the input question according to the degree of similarity comprises:

12. The method of claim 11, wherein said determining the similarity of the input question to the known question based on the JS divergence value comprises:

13. The method of claim 11, wherein said determining the similarity of the input question to the known question based on the JS divergence value further comprises:

comparing the magnitudes of the JS divergence values;

14. The method of claim 11, wherein determining the output answer corresponding to the input question based on the similarity of the input question to the known question comprises: and taking the answer of the known question with the highest similarity with the input question as the output answer corresponding to the input question.

15. A question-answering system, comprising: the system comprises an entity identification unit, an alternative relation acquisition unit, an input problem probability distribution acquisition unit, a known problem probability distribution acquisition unit and a similarity calculation unit;

16. An electronic device, comprising: a memory, a processor;

the memory to store one or more computer instructions;

the processor to execute the one or more computer instructions to implement the method of claims 1-14.

17. A computer-readable storage medium having stored thereon one or more computer instructions which, when executed by a processor, perform the method of claims 1-14.