CN108446286B

CN108446286B - Method, device and server for generating natural language question answers

Info

Publication number: CN108446286B
Application number: CN201710084397.0A
Authority: CN
Inventors: 张增明; 姜飞俊; 陈永健; 汪洋; 胡于响; 沈慧; 王成龙; 杨洋
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-02-16
Filing date: 2017-02-16
Publication date: 2023-04-25
Anticipated expiration: 2037-02-16
Also published as: CN108446286A; TW201832104A; WO2018149326A1; TWI746690B

Abstract

The application provides a method, a device and a server for generating natural language question answers, wherein the method for generating the question answers comprises the following steps: responding to triggering of a question, and marking the question at least according to a pre-trained keyword extraction model to obtain a word marking result, wherein the word marking result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword; determining the target intention of the question according to the word labeling result and a pre-trained intention recognition model; obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention. According to the method and the device for identifying the intention, the intention recognition model which can accurately recognize the intention of the user is trained based on the intention template, and the keyword extraction model is used for extracting the keywords of the question of the user to assign the intention parameters in the intention template, so that the logic in the intention template can be executed to automatically generate the answer of the question.

Description

Method, device and server for generating natural language question answers

Technical Field

The application relates to the technical field of intelligent question and answer, in particular to a method, a device and a server for generating natural language question answers based on an intention recognition model and a keyword extraction model.

Background

Today, with the growing development of the internet, many websites or online stores and the like employ an intelligent question answering system to answer the consultation of various questions. The intelligent question-answering system is a system which can accurately position question knowledge required by users such as websites or online stores in a question-answer mode and provide personalized information services for the users by interacting with the users.

In the prior art, intelligent question and answer is realized by adopting a mode of presetting rules. For example, enumerating all possible questions and answers to the questions, and then abstracting a rule for each question. Specifically, when generating an answer of the current question, checking whether the current question is matched with each rule in sequence, and if so, executing an answer mode appointed by the rule to acquire the answer.

Disclosure of Invention

However, the inventor finds that in the prior art, the configuration of the rules is troublesome, the possibility of mismatching the rules is high, the rules may contradict each other, and the new rules need not conflict with other existing rules when being added. Therefore, the maintenance cost of rule-based intelligent questioning and answering is large. In the specific implementation, because the question is tried to be matched with the corresponding relation of each rule, if the number of the rules is large, the matching efficiency is low. In addition, when the matching is not completed, the question set by the user cannot be answered, so that the flexibility is low, namely, if one question is hoped to be answered, one rule corresponding to the question needs to be added.

Based on the above, the application provides an automatic generation method of question answers based on a pre-trained intention recognition model and a keyword extraction model, which is used for enumerating all intentions in the field, namely the query purpose of one question by adopting a pre-established body mode, configuring a corresponding intention template according to the intentions, namely what question is configured to be answered by the intention template and how to answer, further training based on the intention template to obtain an intention recognition model capable of more accurately recognizing the intention of a user, and extracting keywords of the question of the user by adopting the keyword extraction model to assign values to the intention parameters in the intention template, so that logic in the intention template can be executed to automatically generate the answers of the question.

The application also provides an automatic generation device and a server of question answers, such as a question answering engine and the like, which are used for guaranteeing the realization and application of the method in practice.

In order to solve the above problems, the present application discloses a method for generating answers to questions, which includes:

responding to triggering of a question, and marking the question at least according to a pre-trained keyword extraction model to obtain a word marking result, wherein the word marking result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword;

Determining the target intention of the question according to the word labeling result and a pre-trained intention recognition model;

obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention.

The responding to triggering the question, marking the question at least according to a pre-trained keyword extraction model to obtain a word marking result, comprising the following steps:

word segmentation is carried out on the question to obtain a word segmentation set of the question;

based on a domain dictionary corresponding to the domain to which the question belongs, keyword marking is carried out on each word in the word segmentation set to obtain a first marking result; based on a pre-trained keyword extraction model, keyword marking is carried out on each word in the word segmentation set to obtain a second marking result; the domain dictionary is used for storing the corresponding relation between each domain word and the corresponding word type;

and determining word labeling results of each word in the word segmentation set according to the first labeling result and the second labeling result.

The determining the word labeling result of each word in the word segmentation set according to the first labeling result and the second labeling result comprises the following steps:

Judging whether the first marking result and the second marking result are marked as keywords or not according to each segmentation word in the segmentation word set, if so, determining the segmentation word as the keyword, and determining the word type of the keyword according to the first marking result or the second marking result;

if the first labeling result is labeled as a keyword and the second labeling result is labeled as a non-keyword, the segmentation is determined to be a non-keyword.

Wherein the domain dictionary is generated by:

acquiring each attribute of each entity in the field;

taking the attribute value corresponding to each attribute as a keyword, taking the entity tag and the attribute name as word types of the keywords, and determining the associated vocabulary of the keywords as the keywords, wherein the associated vocabulary comprises the following components: synonyms, paraphrasing, spoken expressions and aliases.

The determining the target intention of the question according to the word labeling result and a pre-trained intention recognition model comprises the following steps:

the word labeling result is used as input of the pre-trained intention recognition model, and a plurality of candidate intentions and corresponding credibility are obtained;

determining the candidate intention with the highest credibility among the plurality of candidate intentions as the current intention;

Judging whether each keyword in the word labeling result is complete or not according to the current intention template corresponding to the current intention; the intent template includes: determining all intention parameters required by an intention;

if so, determining the current intention as a target intention;

if not, selecting the next candidate intention as the current intention according to the sequence from high to low of the credibility, and executing the current intention template corresponding to the current intention to judge whether each keyword in the word labeling result is complete or not until the candidate intentions are selected.

The step of judging whether each keyword in the word labeling result is complete according to the current intention template corresponding to the current intention comprises the following steps:

acquiring all intention parameters configured in the current intention template;

and assigning the all intention parameters by adopting the word type of each keyword in the word labeling result, and judging whether the all intention parameters can be successfully assigned.

At least using the word labeling result as input of the pre-trained intention recognition model to obtain a plurality of candidate intentions and corresponding credibility thereof;

judging whether the credibility of the current intention is larger than a preset credibility threshold, if so, determining the current intention as the target intention;

if the candidate intentions are not greater than the preset number, selecting the first plurality of candidate intentions as intentions to be selected, sending the intentions to be selected to a terminal for selection by a user, and determining the intentions returned by the terminal and selected by the user as target intentions;

judging whether each keyword in the word labeling result is complete according to the target intention template corresponding to the target intention, and if not, determining intention parameters corresponding to the incomplete keywords as missing intention parameters; generating a parameter problem related to the target intention according to the missing intention parameter and the target intention template; returning the parameter questions to the client for display to the user; the intent template includes: the correspondence between the intent parameters and the parameter questions;

And acquiring the missing intention parameters according to the parameter answers returned by the client and input by the user.

Wherein the method further comprises:

storing dialogue content input by a user into a dialogue database, wherein the dialogue content comprises the following components: the user selected intent and/or the user entered parameter answer.

Wherein the method further comprises:

updating the dialogue content in the dialogue database,

or deleting the dialogue content input by the user when the preset time period is reached.

The method at least uses the word labeling result as input of the pre-trained intention recognition model to obtain a plurality of candidate intentions and corresponding credibility thereof, and comprises the following steps:

acquiring the dialogue content from a dialogue database;

and inputting the word labeling result and the dialogue content into a pre-trained intention recognition model to obtain a plurality of candidate intentions and the corresponding credibility thereof.

The step of judging whether each keyword in the word labeling result is complete according to the target intention template corresponding to the target intention comprises the following steps:

acquiring all intention parameters configured in the target intention template;

and assigning all the intention parameters in the target intention template by adopting the word type of each keyword in the word labeling result, and judging whether the assignment of all the intention parameters is successful or not.

Wherein the intent template is generated by:

determining the intention of all questions in each field according to the relation among the entities represented by the ontology in each field; the intent is for query purposes expressing questions, the intent comprising: querying the attributes of the entities, querying all relationships between the entities, querying other or all attributes of the entities by one or more attributes of the entities, and/or querying relationships of the entities by one or more attributes of the entities;

configuring a corresponding intention template for each determined intention, wherein the intention template comprises: template identification, template description, intention parameters, parameter problems corresponding to the intention parameters, example sentence groups corresponding to the intention, and combination rules of the example sentence groups; the template identifier is used for uniquely representing an intention template, and the template description is used for describing the purpose of the intention template; the intention parameter is a parameter corresponding to determining an intention, and the parameter problem is as follows: when the corresponding intention parameter is missing, the problem needs to be sent to the user, the example sentences in the example sentence groups comprise variable names of entities and are used for representing the problem which can be solved by the intention template, and the combination rule of each example sentence group is used for representing: and combining the example sentences in different example sentence groups.

Wherein the intent recognition model and keyword extraction model are trained by:

generating training corpus according to example sentence groups and example sentence combination rules configured in the intention template;

and training according to the training corpus by adopting a machine learning algorithm to obtain an intention classification model and a keyword extraction model.

The generating training corpus according to the example sentence group and the example sentence combination rule configured in the intention template comprises the following steps:

obtaining a variable value list of an entity, wherein the variable value list comprises: the variable name of the entity, the variable value of the entity and the corresponding relation of the entity and the variable value of the entity;

generating variable example sentences comprising the variable names of the entities according to the combination rules of the example sentence groups and the example sentence groups configured in the intention template;

the variable values of the entities in the variable value list are respectively and correspondingly replaced with variable names in the variable example sentences to obtain real example sentences;

marking the real example sentences to obtain training corpus; the training corpus comprises: and the template identifier corresponding to the real example sentence and each keyword type contained in the real example sentence.

According to a pre-configured target intention template corresponding to the target intention, obtaining a target answer corresponding to the target intention comprises the following steps:

Assigning a value to the intention parameter according to the intention parameter configured in the target intention template to obtain a query parameter;

acquiring source data of the target answer according to the query parameters;

processing the source data according to a preset processing mode to obtain processed source data;

and generating the target answer according to the processed source data.

The application also discloses a generating device of natural language question answers, the generating device is integrated on a server, and the generating device comprises:

the keyword labeling unit is used for responding to the triggering question, labeling the question at least according to a pre-trained keyword extraction model to obtain a word labeling result, and the word labeling result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword;

the intention determining unit is used for determining the target intention of the question according to the word labeling result and a pre-trained intention recognition model;

the answer acquisition unit is used for acquiring a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention.

Wherein, the keyword labeling unit includes:

The word segmentation subunit is used for segmenting the question to obtain a word segmentation set of the question;

the first marking subunit is used for marking keywords of each word in the word segmentation set based on a domain dictionary corresponding to the domain to which the question belongs to obtain a first marking result;

the second marking subunit is used for marking the keywords of each word in the word segmentation set based on a pre-trained keyword extraction model to obtain a second marking result; the domain dictionary is used for storing the corresponding relation between each domain word and the corresponding word type;

and the first determining subunit is used for determining word labeling results of each word in the word segmentation set according to the first marking result and the second marking result.

Wherein the first determining subunit includes:

the first judging subunit is used for judging whether the first marking result and the second marking result are marked as keywords or not according to each word in the word segmentation set;

the second determining subunit is configured to determine the word segmentation as a keyword if the first judging subunit is yes, and determine a word type of the keyword according to the first marking result or the second marking result;

And the third determination subunit is used for determining the word segmentation as a non-keyword in the case that the result of the first judgment subunit is that the first marking result is marked as a keyword and the second marking result is marked as a non-keyword.

Wherein the intention determining unit includes:

the first model processing subunit is used for taking the word labeling result as the input of the pre-trained intention recognition model to obtain a plurality of candidate intentions and corresponding credibility;

a fourth determining subunit, configured to determine, as a current intention, a candidate intention with highest reliability among the plurality of candidate intentions;

the second judging subunit is used for judging whether each keyword in the word labeling result is complete or not according to the current intention template corresponding to the current intention; the intent template includes: determining all intention parameters required by an intention;

a fifth determining subunit, configured to determine the current intention as a target intention if the result of the second judging subunit is yes;

a selecting subunit, configured to select, if the result of the second judging subunit is negative, a next candidate intention as a current intention in order of high reliability;

And the triggering subunit is used for triggering the second judging subunit after the selecting subunit selects the next candidate intention until the candidate intents are all selected.

Wherein the intention determining unit includes:

a second model processing subunit, configured to at least use the word labeling result as an input of the pre-trained intent recognition model, to obtain a plurality of candidate intentions and corresponding credibility thereof;

a sixth determining subunit, configured to determine, as the current intention, a candidate intention with the highest reliability among the plurality of candidate intents;

the third judging subunit is used for judging whether the credibility of the current intention is larger than a preset credibility threshold value or not;

a seventh determining subunit, configured to determine the current intention as the target intention if the result of the third determining subunit is greater than the target intention;

a selecting subunit, configured to select a preset number of first candidate intents as intents to be selected if the result of the third judging subunit is negative;

a sending subunit, configured to send the intention to be selected to a terminal for selection by a user;

an eighth determining subunit, configured to determine, as a target intention, a user-selected intention returned by the terminal;

A fourth judging subunit, configured to judge whether each keyword in the word labeling result is complete according to a target intention template corresponding to the target intention;

a ninth determining subunit, configured to determine, if the result of the fourth determining subunit is negative, an intent parameter corresponding to the incomplete keyword as a missing intent parameter;

a question generation subunit, configured to generate, according to the missing intent parameters and the target intent template, parameter questions related to the target intent;

a question return subunit, configured to return the parameter question to the client for displaying to the user; the intent template includes: the correspondence between the intent parameters and the parameter questions;

and the parameter acquisition subunit is used for acquiring the absent intention parameter according to the parameter answer returned by the client and input by the user.

Wherein the answer acquisition unit includes:

the assignment subunit is used for assigning values for the intention parameters according to the intention parameters configured in the target intention template to obtain query parameters;

the source data acquisition subunit is used for acquiring the source data of the target answer according to the query parameters;

The processing subunit is used for processing the source data according to a preset processing mode to obtain processed source data;

and the generating subunit is used for generating the target answer according to the processed source data.

The application also discloses a server, its characterized in that, the server includes: the aforementioned generating device.

Compared with the prior art, the embodiment of the application has the following advantages:

in the embodiment of the application, because the trained intention recognition model and the keyword extraction model can more accurately recognize the intention, and more accurate keywords are extracted to prepare intention parameters for executing intention logic, not only is the efficiency and accuracy of answer generation improved, but also a dialogue can be performed with a user, the true intention of the user is further clarified when the user does not express the intention, or complete intention parameters and the like are further acquired under the condition that the intention of the user is clarified, so that the flexibility of the embodiment of the application is better than that of the prior art based on a rule mode, the dialogue can be performed with the user, and the experience of the user is also higher.

Of course, not all of the above-described advantages need be achieved at the same time in practicing any one of the products of the present application.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.

FIG. 1 is a flow chart of a collocation intent template in the present application;

FIG. 2 is a schematic structural view of the body of the present application;

FIG. 3 is another structural schematic of the body of the present application;

FIG. 4 is a flow chart of training an intent recognition model and a keyword extraction model in the present application;

FIG. 5 is a view of a scene architecture in practical application of the present application;

FIG. 6 is a flowchart of an embodiment 1 of a method for generating answers to questions of the present application;

FIG. 7 is a flowchart of example 2 of a method for generating answers to questions of the present application;

FIG. 8 is a flowchart of example 3 of a method for generating answers to questions of the present application;

fig. 9 is a block diagram of the structure of the question answer generating apparatus embodiment 1 of the present application;

fig. 10 is a block diagram of the structure of the intention determining unit 902 in the embodiment 2 of the question answer generating apparatus of the present application;

Fig. 11 is a block diagram of the structure of the intention determining unit 902 in embodiment 3 of the question answer generating apparatus of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

To facilitate an understanding of the present application by those skilled in the art, the terms appearing in the present application are first described below.

Body (ontology)

An ontology is a concept within a knowledge graph that can be understood as an abstraction of the real world. After the analysis of the real world, the human beings can abstract many entities, and each entity has own attribute, and a certain relationship exists between the entities. Then the ontology is to be understood as an expression and description of these entities and relationships between the entities.

For example, in the poetry field, poetry and poetry are two entities, each having respective attributes, wherein the attributes of poetry may include: name, dynasty, etc.; the properties of the poems may include: content, title, dynasty, genre, etc. And, there is a relationship between writing and being written between the poetry and the two entities of poetry.

Because an ontology describes entities and relationships between entities, all patterns of this area are known from the ontology, so that intent regarding all problems of this area can be enumerated. For example, in the poetry field, from the ontology existing in the poetry field, the following intended questions can be enumerated: the composition of the poetry, the dynasty of the poetry, the sex of the poetry, the content of the poetry, the dynasty of the poetry, the type of the poetry, or the author of the poetry.

Question and answer engine

The question and answer engine automatically gives an answer after the user inputs a question, and allows the user to input the question in a natural language form. The question and answer engine obtains the process of the answer according to the question input by the user: firstly, analyzing key words given by a user to understand the intention expressed by the user; then obtaining an answer by querying a knowledge base or by a certain algorithm; and finally giving an answer in the form of natural language. The question-answering engine may include a single round of question-answering engine and multiple rounds of question-answering engine. Wherein, the single round of question-answering engine is a question-answering, i.e. the user asks a question in which the intention and the key word (slot) are explicitly given, and the question-answering engine outputs an answer. The multi-round question and answer engine can comprehensively recognize the intention of the user and analyze slots according to the current questions and past question and answer records (contexts) of the user, and when the intention is unclear or various parameters required by the intention do not completely meet the conditions, the question and answer engine can control the dialogue to question the user until the intention and the complete slots are clearly expressed by the user, and the question and answer engine can not give the answer. The multiple round question and answer engine asks questions in the form of conversations, giving the user an experience as if it were communicating with a real person.

Intent template

The intention is the purpose that the user wants to express in the question the user is currently posed. For example, the user's intention to ask "Zhou Jielun what songs were sung" is to find songs sung by a singer; for example, the user's intention to ask what is the next sentence of the bright moon light before bed' is to find the next sentence of a poem. In the present embodiment, an intent template is used to represent an intent. One intent template may include the name of the intent, various parameters (slots) that satisfy the intent execution, and the execution logic of the intent, which in the present embodiment is configured in the intent template in the form of a groovy dynamic script.

When a user answers a question with intention, firstly, slots in the user question are extracted, assignment is given to parameters in an intention template, then execution logic configured by a grovy script is executed, and finally, an answer returned by the grovy script is obtained.

Keyword slot

A slot is a parameter that is intended to be executed, for example, the question of which songs are "Zhou Jielun" and "Zhou Jielun" is a slot. The problem of "what is the next sentence of the pre-bed bright moon light", which is a slot. If the intent is treated as a sentence, then slot is the keyword in the question; if the intent is seen as a function, then the slot is a parameter.

slot extraction is the extraction of slots in the user problem. Slot extraction is important if a question is to be answered correctly. If the slot extraction is wrong, the parameter error is directly caused, and the result error of the intended execution is further caused. Therefore, slot extraction is important to answer a question correctly.

grovy dynamic scenario

The groovy is a scripting language, is fully compatible with java grammar, and can be dynamically executed in java. Based on this feature, the groovy language can be used as a carrier of the execution logic in the intent template, i.e., the execution logic can be configured, so that the execution logic can be rewritten without recompilation or restarting the system, which greatly facilitates maintenance of the system and enhances the expandability of the system.

Knowledge base

Knowledge bases, i.e., knowledge graphs, are systems that model and store entities, attributes, and relationships between entities. Knowledge maps are graphs in which knowledge is stored, which is distinguished from relational databases.

Reasoning (reducing)

The embodiment of the application comprises two types of reasoning, wherein one type of reasoning is used for reasoning about knowledge, and the other type of reasoning is used for reasoning about the intention of a user. The embodiments of the present application focus mainly on reasoning about user intent, as reasoning about knowledge is the category of knowledge-graph systems. For example, it is noted in the knowledge base that a is the father of B, and B is the father of C, and when querying who is the grandmaster of C, it is necessary to use the "grandmaster = father' reasoning rule to reason. The reasoning of the user intention is to infer the user intention according to the past context and the current user questions, and automatically generated questions and answers of the user. For example, a human-machine dialogue about travel domain, shown in Table 1 below, where Q is the user question and A is the engine answer:

TABLE 1

As can be seen from the above-described human-machine conversation of table 1, the machine will attempt to query the user's intent and infer the user's current intent from the user's answer and context. When the user only speaks of the eiffel tower, the machine does not know the specific intent of the user, so the machine presents a question and then the user expresses how high the eiffel tower is. At this point, the machine understands that the user's intent is to query the altitude of an entity, and the entity is the Eiffel tower, so the machine gets the height attribute of the Eiffel tower from the knowledge base and returns to the user.

The user then proposes "how to go there", expressing an intention to explicitly plan a route to a place. But there are two parameters for the intent to plan a route to a location, one is the start point and one is the end point. The machine can get the end point from the context to be the eiffel tower, but does not know the start point, so the machine cannot perform this purpose, then the machine presents a problem about the start point, and finally the machine gets the user the start point to say that is the kefir gate.

In general, when a user speaks the name of an entity but does not express any intent, the machine will ask the user's intent like the first Q. When the machine has sufficient context, the machine may understand what the user says the trivia gate is a starting point parameter for the intention to plan a route. At this time, the starting point of the machine for planning the route is a Kernel gate, the end point is the Eiffel tower, and the planned route is obtained by executing the intention. When the user presents the problem of "if you go from holy to Paris", the machine, by context, can be drawn that the user presents the problem with the purpose of changing the starting point, whereupon the machine presents the result of execution of the planned route from the holy to the Eiffel tower.

For a more thorough understanding of the intent templates in the present application by those skilled in the art, referring to fig. 1, a flowchart of configuring intent templates in the present application is shown, and the process of configuring intent templates may include the steps of:

step 101: and establishing an ontology in the field.

When designing an intelligent question-and-answer engine for a new domain, an ontology within the domain needs to be built. Specifically, by analyzing the domain, what content the domain contains and the relationship between the contents are abstracted, then the abstracted content is entity (entity), the relationship between the contents is relationship between the entities (relationship), and finally the ontology (ontology) of the domain is established.

For example, when an intelligent question and answer engine is designed for the poetry field, poeters and poems are found to exist in the field through data investigation, and then the poeters and the poems are entities. Also, both poeters and poeters have their own attributes, for example, poeters have names, sexes, dynasties, etc., and poems have names, contents, etc. In addition, there is a relationship between writing and being written between poetry and poetry, and an ontology as shown in fig. 2 is established.

For example, when analyzing the travel field, the problem is simplified for convenience, only two entities, namely a city and a scenic spot, are considered in the travel field. The city has the attributes of name, country, etc., and the scenic spot has the attributes of name, type, etc. The relationship between the city and the attraction, including the relationship between how the attraction arrives, may be established as shown in fig. 3.

Step 102: generating a domain dictionary according to the ontology in the domain.

And generating a domain dictionary containing entity attribute values and entity attribute names according to the determined ontology comprising the entity and the relation between the entities. Specifically, the domain dictionary may be generated according to the following steps A1 to A2:

step A1: and acquiring each attribute of each entity in the field.

And acquiring all the attributes contained in all the entities in the body according to the determined body in the field. For example, for an entity determined in the travel field, it may be obtained that the entity included in the entity includes a city, a scenic spot, and the like, and it may be obtained that the city has a name, a country to which the city belongs, and the like, and the scenic spot has a name, a type, a business hours, and the like.

Step A2: and taking the attribute value corresponding to each attribute as a keyword, taking the entity tag and the attribute name as word types of the keywords, and determining synonyms, hyponyms and aliases of the keywords as keywords.

After obtaining the attributes such as the names of the cities and the countries, the obtained attribute values corresponding to the attributes are used as keywords, and the entity labels and the attribute names are used as word types of the keywords. For example, the names of cities are "Beijing" and "Shanghai", and the country is "China", and then "Beijing", "Shanghai" and "China" are used as keywords; "City name" and "country to which" belong are taken as word types. Further, synonyms, paraphraseology and aliases of the determined keywords may also be determined as keywords. For example, "capital of china" is an alias of the keyword "beijing", and therefore, "capital of china" is also used as the keyword.

In addition, keywords and word types of keywords can be determined by manual addition and supplement.

At this time, the domain dictionary in the art can be constructed by using the acquired attribute values corresponding to the attributes of the entities and the attribute names of the entities as word types. Since the keywords in one sentence can be determined by referring to the domain dictionary, in the subsequent flow, the sentence can be parsed and the keywords (slots) in the sentence can be extracted using the domain dictionary. In particular, the domain dictionary is very important for the role of the cold domain.

For example, there is a dictionary as shown in Table 2 below:

TABLE 2

Wherein, "# #" is a separator, the front of which is a keyword, and the rear of which is a word type corresponding to the keyword. In particular, the domain dictionary only has a keyword list, that is, only has keywords, and has no word type corresponding to the keywords, so that the domain dictionary only can identify the keywords in sentences, and cannot determine the types of the keywords.

When processing the following sentences: is the window front moon light is the litz write? Then the keywords shown below can be extracted: is? }.

Step 103: and determining the intention of all questions in each field according to the relation among the entities represented by the ontologies in each field.

The intent in this embodiment is for the purpose of query for expressing a question, and the intent may include the following four points:

the first point, inquires about the properties of the entity.

For example, for the above-mentioned city entity, there may be an intent as shown in table 3 below:

TABLE 3 Table 3

For example, for the above-mentioned attraction entity, there may be the intents shown in the following table 4:

TABLE 4 Table 4

All relationships between the second point, querying entities.

For example, the travel domain ontology described above, which has three relationships, may have three intentions as follows:

{ what scenery spots are in the city? (City to sight relationship) what city a sight belongs to? (attraction to city relationship) how from this attraction to another attraction? (scenic spot to scenic spot relationship) }

A third point, querying the entity for other or all attributes through one or more attributes of the entity.

Specifically, an entity is located through one or more attributes of the entity, and then other attributes of the entity are queried.

For example, the above-mentioned city entity may have the following intent: { is the name of which country is the city of xxx? (locating to cities by name, querying for country attributes) which cities are contained by countries with the name xxx? (by country to city, query name attribute) }.

Fourth, query the relationship of the entity through one or more attributes of the entity.

Specifically, the fourth point is the same as the method described in the third point, and is to locate the entity first; except that the third point is an attribute of the querying entity and the fourth point is a relationship of the querying entity. For example, for the above city entity, there may be an intent as shown in table 5 below:

TABLE 5

Through the four methods described in the first to fourth points, all possible intents can be enumerated through the ontology, and the four methods form an intent enumeration algorithm. Some of this enumerated intent is not practical in real scenes. For example, the ID attribute is questioned, so the result of intent enumeration may require some manual verification. The first point is actually a special case of the third point, namely, locating the entity by name attribute and then querying other attributes of the entity.

Step 104: corresponding intention templates are configured for each intention determined.

Specifically, the intent templates are responsible for solving a certain class of intents, such as the question shown below: { what scenery points are contained in Hangzhou? What sceneries are western? What places in south Beijing are interesting? Each question is an intention, i.e. ask a city which sights are available, and all these questions can be solved with an intention template. For another example, { howhow to remove the Lu from the Eiffel iron tower? The question of planning a route from the holy of Paris to the Eiffel tower, i.e. going from the holy of Paris to the Lufei }, and the shown route to a certain place can be solved by using an intention template. The intent template may be considered to be a profile of intents that is used to represent how the intent corresponding to the profile should generate an answer.

Specifically, the intent template may include: template identification, template description, intention parameters, parameter problems corresponding to the intention parameters, example sentence groups corresponding to the intention, and combination rules of the example sentence groups.

Wherein the template identifier is used for uniquely representing an intention template, and the template description is used for describing the purpose of the intention template; the intention parameter is a parameter corresponding to an intention; the parameter problems are as follows: a problem that the corresponding intention parameter needs to be sent to the user when the intention parameter is missing; the example sentences in the example sentence group comprise variable names of entities and are used for representing the problems which can be solved by the intention templates; the combination rule of each example sentence group is used for representing: and combining the example sentences in different example sentence groups.

Specifically, the template description of the first intention template may be "find scenic spots of a certain city"; the template description of the second intent template may be "plan route from place to place". The intent parameter of the first intent template is a city name; the second intent template has two intent parameters, one starting (startPoi) and one ending (endPoi). If the user does not give the intent parameters during the dialog, the user needs to be queried using the default parameter questions, table 6 below is a complete configuration of the intent parameters:

TABLE 6

The sentence beginning with 'q:' behind the above intent parameter is a parameter problem, and one intent parameter can correspond to a plurality of parameter problems, and one parameter problem can be randomly selected when the intent parameter is specifically used.

Specifically, the example sentence group corresponding to the intention template indicates what kind of problem is suitable for being solved by this template. The example sentence group is filled in by a designer of the template, and is generally obtained by analyzing frequently asked questions in the field. For example, the first intention template for finding a scenic spot in a city, and the example sentence set may be as shown in the following table 7:

TABLE 7

Since the above intent to find a city's sight is suitable for interrogation with this type of statement, this type of problem is classified into this intent template in the problem classification.

The specific example sentence patterns are given in the example sentence group, and the emphasis is on summarizing the sentence patterns, so that a set of all possible problem patterns is formed. Since the entity is a variable in the actual application scenario, after the variable is filled into the example sentences, a plurality of real example sentences can be generated. For example, $ { city_name }, which may be western, london, hangzhou, shanghai, etc., is filled into $ { city_name } of the example sentence shown in Table 7, a real sentence may be generated as follows: { what scenery points are contained in Hangzhou? What scenery spots are in Hangzhou? What places the Hangzhou has to play? What is the scenery spot the western security contains? What sceneries are western? What places are western in play? … … }.

The example sentence groups listed above have no group marks, default to a group of example sentences, so that no assembly logic exists between example sentences, and the intent templates adopting a one-to-one-answer mode for dialogue belong to single-round dialogue intent templates. The single-round dialogue intent template is suitable for a one-to-one application scene, and real example sentences generated by adopting example sentences in the single-round dialogue intent template do not contain contextual information.

In addition to single-round dialog intention templates, there are multiple rounds of dialog intention templates in the intention template. There are multiple questions in the multi-round dialog intention template, and the multi-sentence questions including the Context (Context), the specific multi-sentence questions can be as shown in table 8 below:

TABLE 8

From the above-mentioned multi-sentence problem, the server inquires the user about the multi-sentence, so that the intention of the user is obtained, and more intention parameters are obtained. Thus, the server needs to consider the context when answering questions of multiple rounds of conversations, and further queries to the user are needed when there is no clear intent of the user, or no more intent parameters are available. Further, the intent parameters or intent expressed by the user in the context may be fully considered when answering questions for multiple rounds of conversations.

It should be noted that, when configuring example sentences, the multi-round dialog intention template needs to explicitly set the concept of a group, for example, the example sentence configuration shown in the following table 9: there are six total groups, each starting with a "#" symbol. If the group number is followed by an "intent" key, this group is indicated as being given with explicit intent, otherwise, this group may simply express some intent parameter.

TABLE 9

/>

From the example sentences of the multi-round dialog intention template configuration, it can be seen that the first group gives both the intention and all the parameters required, which is a complete expression of the problem. If there is only a first set, the multi-round dialog intention template is converted into a single-round dialog intention template, which is seen to be a special form of the multi-round dialog intention template.

If the example sentence groups in the multi-round dialog intention template are combined according to a certain combination rule, a specific combination mode is as follows in table 10:

table 10

/>

Each row in the above-described combination rule table represents a combination rule, wherein the numbers represent group numbers. For example, the combination rule 10"2,3,6" means that a sentence is randomly selected from the second group, a sentence is randomly selected from the third group, a sentence is randomly selected from the sixth group, and finally, the three sentences are sequentially connected to form a group of multi-sentence example sentences.

In the same manner, each combination rule in table 10 can obtain an example sentence of a multi-sentence problem. For example, for combining rule 1: "1", which is an individual combination of the first group of example sentences in table 9, after filling the variable names of the entities included in the example sentences, can obtain three different real example sentences of the multi-sentence problem as follows: { howhow to get the Eiffel iron tower from the holy of Paris? What route to the eiffel tower from the holy of paris? Is a route to the eiffel tower planned from the holy of paris? And the three real example sentences of the multi-sentence problem all represent the same meaning.

As another example, according to the combination rule 12: "3,4,6", real example sentences of the multi-sentence problem as shown in the following table 11 can be obtained, wherein each set of real example sentences contains three sentences:

TABLE 11

It can be seen that, the example sentences corresponding to the multi-sentence problem may be incomplete, for example, lack of a certain parameter, or complete, for example, the example sentences combined by the combination rule 5. However, the example sentences of the multi-sentence problem generated by each combination rule always contain one example sentence expressing the intention. For example, the example sentence groups 1 and 3 shown in table 9 are both example sentence groups expressing intent, so sentences in the example sentence group 1 or example sentence group 3 are always used for combination in the combination rule in table 10. This is to simulate a real scene, where the user's problem is not classified into any one of the intent templates when the user does not provide enough parameters and intent. Therefore, in order to acquire the user intention, at least one sentence expressing the intention is contained in all the example sentence combination rules.

In addition, besides the template identifier, the template description, the intent parameter, the parameter problem corresponding to the intent parameter, the example sentence group corresponding to the intent, and the combination rule of each example sentence group are configured in the intent template, execution logic represented by a grovy script can be configured, and an answer to the user question can be generated by executing the grovy script. Specifically, the step of executing the grovy script to generate an answer to the user question may include: acquiring data, processing the data and generating an answer.

Wherein, the acquisition data refers to source data to be acquired for generating an answer. Corresponding data is needed to solve any problem. For example, it is necessary to answer which scenery points are in a city, and information of all scenery points in the city is acquired, and the scenery point information is source data for generating an answer. The open and close time of answering a attraction requires entity data for the attraction.

Processing data refers to processing source data that generates an answer. In general, after the source data is obtained, the source data cannot be directly displayed to a user, and the source data needs to be processed, including control inspection, format conversion, algorithm processing and the like, so that the data which can be directly displayed to the user can be obtained.

The answer is generated by colloquially speaking the processed source data and displaying the source data to the user. The data obtained after the processing of the source data is somewhat hard if directly displayed to the user. Therefore, the processed source data is subjected to the spoken language, and some reply templates can be applied in the process of the spoken language, or a character string splicing mode is adopted.

It should be noted that the intention template is a configuration and not code. When it is desired to change execution logic in the intent template, this may be accomplished by changing the configuration of the execution logic in the intent template. When the answer to a new intention question needs to be added, the configuration of execution logic in the corresponding intention template only needs to be added. To increase the flexibility of the intent templates, embodiments of the present application use a groovy script to implement the three steps described above of acquiring data, processing data, and generating answers. When a user question is assigned to a specific intent template, an answer can be generated by only dynamically executing a grovy script in the corresponding intent template.

For a clearer description of the intent templates, typical structures of the intent templates are shown in the following table 12:

table 12

In more detail, an example of a multi-turn dialog intent template intended to "plan a route from place to place" is shown in Table 13 below:

TABLE 13

/>

/>

/>

In table 13, "ID" is a template identifier for uniquely identifying a template, "Desc" is a template description for the purpose of describing a template in natural language, "arms" is an intended parameter, all parameters required for the intended template, "image_script" is a script for assigning an intended parameter, "data_retrieve_script" is a Data acquisition script, "data_process_script" is a Data processing script, "answer_generate_script" is an Answer script, "Corpus" is an example sentence group, and "assembly_rule" is a combination rule of an example sentence group.

Note that, in the field "data_retrieve_script" of table 13, a plan function of a plan variable is directly used to obtain a route. The script does not show how the planner variables are initialized. In practice the planner variable is a route planning engine and the plan function is an entry function of the route plan. When executing the script, the intention template needs to initialize the planner variable before initializing the groovy executor, the planner variable is completed in the actual engine, after the initialization is completed, the planner variable is set in the environment variable of the groovy execution engine, and then the environment variable can be directly used in the groovy script, so that the script is not wrongly reported.

The adoption of the grovy script can enable the question-answering engine of the intention template to answer any question, because the execution of the data acquisition script can acquire any data, whether the data is directly queried in a database, a knowledge graph is searched, a search engine is called, or other related systems are called, and the like. For other non-grovy scripts, complex data processing operations (which may require complex algorithms for data processing) may be implemented first in the question-and-answer engine, and then the access portal is placed in the environment variable of the grovy, so that the environment variable may be directly invoked for data processing. Therefore, the embodiment of the application adopts the oovy scripts, so that the quantity of the oovy scripts in the intention template can be greatly reduced, and the oovy scripts can be used for executing any logic operation from simple to complex.

Through the introduction of the single-round dialog intention template and the multi-round dialog intention template, the two have specific use scenes and are characterized. The single-round dialog intention template is simple, example sentences are simple, maintenance is also simple, the single-round dialog intention template is suitable for a simple scene of asking for a answer (namely the embodiment shown in fig. 7), and the single-round dialog intention template is a special form of the multi-round dialog intention template. The multi-round dialog intention template is somewhat complex but may be adapted to more scenarios (i.e. the embodiment shown in fig. 8), and the multi-round dialog intention template may communicate with the user in the form of a dialog to mine the user's potential intention, and thus answer questions according to the intention, thus being more intelligent.

After describing how the intention template is configured, the intention template contains example sentence groups corresponding to each problem possibly proposed by a user, a grovy script capable of completing execution logic of intention of each problem proposed by the user is configured in the intention template, and then the intention recognition model and the keyword extraction model in the embodiment of the application need to be trained by using the configured intention. The intention recognition model may be used to recognize the intention of a question for one question, for example, one question is "how to ask from the holy of Paris to the Lufeu", and inputting the word labeling result of the question into the awareness recognition model may obtain multiple candidate intentions of the question and the credibility of the multiple candidate intentions, where the credibility may be used to represent the likelihood of the candidate intentions, and the higher the credibility is, the greater the likelihood that the corresponding candidate intentions are real intentions of the user query. Wherein the candidate intent may be represented with a template identification ID.

In the embodiment of the application, the intention recognition model is a natural language classifier, and the natural language classifier is a machine learning model, and no matter which machine learning algorithm is adopted, the same training corpus, namely the natural language text with the mark, is required. Wherein reference is made to table 14, which schematically illustrates an example of natural language text with labels.

TABLE 14

Natural language sentence	Label (label)
		Hangzhou scenic spots	get_city_scenery
Western An has places of play	get_city_scenery
		When the holy of Paris was closed?	get_scenery_opentime
How do you get the eiffel tower from the rufigo?	plan_route_to_poi

With the large amount of training corpus shown in table 14 above, the intent recognition model can be trained using a machine learning algorithm. For example, support vector machines (SVM, support Vector Machine) are a supervised machine learning model commonly used for pattern recognition, classification, regression analysis, and the like. The process of generating the training corpus is that an example sentence combination rule in the configured intention template is executed, and the variable names of the entities in the example sentences are replaced by variable values to generate real example sentences.

The keyword extraction model is a sequence labeling model, and sequence labeling is to label types of sequences (namely, each word after word segmentation) in sentences. Therefore, when the keyword extraction model is trained, training may be performed according to the word labeling result of each real example sentence (for example, the real example sentence in table 14), where the word labeling result may include: part of speech tagging of each word, tagging of whether each word is a keyword, tagging of the part of speech of each keyword, and the like.

For example, for the real example sentence "how to remove the eiffel tower from the rufigo? The word label result may be: "how_r goes from_p luffos_ns_v eiffel pylon_ns # # plan_route_to_poi # # O B-startPoi O B-endPoi", in the labeling result, what_r is the part of speech labeling result from_p luffos_ns to_v eiffel pylon_ns "in front of the first separator # # O B-startPoi O B-endPoi", what "O B-startPoi" is the intended template ID corresponding to the actual example sentence, and what "O B-startPoi O B-endPoi" is the keyword labeling result after the second separator, what "O B-startPoi" and "c" are not keywords, and the type of the iron pylon is a keyword, the type of the part of speech is "start" and the type of the part of speech is also not keywords, and the type of "fei" is the keyword, and the type of the field "is the end". And training to obtain a keyword extraction model according to word labeling results of a large number of real example sentences, wherein the keyword extraction model is used for extracting keywords of on-line questions.

Thus, after describing the configuration process of the intent template, referring to fig. 4, which shows a flowchart of the training intent recognition model and keyword extraction model of the present application, the present embodiment may include the following steps 401 to 405:

Step 401: obtaining a variable value list of an entity, wherein the variable value list comprises: the variable name of the entity, the variable value of the entity and the corresponding relation of the entity and the variable value.

In the flow of the present embodiment, steps 401 to 404 are processes for generating a training corpus.

In the present embodiment, although example sentences are configured in the intention template, these example sentences include variable names of entities, such as "city_name", and are just a set of question methods, which are a summary of patterns of all possible question sentences. Such example sentences are not real sentences because there are variable names in example sentences. And if the variable names are replaced by real variable values, for example, the "City_name" is replaced by the "Beijing", the real example sentence can be generated. Thus, the premise of generating a real record is the collection of all possible variable values for the variable name of an entity.

For example, for which country the example sentence "$ { City_name } configured in the intention template is? "wherein the variable name is" City_name ", and the set of possible variable values is: { Hangzhou, suzhou, beijing, shanghai, london, paris, etc }. Therefore, replacing the variable value in the set of variable values with the variable name in the example sentence can generate a set of real example sentences as follows: { which country is Hangzhou? What country is the state of su? What is Beijing? What country is the Shanghai? What country is london? What is paris? Etc }, etc

In this step, for one domain, in order to obtain a list of variable values of an entity, there may be the following method: first, the type of the entity and the value of the attribute name are obtained from a knowledge base. For example, the variable name "city_name" is the type of entity "City", and the attribute name is "name", so that the knowledge base can be queried to obtain the attribute value of the attribute name "of the entity" City ", and a considerable number of variable values of the entity can be obtained. Second, from the outside. For example, crawling from an external web page, etc. Thirdly, manually maintaining. The corresponding variable values are added by themselves depending on the experience of the user. Fourth, randomly generated, but the fourth approach is suitable for some non-enumeratable variable values, e.g., the variable name for height is "$ { height }, the variable name for weight is" $ { weight }, etc. The variable values of such variable names may be generated in a random manner, and the "height" may generate random integers between 100 and 230, representing possible variable values (in centimeters).

In general, if a real example sentence that is desired to be generated is not repeated, that is, the same variable value is filled in the variable names of the example sentences, the number of variable values is generally greater than the number of generated real example sentences. Of course, if the variable values are too many, the workload of acquiring the entity's variable value list is increased, and the system performance is affected, so that it is not as good as possible. The number of variable values can be selected autonomously by a person skilled in the art when obtaining a list of variable values for an entity.

Step 402: and generating variable example sentences comprising the variable names of the entities according to the combination rules of the example sentence groups and the example sentence groups configured in the intention template.

In this step, a variable example sentence including a variable name is generated in accordance with the combination rule of the example sentence group and the example sentence group in the intention template that has been configured. The example sentences in the intention template are unstructured, and the edited example sentences in the intention template can be analyzed and stored in a certain data structure, for example, the example sentences are stored in a memory, so that the subsequent use is realized. The data structure of the storage example sentence can have a plurality of choices, and mainly can see whether the data structure is convenient to output to a subsequent processing program for use.

Note here that, because in the intention template, for the case of generating a single round form of an answer directly from a question of a user query, the example sentence group may be only one group, that is, all example sentences are numbered "1", or other default group numbers may be added. In the case of a multi-round form in which a question needs to be returned to the user and the intention of the user can be determined based on the answer of the user, a plurality of example sentence groups are provided, and each example sentence group may be numbered by "1, 2, 3 … …" or the like.

The rule of combining the example sentence group can also be obtained by reading the configured intent template. Likewise, the data structure storing the combining rule of the example sentence sets may be selected in a variety of ways, depending on whether the subsequent processing procedure is convenient to use. The combination rule of the example sentence sets is used to indicate which example sentences of the example sentence sets need to be combined together, for example, the combination rule of the example sentence sets is "1,2", which indicates that one example sentence needs to be selected from the first example sentence set and the second example sentence set respectively to generate a variable example sentence. For the intent target in the form of single round, the combination rule of the example sentence group can be directly "1", or other default rules.

Because the example sentence group and the combination rule of the example sentence group are both stored in a specific data structure, then the example sentence group and the combination rule thereof can be read to generate a certain number of real example sentences. It is assumed that the example sentence group and its combination rule have been parsed into a data structure (json format) as shown in table 15.

TABLE 15

In practical application, in order to avoid that too many real example sentences will cause the reduction of system performance, a combination rule may be preset to generate at most how many real example sentences, for example, at most 100 real example sentences.

In the process of generating the real example sentences, one example sentence can be randomly selected from each example sentence group appointed in the combination rule, and the types of the example sentence groups are carried. If the type of the example sentence group is 'intent', the real example sentence is marked as a template identification ID, and if the type of the example sentence group is a keyword slot, a mark '-slot' is added behind the template ID, which indicates that the real example sentence does not directly indicate intent, but provides an intent parameter. And then according to the example sentence group specified by the combination rule, the example sentences comprising the variable names can be spliced to form a plurality of rounds of variable example sentences.

Referring to table 16, examples of variable example sentences obtained according to the combination rules 1 and 2 are shown.

Table 16

Step 403: and respectively and correspondingly replacing variable names in the variable example sentences by variable values of all entities in the variable value list to obtain real example sentences.

Then, the variable values of the entities in the variable value list obtained in step 401 are correspondingly replaced with the variables in the variable example sentences shown in table 16, so that the real example sentences can be obtained. For example, a variable value is randomly obtained from the variable value list, and the corresponding variable name in the variable example sentence is replaced. Thus, the generated variable example sentence in the above table 16 can be subjected to step 403 to obtain an example of a real example sentence shown in table 17. In table 17, the "#" symbol is a separator, the content preceding the separator is a true example sentence, and the content following the separator is a template identification ID corresponding to the true example sentence.

TABLE 17

Step 404: marking the real example sentences to obtain training corpus; the training corpus comprises: and the template identifier corresponding to the real example sentence and each keyword type contained in the real example sentence.

The real example sentences shown in table 17 are required to be marked in the step, so that a training corpus is obtained, and the training corpus can be used for training an intention recognition model and a keyword extraction model in the embodiment of the application. The corpus obtained in the step can be directly used as input in a specific format required by a training intention recognition model and a keyword extraction model. Wherein table 18 shows an example of a format of a training corpus.

TABLE 18

Where "BOS (begin of session)" is the beginning of a set of training corpora and "EOS (end of session)" is the ending of a set of training corpora. The word segmentation and part-of-speech tagging result can be the result of word segmentation and part-of-speech tagging of a real example sentence, the sentence tagging is the template identification ID of the real example sentence, and the keyword slot tagging is the tagging of whether each word segmentation in the real example sentence is a keyword slot or not and the keyword slot type or not. In the "slot mark", an "O" may be used to indicate that the word is not a slot, a "B" may be used to indicate that the word is a slot and is the beginning of the slot, and an "I" may be used to indicate that the word is a continuation of the slot, so that a complete slot is formed from the nearest neighboring "B" before to the corresponding word of the continuously occurring "I". The tag content after "B" and "I" is the word type of the slot.

Therefore, for the real example sentences generated in the table 18, the word labeling result shown in the table 19 can be obtained through the labeling process, and the word labeling result can be directly used for training input of a machine learning algorithm.

TABLE 19

/>

In table 19, for the first training corpus "how_r goes from_p holy_ns to_v_floating_ns # # plan_route_to_poi # # O B-startPoi O B-endPoi", what goes from_p holy_ns to_v floating_ns ", i.e., the content before the first separator" # # ", is the word segmentation and part-of-speech tagging result for the real example sentence. The "plan_route_to_poi", i.e., the content between the first separator "# #" and the second separator "# #", is the template identification of the intended template. Wherein "O O B-startPoi O B-endPoi", i.e. the content after the second separator, is the result of keyword marking, wherein the first character "O" is used for indicating that the corresponding word "how" is not a keyword, the second character "O" is used for indicating that the corresponding word "from" is not a keyword, the third character "B" is used for indicating that the corresponding word "holy-salon of Paris" is a keyword, the word class is the starting point "startPoi", the fourth character "O" is used for indicating that the corresponding word "go" is not a keyword, and the fifth character "B" is used for indicating that the corresponding word "Lu floating" is a keyword, and the word class is the end point "endPoi". With this, one skilled in the art can determine the meaning of other training corpora in table 19.

The steps 401 to 404 are processes of generating a training corpus according to the example sentence group and the example sentence combination rule configured in the intent template.

Step 405: and training according to the training corpus by adopting a machine learning algorithm to obtain an intention classification model and a keyword extraction model.

With the training corpus generated in step 404, the intent classification model and keyword extraction may be trained using machine learning algorithms. Of course, in practical applications, one skilled in the art can train only one model to perform template classification and keyword slot extraction at the same time, which is a joint model. For example, the most efficient Recurrent Neural Network (RNN).

Those skilled in the art may also train only the intent recognition model, e.g., the most efficient Convolutional Neural Network (CNN) for text classification, and may employ multiple channels for input, enhancing the accuracy of model judgment. The corpus as shown in table 19 can provide a total of three different channels, one is a word vector channel, the second is a part-of-speech channel, and the third is a slot annotation channel.

Of course, the person skilled in the art can also train out an intention recognition model and a keyword extraction model at the same time, wherein the intention recognition model is used for carrying out intention recognition of a question, and the keyword extraction model is used for carrying out keyword marking and type marking on the question.

In addition, the single-round dialogue mode and the multi-round dialogue mode need different models, the single-round dialogue mode only needs to consider the judgment of the current question sentence, and only the word segmentation and the part-of-speech marking, the intention recognition and the slot extraction are carried out on the current question sentence; the intention recognition model and the keyword extraction model required by the multi-turn dialogue mode are more complex, the input of the model not only comprises the marking result of the current sentence, but also comprises the historical dialogue content (namely context) before the sentence, and the intention recognition model and the keyword extraction model are used for determining the intention of the current sentence and performing slot extraction by referring to the context. In the embodiment of the application, no matter in a single-round dialogue mode or a multi-round dialogue mode, the method can be solved by adopting a corresponding deep learning algorithm, so that the method can realize the intention recognition of the single-round dialogue mode, can realize the intention recognition and slot extraction of the single-round dialogue mode at the same time, and can realize the intention recognition and slot extraction of the multi-round dialogue mode.

In the embodiment of the application, the intention recognition model and the keyword extraction model are obtained through training at the same time as the description. After training to obtain the intention recognition model and the keyword extraction model, the intention recognition model can be adopted to determine the intention of a question, and the keyword extraction model is adopted to extract the keywords of the question, so as to obtain the marking result of whether each word is a keyword and the word type. Therefore, the embodiment of the application can be applied to the field of automatic generation of answers to questions, such as a question and answer engine. Referring to fig. 5, a scene structure diagram of the method for generating answers to questions in the present application in practical application is shown. In fig. 5, a user sends a question to a server 102 through a client 101 to ask a user for a desired answer, for example, the Hangzhou scenery includes which scenery, the server 102 may use a pre-trained keyword extraction model to extract two keywords of Hangzhou and scenery, and the word types are "city name" and "travel location", then the user may recognize the question according to the pre-trained intent recognition model to obtain that the question is a scenery for asking for Hangzhou, and then the answer to the question may be generated through the determined intent and presented to the user through the client 101. Wherein the server 102 may be implemented using a question and answer engine or the like.

Referring to fig. 6, a flowchart of an embodiment 1 of a method for generating a question answer according to the present application is shown, which is based on the scene structure diagram shown in fig. 5, and the present embodiment may be applied to the server 102 shown in fig. 5, for example, a question-answering engine, etc., and the present embodiment may include the following steps:

step 601: responding to triggering of a question, and marking the question at least according to a pre-trained keyword extraction model to obtain a word marking result, wherein the word marking result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword.

In this step, when the user sends a question to the server, for example, the question-answering engine, through the client, the server may analyze the question, and not only determine the keywords included in the question and the word types of the keywords according to the keyword extraction model, but also determine the intention of the query of the question according to the intention recognition model. When extracting keywords, firstly, word segmentation and part-of-speech tagging can be performed on a question, for example, whether the word segmentation in the question is noun or verb and the like, and then the question is tagged according to a pre-trained keyword extraction model to obtain a word tagging result, wherein the word tagging result can comprise: whether each word segment of the question is a keyword or not and the word type of each keyword.

Specifically, step 601 may include steps B1 to B3:

step B1: and segmenting the question sentence to obtain a segmentation set of the question sentence.

Firstly, performing word segmentation processing on a question sentence and performing part-of-speech tagging to obtain a word segmentation set with part-of-speech tagging.

Step B2: based on a domain dictionary corresponding to the domain to which the question belongs, keyword marking is carried out on each word in the word segmentation set to obtain a first marking result; and carrying out keyword marking on each word in the word segmentation set based on a pre-trained keyword extraction model to obtain a second marking result.

And carrying out keyword marking on each segmentation word in the segmentation word set according to a domain dictionary corresponding to the domain to which the question belongs to so as to obtain a first marking result. The domain dictionary is used for storing the corresponding relation between each domain word and the corresponding word type, or some domain words are not marked with word types, so that the first marking result obtained by marking each word based on the domain dictionary may include both the keyword and the corresponding word type, or may include only the keyword. Since a phenomenon that one domain word contains another domain word may occur in the domain dictionary, if the word types of the two domain words are different, a problem that it is impossible to determine which word type to use occurs at the time of marking. The longest priority marking algorithm may be used here in this embodiment to mark, namely: when there is a part-of-speech collision, the domain word with the longest length is preferentially marked.

For example, the domain dictionary has the following words: hangzhou, west lake, museum, and West lake museum, and the question is "a West lake museum on the side of Hangzhou West lake," the first labeling result of labeling the question based on the domain dictionary is: a seat of O/West lake museum B-scenic spots name is arranged on the side of Hangzhou_B-City-name/West lake_B. Referring to the domain words in the domain dictionary, "West lake museum" may be labeled either as "West lake museum_B-scenic spots name" or as: "West lake_B museum_B", but according to the longest preference labeling algorithm, the length of "West lake museum" is longer than both "West lake" and "museum", so is preferentially labeled "West lake museum_B-scenic spots name". It can be seen that the keywords "Hangzhou", "West lake" and "West lake museum" are all labeled as keywords, and that "Hangzhou" and "West lake museum" are also labeled with word types at the same time, while "West lake" is also not labeled with word types because the word types are not given in the domain dictionary.

As described above, labeling based on a domain dictionary is the most straightforward way to identify the keyword slots in a question, and only considering each word is a local priority method. In practical applications, a user may have an input error, etc., resulting in that the first labeling result based on the domain dictionary is blank, and the domain words in the domain dictionary may be relatively mixed, so that the unimportant word in the question sentence is labeled, which may have a relatively large influence on the answer generated finally. In this case, it is necessary to use a keyword extraction model trained in advance to simultaneously tag the question.

The method is characterized in that the keyword extraction model is used for extracting keywords and marking word types of the questions, and the structure of the questions is considered at the same time, so that whether each word is a keyword or not and the word types of the keywords are determined. For example, assume the question "how from Eiffel iron tower to holy bus of Paris? "how do it walk from $ { startPoi } to $ { endPoi? "so the following second labeling result can be made: "what from the Eiffel iron tower_B-startPoi to the_O Paris_B holy-hospital_I-endPoi goes to the_O? O'

Step B3: and determining word labeling results of each word in the word segmentation set according to the first labeling result and the second labeling result.

In this step, for the first marking result and the second marking result obtained in step B2, a final word marking result of the question may be determined. For example, for each word in the word segmentation set, it may be determined whether the first and second labeling results are both labeled as keywords, if so, the word is determined to be a keyword, and the word type of the keyword is determined according to the word type labeled in the first or second labeling results. And if the first marked result is marked as a keyword and the second marked result is marked as a non-keyword, determining the segmentation as a non-keyword. That is, if a word is considered to be a keyword based on the first labeling result of the domain dictionary and is not considered to be a keyword based on the second labeling result of the keyword extraction model, the word is not determined to be a keyword any more based on the second labeling result.

For example, the question the user sends is "help me plan a route from the holy bus to the trivia? The term marking result may be generated according to the question, and the term marking result may be in the form of a term marking matrix, including each word and whether the word is a keyword or not and a marking result of a word type. An example of a word marking matrix may be shown with reference to table 20.

Table 20

Wherein "-" in table 20 represents no labeling result.

Step 602: and determining the target intention of the question according to the word labeling result and a pre-trained intention recognition model.

In this step, the word labeling result of the question obtained in step 601 is used as input of a pre-trained intention recognition model, so as to obtain a plurality of candidate intentions of the question and the credibility corresponding to each candidate intention. For example, for the word labeling results in table 20, according to the labeling results of each keyword and word type, the intent recognition model outputs the following three candidate intents: the plan_route_to_poi, get_scene_of_city, and get_name_of_city correspond to 0.9, 0.4, and 0.3 confidence levels, respectively. It can be seen that the candidate intent is in fact the template identification ID of the configured intent template. The candidate intent "plan_route_to_poi" corresponds to an intent for planning a route, while the candidate intent "get_scene_of_city" corresponds to an intent for obtaining a sight of a city, and the candidate intent "get_name_of_city" corresponds to an intent for obtaining a name of a city, wherein the candidate intent "plan_route_to_poi" corresponds to a confidence score of 0.9 that is highest, indicating that the candidate intent "plan_route_to_poi" is most likely to be the target intent of the question, most likely reflecting the actual intent of the user question.

Step 603: obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention.

In this step, the target intentions may be respectively corresponding to the target intention templates according to the determined target intentions, and then the target answers corresponding to the target intentions may be obtained according to the method for obtaining target answers configured in the target intention templates according to the target intention templates, and returned to the client.

Specifically, the implementation process of step 603 may include steps C1 to C4:

step C1: and assigning values to the intention parameters according to the intention parameters configured in the target intention template to obtain query parameters.

In this step, since the intent parameters are configured in one intent template, the word labeling result obtained in step 601 is used to assign the intent parameters, so as to obtain the query parameters. For example, for the target intent "plan_route_to_poi", the good-looking intent parameters have been configured in the target intent template, which may include a start parameter "startPoi" and an end parameter "endPoi". The names of the starting keywords may be according to the word labeling result illustrated in table 20: the name of the terminal keyword is "Kelvin", and the starting point parameter "startPoi" and the terminal parameter "endPoi" are respectively assigned to obtain the query parameters "Kelvin" and "Kelvin".

In practical application, a script can be configured in the intent template, for example, a parameter execution script for assigning a parameter can be configured in the intent template, and then the parameter execution script configured in the target intent template can be directly operated in this step, so that the intent parameter is assigned.

It can be appreciated that, in practical applications, a groovy script may be used to implement all execution scripts in the embodiments of the present application. Before executing the script, the groovy execution engine may be initialized, and corresponding variables may be set in the groovy engine, where the variables may be directly used in the groovy script configured in the intent template without re-declaration. In this way, the number of the groovy scripts can be greatly simplified, and the functions of the groovy scripts are enhanced. For example, a java class is initialized, which is used to perform a relatively complex operation, and then an instance of the class is set in the groovy execution engine, and then the method of the class can be directly invoked in other groovy scripts.

Step C2: and acquiring the source data of the target answer according to the query parameters.

And then, acquiring source data of the template answers according to the assigned query parameters. For example, the data acquisition script configured in the target intent template may be directly executed. The data acquisition script defines how to acquire related data, executes the script, directly acquires the data, and returns according to the type of the user-defined return value.

Step C3: and processing the source data according to a preset processing mode to obtain processed source data.

In this step, the data processing script configured in the target intention template may be executed to realize the processing of the source data. The data processing script may receive a return value of the data acquisition script as input, process the data, and return a user-defined return value type.

Step C4: and generating the target answer according to the processed source data.

In this step, an answer generation script in the target intention template may be executed to realize generation of a target answer. The answer assembly can be carried out by taking the return value of the data processing script as input, or specific action is executed, the user-defined return value type is returned, the return value of the answer generation script is taken as a target answer, the target answer can be returned to the user through the client, and the user can obtain the answer corresponding to the question of the question or the execution result of the final action.

Therefore, by adopting the embodiment of the application, through the configuration of the intention template, enough training corpus can be generated, so that the keyword extraction model and the intention recognition model are trained in advance, and the keyword extraction and the intention determination of the question are more accurate. In addition, the intention template comprises an execution script for obtaining the answer, and the execution script can be modified later, so that the maintainability of the intention template is better. And, according to the intention template corresponding to the determined intention, the script can be directly executed to obtain an answer, so that the efficiency of a server such as a question and answer engine is improved. In addition, the intention template is also provided with example sentences which can answer questions, so that the user can conveniently answer questions asked by the user by using the intention template, and answers to the questions can be automatically given.

Referring to fig. 7, a flowchart of an embodiment 2 of a method for generating answers to questions in the present application is shown, and the embodiment is mainly an example of a single-round dialogue form, that is, according to an application scenario in which a question asked by a user may directly return an answer, the embodiment may include the following steps:

step 701: in response to triggering a question, marking the question according to a pre-trained keyword extraction model to obtain a word marking result, wherein the word marking result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword.

In this embodiment, the implementation of step 701 may refer to the description of step 601, which is not described herein.

Step 702: and taking the word labeling result as input of the pre-trained intention recognition model to obtain a plurality of candidate intentions and corresponding credibility.

In this step, the word labeling result is used as the input of the intention recognition model, so that a plurality of candidate intentions and corresponding credibility can be obtained. Reference may be made in particular to the description of step 602.

Step 703: and determining the candidate intention with the highest credibility among the plurality of candidate intentions as the current intention.

In the single-round dialogue form, the target intention is determined based on the credibility of each candidate intention, and first, the candidate intention with the highest credibility is taken as the current intention to make an intention decision, namely, the intention which best meets the actual situation is selected from a plurality of candidate intentions as the target intention.

Step 704: judging whether each keyword in the word labeling result is complete according to the current intention template corresponding to the current intention, if so, entering step 705, and if not, entering step 706.

For the current intention template corresponding to the current intention, all intention parameters configured in the current intention template can be obtained, and because the answer for generating a question needs the values of all intention parameters, all intention parameters configured in the current intention template can be consulted to judge whether each keyword in the word labeling result is complete, namely whether all intention parameters in the current intention template can be successfully assigned according to each keyword and the corresponding word type.

Step 705: the current intent is determined as a target intent.

If all the intent parameters of the current intent template can be assigned, then the current intent is determined to be the target intent.

Step 706: and selecting the next candidate intention as the current intention according to the order of the reliability from high to low, returning to step 704, and returning to the client side to be incapable of generating a question answer until the plurality of candidate intentions are all selected and the target intention is still not determined.

If all the intention parameters of the current intention template cannot be assigned, selecting the next candidate intention as the current intention according to the order of the credibility from high to low, and continuously judging whether all the intention parameters in the newly selected current intention can be assigned or not until all the candidate intents are selected. If all candidate intents are not selected, the answer of the question can not be generated, and further information that the answer can not be generated can be returned to the user through the client.

Step 707: obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention.

If the target intention can be determined, each script configured in the target intention template is further executed, so that a target answer of the question is obtained.

In this embodiment, whether all the intent parameters of a certain intent template can be assigned is adopted as a basis for determining the target intent, because the intent classification model may have a certain failure rate, and the candidate intent with the highest reliability may not be the actual user intent, so that the actual user intent may be determined by constraining the intent parameters after obtaining the candidate intent.

In this embodiment, all possible intents of the user may be enumerated by pre-building an ontology, and then the respective intents are stored in the form of intent templates, each of which is responsible for solving the problem of the corresponding intent. The intention template can comprise intention parameters, intention solution logic and possible question forms of the intention, aiming at the problem posed by the user, the question is classified into a specific intention through a pre-trained keyword extraction model and, then the extracted keyword is used as the parameter of the intention, and the corresponding logic in the intention template, namely, the script configured in the intention template is executed so as to obtain the answer of the problem. A large number of training corpus of the training model is generated according to the combination rule of the example sentence group and the example sentence group in the intention template, so that the keyword extraction model and the intention recognition model obtained through training can extract keywords more accurately and classify intention, and answers generated in the embodiment can be more accurate.

Referring to fig. 8, a flowchart of an embodiment 3 of a method for generating a question answer in the present application is shown, which is mainly an example of a multi-round dialogue form, that is, an application scenario in which an answer cannot be directly returned according to a question asked by a user, and a question needs to be returned to the user and a target intention can be determined according to an input of the user, where the embodiment may include the following steps:

Step 801: responding to triggering of a question, and marking the question at least according to a pre-trained keyword extraction model to obtain a word marking result, wherein the word marking result comprises the following steps: whether each word of the question is a keyword or not and the word type of each keyword.

In this embodiment, unlike step 701, for each current question that is presented by the user, the server uses the word labeling result of the current question and the keyword extraction result of the history question as the input of the keyword extraction model to extract keywords from the current question.

In the multi-turn dialogue form, for the historical dialogue content of the user, for example, a historical question, an intention recognition result of the historical question, a keyword extraction result and the like can be stored in a dialogue database, and then the dialogue content can be obtained from the dialogue database in this step. Because the history question has been subjected to keyword extraction, and the keyword extraction results are stored in the history dialogue content, i.e. the context, the keyword extraction results of the history question can be directly obtained from the dialogue database, and the keyword extraction results and the word labeling results of the current question are used as the input of the keyword extraction model to extract the keywords of the current question. It will be appreciated that the keyword extraction results are not constant and may change as the dialog content changes.

Step 802: and at least using the word labeling result as input of the pre-trained intention recognition model to obtain a plurality of candidate intentions and corresponding credibility thereof.

The step is different from step 702 in that the input of the intent recognition model in the step may include, in addition to the word labeling result of the current question set forth by the user, a keyword extraction result and an intent recognition result of the history question set forth by the user, where the keyword extraction result and the intent recognition result may be used to assist in recognizing the intent for the current question. Because the application scenario of the multi-turn dialogue form is applied, the content input by the user each time, for example, the question, also includes necessary information, for example, intent parameters, etc., so that after the word labeling is performed on the content input by the user subsequently, the intent recognition result and the keyword extraction result of the historical question are also used as the input of the intent recognition model, besides the word labeling result is used as the input of the intent recognition model, and thus a plurality of candidate intentions and corresponding credibility of the current question are obtained.

The session content can be obtained from the session database in this step. Since the history question has been subjected to keyword extraction and the keyword extraction results are stored in the history dialogue content, i.e., the context, after the candidate intention is parsed, the parameters of the candidate intention are partly derived from the keyword extraction results of the current sentence and partly derived from the context of the dialogue content. Specifically, for each current question input by the user, the word labeling result of the current question, the intention recognition result of the historical question and the keyword extraction result are taken as the input of the intention recognition model to acquire the intention recognition result of the current question. It will be appreciated that the candidate intent is not constant and may change as the dialog content changes.

Step 803: and determining the candidate intention with the highest credibility among the plurality of candidate intentions as the current intention.

In this step, the candidate intention with the highest reliability among the plurality of candidate intentions obtained in step 802 is determined as the current intention.

Step 804: judging whether the credibility of the current intention is greater than a preset credibility threshold, if so, entering step 806; if not, step 805 is entered.

In this step, it is determined whether the maximum value of the confidence level is greater than a preset confidence level threshold, for example, 0.8, and if so, the current intention with the maximum confidence level may be considered as the target intention.

Step 805: selecting a preset number of first candidate intents as intents to be selected, and sending the intents to be selected to a terminal for selection by a user.

If the maximum value of the credibility is smaller than the preset credibility threshold, the candidate intents can be ranked according to the sequence of the credibility from large to small, and a plurality of candidate intents with the preset number are selected as intents to be selected. For example, the first two candidate intents are selected as intents to be selected and sent to the client, so that the user can select an intention corresponding to the actual purpose of the user from the two intents to be selected, and further, the intention selected by the user can be used as a target intention later.

Specifically, the server may generate a selection question including the two intents to be selected and return the selection question to the user. The content of the selection question may be from the "desc" field of the intent template, e.g., the "desc" field with two intents to be selected are the content shown in table 20, respectively.

Table 20

plan_route_to_poi	Planning a route to $ { poi }
		get_poi_desc	Get an introduction to $ { poi }

The generated selection question may be: "please ask you if you want to plan a route to the trivia or get an introduction of the trivia? ".

Step 806: and determining the intention returned by the terminal and selected by the user as a target intention, or determining the current intention as the target intention.

After returning the selection question to the client and displaying the selection question to the user, the user inputs an answer according to the selection question, for example, inputs "route planned to the trivia", and then the intention selected by the user can be considered as "route planned to the trivia", so that the intention is determined as a target intention.

It can be understood that if the target intention cannot be determined by sending a selection question once, the server may send the selection question to the user again or for multiple times until the target intention of the user is determined, or if all the "desc" fields are combined and the target intention of the user cannot be determined, return the content of the undetermined intention to the client, and end the flow of generating the answer of the question; alternatively, if the user changes his original intention, in this case, the answer generation flow of the current question may be ended, and the new intention of the user may be processed and executed.

Step 807: judging whether each keyword in the word labeling result is complete according to the target intention template corresponding to the target intention, if not, entering step 808, and if so, entering step 811.

After the target intention is determined, judging whether each keyword in the word labeling result is complete or not according to all intention parameters configured in the corresponding target intention template. The specific determination process may refer to the description of step 704, and will not be described herein.

Step 808: determining intent parameters corresponding to the incomplete keywords as missing intent parameters; and generating the parameter problems related to the target intention according to the missing intention parameters and the target intention template.

In this embodiment, if the intent parameters in the target intent template cannot be assigned according to the word labeling result, the keywords in the description word labeling result are incomplete, the incomplete keywords correspond to those intent parameters that cannot be assigned, and the intent parameters are determined to be missing intent parameters.

Step 809: returning the parameter questions to the client for display to the user, the intent templates comprising: correspondence between the intent parameters and parameter questions.

Further, according to the missing intention parameters and the parameter questions corresponding to the missing intention parameters in the target intention template, generating inquiry questions and sending the inquiry questions to the client for display to the user.

The generation of the parameter questions is from an intention template, in which each intention parameter is set with questions to be queried if the intention parameter is missing when the intention parameter is defined, so that a question is randomly selected from a parameter question list corresponding to the missing parameter and returned to a user.

Step 810: and acquiring the missing intention parameters according to the parameter answers returned by the client and input by the user, and entering a step 811.

After the parameter question is generated and the user is inquired, the user possibly gives corresponding intention parameters in the next dialog, the missing intention parameters can be obtained from the parameter answers input by the user, and in this case, all the intention parameters of the target intention template can be assigned. Alternatively, if the user changes intent, the server may also stop asking for the intent parameters for which the current intent is missing.

Step 811: obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention.

In this step, if all the intent parameters of the target intent template can be assigned, then a script pre-configured in the target intent template can be executed to generate a target answer of the target intent, and the specific procedure can be shown in step 603, which is not described herein.

It should be noted that, in the multi-round dialogue form, the integrity judgment of the target intention and the intention parameter of the target intention is actually an inference, that is, the actual intention of the user is inferred according to the content provided by the user. Unlike the single-round dialog form, the present embodiment determines the target intention not based on the intention parameter priority matching but based on the principle of confidence priority. Because the user would express intent and intent parameters as fully as possible in a question in a single-round dialog form, while in a multi-round dialog form the user may tend to express at will, i.e., a question or a sentence may have only intent expressed therein, but without any parameters, such as "can help me plan a route? For the question, if the candidate intentions are matched according to the principle of preferential matching of the intention parameters, the candidate intentions are not matched, and the user does not provide any intention parameters, but the user clearly expresses the intentions, so that the intention of the user can be more accurately discovered by adopting a mode of preferential credibility in a multi-round dialogue form.

In this embodiment, in order to conveniently mine the potential intention of the user, to make reasoning about the true intention of the user, the embodiment may further include:

After the server generates a question and returns the question to the client and presents the question to the user, the user may input his answer according to the question, or the user may select his intention according to the question, so that the dialog contents input by the user may be temporarily saved in the dialog database. Alternatively, the data may be buffered in the memory. The intent selected by the user can be obtained from the session database or memory later, or the intent parameters can be queried from the session content.

Further, after the dialogue content is stored in the dialogue database or the memory, the dialogue content in the dialogue database can be updated, or when a preset time period is reached, the dialogue content input by the user is deleted. Because in the form of multiple rounds of conversations, conversations between the user and a server, such as a question and answer engine, are involved, it is necessary to save the historical conversational content of the user as well as the involved intents, intent parameters, etc. It will be appreciated that these dialog contents are not constant over time, and that as the dialog contents of the user and the server advance, the original intent or the value of the intent parameter will be covered by the new value. Also, the dialog content is not permanently present, for example a time threshold may be set, which may be deleted when reached. Alternatively, when a new dialogue content is generated by a user sending a new question, the previous dialogue content cannot influence the current dialogue, and therefore, the previous dialogue content can be deleted.

It can be seen that, in this embodiment, not only the question sentence of the user that is firstly asked is used as the input of the keyword extraction model or the consciousness recognition model, but also the question is continuously asked to the user so that the user can select his own intention or supplement the intention parameter according to the question, in the case that the intention of the user cannot be determined or the intention of the user can be determined to be incomplete. Therefore, the reasonable reasoning in this embodiment may enable the intention of the user to be clarified and complete intention parameters to be obtained, so as to realize a dialogue with the user and automatically generate the answer required by the user according to the dialogue content, thereby not only improving the accuracy of answer generation, but also enabling the generated answer to be more in line with the habit of the user, and very friendly to interact with the dialogue of the user, and the user can communicate with a server such as a question-answering engine directly through the dialogue in daily life without learning a special interaction mode.

For the purposes of simplicity of explanation, the foregoing method embodiments are depicted as a series of acts, but it will be appreciated by those skilled in the art that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently in accordance with the application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.

Corresponding to the method provided in the foregoing embodiment 1 of the method for generating a question answer of the present application, referring to fig. 9, the present application further provides an embodiment of a generating device for a question answer, where in this embodiment, the generating device may be integrated on a server side, for example, a question-answering engine, and the generating device may include:

the keyword labeling unit 901 is configured to respond to triggering a question, label the question at least according to a pre-trained keyword extraction model to obtain a word labeling result, where the word labeling result includes: whether each word of the question is a keyword or not and the word type of each keyword.

The keyword labeling unit 901 may include:

the word segmentation subunit is used for segmenting the question to obtain a word segmentation set of the question; the first marking subunit is used for marking keywords of each word in the word segmentation set based on a domain dictionary corresponding to the domain to which the question belongs to obtain a first marking result; the second marking subunit is used for marking the keywords of each word in the word segmentation set based on a pre-trained keyword extraction model to obtain a second marking result; the domain dictionary is used for storing the corresponding relation between each domain word and the corresponding word type; and the first determining subunit is used for determining word labeling results of each word in the word segmentation set according to the first marking result and the second marking result.

Wherein the domain dictionary may be generated by: acquiring each attribute of each entity in the field; and taking the attribute value corresponding to each attribute as a keyword, taking the entity tag and the attribute name as word types of the keywords, and determining synonyms, hyponyms and aliases of the keywords as keywords.

Wherein the first determining subunit may include:

the first judging subunit is used for judging whether the first marking result and the second marking result are marked as keywords or not according to each word in the word segmentation set; the second determining subunit is configured to determine the word segmentation as a keyword if the first judging subunit is yes, and determine a word type of the keyword according to the first marking result or the second marking result; and a third determining subunit configured to determine the word as a non-keyword when the result of the first judging subunit is that the first label result is labeled as a keyword and the second label result is labeled as a non-keyword.

The intention determining unit 902 is configured to determine a target intention of the question according to the word labeling result and a pre-trained intention recognition model.

The answer obtaining unit 903 is configured to obtain a target answer corresponding to the target intention according to a target intention template configured in advance and corresponding to the target intention.

The answer obtaining unit 903 may include:

the assignment subunit is used for assigning values for the intention parameters according to the intention parameters configured in the target intention template to obtain query parameters; the source data acquisition subunit is used for acquiring the source data of the target answer according to the query parameters; the processing subunit is used for processing the source data according to a preset processing mode to obtain processed source data; and a generation subunit, configured to generate the target answer according to the processed source data.

Wherein the intent template may be generated by:

determining the intention of all questions in each field according to the relation among the entities represented by the ontology in each field; corresponding intention templates are configured for each intention determined. Wherein the intent is for an interrogation purpose that expresses a question, the intent comprising: querying the attributes of the entities, querying all relationships between the entities, querying other or all attributes of the entities by one or more attributes of the entities, and/or querying relationships of the entities by one or more attributes of the entities; the intent template includes: template identification, template description, intention parameters, parameter problems corresponding to the intention parameters, example sentence groups corresponding to the intention, and combination rules of the example sentence groups; the template identifier is used for uniquely representing an intention template, and the template description is used for describing the purpose of the intention template; the intention parameter is a parameter corresponding to determining an intention, and the parameter problem is as follows: when the corresponding intention parameter is missing, the problem needs to be sent to the user, the example sentences in the example sentence groups comprise variable names of entities and are used for representing the problem which can be solved by the intention template, and the combination rule of each example sentence group is used for representing: and combining the example sentences in different example sentence groups.

Wherein the intent recognition model and the keyword extraction model may be trained by:

generating training corpus according to example sentence groups and example sentence combination rules configured in the intention template; and training according to the training corpus by adopting a machine learning algorithm to obtain an intention classification model and a keyword extraction model.

Wherein, the training corpus can be generated by the following ways:

obtaining a variable value list of an entity, wherein the variable value list comprises: the variable name of the entity, the variable value of the entity and the corresponding relation of the entity and the variable value of the entity; generating variable example sentences comprising the variable names of the entities according to the combination rules of the example sentence groups and the example sentence groups configured in the intention template; the variable values of the entities in the variable value list are respectively and correspondingly replaced with variable names in the variable example sentences to obtain real example sentences; marking the real example sentences to obtain training corpus; the training corpus comprises: and the template identifier corresponding to the real example sentence and each keyword type contained in the real example sentence.

Therefore, by adopting the generating device of the embodiment of the application, enough training corpus can be generated in advance through the configuration of the intention template, so that the keyword extraction model and the intention recognition model are trained in advance, and the keyword extraction and the intention determination of the question are more accurate. In addition, the intention template comprises an execution script for obtaining the answer, and the execution script can be modified later, so that the maintainability of the intention template is better. And, according to the intention template corresponding to the determined intention, the script can be directly executed to obtain an answer, so that the efficiency of a server such as a question and answer engine is improved. In addition, the intention template is also provided with example sentences which can answer questions, so that the user can conveniently answer questions asked by the user by using the intention template, and answers to the questions can be automatically given.

In correspondence to the method provided in the foregoing embodiment 2 of the method for generating a question answer of the present application, referring to fig. 10, the present application further provides an embodiment 2 of a generating device for a question answer, in this embodiment, the generating device may be integrated on a server side, for example, a question-answering engine, etc., the embodiment may generate an answer about a question of a user in a single-round dialogue form, and the intention determining unit 902 in the generating device may include:

the first model processing subunit 1001 is configured to use the word labeling result as an input of the pre-trained intent recognition model to obtain a plurality of candidate intentions and corresponding credibility.

A fourth determining subunit 1002, configured to determine, as the current intention, a candidate intention with the highest reliability among the plurality of candidate intents.

A second judging subunit 1003, configured to judge whether each keyword in the word labeling result is complete according to the current intention template corresponding to the current intention; the intent template includes: all intent parameters required for an intent are determined.

A fifth determining subunit 1004, configured to determine the current intention as a target intention if the second judging subunit has a yes result.

A selecting subunit 1005, configured to select, if the result of the second determining subunit is negative, the next candidate intention as the current intention in order of high reliability.

And a triggering subunit 1006, configured to trigger the second determining subunit after the selecting subunit selects the next candidate intention, until the candidate intents are all selected.

The second judging subunit may specifically include:

a parameter obtaining subunit, configured to obtain all intention parameters configured in the current intention template; a value assignment subunit, configured to assign values to the all the intent parameters by using word types of the keywords in the word labeling result; and the assignment judging subunit is used for judging whether the all intention parameters can be successfully assigned.

By adopting the generating device of the embodiment, all possible intents of the user can be enumerated by pre-establishing the ontology, and then the intents are stored in the form of intent templates, and each intent template is responsible for solving the problem of the corresponding intent. The intention template can comprise intention parameters, intention solution logic and possible question forms of the intention, aiming at the problem posed by the user, the question is classified into a specific intention through a pre-trained keyword extraction model and, then the extracted keyword is used as the parameter of the intention, and the corresponding logic in the intention template, namely, the script configured in the intention template is executed so as to obtain the answer of the problem. A large number of training corpus of the training model is generated according to the combination rule of the example sentence group and the example sentence group in the intention template, so that the keyword extraction model and the intention recognition model obtained through training can extract keywords more accurately and classify intention, and answers generated in the embodiment can be more accurate.

In correspondence to the method provided in the foregoing embodiment 3 of the method for generating a question answer of the present application, referring to fig. 11, the present application further provides an embodiment 3 of a generating device for a question answer, in this embodiment, the generating device may be integrated on a server side, for example, a question-answering engine, etc., the embodiment may generate an answer about a question of a user in a single-round dialogue form, and the intention determining unit 902 in the generating device may include:

the second model processing subunit 1101 is configured to obtain a plurality of candidate intents and corresponding credibility by using at least the word labeling result as an input of the pre-trained intent recognition model.

A sixth determining subunit 1102 is configured to determine, as the current intention, a candidate intention with the highest reliability among the plurality of candidate intents.

A third determining subunit 1103 is configured to determine whether the credibility of the current intention is greater than a preset credibility threshold.

A seventh determining subunit 1104, configured to determine the current intention as the target intention if the result of the third determining subunit is greater than the target intention.

A selecting subunit 1105, configured to select a preset number of first candidate intents as the intents to be selected if the result of the third judging subunit is no.

And the sending subunit 1106 is configured to send the intent to be selected to a terminal for selection by a user.

An eighth determination subunit 1107 is configured to determine, as a target intention, the intention selected by the user and returned by the terminal.

The fourth judging subunit 1108 is configured to judge whether each keyword in the word labeling result is complete according to the target intention template corresponding to the target intention.

A ninth determining subunit 1109, configured to determine, if the result of the fourth determining subunit is negative, an intent parameter corresponding to the incomplete keyword as a missing intent parameter.

A question generation subunit 1110, configured to generate the parameter question related to the target intention according to the missing intention parameter and the target intention template.

A question return sub-unit 1111 for returning the parameter questions to the client for display to the user; the intent template includes: correspondence between the intent parameters and parameter questions.

A parameter obtaining subunit 1112, configured to obtain the missing intent parameter according to the parameter answer returned by the client and input by the user.

Wherein the intention determining unit 902 may further include:

A storing subunit, configured to store dialogue content input by a user into a session database, where the dialogue content includes: the user selected intent and/or the user entered parameter answer.

Wherein the intention determining unit 902 may further include:

and the updating subunit is used for updating the dialogue content in the dialogue database or deleting the dialogue content input by the user when a preset time period is reached.

The second model processing subunit 1101 may specifically include:

a dialogue acquisition subunit for acquiring the dialogue content from a dialogue database; and the input subunit is used for inputting the word labeling result and the dialogue content into a pre-trained intention recognition model to obtain a plurality of candidate intentions and the corresponding credibility thereof.

The fourth determining subunit 1108 may specifically include:

a parameter acquisition subunit, configured to acquire all intention parameters configured in the target intention template; and a valuation subunit, configured to perform valuation on all the intent parameters in the target intent template by using the word types of the keywords in the word labeling result, and a valuation judgment subunit, configured to judge whether the valuation of all the intent parameters is successful.

By adopting the generating device in the embodiment, not only the question sentence which is firstly questioned by the user is used as the input of the keyword extraction model or the consciousness recognition model, but also the question is continuously questioned to the user so that the user can select the intention or supplement the intention parameter according to the question under the condition that the intention of the user cannot be determined or the intention of the user can be determined to be incomplete. Therefore, the reasonable reasoning in this embodiment may enable the intention of the user to be clarified and complete intention parameters to be obtained, so as to realize a dialogue with the user and automatically generate the answer required by the user according to the dialogue content, thereby not only improving the accuracy of answer generation, but also enabling the generated answer to be more in line with the habit of the user, and very friendly to interact with the dialogue of the user, and the user can communicate with a server such as a question-answering engine directly through the dialogue in daily life without learning a special interaction mode.

It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is provided in detail for a method, a device and a server for generating a natural language question answer, and specific examples are applied to explain the principles and embodiments of the present application, and the description of the above examples is only used to help understand the method and core ideas of the present application; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. A method for generating natural language question answers is characterized in that the method comprises the following steps:

obtaining a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention,

2. The method of claim 1, wherein determining word labeling results for each word in the set of words according to the first labeling result and the second labeling result comprises:

3. The method of claim 1, wherein the domain dictionary is generated by:

acquiring each attribute of each entity in the field;

4. The method of claim 1, wherein said determining the target intent of the question based on the word labeling result and a pre-trained intent recognition model comprises:

if so, determining the current intention as a target intention;

5. The method of claim 4, wherein the determining whether each keyword in the word labeling result is complete according to the current intent template corresponding to the current intent comprises:

6. The method of claim 1, wherein said determining the target intent of the question based on the word labeling result and a pre-trained intent recognition model comprises:

7. The method as recited in claim 6, further comprising:

8. The method as recited in claim 7, further comprising:

updating the dialogue content in the dialogue database,

9. The method of claim 7, wherein the obtaining a plurality of candidate intents and their corresponding trustworthiness using at least the word labeling results as input to the pre-trained intent recognition model comprises:

acquiring the dialogue content from a dialogue database;

10. The method of claim 6, wherein the determining whether each keyword in the word labeling result is complete according to the target intention template corresponding to the target intention comprises:

Acquiring all intention parameters configured in the target intention template;

11. The method of claim 1, wherein the intent template is generated by:

12. The method of claim 11, wherein the intent recognition model and keyword extraction model are trained by:

13. The method of claim 12, wherein the generating a training corpus according to the example sentence sets and the example sentence combination rules configured in the intent template comprises:

14. The method of claim 1, wherein obtaining a target answer corresponding to the target intent from a pre-configured target intent template corresponding to the target intent comprises:

acquiring source data of the target answer according to the query parameters;

and generating the target answer according to the processed source data.

15. A device for generating answers to questions in natural language, wherein the device is integrated on a server, and the device comprises:

An answer acquisition unit for acquiring a target answer corresponding to the target intention according to a pre-configured target intention template corresponding to the target intention,

wherein, the keyword labeling unit includes:

16. The apparatus of claim 15, wherein the first determination subunit comprises:

17. The apparatus according to claim 15, wherein the intention determining unit comprises:

18. The apparatus according to claim 15, wherein the intention determining unit comprises:

19. The apparatus according to claim 15, wherein the answer acquisition unit includes:

20. A server, the server comprising: the device of any one of claims 15 to 19.