CN114153955B - Construction method of multi-skill task type dialogue system fusing chatting and common knowledge - Google Patents

Construction method of multi-skill task type dialogue system fusing chatting and common knowledge Download PDF

Info

Publication number
CN114153955B
CN114153955B CN202111334457.2A CN202111334457A CN114153955B CN 114153955 B CN114153955 B CN 114153955B CN 202111334457 A CN202111334457 A CN 202111334457A CN 114153955 B CN114153955 B CN 114153955B
Authority
CN
China
Prior art keywords
chatting
entity
final
degree
dialogue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111334457.2A
Other languages
Chinese (zh)
Other versions
CN114153955A (en
Inventor
陈楷
熊京萍
廖奇
王辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kexun Jialian Information Technology Co ltd
Original Assignee
Kexun Jialian Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kexun Jialian Information Technology Co ltd filed Critical Kexun Jialian Information Technology Co ltd
Priority to CN202111334457.2A priority Critical patent/CN114153955B/en
Publication of CN114153955A publication Critical patent/CN114153955A/en
Application granted granted Critical
Publication of CN114153955B publication Critical patent/CN114153955B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention relates to a dialogue system, in particular to a multi-skill task type dialogue system construction method fusing chatting and common knowledge, which comprises the steps of collecting large-scale open-source Chinese chatting linguistic data, taking adjacent chats as chatting linguistic data pairs, constructing a chatting model, collecting common sense question-answer linguistic data and triple data in various large-scale fields, constructing a knowledge graph, scoring a model based on question-answer linguistic data training similarity, processing an original problem to obtain a candidate entity common sense set, obtaining all out-degree and in-degree relations in related two-hop relations from the knowledge graph, obtaining a final candidate entity, screening out a relation path by using the final candidate entity, preferably selecting the entity relation path to obtain a final path, inquiring a common sense answer from the knowledge graph based on the final path, and completing construction of a common sense dialogue model; the technical scheme provided by the invention can effectively overcome the defects that the prior art cannot have the common sense reasoning capability and the chatting function and has lower conversation reply recall accuracy.

Description

Construction method of multi-skill task type dialogue system fusing chatting and common knowledge
Technical Field
The invention relates to a dialogue system, in particular to a construction method of a multi-skill task type dialogue system fusing chatting and common knowledge.
Background
The dialog system may be classified into a chatting type dialog system, a question-and-answer type dialog system, and a task type dialog system according to task types.
The chatting dialogue system mainly performs emotion interactive communication with a user to help the user relieve worries and stuffiness, generally utilizes a large amount of chatting linguistic data to train data, for example, dialoDG, T5 and the like adopt a generative model, so that the model has the capability of generating corresponding output according to input, but the generated result is random, and the whole process is difficult to evaluate and control. The chatty type dialogue system does not have the common sense reasoning ability and can not provide vertical tasks in a specific field.
The question-answering type dialogue system is also called an FAQ question-answering system, and is mainly used for providing vertical consulting services, such as government affair consultation and legal consultation, for the user. The current mainstream question-answer dialogue systems are all based on a retrieval scheme, namely an ES question-answer pair library is constructed in advance according to questions and answers, when a user asks questions, the questions in the library are retrieved, topN candidate questions most similar to the user questions are returned, then the most similar standard questions are selected by using a grading rearrangement mechanism, and the answers corresponding to the standard questions are returned as answers to the questions. At present, a common FAQ question-answering system fully utilizes text and semantic feature vectors by constructing a large number of similar questions or high-quality knowledge maps and adopts a recall and scoring strategy to ensure controllability and precision, but the question-answering type dialogue system does not have the common knowledge reasoning capability and cannot support the chatting function.
The task type dialogue system, also called an intelligent customer service system, has wide application prospects in various fields of logistics, finance, insurance, manufacturing, e-commerce and the like, can replace part of manual customer service to provide 24h service for users, greatly reduces the pressure of the manual customer service, and reduces the enterprise cost. At present, mainstream task type conversation systems in the market are realized in a pipeline mode, the whole system is divided into a plurality of modules, for example, a conversation understanding NLU module mainly identifies user intentions and extracts key slot positions in conversations, a conversation tracking DST module mainly records conversation states and slot positions, a conversation strategy DPL module mainly decides what response should be made to a user in the conversation according to current input and historical records, and a conversation generation NLG module mainly generates contents of final responses to the user. In view of the complexity of the task-based dialog system and the high standard requirement for task completion, the system does not have the common sense reasoning capability and cannot well support the chatting function.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects in the prior art, the invention provides the construction method of the multi-skill task type dialogue system integrating chatting and common knowledge, which can effectively overcome the defects that the prior art cannot combine the function of common sense reasoning and chatting and has lower dialogue reply recall accuracy.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
a multi-skill task type dialogue system construction method fusing chatting and common knowledge comprises the following steps:
s1, collecting large-scale open-source Chinese chatting linguistic data, taking adjacent dialogues as chatting linguistic data pairs, and constructing a chatting model;
s2, collecting the common sense question and answer corpus and the triple data of each large-scale field, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus;
s3, processing the original problem to obtain a candidate entity set, acquiring all out-degree and in-degree relations in the relevant two-hop relation from the knowledge graph, and obtaining a final candidate entity;
s4, screening out entity relation paths by using the final candidate entities, optimizing the entity relation paths to obtain final paths, and inquiring common sense answers from the knowledge graph based on the final paths to complete construction of a common sense dialogue model;
s5, analyzing the task dialogue corpus, abstracting out fixed body data, storing the fixed body data into a database, and constructing a task dialogue model at a session level;
and S6, generating a topic discrimination model by utilizing common sense reasoning and task dialogue corpus training.
Preferably, the step S4 of screening out the entity relationship path by using the final candidate entity, and performing optimization on the entity relationship path to obtain a final path, and querying a common sense answer from the knowledge graph based on the final path includes:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
Preferably, the step of splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relation into a complete sentence includes:
all final candidate entities correspond to a spliced complete sentence which comprises the following steps:
Figure BDA0003350018620000031
wherein the content of the first and second substances,
Figure BDA0003350018620000032
sentences composed of all relations in representative degree and the ith final candidate entity, I i Is the final candidate entity of the ith input, R r1 Being an in-degree relationship, T, within a two-hop relationship r1 Is an in-degree entity within a two-hop relationship,
Figure BDA0003350018620000033
represents all relations in the degree and the sentence composed by the ith final candidate entity, R r2 For out-of-degree relations within a two-hop relation, T r2 And M is the number of final candidate entities.
Preferably, the calculating the similarity between the original problem and each complete sentence by using the similarity scoring model and screening out the entity relationship path includes:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, outputting the 3 x N vector after passing through two full-connection layers, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
Preferably, in S2, collecting the general knowledge question and answer corpus and the triple data in each large-scale field, constructing a knowledge graph, and training a similarity scoring model based on the general knowledge question and answer corpus, including:
the method comprises the steps of expanding diversity of the common sense question-answer corpus by utilizing triple data, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge map, and training an SimCSE-RoBerta similarity scoring model based on the common sense question-answer corpus.
Preferably, the scoring each entity relationship path and the original question, selecting the entity relationship path with the highest score as the final path, and querying a common sense answer from the knowledge graph based on the final path includes:
calculating the total score source between the ith entity relationship path and the original problem by adopting the following formula i ：
source i ＝L i +D i +R i +C i
Wherein L is i The intersection length, D, obtained after the duplication of the ith entity relationship path and the original problem is removed on all word granularities i Is the inverse of the length of the ith entity relationship path, R i Is the reciprocal of the hop count of the ith entity relationship path, C i The frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score source i And taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query the common sense answers from the knowledge graph.
Preferably, the task dialog model is built at a session level in S5, including:
the method for performing fine adjustment on the dialog neural network by combining the words, the belief states, the database results, the system actions and the system replies of the users in each dialogue turn into a dialogue sequence comprises the following steps:
in the first dialog turn, the user inputs the utterance as U 0 According to the speech U 0 The generated belief state is B 0 Belief state B 0 For database retrieval to retrieve satisfaction of belief state B 0 The number of entities under the constraint and the search result are D 0 According to { U 0 ，B 0 ，D 0 Generation System action A 0 And system reply R 0 ；
At the tth dialog turn, based on the user utterance U t And all previously generated outputs U 0 ，B 0 ，D 0 ，A 0 ，R 0 ，…，U t-1 ，B t-1 ，D t-1 ，A t-1 ，R t-1 ，U t Is multiplied by the sum of the coefficients to generate B t 、A t And R t 。
Preferably, the processing the original problem in S3 to obtain a candidate entity set includes:
and performing word segmentation and part-of-speech extraction on the original problem through a lac tool, reserving words and phrases with the part-of-speech, removing stop words, and respectively obtaining substrings of the words and phrases with the part-of-speech from 2 to the original problem length to obtain a candidate entity set.
Preferably, in S3, obtaining all the out-degree and in-degree relations in the relevant two-hop relation from the knowledge graph, and obtaining a final candidate entity, includes:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount count qr The similarity s between the original problem and each candidate entity is calculated using the following formula qe ：
Figure BDA0003350018620000051
The similarity s between the original problem and each relation in the two-hop relation is calculated by using the following formula qc ：
Figure BDA0003350018620000052
Wherein S is q For the original set of problems, S e As a set of candidate entities, S c Is a set of relationships;
calculating the relation data amount count using the following equation qr The similarity s between the original question and each candidate entity qe Similarity s of original problem and each relation in two-hop relation qc Weighted score candidate between s ：
candidate s ＝count qr *W qr +s qr *W qe +s qc *W qc
Wherein, W qr 、W qe 、W qc Are all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
Preferably, the method for constructing the Chinese chatting model includes the steps of collecting Chinese chatting corpus of a large-scale open source in S1, taking adjacent chatting as a chatting corpus pair, and constructing the chatting model, wherein the steps include:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by utilizing a Dialdo DG neural network.
(III) advantageous effects
Compared with the prior art, the construction method of the multi-skill task type dialogue system fusing the chatting and the common knowledge, provided by the invention, has the following beneficial effects:
1) In the construction process of a common sense dialogue model, a unique candidate entity extraction strategy is designed, an additional entity link model is not needed, the cost of manual labeling and model training is reduced, feature vectors are extracted by using an SimCSE-RoBerta similarity scoring model at an entity relation path stage for differential fusion, feature information is enhanced, and recall scoring is carried out by combining with the original problem when a final path is optimized, so that the recall accuracy is effectively improved;
2) In the construction process of the task dialogue model, a pipeline multi-module scheme is abandoned, a unified data input format is constructed, mutual information of dialogue texts and system responses is improved, and model training is carried out in a DialoDG end-to-end mode.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a schematic flow diagram of the present invention;
FIG. 2 is a schematic flow chart of the method for constructing a chat model according to the present invention;
FIG. 3 is a schematic flow chart of the construction of a common sense dialogue model in the present invention;
FIG. 4 is a schematic flow chart of the task dialogue model construction method of the present invention;
fig. 5 is a working diagram of the dialog neural network in the process of constructing the task dialogue model in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A multi-skill task type dialogue system construction method fusing chat and common knowledge is disclosed, as shown in FIG. 2, collecting large-scale open-source Chinese chat linguistic data, taking adjacent dialogues as chat linguistic data pairs, and constructing a chat model, and specifically comprises the following steps:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by using a DiaoloDG neural network.
As shown in fig. 3, collecting the common sense question and answer corpus and the triple data in various large-scale fields, constructing a knowledge graph, and training a similarity scoring model based on the common sense question and answer corpus, specifically including:
expanding the diversity of the common sense question and answer corpus by using 1.4 hundred million Chinese entity triple data sourced from ownthink, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge map, and training a SimCSE-RoBerta similarity scoring model based on the common sense question and answer corpus.
As shown in fig. 3, the original problem is processed to obtain a candidate entity set, all the relationships between out-degree and in-degree in the relevant two-hop relationship are obtained from the knowledge graph, and a final candidate entity is obtained.
(1) Processing the original problem to obtain a candidate entity set, including:
and performing word segmentation and part-of-speech extraction on the original problem through a lac tool, reserving words and phrases with the part-of-speech, removing stop words, and respectively obtaining substrings of the words and phrases with the part-of-speech from 2 to the original problem length to obtain a candidate entity set.
(2) Obtaining all the relationships of out-degree and in-degree in the related two-hop relationship from the knowledge graph, and obtaining a final candidate entity, wherein the steps comprise:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount count qr The similarity s between the original problem and each candidate entity is calculated using the following formula qe ：
Figure BDA0003350018620000081
The similarity s between the original problem and each relation in the two-hop relation is calculated by adopting the following formula qc ：
Figure BDA0003350018620000082
Wherein S is q For the original set of problems, S e As a set of candidate entities S c Is a set of relationships;
calculating the relation data amount count using the following equation qr The similarity s between the original question and each candidate entity qe Similarity s of original problem and each relation in two-hop relation qc Weighted score candidate between s ：
candidate s ＝count qr *W qr +s qr *W qe +s qc *W qc
Wherein, W qr 、W qe 、W qc Are all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
As shown in fig. 3, the entity relationship path is screened out by using the final candidate entity, the entity relationship path is optimized to obtain a final path, and a common sense answer is queried from the knowledge graph based on the final path to complete the construction of the common sense dialogue model, which specifically includes:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original question, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
(1) And splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relations respectively to form a complete sentence, wherein the steps comprise:
all final candidate entities correspond to the complete sentence spliced by:
Figure BDA0003350018620000083
/>
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003350018620000091
represents all relationships in the degree of entry and a sentence composed of the ith final candidate entity, I i Is the final candidate entity of the ith input, R r1 Is an in-degree relationship, T, within a two-hop relationship r1 Is an in-degree entity within a two-hop relationship,
Figure BDA0003350018620000092
represents all relations in the degree and the sentence composed by the ith final candidate entity, R r2 In a two-hop relationship, T r2 And M is the number of final candidate entities.
(2) Respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening an entity relationship path, wherein the similarity scoring model comprises the following steps:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, outputting the 3 x N vector after passing through two full-connection layers, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
(3) Scoring each entity relationship path and the original question, selecting the entity relationship path with the highest score as a final path, and inquiring a common sense answer from the knowledge graph based on the final path, wherein the method comprises the following steps:
calculating the total score source between the ith entity relationship path and the original problem by adopting the following formula i ：
source i ＝L i +D i +R i +C i
Wherein L is i The intersection length, D, obtained after the duplication of the ith entity relationship path and the original problem is removed on all word granularities i Is the inverse of the length of the ith entity relationship path, R i Is the reciprocal of the hop count of the ith entity relationship path, C i The frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score source i And taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query common sense answers from the knowledge graph.
As shown in fig. 4, the task dialogue corpus is analyzed, the fixed ontology data is abstracted and stored in the database, and the task dialogue model is constructed at the session level.
(1) Analyzing the task dialogue corpus, abstracting out fixed ontology data and storing the fixed ontology data into a database, wherein the method comprises the following steps:
take restaurant information as an example, [ { "pname": anhui province, "city": compost city, "adname": "address" in the mountainous region of hollyhock: innovation Daotai No. 2800 in Sichuan province, "location": [117.26104,31.85117], "name": old and rural chicken innovation industry park shop, "quality":5, "license range": cheap } ], generally if a third party has more accurate external information, the third party can be butted in the form of an HTTP interface, then the diversity of language expressions in linguistic data from different sources is removed, mutual information between a dialogue semantic structure and response is fully utilized, all data are differentiated, specific slot positions in dialogue data are replaced in a uniform alias mode, and the words, the belief states, database results (also can be results of the external interface), system actions and system replies of a user are all surrounded in a specific coding format as input;
the specific data format of the conventional scheme is as follows:
user i want to see a cheap, good and good restaurant near the lake park of Sichuan
[ Slot ] [ Shuxi lake park, cheap, good public praise ]
[ System ] the old and rural chicken innovation industry park store is good, and the popular comment is a 5-star good comment
User is good, help me order today's 4-person compartment
The specific data format in the invention is as follows:
< sos _ u > ] I want to see the restaurant at the privacy quality around address [ < eos _ u > ]
[<sos_a>][address][pricerange][quality][restaurant][<eos_u>]
[ < sos _ r > ] name is good, and popular is quality. [ < eos _ r > ]
[sos_db][tb_restaurant][address][pricerange][quality][eos_db]
Good [ < sos _ u > ] person's compartment [ < eos _ u > ] helping me to schedule time
[<sos_a>][booking][address][pricerange][quality][time][person][<eos_u>]
[ < sos _ r > ] good, has succeeded in booking you [ old and rural chicken innovation industry park store ] [ today ] people's booth [ 4 ]. Ask what can help you? [ < eos _ r > ]
(2) Constructing a task dialog model at the session level, as shown in FIG. 5, includes:
the method for performing fine adjustment on the dialog neural network by combining the words, the belief states, the database results, the system actions and the system replies of the users in each dialogue turn into a dialogue sequence comprises the following steps:
in the first dialog turn, the user inputs the utterance as U 0 According to the speech U 0 The generated belief state is B 0 Belief state B 0 For database retrieval to retrieve satisfaction of belief state B 0 The number of entities under the constraint and the search result are D 0 According to { U 0 ，B 0 ，D 0 Generate system actions A 0 And system reply R 0 ；
At the tth dialog turn, based on the user utterance U t And all previously generated outputs { U } 0 ，B 0 ，D 0 ，A 0 ，R 0 ，…，U t-1 ，B t-1 ，D t-1 ，A t-1 ，R t-1 ，U t Is multiplied by the sum of the coefficients to generate B t 、A t And R t 。
And generating a topic discrimination model by using common sense reasoning and task dialogue corpus training. The method comprises the following steps that a user inputs a topic type judgment through a topic judgment model, and if a result output by the topic judgment model is a common sense type, a common sense conversation model is called; if the result output by the topic judging model is the task type, calling a task dialogue model; and if the topic discrimination model judges that the input does not belong to the common sense type or the task type, calling the chatting model.
The above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (8)

1. A multi-skill task type dialogue system construction method fusing chatting and common knowledge is characterized in that: the method comprises the following steps:
s1, collecting large-scale open-source Chinese chatting linguistic data, taking adjacent dialogues as chatting linguistic data pairs, and constructing a chatting model;
s2, collecting the common sense question and answer corpus and the triple data of each large-scale field, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus;
s3, processing the original problem to obtain a candidate entity set, acquiring all out-degree and in-degree relations in the relevant two-hop relation from the knowledge graph, and obtaining a final candidate entity;
s4, screening out entity relationship paths by using the final candidate entities, optimizing the entity relationship paths to obtain final paths, and inquiring common sense answers from the knowledge graph based on the final paths to complete the construction of a common sense dialogue model;
s5, analyzing the task dialogue corpus, abstracting out fixed body data, storing the fixed body data into a database, and constructing a task dialogue model at a session level;
s6, generating a topic discrimination model by utilizing common sense reasoning and task dialogue corpus training;
the method includes the steps of collecting large-scale open-source Chinese chatting corpus in S1, taking adjacent dialogues as chatting corpus pairs, and constructing a chatting model, wherein the steps include:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by utilizing a dialogue neural network;
in S2, collecting the common sense question and answer corpus and the triple data of each large-scale field, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus, wherein the model comprises the following steps:
the triple data is used for expanding the diversity of the common sense question and answer corpus, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge graph, and training a SimCSE-RoBerta similarity scoring model based on the common sense question and answer corpus.
2. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: in S4, screening out entity relationship paths by using the final candidate entities, optimizing the entity relationship paths to obtain final paths, and inquiring common sense answers from the knowledge graph based on the final paths, wherein the steps comprise:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
3. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the step of splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relations respectively into a complete sentence comprises the following steps:
all final candidate entities correspond to the complete sentence spliced by:
Figure FDA0003952495440000021
wherein the content of the first and second substances,
Figure FDA0003952495440000022
represents all relationships in the degree of entry and a sentence composed of the ith final candidate entity, I i Is the final candidate entity of the ith input, R r1 Being an in-degree relationship, T, within a two-hop relationship r1 Is an in-degree entity within a two-hop relationship,
Figure FDA0003952495440000023
represents all relations in the degree and the sentence composed of the ith final candidate entity, R r2 For out-of-degree relations within a two-hop relation, T r2 The entity number is the out degree entity in the two-hop relationship, M is the number of the final candidate entities, R represents the total number of the relationship, E represents the total number of the entities, i represents the final candidate mark, R1 represents the in degree correlation coefficient, E1 represents the number of the in degree entities, R2 represents the out degree correlation coefficient, and E2 represents the number of the out degree entities.
4. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the method for respectively calculating the similarity between an original problem and each complete sentence by utilizing the similarity scoring model and screening out an entity relationship path comprises the following steps:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, outputting the 3 x N vector after passing through two full-connection layers, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
5. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the scoring of each entity relationship path and the original problem, selecting the entity relationship path with the highest score as the final path, and inquiring the common sense answer from the knowledge graph based on the final path comprises the following steps:
calculating the total score source between the ith entity relationship path and the original question by adopting the following formula:
source i ＝L i +D i +R i +C i
wherein L is i The intersection length, D, obtained by removing the duplication of the ith entity relationship path and the original problem on all word granularities i Is the inverse of the length of the ith entity relationship path, R i Is the reciprocal of the hop count of the ith entity relationship path, C i The frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score Source i And taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query common sense answers from the knowledge graph.
6. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: and S5, constructing a task dialog model at a session level, wherein the task dialog model comprises the following steps:
the method comprises the following steps of forming a dialogue sequence by the words, the belief states, the database results, the system actions and the system replies of each dialogue turn user, and finely adjusting the dialogue sequence on a dialogue neural network, wherein the dialogue sequence specifically comprises the following steps:
in the first dialog turn, the user inputs the utterance as U 0 According to the speech U 0 The generated belief state is B 0 Belief state B 0 For database retrieval to satisfy belief state B 0 The number of entities under the constraint and the search result are D 0 According to { U 0 ，B 0 ，D 0 Generation System action A 0 And system reply R 0 ；
At the tth dialog turn, based on the user utterance U t And all previously generated outputs { U } 0 ，B 0 ，D 0 ，A 0 ，R 0 ，…，U t-1 ，B t-1 ，D t-1 ，A t-1 ，R t-1 ，U t Is multiplied by the sum of the coefficients to generate B t 、A t And R t ；
Wherein, U t-1 Representing utterances entered by the user during the t-1 st dialog turn, B t-1 Representing the belief states generated from the utterances input by the user at the t-1 st dialogue turn, D t-1 Is represented by the formula A and B t-1 Corresponding search result, A t-1 Represents the system action at the t-1 st dialogue turn, R t-1 The system replies at t-1 dialog turns on the presentation.
7. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: processing the original problem in S3 to obtain a candidate entity set, which comprises:
extracting words and parts of speech of the original problem by a lac tool, reserving words and phrases with parts of speech, removing stop words, and respectively obtaining substrings of the words and phrases with parts of speech from 2 to the original problem length so as to obtain a candidate entity set.
8. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 7, wherein: s3, acquiring all out-degree and in-degree relations in the related two-hop relations from the knowledge graph, and obtaining a final candidate entity, wherein the steps comprise:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount count qr The similarity s between the original problem and each candidate entity is calculated using the following formula qe :
Figure FDA0003952495440000041
The similarity s between the original problem and each relation in the two-hop relation is calculated by adopting the following formula qc ：
Figure FDA0003952495440000042
Wherein S is q For the original set of problems, S e As a set of candidate entities S c Is a set of relationships;
calculating the relation data amount count using the following equation qr The similarity s between the original question and each candidate entity qe Similarity s of original problem and each relation in two-hop relation qc Weighted score candidate between s ：
candidate s ＝count qr *W qr +s qr *W qe +s qc *W qc
Wherein, W qr 、W qe 、W qc Are all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
CN202111334457.2A 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge Active CN114153955B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111334457.2A CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111334457.2A CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Publications (2)

Publication Number Publication Date
CN114153955A CN114153955A (en) 2022-03-08
CN114153955B true CN114153955B (en) 2023-04-07

Family

ID=80460125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111334457.2A Active CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Country Status (1)

Country Link
CN (1) CN114153955B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114626368B (en) * 2022-03-18 2023-06-09 中国电子科技集团公司第十研究所 Method and system for acquiring rule common sense knowledge in vertical field
CN115017276B (en) * 2022-03-28 2022-11-29 连芷萱 Multi-turn conversation method and system for government affair consultation, government affair robot and storage medium
CN115878775B (en) * 2022-12-23 2024-04-12 北京百度网讯科技有限公司 Method and device for generating cross-type dialogue data

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105068661B (en) * 2015-09-07 2018-09-07 百度在线网络技术（北京）有限公司 Man-machine interaction method based on artificial intelligence and system
CN113505586A (en) * 2021-06-07 2021-10-15 中电鸿信信息科技有限公司 Seat-assisted question-answering method and system integrating semantic classification and knowledge graph
CN113515613A (en) * 2021-06-25 2021-10-19 华中科技大学 Intelligent robot integrating chatting, knowledge and task question answering

Also Published As

Publication number Publication date
CN114153955A (en) 2022-03-08

Similar Documents

Publication Publication Date Title
CN115238101B (en) Multi-engine intelligent question-answering system oriented to multi-type knowledge base
CN114153955B (en) Construction method of multi-skill task type dialogue system fusing chatting and common knowledge
CN111046132B (en) Customer service question-answering processing method and system for searching multiple rounds of conversations
Tur et al. Spoken language understanding: Systems for extracting semantic information from speech
CN109918650B (en) Interview intelligent robot device capable of automatically generating interview draft and intelligent interview method
US11823074B2 (en) Intelligent communication manager and summarizer
Kim et al. Design principles and architecture of a second language learning chatbot
CN114691852B (en) Man-machine conversation system and method
CN111914185A (en) Graph attention network-based text emotion analysis method in social network
CN109325780A (en) A kind of exchange method of the intelligent customer service system in E-Governance Oriented field
CN114722839A (en) Man-machine collaborative dialogue interaction system and method
CN112287090A (en) Financial question asking back method and system based on knowledge graph
CN110597968A (en) Reply selection method and device
CN111339772B (en) Russian text emotion analysis method, electronic device and storage medium
Li et al. Development of an intelligent NLP-based audit plan knowledge discovery system
Shin et al. End-to-end task dependent recurrent entity network for goal-oriented dialog learning
Pardeshi et al. A survey on Different Algorithms used in Chatbot
Acharya et al. AtheNA an avid traveller using LSTM based RNN architecture
Riou et al. Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems
CN114912020A (en) Multi-sub-target dialogue recommendation method based on user preference graph
CN112395394B (en) Short text semantic understanding template checking method, template generating method and device
Ai et al. A Two-Stage Multimodal Emotion Recognition Model Based on Graph Contrastive Learning
Kreyssig Deep learning for user simulation in a dialogue system
CN115186071A (en) Intention recognition method and device, electronic equipment and readable storage medium
Zukerman et al. A probabilistic approach for argument interpretation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant