CN114153955A - Construction method of multi-skill task type dialogue system fusing chatting and common knowledge - Google Patents

Construction method of multi-skill task type dialogue system fusing chatting and common knowledge Download PDF

Info

Publication number
CN114153955A
CN114153955A CN202111334457.2A CN202111334457A CN114153955A CN 114153955 A CN114153955 A CN 114153955A CN 202111334457 A CN202111334457 A CN 202111334457A CN 114153955 A CN114153955 A CN 114153955A
Authority
CN
China
Prior art keywords
chatting
entity
final
path
degree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111334457.2A
Other languages
Chinese (zh)
Other versions
CN114153955B (en
Inventor
陈楷
熊京萍
廖奇
王辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kexun Jialian Information Technology Co ltd
Original Assignee
Kexun Jialian Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kexun Jialian Information Technology Co ltd filed Critical Kexun Jialian Information Technology Co ltd
Priority to CN202111334457.2A priority Critical patent/CN114153955B/en
Publication of CN114153955A publication Critical patent/CN114153955A/en
Application granted granted Critical
Publication of CN114153955B publication Critical patent/CN114153955B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G06F40/35—Discourse or dialogue representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention relates to a dialogue system, in particular to a multi-skill task type dialogue system construction method fusing chatting and common knowledge, which comprises the steps of collecting large-scale open-source Chinese chatting linguistic data, taking adjacent chats as chatting linguistic data pairs, constructing a chatting model, collecting common sense question-answer linguistic data and triple data in various large-scale fields, constructing a knowledge graph, scoring a model based on question-answer linguistic data training similarity, processing an original problem to obtain a candidate entity common sense set, obtaining all out-degree and in-degree relations in related two-hop relations from the knowledge graph, obtaining a final candidate entity, screening out a relation path by using the final candidate entity, preferably selecting the entity relation path to obtain a final path, inquiring a common sense answer from the knowledge graph based on the final path, and completing construction of a common sense dialogue model; the technical scheme provided by the invention can effectively overcome the defects that the prior art cannot have the common sense reasoning capability and the chatting function and has lower conversation reply recall accuracy.

Description

Construction method of multi-skill task type dialogue system fusing chatting and common knowledge
Technical Field
The invention relates to a dialogue system, in particular to a construction method of a multi-skill task type dialogue system fusing chatting and common knowledge.
Background
The dialog systems may be classified into a chatting type dialog system, a question-and-answer type dialog system, and a task type dialog system according to task types.
The chatting dialogue system mainly performs emotion interaction with a user to help the user relieve worries and stuffiness, generally trains data by using a large amount of chatting linguistic data, for example, dialog, T5 and the like adopt a generative model, so that the model has the capability of generating corresponding output according to input, but the generated result is random, and the whole process is difficult to evaluate and control. The chatty type dialogue system does not have the common sense reasoning ability and can not provide vertical tasks in a specific field.
The question-answering type dialogue system is also called an FAQ question-answering system, and is mainly used for providing vertical consulting services, such as government affair consultation and legal consultation, for the user. The current mainstream question-answer dialogue systems are all based on a retrieval scheme, namely an ES question-answer pair library is constructed in advance according to questions and answers, when a user asks questions, the questions in the library are retrieved, topN candidate questions most similar to the user questions are returned, then the most similar standard questions are selected by using a grading rearrangement mechanism, and the answers corresponding to the standard questions are returned as answers to the questions. At present, a common FAQ question-answering system fully utilizes text and semantic feature vectors by constructing a large number of similar questions or high-quality knowledge maps and adopts a recall and scoring strategy to ensure controllability and precision, but the question-answering type dialogue system does not have the common knowledge reasoning capability and cannot support the chatting function.
The task type dialogue system, also called an intelligent customer service system, has wide application prospects in various fields of logistics, finance, insurance, manufacturing, e-commerce and the like, can replace part of manual customer service to provide 24h service for users, greatly reduces the pressure of the manual customer service, and reduces the enterprise cost. At present, a mainstream task-based dialog system in the market is realized in a pipeline mode, the whole system is divided into a plurality of modules, for example, a dialog understanding NLU module mainly identifies user intentions and extracts key slot positions in a dialog, a dialog tracking DST module mainly takes charge of recording dialog states and slot positions, a dialog strategy DPL module mainly takes charge of deciding what response should be made to a user in the dialog according to current input and historical records, and a dialog generating NLG module mainly takes charge of generating content of a final response to the user. In view of the complexity of the task-based dialog system and the high standard requirement for task completion, the system does not have the common sense reasoning capability and cannot well support the chatting function.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects in the prior art, the invention provides the construction method of the multi-skill task type dialogue system fusing the chatting and the common knowledge, which can effectively overcome the defects that the prior art cannot have the common sense reasoning ability and the chatting function and has lower accuracy of the dialogue reply recall.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
a multi-skill task type dialogue system construction method fusing chatting and common knowledge comprises the following steps:
s1, collecting large-scale open-source Chinese chatting linguistic data, taking adjacent dialogues as chatting linguistic data pairs, and constructing a chatting model;
s2, collecting the common sense question and answer corpus and the triple data of each large-scale field, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus;
s3, processing the original problem to obtain a candidate entity set, acquiring all out-degree and in-degree relations in the relevant two-hop relation from the knowledge graph, and obtaining a final candidate entity;
s4, screening out entity relationship paths by using the final candidate entities, optimizing the entity relationship paths to obtain final paths, and inquiring common sense answers from the knowledge graph based on the final paths to complete the construction of a common sense conversation model;
s5, analyzing the task dialogue corpus, abstracting fixed ontology data to store in a database, and constructing a task dialogue model at a session level;
and S6, generating a topic discrimination model by using common sense reasoning and task dialogue corpus training.
Preferably, in S4, screening out entity relationship paths by using the final candidate entities, and performing optimization on the entity relationship paths to obtain final paths, and querying a common sense answer from the knowledge graph based on the final paths, including:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
Preferably, the splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relation into a complete sentence includes:
all final candidate entities correspond to the complete sentence spliced by:
Figure BDA0003350018620000031
wherein the content of the first and second substances,
Figure BDA0003350018620000032
representative of degree of incomeAll relationships in and the I-th final candidate entityiIs the final candidate entity of the ith input, Rr1Is an in-degree relationship, T, within a two-hop relationshipr1Is an in-degree entity within a two-hop relationship,
Figure BDA0003350018620000033
represents all relations in the degree and the sentence composed of the ith final candidate entity, Rr2For out-of-degree relations within a two-hop relation, Tr2And M is the number of final candidate entities.
Preferably, the calculating the similarity between the original problem and each complete sentence by using the similarity scoring model and screening out the entity relationship path includes:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, passing through two full-connection layers, outputting, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
Preferably, in S2, collecting the general knowledge question and answer corpus and the triple data in each large-scale field, constructing a knowledge graph, and training a similarity scoring model based on the general knowledge question and answer corpus, including:
the triple data is used for expanding the diversity of the common sense question and answer corpus, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge map, and training a SimCSE-RoBerta similarity scoring model based on the common sense question and answer corpus.
Preferably, the scoring each entity relationship path and the original question, selecting the entity relationship path with the highest score as the final path, and querying a common sense answer from the knowledge graph based on the final path includes:
calculating the total score source between the ith entity relationship path and the original problem by adopting the following formulai：
sourcei＝Li+Di+Ri+Ci
Wherein L isiThe intersection length, D, obtained after the duplication of the ith entity relationship path and the original problem is removed on all word granularitiesiIs the inverse of the length of the ith entity relationship path, RiIs the reciprocal of the hop count of the ith entity relationship path, CiThe frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score sourceiAnd taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query the common sense answers from the knowledge graph.
Preferably, the task dialog model is constructed at a session level in S5, including:
the method for performing fine adjustment on the dialog neural network by combining the words, the belief states, the database results, the system actions and the system replies of the users in each dialogue turn into a dialogue sequence comprises the following steps:
in the first dialog turn, the user inputs the utterance as U0According to the speech U0The generated belief state is B0Belief state B0For database retrieval to retrieve satisfaction of belief state B0The number of entities under the constraint and the search result are D0According to { U0，B0，D0Generation System action A0And system reply R0；
At the tth dialog turn, based on the user utterance UtAnd all previously generated outputs { U }0，B0，D0，A0，R0，…，Ut-1，Bt-1，Dt-1，At-1，Rt-1，UtIs multiplied by the sum of the coefficients to generate Bt、AtAnd Rt。
Preferably, the processing the original problem in S3 to obtain a candidate entity set includes:
and performing word segmentation and part-of-speech extraction on the original problem through a lac tool, reserving words and phrases with the part-of-speech, removing stop words, and respectively obtaining substrings of the words and phrases with the part-of-speech from 2 to the original problem length to obtain a candidate entity set.
Preferably, in S3, obtaining all the relationships between out-degree and in-degree in the relevant two-hop relationship from the knowledge graph, and obtaining the final candidate entity includes:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount countqrThe similarity s between the original problem and each candidate entity is calculated using the following formulaqe：
Figure BDA0003350018620000051
The similarity s between the original problem and each relation in the two-hop relation is calculated by adopting the following formulaqc：
Figure BDA0003350018620000052
Wherein S isqFor the original set of problems, SeAs a set of candidate entities, ScIs a set of relationships;
calculating the relation data amount count using the following equationqrThe similarity s between the original question and each candidate entityqeSimilarity s of original problem and each relation in two-hop relationqcWeighted score candidate betweens：
candidate s＝countqr*Wqr+sqr*Wqe+sqc*Wqc
Wherein, Wqr、Wqe、WqcAre all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
Preferably, in S1, collecting large-scale open-source chinese chatting corpus, taking adjacent dialogues as chatting corpus pairs, and constructing a chatting model, including:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by utilizing a Dialdo DG neural network.
(III) advantageous effects
Compared with the prior art, the construction method of the multi-skill task type dialogue system fusing the chatting and the common knowledge, provided by the invention, has the following beneficial effects:
1) in the construction process of a common sense dialogue model, a unique candidate entity extraction strategy is designed, an additional entity link model is not needed, the cost of manual labeling and model training is reduced, feature vectors are extracted by using an SimCSE-RoBerta similarity scoring model at an entity relation path stage for differential fusion, feature information is enhanced, and recall scoring is carried out by combining with the original problem when a final path is optimized, so that the recall accuracy is effectively improved;
2) in the construction process of the task dialogue model, a pipeline multi-module scheme is abandoned, a unified data input format is constructed, mutual information of dialogue texts and system responses is improved, and model training is carried out in a DialoDG end-to-end mode.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a schematic flow diagram of the present invention;
FIG. 2 is a schematic flow chart of the method for constructing a chat model according to the present invention;
FIG. 3 is a schematic flow chart of the common sense dialogue model construction in the present invention;
FIG. 4 is a schematic flow chart of the task dialogue model construction method of the present invention;
fig. 5 is a working diagram of the dialog neural network in the process of constructing the task dialogue model in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A multi-skill task type dialogue system construction method fusing chat and common knowledge is disclosed, as shown in FIG. 2, collecting large-scale open-source Chinese chat linguistic data, taking adjacent dialogues as chat linguistic data pairs, and constructing a chat model, and specifically comprises the following steps:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by utilizing a Dialdo DG neural network.
As shown in fig. 3, collecting the common sense question and answer corpus and the triple data in each large-scale field, constructing a knowledge graph, and training a similarity scoring model based on the common sense question and answer corpus specifically includes:
the method comprises the steps of expanding diversity of common sense question and answer corpora by utilizing 1.4 hundred million Chinese entity triple data sourced by ownthink, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge graph, and training a SimCSE-RoBerta similarity scoring model based on the common sense question and answer corpora.
As shown in fig. 3, the original problem is processed to obtain a candidate entity set, all the relationships between out-degree and in-degree in the relevant two-hop relationship are obtained from the knowledge graph, and a final candidate entity is obtained.
Processing an original problem to obtain a candidate entity set, including:
and performing word segmentation and part-of-speech extraction on the original problem through a lac tool, reserving words and phrases with the part-of-speech, removing stop words, and respectively obtaining substrings of the words and phrases with the part-of-speech from 2 to the original problem length to obtain a candidate entity set.
Acquiring all the relations of the out-degree and the in-degree in the related two-hop relation from the knowledge graph, and acquiring a final candidate entity, wherein the relations comprise:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount countqrThe similarity s between the original problem and each candidate entity is calculated using the following formulaqe：
Figure BDA0003350018620000081
The similarity s between the original problem and each relation in the two-hop relation is calculated by adopting the following formulaqc：
Figure BDA0003350018620000082
Wherein S isqFor the original set of problems, SeAs a set of candidate entities, ScIs a set of relationships;
calculating the relation data amount count using the following equationqrThe similarity s between the original question and each candidate entityqeSimilarity s of original problem and each relation in two-hop relationqcWeighted score candidate betweens：
candidate s＝countqr*Wqr+sqr*Wqe+sqc*Wqc
Wherein, Wqr、Wqe、WqcAre all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
As shown in fig. 3, screening out an entity relationship path by using the final candidate entity, and performing optimization on the entity relationship path to obtain a final path, and querying a common sense answer from the knowledge graph based on the final path to complete the construction of the common sense dialogue model, which specifically includes:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
Splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relations respectively to form a complete sentence, wherein the method comprises the following steps:
all final candidate entities correspond to the complete sentence spliced by:
Figure BDA0003350018620000083
wherein the content of the first and second substances,
Figure BDA0003350018620000091
represents all relationships in the degree of entry and a sentence composed of the ith final candidate entity, IiIs the final candidate entity of the ith input, Rr1Is an in-degree relationship, T, within a two-hop relationshipr1Is an in-degree entity within a two-hop relationship,
Figure BDA0003350018620000092
represents all relations in the degree and the sentence composed of the ith final candidate entity, Rr2For out-of-degree relations within a two-hop relation, Tr2And M is the number of final candidate entities.
Secondly, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out an entity relationship path, wherein the similarity scoring model comprises the following steps:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, passing through two full-connection layers, outputting, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
Scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring a common sense answer from the knowledge graph based on the final path, wherein the method comprises the following steps:
calculating the total score source between the ith entity relationship path and the original problem by adopting the following formulai：
sourcei＝Li+Di+Ri+Ci
Wherein L isiThe intersection length, D, obtained after the duplication of the ith entity relationship path and the original problem is removed on all word granularitiesiIs the inverse of the length of the ith entity relationship path, RiIs the reciprocal of the hop count of the ith entity relationship path, CiThe frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score sourceiAnd taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query the common sense answers from the knowledge graph.
As shown in fig. 4, the task dialogue corpus is analyzed, the fixed ontology data is abstracted and stored in the database, and the task dialogue model is constructed at the session level.
Analyzing task dialogue linguistic data, abstracting fixed ontology data and storing the fixed ontology data into a database, wherein the method comprises the following steps:
take restaurant information as an example, [ { "pname": anhui province, "city": compost city, "adname": "address" in the Shushan area: innovation Daotai No. 2800 in Sichuan province, "location": [117.26104,31.85117], "name": old and rural chicken innovation industry park shop, "quality": 5, "license range": cheap } ], generally, if a third party has more accurate external information, the third party can be butted in the form of an HTTP interface, then the diversity of language expressions in linguistic data of different sources is removed, mutual information between a dialogue semantic structure and response is fully utilized, all data are differentiated, specific slot positions in dialogue data are replaced in a uniform alias mode, and the words, the belief states, database results (also can be the results of the external interface), system actions and system replies of a user are surrounded in a specific coding format as input;
the specific data format of the conventional scheme is as follows:
user i want to see a cheap, good and good restaurant near the lake park of Sichuan
[ Slot ] [ Shuxi lake park, cheap, good public praise ]
[ System ] the old and rural chicken innovation industry park store is good, and the popular comment is a 5-star good comment
User is good, help me order today's 4-person compartment
The specific data format in the invention is as follows:
< sos _ u > ] I want to see the restaurant at the privacy quality around address [ < eos _ u > ]
[<sos_a>][address][pricerange][quality][restaurant][<eos_u>]
[ < sos _ r > ] name is good, and popular is quality. [ < eos _ r > ]
[sos_db][tb_restaurant][address][pricerange][quality][eos_db]
Good [ < sos _ u > ] person's compartment [ < eos _ u > ] helping me to schedule time
[<sos_a>][booking][address][pricerange][quality][time][person][<eos_u>]
Good, [ < sos _ r > ], has succeeded in helping you book [ old and rural chicken Innovation industry park store ] [ today ] people's booth [ 4 ]. Ask what can help you? [ < eos _ r > ]
Secondly, constructing a task dialogue model at a conversation level, as shown in fig. 5, including:
the method for performing fine adjustment on the dialog neural network by combining the words, the belief states, the database results, the system actions and the system replies of the users in each dialogue turn into a dialogue sequence comprises the following steps:
in the first dialog turn, useThe speech input by the user is U0According to the speech U0The generated belief state is B0Belief state B0For database retrieval to retrieve satisfaction of belief state B0The number of entities under the constraint and the search result are D0According to { U0，B0，D0Generation System action A0And system reply R0；
At the tth dialog turn, based on the user utterance UtAnd all previously generated outputs { U }0，B0，D0，A0，R0，…，Ut-1，Bt-1，Dt-1，At-1，Rt-1，UtIs multiplied by the sum of the coefficients to generate Bt、AtAnd Rt。
And generating a topic discrimination model by using common sense reasoning and task dialogue corpus training. The method comprises the following steps that a user inputs a topic type judgment through a topic judgment model, and if a result output by the topic judgment model is a common sense type, a common sense conversation model is called; if the result output by the topic judging model is the task type, calling a task dialogue model; and if the topic discrimination model judges that the input does not belong to the common sense type or the task type, calling the chatting model.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims (10)

1. A multi-skill task type dialogue system construction method fusing chatting and common knowledge is characterized in that: the method comprises the following steps:
s1, collecting large-scale open-source Chinese chatting linguistic data, taking adjacent dialogues as chatting linguistic data pairs, and constructing a chatting model;
s2, collecting the common sense question and answer corpus and the triple data of each large-scale field, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus;
s3, processing the original problem to obtain a candidate entity set, acquiring all out-degree and in-degree relations in the relevant two-hop relation from the knowledge graph, and obtaining a final candidate entity;
s4, screening out entity relationship paths by using the final candidate entities, optimizing the entity relationship paths to obtain final paths, and inquiring common sense answers from the knowledge graph based on the final paths to complete the construction of a common sense conversation model;
s5, analyzing the task dialogue corpus, abstracting fixed ontology data to store in a database, and constructing a task dialogue model at a session level;
and S6, generating a topic discrimination model by using common sense reasoning and task dialogue corpus training.
2. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: in S4, screening an entity relationship path by using the final candidate entity, and performing optimization on the entity relationship path to obtain a final path, and querying a general knowledge answer from the knowledge graph based on the final path, including:
splicing the final candidate entities with all out-degree and in-degree relations in the corresponding two-hop relations to form complete sentences, respectively calculating the similarity between the original problem and each complete sentence by using a similarity scoring model, and screening out entity relation paths;
and scoring each entity relationship path and the original problem, selecting the entity relationship path with the highest score as a final path, and inquiring the common sense answer from the knowledge graph based on the final path.
3. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the step of splicing the final candidate entities with all the out-degree and in-degree relations in the corresponding two-hop relations respectively into a complete sentence comprises the following steps:
all final candidate entities correspond to the complete sentence spliced by:
Figure FDA0003350018610000021
wherein the content of the first and second substances,
Figure FDA0003350018610000022
represents all relationships in the degree of entry and a sentence composed of the ith final candidate entity, IiIs the final candidate entity of the ith input, Rr1Is an in-degree relationship, T, within a two-hop relationshipr1Is an in-degree entity within a two-hop relationship,
Figure FDA0003350018610000023
represents all relations in the degree and the sentence composed of the ith final candidate entity, Rr2For out-of-degree relations within a two-hop relation, Tr2And M is the number of final candidate entities.
4. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the method for respectively calculating the similarity between an original problem and each complete sentence by utilizing the similarity scoring model and screening out an entity relationship path comprises the following steps:
and respectively passing the original problem and the complete sentence through a similarity scoring model, adding the last layer of N-dimensional feature vectors, splicing the last layer of N-dimensional feature vectors to form a 3 x N vector, passing through two full-connection layers, outputting, and finally scoring by using a softmax function to screen out an entity relationship path corresponding to a preset number of complete sentences with the highest score.
5. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 4, wherein: collecting the common sense question and answer corpus and the triple data in each large-scale field in S2, constructing a knowledge map, and training a similarity scoring model based on the common sense question and answer corpus, wherein the model comprises the following steps:
the triple data is used for expanding the diversity of the common sense question and answer corpus, cleaning and preprocessing the triple data, storing the triple data into a neo4j database to construct a knowledge map, and training a SimCSE-RoBerta similarity scoring model based on the common sense question and answer corpus.
6. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 2, wherein: the scoring of each entity relationship path and the original problem, selecting the entity relationship path with the highest score as the final path, and inquiring the common sense answer from the knowledge graph based on the final path comprises the following steps:
calculating the total score source between the ith entity relationship path and the original problem by adopting the following formulai：
sourcei＝Li+Di+Ri+Ci
Wherein L isiThe intersection length, D, obtained after the duplication of the ith entity relationship path and the original problem is removed on all word granularitiesiIs the inverse of the length of the ith entity relationship path, RiIs the reciprocal of the hop count of the ith entity relationship path, CiThe frequency of the final candidate entity appearing in the ith entity relationship path is obtained;
selecting a Total score sourceiAnd taking the highest entity relationship path as a final path, and assembling cypher query sentences based on the final path to query the common sense answers from the knowledge graph.
7. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: and S5, constructing a task conversation model at a conversation level, wherein the task conversation model comprises the following steps:
the method for performing fine adjustment on the dialog neural network by combining the words, the belief states, the database results, the system actions and the system replies of the users in each dialogue turn into a dialogue sequence comprises the following steps:
in the first dialog turn, the user inputs the utterance as U0According to the speech U0The generated belief state is B0Belief state B0For database retrieval to retrieve satisfaction of belief state B0The number of entities under the constraint and the search result are D0According to { U0，B0，D0Generation System action A0And system reply R0；
At the tth dialog turn, based on the user utterance UtAnd all previously generated outputs { U }0，B0，D0，A0，R0，…，Ut-1，Bt-1，Dt-1，At-1，Rt-1，UtIs multiplied by the sum of the coefficients to generate Bt、AtAnd Rt。
8. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: in S3, processing the original problem to obtain a candidate entity set, including:
and performing word segmentation and part-of-speech extraction on the original problem through a lac tool, reserving words and phrases with the part-of-speech, removing stop words, and respectively obtaining substrings of the words and phrases with the part-of-speech from 2 to the original problem length to obtain a candidate entity set.
9. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 8, wherein: in S3, obtaining all the relationships between out-degree and in-degree in the two-hop relationship from the knowledge graph, and obtaining a final candidate entity, including:
obtaining all out-degree and in-degree relations in the related two-hop relation from the knowledge graph, and counting the relation data amount countqrThe similarity s between the original problem and each candidate entity is calculated using the following formulaqe：
Figure FDA0003350018610000041
The similarity s between the original problem and each relation in the two-hop relation is calculated by adopting the following formulaqc：
Figure FDA0003350018610000042
Wherein S isqFor the original set of problems, SeAs a set of candidate entities, ScIs a set of relationships;
calculating the relation data amount count using the following equationqrThe similarity s between the original question and each candidate entityqeSimilarity s of original problem and each relation in two-hop relationqcWeighted score candidate betweens：
candidates＝countqr*Wqr+sqr*Wqe+sqc*Wqc
Wherein, Wqr、Wqe、WqcAre all fixed adjustment values;
and reserving the preset number of candidate entities with the highest scores as final candidate entities.
10. The method for building a multi-skill task type dialogue system fusing chatting and general knowledge according to claim 1, wherein: collecting large-scale open-source Chinese chatting corpus in S1, taking adjacent chats as chatting corpus pairs, and constructing a chatting model, including:
collecting large-scale open-source Chinese chatting linguistic data, performing noise cleaning and quality inspection on the Chinese chatting linguistic data, removing sensitive subject samples, taking the upper part of an adjacent conversation as a question query and the lower part as an Answer, and training a generating chatting model by utilizing a Dialdo DG neural network.
CN202111334457.2A 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge Active CN114153955B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111334457.2A CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111334457.2A CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Publications (2)

Publication Number Publication Date
CN114153955A true CN114153955A (en) 2022-03-08
CN114153955B CN114153955B (en) 2023-04-07

Family

ID=80460125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111334457.2A Active CN114153955B (en) 2021-11-11 2021-11-11 Construction method of multi-skill task type dialogue system fusing chatting and common knowledge

Country Status (1)

Country Link
CN (1) CN114153955B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114626368A (en) * 2022-03-18 2022-06-14 中国电子科技集团公司第十研究所 Method and system for acquiring common knowledge of vertical domain rules
CN115017276A (en) * 2022-03-28 2022-09-06 连芷萱 Multi-turn conversation method and system for government affair consultation by combining fuzzy logic and R-GCN
CN115878775A (en) * 2022-12-23 2023-03-31 北京百度网讯科技有限公司 Method and device for generating cross-type dialogue data

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017041372A1 (en) * 2015-09-07 2017-03-16 百度在线网络技术（北京）有限公司 Man-machine interaction method and system based on artificial intelligence
CN113505586A (en) * 2021-06-07 2021-10-15 中电鸿信信息科技有限公司 Seat-assisted question-answering method and system integrating semantic classification and knowledge graph
CN113515613A (en) * 2021-06-25 2021-10-19 华中科技大学 Intelligent robot integrating chatting, knowledge and task question answering

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017041372A1 (en) * 2015-09-07 2017-03-16 百度在线网络技术（北京）有限公司 Man-machine interaction method and system based on artificial intelligence
CN113505586A (en) * 2021-06-07 2021-10-15 中电鸿信信息科技有限公司 Seat-assisted question-answering method and system integrating semantic classification and knowledge graph
CN113515613A (en) * 2021-06-25 2021-10-19 华中科技大学 Intelligent robot integrating chatting, knowledge and task question answering

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114626368A (en) * 2022-03-18 2022-06-14 中国电子科技集团公司第十研究所 Method and system for acquiring common knowledge of vertical domain rules
CN114626368B (en) * 2022-03-18 2023-06-09 中国电子科技集团公司第十研究所 Method and system for acquiring rule common sense knowledge in vertical field
CN115017276A (en) * 2022-03-28 2022-09-06 连芷萱 Multi-turn conversation method and system for government affair consultation by combining fuzzy logic and R-GCN
CN115017276B (en) * 2022-03-28 2022-11-29 连芷萱 Multi-turn conversation method and system for government affair consultation, government affair robot and storage medium
CN115878775A (en) * 2022-12-23 2023-03-31 北京百度网讯科技有限公司 Method and device for generating cross-type dialogue data
CN115878775B (en) * 2022-12-23 2024-04-12 北京百度网讯科技有限公司 Method and device for generating cross-type dialogue data

Also Published As

Publication number Publication date
CN114153955B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
CN115238101B (en) Multi-engine intelligent question-answering system oriented to multi-type knowledge base
CN114153955B (en) Construction method of multi-skill task type dialogue system fusing chatting and common knowledge
Kreyssig et al. Neural user simulation for corpus-based policy optimisation for spoken dialogue systems
Shou et al. Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis
US11823074B2 (en) Intelligent communication manager and summarizer
Oraby et al. " How May I Help You?" Modeling Twitter Customer ServiceConversations Using Fine-Grained Dialogue Acts
CN111353013A (en) Method and system for realizing intelligent delivery and reception
CN115392259B (en) Microblog text sentiment analysis method and system based on confrontation training fusion BERT
CN109325780A (en) A kind of exchange method of the intelligent customer service system in E-Governance Oriented field
Preiser et al. Qualitative content analysis
CN109871441A (en) One kind knowledge neural network based of leading answers system and method
Li et al. Development of an intelligent NLP-based audit plan knowledge discovery system
CN114691852A (en) Man-machine conversation system and method
Anupama et al. Real time Twitter sentiment analysis using natural language processing
Maher et al. AI and deep learning-driven chatbots: a comprehensive analysis and application trends
Shin et al. End-to-end task dependent recurrent entity network for goal-oriented dialog learning
CN112395885B (en) Short text semantic understanding template generation method, semantic understanding processing method and device
CN117407516A (en) Information extraction method, information extraction device, electronic equipment and storage medium
Pardeshi et al. A survey on Different Algorithms used in Chatbot
Acharya et al. AtheNA an avid traveller using LSTM based RNN architecture
Riou et al. Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems
CN114912020A (en) Multi-sub-target dialogue recommendation method based on user preference graph
Kreyssig Deep learning for user simulation in a dialogue system
Duran et al. Inter-annotator agreement using the conversation analysis modelling schema, for dialogue
Gelbukh Sentiment analysis and opinion mining: Keynote address

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant