CN113220856A - Multi-round dialogue system based on Chinese pre-training model - Google Patents
Multi-round dialogue system based on Chinese pre-training model Download PDFInfo
- Publication number
- CN113220856A CN113220856A CN202110588492.0A CN202110588492A CN113220856A CN 113220856 A CN113220856 A CN 113220856A CN 202110588492 A CN202110588492 A CN 202110588492A CN 113220856 A CN113220856 A CN 113220856A
- Authority
- CN
- China
- Prior art keywords
- question
- module
- model
- training
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q30/016—After-sales
Abstract
The invention relates to a multi-round dialogue system based on a Chinese pre-training model, which comprises: a data processing module; the retrieval module is used for retrieving the question and answer data set by utilizing the vector inner product similarity; the generating module is used for improving a Chinese pre-training model NEZHA pre-training model so as to enable the Chinese pre-training model NEZHA to be used for generating tasks; the compression module is used for carrying out knowledge distillation on the NEZHA model of the generation module by using a replaceable strategy; and the dialogue management module is used for managing high-frequency user problems.
Description
Technical Field
The invention belongs to the field of intelligent dialogue, and mainly relates to a multi-round dialogue system based on a Chinese pre-training model.
Background
With the rapid development of deep learning technology and network technology, data-driven models are becoming more and more popular. Building a human-like dialog agent is considered one of the most challenging tasks in artificial intelligence. For a task-specific dialog system, it can be regarded as a continuous decision process. It relies on a large amount of information to continue the conversation, such as conversation context, intentions, external knowledge, common sense, emotions, participant's background and personas, etc. All of this can have an effect on the response in the conversation, and these uncertainties make the conversation extremely daunting.
In addition, as computer computing power is improved, a large amount of real conversation data is generated in daily life, and a large pre-trained neural network model (such as NEZHA) is greatly developed, so that a plurality of natural language understanding tasks based on the real conversation data even exceed the human level. However, at present, the task-based multi-turn dialog class is created directly based on the Chinese pre-training neural network, so that relatively few tasks are required, and the dialog quality and diversity are to be improved.
For a dialog system in a specific scene (such as an e-commerce platform), dialog data in the field is often focused, and although the dialog system can solve the problem of high-frequency users, the dialog system is limited by the dialog data and lacks generalization and semantic understanding ability for some long-tail problems. Although the end-to-end model has become a hot spot in current research, we still need to rely on the traditional pipelined dialog system in the actual dialog system, especially in the pre-heating stage of a new domain.
The retrieval type and the generation type dialogue systems have different realization principles and have respective advantages and disadvantages. While a search-based dialog can provide more fluid and relevant responses, a generative dialog can model more complex contextual semantics (e.g., user emotions). Thus, the present invention explores a combination of search and generation strategies to seek better dialog system performance.
Reference documents:
[1] ScaNN vector search tool:
https://github.com/google-research/google-research/tree/master/scann
[2]Wei J,Ren X,Li X,et al.NEZHA:Neural contextualized representation for chinese language understanding[J].arXiv preprint arXiv:1909.00204,2019.
[3]Guo R,Sun P,Lindgren E,et al.Accelerating large-scale inference with anisotropic vector quantization[C].In International Conference on Machine Learning,2020:3887–3896.
[4]Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:6000-6010.
disclosure of Invention
The invention aims to provide a multi-round dialogue system based on a Chinese pre-training model, which adopts the following technical scheme:
a multi-turn dialogue system based on a Chinese pre-training model comprises:
the data processing module is used for segmenting the dialogue data by a proper method, removing stop words, replacing English punctuations, constructing a question-answer data set and constructing a sentence vector index database, the concrete process of constructing the sentence vector index database is as follows,
firstly, segmenting words of user problems by using a Jieba Chinese word segmentation tool;
secondly, loading the open-source Chinese word vectors, and mapping the user problem after word segmentation into a plurality of word vectors;
thirdly, carrying out weighted summation and averaging on a plurality of word vectors to convert the word vectors into sentence vectors, and constructing a user question sentence vector index database;
the retrieval module retrieves the question-answer data set by utilizing the similarity of the vector inner product, and the specific process is as follows,
firstly, segmenting words of a user input problem by using a Jieba Chinese word segmentation tool;
secondly, inputting the user question after word segmentation into a FastText text classification model for user intention identification, and judging whether the question belongs to chatting;
thirdly, if the question belongs to chatting, adding the historical dialogue questions of the last two rounds as the user questions;
fourthly, converting the user problem into a sentence vector, and calculating the similarity of vector inner products with a sentence vector index database constructed by a data processing module by using a ScaNN vector retrieval tool;
fifthly, obtaining a plurality of candidate answers corresponding to the question with the highest similarity score;
sixthly, inputting the question and a plurality of candidate answers into a Chinese pre-training language model NEZHA for reordering to obtain the answer with the highest score;
the generation module improves a Chinese pre-training model NEZHA pre-training model to enable the Chinese pre-training model NEZHA pre-training model to be used for generating tasks, and comprises the following specific processes,
loading a question and answer data set obtained by a data processing module into a generating module, and loading a pre-training weight of a Chinese pre-training model NEZHA for training a generating model aiming at the question and answer data;
secondly, recording different roles in the question and answer data, such as the words of the user as all 0, and the words of the customer service as all 1, and embedding the words as paragraphs;
thirdly, according to the paragraph embedding in the second step, obtaining the length m of the question-answer data, and then constructing a self-attention matrix with m rows and m columns;
setting the upper triangular part of the self-attention matrix in the third step as- ∞, and setting other position elements of the matrix as 0 as the input attention matrix of question-answer pairs;
fifthly, inputting the embedded sequence into a 12-layer transform network for training;
sixthly, generating 10 candidate replies by using topK random decoding;
the compression module utilizes an alternative strategy to carry out knowledge distillation on the NEZHA model of the generation module, and the specific process is as follows,
inputting a question and answer data set obtained by a data processing module, loading the NEZHA model weight finely adjusted by a generating module, and marking as a ancestor layer;
secondly, setting the probability rr of replacing the ancestor layer by the inheritor layer to be 0.5, and compressing the layer number of the ancestor layer to be half of the original layer number;
thirdly, continuing training by using the question-answer pairs constructed by the data processing module, and then generating candidate answers by using the compressed inheritor layer;
and the dialogue management module is used for managing high-frequency user problems.
The invention provides an e-commerce customer service dialogue system which combines retrieval and generation based on a Chinese pre-training model and is assisted by a task template. The current advanced pre-training model is combined with the dialogue system, and in order to improve the satisfaction degree of users, knowledge distillation is applied to the generation model, so that the reasoning efficiency of the pre-training model is optimized, and the system can achieve good performance in the aspects of reply generation quality and operation efficiency. The invention develops a new idea for combining a multi-turn dialogue system with the current advanced Chinese pre-training language model and makes a contribution to improving the reply generation quality and efficiency of the dialogue system.
Drawings
FIG. 1 is a multi-turn dialog system framework diagram;
FIG. 2 is a schematic diagram of a retrieval module;
FIG. 3 is a diagram of a pre-training model base composition transform encoder;
FIG. 4 is a schematic diagram of a generation module;
Detailed Description
The present invention will be described and demonstrated in further detail below with reference to experimental procedures and experimental results.
The invention designs a multi-round dialogue system based on a Chinese pre-training model. The system comprises: the device comprises a data processing module, a retrieval module, a generation module, a compression module and a conversation management module. The invention combines the current advanced pre-training model with the dialogue system, and applies knowledge distillation in the generating model in order to improve the satisfaction degree of users, thereby optimizing the reasoning efficiency of the pre-training model and leading the system to achieve good performance in the aspects of reply generation quality and operation efficiency. Fig. 1 presents the overall framework of the proposed dialog system. The key points of the specific technical scheme are divided into the following five parts:
(1) data processing module
The method provided by the patent firstly carries out word segmentation on the dialogue data through a proper method, removes stop words, replaces English punctuations, constructs a question-answer data set and constructs a sentence vector index database. The specific process of constructing the sentence vector index database is as follows,
firstly, using a Jieba Chinese word segmentation tool to segment the words of the user questions,
secondly, loading the Chinese word vectors of the open source, mapping the user questions after word segmentation into a plurality of word vectors,
and thirdly, carrying out weighted summation and averaging on a plurality of word vectors to convert the word vectors into sentence vectors, and constructing a user question sentence vector index database.
(2) Retrieval module
The retrieval module mainly utilizes the similarity of the vector inner product to retrieve the question-answering data, and a main flow chart of the retrieval module is shown in figure 2,
firstly, the input problems of the user are segmented by utilizing a Jieba Chinese segmentation tool,
secondly, inputting the user question after word segmentation into a FastText text classification model for user intention identification, judging whether the question belongs to chatting,
third, if the question belongs to chat, add its last two rounds of historical dialogue questions to the back as user questions
Fourthly, converting the user problem into a sentence vector, and utilizing a ScaNN (1, 3) vector retrieval tool to calculate the similarity of the neighbors with a sentence vector index database constructed by a data processing module
The fifth step, get 10 candidate answers that the question that the degree of similarity score is the highest corresponds to
Sixthly, inputting the question and 10 candidate responses into Chinese pre-training language model NEZHA [2] for re-ranking to obtain the response with the highest score
(3) Generation module
For the generation module, we improve the improvement of the Chinese pre-training model NEZHA pre-training model to enable it to be used for generating tasks. FIG. 3 shows the basic components of the NEZHA encoder, the transform [4] module. As shown in FIG. 4, NEZHA is composed of 12 layers of transformers, and the specific process is as follows,
firstly, a question and answer data set obtained by a data processing module is loaded into a generating module, and the pre-training weight of a Chinese pre-training model NEZHA is loaded for training the generating model aiming at the question and answer data.
And secondly, recording different roles in the question and answer data, such as the words of the user as all 0, and the words of the customer service as all 1, and Embedding as paragraphs (Segment Embedding).
And thirdly, according to the paragraph embedding in the second step, obtaining the length m of the question-answer data, and then constructing a self-attention matrix with m rows and m columns.
And fourthly, setting the upper triangular part of the self-attention matrix in the third step as ∞ and setting other position elements of the matrix as 0 as the input attention matrix of the question-answer pair.
And fifthly, inputting the embedded sequence into a 12-layer transform network for training.
Sixth, 10 candidate replies are generated using topK random decoding
(4) Compression module
We used an alternative strategy to distill the knowledge of the NEZHA model of the generation module. The specific process is as follows,
firstly, inputting a question and answer data set obtained by a data processing module, loading the NEZHA model weight finely adjusted by a generating module, and marking as a ancestor layer.
And secondly, setting the probability rr of replacing the ancestor layer by the successor layer (Suc layer) to be 0.5, and compressing the layer number of the ancestor layer to be half of the original layer number.
And thirdly, continuing training by using the question-answer pairs constructed by the data processing module, and then generating candidate answers by using the compressed inheritor layer.
(5) Dialogue management module
The question and answer data set constructed by the data processing module is provided with a plurality of high-frequency user questions (such as goods returns, order modification, price protection and the like), and the questions are processed by corresponding flow answers. As shown in fig. 1, we collate these problems and add a task dialog module. The specific process is as follows,
firstly, after the system preprocesses the user input question, firstly, the user input question is input to a task module for task matching, if a response task is matched, the system directly returns a response corresponding to a predefined template
And secondly, if the predefined task template is not matched, executing a subsequent retrieval module.
And thirdly, if the semantic matching score in the candidate answer obtained by the retrieval module is greater than the set threshold value of 0.5, the retrieval is successful, and the candidate answer with the highest score is returned.
And fourthly, if the candidate reply score of the retrieval module is smaller than the threshold value of 0.5, executing the generation module, reordering the generated candidate and the retrieval candidate together, and returning the reply with the highest score.
And selecting dialogues with session sessions more than 2 rounds in the original data set, and selecting three rounds of dialogues for training the model. In addition, we filter out sessions with a number of response words less than 4, because they tend to be generic responses. The data partitioning is shown in table 1.
TABLE 1 Experimental database partitioning and basic cases thereof
As shown in table 2, we performed detailed experiments on the above components from the above 5 indexes, and compared the performances of the current mainstream model on the constructed data set respectively. In the table c _ a is an abbreviation of copy with attribute, i.e. attention and copy mechanism is introduced in seq2seq, l2r denotes the previously mentioned attention mask scheme of NEZHA as generative model, and t _ l denotes the knowledge distillation of generative model.
TABLE 2 comparison of model for each module
Our model has good results both on individual components and overall, compared to the traditional model, which also demonstrates the effectiveness of our designed system framework.
While the invention has been described in connection with the drawings, the invention is not limited to the specific embodiments described above, which are intended to be illustrative rather than limiting, and that many modifications may be made by those skilled in the art without departing from the spirit of the invention, which will fall within the scope of the appended claims.
Claims (1)
1. A multi-turn dialogue system based on a Chinese pre-training model comprises:
the data processing module is used for segmenting the dialogue data by a proper method, removing stop words, replacing English punctuations, constructing a question-answer data set and constructing a sentence vector index database, the concrete process of constructing the sentence vector index database is as follows,
firstly, segmenting words of user problems by using a Jieba Chinese word segmentation tool;
secondly, loading the open-source Chinese word vectors, and mapping the user problem after word segmentation into a plurality of word vectors;
and thirdly, carrying out weighted summation and averaging on a plurality of word vectors to convert the word vectors into sentence vectors, and constructing a user question sentence vector index database.
The retrieval module retrieves the question-answer data set by utilizing the similarity of the vector inner product, and the specific process is as follows,
firstly, segmenting words of a user input problem by using a Jieba Chinese word segmentation tool;
secondly, inputting the user question after word segmentation into a FastText text classification model for user intention identification, and judging whether the question belongs to chatting;
thirdly, if the question belongs to chatting, adding the historical dialogue questions of the last two rounds as the user questions;
fourthly, converting the user problem into a sentence vector, and calculating the similarity of vector inner products with a sentence vector index database constructed by a data processing module by using a ScaNN vector retrieval tool;
fifthly, obtaining a plurality of candidate answers corresponding to the question with the highest similarity score;
sixthly, inputting the question and a plurality of candidate answers into a Chinese pre-training language model NEZHA for reordering to obtain the answer with the highest score;
the generation module improves a Chinese pre-training model NEZHA pre-training model to enable the Chinese pre-training model NEZHA pre-training model to be used for generating tasks, and comprises the following specific processes,
loading a question and answer data set obtained by a data processing module into a generating module, and loading a pre-training weight of a Chinese pre-training model NEZHA for training a generating model aiming at the question and answer data;
secondly, recording different roles in the question and answer data, such as the words of the user as all 0, and the words of the customer service as all 1, and embedding the words as paragraphs;
thirdly, according to the paragraph embedding in the second step, obtaining the length m of the question-answer data, and then constructing a self-attention matrix with m rows and m columns;
setting the upper triangular part of the self-attention matrix in the third step as- ∞, and setting other position elements of the matrix as 0 as the input attention matrix of question-answer pairs;
fifthly, inputting the embedded sequence into a 12-layer transform network for training;
sixthly, generating candidate replies by using topK random decoding;
the compression module utilizes an alternative strategy to carry out knowledge distillation on the NEZHA model of the generation module, and the specific process is as follows,
inputting a question and answer data set obtained by a data processing module, loading the NEZHA model weight finely adjusted by a generating module, and marking as a ancestor layer;
secondly, setting the probability rr of replacing the ancestor layer by the inheritor layer to be 0.5, and compressing the layer number of the ancestor layer to be half of the original layer number;
thirdly, continuing training by using the question-answer pairs constructed by the data processing module, and then generating candidate answers by using the compressed inheritor layer;
and the dialogue management module is used for managing high-frequency user problems.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110588492.0A CN113220856A (en) | 2021-05-28 | 2021-05-28 | Multi-round dialogue system based on Chinese pre-training model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110588492.0A CN113220856A (en) | 2021-05-28 | 2021-05-28 | Multi-round dialogue system based on Chinese pre-training model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113220856A true CN113220856A (en) | 2021-08-06 |
Family
ID=77098976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110588492.0A Pending CN113220856A (en) | 2021-05-28 | 2021-05-28 | Multi-round dialogue system based on Chinese pre-training model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113220856A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113641789A (en) * | 2021-08-11 | 2021-11-12 | 福州大学 | Viewpoint retrieval method and system based on hierarchical fusion of multi-head attention network and convolutional network |
| CN114090757A (en) * | 2022-01-14 | 2022-02-25 | 阿里巴巴达摩院(杭州)科技有限公司 | Data processing method of dialogue system, electronic device and readable storage medium |
| CN114417892A (en) * | 2022-01-27 | 2022-04-29 | 北京中科深智科技有限公司 | Generation model of small sample multi-turn conversation for E-commerce live broadcast scene |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107247868A (en) * | 2017-05-18 | 2017-10-13 | 深思考人工智能机器人科技（北京）有限公司 | A kind of artificial intelligence aids in interrogation system |
| CN108280218A (en) * | 2018-02-07 | 2018-07-13 | 逸途（北京）科技有限公司 | A kind of flow system based on retrieval and production mixing question and answer |
| CN108399169A (en) * | 2017-02-06 | 2018-08-14 | 阿里巴巴集团控股有限公司 | Dialog process methods, devices and systems based on question answering system and mobile device |
| CN109858020A (en) * | 2018-12-29 | 2019-06-07 | 航天信息股份有限公司 | A kind of method and system obtaining taxation informatization problem answers based on grapheme |
| CN109933661A (en) * | 2019-04-03 | 2019-06-25 | 上海乐言信息科技有限公司 | It is a kind of that the semi-supervised question and answer of model are generated to inductive method and system based on depth |
| CN110046221A (en) * | 2019-03-01 | 2019-07-23 | 平安科技（深圳）有限公司 | A kind of machine dialogue method, device, computer equipment and storage medium |
| CN110309287A (en) * | 2019-07-08 | 2019-10-08 | 北京邮电大学 | The retrieval type of modeling dialog round information chats dialogue scoring method |
| CN110334347A (en) * | 2019-06-27 | 2019-10-15 | 腾讯科技（深圳）有限公司 | Information processing method, relevant device and storage medium based on natural language recognition |
| CN110347792A (en) * | 2019-06-25 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Talk with generation method and device, storage medium, electronic equipment |
| CN110362651A (en) * | 2019-06-11 | 2019-10-22 | 华南师范大学 | Dialogue method, system, device and the storage medium that retrieval and generation combine |
| CN110413761A (en) * | 2019-08-06 | 2019-11-05 | 浩鲸云计算科技股份有限公司 | A kind of method that the territoriality in knowledge based library is individually talked with |
| CN110516035A (en) * | 2019-07-05 | 2019-11-29 | 同济大学 | A kind of man-machine interaction method and system of mixing module |
| CN111177359A (en) * | 2020-04-10 | 2020-05-19 | 支付宝(杭州)信息技术有限公司 | Multi-turn dialogue method and device |
| CN112131367A (en) * | 2020-09-24 | 2020-12-25 | 民生科技有限责任公司 | Self-auditing man-machine conversation method, system and readable storage medium |
| CN112364150A (en) * | 2021-01-12 | 2021-02-12 | 南京云创大数据科技股份有限公司 | Intelligent question and answer method and system combining retrieval and generation |
| CN112541063A (en) * | 2020-12-08 | 2021-03-23 | 山东师范大学 | Man-machine conversation method and system based on self-learning conversation model |
-
2021
- 2021-05-28 CN CN202110588492.0A patent/CN113220856A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108399169A (en) * | 2017-02-06 | 2018-08-14 | 阿里巴巴集团控股有限公司 | Dialog process methods, devices and systems based on question answering system and mobile device |
| CN107247868A (en) * | 2017-05-18 | 2017-10-13 | 深思考人工智能机器人科技（北京）有限公司 | A kind of artificial intelligence aids in interrogation system |
| CN108280218A (en) * | 2018-02-07 | 2018-07-13 | 逸途（北京）科技有限公司 | A kind of flow system based on retrieval and production mixing question and answer |
| CN109858020A (en) * | 2018-12-29 | 2019-06-07 | 航天信息股份有限公司 | A kind of method and system obtaining taxation informatization problem answers based on grapheme |
| CN110046221A (en) * | 2019-03-01 | 2019-07-23 | 平安科技（深圳）有限公司 | A kind of machine dialogue method, device, computer equipment and storage medium |
| CN109933661A (en) * | 2019-04-03 | 2019-06-25 | 上海乐言信息科技有限公司 | It is a kind of that the semi-supervised question and answer of model are generated to inductive method and system based on depth |
| CN110362651A (en) * | 2019-06-11 | 2019-10-22 | 华南师范大学 | Dialogue method, system, device and the storage medium that retrieval and generation combine |
| CN110347792A (en) * | 2019-06-25 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Talk with generation method and device, storage medium, electronic equipment |
| CN110334347A (en) * | 2019-06-27 | 2019-10-15 | 腾讯科技（深圳）有限公司 | Information processing method, relevant device and storage medium based on natural language recognition |
| CN110516035A (en) * | 2019-07-05 | 2019-11-29 | 同济大学 | A kind of man-machine interaction method and system of mixing module |
| CN110309287A (en) * | 2019-07-08 | 2019-10-08 | 北京邮电大学 | The retrieval type of modeling dialog round information chats dialogue scoring method |
| CN110413761A (en) * | 2019-08-06 | 2019-11-05 | 浩鲸云计算科技股份有限公司 | A kind of method that the territoriality in knowledge based library is individually talked with |
| CN111177359A (en) * | 2020-04-10 | 2020-05-19 | 支付宝(杭州)信息技术有限公司 | Multi-turn dialogue method and device |
| CN112131367A (en) * | 2020-09-24 | 2020-12-25 | 民生科技有限责任公司 | Self-auditing man-machine conversation method, system and readable storage medium |
| CN112541063A (en) * | 2020-12-08 | 2021-03-23 | 山东师范大学 | Man-machine conversation method and system based on self-learning conversation model |
| CN112364150A (en) * | 2021-01-12 | 2021-02-12 | 南京云创大数据科技股份有限公司 | Intelligent question and answer method and system combining retrieval and generation |
Non-Patent Citations (2)
| Title |
|---|
| 廖胜兰: "面向服务机器人自然语言交互的语义解析方法研究", 《中国优秀博士学位论文全文数据库 信息科技辑》 * |
| 苏剑林: "动手做个DialoGPT：基于LM的生成式多轮对话模型", 《科学空间》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113641789A (en) * | 2021-08-11 | 2021-11-12 | 福州大学 | Viewpoint retrieval method and system based on hierarchical fusion of multi-head attention network and convolutional network |
| CN113641789B (en) * | 2021-08-11 | 2023-08-04 | 福州大学 | Viewpoint retrieval method and system based on hierarchical fusion multi-head attention network and convolution network |
| CN114090757A (en) * | 2022-01-14 | 2022-02-25 | 阿里巴巴达摩院(杭州)科技有限公司 | Data processing method of dialogue system, electronic device and readable storage medium |
| CN114417892A (en) * | 2022-01-27 | 2022-04-29 | 北京中科深智科技有限公司 | Generation model of small sample multi-turn conversation for E-commerce live broadcast scene |
| CN114417892B (en) * | 2022-01-27 | 2022-08-02 | 北京中科深智科技有限公司 | Generation model of small sample multi-turn conversation for E-commerce live broadcast scene |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781680B (en) | Semantic similarity matching method based on twin network and multi-head attention mechanism | |
| Zhu et al. | Simple is not easy: A simple strong baseline for textvqa and textcaps | |
| CN113591902B (en) | Cross-modal understanding and generating method and device based on multi-modal pre-training model | |
| Li et al. | Learnable aggregating net with diversity learning for video question answering | |
| CN113220856A (en) | Multi-round dialogue system based on Chinese pre-training model | |
| CN112100351A (en) | Method and equipment for constructing intelligent question-answering system through question generation data set | |
| Nguyen et al. | Efficient attention mechanism for visual dialog that can handle all the interactions between multiple inputs | |
| CN113297364B (en) | Natural language understanding method and device in dialogue-oriented system | |
| CN108628935A (en) | A kind of answering method based on end-to-end memory network | |
| CN110990555B (en) | End-to-end retrieval type dialogue method and system and computer equipment | |
| CN110781306A (en) | English text aspect layer emotion classification method and system | |
| CN111027292B (en) | Method and system for generating limited sampling text sequence | |
| Zhang et al. | Hierarchical scene parsing by weakly supervised learning with image descriptions | |
| CN112559706B (en) | Training method of dialogue generating model, dialogue method, device and storage medium | |
| CN116150335A (en) | Text semantic retrieval method under military scene | |
| Song et al. | LSTM-in-LSTM for generating long descriptions of images | |
| Chen et al. | News image captioning based on text summarization using image as query | |
| Song et al. | avtmNet: adaptive visual-text merging network for image captioning | |
| Dua et al. | Learning with instance bundles for reading comprehension | |
| Mou et al. | Multimodal dialogue state tracking by qa approach with data augmentation | |
| Karayil et al. | Conditional GANs for image captioning with sentiments | |
| Chen et al. | Multimodal fusion of visual dialog: A survey | |
| Vilalta et al. | Full-network embedding in a multimodal embedding pipeline | |
| Sharma et al. | A framework for image captioning based on relation network and multilevel attention mechanism | |
| Qin et al. | Towards complex scenarios: building end-to-end task-oriented dialogue system across multiple knowledge bases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210806 |