CN113239164A - Multi-round conversation process construction method and device, computer equipment and storage medium - Google Patents

Multi-round conversation process construction method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113239164A
CN113239164A CN202110520914.0A CN202110520914A CN113239164A CN 113239164 A CN113239164 A CN 113239164A CN 202110520914 A CN202110520914 A CN 202110520914A CN 113239164 A CN113239164 A CN 113239164A
Authority
CN
China
Prior art keywords
question
answer
constructing
conversation
robot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110520914.0A
Other languages
Chinese (zh)
Other versions
CN113239164B (en
Inventor
高鹏
康维鹏
袁兰
吴飞
周伟华
高峰
潘晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Mjoys Big Data Technology Co ltd
Original Assignee
Hangzhou Mjoys Big Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Mjoys Big Data Technology Co ltd filed Critical Hangzhou Mjoys Big Data Technology Co ltd
Priority to CN202110520914.0A priority Critical patent/CN113239164B/en
Publication of CN113239164A publication Critical patent/CN113239164A/en
Application granted granted Critical
Publication of CN113239164B publication Critical patent/CN113239164B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3343—Query execution using phonetics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/231—Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The embodiment of the invention discloses a method and a device for constructing a multi-round conversation process, computer equipment and a storage medium. The method comprises the following steps: collecting dialogue linguistic data; performing word segmentation and recognition on the dialogue corpus to obtain a recognition result; constructing a question answer set according to the recognition result; and constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set. By implementing the method provided by the embodiment of the invention, the efficient automatic construction of a higher-quality conversation process on the online data can be realized, and a conventional question-answer set is mined in a question-answer pair mining mode.

Description

Multi-round conversation process construction method and device, computer equipment and storage medium
Technical Field
The invention relates to the technical field of intelligent question answering, in particular to a method and a device for constructing a multi-turn conversation process, computer equipment and a storage medium.
Background
In an intelligent dialogue system, correct understanding and processing of a Question are key points of whole dialogue interaction, and the key points need to be established on the basis of a huge knowledge base, the construction of the current knowledge base comprises entity words, entity relations, QA (Question Answering) pairs, similar questions, a dialogue process and the like, the method of combining manual examination and machine statistics is basically adopted, the mining of the entity words, the entity relations and the Question sentences is mainly focused, and the automatic mining construction of the dialogue process is very limited in practical application.
However, in the actual production environment, a large amount of dialogue linguistic data exist, for example, in a bank system, centralized outbound or customer service agents exist, and the huge amount of dialogue linguistic data are collected after long-term accumulation and precipitation, and have very strong bank domain characteristics, but at present, no method is available for deeply and better utilizing the dialogue knowledge linguistic data, so that an intelligent dialogue system knowledge base is automatically enriched and perfected, and a dialogue flow with the domain characteristics is automatically constructed.
Therefore, it is necessary to design a new method to implement effective and automatic construction of a higher-quality dialogue flow for online data, and to mine a conventional question-answer set by a question-answer pair mining method.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a device for constructing a multi-turn conversation process, a computer device and a storage medium.
In order to achieve the purpose, the invention adopts the following technical scheme: the multi-round conversation process construction method comprises the following steps:
collecting dialogue linguistic data;
performing word segmentation and recognition on the dialogue corpus to obtain a recognition result;
constructing a question answer set according to the recognition result;
and constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set.
The further technical scheme is as follows: the collecting dialogue corpus includes:
acquiring a conversation recording file;
carrying out voice recognition on the conversation sound recording file to obtain conversation stream data;
capturing question-answer pairs through a crawler technology;
and integrating the conversation streaming data and the question-answer pairs to obtain a conversation corpus.
The further technical scheme is as follows: the segmenting and recognizing the dialogue corpus to obtain a recognition result comprises the following steps:
and performing word segmentation and entity recognition on the dialogue corpus by using a Jieba word segmentation tool to obtain a recognition result.
The further technical scheme is as follows: the step of constructing a question answer set according to the recognition result comprises the following steps:
constructing a user question set according to the identification result;
constructing a robot response set corresponding to the user question set by adopting a clustering analysis mode;
the question answer set comprises a user question set and a robot answer set corresponding to the user question set.
The further technical scheme is as follows: the method for constructing the robot response set corresponding to the user question set by adopting a cluster analysis mode comprises the following steps:
and constructing a robot response set corresponding to the user question set by adopting a Canopy clustering algorithm.
The further technical scheme is as follows: the method for constructing the robot response set corresponding to the user question set by adopting the Canopy clustering algorithm comprises the following steps:
determining a first similarity threshold and a second similarity threshold;
initializing a user question set and initializing a clustering result set;
randomly selecting a question and creating a cluster centered on the question;
traversing each answer in the user question set, and calculating the distance from the answer to each cluster;
classifying the answers with the distance smaller than a first similarity threshold value into the clusters, deleting the answers with the distance smaller than a second similarity threshold value from the user, and determining the question as a new cluster center point when the distance from the answer corresponding to the question to each cluster is larger than the first similarity threshold value;
and determining the cluster as a robot response set corresponding to the user question set.
The further technical scheme is as follows: the establishing of the conversation process tree according to the question answer set in a hierarchical clustering mode comprises the following steps:
acquiring a robot question and an answer of a starting node;
obtaining answers of subsequent robots, and constructing clustering branches for each robot by adopting a Canopy clustering algorithm;
and acquiring response branch response nodes of the robot, and labeling the response nodes which belong to the question nodes but have no answer.
The invention also provides a multi-round conversation process constructing device, which comprises:
the corpus collection unit is used for collecting dialogue corpuses;
the recognition unit is used for performing word segmentation and recognition on the dialogue corpus to obtain a recognition result;
the set construction unit is used for constructing a question answer set according to the identification result;
and the flow tree construction unit is used for constructing a conversation flow tree by adopting a hierarchical clustering mode according to the question answer set.
The invention also provides computer equipment which comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the method when executing the computer program.
The invention also provides a storage medium storing a computer program which, when executed by a processor, is operable to carry out the method as described above.
Compared with the prior art, the invention has the beneficial effects that: the method and the device have the advantages that the existing dialogue linguistic data are collected, the words of the dialogue linguistic data are segmented and recognized, the question answer set is constructed on the basis, the dialogue process tree is constructed on the basis of the question answer set in a hierarchical clustering mode, the efficient automatic construction of a high-quality dialogue process for the on-line data is realized, and the conventional question-answer set is mined in a question-answer pair mining mode.
The invention is further described below with reference to the accompanying drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 3 is a sub-flow diagram of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 4 is a sub-flow diagram of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 5 is a sub-flow diagram of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 6 is a sub-flow diagram of a multi-round conversation process building method according to an embodiment of the present invention;
fig. 7 is a schematic flow chart of a Canopy clustering algorithm according to an embodiment of the present invention;
fig. 8 is a schematic flow chart of a dialog flow tree construction provided in the embodiment of the present invention;
FIG. 9 is a schematic block diagram of a multi-round conversation process building apparatus provided by an embodiment of the present invention;
FIG. 10 is a schematic block diagram of a corpus collection unit of a multi-round conversation process building apparatus according to an embodiment of the present invention;
FIG. 11 is a schematic block diagram of a set building unit of a multi-round conversation process building apparatus according to an embodiment of the present invention;
FIG. 12 is a schematic block diagram of a response set constructing subunit of the multi-round conversation process constructing apparatus according to the embodiment of the present invention;
fig. 13 is a schematic block diagram of a flow tree building unit of the multi-round conversation flow building apparatus according to the embodiment of the present invention;
FIG. 14 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a multi-round conversation process building method according to an embodiment of the present invention. Fig. 2 is a schematic flowchart of a multi-round conversation process building method according to an embodiment of the present invention. The multi-round conversation process construction method is applied to a server, the server performs data interaction with a robot and a terminal, after conversation linguistic data are obtained from the robot and the terminal, the linguistic data are identified, a question answer set is constructed, and a conversation process tree is constructed from the question answer set.
Fig. 2 is a flow diagram of a multi-round conversation flow building method according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S140.
And S110, collecting dialogue linguistic data.
In this embodiment, the dialogue corpus refers to dialogue contents between a robot and a client or between a customer service and a client in a banking system.
In an embodiment, referring to fig. 3, the step S110 may include steps S111 to S114.
And S111, acquiring the conversation sound recording file.
In this embodiment, the session recording file refers to a recording file of a manual outgoing call or a customer service telephone session of the bank system, and the manual outgoing call or the customer service telephone session of the bank system generally exists in a manner of a session recording file.
And S112, carrying out voice recognition on the conversation sound recording file to obtain conversation stream data.
In the present embodiment, the dialogue flow data is data obtained by performing speech recognition on the dialogue record file, and is used to distinguish information such as dialogue order and dialogue time of the two speakers.
Specifically, a speech recognition technology is adopted to convert the conversation sound recording into conversation stream data in a text form, and information such as conversation sequence, time and the like of two speakers is distinguished. Common methods used in speech recognition technology are linguistic and acoustic based methods; a random model method; a method using an artificial neural network; probability grammar analysis; the details of the four methods for converting the dialogue record into the dialogue stream data in the text form belong to the prior art, and are not described herein again.
And S113, grabbing question and answer pairs through a crawler technology.
In this embodiment, the question-answering and comment data in the bank community-type question-answering system can be obtained by a crawler specially grabbing a message board of a Baidu, Zhi and Bank homepage website.
And S114, integrating the conversation streaming data and the question-answer pairs to obtain conversation corpora.
In this embodiment, the robot calls out the conversational data of the system, which are the conversation between the telephone robot and the client, while there may be some deficiencies in the conversational answer of the telephone robot, the other party of the conversation is a natural person, who can give reasonably correct feedback according to the current robot answer, even including complaints or even emotional answers that the other party cannot understand the own answer. This is also the question that needs to be solved to build knowledge, which is a better answer to the customer's question.
The knowledge domain of banking business, such as account password, asset query, credit card, deposit and financing, loan repayment, regulation policy, etc., can be basically covered by the above-mentioned reorganization of these question and answer linguistic data. The following is a corpus data example of a one-way conversation in a telephone outbound system:
the robot comprises: your good, i here is the customer manager of bank, you are the premium customers in my bank, there is a deposit product with good income worth paying attention, the demand for starting deposit is low, but the earning rate is similar to a large-volume deposit list, and can be bought at any time, i introduce you to get down in a bar? (R131)
The user: that Moba I went to a lap bar tomorrow. (Y3)
The robot comprises: the Bohai deposit reservation is an innovative deposit released by the Bohai bank of China, can be bought at any time in advance, has the highest annual income rate of 4.125%, enables a customer to obtain high income without influencing the urgent use of capital, and enables a customer manager in charge of the Bohai deposit reservation to contact the customer later to introduce a detail of a bar? (R134)
The user: that prayer comes back, but i are now out. (Q32)
The robot comprises: you can transact the network through an online bank, a mobile phone bank or a nearby network. (R156)
The user: can the mobile bank of that mobile bank also find, is bar? (Q29)
The robot comprises: can be handled by a mobile phone bank. Will you give you a detailed introduction to do you not later let me go to my customer manager and you? (R157)
The user: the rows are rows by rows. (Y3)
The robot comprises: good, thank you for support, later I go to customer's regular meeting to contact you, please note to answer, congratulate you for life pleasure, see again! (R135)
Converting according to the sound recording file, and marking conversation contents of two parties of the conversation, wherein for example Q, R, Y, Q is a question; r is answer, Y is positive answer, so that the corresponding relation between the question and the answer can be obtained.
And S120, performing word segmentation and recognition on the dialogue corpus to obtain a recognition result.
In this embodiment, the recognition result refers to the meaning of each word in each dialog content of the dialog corpus, such as an adjective.
Specifically, word segmentation and entity recognition are carried out on the dialogue corpus by using a Jieba word segmentation tool to obtain a recognition result.
The method comprises the steps that a word is a semantic basic unit, word segmentation and entity recognition are carried out by utilizing an open-source Jieba word segmentation tool, part of speech recognition such as nouns, verbs, adjectives, adverbs and auxiliary words is mainly carried out on each text in a dialogue corpus, and entity phrase words and the like are recognized. Since individual words or phrases are converted into homonymous words or wrongly written words in the telephone speech, the word spelling information and the like are also analyzed during the lexical analysis. In addition, potential ngram phrases are identified according to some rules.
The following are examples of word segmentation results:
our/r this/r/q "/wp Bohai/j reserve/n"/wp is/v me/n push out/v/u innovative/b reserve/v deposit/n,/wp can/v get ahead/v draw ahead/v,/wp max/a annualization/v profitability/n can/v reach/v 4.125%/m,/wp let/v you/r get/v high/a profit/n/u while/n,/wp again/d does not/d affect/v you/r fund/n rush use/v,/wp me/r let/v be responsible for/v this block/r/u customer/n principle/n later/d link/v you/r And/wp detail/a introduction/v at once/m bar/u? /wp
After each word of each dialogue content is split, the meaning is divided, and therefore each dialogue content of the dialogue corpus can be identified.
And S130, constructing a question answer set according to the recognition result.
In this embodiment, the question-answer set refers to a set formed by one-to-one correspondence of questions and responses.
The historical dialogue corpus contains a large number of question-answering dialogues. The following examples contain question-answer pairs.
User (B): can the mobile bank of that mobile bank also find, is bar? (Q29)
Robot (a): can be handled by a mobile phone bank. Will you give you a detailed introduction to do you not later let me go to my customer manager and you? (R157)
In an embodiment, referring to fig. 4, the step S130 may include steps S131 to S132.
S131, constructing a user question set according to the identification result.
In this embodiment, the user question set refers to all questions initiated by the client.
Specifically, a question classifier needs to be constructed to judge whether the user text in the recognition result is an effective question or not, and identify and screen out potential effective questions in all dialog corpuses. The training corpus of the question recognition classifier is sorted by combining rules and manual inspection, and the rules for question discrimination mainly discriminate whether question sentences contain question marks, whether question words, whether verb words and whether specific rule sentence patterns are met or not. The classification type Y is {0,1}, each user question X is converted into a TF-IDF (term frequency-inverse document frequency) feature representation based on question words, and a prepared sample T is { (X)1,Y1),(X2,Y2)(X3,Y3) ... } N samples, the SVM classification hyperplane (dividing line) is wx + b ═ 0; the geometric distance from the sample point to the hyperplane is:
Figure BDA0003063929100000081
the goal of SVM (Support Vector Machine) training is to find the hyperplane corresponding to the largest value of all outer margins. Thus, in mathematical language, w, b are determined so that the outer edge distance is maximized:
Figure BDA0003063929100000082
thereby determining a user question set for the user.
And S132, constructing a robot response set corresponding to the user question set by adopting a clustering analysis mode.
In this embodiment, the robot answer set refers to an answer set corresponding to a set of answers to a user question.
The question answer set comprises a user question set and a robot answer set corresponding to the user question set.
In this embodiment, as shown in fig. 7, a robot response set corresponding to the user question set is constructed by using a Canopy clustering algorithm.
In one embodiment, referring to fig. 5, the step S132 may include steps S1321 to S1326.
S1321, determining a first similarity threshold and a second similarity threshold.
In this embodiment, after question recognition, the answer to the question needs to be mined. Considering that the effective question-answer pairs are in a question-answer form, the original positions of actually distinguished question sentences are traced, the text dialogues next to the text dialogues are used as potential answers of the question sentences, and the frequency of occurrence of a question-answer is relatively high, so that a cluster analysis mode is needed to directly gather potential question answers to the same category. I will answer the following as a basic answer to the question to sample X. In this embodiment, a Canopy clustering algorithm is adopted to perform clustering, and it is necessary to determine the first similarity threshold t1 and the second similarity threshold t2, t1> t 2.
S1322, initializing a user question set and initializing a clustering result set.
In this embodiment, a question and answer a in the user question set is initialized, the text a below is vectorized and placed in a, and a clustering result set Canopy is initialized.
S1323, randomly selecting a question, and creating a cluster centered on the question.
In this embodiment, a question qa is randomly selected, and a cluster canopy centered on the question qa is created.
S1324, traversing each answer in the user question set, and calculating the distance from the answer to each cluster.
In this embodiment, traverse each answer a in the remaining set of user questions, calculate the distance d from the answer to each cluster.
S1325, classifying the answers with the distance smaller than the first similarity threshold value into the clusters, deleting the answers with the distance smaller than the second similarity threshold value from the user, and determining the question as a new cluster center point when the distance from the answer corresponding to the question to each cluster is larger than the first similarity threshold value.
Specifically, if d < t1, the problem qa is classified into the cluster; if d < t2, deleting the question from the user question set; if for the problem QA, each d is d > t1, the problem QA is the center point of a new cluster, i.e., the problem QA is a potential new problem QA. The above steps S1324 and S1325 are repeated until the user question set is empty. Thus each cluster is a set containing a plurality of similar questions and answers.
S1326, determining the cluster as a robot response set corresponding to the user question set.
After clustering is completed, the potential high-quality answer can be obtained by using the cluster with higher convergence similarity average value or larger aggregation category number. Each clustered big question or answer text represents the size of the cluster, which can be used in the transition probability calculation.
And S140, constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set.
In this embodiment, the dialog flow tree refers to a result formed by building questions and answers according to a tree.
As shown in FIG. 8, a dialog knowledge base is in the form of a tree hierarchy, and a question of a user corresponds to an answer of a customer service; the user can extend or further ask the relevant service problem, so that the dialogue is developed into a dialogue tree hierarchy, the answering dialogs of the robot are regarded as nodes, the answering dialogs of the user are regarded as transition edges, and one answering dialog of the robot can lead out a plurality of different user answering interactions. Therefore, the establishment of the dialog hierarchical tree is to construct a transfer relationship between nodes, and each node has a plurality of child nodes. In addition, because the probability of transferring one speech operation of the robot to another speech operation is not completely consistent, the constructed dialogue logic diagram transfer tree is also probabilistic.
In an embodiment, referring to fig. 6, the step S140 may include steps S141 to S143.
And S141, acquiring the robot question and answer of the starting node.
Specifically, the first sentence questions and answers which are robots in all the dialogue records are clustered, so that the subsequent branches of the starting nodes are obtained.
And S142, obtaining answers of subsequent robots, and constructing clustering branches for each robot by adopting a Canopy clustering algorithm.
Specifically, for each clustering branch of the robot, different responses of subsequent users are firstly obtained, and a Canopy clustering algorithm is adopted by a specific clustering method. If the user here hangs up the phone, such states are classified individually as special states. And further mining the speeches of different users by adopting a Canopy clustering method so as to obtain the question and answer or question of the users.
And S143, acquiring response branch response nodes of the robot, and marking the response nodes which belong to the question nodes but have no answer.
For the response of the user, when the subsequent robot response branch response node is obtained, whether the subsequent robot response branch response node is a question node or not needs to be judged according to a question classification discriminator, and if the subsequent robot response branch response node is a question node but has no answer, the question node needs to be subjected to key labeling to indicate that the current user question does not have correct response. After hierarchical clustering is completed, the transition probability between the cluster Q1 and question cluster Q2 can be calculated, and the transition probability is based on the cluster size.
According to the multi-round conversation process construction method, the existing conversation corpus is collected, the conversation corpus is segmented and recognized, the question answer set is constructed on the basis, the conversation process tree is constructed on the basis of the question answer set in a hierarchical clustering mode, the efficient automatic construction of a high-quality conversation process for online data is achieved, and the conventional question-answer set is mined in a question-answer pair mining mode.
Fig. 9 is a schematic block diagram of a multi-turn conversation process building apparatus 300 according to an embodiment of the present invention. As shown in fig. 9, the present invention further provides a multi-round conversation process constructing apparatus 300 corresponding to the above multi-round conversation process constructing method. The multi-round conversation process constructing apparatus 300, which includes a unit for performing the above-described multi-round conversation process constructing method, may be configured in a server. Specifically, referring to fig. 9, the multi-turn conversation process constructing apparatus 300 includes a corpus collecting unit 301, a recognizing unit 302, a set constructing unit 303, and a process tree constructing unit 304.
A corpus collection unit 301, configured to collect dialog corpuses; the recognition unit 302 is configured to perform word segmentation and recognition on the dialog corpus to obtain a recognition result; a set construction unit 303, configured to construct a question answer set according to the recognition result; and a flow tree construction unit 304, configured to construct a conversation flow tree according to the question answer set in a hierarchical clustering manner.
In one embodiment, as shown in fig. 10, the corpus collection unit 301 includes a recording file acquisition subunit 13011, a speech recognition subunit 3012, a crawling subunit 3013, and an integration subunit 3014.
A recording file acquisition subunit 13011 configured to acquire a session recording file; a voice recognition subunit 3012, configured to perform voice recognition on the session recording file to obtain session stream data; a crawling subunit 3013, configured to crawl question-answer pairs through a crawler technology; the integrating subunit 3014 is configured to integrate the dialog flow data and the question-answer pairs to obtain a dialog corpus.
In an embodiment, the identifying unit 302 is configured to perform word segmentation and entity identification on the dialog corpus by using a Jieba word segmentation tool to obtain an identification result.
In one embodiment, as shown in fig. 11, the set constructing unit 303 includes a question set constructing sub-unit 3031 and a response set constructing sub-unit 3032.
A question set constructing subunit 3031, configured to construct a user question set according to the recognition result; a response set constructing subunit 3032, configured to construct a robot response set corresponding to the user question set in a cluster analysis manner; the question answer set comprises a user question set and a robot answer set corresponding to the user question set.
In an embodiment, the answer set constructing subunit 3032 is configured to construct a robot answer set corresponding to the user question set by using a Canopy clustering algorithm.
In one embodiment, as shown in fig. 12, the answer set constructing sub-unit 3032 includes a threshold determining module 30321, an initialization module 30322, a cluster constructing module 30323, a traversing module 30324, a classifying module 30325 and a determining module 30326.
A threshold determination module 30321, configured to determine a first similarity threshold and a second similarity threshold; an initialization module 30322, configured to initialize a user question set and initialize a clustering result set; a cluster construction module 30323, configured to randomly select a question and create a cluster centered on the question; a traversing module 30324, configured to traverse each answer in the user question set, and calculate a distance from the answer to each cluster; a classifying module 30325, configured to classify the answer with the distance smaller than the first similarity threshold into the clusters, delete the answer with the distance smaller than the second similarity threshold from the user, and determine that the question is a new cluster center point when the distance from the answer corresponding to the question to each cluster is greater than the first similarity threshold; a determining module 30326, configured to determine that the cluster is a robot response set corresponding to the user question set.
In an embodiment, as shown in fig. 13, the flow tree construction unit 304 includes an obtaining subunit 3041, a branch construction subunit 3042, and a labeling subunit 3043.
An obtaining subunit 3041, configured to obtain a robot question and an answer of the start node; a branch construction subunit 3042, configured to obtain answers of subsequent robots, and construct a clustering branch for each robot by using a Canopy clustering algorithm; and a labeling subunit 3043, configured to obtain response branch response nodes of the robot, and label response nodes that belong to question nodes but do not have answers.
It should be noted that, as can be clearly understood by those skilled in the art, the specific implementation processes of the multi-round conversation process constructing apparatus 300 and each unit may refer to the corresponding descriptions in the foregoing method embodiments, and for convenience and brevity of description, no further description is provided herein.
The multi-turn conversation process building apparatus 300 described above may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 14.
Referring to fig. 14, fig. 14 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 is a server, wherein the server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 14, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer programs 5032 include program instructions that, when executed, cause the processor 502 to perform a multi-pass conversation process construction method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be caused to execute a multi-round conversation process building method.
The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the configuration shown in fig. 14 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation of the computer device 500 to which the present application is applied, and that a particular computer device 500 may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:
collecting dialogue linguistic data; performing word segmentation and recognition on the dialogue corpus to obtain a recognition result; constructing a question answer set according to the recognition result; and constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set.
In an embodiment, when the processor 502 implements the step of collecting dialog corpus, the following steps are specifically implemented:
acquiring a conversation recording file; carrying out voice recognition on the conversation sound recording file to obtain conversation stream data; capturing question-answer pairs through a crawler technology; and integrating the conversation streaming data and the question-answer pairs to obtain a conversation corpus.
In an embodiment, when the processor 502 implements the step of performing word segmentation and recognition on the dialog corpus to obtain a recognition result, the following steps are specifically implemented:
and performing word segmentation and entity recognition on the dialogue corpus by using a Jieba word segmentation tool to obtain a recognition result.
In an embodiment, when the processor 502 implements the step of constructing the answer set of questions according to the recognition result, the following steps are specifically implemented:
constructing a user question set according to the identification result; constructing a robot response set corresponding to the user question set by adopting a clustering analysis mode; the question answer set comprises a user question set and a robot answer set corresponding to the user question set.
In an embodiment, when the processor 502 implements the step of constructing the robot answer set corresponding to the user question set by using the cluster analysis method, the following steps are specifically implemented:
and constructing a robot response set corresponding to the user question set by adopting a Canopy clustering algorithm.
In an embodiment, when the step of constructing the robot answer set corresponding to the user question set by using the Canopy clustering algorithm is implemented, the processor 502 specifically implements the following steps:
determining a first similarity threshold and a second similarity threshold; initializing a user question set and initializing a clustering result set; randomly selecting a question and creating a cluster centered on the question; traversing each answer in the user question set, and calculating the distance from the answer to each cluster; classifying the answers with the distance smaller than a first similarity threshold value into the clusters, deleting the answers with the distance smaller than a second similarity threshold value from the user, and determining the question as a new cluster center point when the distance from the answer corresponding to the question to each cluster is larger than the first similarity threshold value; and determining the cluster as a robot response set corresponding to the user question set.
In an embodiment, when implementing the step of constructing the dialog flow tree by using a hierarchical clustering method according to the question answer set, the processor 502 specifically implements the following steps:
acquiring a robot question and an answer of a starting node; obtaining answers of subsequent robots, and constructing clustering branches for each robot by adopting a Canopy clustering algorithm; and acquiring response branch response nodes of the robot, and labeling the response nodes which belong to the question nodes but have no answer.
It should be understood that in the embodiment of the present Application, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be understood by those skilled in the art that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program instructing associated hardware. The computer program includes program instructions, and the computer program may be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the steps of:
collecting dialogue linguistic data; performing word segmentation and recognition on the dialogue corpus to obtain a recognition result; constructing a question answer set according to the recognition result; and constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set.
In an embodiment, when the processor executes the computer program to implement the step of collecting dialog corpus, the following steps are specifically implemented:
acquiring a conversation recording file; carrying out voice recognition on the conversation sound recording file to obtain conversation stream data; capturing question-answer pairs through a crawler technology; and integrating the conversation streaming data and the question-answer pairs to obtain a conversation corpus.
In an embodiment, when the processor executes the computer program to implement the step of performing word segmentation and recognition on the dialog corpus to obtain a recognition result, the following steps are specifically implemented:
and performing word segmentation and entity recognition on the dialogue corpus by using a Jieba word segmentation tool to obtain a recognition result.
In an embodiment, when the step of constructing the answer set of questions according to the recognition result is implemented by the processor by executing the computer program, the following steps are specifically implemented:
constructing a user question set according to the identification result; constructing a robot response set corresponding to the user question set by adopting a clustering analysis mode; the question answer set comprises a user question set and a robot answer set corresponding to the user question set.
In an embodiment, when the processor executes the computer program to implement the step of constructing the robot answer set corresponding to the user question set in a cluster analysis manner, the following steps are specifically implemented:
and constructing a robot response set corresponding to the user question set by adopting a Canopy clustering algorithm.
In an embodiment, when the processor executes the computer program to implement the step of constructing the robot answer set corresponding to the user question set by using the Canopy clustering algorithm, the following steps are specifically implemented:
determining a first similarity threshold and a second similarity threshold; initializing a user question set and initializing a clustering result set; randomly selecting a question and creating a cluster centered on the question; traversing each answer in the user question set, and calculating the distance from the answer to each cluster; classifying the answers with the distance smaller than a first similarity threshold value into the clusters, deleting the answers with the distance smaller than a second similarity threshold value from the user, and determining the question as a new cluster center point when the distance from the answer corresponding to the question to each cluster is larger than the first similarity threshold value; and determining the cluster as a robot response set corresponding to the user question set.
In an embodiment, when the processor executes the computer program to implement the step of constructing the dialog flow tree according to the question answer set in a hierarchical clustering manner, the following steps are specifically implemented:
acquiring a robot question and an answer of a starting node; obtaining answers of subsequent robots, and constructing clustering branches for each robot by adopting a Canopy clustering algorithm; and acquiring response branch response nodes of the robot, and labeling the response nodes which belong to the question nodes but have no answer.
The storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, which can store various computer readable storage media.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be merged, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. The multi-round conversation process construction method is characterized by comprising the following steps:
collecting dialogue linguistic data;
performing word segmentation and recognition on the dialogue corpus to obtain a recognition result;
constructing a question answer set according to the recognition result;
and constructing a conversation process tree by adopting a hierarchical clustering mode according to the question answer set.
2. The method for building multi-turn conversation process according to claim 1, wherein said collecting conversation corpus comprises:
acquiring a conversation recording file;
carrying out voice recognition on the conversation sound recording file to obtain conversation stream data;
capturing question-answer pairs through a crawler technology;
and integrating the conversation streaming data and the question-answer pairs to obtain a conversation corpus.
3. The multi-round conversation process construction method according to claim 1, wherein the segmenting and recognizing the conversation corpus to obtain a recognition result comprises:
and performing word segmentation and entity recognition on the dialogue corpus by using a Jieba word segmentation tool to obtain a recognition result.
4. The multi-round conversation process construction method according to claim 1, wherein said constructing a question answer set according to the recognition result comprises:
constructing a user question set according to the identification result;
constructing a robot response set corresponding to the user question set by adopting a clustering analysis mode;
the question answer set comprises a user question set and a robot answer set corresponding to the user question set.
5. The multi-round conversation process construction method according to claim 4, wherein the constructing the robot response set corresponding to the user question set by using a cluster analysis method comprises:
and constructing a robot response set corresponding to the user question set by adopting a Canopy clustering algorithm.
6. The method for building the multi-round conversation process according to claim 5, wherein the building the robot response set corresponding to the user question set by using a Canopy clustering algorithm comprises:
determining a first similarity threshold and a second similarity threshold;
initializing a user question set and initializing a clustering result set;
randomly selecting a question and creating a cluster centered on the question;
traversing each answer in the user question set, and calculating the distance from the answer to each cluster;
classifying the answers with the distance smaller than a first similarity threshold value into the clusters, deleting the answers with the distance smaller than a second similarity threshold value from the user, and determining the question as a new cluster center point when the distance from the answer corresponding to the question to each cluster is larger than the first similarity threshold value;
and determining the cluster as a robot response set corresponding to the user question set.
7. The multi-round conversation process construction method according to claim 1, wherein constructing a conversation process tree according to the question answer set in a hierarchical clustering manner comprises:
acquiring a robot question and an answer of a starting node;
obtaining answers of subsequent robots, and constructing clustering branches for each robot by adopting a Canopy clustering algorithm;
and acquiring response branch response nodes of the robot, and labeling the response nodes which belong to the question nodes but have no answer.
8. The multi-round conversation process constructing device is characterized by comprising the following steps:
the corpus collection unit is used for collecting dialogue corpuses;
the recognition unit is used for performing word segmentation and recognition on the dialogue corpus to obtain a recognition result;
the set construction unit is used for constructing a question answer set according to the identification result;
and the flow tree construction unit is used for constructing a conversation flow tree by adopting a hierarchical clustering mode according to the question answer set.
9. A computer device, characterized in that the computer device comprises a memory, on which a computer program is stored, and a processor, which when executing the computer program implements the method according to any of claims 1 to 8.
10. A storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.
CN202110520914.0A 2021-05-13 2021-05-13 Multi-round dialogue flow construction method and device, computer equipment and storage medium Active CN113239164B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110520914.0A CN113239164B (en) 2021-05-13 2021-05-13 Multi-round dialogue flow construction method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110520914.0A CN113239164B (en) 2021-05-13 2021-05-13 Multi-round dialogue flow construction method and device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113239164A true CN113239164A (en) 2021-08-10
CN113239164B CN113239164B (en) 2023-07-04

Family

ID=77134175

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110520914.0A Active CN113239164B (en) 2021-05-13 2021-05-13 Multi-round dialogue flow construction method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113239164B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015058604A1 (en) * 2013-10-21 2015-04-30 北京奇虎科技有限公司 Apparatus and method for obtaining degree of association of question and answer pair and for search ranking optimization
CN105335400A (en) * 2014-07-22 2016-02-17 阿里巴巴集团控股有限公司 Method and apparatus for obtaining answer information for questioning intention of user
CN106294321A (en) * 2016-08-04 2017-01-04 北京智能管家科技有限公司 The dialogue method for digging of a kind of specific area and device
US20170308791A1 (en) * 2014-10-22 2017-10-26 Baidu Online Network Technology (Beijing) Co., Ltd. Multi-round session interaction method and system, and computer device
US20180341870A1 (en) * 2017-05-23 2018-11-29 International Business Machines Corporation Managing Indecisive Responses During a Decision Tree Based User Dialog Session
US20190182382A1 (en) * 2017-12-13 2019-06-13 Genesys Telecomminications Laboratories, Inc. Systems and methods for chatbot generation
CN111274378A (en) * 2020-02-13 2020-06-12 南京云问网络技术有限公司 Data processing method and device for question answering, equipment and storage medium
US20200257861A1 (en) * 2019-02-13 2020-08-13 Oracle International Corporation Chatbot conducting a virtual social dialogue
US20200286463A1 (en) * 2017-05-10 2020-09-10 Oracle International Corporation Using communicative discourse trees to create a virtual persuasive dialogue
CN111767366A (en) * 2019-04-01 2020-10-13 北京百度网讯科技有限公司 Question and answer resource mining method and device, computer equipment and storage medium

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015058604A1 (en) * 2013-10-21 2015-04-30 北京奇虎科技有限公司 Apparatus and method for obtaining degree of association of question and answer pair and for search ranking optimization
CN105335400A (en) * 2014-07-22 2016-02-17 阿里巴巴集团控股有限公司 Method and apparatus for obtaining answer information for questioning intention of user
US20170308791A1 (en) * 2014-10-22 2017-10-26 Baidu Online Network Technology (Beijing) Co., Ltd. Multi-round session interaction method and system, and computer device
CN106294321A (en) * 2016-08-04 2017-01-04 北京智能管家科技有限公司 The dialogue method for digging of a kind of specific area and device
US20200286463A1 (en) * 2017-05-10 2020-09-10 Oracle International Corporation Using communicative discourse trees to create a virtual persuasive dialogue
US20180341870A1 (en) * 2017-05-23 2018-11-29 International Business Machines Corporation Managing Indecisive Responses During a Decision Tree Based User Dialog Session
US20190182382A1 (en) * 2017-12-13 2019-06-13 Genesys Telecomminications Laboratories, Inc. Systems and methods for chatbot generation
US20200257861A1 (en) * 2019-02-13 2020-08-13 Oracle International Corporation Chatbot conducting a virtual social dialogue
CN111767366A (en) * 2019-04-01 2020-10-13 北京百度网讯科技有限公司 Question and answer resource mining method and device, computer equipment and storage medium
CN111274378A (en) * 2020-02-13 2020-06-12 南京云问网络技术有限公司 Data processing method and device for question answering, equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
周小强;王晓龙;陈清财;: "交互式问答的关系结构体系及标注", 中文信息学报, no. 05 *

Also Published As

Publication number Publication date
CN113239164B (en) 2023-07-04

Similar Documents

Publication Publication Date Title
US11249774B2 (en) Realtime bandwidth-based communication for assistant systems
WO2020244073A1 (en) Speech-based user classification method and device, computer apparatus, and storage medium
US20220337538A1 (en) Customized message suggestion with user embedding vectors
US8639517B2 (en) Relevance recognition for a human machine dialog system contextual question answering based on a normalization of the length of the user input
US20210350209A1 (en) Intent and context-aware dialogue based virtual assistance
CN112818109B (en) Intelligent reply method, medium, device and computing equipment for mail
CN110929043A (en) Service problem extraction method and device
WO2021036439A1 (en) Method for responding to complaint, and device
CN115274086B (en) Intelligent diagnosis guiding method and system
US20230395075A1 (en) Human-machine dialogue system and method
CN111429157A (en) Method, device and equipment for evaluating and processing complaint work order and storage medium
CN113407677A (en) Method, apparatus, device and storage medium for evaluating quality of consultation session
CN110399473B (en) Method and device for determining answers to user questions
US11563852B1 (en) System and method for identifying complaints in interactive communications and providing feedback in real-time
Ali et al. K-means clustering to improve the accuracy of decision tree response classification
WO2023207566A1 (en) Voice room quality assessment method, apparatus, and device, medium, and product
TWI734085B (en) Dialogue system using intention detection ensemble learning and method thereof
CN111143508A (en) Event detection and tracking method and system based on communication short text
CN113239164B (en) Multi-round dialogue flow construction method and device, computer equipment and storage medium
WO2020091619A1 (en) Automated assessment of the quality of a dialogue system in real time
CN111818290B (en) Online interviewing method and system
CN114239602A (en) Session method, apparatus and computer program product
CN113094471A (en) Interactive data processing method and device
KR102540562B1 (en) Method to analyze consultation data
CN117453895B (en) Intelligent customer service response method, device, equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant