CN114519093A - Question answering method, device, electronic equipment and computer readable storage medium - Google Patents
Question answering method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114519093A CN114519093A CN202210116820.1A CN202210116820A CN114519093A CN 114519093 A CN114519093 A CN 114519093A CN 202210116820 A CN202210116820 A CN 202210116820A CN 114519093 A CN114519093 A CN 114519093A
- Authority
- CN
- China
- Prior art keywords
- target
- question
- reply
- theme
- topic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Finance (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a question answering method, a question answering device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: the method comprises the steps of receiving target questions of a client, classifying according to semantics of the target questions to obtain target topics to which the target questions belong and confidence degrees that the target questions belong to the target topics, replying the target questions by adopting a question-answering model corresponding to the target topics to obtain target replies, sending the target replies to the client when the confidence degrees are larger than a set threshold, and sending the target replies to a customer service working end as reply suggestions when the confidence degrees are smaller than or equal to the set threshold.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a question answering method, a question answering device, an electronic device, and a computer-readable storage medium.
Background
With the development of the internet and electronic commerce, many electronic commerce or websites provide online customer service to solve the confusion of users. The online customer service extracts key information from a question of a user by chatting with the user, and feeds back corresponding answers through internal analysis, wherein the online customer service comprises robot customer service and manual customer service. However, the robot customer service cannot understand the semantics of the user problem well, and cannot solve the user problem, so that the user can select manual customer service, and the working pressure of the manual customer service is too high.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the application provides a question-answering method, a question-answering device, electronic equipment and a computer readable storage medium, so that a robot helps a customer service to answer questions of a user in an auxiliary manner based on a question-answering model obtained through training, different answering modes are determined based on confidence degrees of topic categories, and the accuracy and efficiency of question-answering are improved.
An embodiment of a first aspect of the present application provides a question answering method, including:
receiving a target problem of a client;
classifying according to the semantics of the target problem to obtain a target subject to which the target problem belongs and a confidence degree of the target problem belonging to the target subject;
replying the target question by adopting a question-answer model corresponding to the target theme to obtain a target reply;
sending the target reply to the client under the condition that the confidence coefficient is greater than a set threshold value;
and under the condition that the confidence degree is less than or equal to the set threshold value, sending the target reply as a reply suggestion to the customer service working end.
An embodiment of a second aspect of the present application provides a question answering device, including:
the receiving module is used for receiving the target problem of the client;
the classification module is used for classifying according to the semantics of the target problem to obtain a target subject to which the target problem belongs and a confidence coefficient of the target problem belonging to the target subject;
the reply module is used for replying the target question by adopting a question-answer model corresponding to the target theme to obtain a target reply;
the processing module is used for sending the target reply to the client under the condition that the confidence coefficient is greater than a set threshold value;
and the processing module is further used for sending the target reply as a reply suggestion to the customer service working end under the condition that the confidence degree is less than or equal to the set threshold value.
An embodiment of a third aspect of the present application provides an electronic device, including:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
A fourth aspect of the present application is directed to a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of the first aspect.
An embodiment of the fifth aspect of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the method of the first aspect.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
the method comprises the steps of receiving target questions of a client, classifying according to semantics of the target questions to obtain target topics to which the target questions belong and confidence degrees that the target questions belong to the target topics, replying the target questions by adopting a question-answering model corresponding to the target topics to obtain target replies, sending the target replies to the client when the confidence degrees are larger than a set threshold, and sending the target replies to a customer service working end as reply suggestions when the confidence degrees are smaller than or equal to the set threshold.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of a question answering method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of another question answering method provided in the embodiment of the present application;
fig. 3 is a schematic flow chart of another question answering method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a question-answer interaction provided in an embodiment of the present application;
FIG. 5 is a schematic diagram of a knowledge base provided by an embodiment of the present application; and
fig. 6 is a schematic structural diagram of a question answering device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
A question answering method, apparatus, electronic device, and computer-readable storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a question answering method according to an embodiment of the present application.
As shown in fig. 1, the method comprises the steps of:
The execution subject of the embodiment of the application is a question answering device, and the question answering device may be an electronic device or may be disposed in the electronic device, where the electronic device includes a robot, but is not limited to a robot, and may also be a mobile phone, a palm computer, and the like, and is not limited in this embodiment.
The client is an application installed on the electronic device, such as an e-commerce application, an instant messaging application, and the like.
The target question is a question that the user sends through the client, e.g., hello, at? Or the following steps: when did my orders ship? Or the following steps: what is the size of the washing machine? Or the following steps: what model of CPU, how large memory, and how large size of computer? And so on, which are not listed in this embodiment.
And 102, classifying according to the semantics of the target problem to obtain a target subject to which the target problem belongs and a confidence degree of the target problem belonging to the target subject.
In an implementation manner of the embodiment of the present application, a target problem is input into a recognition model obtained by training, and a target topic to which the target problem belongs and a confidence level of the target problem belonging to the target topic are output.
In another implementation manner of the embodiment of the present application, according to the vector code of the question text of the corpus pair included in each topic category, the vector code of each topic category is determined, the target problem is coded, the vector code of the target problem is obtained, according to the distance between the vector code of the target problem and the vector code of each topic category, the target topic to which the target problem belongs is determined from each topic category, and according to the distance between the vector code of the target problem and the vector code of the target topic, the confidence that the target problem belongs to the target topic is determined.
For example, the target question is "what the size of the type XX refrigerator is", the determined target subject to which the target question belongs is an industry category, specifically a refrigerator industry category, and meanwhile, the confidence coefficient of belonging to the refrigerator industry category is 0.9. And if the target question is 'when the delivery can be carried out', determining that the target subject to which the target question belongs is a general category, and the confidence coefficient of the target subject to which the target question belongs is 0.8.
And 103, replying the target question by adopting the question-answer model corresponding to the target theme to obtain a target reply.
The question-answer model and the target theme have a corresponding relationship, namely each theme has a corresponding question-answer model, and the question-answer model corresponding to the theme has been learned in advance through training to the corresponding relationship of the question and the reply under the theme.
For example, the target question is "what the processor model of this computer is", and the identified target returns to "model X-11".
And 104, sending a target reply to the client under the condition that the confidence coefficient is greater than the set threshold value.
And 105, under the condition that the confidence coefficient is less than or equal to the set threshold value, sending the target reply as a reply suggestion to the customer service working end.
In the embodiment of the application, in order to improve the accuracy of the target question reply, different types of replies are performed according to the accuracy of the classification result of the target question, that is, according to the confidence of the target topic to which the determined target question belongs, in one scenario, the target reply is sent to the client side under the condition that the confidence of the target topic is greater than a set threshold value, that is, the target reply is sent to the client side under the condition that the target reply with higher accuracy is determined; in another scenario, when the confidence of the target topic is less than or equal to a set threshold, the target reply is sent to the customer service working end as a reply suggestion, and the customer service working end sends the target reply to the client side after confirming that the target reply is accurate. In the embodiment of the application, the message channels of the electronic device, the client and the customer service working end are opened, after the electronic device obtains the target problem of the client, the target reply is pushed to different target ends according to the matched routing rule, so that the electronic device and the manual work are tightly combined, the reception efficiency of the customer service working end is improved, and the experience of a client user is improved.
The set threshold may be set by a person skilled in the art according to a service requirement, and the size of the set threshold may be adjusted according to the accuracy of the service requirement.
In the question answering method of the embodiment of the application, the target questions of the client are received, the target topics to which the target questions belong are obtained by classifying according to the semantics of the target questions, the confidence degrees of the target questions belonging to the target topics are obtained, the question answering model corresponding to the target topics is adopted to reply the target questions to obtain target replies, the target replies are sent to the client under the condition that the confidence degrees are larger than a set threshold value, and the target replies are sent to the customer service working end as reply suggestions under the condition that the confidence degrees are smaller than or equal to the set threshold value, so that the question answering method based on the question answering model obtained by training helps the customer service to answer the questions of the user, meanwhile, different answering modes are determined based on the confidence degrees of the topic categories, and the accuracy and the efficiency are improved.
In the above embodiment, it is explained that, for the obtained target question of the client, the target topic described in the target question is determined, and the target question is replied according to the question-answer model corresponding to the target topic to obtain the target reply, so that the training process of the question-answer model corresponding to the target topic is explained in this embodiment. Based on the previous embodiment, fig. 2 is a schematic flow chart of another question-answering method provided in the embodiment of the present application, and specifically illustrates a training process of a question-answering model.
As shown in fig. 2, the method may include the steps of:
In step 201 to step 202, the explanation of the foregoing method embodiment can be referred to, and the principle is the same, which is not described herein again.
The training process of the question-answering models of the topic categories in the above steps 203 to 205 is not limited to be executed after the step 202, and may also be executed before the step 202 or before the step 201, which is not limited in this embodiment.
In the embodiment of the application, a plurality of corpus pairs are obtained by extracting the historical chat corpus and the corpus of the common problem solution accumulated by customer service from the historical customer service dialog according to a tool based on an Extract-Transform-Load (ETL) technology. The corpus pair comprises a question text and a corresponding reply text.
In which ETL is used to extract (extract), transform (transform), and load (load) data from a source to a destination. The term ETL is more commonly used in data warehouses, but its objects are not limited to data warehouses.
And 204, clustering according to the semantics of the plurality of corpus pairs, and dividing the plurality of corpus pairs into at least two subject categories.
In an implementation manner of the embodiment of the application, an unsupervised learning algorithm may be used to cluster semantics of a plurality of corpus pairs, and as an implementation manner, similar corpus pairs may be divided into the same subject category based on a distance between the corpus pairs, for example, the clustering algorithm is a hierarchical clustering algorithm, a K-means clustering algorithm, an Expectation Maximization (EM) clustering algorithm, and the like, which are not listed in this embodiment. Through clustering, a plurality of corpus pairs are divided into at least two subject categories, and the clustering algorithm belongs to an unsupervised learning algorithm, does not need manual labeling, reduces the cost of subject category identification, and improves the efficiency.
In an implementation manner of the embodiment of the application, for any topic category, the question text of the corpus pair in the topic category is input into the question-answer model of the corresponding topic category to obtain the prediction reply, and model parameters are adjusted according to the difference between the prediction reply and the corresponding reply text, so that the question-answer model of the corresponding topic category is trained. In the embodiment of the application, for each topic category, the corpus under the topic category is respectively adopted to train the corresponding question-answer model as a sample, so that the training effect of the question-answer model corresponding to each topic category is improved.
And step 206, replying the target question by adopting the question-answer model corresponding to the target subject to obtain a target reply.
And step 208, under the condition that the confidence coefficient is less than or equal to the set threshold value, sending the target reply as a reply suggestion to the customer service working end.
In step 206 to step 208, the explanation of the aforementioned method embodiments can be referred to, and the principle is the same, which is not described herein again.
According to the question answering method, a plurality of corpus pairs are obtained in historical customer service dialogue, at least two subject categories determined by clustering of the corpus pairs are determined in a clustering mode, the categories of the corpus pairs are determined by clustering, manual marking is not needed, and efficiency is improved. The question-answer models corresponding to at least two topic categories are set, and then the corpus pairs contained in each topic type are adopted to train the question-answer models corresponding to the topic categories, so that the question-answer models obtained through training learn the corresponding relation between the questions and the answers under the corresponding topic categories, and the question answering efficiency under each topic category is improved. The question text and the corresponding reply text contained in each corpus pair are adopted, the question-answer model is trained through a supervised training method, manual labeling is avoided, the model training effect is improved, a large number of historical samples are adopted for training, and the efficiency is improved.
Based on the foregoing embodiments, an embodiment of the present application provides another question answering method, and fig. 3 is a schematic flow chart of the another question answering method provided in the embodiment of the present application, as shown in fig. 3, the method includes the following steps:
In the embodiments of the present application, an electronic device is described as an example of a robot.
As shown in fig. 4, the robot interacts with the user through the client installed on the robot, receives the target question of the user based on the gateway, for example, the target question is determined through voice recognition by inputting the text of the target question by the user or acquiring the voice of the target question.
As shown in fig. 4, according to the obtained target question, based on Natural Language Understanding (NLU), the semantics of the target question are identified to determine the target subject to which the target question belongs and the confidence level that the target question belongs to the target subject.
And 303, replying the target question by adopting the question-answer model corresponding to the target theme to obtain a target reply.
The explanation of steps 301 to 302 in the foregoing embodiments can be referred to, and the principle is the same, which is not described again in this embodiment.
It should be noted that, the standard replying technique for the knowledge point under the target topic in steps 305 to 308 may be executed after step 304 or before step 304, which is not limited in this embodiment.
Referring to the clustering method in step 201 and step 202 in the above embodiment, the target topic to which each question text belongs may be determined, and thus, each question text belonging to the target topic is determined.
And step 306, merging the problems according to the semantic similarity between the problem texts of the target theme to obtain all the merged problems of the target theme.
In the embodiment of the application, vector codes of all problem texts of a target theme are determined, semantic similarity is determined based on the distance between the vector codes of all problem texts, and all problem texts with semantic similarity meeting a threshold are merged, for example, all problem texts are spliced or all problem texts are fused to obtain all merged problems of the target theme.
For example, the target theme is a general type theme, and the question text 1 is "ship while what? "when the question text 2 is" when it can be received "and the question text 3 is" did it have been delivered ", then the combined questions are combined in a fusion manner to be" delivery time "or" time to receive ".
And 307, generating corresponding knowledge points under the target theme according to the combined problems of the target theme.
In the embodiment of the application, a plurality of merging problems are provided under the target theme, and each merging problem is a knowledge point under the target theme. For example, the delivery time, i.e., a corresponding knowledge point under the general type of subject.
And step 308, responding to the configuration operation, and configuring standard answer operation on the knowledge points under the target subject.
In the embodiment of the application, in response to the configuration operation, a standard reply dialect of the user for the configuration of the knowledge point under the target topic, for example, the "delivery time" of the knowledge point, and the corresponding standard reply dialect is "within 48 hours after payment".
And 309, inquiring knowledge points under the target subject in the knowledge base according to the target problem.
The knowledge base is pre-established, and includes knowledge bases of various topics, for example, as shown in fig. 4 and 5, the knowledge base includes a general knowledge base, an industry knowledge base, a frequently asked question and answer FAQ knowledge base, and the like, where the frequently asked question and answer FAQ knowledge base is classified by using standardized "shortcut phrases" and forms a standard FAQ by using a mathematical induction method, and each topic also has a corresponding answer model, where the general topic corresponds to the general model, the industry topic corresponds to the industry model, and the industry model corresponds to the industry model, where the industry includes different industries such as a washing machine, a refrigerator, and an air conditioner, and these industries are not listed in this embodiment. The knowledge base of each topic comprises a plurality of knowledge points.
For example, the knowledge points under the general topic are "delivery time" and "coupon".
In the embodiment of the application, the target problem is matched with each knowledge point under the target topic in the knowledge base, the knowledge point matched with the target problem is determined, for example, matching is performed based on the distance, and whether the target knowledge point matched with the target problem exists under the target topic in the knowledge base is determined.
In an implementation manner of the embodiment of the application, when it is determined that the confidence that the target problem belongs to the target topic is greater than the set threshold and it is determined that there is no target knowledge point matching with the target problem, a target reply is sent to the client, so that the target problem is replied, the pressure of a customer service working end is reduced, and the reply efficiency is improved.
In another implementation manner of the embodiment of the application, when the confidence that the target problem belongs to the target topic is greater than the set threshold, it is determined that the target problem has a matched target knowledge point, and when the target indication point is not configured with a standard reply dialect, a target reply is sent to the client, so that the target problem is replied, the pressure of the customer service working end is reduced, and the reply efficiency is improved.
And 311, sending the standard answer operation to the client under the condition that the target knowledge points matched with the target problem are determined to be configured with the standard answer operation.
In an implementation manner of the embodiment of the application, under the condition that the confidence that the target problem belongs to the target topic is greater than the set threshold, and under the condition that the target knowledge point matched with the target problem is configured with the standard answer operation, the standard answer operation is sent to the client.
It should be noted that the reply sent to the client in fig. 4 may be a target reply obtained by recognition or a standard reply dialog obtained by matching from a knowledge base.
And step 312, under the condition that the confidence degree that the target question belongs to the target subject is less than or equal to the set threshold value, sending the target reply as a reply suggestion to the customer service working end.
Specifically, reference may be made to the description in the foregoing embodiments, which are not repeated herein.
In the question answering method of the embodiment of the application, the knowledge bases of all the topics are generated in advance, the knowledge points of the problems are set in the knowledge bases of all the topics, the operation configuration is responded in advance, the standard answer operation is configured on the knowledge points under the target topic, therefore, in order to improve accuracy, whether the standard answer operation corresponding to the target problem exists or not is determined for the target answer obtained by the target problem, if not, the target answer is sent to the client, if the standard answer operation exists, the standard answer operation is sent to the client, the accuracy of the answer is improved, and the answer efficiency is improved.
In order to implement the above embodiments, the present application further provides a question answering device.
Fig. 6 is a schematic structural diagram of a question answering device according to an embodiment of the present application.
As shown in fig. 6, the apparatus includes:
and a receiving module 61, configured to receive the target question of the client.
And a classification module 62, configured to classify according to the semantics of the target problem to obtain a target topic to which the target problem belongs and a confidence that the target problem belongs to the target topic.
And the reply module 63 is configured to reply to the target question by using the question-answer model corresponding to the target topic, so as to obtain a target reply.
A processing module 64, configured to send the target reply to the client if the confidence is greater than a set threshold.
And the processing module 64 is further configured to send the target reply to the customer service working end as a reply suggestion if the confidence is less than or equal to the set threshold.
Further, in an implementation manner of the embodiment of the present application, the apparatus further includes:
the acquisition module is used for acquiring a plurality of corpus pairs from the historical customer service conversation;
the clustering module is used for clustering according to the semantics of the plurality of corpus pairs and dividing the plurality of corpus pairs into at least two subject categories;
and the training module is used for training the question-answering model corresponding to the theme category by adopting the corpus pairs contained in the theme categories.
In an implementation manner of the embodiment of the present application, each corpus pair includes a question text and a corresponding reply text; a training module to:
aiming at any one theme category, inputting the question text of the corpus pair in the theme category into a question-answer model of the corresponding theme category to obtain a prediction reply; and adjusting model parameters according to the difference between the predicted reply and the corresponding reply text.
In an implementation manner of the embodiment of the present application, the classification module 62 is configured to:
determining the vector code of each topic type according to the vector code of the question text of the corpus pair contained in each topic type; coding the target problem to obtain a vector code of the target problem; determining a target topic to which the target problem belongs from each topic type according to the distance between the vector code of the target problem and the vector code of each topic type; and determining the confidence degree of the target question belonging to the target subject according to the distance between the vector code of the target question and the vector code of the target subject.
Further, in an implementation manner of the embodiment of the present application, the processing module 64 includes:
the query unit is used for querying each knowledge point under the target subject in a knowledge base according to the target problem under the condition that the confidence coefficient is greater than a set threshold value;
a sending unit, configured to send the target reply to the client when it is determined that there is no target knowledge point matching the target problem, or when the target knowledge point is not configured with a standard reply dialect.
In an implementation manner of the embodiment of the present application, the processing module 64 further includes:
the determining unit is used for determining each question text belonging to the target theme from the historical customer service dialogue;
the merging unit is used for merging the problems according to the semantic similarity between the problem texts of the target theme to obtain all the merged problems of the target theme;
the generating unit is used for generating corresponding knowledge points under the target theme according to the merging problems of the target theme;
and the configuration unit is used for responding to configuration operation and configuring the standard answer operation for the knowledge points under the target theme.
It should be noted that the foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and is not repeated herein.
In the question answering device of the embodiment of the application, the target questions of the client are received and classified according to the semantics of the target questions to obtain the target subjects to which the target questions belong and the confidence degrees of the target questions belonging to the target subjects, the question answering model corresponding to the target subjects is adopted to reply to the target questions to obtain the target replies, the target replies are sent to the client under the condition that the confidence degrees are larger than the set threshold values, and the target replies are sent to the customer service working end as reply suggestions under the condition that the confidence degrees are smaller than or equal to the set threshold values, so that the question answering model obtained based on training is realized, the robot helps the customer service to answer the questions of the user in an auxiliary mode, meanwhile, different answering modes are determined based on the confidence degrees of the subject categories, and the accuracy and the efficiency are improved.
In order to implement the foregoing embodiment, an embodiment of the present application further provides an electronic device, including:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of the foregoing method embodiments.
In order to implement the foregoing embodiments, the present application also proposes a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method described in the foregoing method embodiments.
In order to implement the foregoing embodiments, the present application further provides a computer program product including a computer program, where the computer program implements the method described in the foregoing method embodiments when being executed by a processor.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (15)
1. A question-answering method, characterized by comprising the steps of:
receiving a target problem of a client;
classifying according to the semantics of the target problem to obtain a target subject to which the target problem belongs and a confidence degree of the target problem belonging to the target subject;
replying the target question by adopting a question-answer model corresponding to the target theme to obtain a target reply;
sending the target reply to the client under the condition that the confidence coefficient is greater than a set threshold value;
and under the condition that the confidence degree is less than or equal to the set threshold value, sending the target reply as a reply suggestion to the customer service working end.
2. The question-answering method according to claim 1, characterized in that it further comprises:
obtaining a plurality of corpus pairs from historical customer service conversations;
clustering is carried out according to the semantemes of the corpus pairs, and the corpus pairs are divided into at least two subject categories;
and training a question-answer model corresponding to the theme category by adopting the corpus pairs contained in the theme categories.
3. The question-answering method according to claim 2, wherein each corpus pair contains a question text and a corresponding reply text; the training of the question-answer model corresponding to the topic categories by adopting the corpus pairs contained in the topic categories comprises the following steps:
aiming at any one theme category, inputting the question text of the corpus pair in the theme category into a question-answer model of the corresponding theme category to obtain a prediction reply;
and adjusting model parameters according to the difference between the predicted reply and the corresponding reply text.
4. The method according to claim 2, wherein the classifying according to the semantics of the target question to obtain a target subject to which the target question belongs and a confidence that the target question belongs to the target subject comprises:
determining the vector code of each topic type according to the vector code of the question text of the corpus pair contained in each topic type;
coding the target problem to obtain a vector code of the target problem;
determining a target topic to which the target problem belongs from each topic type according to the distance between the vector code of the target problem and the vector code of each topic type;
and determining the confidence degree of the target question belonging to the target subject according to the distance between the vector code of the target question and the vector code of the target subject.
5. The method according to any one of claims 1-4, wherein the sending the target reply to the client if the confidence level is greater than a set threshold comprises:
under the condition that the confidence coefficient is larger than a set threshold value, inquiring all knowledge points under the target subject in a knowledge base according to the target problem;
and sending the target reply to the client under the condition that the target knowledge point matched with the target problem does not exist or the target knowledge point is not configured with standard reply dialogs.
6. The method of claim 5, wherein prior to querying knowledge points under the target topic in a knowledge base according to the target question, the method further comprises:
determining each question text belonging to the target topic from the historical customer service dialog;
according to semantic similarity among the question texts of the target theme, problem combination is carried out, and all combined problems of the target theme are obtained;
generating corresponding knowledge points under the target theme according to the merging problems of the target theme;
and responding to a configuration operation, and configuring the standard answer operation for the knowledge points under the target subject.
7. A question answering device, comprising:
the receiving module is used for receiving the target problem of the client;
the classification module is used for classifying according to the semantics of the target problem to obtain a target subject to which the target problem belongs and a confidence coefficient of the target problem belonging to the target subject;
the reply module is used for replying the target question by adopting a question-answer model corresponding to the target theme to obtain a target reply;
the processing module is used for sending the target reply to the client under the condition that the confidence coefficient is greater than a set threshold value;
and the processing module is further used for sending the target reply as a reply suggestion to the customer service working end under the condition that the confidence degree is less than or equal to the set threshold value.
8. The question-answering device according to claim 7, characterized in that the device further comprises:
the acquisition module is used for acquiring a plurality of corpus pairs from the historical customer service conversation;
the clustering module is used for clustering according to the semantics of the plurality of corpus pairs and dividing the plurality of corpus pairs into at least two subject categories;
and the training module is used for training the question-answering model corresponding to the theme category by adopting the corpus pairs contained in each theme category.
9. The question-answering device according to claim 8, wherein each corpus pair contains a question text and a corresponding reply text; the training module is configured to:
aiming at any topic category, inputting the question text of the corpus pairs in the topic category into a question-answer model corresponding to the topic category to obtain a prediction reply;
and adjusting model parameters according to the difference between the predicted reply and the corresponding reply text.
10. The apparatus of claim 8, wherein the classification module is configured to:
determining the vector code of each topic type according to the vector code of the question text of the corpus pair contained in each topic type;
coding the target problem to obtain a vector code of the target problem;
determining a target theme to which the target problem belongs from each theme category according to the distance between the vector code of the target problem and the vector code of each theme category;
and determining the confidence degree of the target question belonging to the target subject according to the distance between the vector code of the target question and the vector code of the target subject.
11. The apparatus according to any one of claims 7-10, wherein the processing module comprises:
the query unit is used for querying each knowledge point under the target subject in a knowledge base according to the target problem under the condition that the confidence coefficient is greater than a set threshold value;
a sending unit, configured to send the target reply to the client when it is determined that there is no target knowledge point matching the target problem, or when the target knowledge point is not configured with a standard reply dialect.
12. The apparatus of claim 11, wherein the processing module further comprises:
the determining unit is used for determining each question text belonging to the target theme from the historical customer service dialogue;
the merging unit is used for merging the problems according to the semantic similarity between the problem texts of the target theme to obtain all the merged problems of the target theme;
the generating unit is used for generating corresponding knowledge points under the target theme according to the merging problems of the target theme;
and the configuration unit is used for responding to configuration operation and configuring the standard answer operation for the knowledge points under the target theme.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
15. A computer program product comprising a computer program, characterized in that the computer program realizes the method of any of claims 1-6 when executed by a processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116820.1A CN114519093A (en) | 2022-02-07 | 2022-02-07 | Question answering method, device, electronic equipment and computer readable storage medium |
| PCT/CN2022/131497 WO2023147733A1 (en) | 2022-02-07 | 2022-11-11 | Question answering method and apparatus, and electronic device and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116820.1A CN114519093A (en) | 2022-02-07 | 2022-02-07 | Question answering method, device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114519093A true CN114519093A (en) | 2022-05-20 |
Family
ID=81597236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210116820.1A Pending CN114519093A (en) | 2022-02-07 | 2022-02-07 | Question answering method, device, electronic equipment and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114519093A (en) |
| WO (1) | WO2023147733A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114996433A (en) * | 2022-08-08 | 2022-09-02 | 北京聆心智能科技有限公司 | Dialog generation method, device and equipment |
| WO2023147733A1 (en) * | 2022-02-07 | 2023-08-10 | 京东科技信息技术有限公司 | Question answering method and apparatus, and electronic device and computer-readable storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117725185B (en) * | 2024-02-06 | 2024-05-07 | 河北神玥软件科技股份有限公司 | Intelligent dialogue generation method and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9384450B1 (en) * | 2015-01-22 | 2016-07-05 | International Business Machines Corporation | Training machine learning models for open-domain question answering system |
| CN109033221A (en) * | 2018-06-29 | 2018-12-18 | 上海银赛计算机科技有限公司 | Answer generation method, device and server |
| CN114519093A (en) * | 2022-02-07 | 2022-05-20 | 京东科技信息技术有限公司 | Question answering method, device, electronic equipment and computer readable storage medium |
| CN114610854A (en) * | 2022-03-15 | 2022-06-10 | 深圳壹账通智能科技有限公司 | Intelligent question and answer method, device, equipment and storage medium |
-
2022
- 2022-02-07 CN CN202210116820.1A patent/CN114519093A/en active Pending
- 2022-11-11 WO PCT/CN2022/131497 patent/WO2023147733A1/en unknown
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023147733A1 (en) * | 2022-02-07 | 2023-08-10 | 京东科技信息技术有限公司 | Question answering method and apparatus, and electronic device and computer-readable storage medium |
| CN114996433A (en) * | 2022-08-08 | 2022-09-02 | 北京聆心智能科技有限公司 | Dialog generation method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023147733A1 (en) | 2023-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114519093A (en) | Question answering method, device, electronic equipment and computer readable storage medium | |
| CN107168546B (en) | Input prompting method and device | |
| JP2020537777A (en) | Methods and devices for identifying the user's intent of speech | |
| CN111737987B (en) | Intention recognition method, device, equipment and storage medium | |
| CN111182162B (en) | Telephone quality inspection method, device, equipment and storage medium based on artificial intelligence | |
| CN111062220B (en) | End-to-end intention recognition system and method based on memory forgetting device | |
| CN111401071A (en) | Model training method and device, computer equipment and readable storage medium | |
| CN103593412B (en) | A kind of answer method and system based on tree structure problem | |
| CN111143530B (en) | Intelligent reply method and device | |
| CN113051374B (en) | Text matching optimization method and device | |
| CN110413744B (en) | Dialogue message processing method, device and equipment in instant messaging process | |
| CN111400480B (en) | User intention recognition method and device for multi-round dialogue | |
| CN112463949B (en) | Data recall method and system, interaction method and interaction system | |
| CN112766990B (en) | Intelligent customer service auxiliary system and method based on multi-round dialogue improvement | |
| CN111026840A (en) | Text processing method, device, server and storage medium | |
| CN113590778A (en) | Intelligent customer service intention understanding method, device, equipment and storage medium | |
| CN110390109B (en) | Method and device for analyzing association relation among multiple group chat messages | |
| CN109684444A (en) | A kind of intelligent customer service method and system | |
| CN112364622A (en) | Dialog text analysis method, dialog text analysis device, electronic device and storage medium | |
| CN111597821A (en) | Method and device for determining response probability | |
| CN116739004B (en) | Intention recognition method, device, equipment and readable storage medium | |
| CN112884009A (en) | Classification model training method and system | |
| CN117370512A (en) | Method, device, equipment and storage medium for replying to dialogue | |
| CN112256856A (en) | Robot dialogue method, device, electronic device and storage medium | |
| CN116662555A (en) | Request text processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |