CN112131338B - Method and device for establishing question-answer pairs - Google Patents
Method and device for establishing question-answer pairs Download PDFInfo
- Publication number
- CN112131338B CN112131338B CN202010946638.XA CN202010946638A CN112131338B CN 112131338 B CN112131338 B CN 112131338B CN 202010946638 A CN202010946638 A CN 202010946638A CN 112131338 B CN112131338 B CN 112131338B
- Authority
- CN
- China
- Prior art keywords
- question
- reply message
- current
- answer pair
- customer service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000012550 audit Methods 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 description 5
- 230000032683 aging Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The specification discloses a method and apparatus for establishing question-answer pairs. The method comprises the following steps: monitoring the pasting operation of the manual customer service; taking the text corresponding to the paste operation as a current reply message of the current user problem; determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service; if the number of the historical reply messages meets a preset condition, generating a question-answer pair according to the current reply message and the current user question; and storing the question-answer pairs into a knowledge base.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for establishing question and answer pairs.
Background
In the application scenarios of instant messaging technologies such as consultation and customer service, many similar problems often need to be replied. In order to improve the reply efficiency of the message, the system usually presets a standard question-reply pair, so that the reply speed can be improved when the same problem is received later.
However, the establishment of question-answer pairs requires collection of a large number of questions and reply messages, and has a certain hysteresis, so how to establish question-answer pairs in time has become a urgent problem to be solved.
Disclosure of Invention
In view of this, the present disclosure provides a method and apparatus for establishing question-answer pairs.
Specifically, the specification is realized by the following technical scheme:
a method of establishing question-answer pairs, the method comprising:
monitoring the pasting operation of the manual customer service;
taking the text corresponding to the paste operation as a current reply message of the current user problem;
determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service;
if the number of the historical reply messages meets a preset condition, generating a question-answer pair according to the current reply message and the current user question;
and storing the question-answer pairs into a knowledge base.
An apparatus for establishing question-answer pairs, the apparatus comprising:
the monitoring unit is used for monitoring the pasting operation of the manual customer service;
the obtaining unit is used for taking the text corresponding to the pasting operation as a current reply message of the current user problem;
the counting unit is used for determining the number of historical reply messages matched with the current reply message in the current period, and the historical reply messages are input by the manual customer service;
The judging unit is used for generating a question-answer pair according to the current reply message and the current user question if the number of the historical reply messages meets a preset condition;
and the storage unit is used for storing the question-answer pairs into a knowledge base.
An apparatus for creating question-answer pairs, comprising:
a processor;
a memory for storing machine-executable instructions;
wherein, by reading and executing the machine executable instructions stored in the memory corresponding to the method of creating question-answer pairs, the processor is caused to:
monitoring the pasting operation of the manual customer service;
taking the text corresponding to the paste operation as a current reply message of the current user problem;
determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service;
if the number of the historical reply messages meets a preset condition, generating a question-answer pair according to the current reply message and the current user question;
and storing the question-answer pairs into a knowledge base.
According to one embodiment of the specification, the paste operation of the manual customer service is monitored, the text corresponding to the paste operation is used as the current reply message, and then the reply message is matched with the historical reply message in the current period. And under the condition that the number of the matched historical reply messages meets the preset condition, generating question-answer pairs according to the reply messages and the corresponding user questions, and storing the question-answer pairs into a knowledge base. In this embodiment, if a person customer service monitors that a similar reply message is input multiple times in one period, it is indicated that the frequency of the reply message being used is higher, and then a question-answer pair can be generated and stored according to the reply message and the corresponding user question.
Compared with the traditional manual input question-answer pair, the method of the embodiment saves labor cost and has higher efficiency; compared with the method of establishing question-answer pairs through machine learning algorithms, the method of the embodiment not only reduces the performance pressure of the server, but also has better timeliness.
Drawings
Fig. 1 is a flow chart illustrating a method for creating question-answer pairs according to an exemplary embodiment of the present disclosure.
Fig. 2 is a flow chart illustrating another method for creating question-answer pairs according to an exemplary embodiment of the present disclosure.
Fig. 3 is a schematic diagram of a device for creating question-answer pairs according to an exemplary embodiment of the present disclosure.
Fig. 4 is a block diagram of a device for creating question-answer pairs according to an exemplary embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present description as detailed in the accompanying claims.
The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
The customer service system is a system for replying to questions consulted by a user, and in order to improve the replying efficiency, mainly for the replying efficiency of repeated questions, question-answer pairs, which are usually the mapping relationship between questions and answers, can be established. When the customer service system receives the user questions, the customer service system can query question-answer pairs, and if the corresponding answers are queried, the customer service system can automatically reply to the users or give references to reply messages. However, in the face of a large number of user questions and reply messages, how to screen them and establish effective question-answer pairs in time is a urgent problem to be solved.
In the related art, a manual entry mode can be adopted to establish question-answer pairs. For example, the human customer service may empirically input the user questions with high frequency of occurrence and the response messages corresponding to the user questions into a knowledge base storing the question-answer pairs.
However, the manual input method is high in accuracy, but the manual operation is low in efficiency, and question-answer pairs cannot be added timely.
In another related art, intelligent algorithms may also be employed for data mining to establish question-answer pairs. In this approach, customer service systems typically need to acquire a large number of user questions and corresponding reply messages, and then build a model from these data to mine out valid question-answer pairs.
However, the method needs to collect a large amount of data in advance, so that the established question-answer pair is lagged, and the user problem with timeliness cannot be added in time. In addition, the accuracy of question-answer pairs established by the method is low, and a large amount of calculation is needed, so that the equipment performance is consumed.
In view of this, the present specification proposes a method of establishing question-answer pairs. In the present specification, a reply message input by a human customer service for a user problem may be monitored, and then the reply message is matched with a historical reply message in a current period. And under the condition that the number of the matched historical reply messages meets the preset condition, generating a question-answer pair according to the reply messages and the corresponding user questions.
Compared with the related art, the method of the specification has less manual participation, lower pressure on the server, and higher accuracy and timeliness when the question-answer pair is established with high efficiency.
The method is applied to a customer service system, a client of the customer service system can display user problems, reply messages or other prompt messages and the like through a session window, a server of the customer service system stores a knowledge base, and complex processing such as calculation, search and the like is executed. The client and the server may interact through a network.
The knowledge base is a database or a data table for storing question and answer pairs. In the initial stage of customer service system establishment, question-answer pairs with higher use frequency can be added to a knowledge base in a manual input mode by a user, and then, the question-answer pairs can be generated according to the method of the specification and stored in the knowledge base.
Alternatively, question-answer pairs in the knowledge base may have keywords of questions as user questions and complete sentences as reply messages. When receiving a user question and inquiring a matched question and answer, a server side of the customer service system can acquire one or more keywords in the user question by using a preset algorithm, and then inquire a reply message corresponding to the keywords in a knowledge base. The specific method for extracting the keywords may refer to related technology, and will not be described in detail in this embodiment.
Alternatively, question-answer pairs in the knowledge base may also save both the user questions and the reply messages as complete sentences. When receiving user questions and inquiring matched question-answering pairs, a server side of the customer service system can use a preset algorithm to match the received user questions with the user questions in the knowledge base question-answering pairs one by one, and if the matching is successful, a reply message in the matched question-answering pairs is obtained. More methods for matching two complete sentences, for example, zSearch is used to match the word segmentation, which is not described in detail in this embodiment.
In an alternative embodiment, after the customer service system receives the user problem, the server may query the knowledge base for a question-answer pair matching the user problem, for example, extract a keyword in the user problem, and query the knowledge base for the keyword; or directly word segmentation matching is carried out on the user questions and the user questions in the knowledge base question-answering pair.
If the query is successful, a reply message of the queried question-answer pair can be returned to the user. This process does not require human customer service, and may also be referred to as intelligent customer service.
If the inquiry fails, assigning a manual customer service for the user problem, and monitoring a reply message input by the manual customer service in a session window, wherein a question-answer pair can be generated subsequently based on the reply message of the manual customer service.
In another alternative embodiment, after the customer service system receives the user problem, on one hand, a manual customer service is assigned to the user problem, and on the other hand, the server can query whether a question-answer pair matched with the user problem exists in the knowledge base.
If the inquiry is successful, the server returns the reply message in the inquired question-answer pair to the client, the client displays the received reply message in a floating window or embedded interface mode and the like, then the manual customer service can judge whether the reply message is suitable for being used as the reply message of the user problem, and can execute editing, sending and other operations on the reply message.
If the inquiry fails, the server side informs the client side that the inquiry result is the inquiry failure, the client side can inform the manual customer service of the inquiry failure in a popup window and other modes, monitor the reply message input by the manual customer service in the session window, and generate a question-answer pair based on the reply message of the manual customer service.
It can be seen that the more perfect the question-answer pair content in the knowledge base, the higher the efficiency of the customer service system to answer the user's question.
The method of creating question-answer pairs in this specification will be described with reference to fig. 1.
As shown in fig. 1, the method comprises the following steps:
Step S102: monitoring the pasting operation of the manual customer service;
step S104: and taking the text corresponding to the paste operation as a current reply message of the current user problem.
In this step, the customer service system may assign a manual customer service to the user problem through the server, newly establish a session window at the client of the assigned manual customer service, display the user problem, and then monitor the session window through the client, obtain the reply message input by the manual customer service, so as to facilitate distinction, and may refer to the reply message monitored by the client as the current reply message.
The method for assigning the artificial customer service is various, for example, a person customer service in an idle state is selected randomly, or a corresponding category of artificial customer service is assigned to the user problem according to the category to which the user belongs, and the method can be specifically assigned according to the actual situation, which is not limited herein.
After receiving the user problem, the manual customer service can manually input a corresponding reply message in an input box of the session window. The client may monitor for an input operation in the session window, with the input content as the current reply message.
Alternatively, the human customer service may copy the reply message from other material and paste it into the input box. The client can monitor the paste operation in the session window and take the text corresponding to the paste operation as the current reply message.
For example, assume that the user problem is: "please ask when to ship", the human customer service may manually input or paste the input reply message "warehouse already packaged, ship today" for the question, and the client takes this "warehouse already packaged, ship today" as the current reply message.
It should be noted that, compared with the input text, the text pasted in general is more fixed and has higher frequency of use, the pasting operation of the monitoring client can improve the generation threshold of question-answer pairs, reduce low-quality question-answer pairs, reduce the workload of customer service staff during auditing, and simultaneously, when the customer service staff does not audit but automatically stores the question-answer pairs, the quality of the question-answer pairs can be ensured to a certain extent.
Step S106: and determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service.
The historical reply message can be manually input by a manual customer service or can be input by the manual customer service in a pasting mode.
In this embodiment, the customer service system may save the historical reply message that the manual customer service has replied to at the server, for example, maintain a list of the historical reply messages.
As an alternative embodiment, a historical reply message and a timestamp of the historical reply message may be recorded, where the timestamp may be a reply time of the historical reply message. Optionally, to prevent excessive data from reducing query efficiency, historical reply messages may be cleaned up periodically.
Taking the time period of 1 month as an example, each historical reply message can be polled regularly, whether the time length of the corresponding historical reply message exceeds one month or not is judged according to the time stamp, and if the time length exceeds one month, the historical reply message can be deleted.
As another alternative embodiment, the historical reply message may be recorded in a list of historical reply messages, and the aging time may be set for the historical reply message. The aging time is a period of time, such as 1 day, 1 week, or 1 month. For each historical reply message, timing is started from the time when the historical reply message is stored in the list, and when the aging time is exceeded, the historical reply message is deleted. It will be appreciated that if the method of this embodiment is employed, then all of the historical reply messages in the list are in the "current cycle".
Optionally, the reply message stored in the history reply message list is a complete sentence.
For example, for a user problem, the manually customer service inputs a reply message of "shipment available today", and the reply message is saved in the history reply message list.
Optionally, the list of historical reply messages may also hold one or more keywords of the reply message. For example, in the above example, the keywords of the reply message may be determined to be "today", "ship", and then both keywords may be saved.
Optionally, the historical reply message list may also store a correspondence between reply messages and reply message keywords. For example, in the above example, the entries shown in table 1 may be saved.
| Reply message | Reply message keywords | Time stamp |
| Can be shipped today | Today, delivery | 2020/1/1 00:00 |
TABLE 1
Of course, the user questions corresponding to the reply messages may also be stored in the historical reply message list, which is not limited herein according to the actual requirements.
As an alternative embodiment, the number of historical reply messages that match the current reply message in the current period may be determined by steps as shown in fig. 2.
Step S1062: one or more keywords in the current reply message are obtained.
Taking the current reply message in step S104 as an example, it can be determined that the keywords in "warehouse already packaged, shipment today" are "today" and "shipment".
Step S1064: judging whether the current reply message is matched with each historical reply message in the current period according to the keywords in the current reply message.
In this step, the historical reply message in the current period is first searched. The method for searching corresponds to the method for recording the historical reply message, and taking the method for recording the historical reply message in the two embodiments as an example, the following method can be adopted to search the historical reply message in the current period.
If a method for recording the time stamp of the historical reply message is adopted, judging whether each historical reply message is in the current period or not according to the time stamp. For example, a time difference between the current time and the timestamp of the historical reply message may be calculated, and if the time difference is less than a period of time, the historical reply message is determined to be in the current period.
If the method of setting the aging time for each historical reply message is adopted when the historical reply message is recorded, the recorded historical reply messages are all in the current period, and the judgment is not needed again.
After determining the historical reply messages in the current period, the current reply message and the historical reply message can be compared one by one.
Alternatively, the keywords of the historical reply message may be compared with the keywords of the current reply message. And determining the ratio of the same or similar keywords as the keyword matching rate, and if the keyword matching rate exceeds a preset threshold, considering that the current reply message is matched with the historical reply message. The method for judging whether the keywords are similar may include whether a distance between word vectors of the keywords meets a preset condition; or a word library of synonyms can be pre-established, and the specific method is determined according to actual requirements, and is not limited herein.
It is assumed that a history reply message list is used to hold history reply messages, and that the time stamp of each history reply message is also recorded in the list, as shown in table 2.
| Sequence number | Reply message | Reply message keywords | Time stamp |
| 1 | Can be shipped today | Today, delivery | 2020/1/1 7:00 |
| 2 | Commodity is not available, please apply for return goods | No goods | 2020/1/1 8:00 |
| 3 | The delivery site is Guangzhou | Guangzhou and delivery of goods | 2020/1/1 9:00 |
TABLE 2
Assuming the current reply message is "warehouse already packaged, shipped today," the current time is 2020/1/110:00. Calculating the time difference between each historical reply message in table 2 and the current time, which is 3 hours, 2 hours and 1 hour respectively, taking the period time as 1 week as an example, it can be determined that each historical reply message in table 2 is in the current period.
Comparing the keywords of the current reply message (i.e. "today", "shipping") with the keywords of each historical reply message in table 2, it can be determined that the matching rate of the historical reply message with the sequence number of 1 and the keywords of the current reply message is 100%; the keyword matching rate of the historical reply message with the sequence number of 2 and the current reply message is 0%; the keyword matching rate of the historical reply message with the sequence number of 3 and the current reply message is 50%.
Assuming that the threshold of the keyword matching rate is 90%, it can be determined that the historical reply message with the sequence number of 1 matches the current reply message, and other historical reply messages do not match the current reply message.
In another embodiment, if only the complete sentence of the reply message is saved in the history message list and the keyword of the reply message is not saved, it is also necessary to extract the keyword of the history reply message to be compared first, and then compare the extracted keyword with the keyword of the current reply message.
Step S1066: the number of historical reply messages that match the current reply message is counted.
In the above example, it may be determined that there are 1 in total historical reply messages that match the current reply message.
Step S108: and if the number of the historical reply messages meets the preset condition, generating a question-answer pair according to the current reply message and the current user question.
The preset conditions may include a quantity threshold and a judgment condition. For example, the preset condition is satisfied when the number of historical reply messages is greater than a number threshold; or, the preset condition is satisfied when the number of the history reply messages is not less than the number threshold.
If the number of question-answer pairs in the knowledge base is large, the requirement for the establishment of question-answer pairs is generally high, and a relatively high number threshold may be set, for example, if the number of historical reply messages matched with the current reply message is not less than 10, the question-answer pairs are generated.
Alternatively, in the initial stage of knowledge base establishment, the number of questions and answers in the knowledge base is less, and generally the requirements for question and answer pair establishment are lower, a lower number threshold may be set, for example, in the case that the number of historical reply messages matched with the current reply message is not less than 1, the question and answer pair is generated.
It can be understood that, since the single answer does not have generalizability, in this embodiment, all the questions and reply messages of the user are not generated into question-answer pairs, but a threshold for generating question-answer pairs is set by using preset conditions, and the question-answer pairs reaching the generated threshold generally have better replicability and generalizability.
In this embodiment, the established question-answer pair may include a correspondence between a keyword of a user question and a current reply message, and the specific method is as follows:
first, keywords in a user question are determined according to the user question, and a specific method of determining the keywords may refer to the related art, which is not limited herein.
Still taking the example in step S104 as an example, the user question is "ask when to ship", and the keyword of the user question may be determined as "shipping time".
And then, storing the corresponding relation between the key words and the current reply message as the generated question-answer pair.
In the above example, the generated question-answer pairs may be as shown in table 3.
| User problems | Reply message |
| Delivery time | Warehouse already in packaging and shipped today |
TABLE 3 Table 3
Step S110: and storing the question-answer pairs into a knowledge base.
As an alternative embodiment, the customer service system may store the established question-answer pairs directly into the knowledge base for use in subsequent reply messages.
As another alternative embodiment, the customer service system stores the question-answer pairs passed by the manual customer service audit into a knowledge base.
Firstly, displaying the question-answer pairs to a manual customer service through a visual interface of a client, and providing auditing options, for example, displaying the question-answer pairs through a floating window by the client, and providing selectable options such as pass, reject and the like; then, the client acquires an audit result of the manual customer service aiming at the question-answer pair, and sends the audit result to the server; then, the server judges the auditing result received from the client, if the auditing result indicates that the question-answer pair passes the auditing, the question-answer pair is added into the knowledge base; and if the auditing result indicates that the question-answer pair fails to pass the auditing, deleting the question-answer pair.
Alternatively, the customer service system may be preset with two kinds of knowledge bases, one for each of the personal knowledge bases of the customer service, for use by the corresponding person of the customer service, and the other for use by all of the customer service.
For a newly established question-answer pair, the customer service system can display the question-answer pair to a manual customer service which sends a current reply message of the question-answer pair, the manual customer service audits the question-answer pair, and if the audit is passed, the question-answer pair is stored in a personal knowledge base of the manual customer service.
After the manual customer service auditing is passed, the question-answer pair can be displayed to a plurality of manual customer service in a specific auditing group, auditing results of the plurality of manual customer service are synthesized, and if the auditing is passed, the question-answer pair is stored in a public knowledge base.
As another alternative, the customer service system may store pairs of questions and answers that are not audited by the human customer service in a knowledge base. The customer service system may preset a timeout period, for example, 3 days, and if the manual customer service does not audit the question-answer pair during the timeout period, the customer service system automatically adds the question-answer pair to the knowledge base.
Thus, the flow shown in fig. 1 is completed. The steps in fig. 1 may be applied to a server of a customer service system, for example, the server may receive a reply message monitored by a client, then perform processing such as matching on the reply message, and finally generate and store a question-answer pair. Alternatively, the steps of fig. 1 may be applied to a client, for example, the client listens to the current reply message, and then performs the processes of determining the number of historical reply messages, generating and saving the question-answer pair, and the like. Of course, the steps of fig. 1 may also be performed by the client and the server separately, which will not be described herein.
By the method shown in fig. 1, a new question-answer pair can be generated. The method in the embodiment can efficiently screen reply messages with higher use frequency, has less human participation, saves labor cost, has good timeliness and has less energy consumption on the server.
By storing the generated question-answer pairs in the knowledge base, the embodiment can greatly improve the efficiency of the customer service system for replying to the message. After receiving the user problem, the customer service system can extract the keywords in the user problem and then inquire the question-answer pairs matched with the keywords in the knowledge base. If the inquiry is successful, the reply message in the question-answer pair can be sent to the user, so that the automatic reply rate is improved, and the manual participation rate is reduced; or the reply message in the question-answer pair can be displayed in a session window for manual customer service reference, and the manual customer service can directly refer to the question-answer pair for reply, thereby avoiding manual input or data reference and the like and greatly improving the reply speed.
In addition, in order to prevent excessive numbers of questions and answers in the knowledge base from affecting query efficiency, the embodiment can delete the questions and answers in the knowledge base.
Alternatively, less frequently used question-answer pairs may be deleted from the knowledge base. For example, the customer service system may preset a timeout period, e.g., 30 days, for each question-answer pair in the knowledge base, and delete the question-answer pair from the knowledge base if no user questions matching the question-answer pair are received within the timeout period.
Corresponding to the foregoing embodiments of the method for creating question-answer pairs, the present disclosure also provides embodiments of a device for creating question-answer pairs.
The embodiment of the question and answer pair device established in the specification can be applied to a server. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory by a processor of a server where the device is located. In terms of hardware, as shown in fig. 3, a schematic structural diagram of a device for creating question-answer pairs in an exemplary embodiment of the present disclosure is shown, which is used to illustrate a hardware structure of a server where the device for creating question-answer pairs is located in the present disclosure, and in addition to a processor, a memory, a network interface, and a nonvolatile memory shown in fig. 3, the server where the device is located in the embodiment generally includes other hardware according to an actual function of the server, which is not described herein again.
Fig. 4 is a block diagram of a device for creating question-answer pairs according to an exemplary embodiment of the present disclosure.
Referring to fig. 4, the device 400 for creating question-answer pairs may be applied to the server shown in fig. 3, and includes:
a monitoring unit 402, configured to monitor a paste operation of a manual customer service;
an obtaining unit 404, configured to use the text corresponding to the paste operation as a current reply message of the current user question;
a counting unit 406, configured to determine a number of historical reply messages that match the current reply message in a current period, where the historical reply message is input by the human customer service;
a judging unit 408, configured to generate a question-answer pair according to the current reply message and the current user question if the number of the historical reply messages meets a preset condition;
and a storage unit 410, configured to store the question-answer pair in a knowledge base.
Optionally, the counting unit 406 is configured to:
acquiring one or more keywords in the current reply message;
judging whether the current reply message is matched with each historical reply message in the current period according to the keywords;
and counting the number of historical reply messages matched with the current reply message.
Optionally, the generating a question-answer pair according to the current reply message and the current user question includes:
acquiring one or more keywords in the current user question;
and storing the corresponding relation between the keywords and the current reply message as the generated question-answer pair.
Optionally, the storage unit 410:
displaying the question-answer pair to the manual customer service through a visual interface;
receiving an audit result sent by the manual customer service aiming at the question-answer pair;
storing the question-answer pair to a knowledge base under the condition that the auditing result indicates that the question-answer pair passes auditing;
and under the condition that the auditing result sent by the manual customer service for the question-answer pair is not received within a first preset time, storing the question-answer pair into a knowledge base.
Optionally, the apparatus further comprises:
a first assigning unit 412 for:
when a user problem is received, inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
returning a reply message of the matched question-answer pair as a reply message of the user problem under the condition that the question-answer pair matched with the user problem exists;
In the absence of a question-answer pair matching a user question, assigning a human customer service to the user question.
Optionally, the apparatus further comprises:
a second assigning unit 414, configured to:
when receiving a user problem, assigning a manual customer service for the user problem, and inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
and in the case that the question-answer pair matched with the user problem exists, presenting a reply message of the matched question-answer pair to the manual customer service.
Optionally, the apparatus further comprises:
a deleting unit 416, configured to:
and deleting each question-answer pair in the knowledge base under the condition that the user problem matched with the question-answer pair is not received within a second preset time.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present description. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.
Corresponding to the foregoing embodiments of the method for creating a question-answer pair, the present disclosure further provides an apparatus for creating a question-answer pair, where the apparatus includes: a processor and a memory for storing machine executable instructions. Wherein the processor and the memory are typically interconnected by means of an internal bus. In other possible implementations, the device may also include an external interface to enable communication with other devices or components.
In this embodiment, the processor is caused to, by reading and executing machine-executable instructions stored by the memory corresponding to the logic for creating question-answer pairs:
Monitoring the pasting operation of the manual customer service;
taking the text corresponding to the paste operation as a current reply message of the current user problem;
determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service;
if the number of the historical reply messages meets a preset condition, generating a question-answer pair according to the current reply message and the current user question;
and storing the question-answer pairs into a knowledge base.
Optionally, the determining the number of the historical reply messages matched with the current reply message in the current period includes:
acquiring one or more keywords in the current reply message;
judging whether the current reply message is matched with each historical reply message in the current period according to the keywords;
and counting the number of historical reply messages matched with the current reply message.
Optionally, the generating a question-answer pair according to the current reply message and the current user question includes:
acquiring one or more keywords in the current user question;
and storing the corresponding relation between the keywords and the current reply message as the generated question-answer pair.
Optionally, the storing the question-answer pair in a knowledge base includes:
displaying the question-answer pair to the manual customer service through a visual interface;
receiving an audit result sent by the manual customer service aiming at the question-answer pair;
and if the auditing result indicates that the question-answer pair passes the auditing, storing the question-answer pair into a knowledge base.
Optionally, the storing the question-answer pair in a knowledge base further includes:
and if the auditing result sent by the manual customer service for the question-answer pair is not received within the first preset time, storing the question-answer pair into a knowledge base.
Optionally, the following steps are also implemented:
when a user problem is received, inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
if yes, returning the reply message in the matched question-answer pair as the reply message of the user question;
if not, assigning a manual customer service to the user problem.
Optionally, the following steps are also implemented:
when receiving a user problem, assigning a manual customer service for the user problem, and inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
And if so, displaying the reply message in the matched question-answer pair to the manual customer service.
Optionally, the following steps are also implemented:
and aiming at each question-answer pair in the knowledge base, if no user problem matched with the question-answer pair is received within a second preset time, deleting the question-answer pair from the knowledge base.
Corresponding to the foregoing embodiments of the method for creating question-answer pairs, the present disclosure further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
monitoring the pasting operation of the manual customer service;
taking the text corresponding to the paste operation as a current reply message of the current user problem;
determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service;
if the number of the historical reply messages meets a preset condition, generating a question-answer pair according to the current reply message and the current user question;
and storing the question-answer pairs into a knowledge base.
Optionally, the determining the number of the historical reply messages matched with the current reply message in the current period includes:
Acquiring one or more keywords in the current reply message;
judging whether the current reply message is matched with each historical reply message in the current period according to the keywords;
and counting the number of historical reply messages matched with the current reply message.
Optionally, the generating a question-answer pair according to the current reply message and the current user question includes:
acquiring one or more keywords in the current user question;
and storing the corresponding relation between the keywords and the current reply message as the generated question-answer pair.
Optionally, the storing the question-answer pair in a knowledge base includes:
displaying the question-answer pair to the manual customer service through a visual interface;
receiving an audit result sent by the manual customer service aiming at the question-answer pair;
and if the auditing result indicates that the question-answer pair passes the auditing, storing the question-answer pair into a knowledge base.
Optionally, the storing the question-answer pair in a knowledge base further includes:
and if the auditing result sent by the manual customer service for the question-answer pair is not received within the first preset time, storing the question-answer pair into a knowledge base.
Optionally, the following steps are also implemented:
when a user problem is received, inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
if yes, returning the reply message in the matched question-answer pair as the reply message of the user question;
if not, assigning a manual customer service to the user problem.
Optionally, the following steps are also implemented:
when receiving a user problem, assigning a manual customer service for the user problem, and inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
and if so, displaying the reply message in the matched question-answer pair to the manual customer service.
Optionally, the following steps are also implemented:
and aiming at each question-answer pair in the knowledge base, if no user problem matched with the question-answer pair is received within a second preset time, deleting the question-answer pair from the knowledge base.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The foregoing description of the preferred embodiments is provided for the purpose of illustration only, and is not intended to limit the scope of the disclosure, since any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the disclosure are intended to be included within the scope of the disclosure.
Claims (16)
1. A method of establishing question-answer pairs, the method comprising:
monitoring text entered by a human customer service assigned to reply to a current user question;
taking the input text as a current reply message of the current user problem;
determining the number of historical reply messages matched with the current reply message in the current period, wherein the historical reply messages are input by the manual customer service;
and if the number of the historical reply messages meets the preset condition, generating a question-answer pair according to the current reply message and the current user question.
2. The method of claim 1, the determining a number of historical reply messages matching the current reply message within a current period comprising:
acquiring one or more keywords in the current reply message;
judging whether the current reply message is matched with each historical reply message in the current period according to the keywords;
And counting the number of historical reply messages matched with the current reply message.
3. The method of claim 1, the generating a question-answer pair from the current reply message and the current user question, comprising:
acquiring one or more keywords in the current user question;
and storing the corresponding relation between the keywords and the current reply message as the generated question-answer pair.
4. The method of claim 1, the method further comprising:
displaying the question-answer pair to the manual customer service through a visual interface;
receiving an audit result sent by the manual customer service aiming at the question-answer pair;
and if the auditing result indicates that the question-answer pair passes the auditing, storing the question-answer pair into a knowledge base.
5. The method of claim 4, the method further comprising:
and if the auditing result sent by the manual customer service for the question-answer pair is not received within the first preset time, storing the question-answer pair into a knowledge base.
6. The method of claim 4 or 5, the method further comprising:
when a user problem is received, inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
If yes, returning the reply message in the matched question-answer pair as the reply message of the user question;
if not, assigning a manual customer service to the user problem.
7. The method of claim 4 or 5, the method further comprising:
when receiving a user problem, assigning a manual customer service for the user problem, and inquiring whether a question-answer pair matched with the user problem exists in the knowledge base;
and if so, displaying the reply message in the matched question-answer pair to the manual customer service.
8. The method of claim 4 or 5, the method further comprising:
and aiming at each question-answer pair in the knowledge base, if no user problem matched with the question-answer pair is received within a second preset time, deleting the question-answer pair from the knowledge base.
9. An apparatus for establishing question-answer pairs, the apparatus comprising:
a monitoring unit for monitoring text entered by a human customer service assigned to reply to a current user question;
the acquisition unit is used for taking the input text as a current reply message of the current user problem;
the counting unit is used for determining the number of historical reply messages matched with the current reply message in the current period, and the historical reply messages are input by the manual customer service;
And the judging unit is used for generating a question-answer pair according to the current reply message and the current user question if the number of the historical reply messages meets a preset condition.
10. The apparatus of claim 9, the counting unit to:
acquiring one or more keywords in the current reply message;
judging whether the current reply message is matched with each historical reply message in the current period according to the keywords;
and counting the number of historical reply messages matched with the current reply message.
11. The apparatus of claim 9, the generating a question-answer pair from the current reply message and the current user question, comprising:
acquiring one or more keywords in the current user question;
and storing the corresponding relation between the keywords and the current reply message as the generated question-answer pair.
12. The apparatus of claim 9, further comprising a storage unit that:
displaying the question-answer pair to the manual customer service through a visual interface;
receiving an audit result sent by the manual customer service aiming at the question-answer pair;
and storing the question-answer pair to a knowledge base in the case that the auditing result indicates that the question-answer pair passes the auditing.
13. The apparatus of claim 12, the storage unit:
and under the condition that the auditing result sent by the manual customer service for the question-answer pair is not received within a first preset time, storing the question-answer pair into a knowledge base.
14. The apparatus according to claim 12 or 13, the apparatus further comprising:
the first assignment unit is used for inquiring whether a question-answer pair matched with the user problem exists in the knowledge base or not when the user problem is received; returning a reply message of the matched question-answer pair as a reply message of the user problem under the condition that the question-answer pair matched with the user problem exists; in the absence of a question-answer pair matching a user question, assigning a human customer service to the user question.
15. The apparatus according to claim 12 or 13, the apparatus further comprising:
the second assignment unit is used for assigning a manual customer service for the user problem when the user problem is received, and inquiring whether a question-answer pair matched with the user problem exists in the knowledge base or not; and in the case that the question-answer pair matched with the user problem exists, presenting a reply message of the matched question-answer pair to the manual customer service.
16. The apparatus according to claim 12 or 13, the apparatus further comprising:
and the deleting unit is used for deleting the question-answer pairs from the knowledge base when the user questions matched with the question-answer pairs are not received within a second preset time aiming at the question-answer pairs in the knowledge base.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010946638.XA CN112131338B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010504143.1A CN111400482B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
| CN202010946638.XA CN112131338B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010504143.1A Division CN111400482B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112131338A CN112131338A (en) | 2020-12-25 |
| CN112131338B true CN112131338B (en) | 2024-02-09 |
Family
ID=71429940
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010504143.1A Active CN111400482B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
| CN202010946638.XA Active CN112131338B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010504143.1A Active CN111400482B (en) | 2020-06-05 | 2020-06-05 | Method and device for establishing question-answer pairs |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN111400482B (en) |
| WO (1) | WO2021244574A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111400482B (en) * | 2020-06-05 | 2020-09-01 | 支付宝(杭州)信息技术有限公司 | Method and device for establishing question-answer pairs |
| CN112632232B (en) * | 2021-03-09 | 2022-03-15 | 北京世纪好未来教育科技有限公司 | Text matching method, device, equipment and medium |
| CN114490975B (en) * | 2021-12-31 | 2023-02-07 | 马上消费金融股份有限公司 | User question labeling method and device |
| CN115907785B (en) * | 2022-11-22 | 2023-09-19 | 深圳市米糠云科技有限公司 | Customer service question-answering intelligent auxiliary method and device based on associated keywords |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335400A (en) * | 2014-07-22 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Method and apparatus for obtaining answer information for questioning intention of user |
| CN105786851A (en) * | 2014-12-23 | 2016-07-20 | 北京奇虎科技有限公司 | Question and answer knowledge base construction method as well as search provision method and apparatus |
| CN109308240A (en) * | 2017-07-26 | 2019-02-05 | 腾讯科技（深圳）有限公司 | Operation processing method and device, storage medium |
| CN109408624A (en) * | 2018-11-06 | 2019-03-01 | 江西师范大学 | A kind of vision chat robots session generation method |
| CN110162603A (en) * | 2018-11-30 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of Intelligent dialogue method, dynamic storage method and device |
| CN110445707A (en) * | 2019-07-23 | 2019-11-12 | 精硕科技（北京）股份有限公司 | Message processing method and device |
| CN110457453A (en) * | 2019-07-12 | 2019-11-15 | 平安普惠企业管理有限公司 | Customer problem and customer service interconnection method, device, medium, electronic equipment |
| WO2019228232A1 (en) * | 2018-05-30 | 2019-12-05 | 苏州思必驰信息科技有限公司 | Method for sharing knowledge between dialog systems, and dialog method and apparatus |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9401881B2 (en) * | 2013-09-26 | 2016-07-26 | International Business Machines Corporation | Automatic question generation and answering based on monitored messaging sessions |
| US9858335B2 (en) * | 2015-04-21 | 2018-01-02 | International Business Machines Corporation | Providing searching strategy in connection with answering question in message |
| CN111400482B (en) * | 2020-06-05 | 2020-09-01 | 支付宝(杭州)信息技术有限公司 | Method and device for establishing question-answer pairs |
-
2020
- 2020-06-05 CN CN202010504143.1A patent/CN111400482B/en active Active
- 2020-06-05 CN CN202010946638.XA patent/CN112131338B/en active Active
-
2021
- 2021-06-02 WO PCT/CN2021/097937 patent/WO2021244574A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335400A (en) * | 2014-07-22 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Method and apparatus for obtaining answer information for questioning intention of user |
| CN105786851A (en) * | 2014-12-23 | 2016-07-20 | 北京奇虎科技有限公司 | Question and answer knowledge base construction method as well as search provision method and apparatus |
| CN109308240A (en) * | 2017-07-26 | 2019-02-05 | 腾讯科技（深圳）有限公司 | Operation processing method and device, storage medium |
| WO2019228232A1 (en) * | 2018-05-30 | 2019-12-05 | 苏州思必驰信息科技有限公司 | Method for sharing knowledge between dialog systems, and dialog method and apparatus |
| CN109408624A (en) * | 2018-11-06 | 2019-03-01 | 江西师范大学 | A kind of vision chat robots session generation method |
| CN110162603A (en) * | 2018-11-30 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of Intelligent dialogue method, dynamic storage method and device |
| CN110457453A (en) * | 2019-07-12 | 2019-11-15 | 平安普惠企业管理有限公司 | Customer problem and customer service interconnection method, device, medium, electronic equipment |
| CN110445707A (en) * | 2019-07-23 | 2019-11-12 | 精硕科技（北京）股份有限公司 | Message processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 基于自动生成知识库的智能问答系统设计;王飞鸿;;中国科技信息(12);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021244574A1 (en) | 2021-12-09 |
| CN111400482A (en) | 2020-07-10 |
| CN112131338A (en) | 2020-12-25 |
| CN111400482B (en) | 2020-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112131338B (en) | Method and device for establishing question-answer pairs | |
| CN109299157B (en) | Data export method and device for distributed big single table | |
| US20120158793A1 (en) | Automatic Elimination Of Functional Dependencies Between Columns | |
| US20230126706A1 (en) | Information traceability method and system based on blockchain | |
| CN112396462B (en) | Crowd circling method and device based on click house | |
| CN108268591A (en) | It is chatted based on active and builds the method and apparatus of user's portrait | |
| JP6276279B2 (en) | Method and apparatus for graphic code database update and retrieval | |
| CN111881221A (en) | Method, device and equipment for customer portrait in logistics service | |
| CN103533152A (en) | Short message processing method and system of mobile terminal | |
| CN111737443B (en) | Answer text processing method and device and key text determining method | |
| CN107918618A (en) | Data processing method and device | |
| CN109840804A (en) | Third party's information displaying method, device, equipment and storage medium | |
| US20090083221A1 (en) | System and Method for Estimating and Storing Skills for Reuse | |
| CN111027838A (en) | Crowdsourcing task pushing method, device, equipment and storage medium thereof | |
| CN116739836B (en) | Restaurant data analysis method and system based on knowledge graph | |
| CN112184138A (en) | RPA and AI combined power grid work order processing method and device and electronic equipment | |
| CN107257358A (en) | A kind of information-pushing method, terminal and server | |
| CN109145092B (en) | Database updating and intelligent question and answer management method, device and equipment | |
| CN112016009A (en) | Data processing method, balance obtaining method, device, equipment and storage medium | |
| CN112132609A (en) | Method and device for verifying popularization effect and storage medium | |
| CN113407587B (en) | Data processing method, device and equipment for online analysis processing engine | |
| CA3168300A1 (en) | Method for selecting member data for e-commerce platforms and system thereof | |
| CN113486254B (en) | Activity recommendation method and system based on big data | |
| CN111949772A (en) | Intelligent customer service and knowledge base system and management method | |
| CN115829579A (en) | Resource data processing method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |