CN116431791A

CN116431791A - Session information processing method and device

Info

Publication number: CN116431791A
Application number: CN202310441336.0A
Authority: CN
Inventors: 刘丹; 邹波
Original assignee: Jingdong Technology Information Technology Co Ltd
Current assignee: Jingdong Technology Information Technology Co Ltd
Priority date: 2023-04-23
Filing date: 2023-04-23
Publication date: 2023-07-14

Abstract

The invention discloses a session information processing method and device, and relates to the technical field of customer service. The method embodiment comprises the following steps: splitting a single sentence from a conversation sentence generated by customer service; classifying the single sentences by using a preset positive factor model and a preset negative factor model; the positive factor model is obtained through a plurality of positive clustering cluster training classification models clustered by positive key sentences, and the negative factor model is obtained through a plurality of negative clustering cluster training classification models clustered by negative key sentences; determining a target evaluation type to which the single sentence belongs according to the evaluation type set by the classification results corresponding to the positive factor model and the negative factor model and the classification result of the single sentence; and determining service evaluation for the customer service based on a preset evaluation system containing the evaluation type and a target evaluation type to which the single sentence belongs. According to the embodiment, objective problems existing in the session service can be determined based on the session, and the accuracy of session analysis is effectively improved.

Description

Session information processing method and device

Technical Field

The present invention relates to the field of customer service technologies, and in particular, to a method and an apparatus for processing session information.

Background

On the e-commerce platform, the session between customer service and consumer can reflect the service condition of customer service and the reasons that consumers are satisfied or not satisfied with the consumption process, etc. Thus, by analyzing or processing a session between customer service and a consumer, evaluation and improvement of customer service is facilitated to enhance the consumer's consumption experience.

The current customer service evaluation method mainly divides the session between customer service and consumer into a customer dissatisfaction session or a customer satisfaction session through a classification model to evaluate whether the consumer is satisfied with the customer service, and cannot acquire specific reasons of customer dissatisfaction, and the accuracy of the session analysis result is lower.

Disclosure of Invention

In view of the above, the embodiments of the present invention provide a method and an apparatus for processing session information, which can determine objective problems existing in a session service based on a session, and effectively improve accuracy of session analysis.

To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a customer service session processing method, including:

splitting a plurality of single sentences from session sentences generated by customer service;

for each single sentence, classifying the single sentence by using a preset positive factor model and a preset negative factor model respectively; the positive factor model is obtained by training a classification model through a plurality of positive clustering clusters of positive key sentences, and the negative factor model is obtained by training a classification model through a plurality of negative clustering clusters of negative key sentences;

Determining a target evaluation type of each single sentence according to the evaluation type set by each classification result respectively corresponding to the positive factor model and the negative factor model and the classification result of each single sentence;

and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each single sentence belongs.

Optionally, the customer service session processing method further includes:

for each positive session sample or negative session sample determined, the following operations are circularly executed until a preset first condition for stopping the circulation is met:

deleting one or more sample single sentences included in the positive session sample or the negative session sample, and constructing a new sample session by using the rest sample single sentences; predicting whether the new sample session belongs to positive evaluation or negative evaluation by using a preset two-class prediction model;

and after stopping the circulation, determining positive key sentences of the positive session samples or negative key sentences of the negative session samples according to the prediction result of each circulation.

Optionally, the customer service session processing method further includes:

Clustering a plurality of positive key sentences from a plurality of positive session samples and a plurality of negative key sentences from a plurality of negative session samples respectively;

screening a plurality of positive clustering clusters and a plurality of negative clustering clusters which meet preset clustering conditions from the clustering result;

and respectively training a classification model by utilizing the plurality of positive clustering clusters and the plurality of negative clustering clusters which are screened out to meet the preset clustering condition.

Optionally, the customer service session processing method further includes:

and respectively carrying out the following operations in a circulating way aiming at each positive key sentence included in each positive clustering cluster and each negative key sentence included in each negative clustering cluster until a preset second condition for stopping circulation is met:

word segmentation is carried out on the positive key sentences or the negative key sentences,

deleting one or more words included in the positive key sentence or the negative key sentence according to the word segmentation result, and constructing a new key sentence by using the residual words;

predicting the new key sentence using the positive and negative factor models;

after stopping the circulation, determining keywords included in the positive key sentence or keywords included in the negative key sentence according to a prediction result of each circulation;

Setting an evaluation type corresponding to a classification result of the forward clustering cluster based on the determined keywords included in each forward keyword in the same forward clustering cluster;

and setting an evaluation type corresponding to the classification result of the negative clustering cluster based on the determined keywords included in each negative keyword in the same negative clustering cluster.

Optionally, the evaluation system includes a plurality of evaluation levels with association relationships, wherein the evaluation type set by each classification result is set at the last evaluation level of the evaluation system, and other evaluation levels are set with reasons for generating evaluation;

the determining a service valuation for the customer service comprises: and searching the reasons for generating the evaluation associated with the target evaluation type according to the reasons for generating the evaluation set by a plurality of evaluation levels and other evaluation levels with association relations, and determining the reasons for generating the evaluation associated with the target evaluation type as the reasons for the service evaluation of the customer service.

Optionally, the customer service session processing method further includes:

extracting customer service indicating negative evaluation from a questionnaire of the customer service;

Acquiring a conversation statement of customer service indicating negative evaluation;

the splitting the plurality of single sentences from the conversation sentences generated by the customer service comprises the following steps: a plurality of individual sentences are split from a conversation sentence indicating a negatively rated customer service.

Optionally, the determining the positive key sentence of the positive session sample or the negative key sentence of the negative session sample includes:

screening out a prediction result indicating that a new sample session corresponding to the positive session sample belongs to negative evaluation, and determining that a sample sentence lacking in the new sample session corresponding to the screened prediction result is a positive key sentence of the positive session sample;

and screening out a prediction result indicating that the new sample session corresponding to the negative session sample belongs to positive evaluation, and determining that sample single sentences lacking in the new sample session corresponding to the screened prediction result are negative key sentences of the negative session sample.

Optionally, the determining the keywords included in the positive key sentence or the keywords included in the negative key sentence includes:

and screening out new key sentences corresponding to prediction results inconsistent with the classification results of the positive key sentences or the negative key sentences, and determining the keywords lacking in the screened new key sentences as the key words of the positive key sentences or the negative key sentences.

In a second aspect, an embodiment of the present invention provides a customer service session processing apparatus, including: a sentence splitting module, a classifying module and a service evaluating module, wherein,

the sentence splitting module is used for splitting a plurality of single sentences from session sentences generated by customer service;

the classification module is used for classifying each single sentence by respectively utilizing a preset positive factor model and a preset negative factor model; the positive factor model is obtained by training a classification model through a plurality of positive clustering clusters of positive key sentences, and the negative factor model is obtained by training a classification model through a plurality of negative clustering clusters of negative key sentences;

the service evaluation module is used for determining the target evaluation type of each single sentence according to the evaluation type set by each classification result corresponding to the positive factor model and the negative factor model respectively and the classification result of each single sentence; and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each single sentence belongs.

One embodiment of the above invention has the following advantages or benefits: the positive factor model is obtained by training the classification model through a plurality of positive clustering clusters clustered by positive key sentences, and the negative factor model is obtained by training the classification model through a plurality of negative clustering clusters clustered by negative key sentences, so that the positive factor model and the negative factor model form a multi-classification model, and the positive factor model and the negative factor model have higher classification accuracy.

Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

FIG. 2 is a schematic diagram of the main flow of a customer service session processing method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the main flow of finding a key sentence according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a main flow of determining key sentences according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of the main flow of training a classification model according to an embodiment of the invention;

fig. 6 is a schematic diagram of a main flow of setting a rating type according to a search keyword of an embodiment of the present invention;

FIG. 7 is a schematic diagram of the architecture of an evaluation architecture according to an embodiment of the invention;

FIG. 8 is a schematic diagram of the main flow of a customer service session processing method according to another embodiment of the present invention;

FIG. 9 is a schematic diagram of the main modules of a customer service session processing apparatus according to an embodiment of the present invention;

fig. 10 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the prior art, the customer service is generally evaluated by a questionnaire aiming at the customer service, but as the questionnaire evaluates the customer service more singly, for example, the questionnaire only can give satisfaction or dissatisfaction to consumers, can only give the solution rate and the resolution rate of the customer service, does not know what is the specific reason of the change of the solution rate, does not have a way to quickly know the short board of the customer service, and can not guide the improvement of the customer service. In addition, the prior art also fails to provide a true cause of consumer dissatisfaction such as non-customer service causes due to backout, slower shipping, slower logistics, etc.

The method aims to solve the problems of insufficient objectivity and lower accuracy of customer service evaluation in the prior art. The embodiment of the invention provides a session information processing method and a session information processing device. Fig. 1 illustrates an exemplary system architecture 100 to which the session information processing method of the embodiment of the present invention may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, a client service server 105, and a session processing server 106. The network 104 is a medium for providing a communication link between the client service server 105 and the

terminal apparatuses

101, 102, and 103, and between the session processing server 106 and the client service server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user interacts with the client service server 105 via the network 104 using the

terminal devices

101, 102, 103 to send an evaluation questionnaire for the client service to the client service server 105, and records the dialogue content or dialogue log or the like between the user and the customer service via the

terminal devices

101, 102, 103. Various communication client applications, such as shopping applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the

terminal devices

101, 102, 103.

The session processing server 106 may be a server providing various services, which acquires customer service session contents or customer service session logs from the customer service server 105, and a background management server providing support for processing, analyzing, and the like of the customer service session contents or customer service session logs (merely by way of example).

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to desktop computers, notebooks, smartphones, tablets, etc.

It should be noted that, the session information processing method provided in the embodiment of the present invention is generally completed by the session processing server 106. Accordingly, the session information processing apparatus is installed in the session processing server 106.

It should be understood that the number of terminal devices, networks, customer service servers, and session handling servers in fig. 1 are merely illustrative. There may be any number of terminal devices, networks, customer service servers, and session handling servers, as desired for implementation.

Fig. 2 is a main flow diagram of a customer service session processing method according to an embodiment of the present invention. As shown in fig. 2, the customer service session processing method may include the following steps:

step S201: splitting a plurality of single sentences from session sentences generated by customer service;

the customer service can be artificial customer service or robot customer service.

The session statement generated by the customer service generally refers to the dialogue content between the customer service and the customer in the process that the customer service serves the customer, and the dialogue content can be the text dialogue content of text dialogue between the customer service and the customer directly through an instant messaging tool, or can be the text dialogue converted from voice dialogue.

The splitting into single sentences generally uses a complete dialogue as a single sentence. Such as a period, question mark, exclamation mark, etc., as the end of a sentence, then the sentence may be split according to punctuation (period, question mark, exclamation mark, etc.). For sentences without punctuation, separate single sentences are split according to dialogue objects, for example, the dialogue between customer service A and customer B is interactive (i.e. the dialogue process is A-B-A-B-A- …), then each dialogue content of A is ase:Sub>A single sentence, and each dialogue content of B is ase:Sub>A single sentence.

Step S202: for each single sentence, classifying the single sentence by using a preset positive factor model and a preset negative factor model respectively; the positive factor model is obtained by training a classification model through a plurality of positive clustering clusters clustered by positive key sentences, and the negative factor model is obtained by training a classification model through a plurality of negative clustering clusters clustered by negative key sentences;

the positive key sentences are single sentences which are screened from the historical conversation sentences and indicate that the customer service solves the problem, and the negative key sentences are single sentences which are screened from the historical conversation sentences and indicate that the customer service does not solve the problem. The positive factor model and the negative factor model are both multi-classification models, wherein one positive cluster is used for training one classification branch in the multi-classification model, and one negative cluster is used for training one classification branch in the multi-classification model.

Wherein, the positive clustering cluster and the negative clustering cluster are obtained by the existing text clustering technology.

Step S203: determining the target evaluation type of each single sentence according to the evaluation type set by each classification result respectively corresponding to the positive factor model and the negative factor model and the classification result of each single sentence;

step S204: and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each sentence belongs.

The classifying result and the target evaluation type of each single sentence can be comprehensively examined through the process to determine the service evaluation of the customer service, so that the service evaluation can reflect the customer service more objectively and truly.

In the embodiment shown in fig. 2, since the positive factor model is obtained by training the classification model through a plurality of positive clustering clusters of positive key sentences, and the negative factor model is obtained by training the classification model through a plurality of negative clustering clusters of negative key sentences, so that the positive factor model and the negative factor model form a multi-classification model, and the positive factor model and the negative factor model have higher classification accuracy, then the single sentences in the session sentences generated by the customer service are classified through the positive factor model and the negative factor model respectively, and the target evaluation type of each single sentence can be determined more accurately according to the evaluation type set by each classification result respectively corresponding to the positive factor model and the negative factor model and the classification result of each single sentence, thereby enabling the service evaluation for the customer service to reflect the customer service more truly, realizing objective problem of determining the existence of the session service based on the session, and effectively improving the accuracy of the session analysis.

In addition, the technical scheme provided by the invention not only can give the result of the customer service, namely the result is solved or not solved, but also can give the reason for generating the result of the customer service, namely the service evaluation, so as to better guide the improvement of the customer service.

Further, in order to enable positive key sentences and negative key sentences used for constructing the positive factor model and the negative factor model to be accurate, the accuracy of the trained positive factor model and the trained negative factor model is effectively improved. As shown in fig. 3, the above customer service session processing method may further execute the following steps S301 and S302 in a loop for each positive session sample or negative session sample determined until a preset first condition for stopping the loop is satisfied:

the first condition may be to stop after finding a key sentence, or stop when all new sample sessions that can be constructed are constructed.

Step S301: deleting one or more sample single sentences included in the positive session sample or the negative session sample, and constructing a new sample session by using the rest sample single sentences;

wherein the positive session sample and the negative session sample are a complete session between a customer service and a consumer obtained from historical session statements. In addition, the positive session sample and the negative session sample are primarily divided by the existing classification model, namely, the positive session sample is session content of which the problem is primarily divided by the classification model and the negative session sample is session content of which the problem is not primarily divided by the classification model.

Such as one example of a negative going session sample shown in table 1.

TABLE 1

Performing this step S301 process in a loop may be: sequentially deleting a single sentence to construct a new sample session, for example, deleting a single sentence with a sequence number 1, constructing a new sample session with the remaining sequence numbers 2-13, deleting a single sentence with a sequence number 2, constructing a new sample session with the remaining sequence numbers 2-13, and the like, deleting a single sentence with a sequence number 13, and constructing a new sample session with the remaining sequence numbers 1-12. Sequentially deleting the combination of the two single sentences to construct a new sample session, for example, deleting the single sentences of sequence numbers 1 and 2, constructing a new sample session by the residual sequence numbers 3 to 13, deleting the single sentences of sequence numbers 1 and 3, constructing a new sample session by the residual sequence numbers 2 and 4 to 13, and so on, constructing a new sample session by deleting the single sentences of sequence numbers 1 and 13 and the residual sequence numbers 2 to 12. For example, delete the single sentence of sequence numbers 2 and 3, the remaining sequence numbers 1 and 4-13 construct a new sample session, for example, delete the single sentence of sequence numbers 2 and 4, the remaining sequence numbers 1, 3 and 4-13 construct a new sample session, and so on, delete the single sentence of sequence numbers 2 and 13, the remaining sequence numbers 1 and 3-12 construct a new sample session, and so on. The combination of the three single sentences is sequentially deleted to construct a new sample session, e.g., delete single sentences of sequence numbers 1, 2 and 3, the remaining sequence numbers 4-13 construct a new sample session, delete single sentences of sequence numbers 1, 2 and 4, the remaining sequence numbers 2 and 4-13 construct a new sample session, and so on. There are also combinations of four single sentences in sequence, combinations of five single sentences in sequence, combinations of six single sentences in sequence, combinations of seven single sentences in sequence, combinations of eight single sentences in sequence, and the like.

Step S302: predicting whether a new sample session belongs to positive evaluation or negative evaluation by using a preset two-class prediction model;

the preset two-class prediction model is an existing positive evaluation or an unresolved negative evaluation which directly identifies the sample session as being resolved.

Illustratively, the negative session samples given in table 1 are processed by the above-described step S301 and step S302, resulting in the following table 2.

TABLE 2

Wherein, 0 corresponding to the new conversation constructed in table 2 represents deleting the single sentence corresponding to the original sentence number, and 1 corresponding to the new conversation constructed represents reserving the single sentence corresponding to the original sentence number. m represents the total number of new sessions built in total.

Step S303: after stopping the circulation, determining positive key sentences of the positive session samples or negative key sentences of the negative session samples according to the prediction result of each circulation.

For example, through the above process, analysis of the sentence in table 1 finds that after deleting the 2 nd sentence, the 9 th sentence and the 11 th sentence, the type of the conversation sentence is changed from unresolved to resolved, and then the 2 nd sentence, the 9 th sentence and the 11 th sentence in the negative direction conversation sample in table 1 can represent specific reasons that the problem is unresolved, and the 2 nd sentence, the 9 th sentence and the 11 th sentence are negative direction key sentences in table 1.

Through the process, positive key sentences and negative key sentences can be accurately searched, and interference of non-key sentences to the training process is eliminated, so that the accuracy of the trained positive factor model and negative factor model is effectively improved.

In addition, as shown in fig. 4, the specific embodiment of determining the positive key sentence of the positive session sample or the negative key sentence of the negative session sample may include the following steps:

step S401: screening out a prediction result indicating that a new sample session corresponding to a positive session sample belongs to negative evaluation, and determining that a sample single sentence lacking in the new sample session corresponding to the screened prediction result is a positive key sentence of the positive session sample;

step S402: and screening out a prediction result indicating that a new sample session corresponding to the negative session sample belongs to positive evaluation, and determining that a sample single sentence lacking in the new sample session corresponding to the screened prediction result is a negative key sentence of the negative session sample.

The specific implementation manner of determining the keywords included in the positive keywords or the keywords included in the negative keywords may include: and screening out new key sentences corresponding to prediction results inconsistent with the classification results of the positive key sentences or the negative key sentences, and determining the keywords lacking in the screened new key sentences as the key words of the positive key sentences or the negative key sentences.

Further, as shown in fig. 5, the above customer service session processing method may further include the following steps:

step S501: clustering a plurality of positive key sentences from a plurality of positive session samples and a plurality of negative key sentences from a plurality of negative session samples respectively;

the clustering process is implemented by the existing text clustering technology, and is not described herein.

Step S502: screening a plurality of positive clustering clusters and a plurality of negative clustering clusters which meet preset clustering conditions from the clustering result;

the preset clustering condition can be set correspondingly according to actual requirements, for example, the preset clustering condition can be a cluster with the number of samples of the session sample not less than 1000. The preset clustering condition can also be a cluster in which the number of samples of the session samples is ranked in the front N.

Step S503: and respectively training a classification model by utilizing the plurality of positive clustering clusters and the plurality of negative clustering clusters which are screened out to meet the preset clustering condition.

The consistency of samples used for training each classification branch can be ensured through clustering, so that the classification accuracy of the positive factor model and the negative factor model is effectively improved.

Further, the customer service session processing method may further include: step S601 to step S603 shown in fig. 6 are executed in a loop for each positive key sentence included in each positive cluster and each negative key sentence included in each negative cluster, respectively, until a preset second condition for stopping the loop is satisfied:

The second condition may be stopping after the keyword is found, or stopping after all new keywords are constructed.

Step S601: word segmentation is carried out on the positive key sentences or the negative key sentences;

the word segmentation process can be realized by selecting the existing word segmentation mode, and the list 1 and the list 2 of the 2 nd sentence given in the table 1 are listed with the list, and the list 1 and the list 2 of the word sequence number after word segmentation are shown in the following table 3 when the phone is called.

TABLE 3 Table 3

Word sequence number	Sentence segmentation	Structure sentence 1	Construction sentence 2	Construction sentence 3	......	Construction sentence m-1	Construction sentence m
								1	Ordering sheet	0	1	1		1	1
2	A kind of electronic device	1	0	1		1	1
								3	When in use	1	1	1		1	1
4	Mingming	1	1	1		1	1
								5	With goods	1	1	1		1	1
6	，	1	1	1		1	1
								7	Now	1	1	1	......	1	1
8	And also (b)	1	1	1	......	1	1
								9	Beating machine	1	1	1	......	1	1
10	Telephone set	1	1	1	......	0	1
								11	Come to	1	1	1	......	0	0
12	Say that	1	1	1	......	0	0
								14	Without goods	1	1	0	......	1	0
Prediction result	Commodity shortage	Commodity shortage	Commodity shortage	Other	......	Commodity shortage	Commodity shortage

Step S602: deleting one or more words included in the positive key sentence or the negative key sentence according to the word segmentation result, and constructing a new key sentence by using the residual words;

the specific implementation of the steps is as follows: after deleting one word in sequence, the rest of the words construct new key sentences, for example, in table 3, the words with word numbers 1 are deleted, word numbers 2-14 construct new key sentences (i.e. construct sentence 1 in table 3), the words with word numbers 2 are deleted, word numbers 1 and 3-14 construct new key sentences (i.e. construct sentence 2 in table 3), and so on; then, after sequentially deleting two words, the remaining words construct new key sentences, for example, the words with word numbers 1 and 2 in the following table 3 are deleted, the words with word numbers 3 to 14 construct new key sentences, the words with word numbers 1 and 3 are deleted, the words with word numbers 2 and 4 to 14 construct new key sentences and the like, the words with word numbers 2 and 3 are deleted, the words with word numbers 1 and 4 to 14 construct new key sentences and the like, and so on; then, after three words are sequentially combined and deleted, the rest words construct a new key sentence; after the four words are sequentially combined and deleted, the rest words construct a new key sentence; after the five words are sequentially combined and deleted, the rest words construct a new key sentence; after six words are sequentially combined and deleted, the rest words construct a new key sentence; after seven words are sequentially combined and deleted, the rest words construct a new key sentence; after eight words are sequentially combined and deleted, the rest words construct new key sentences and the like. That is, sentences formed by any word combinations can be obtained through the step.

Step S603: predicting new key sentences by utilizing the positive factor model and the negative factor model;

through this step, the new key sentence constructed is classified, a new key sentence with a changed type is screened out, then the default word of the new key sentence with a changed type is the key word of the key sentence, in the new key sentence constructed in the following table 3, when the 14 th word is absent, the classified type is no longer commodity absent, and then the 14 th word is the key sentence which is exposed when the order is placed, and the key word of no-stock is called.

Step S604: after stopping the circulation, determining keywords included in the positive key sentences or keywords included in the negative key sentences according to the prediction result of each circulation;

step S605: setting an evaluation type corresponding to a classification result of the forward clustering cluster based on the determined keywords included in each forward keyword in the same forward clustering cluster;

for example, two clusters obtained through the above process include the key sentences and the keywords of the key sentences as shown in table 4 below. And summarizing different keywords of the same cluster to obtain an evaluation type corresponding to the cluster. The summarization process may be summarized manually.

TABLE 4 Table 4

Step S606: and setting an evaluation type corresponding to the classification result of the negative cluster based on the determined keywords included in each negative keyword in the same negative cluster.

Through the process of redefining the evaluation type, standardized management of classification results is realized.

In addition, the evaluation system can comprise a plurality of evaluation levels with association relations, wherein the evaluation type set by each classification result is set at the last evaluation level of the evaluation system, and other evaluation levels are provided with reasons for generating evaluation; accordingly, embodiments of determining service valuations for customer services may include: according to the reasons for generating the evaluation, which are set by a plurality of evaluation levels and other evaluation levels with the association relationship, the reasons for generating the evaluation, which are associated with the target evaluation type, are searched, and the reasons for generating the evaluation, which are associated with the target evaluation type, are determined to be the reasons for the service evaluation of the customer service.

A schematic of the architecture of an evaluation system is shown in fig. 7. The evaluation system comprises a first level: an exogenous-global shopping experience and an endogenous-customer service shopping experience; the second tier corresponding to the exogenous-global shopping experience may be divided into dispatches, after-market, merchandise, order related, and activities corresponding to the negative-going type, the second tier corresponding to the endogenous-customer service shopping experience may be divided into service floors, service attitudes, and service capabilities corresponding to the negative-going type, and further, the second tier corresponding to the positive-going type is default regardless of exogenous-global shopping experience or endogenous-customer service shopping experience, wherein the third tier comprises: delivery timeout, delivery out of stock corresponding to delivery; corresponding to after-sale abnormality, unable price protection and invoice abnormality after sale; abnormal commodity packaging, abnormal gift and unqualified commodity corresponding to the commodity; the payment corresponding to the order is abnormal, the balance is abnormal and the white bar is abnormal; killing abnormality, coupon abnormality, discount abnormality, full-subtraction abnormality and evaluation sun-drying abnormality corresponding to the emergency purchase seconds of the activity; good commodity of the forward type corresponding to the external factor, quick logistics, low cost and the like; a revealing company secret corresponding to the service bottom line drains others; bad attitudes corresponding to service attitudes, desertification complaints, withholding each other, and emotional unclassified; communication corresponding to the service capability is not agreed upon, promised not honored, and the single scheme is repeated; the type of warmth corresponding to the internal cause, agreement on communication, promise redemption, etc. For example, if the evaluation type is determined to be a commodity shortage, the cause of the service evaluation of the customer service may be determined according to the evaluation level distribution, the negative type and the exogenous-overall shopping experience of the association relationship between the commodity shortage and the commodity shortage: externally caused by negative type of distribution in the overall shopping experience.

Further, the customer service session processing method may further include: extracting customer service indicating negative evaluation from a questionnaire of the customer service; acquiring a conversation statement of customer service indicating negative evaluation; accordingly, an embodiment of splitting a plurality of single sentences from a conversation sentence generated by a customer service may include: a plurality of individual sentences are split from a conversation sentence indicating a negatively rated customer service. The customer service evaluation method and the customer service evaluation system are mainly used for further analyzing the reasons of negative evaluation aiming at the customer service which is evaluated as negative evaluation in advance, so that the customer service can be better improved while the workload is reduced.

In summary, the above-mentioned customer service session processing procedure is mainly divided into two parts: one part is a process of constructing a positive factor model and a negative factor model, setting classification types for each classification branch of the positive factor model and the negative factor model and determining an evaluation system, and the other part is a process of evaluating customer service session. The following describes in detail the processing procedure included in the customer service session processing method in a specific example. As shown in fig. 8, the customer service session process may include the steps of:

step S801: constructing a positive factor model and a negative factor model, setting an evaluation type for each classification branch of the positive factor model and the negative factor model, and constructing an evaluation system for the evaluation type;

The specific implementation of this step may include: respectively searching positive key sentences of the positive session samples and negative key sentences of the negative session samples; clustering positive key sentences and negative key sentences respectively; and training the classification model by using key sentences in the clustered clusters to obtain a positive factor model and a negative factor model. And further searching keywords from the positive keywords and the negative keywords respectively, and setting evaluation types for keyword integration.

Step S802: splitting a plurality of single sentences from session sentences generated by customer service;

for example, a single sentence obtained by splitting a conversational sentence for one example is shown in table 5 below.

TABLE 5

Step S803: for each sentence in table 5, classifying the sentence by using a preset negative factor model;

for conversational sentences that have been determined to be negatively rated, the individual sentences are classified directly by the negative factor model.

Step S804: determining a target evaluation type of each single sentence according to the evaluation type set by each classification result and the classification result of each single sentence respectively corresponding to the negative factor model;

the classification types of the individual sentences are shown in table 5.

Step S805: and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each sentence belongs.

The target rating types for Table 5 are delivery timeout and out of stock, then the service rating is a negative rating due to a negative delivery cause under an exogenous-global shopping experience, not an endogenous customer service cause.

Fig. 9 is a schematic structural diagram of a customer service session processing device according to an embodiment of the present invention. As shown in fig. 9, the customer service session processing apparatus 900 may include: a sentence splitting module 901, a classifying module 902, and a service evaluation module 903, wherein,

the sentence splitting module 901 is configured to split a plurality of single sentences from a conversation sentence generated by a customer service;

the classification module 902 is configured to classify each sentence by using a preset positive factor model and a preset negative factor model; the positive factor model is obtained by training a classification model through a plurality of positive clustering clusters clustered by positive key sentences, and the negative factor model is obtained by training a classification model through a plurality of negative clustering clusters clustered by negative key sentences;

the service evaluation module 903 is configured to determine, according to an evaluation type set by each classification result corresponding to the positive factor type and the negative factor type, and a classification result of each single sentence, a target evaluation type to which each single sentence belongs; and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each sentence belongs.

In an embodiment of the present invention, as shown in fig. 9, the customer service session processing apparatus 900 may further include: the key sentence retrieval module 904, wherein,

the key sentence retrieval module 904 is configured to, for each positive session sample or negative session sample determined, circularly execute deletion of one or more sample sentences included in the positive session sample or the negative session sample, construct a new sample session by using the remaining sample sentences, and predict whether the new sample session belongs to positive evaluation or negative evaluation by using a preset binary classification prediction model until a first condition of a preset stop cycle is satisfied; and after stopping the circulation, determining positive key sentences of a positive session sample or negative key sentences of the negative session sample according to the prediction result of each circulation.

In an embodiment of the present invention, as shown in fig. 9, the customer service session processing apparatus 900 may further include: training module 905 is configured to provide, among other things,

training module 905, configured to cluster a plurality of positive key sentences derived from a plurality of positive session samples and a plurality of negative key sentences derived from a plurality of negative session samples, respectively; screening a plurality of positive clustering clusters and a plurality of negative clustering clusters which meet preset clustering conditions from the clustering result; and respectively training a classification model by utilizing the plurality of positive clustering clusters and the plurality of negative clustering clusters which are screened out to meet the preset clustering condition.

In an embodiment of the present invention, as shown in fig. 9, the customer service session processing apparatus 900 may further include: a keyword retrieval module 906, and a model setting module 907, wherein,

a keyword retrieval module 906, configured to perform word segmentation on the positive keyword or the negative keyword in a circulating manner for each positive keyword included in each positive cluster and each negative keyword included in each negative cluster, delete one or more terms included in the positive keyword or the negative keyword according to the word segmentation result, and construct a new keyword by using the remaining terms; predicting new key sentences by utilizing the positive factor model and the negative factor model until a preset second condition for stopping circulation is met; after stopping the circulation, determining keywords included in the positive key sentences or keywords included in the negative key sentences according to the prediction result of each circulation;

a model setting module 907, configured to set an evaluation type corresponding to a classification result of the forward cluster based on the determined keywords included in each forward keyword in the same forward cluster; and setting an evaluation type corresponding to the classification result of the negative cluster based on the determined keywords included in each negative keyword in the same negative cluster.

In the embodiment of the invention, the evaluation system comprises a plurality of evaluation levels with association relations, wherein the evaluation type set by each classification result is set at the last evaluation level of the evaluation system, and other evaluation levels are provided with reasons for generating evaluation;

the service evaluation module 903 is further configured to search for a reason for generating an evaluation associated with the target evaluation type according to the reasons for generating the evaluation set by the plurality of evaluation levels and the other evaluation levels having the association relationship, and determine that the reason for generating the evaluation associated with the target evaluation type is a reason for service evaluation of the customer service.

In the embodiment of the present invention, the sentence splitting module 901 is further configured to extract a customer service indicating a negative evaluation from a questionnaire of the customer service; acquiring a conversation statement of customer service indicating negative evaluation; a plurality of individual sentences are split from a conversation sentence indicating a negatively rated customer service.

In the embodiment of the present invention, the keyword search module 904 is further configured to screen out a prediction result indicating that a new sample session corresponding to a positive session sample belongs to negative evaluation, and determine that a sample sentence missing in the new sample session corresponding to the screened out prediction result is a positive keyword of the positive session sample; and screening out a prediction result indicating that a new sample session corresponding to the negative session sample belongs to positive evaluation, and determining that a sample single sentence lacking in the new sample session corresponding to the screened prediction result is a negative key sentence of the negative session sample.

In the embodiment of the present invention, the keyword search module 906 is further configured to screen out a new keyword corresponding to a prediction result inconsistent with the classification result of the positive keyword or the negative keyword, and determine that the word missing from the screened new keyword is the keyword of the positive keyword or the negative keyword.

Referring now to FIG. 10, there is shown a schematic diagram of a computer system 1000 suitable for use in implementing a server hosting an electronic prescription processing device in accordance with an embodiment of the present invention. The server illustrated in fig. 10 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.

As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can execute various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk or the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The drive 1010 is also connected to the I/O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed in the drive 1010, so that a computer program read out therefrom is installed as needed in the storage section 1008.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 1009, and/or installed from the removable medium 1011. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 1001.

The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a statement splitting module, a classification module, and a service evaluation module. The names of these modules do not limit the module itself in some cases, and for example, the sentence splitting module may also be described as "a module that splits a plurality of individual sentences from a conversational sentence generated by a customer service".

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: splitting a plurality of single sentences from session sentences generated by customer service; classifying each single sentence by using a preset single sentence; the positive factor model is obtained by training a classification model through a plurality of positive clustering clusters clustered by positive key sentences, and the negative factor model is obtained by training a classification model through a plurality of negative clustering clusters clustered by negative key sentences; determining the target evaluation type of each single sentence according to the evaluation type set by each classification result respectively corresponding to the positive factor model and the negative factor model and the classification result of each single sentence; and determining service evaluation for the customer service based on a preset evaluation system containing evaluation types and a target evaluation type to which each sentence belongs.

According to the technical scheme of the embodiment of the invention, the positive factor model is obtained by training the classification model through a plurality of positive clustering clusters formed by clustering positive key sentences, and the negative factor model is obtained by training the classification model through a plurality of negative clustering clusters formed by clustering negative key sentences, so that the positive factor model and the negative factor model form a multi-classification model, and the positive factor model and the negative factor model have higher classification accuracy, then the single sentences in the conversation sentences generated by customer service are respectively classified through the positive factor model and the negative factor model, and the target evaluation type of each single sentence can be more accurately determined according to the evaluation type set by each classification result respectively corresponding to the positive factor model and the negative factor model and the classification result of each single sentence, thereby enabling the service evaluation aiming at the customer service to more truly reflect the customer service, realizing the objective problem of determining the existence of the conversation service based on the conversation, and effectively improving the accuracy of conversation analysis.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. The customer service session processing method is characterized by comprising the following steps of:

2. The customer service session processing method according to claim 1, further comprising:

3. The customer service session processing method according to claim 1 or 2, further comprising:

4. The customer service session processing method according to claim 1 or 2, further comprising:

predicting the new key sentence using the positive and negative factor models;

5. The customer service session processing method according to claim 1, wherein,

the evaluation system comprises a plurality of evaluation levels with association relations, wherein the evaluation type set by each classification result is set at the last evaluation level of the evaluation system, and other evaluation levels are provided with reasons for generating evaluation;

6. The customer service session processing method according to claim 1, further comprising:

7. The customer service session processing method according to claim 2, wherein the determining the positive key sentence of the positive session sample or the negative key sentence of the negative session sample includes:

8. A customer service session processing method according to claim 3, wherein the determining the keywords included in the positive key sentence or the keywords included in the negative key sentence includes:

9. A customer service session processing apparatus, comprising: a sentence splitting module, a classifying module and a service evaluating module, wherein,

10. An electronic device, comprising:

One or more processors;

storage means for storing one or more programs,

when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-8.

11. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-8.