TW201935370A - System and method for evaluating customer service quality from text content - Google Patents

Abstract

A system and method for evaluating customer service quality from text content are disclosed. The method comprises: performing speech segmentation, speech-to-text and text parsing for dialogue voice records between a customer and a customer service staff to obtain a dialogue vocabulary therebetween; performing word vector conversion for the dialog vocabulary to obtain the word vector, and getting a customer emotion analysis value from the word vector to obtain a result of customer satisfaction; performing intent and keyword analysis for the dialog vocabulary to obtain the intent and the keyword of the dialog vocabulary, finding out a model answer of a customer question from a task list and a knowledge database according to the intent and the keyword, and comparing similarity between answer content of the customer service staff and the model answer of the knowledge database to obtain professional degree of the customer service staff; and optimizing service content evaluation according to the result of the customer satisfaction or the professional degree of the customer service staff.

Description

System and method for evaluating customer service quality from text content

The present invention relates to a technology for evaluating customer service quality, and particularly to a system and method for evaluating customer service quality from text content.

Most of the existing customer service quality testing is done by manual listening. In addition, due to the limited raters, each customer service staff will only be drawn out a small number of calls (such as single-digit communication) each month to score.

In addition, there will be different standards in the evaluation performed by the raters. Because the raters will make scores in accordance with their own subjective consciousness, there will be unstable or uncontrollable factors. Therefore, it is necessary to calibrate the scores for all the raters on a regular basis. operation.

In addition, some conventional technologies use the historical data of customers and companies for analysis to evaluate the ability of customer service, but it is easy to cause the distortion of the evaluation due to the current situation of customers.

Therefore, how to solve the shortcomings of the conventional techniques has become a major issue for those skilled in the art.

The present invention provides a system and method for evaluating customer service quality from text content, which can replace a rater to automatically evaluate customer service quality, so as to provide a more accurate and rapid evaluation technology.

The system for assessing customer service quality from text content in the present invention includes: a voice-to-speech module that sequentially analyzes the voice data of the dialogue between the customer and the customer service, and performs speech segmentation, speech-to-text, and text analysis in order to obtain the customer and customer service Dialogue vocabulary; a customer satisfaction analysis module that converts the dialogue vocabulary of the speech-to-speech module into word vectors to obtain the word vector, and then obtains the semantic vector of the paragraph of the dialogue vocabulary from the word vector, and then converts the semantic meaning Input the sentiment analysis model to obtain the sentiment analysis value of the customer, and predict the customer satisfaction analysis result based on the sentiment analysis value of the customer; a professional degree analysis module, which analyzes the vocabulary of the speech-to-semantic module for intent and keyword analysis, In order to obtain the intention and keywords of the dialogue vocabulary, and then query the standard answers of the customer's questions from the task list and the knowledge base according to the intents and keywords, to compare the similarity between the response content of the customer service and the standard answers of the knowledge base To obtain customer service professionalism; and an optimized service content evaluation module, which is based on the results of customer satisfaction analysis or Degree in professional clothes and pick out the differences in scores artificial score exceeds the threshold value were to optimize the task, according to the table, the knowledge base, the word vector model and sentiment analysis model in which at least one.

The method for assessing the quality of customer service from text content in the present invention includes: sequentially performing speech segmentation, speech-to-text, and text analysis on the speech data of the dialogue between the customer and the customer service to obtain the dialogue vocabulary between the customer and the customer service; and performing the conversation vector on the word vector Convert to get the word vector, and then get the semantic vector of the paragraph of the dialogue vocabulary from the word vector, and then input the semantic vector to the emotional score Analyze the model to obtain the customer sentiment analysis value, and predict the customer satisfaction analysis result based on the customer sentiment analysis value; analyze the dialogue vocabulary for intent and keywords to obtain the dialogue vocabulary intention and keywords, and then use the intent and keywords to get the task The standard answers to customer questions are queried in tables and knowledge bases, and the professional level of customer service is obtained by comparing the similarity between the response content of customer service and the standard answers in the knowledge base; and from the analysis results of customer satisfaction or professional level of customer service Pick out those who differ from the artificial scores by more than a threshold, and use them to optimize at least one of the task list, knowledge base, word vector model, and sentiment analysis model.

In order to make the above features and advantages of the present invention more comprehensible, embodiments are described below in detail with reference to the accompanying drawings. Additional features and advantages of the present invention will be partially explained in the following description, and these features and advantages will be partially obvious from the description, or may be learned through practice of the present invention. The features and advantages of the invention are realized and achieved by means of elements and combinations specifically pointed out in the scope of the patent application. It should be understood that both the foregoing general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the scope of the invention as claimed.

1‧‧‧ System for evaluating customer service quality from text content

10‧‧‧Voice-to-speech module

11‧‧‧ Phonetic Sentence Unit

12‧‧‧ Speech-to-text unit

13‧‧‧Text Analysis Unit

2‧‧‧ Semantic Analysis Module

20‧‧‧Customer Satisfaction Analysis Module

21‧‧‧Word Vector Conversion Unit

22‧‧‧Satisfaction trajectory change prediction unit

30‧‧‧Professional Level Analysis Module

31‧‧‧ Intent and Keyword Analysis Unit

32‧‧‧ Inquiry for reply task unit

33‧‧‧Task List

34‧‧‧Query Standard Answer Unit

35‧‧‧ Knowledge Base

36‧‧‧Content Similarity Comparison Unit

40‧‧‧Optimized service content evaluation module

41‧‧‧Adjust professional analysis model

42‧‧‧Adjust customer satisfaction analysis model

43‧‧‧Word Vector Model

44‧‧‧ sentiment analysis model

A‧‧‧Customer service voice input

B‧‧‧ Customer Service Quality Score

C‧‧‧ Customer service manual scoring result input

D‧‧‧ Customer Service Text Semantic Record

E‧‧‧Client conversation record

F‧‧‧Conversation record of client

G‧‧‧similarity

H‧‧‧Automatic analysis results and manual scoring results

Steps S1 to S4‧‧‧‧

Figure 1 shows a schematic block diagram of the system for evaluating customer service quality from text content in the present invention; Figures 2A to 2D show the speech-to-speech module and customer satisfaction analysis module in Figure 1 of the present invention, respectively. Schematic block diagram of group, professional level analysis module and optimization service content evaluation module; Figure 3 shows the schematic diagram of the voice input of the client and the client in the present invention 4A and 4B are schematic diagrams of semantic vector conversion of a word vector conversion unit in the present invention; FIG. 5A is a schematic diagram of training and application of an emotional analysis model in the present invention; FIG. 5B FIG. 6 is a schematic diagram of a change line of a satisfaction trajectory composed of a customer sentiment analysis value and a customer inquiry sentence in the present invention; FIG. 6 is a schematic diagram of an embodiment of the professional degree analysis module in FIG. 2C of the present invention; The figure shows a schematic flowchart of a method for evaluating customer service quality from text content in the present invention.

The following describes the embodiments of the present invention with specific specific implementation forms. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this description, and can also be implemented by other different specific implementation forms. Or apply.

FIG. 1 is a schematic block diagram showing the system 1 for evaluating customer service quality from text content in the present invention. FIGS. 2A to 2D show the speech-to-speech module 10 in the first aspect of the present invention and customer satisfaction. Schematic block diagrams of degree analysis module 20, professional degree analysis module 30, and optimized service content evaluation module 40.

As shown in Figure 1, the system 1 for evaluating customer service quality from text content includes a voice-to-speech module 10, a semantic analysis module 2 with a customer satisfaction analysis module 20 and a professional level analysis module 30. And an optimization Service content evaluation module 40.

As shown in FIG. 2A, the speech-to-speech module 10 in FIG. 1 is composed of a speech segmentation unit 11, a speech-to-text unit 12, and a text analysis unit 13. As shown in FIG. 2B, the customer satisfaction analysis module 20 in FIG. 1 is composed of a word vector conversion unit 21 and a satisfaction trajectory change prediction unit 22. As shown in FIG. 2C, the professional degree analysis module 30 in FIG. 1 includes an intention and keyword analysis unit 31, a query response task unit 32, a task table 33, a query standard answer unit 34, a knowledge base 35, and content similarity. The comparison unit 36 is composed. As shown in FIG. 2D, the optimization service content evaluation module 40 in FIG. 1 is composed of an adjustment professional analysis model 41, an adjustment customer satisfaction analysis model 42, a word vector model 43 and an emotion analysis model 44. Or connect task table 33 and knowledge base 35 in FIG. 2C.

As shown in Fig. 1 and Fig. 2A, the voice-to-speech module 10 can convert the voice data of the conversation between the customer and the customer service (such as customer service voice input A), in order from the speech segmentation unit 11, speech to text unit 12, The analysis unit 13 performs speech segmentation, speech-to-text, and text analysis, respectively, to obtain the dialogue vocabulary of the customer and the customer service (such as the customer service customer text semantic record D).

Specifically, in the voice-to-speech module 10 in FIG. 1 and FIG. 2A, the continuous voice signals of the customer service and the client (such as customer service voice input A) can be input to the voice-to-speech module 10 so that The voice segmentation unit 11 captures the voice signal of the entire recorded conversation between the customer and the customer service, and then uses the energy of the voice signal as a judgment condition to cut the voice signal into a conversation (may consist of one or more sentences) The signal for the unit. Then, send the voice signal that has been cut into paragraphs into a voice-to-text list Element 12 (such as a speech-to-text decoder) obtains a candidate word map, and then selects a path with the best score from the candidate word map as the recognition result. This recognition result is the text data representing the semantic content. Then, the word data is used to obtain the word content in terms of words and the vocabulary of key words representing a specific entity through word segmentation and entity recognition.

In the customer satisfaction analysis module 20 shown in FIG. 1 and FIG. 2B, the conversation vocabulary of the speech-to-speech module 10 can be converted into a word vector to obtain a word vector, and the paragraph of the conversation vocabulary can be obtained from the word vector. The semantic vector is then input to the sentiment analysis model 44 to obtain a customer sentiment analysis value, and a customer satisfaction analysis result is predicted based on the customer sentiment analysis value.

In summary, the word vector conversion unit 21 of the customer satisfaction analysis module 20 can convert the word vocabulary of the speech to semantic module 10 (such as the client conversation record E) to obtain a word vector, and then extract the word vector from the word vector. Find the semantic vector of the paragraph of the dialogue vocabulary. The satisfaction trajectory change prediction unit 22 of the customer satisfaction analysis module 20 can input the semantic vector of the paragraph of the dialogue vocabulary into the sentiment analysis model to obtain the sentiment analysis value of the customer, and obtain the change trend of the satisfaction from several sentiment analysis values of the customer. Line, showing the analysis results of customer satisfaction, and the trend line of satisfaction change can be calculated by the formula Y = βX + ε (see Figure 5B).

Specifically, in FIG. 2B, from the customer service text semantic record D, which is sufficient to represent the interaction information between the customer and the customer service, all client conversation records E in a conversation unit E to the word vector conversion unit 21 are taken out, Segments to filter vocabulary that has nothing to do with emotion, and then perform word vector For conversion, the word vector conversion unit 21 can be combined with the word vector model 43 to convert words (such as vocabulary combinations) into word vectors (such as numerical values). Next, each word can get a fixed-length multi-dimensional word vector. By arithmetically averaging several multi-dimensional word vectors in this paragraph, the semantic vector of the paragraph (such as the output result in Figure 4B) is input, and then input The sentiment analysis model 44 trained in advance can calculate the sentiment analysis value in units of paragraphs, and after statistically completing the sentiment values of customers in all the dialogue paragraphs, then use linear regression to find out the trend line of customer satisfaction changes in the phone call. This trend line is the result of customer satisfaction analysis.

In the professional degree analysis module 30 in FIG. 1 and FIG. 2C, the intent and keyword analysis unit 31 may perform intent and keyword analysis on the conversation vocabulary (such as the client conversation record E) of the speech to semantic module 10, To get the intent and keywords of each paragraph in the conversation vocabulary. In summary, the intent and keyword analysis unit 31 can obtain the intent and keywords of each paragraph in the dialogue vocabulary (such as the client dialogue record E) with a machine learning algorithm.

At the same time, the query response task unit 32 can query the task table 33 to obtain the tasks that the customer service needs to engage in, and then a query system standard answer unit 34 queries the standard answer of the customer's question from the knowledge base 35 based on the intent and keywords. The similarity G between the response content of the customer service (the customer service conversation record F) and the standard answer obtained by the knowledge base 35 is compared by the content similarity comparison unit 36, and the result of the similarity G is used as the professional level of customer service.

In the optimized service content evaluation module 40 in Fig. 1 and Fig. 2D, cases with large score differences (such as exceeding the first threshold) can be selected from the automatic analysis result and the manual scoring result H, and adjusted accordingly. Task list 33 and knowledge Library 35, and then adjust the word vector model 43 and sentiment analysis model 44 by adjusting the customer satisfaction analysis model 42, and then update the task list 33, knowledge base 35, word vector model 43, and sentiment analysis model 44 to Figure 2B and The corresponding model of the 2C chart achieves the effect of optimizing the evaluation of service content.

For example, the optimized service content assessment module 40 can compare the professionalism of customer service with the scorer's score results (scores of manual scores) to find cases with large score differences (such as exceeding the first threshold) for analysis. Then, the adjustment of the score difference between the two by updating the task table 33 or the knowledge base 35 through an adjustment professional analysis model 41. Alternatively, the optimized service content evaluation module 40 may compare the customer satisfaction analysis result with the scorer's score result (a score manually scored) to find cases with large score differences (such as exceeding the first threshold). Perform analysis, and then update the word vector model 43 and the sentiment analysis model 44 to adjust the score difference between the two.

Specifically, in FIG. 2D, the optimized service content evaluation module 40 can obtain the customer satisfaction change trend line (from the result of customer satisfaction analysis) and the similarity score (from the professionalism of customer service) (Compared with the results of the evaluation), and combined with the results of the scorers who manually filled out in advance to optimize the customer satisfaction analysis module 20 and professional level analysis module 30. Since the results of both automatic scoring and manual scoring have been obtained, the cases with large gaps between the two are updated through the score gap comparison. If it is a paragraph with a large slope gap in the trend line of customer satisfaction, you can retrain the word vector model 43 and sentiment analysis model 44 by retraining the word vector and the classification method of machine learning to bring the sentiment analysis results closer. Results for manual annotation. If it is the result of the professional evaluation of customer service The difference is large, and it may be that the content of the task list 33 or the knowledge base 35 must be adjusted accordingly due to factors such as policies or time (such as adding or deleting task lists, and updating the knowledge base in accordance with the new promotion plan).

FIG. 3 is a schematic diagram showing voice input of a client and a client in the present invention. As shown in FIG. 3, the voice-to-speech module 10 in FIG. 2A takes out the audio files on both sides of the customer and customer service from the recording equipment of the customer service line. The audio files on both ends are separated and do not mix or interfere with each other. The voice signals of the client and the client are distinguished by the left and right channels of the audio file (such as the voice input A of the customer service).

At the same time, the voice segmentation unit 11 in Figure 2A captures the voice signal of the complete conversation between the customer and the customer service, and then cuts the voice signal into paragraphs based on the energy level of the voice signal as a judgment condition, and the voice of each paragraph The signals are sorted in chronological order.

Next, the speech-to-text unit 12 in FIG. 2A sends the speech signal that has been cut into paragraphs to the speech-to-text decoder to obtain a candidate word map, and then selects a best score from the candidate word map. The path is used as the recognition result, and this recognition result is the text data representing the speech content. As shown in Figure 3, paragraph one for the customer service said: "China Telecom Good Afternoon, and the surname Zhong is honored to serve you." Section two is: "Miss, I want to complain to you NCC for this bad service." Three: "I'm really sorry, what can we do for you?"

Then, the text analysis unit 13 in FIG. 2A obtains the text data obtained by word segmentation and entity recognition through the word segmentation unit 13 and the keyword collection representing the specific entity, that is, the “NCC” of the semantic analysis result in FIG. 3 " And "Bad Services" and other terms, only with the "company line number" label and "complaint" label generated by the entity identification.

Before performing the customer satisfaction analysis module 20 in FIG. 2B, the sentiment analysis model 44 used in the process must be trained in advance. The purpose of training the sentiment analysis model 44 is to obtain the sentiment analysis value of the customer, and the sentiment analysis model is trained. The input part is the semantic vector of the paragraph, and the output part is the manually analyzed sentiment analysis value. Therefore, the entire text data must be converted into the semantic vector of the paragraph.

4A and 4B are schematic diagrams illustrating semantic vector conversion of a word vector conversion unit in the present invention. As shown in Figure 4A, first collect the text training data recorded by the customer service line and processed by word segmentation, and use paragraphs as the unit. For example, the client in Figure 4A said, "Miss, I want to complain to you NCC for your bad service." Set a text box with a size of 5 words. The text box will move from the left to the right of the sentence to the end of the sentence. And set the word in the middle (the third word from the left) as the input word, and the words in other text boxes as adjacent words.

Each word in the text training data will correspond to a number in the dictionary, and the total number of words in the text training data will be equal to the dimensional length (N) of the input word vector and the adjacent word vector in Figure 4A. Therefore, it is assumed in the dictionary There are 1000 vocabularies, and the number of the word "you" is 2. Therefore, the input word vector I _1XN with a dimension length of 1000 is only 1 in the second dimension and 0 in the other dimensions. Similarly, the adjacent word vector O _{1xN is} composed of 4 words (complaint), NCC, this kind, and bad service (because the text box length is only 5 words), so this vector has only the first and the first words. 3. The 4th and 5th dimensions are 1 and the other dimensions are 0. Here, the method adopted by the present invention is Chinese word vector conversion (Word2vec). When the input word vector and the adjacent word vector are known, the word vector representing semantic information is obtained through the Chinese word vector conversion (Word2vec). Dimensions can be reduced from high to low.

As shown in FIG. 4B, the word vector conversion unit 21 in FIG. 2B can convert each word through the word vector to obtain a corresponding word vector (the number of dimensions is M), and perform arithmetic on each dimension of all the word vectors. Average to get the semantic vector of this paragraph (the number of dimensions is M).

FIG. 5A is a schematic diagram showing the training of the emotion analysis model and the application of the emotion analysis model in the present invention, and FIG. 5B is a diagram showing the change line of the satisfaction trajectory formed by the customer's emotion analysis value and the customer's query in the present invention. schematic diagram.

In FIG. 5A, the satisfaction trajectory change prediction unit 22 in FIG. 2B can input the semantic vector of the paragraph through the sentiment analysis model 44 to obtain the customer sentiment analysis value. The training of the sentiment analysis model 44 is as shown in FIG. 5A. Shown in the left half. Under the condition that the input layer of the entire word vector is input and the output layer is filled with the customer sentiment analysis value (in this embodiment, this part is manually labeled, and the customer sentiment of each paragraph in the training data is first analyzed. Artificial scoring of analysis values), a deep neural network model representing the sentiment analysis model 44 is trained through a deep learning algorithm.

The application of sentiment analysis model 44 is shown in the right half of Figure 5A. Enter the entire average sentiment word vector to calculate a customer sentiment analysis value. The value range of customer sentiment analysis value is between 0 and 1. Close to 0 table Indicates that the customer is in a negative mood, and closer to 1 indicates that the customer is in a positive mood.

As shown in Figure 5B, the customer sentiment analysis value of each passage in the conversation can be combined into Y, and each passage can be combined into X in accordance with the time sequence, which is brought into the formula Y = βX + ε by linear regression, and accordingly Calculate the slope β of the satisfaction trajectory change line (such as a diagonal line), where ε represents the initial value of the customer's emotions. The slope β represents the change of customer satisfaction with time series, and customer satisfaction can be divided into one or more levels for judging whether the customer service properly appeases the customer's mood. If the slope β of the change line of the satisfaction trajectory indicates that the customer's satisfaction has increased due to the appropriate explanation of the customer service, the downward slope indicates that the customer service has not properly comforted the customer's emotions. This satisfaction trend line is the output of the customer satisfaction analysis.

FIG. 6 is a schematic diagram illustrating an embodiment of the professional degree analysis module 30 in FIG. 2C of the present invention.

The professional degree analysis module 30 in FIG. 6 and FIG. 2C is to input the client conversation record E (text form), and the intention and keyword analysis unit 31 is used to detect the customer's intention and keywords for each question in paragraphs. The categories of customer intents can be manually defined from the questions often mentioned in customer service lines. In addition to the categories of intents related to business, these customer intent categories also contain greeting greetings and confirmations to confirm personal information. The intention of a paragraph of customer questions is to filter in advance questions that are not related to customer service professionalism.

Keyword analysis In order to find other necessary information for matching intents, if the intent category is billing information, the keywords that must be matched are device numbers or Billing month so that subsequent inquiries can be made. The intent and keyword analysis unit 31 can predict the intent and keywords using a pattern matching method.

As shown in Figure 6, the customer said: "My mobile phone number 0912345678 requires a SIM card to unlock." After analysis by the intent and keyword analysis unit 31, the intent is "unlocked" and the keyword is "device number (0912345678)" ".

After knowing the information of the intent and keywords, the query response task unit 32 can query the task table 33 to obtain what kind of corresponding task response the customer service should do. To determine whether the intent category of this paragraph needs professional analysis, the task table 33 can be used. Define it. For example, greetings do not require follow-up professional analysis. After querying the task list 33, the customer service task content is "certified".

Next, the standard answer unit 34 of the query system in FIG. 6 queries the knowledge base 35 that has been established in advance to obtain the standard answer. The standard answer obtained is "Is this the person? The name of the device holder?" Then, the content similarity comparison unit 36 compares the content of the customer service conversation record F with the standard answer of the knowledge base 35. The similarity comparison method can compare the customer service conversation record F with the knowledge base 35. The vocabulary of each paragraph of the standard answer is converted into the semantic vector of the paragraph. That is, after removing the content of the vocabulary of the paragraph, the vocabulary is converted into a word vector, and the same dimensions of different words are added up and averaged to obtain Two semantic vectors V (customer service response) and semantic vector V (standard answer) representing the paragraph, and then substitute the semantic vector V (customer service response) and the semantic vector V (standard answer) into formula 1 below to calculate Cosine similarity, a higher similarity indicates a professional customer service process High degree and low similarity means customer service cannot accurately grasp customer issues.

In addition, as shown in FIG. 2D, in order to narrow the difference between the automatic analysis result and the manual scoring result H, it can be achieved through the optimized service content evaluation module 40, and the similarity score (the evaluation result of the professionalism of the customer service), Customer satisfaction change trend line (customer satisfaction analysis result), and manual fill-in of the scorer's evaluation results (including customer satisfaction analysis and customer service professionalism) are automatically compared with each other to find the difference. Update.

Furthermore, the professional degree analysis model 41 of the 2D graph is adjusted. If the difference in the judgment result of professional degree exceeds the second threshold value (because the cosine similarity distance is used, the second threshold value is preset to 0.5) to indicate that it is professional. In cases where the results of the evaluation differ greatly, the content of the task list or the knowledge base must be adjusted due to policy or time factors. For example, re-examine whether to add or modify the content of the task table 33, such as deleting one of the tasks corresponding to one of the intent categories in the task table 33 to notify the knowledge base 35 that the professional analysis is not required in this paragraph. Or, adjust the knowledge of the knowledge base 35, such as modifying the content of the standard answer, to make the similarity result closer.

Next, carry out the adjustment of the customer satisfaction analysis model 42 in the 2D diagram to find out those who have greater differences in the evaluation results of the customer satisfaction analysis. If the difference in the slope value of the satisfaction change trend line exceeds the third threshold value (the default value is 5) Retrain the words in cases where the score difference in the test data is greater than the third threshold For vectors, if a new vocabulary cannot be used for word vector conversion and the score difference is greater than the third threshold, add the entire paragraph to the training data set and retrain the word vector model 43. Then adjust the sentiment analysis model 44. The training method of the sentiment analysis model 44 is as described in detail in Figure 5A above. When the input layer data is unchanged, only the sentiment level of the output layer is adjusted, and the optimized sentiment analysis model is retrained. 44), the above steps can bring the sentiment analysis of this paragraph closer to the result of manual annotation. The first threshold value, the second threshold value, and the third threshold value may be the same or different threshold values.

FIG. 7 is a schematic flowchart showing a method for evaluating customer service quality from text content in the present invention. The main technical content of FIG. 7 is as follows, and the remaining technical content is as described in FIGS. 1 to 6 above. This is no longer repeated.

In step S1 in FIG. 7, the speech data of the dialogue between the customer and the customer service is sequentially analyzed by voice segmentation, speech-to-text, and text analysis to obtain the dialogue vocabulary of the customer and the customer service.

In the above step S1, the voice data of the conversation between the customer and the customer service or the voice signal of the recorded conversation content can also be retrieved first, and the voice signal is cut into units of paragraphs based on the magnitude of the voice energy of the voice signal. At the same time, the speech signal that has been cut into paragraphs can be sent to a speech-to-text decoder to obtain a candidate word map of the speech signal, and then a path with the best score can be selected from the candidate word map as the recognition result. Text data representing the voice content of the voice signal.

In step S2 of FIG. 7, the dialogue vocabulary is converted into a word vector to obtain a word vector, and then a segment of the dialogue vocabulary is obtained from the word vector. Fall in the semantic vector, and then input the semantic vector into the sentiment analysis model to get the customer sentiment analysis value, and then predict the customer satisfaction analysis result based on the sentiment analysis value.

In the above step S2, it is also possible to filter irrelevant words from the conversation vocabulary of the customer and customer service, and then perform word vector conversion on the conversation vocabulary of the filtered irrelevant words to obtain the word vector of the corresponding paragraph in the conversation vocabulary, and then from the word vector Find the semantic vector of the paragraph of the dialogue vocabulary in. Alternatively, the word vector is mathematically calculated according to the classification algorithm to obtain the customer sentiment analysis value, and then all the customer sentiment analysis values in the conversation vocabulary are counted to calculate the satisfaction change trend line as the result of the customer satisfaction analysis.

In step S3 of FIG. 7, the dialogue vocabulary is analyzed for intent and keywords to obtain the intention and keywords of the dialogue vocabulary, and then the standard answer of the customer's question is queried from the knowledge base based on the intent and keywords. The similarity between the response content of the customer service and the standard answer of the knowledge base gives the professional level of customer service.

In the above step S3, the intent and keywords of each paragraph in the dialogue vocabulary can also be obtained through an intent and keyword analysis model and a machine learning algorithm.

In step S4 of FIG. 7, from the customer satisfaction analysis result or the professionalism of customer service, select a case that is significantly different from the manual score (such as exceeding the first threshold) to optimize the task list and knowledge base. At least one of a word vector model and a sentiment analysis model. For example, compare the professionalism of customer service with the scorer ’s score (the score of manual scores) to find out cases where the scores are significantly different (such as exceeding the first threshold) for analysis, and then The score difference is adjusted by updating the task list or knowledge base through an adjustment specialty analysis model. Or, compare the customer satisfaction analysis result with the scorer's score result (the score manually scored) to find out the cases with large score differences (such as exceeding the first threshold) for analysis, and then update the word vector model And sentiment analysis models to adjust score differences.

As can be seen from the above, in the system and method for evaluating customer service quality from text content, the present invention can automatically evaluate customer service quality to replace the rater. In addition, the use of speech-to-speech technology makes customer service quality detection no longer limited to only sampling and can be scored one by one, and the evaluation method is more fair and objective. At the same time, the change in customer satisfaction is used to determine the service quality, and the professional level of customer service is determined based on the response content of the customer service and the standard answers in the knowledge base to resolve the phenomenon that the standardizers have different standards in the past.

Furthermore, the automatic scoring is based on customer feedback (customer satisfaction analysis) and preset customer service specifications (knowledge base) for consistency evaluation, so the automatic scoring method is more credible, and it is no longer necessary to periodically score all Perform calibration work. In addition, if the difference between the results of the automatic evaluation and the rater's evaluation is too large, the gap between the automatic evaluation and the manual evaluation can be reduced by optimizing the customer satisfaction analysis model and increasing the content of the knowledge base.

In addition, the present invention starts with the content of the service conversation, and detects the customer's feedback and professionalism of the customer service by using the real-time situation to judge whether the customer service is properly handled as a scoring standard, thereby overcoming the conventional technology using the customer and the company. The historical data was analyzed to evaluate the customer service ability, but the evaluation was distorted due to the current situation of customers.

The above-mentioned embodiment merely exemplarily illustrates the principle, characteristics and The effect is not intended to limit the implementable scope of the present invention, and anyone skilled in the art can modify and change the above implementation form without departing from the spirit and scope of the present invention. Any equivalent changes and modifications made by using the disclosure of the present invention should still be covered by the scope of patent application. Therefore, the scope of protection of the rights of the present invention should be as listed in the scope of patent application.

Claims (10)

A system for evaluating customer service quality from text content, including: a voice-to-speech module, which performs voice segmentation, voice-to-text, and text analysis of the voice data of the dialogue between the customer and the customer service to obtain the customer's dialogue with the customer service Vocabulary; a customer satisfaction analysis module that performs word vector conversion of the dialogue vocabulary of the voice-to-speech module to obtain the word vector, and then obtains the semantic vector of the paragraph of the dialogue vocabulary from the word vector, and The semantic vector is input to a sentiment analysis model to obtain a customer sentiment analysis value, and a customer satisfaction analysis result is predicted based on the customer sentiment analysis value; a professional level analysis module that converts the speech to the semantic word of the conversation module Perform intent and keyword analysis to obtain the intent and keywords of the dialogue vocabulary, and then query the standard answer to the customer ’s question from the task list and knowledge base based on the intent and keywords to compare the content of the response to the customer service The degree of similarity with the standard answer of the knowledge base to obtain the professional level of customer service; and an evaluation module for optimizing service content, Select from the customer satisfaction analysis result or the customer service professional level the difference between the scores from the manual score and the threshold to optimize at least one of the task list, the knowledge base, the word vector model, and the sentiment analysis model . The system described in item 1 of the scope of patent application, wherein the customer satisfaction analysis module further trains a deep neural network model representing the sentiment analysis model through a deep learning algorithm. The system described in item 1 of the scope of patent application, wherein the result of the customer satisfaction analysis is presented as a trend line of the satisfaction change. The system described in item 1 of the scope of patent application, wherein the professional degree analysis module obtains the intent and keywords of each paragraph in the dialogue vocabulary through intent and keyword analysis models combined with machine learning algorithms, and then through the intent And keywords to obtain the standard answer of the knowledge base, and then compare the similarity between the response content of the customer service and the standard answer of the knowledge base to obtain the result of the professionalism of the customer service. The system described in item 1 of the scope of patent application, wherein the optimized service content evaluation module further compares the professionalism of the customer service with the score result of the manual score to find out the difference in scores that exceeds the threshold Perform the analysis, and then adjust the score difference by updating the task list or the knowledge base through an adjustment specialty analysis model. The system as described in item 1 of the scope of patent application, wherein the optimized service content evaluation module further compares the customer satisfaction analysis result with the manual scoring result to find that the difference between the scores exceeds the threshold value The analyst performs analysis, and then adjusts the score difference by updating the word vector model and the sentiment analysis model. A method for assessing the quality of customer service from text content includes: performing speech segmentation, speech-to-text, and text analysis on the speech data of the conversation between the customer and the customer service to obtain the dialog vocabulary of the customer and the customer service; and performing the word vector of the dialog vocabulary Transform to get the word vector, and then get the semantic vector of the paragraph of the dialogue vocabulary from the word vector, Then input the semantic vector into the sentiment analysis model to obtain the customer sentiment analysis value, and then predict the customer satisfaction analysis result based on the customer sentiment analysis value; perform the intention and keyword analysis on the dialogue vocabulary to obtain the intention and Keywords, and then based on the intent and keywords, query the standard answer of the customer ’s question from the task list and the knowledge base, and get the customer service by comparing the similarity between the response content of the customer service and the standard answer of the knowledge base Professionalism; and selecting from the customer satisfaction analysis results or the professionalism of customer service the scores that differ from the manual scores by more than a threshold to optimize the task list, the knowledge base, the word vector model, and the sentiment analysis model At least one of them. The method as described in item 7 of the scope of patent application, further comprising filtering irrelevant words from the conversation vocabulary of the customer and the customer service, and performing the word vector conversion of the conversation vocabulary that has filtered the irrelevant words to obtain the conversation vocabulary. The word vector, and further obtain the semantic vector of the paragraph of the dialogue vocabulary from the word vector. The method described in item 7 of the scope of patent application further includes training a deep neural network model representing the sentiment analysis model through a deep learning algorithm. The method described in item 7 of the scope of patent application, further includes operating the word vector according to a classification algorithm to obtain the customer sentiment analysis value, and then counting all the customer sentiment analysis values in the conversation vocabulary to calculate as the customer Satisfaction Trend of Satisfaction Analysis Results line.