TWI650719B - System and method for evaluating customer service quality from text content - Google Patents

Abstract

The invention discloses a system and method for evaluating customer service quality from text content, the method comprising: performing voice sentence segmentation, voice-to-text and text analysis on the dialogue voice data of the customer and the customer service to obtain a dialogue vocabulary between the customer and the customer service; The vocabulary performs word vector conversion to obtain the word vector, and then obtains the customer sentiment analysis value from the word vector to obtain the customer satisfaction analysis result; the dialogue vocabulary carries out the intention and keyword analysis to obtain the dialogue vocabulary intention and keywords, and then according to the intention And the keyword to query the standard answer of the customer's question from the task table and the knowledge base, and obtain the customer service professional degree by comparing the similarity between the customer service reply content and the standard answer of the knowledge base; and analyzing the result according to the customer satisfaction Or customer service professional level to optimize service content assessment.

Description

System and method for evaluating customer service quality from text content

The present invention relates to a technology for evaluating customer service quality, and more particularly to a system and method for evaluating customer service quality from text content.

Most of the existing customer service quality tests are done by manual listening. In addition to the limited number of scorers, each customer service staff will only be drawn a small number of calls (such as single-digit number) every month.

In addition, the rating by the scorer will have different standards. Because the scorer will score according to his subjective consciousness and cause unstable or uncontrollable factors, it is necessary to regularly calibrate all the scorers. operation.

In addition, some conventional technologies use customer and company data historical data to analyze, in order to evaluate the ability of customer service, but it is easy to cause distortion of the rating due to different conditions of the current customer.

Therefore, how to solve the above disadvantages of the prior art has become one of the major problems of those skilled in the art.

The present invention provides a system and method for evaluating customer service quality from text content, which can replace the scorer to automatically evaluate the customer service quality to provide a more accurate and rapid evaluation technique.

The system for evaluating customer service quality from text content includes: a voice-to-speech module, which sequentially performs voice-sentence, voice-to-text and text analysis on the voice data of the customer and the customer service to obtain the customer and the customer service. Dialogue vocabulary; a customer satisfaction analysis module, which converts the dialogue vocabulary of the phonetic semantic module into a word vector to obtain a word vector, and then obtains the semantic vector of the paragraph of the dialogue vocabulary from the word vector, and then the semantic intention The quantity is input to the sentiment analysis model to obtain the customer sentiment analysis value, and the customer satisfaction analysis result is predicted according to the customer sentiment analysis value; a professional level analysis module, which performs the intent and keyword analysis of the dialogue vocabulary of the speech transfer semantic module. In order to obtain the intent and keywords of the dialogue vocabulary, and then query the standard answers of the customer's questions from the task list and the knowledge base according to the intent and keywords, to compare the similarity between the customer service reply content and the standard answer of the knowledge base. And get the customer service professional level; and an optimization service content evaluation module, which results from the customer satisfaction analysis or Degree in professional clothes and pick out the differences in scores artificial score exceeds the threshold value were to optimize the task, according to the table, the knowledge base, the word vector model and sentiment analysis model in which at least one.

The method for evaluating the customer service quality from the text content in the invention comprises: sequentially synchronizing the voice data of the customer and the customer service into a speech sentence, a voice text and a text analysis to obtain a dialogue vocabulary between the customer and the customer service; Convert to get the word vector, and then obtain the semantic vector of the paragraph of the dialogue vocabulary from the word vector, and then input the semantic vector into the emotion score The model is analyzed to obtain the customer sentiment analysis value, and the customer satisfaction analysis result is predicted according to the customer sentiment analysis value; the dialogue vocabulary is subjected to the intention and keyword analysis to obtain the dialogue vocabulary intention and keywords, and then the task is based on the intention and the keyword. The standard answers to the customer's questions are queried in the table and the knowledge base, and the customer service professional level is obtained by comparing the similarity between the customer service reply content and the standard answer of the knowledge base; and from the customer satisfaction analysis result or the customer service professional level The scores that differ from the scores of the manual scores exceed the threshold value are selected to optimize at least one of the task table, the knowledge base, the word vector model, and the sentiment analysis model.

The above described features and advantages of the invention will be apparent from the description and appended claims. The additional features and advantages of the invention will be set forth in part in the description in the description. The features and advantages of the present invention are realized and attained by the <RTIgt; It is to be understood that both the foregoing general description

1‧‧‧System for assessing customer service quality from textual content

10‧‧‧Voice to semantic module

11‧‧‧Speech sentence unit

12‧‧‧Voice to text unit

13‧‧‧Text analysis unit

2‧‧  semantic analysis module

20‧‧‧Customer Satisfaction Analysis Module

21‧‧ ‧ word vector conversion unit

22‧‧‧Satisfaction trajectory change prediction unit

30‧‧‧Professional Analysis Module

31‧‧‧Intention and Keyword Analysis Unit

32‧‧‧Query reply task unit

33‧‧‧Task Schedule

34‧‧‧Query standard answer unit

35‧‧‧Knowledge base

36‧‧‧Content similarity comparison unit

40‧‧‧Optimized Service Content Evaluation Module

41‧‧‧Adjusting the professional analysis model

42‧‧‧Adjust customer satisfaction analysis model

43‧‧‧word vector model

44‧‧‧Emotional analysis model

A‧‧‧Customer customer service voice input

B‧‧‧Customer Service Quality Rating

C‧‧‧Customer service manual score entry

D‧‧‧ customer service text semantic record

E‧‧‧Client dialogue record

F‧‧‧Customer dialogue record

G‧‧‧similarity

H‧‧‧Automatic analysis results and manual score results

S1 to S4‧‧‧ steps

1 is a schematic block diagram of a system for evaluating customer service quality from text content in the present invention; FIG. 2A to FIG. 2D respectively illustrate a voice-to-speech module and a customer satisfaction analysis module in FIG. 1 of the present invention. Schematic block diagram of the group, professional level analysis module and optimization service content evaluation module; FIG. 3 is a schematic diagram of the voice input of the client terminal and the client terminal in the present invention FIG. 4A and FIG. 4B are schematic diagrams showing the semantic vector conversion of the word vector conversion unit in the present invention; FIG. 5A is a schematic diagram showing the application of the training and emotion analysis model of the emotion analysis model in the present invention; The figure shows a schematic diagram of the satisfaction trajectory change line formed by the customer sentiment analysis value and the customer inquiry statement in the present invention; FIG. 6 is a schematic diagram showing an embodiment of the professional degree analysis module in the 2C figure of the present invention; The figure shows a schematic flow chart of a method for evaluating customer service quality from text content in the present invention.

The embodiments of the present invention are described in the following specific embodiments, and those skilled in the art can easily understand other advantages and functions of the present invention by the disclosure of the present disclosure, and can also be implemented by other different embodiments. Or application.

1 is a schematic block diagram of a system 1 for evaluating customer service quality from text content in the present invention, and FIG. 2A to FIG. 2D respectively illustrate a voice-to-speech module 10 in the first embodiment of the present invention, and customer satisfaction A schematic block diagram of the degree analysis module 20, the professional level analysis module 30, and the optimized service content evaluation module 40.

As shown in FIG. 1, the system 1 for evaluating customer service quality from text content includes a voice transfer semantic module 10, a semantic analysis module having a customer satisfaction analysis module 20 and a professional level analysis module 30. And an optimization Service Content Evaluation Module 40.

As shown in FIG. 2A, the speech-to-speech module 10 of FIG. 1 is composed of a speech segmentation unit 11, a speech-to-text unit 12, and a text analysis unit 13. As shown in FIG. 2B, the customer satisfaction analysis module 20 of FIG. 1 is composed of a word vector conversion unit 21 and a satisfaction trajectory change prediction unit 22. As shown in FIG. 2C, the professional level analysis module 30 of FIG. 1 is composed of an intent and keyword analysis unit 31, a query reply task unit 32, a task table 33, a query standard answer unit 34, a knowledge base 35, and content similarity. The comparison unit 36 is composed of. As shown in FIG. 2D, the optimization service content evaluation module 40 of FIG. 1 is composed of an adjustment professional analysis model 41, an adjusted customer satisfaction analysis model 42, a word vector model 43, and an emotion analysis model 44, and may also be added. Or connect the task table 33 of FIG. 2C with the knowledge base 35.

As shown in FIG. 1 and FIG. 2A, the voice-to-speech module 10 can exchange the voice data of the customer and the customer service (such as the customer service voice input A), sequentially by the voice sentence unit 11, the voice-to-text unit 12, and the text. The parsing unit 13 respectively performs speech segmentation, voice-to-text and text analysis to obtain a dialogue vocabulary between the client and the customer service (such as the customer service text semantic record D).

Specifically, in the voice transfer semantic module 10 of FIG. 1 and FIG. 2A, a continuous voice signal (such as customer service voice input A) of the client terminal and the client terminal may be input to the voice transfer semantic module 10, for example, by The speech segmentation unit 11 captures the voice signal of the entire recording conversation content between the customer and the customer service, and then uses the energy level of the voice signal as a judgment condition to cut the voice signal into a dialogue (may be composed of one or more sentences). Signal for the unit. Next, the voice signal that has been cut into units of paragraphs is sent to the voice-to-text list. Element 12 (such as a speech-to-text decoder) obtains a candidate word map, and then selects a path with the best score as the identification result from the candidate word map, and the identification result is the text data representing the semantic content. Then, the text data is obtained by word segmentation and entity identification, and the word content of the word unit and the word combination representing the key words of the specific entity are obtained.

In the customer satisfaction analysis module 20 of FIG. 1 and FIG. 2B, the dialogue vocabulary of the speech-to-speech module 10 can be converted into a word vector to obtain a word vector, and then the paragraph of the dialogue vocabulary can be obtained from the word vector. The semantic vector is input to the sentiment analysis model 44 to obtain the customer sentiment analysis value, and the customer satisfaction analysis result is predicted based on the customer sentiment analysis value.

In other words, the word vector conversion unit 21 of the customer satisfaction analysis module 20 can perform a word vector conversion on the conversation vocabulary of the speech transfer semantic module 10 (such as the client conversation record E) to obtain a word vector, and then from the word vector. Find the semantic vector of the paragraph of the dialogue vocabulary. The satisfaction trajectory change prediction unit 22 of the customer satisfaction analysis module 20 can input the semantic vector of the paragraph of the conversation vocabulary into the sentiment analysis model to obtain the customer sentiment analysis value, and obtain the satisfaction change trend from the plurality of customer sentiment analysis values. Line, the results of customer satisfaction analysis are presented, and the trend line of satisfaction variation can be calculated by the formula Y=βX+ε (see Figure 5B).

Specifically, in FIG. 2B, from the customer service text semantic record D sufficient to represent the customer and the customer service interaction information, all the client dialogue records E to the word vector conversion unit 21 in a segment of the dialogue are taken out, first The segment performs filtering to process vocabulary that is not related to emotion, and then performs word vector The conversion, word vector conversion unit 21 can be collocated with the word vector model 43 to convert words (such as vocabulary combinations) into word vectors (such as numerical values). Then, each vocabulary can obtain a fixed-length multi-dimensional word vector, and arithmetically average the several multi-dimensional word vectors in the paragraph to obtain the semantic vector of the paragraph (such as the output of FIG. 4B), and then input The pre-trained sentiment analysis model 44 can calculate the sentiment analysis values in units of paragraphs, calculate the emotional values of the clients in all the dialogue paragraphs, and then find the trend line of the change in customer satisfaction in the telephone through linear regression. This trend line is the result of customer satisfaction analysis.

In the professional level analysis module 30 of FIGS. 1 and 2C, the intent and keyword analyzing unit 31 can perform the intent and keyword analysis on the dialogue vocabulary of the voice transfer semantic module 10 (such as the client dialogue record E). In order to obtain the intent and keywords of each paragraph in the dialogue vocabulary. In other words, the intent and keyword analysis unit 31 can cooperate with the machine learning algorithm to obtain the intent and keywords of each paragraph in the dialogue vocabulary (such as the client dialogue record E).

At the same time, the query reply task unit 32 can query the task table 33 to obtain the task of the customer service to engage in related problems, and then query the standard answer unit 34 from the knowledge base 35 according to the intention and the keyword according to the intent and the keyword. The similarity G between the response content of the customer service (the customer service session record F) and the standard answer obtained by the knowledge base 35 is compared by the content similarity comparison unit 36, and then the similarity G result is used as the customer service professional level.

In the optimization service content evaluation module 40 of FIG. 1 and FIG. 2D, a case in which the score difference is large (for example, exceeding the first threshold) can be selected from the automatic analysis result and the manual score result H, and adjusted according to the basis. Task Table 33 and Knowledge The library 35 further adjusts the word vector model 43 and the sentiment analysis model 44 by adjusting the customer satisfaction analysis model 42, and then updates the task table 33, the knowledge base 35, the word vector model 43, and the sentiment analysis model 44 to the 2B map and the The 2C map corresponding model achieves the effect of optimizing the service content assessment.

For example, the optimization service content assessment module 40 can compare the customer service professional level with the scorer's score result (the score of the manual score) to find a case with a large difference in scores (eg, exceeding the first threshold) for analysis. Then, the task table 33 or the knowledge base 35 is updated by an adjustment professional analysis model 41 to adjust the difference between the two scores. Alternatively, the optimization service content assessment module 40 may compare the customer satisfaction analysis result with the scorer's score result (the score of the manual score) to find a case where the score difference is large (eg, exceeding the first threshold). The analysis is performed, and the word vector model 43 and the emotion analysis model 44 are updated to adjust the difference between the two scores.

Specifically, in the 2D figure, the optimized service content evaluation module 40 can obtain the customer satisfaction change trend line (from the customer satisfaction analysis result) and the customer service problem response similarity score (from the customer service professional level). The result of the evaluation is combined with the score of the scorer manually filled in before to optimize the customer satisfaction analysis module 20 and the professional level analysis module 30. Since the results of both automatic scoring and manual scoring have been obtained, the gap between the scores and the gaps is found to be updated. If it is a paragraph with a large gap in the slope of the customer satisfaction trend line, the word vector model 43 and the sentiment analysis model 44 of the paragraph can be retrained by retraining the word vector and the machine learning classification method, so that the sentiment analysis result approaches The result of manual labeling. If it is the result of customer service professional rating The difference is large. It may be that the content of task table 33 or knowledge base 35 must be adjusted correspondingly due to factors such as policy or time (such as adding or deleting a task list, updating the knowledge base in response to a new promotion plan).

Figure 3 is a schematic diagram showing the voice input of the client terminal and the client terminal in the present invention. As shown in FIG. 3, the voice-to-speech module 10 of FIG. 2A respectively takes out the audio files of both the customer and the customer service from the recording equipment of the customer service line, and the sound files at both ends are separated and do not mix or interfere with each other. The voice signal distinguishes the voice signal between the client and the client through the left channel and the right channel of the audio file (such as customer service voice input A).

At the same time, the voice sentence unit 11 of FIG. 2A captures the voice signal of the complete conversation between the client and the customer service, and then according to the energy of the voice signal, the voice signal is cut into paragraphs and the voice of each paragraph is used. The signals are sorted in chronological order.

Then, the voice-to-text unit 12 of FIG. 2A sends the voice signal that has been cut into units of paragraphs into the voice-to-text decoder to obtain the candidate word map, and then selects the best score from the candidate word map. The path of the identification is the result of the identification, and the result of the identification is the text data representing the voice content. As shown in Figure 3, paragraph 1 for the customer service said: "Chunghwa Telecom Good afternoon, my surname is very honored to serve you," paragraph 2 is: "Miss I want to complain to NCC you bad service", paragraph Three: "I am really embarrassed to ask what can we do to help."

Then, the text parsing unit 13 of FIG. 2A obtains the text content in units of words and the key words representing the specific entity through the word segmentation and entity identification, that is, the NCC of the semantic analysis result as shown in FIG. " After the words "bad service", the "company line number" label and the "grievance language" label generated by the entity identification are attached.

Before performing the customer satisfaction analysis module 20 of FIG. 2B, the emotion analysis model 44 used in the process must be trained in advance, and the purpose of training the emotion analysis model 44 is to obtain the customer sentiment analysis value, and the emotion analysis model is trained. The input part is the semantic vector of the paragraph, and the output part is the artificially annotated emotional analysis value, so the entire text data must be converted into the semantic vector of the paragraph.

4A and 4B are schematic diagrams showing semantic vector conversion of the word vector conversion unit in the present invention. As shown in Figure 4A, the text training materials from the customer service line recording and processed by word segmentation are collected first, and are in units of paragraphs. As shown in Figure 4A, the customer said, "Miss I want to complain to NCC about your bad service." Set a text box that can hold 5 words in size. The text box will move from the left to the right of the clause to the end of the sentence. And the words in the middle (the third word from the left) are input words, and the words in other text boxes are adjacent words.

Each word in the text training data will correspond to a number in the dictionary, and the total number of words in the text training data will be equal to the input word vector of Figure 4A and the dimension length (N) of the adjacent word vector, so the hypothesis dictionary There are 1000 vocabulary words, and the word "you" is number 2, so the input word vector I _1XN with a length of 1000 is 1 , and the 1xN vector has only the second dimension is 1, and the other dimensions are 0. Similarly, the adjacent word vector O _1xN consists of four words "complaint", "NCC", "this", "bad service" (because the text box is only 5 words long), so the vector is only the first and the first 3. The 4th and 5th dimensions are 1, and the other dimensions are 0. Here, the method adopted by the present invention is Chinese word vector conversion (Word2vec), and in the case of the known input word vector and adjacent word vector, the word vector representing the semantic information is obtained by the Chinese word vector conversion (Word2vec) method. Dimensions can be reduced from high dimensions to low dimensions.

As shown in FIG. 4B, the word vector conversion unit 21 of FIG. 2B can convert each vocabulary through a word vector to obtain a corresponding word vector (the number of dimensions is M), and perform arithmetic for each dimension of all word vectors. Average to get the semantic vector of the paragraph (the number of dimensions is M).

FIG. 5A is a schematic diagram showing the application of the training and emotion analysis model of the emotion analysis model in the present invention, and FIG. 5B is a diagram showing the variation trajectory of the satisfaction trajectory formed by the customer emotion analysis value and the customer inquiry statement in the present invention. schematic diagram.

In FIG. 5A, the satisfaction trajectory change prediction unit 22 of FIG. 2B can input the semantic vector of the paragraph through the emotion analysis model 44 to obtain the customer emotion analysis value, and the training of the emotion analysis model 44 is as shown in FIG. 5A. Shown in the left half. Under the condition that the input layer input full segment word vector and the output layer are filled in the customer sentiment analysis value (in this embodiment, this part is manually labeled, the customer sentiment is firstly performed on the semantics of each paragraph in the training data). The artificial score of the analysis value is used to train a deep class neural network model representing the emotion analysis model 44 through a deep learning algorithm.

The application of the sentiment analysis model 44 is as shown in the right half of Figure 5A. By inputting the entire average emotional word vector, a customer sentiment analysis value can be calculated. The value range of the customer sentiment analysis value is between 0 and 1. Close to 0 table Show that the customer is in a negative mood, the closer to 1 indicates that the customer is in a positive mood.

As shown in Figure 5B, the customer sentiment analysis values of each paragraph in the dialogue can be combined into Y, and each paragraph is combined into X according to time series, and is brought into the formula Y=βX+ε by linear regression. The slope β of the satisfaction trajectory change line (such as a slash) is calculated, where ε represents the initial value of the customer's emotion. The slope β represents the change in customer satisfaction with the timing, and the customer satisfaction can be divided into one or more steps to judge whether the customer service properly comforts the customer's mood. If the slope β of the satisfaction trajectory change line represents the customer's satisfaction with the customer's appropriate explanation, the slope indicates that the customer service does not properly appease the customer's sentiment. This satisfaction trend line is the output of the customer satisfaction analysis.

Figure 6 is a schematic view showing an embodiment of the professional level analysis module 30 in Figure 2C of the present invention.

The professional level analysis module 30 of FIG. 6 and FIG. 2C is an input client dialogue record E (text form), and the intent and keyword analysis unit 31 detects the intent and keywords of each question by the client in units of paragraphs. The category of customer intent can be manually defined from the questions frequently mentioned in the customer service line. In addition to the business-related intent categories, these categories of customer intent include certain greetings for greetings and confirmation of personal data. The intent of a paragraph of customer questions can be used to filter issues that are not related to customer service expertise.

Keyword analysis is to find other necessary information about the intent of the match. If the intent category is the checklist information, the keyword that must be matched is the device number or The billing month allows for subsequent queries. The intention and keyword analysis unit 31 can employ a method of keyword matching to predict an intention and a keyword.

As shown in Figure 6, the customer said, "My mobile phone number 0912345678 requires a SIM card to unlock". After the analysis by the intent and keyword analysis unit 31, the intent is "unlocked" and the keyword is "device number (0912345678). "."

After learning the intent and the keyword information, the query reply task unit 32 can query the task table 33 to obtain a corresponding task response that the customer service should do, and determine whether the intent category of the paragraph needs to be professionally analyzed. Define it. For example, the slang language does not need to carry out follow-up professional analysis. After querying the task table 33, the task content of the customer service can be obtained as “certification”.

Next, the previously completed knowledge base 35 is queried by the query standard answer unit 34 of FIG. 6 to obtain a standard answer. The standard answer obtained is "Is it my own? Is the device holder name?". Then, the content similarity comparison unit 36 compares the customer service dialogue record F with the standard answer of the knowledge base 35 for content similarity comparison, and the similarity comparison method can connect the customer service dialogue record F with the knowledge base 35. The vocabulary of each paragraph is converted into the semantic vector of the paragraph. After the vocabulary of the paragraph is removed, the vocabulary is converted into a word vector, and the same dimensions of different vocabularies are summed and averaged to obtain Two semantic vectors V (customer reply) and semantic vector V (standard answer) representing the paragraph, and then the semantic vector V (customer reply) and the semantic vector V (standard answer) are substituted into the following formula 1 to calculate Cosine similarity (cosine similarity), high similarity indicates customer service professional High degree, and low similarity means that customer service can't accurately grasp customer problems.

Moreover, as shown in FIG. 2D, in order to narrow the difference between the automatic analysis result and the manual rating result H, the optimization service content evaluation module 40 can achieve the similarity score (the customer service professional rating result), The customer satisfaction change trend line (customer satisfaction analysis results), and the manually filled scorer's score results (including customer satisfaction analysis and customer service professional level) are automatically compared to compare the results to find out the difference The case is updated.

Furthermore, the adjustment professional degree analysis model 41 of the 2D map is performed, and if it is judged that the difference in the professional level evaluation result exceeds the second threshold value (the second threshold value is preset to 0.5 due to the use of the cosine similarity distance), it indicates that it is professional. The case where the degree of evaluation results differ greatly may be that the content of the task table or the knowledge base must be adjusted according to the policy or time factor to make the corresponding task table 33 and the knowledge base 35. For example, re-reviewing whether to add or modify the content of the task table 33, such as deleting the task corresponding to one of the intent categories of the task table 33 to notify the knowledge base 35 that the paragraph does not require professional analysis. Or, adjust the knowledge of the knowledge base 35, such as modifying the content of the standard answer, so that the similarity results are closer.

Next, the adjustment customer satisfaction analysis model 42 of the 2D map is performed to find out that the difference in the satisfaction result of the customer satisfaction analysis is larger, and if the difference in the slope value of the satisfaction trend trend line exceeds the third threshold value (the preset value is 5) Retraining the case in the case where the score difference in the test data is greater than the third threshold The vector, if the score difference is greater than the third threshold because the new word cannot be converted, the whole paragraph is added to the training data set and the word vector model 43 is retrained. Then, the emotion analysis model 44 is adjusted. The training method of the emotion analysis model 44 is as described in detail in FIG. 5A above. When the input layer data is unchanged, only the emotional level of the output layer is adjusted, and the optimized emotion analysis model is retrained. 44) The above steps allow the sentiment analysis of the paragraph to be closer to the result of manual labeling. The first threshold value, the second threshold value and the third threshold value may be the same or different threshold values.

FIG. 7 is a schematic flow chart showing a method for evaluating customer service quality from text content in the present invention, and the main technical contents of FIG. 7 are as follows, and the remaining technical contents are as shown in FIG. 1 to FIG. This is not repeated.

In step S1 of FIG. 7, the conversational voice data of the customer and the customer service are sequentially subjected to speech segmentation, voice-to-text and text analysis to obtain a dialogue vocabulary between the client and the customer service.

In the above step S1, the voice signal of the conversation voice data of the customer and the customer service or the content of the voice recording conversation may be firstly retrieved, and the voice signal is cut into the paragraph unit according to the voice energy of the voice signal. At the same time, the voice signal that has been cut into paragraphs can be sent to a speech-to-text decoder to obtain a candidate word map of the voice signal, and then a path with the best score is selected from the candidate word map as the identification result. A textual representation of the voice content representing the voice signal.

In step S2 of FIG. 7, the dialogue vocabulary is converted into a word vector to obtain a word vector, and then a segment of the dialogue vocabulary is obtained from the word vector. The meaning vector of the falling language is input into the sentiment analysis model to obtain the customer sentiment analysis value, and the customer satisfaction analysis result is predicted based on the customer sentiment analysis value.

In the above step S2, it is also possible to filter the irrelevant words from the dialogue vocabulary between the customer and the customer service, and then convert the vocabulary of the filtered irrelevant words into a word vector to obtain the word vector of the corresponding paragraph in the dialogue vocabulary, and then from the word vector. Find the semantic vector of the paragraph of the dialogue vocabulary. Alternatively, the word vector is mathematically operated according to the classification algorithm to obtain the customer sentiment analysis value, and then all the customer sentiment analysis values in the dialogue vocabulary are counted to calculate the satisfaction change trend line as the result of the customer satisfaction analysis.

In step S3 of FIG. 7, the dialogue vocabulary is subjected to intent and keyword analysis to obtain the intent and keywords of the dialogue vocabulary, and then the standard answer of the customer's question is queried from the knowledge base according to the intention and the keyword, Customer service professionalism is obtained from the similarity between the customer service reply content and the standard answer of the knowledge base.

In the above step S3, the intention and keyword of each paragraph in the dialogue vocabulary can also be obtained through the intention and keyword analysis model and the machine learning algorithm.

In step S4 of FIG. 7 , a case in which the difference between the scores of the manual scores and the scores of the manual scores is large (for example, exceeding the first threshold value) is selected from the customer satisfaction analysis results or the customer service professional level, so as to optimize the task table and the knowledge base according to the optimization. At least one of a word vector model and a sentiment analysis model. For example, compare the customer service professional level with the scorer's score (the score of the manual score) to find out the case where the score difference is large (such as exceeding the first threshold) for analysis. The score difference is adjusted by updating the task table or knowledge base by adjusting the professional analysis model. Or, compare the results of the customer satisfaction analysis with the scores of the scorers (the scores of the manual scores) to find out the cases with large differences in scores (such as exceeding the first threshold), and then update the word vector model. And emotional analysis models to adjust score differences.

As can be seen from the above, the present invention can automatically evaluate the customer service quality to replace the scorer in the system and method for evaluating the customer service quality from the text content. Moreover, the use of voice-to-speech technology to make customer service quality testing is no longer limited to only sampling, can be scored on a per-step basis, and the method of evaluation is more fair and objective. At the same time, the customer's satisfaction change is used to judge the service quality, and the customer service professional level is judged according to the customer service reply content and the standard answer of the knowledge base, and the phenomenon that the scorer has different standards in the past is solved.

In addition, the automatic scoring is based on the customer feedback (customer satisfaction analysis) and the pre-set customer service specification (knowledge base) for consistency evaluation, so the automatic scoring method is more credible, and there is no need to regularly rate all. The staff performs a scoring calibration operation. In addition, if the gap between the automatic rating and the rating of the scorer is too large, the gap between the automatic rating and the manual rating can be reduced by optimizing the customer satisfaction analysis model and increasing the content of the knowledge base.

In addition, the present invention starts from the dialogue content of the service, and cooperates with the detection of customer feedback and customer service professionalism, and judges whether the customer service on-site disposal is appropriate as a scoring standard in the case of an immediate occurrence, thereby overcoming the conventional technology with customers and companies. Data historical data is analyzed to judge customer service capabilities, but the current situation is different due to the different conditions of the customer.

The above embodiments merely exemplarily illustrate the principles and features of the present invention and The exemplifications are not intended to limit the scope of the invention, and those skilled in the art can modify and modify the above embodiments without departing from the spirit and scope of the invention. Any equivalent changes and modifications made by the disclosure of the present invention should still be covered by the scope of the patent application. Therefore, the scope of protection of the present invention should be as set forth in the scope of the patent application.

Claims (10)

A system for evaluating customer service quality from text content, comprising: a voice-to-speech module, which performs voice-sentence, voice-to-text and text analysis on the voice data of the customer and the customer service to obtain a dialogue between the customer and the customer service. Vocabulary; a customer satisfaction analysis module, which converts the dialogue vocabulary of the phonetic semantic module into a word vector to obtain a word vector, and then obtains a semantic vector of the paragraph of the dialogue vocabulary from the word vector, and then The semantic vector is input to the sentiment analysis model to obtain the customer sentiment analysis value, and the customer satisfaction analysis result is predicted according to the customer sentiment analysis value; and a professional degree analysis module, the speech conversion semantic module of the dialogue vocabulary Intent and keyword analysis to obtain the intent and keywords of the dialogue vocabulary, and then query the standard answer of the customer's question from the task list and the knowledge base according to the intention and the keyword, to compare the content of the customer service Customer service expertise with the similarity of the standard answer to the knowledge base; and an optimization service content assessment module Customer satisfaction analysis results or the degree of customer service professionals sort out the differences with the artificial score score exceeds the threshold value were to optimize the task, according to the table, the knowledge base, the word vector model and the mood in which at least one model analysis. The system of claim 1, wherein the customer satisfaction analysis module further trains a deep class neural network model representing the sentiment analysis model through a deep learning algorithm. For example, the system described in claim 1 is characterized in that the result of the customer satisfaction analysis is presented in a trend line of satisfaction change. The system of claim 1, wherein the professional analysis module obtains the intention and keyword of each paragraph in the dialogue vocabulary through the intention and keyword analysis model and the machine learning algorithm, and then uses the intention And the keyword obtains the standard answer of the knowledge base, and then compares the content of the reply to the customer service with the standard answer of the knowledge base to obtain the result of the customer service professional degree. The system of claim 1, wherein the optimization service content evaluation module compares the customer service professional degree with the score of the manual rating to find out that the score difference exceeds the threshold value. The analysis is performed, and the score difference is adjusted by updating the task table or the knowledge base through an adjustment professional analysis model. The system of claim 1, wherein the optimization service content evaluation module compares the customer satisfaction analysis result with the score of the manual rating to find that the score difference exceeds the threshold value. The analysis is performed, and the score difference is adjusted by updating the word vector model and the sentiment analysis model. A method for evaluating customer service quality from text content, comprising: performing a speech segmentation, a voice-to-text and a text analysis of a conversational voice data between a customer and a customer service to obtain a dialogue vocabulary between the client and the customer service; Converting to obtain a word vector, and then obtaining a semantic vector of the paragraph of the dialogue word from the word vector, And inputting the semantic vector into the sentiment analysis model to obtain the customer sentiment analysis value, predicting the customer satisfaction analysis result according to the customer sentiment analysis value; performing the intent and keyword analysis on the dialogue vocabulary to obtain the intention of the dialogue vocabulary and Keyword, and then querying the standard answer of the customer's question from the task table and the knowledge base according to the intention and the keyword, and obtaining the customer service by comparing the content of the customer service reply with the standard answer of the knowledge base. Professional degree; and an optimization service content evaluation module selects a score that differs from the manual score by the customer satisfaction analysis result or the customer service professional level to exceed the threshold value, thereby optimizing the task table and the knowledge base according to the optimization At least one of a word vector model and the sentiment analysis model. The method of claim 7, further comprising filtering the irrelevant word from the conversation vocabulary of the customer and the customer service, and performing the word vector conversion on the conversation vocabulary that has filtered the irrelevant word to obtain the conversation vocabulary The word vector, and then the semantic vector of the paragraph of the dialogue vocabulary is obtained from the word vector. For example, the method described in claim 7 includes training a deep class neural network model representing the sentiment analysis model through a deep learning algorithm. The method of claim 7, further comprising: calculating the word vector according to the classification algorithm to obtain the customer sentiment analysis value, and then counting all the customer sentiment analysis values in the conversation vocabulary to calculate as the customer. The satisfaction trend trend line of satisfaction analysis results.