CN112329437A - Intelligent customer service voice quality inspection scoring method, equipment and storage medium - Google Patents
Intelligent customer service voice quality inspection scoring method, equipment and storage medium Download PDFInfo
- Publication number
- CN112329437A CN112329437A CN202011131622.XA CN202011131622A CN112329437A CN 112329437 A CN112329437 A CN 112329437A CN 202011131622 A CN202011131622 A CN 202011131622A CN 112329437 A CN112329437 A CN 112329437A
- Authority
- CN
- China
- Prior art keywords
- customer service
- scoring
- data
- recording
- voice quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 21
- 238000013077 scoring method Methods 0.000 title claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000013135 deep learning Methods 0.000 claims abstract description 12
- 238000007405 data analysis Methods 0.000 claims description 14
- 230000006698 induction Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 239000003795 chemical substances by application Substances 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000007635 classification algorithm Methods 0.000 claims description 7
- 238000003908 quality control method Methods 0.000 claims description 6
- 238000007637 random forest analysis Methods 0.000 claims description 5
- 230000002567 autonomic effect Effects 0.000 claims description 3
- 230000001939 inductive effect Effects 0.000 claims description 3
- 238000012417 linear regression Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000007547 defect Effects 0.000 abstract description 4
- 238000011161 development Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0281—Customer communication at a business location, e.g. providing product or service information, consulting
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Physics & Mathematics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- General Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to an intelligent customer service voice quality inspection grading method, equipment and a storage medium, aiming at the two defects of the traditional customer service voice quality inspection grading adopting keyword search, namely that the grading rule needs to be manually set and cannot be completed in time according to market change, and when the customer service expresses the same meaning by using a similar word, the grading error can not be caused by the keyword search, the intelligent customer service voice quality inspection grading method is provided, the grading rule can be intelligently and automatically summarized according to the customer service seat recording text in different scenes, the deep learning algorithm is used for automatically identifying the similarity between the customer service seat recording statement and the rule statement so as to give the grading, and meanwhile, the grading rule and suggestion are fed back according to the grading of the customer service seat, so that the customer service seat is helped to find problems and complete the conversation in time.
Description
Technical Field
The invention relates to the technical field of intelligent voice quality inspection, in particular to an intelligent customer service voice quality inspection scoring method, equipment and a storage medium.
Background
The traditional customer service seat voice quality inspection needs to manually extract partial recording and give the customer service seat recording score after the recording is completely listened, and the method not only consumes a large amount of manpower, but also has the defects that the timeliness cannot meet the requirement and the quality inspection coverage rate is low.
In recent years, an intelligent voice quality inspection system appears, and the real-time text conversion of recorded sound, automatic scoring, 100% coverage rate and the like are realized. The automatic scoring mainly adopts a method of searching keywords, the method is that after the recorded sound is converted into a text, the text is searched for the appointed keywords, when the keywords appear, the preset score is given, such as adding or subtracting, and finally, the total scoring is given. The method has the disadvantages that the grading rule is set manually, which needs stronger professional knowledge and service background, and the grading rule can not be adjusted in time according to the development condition of the actual service; when the customer service agent uses the similar meaning word to express the same meaning, the similar meaning word cannot be searched by the keyword, so that scoring errors are caused.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides an intelligent customer service voice quality inspection scoring method, equipment and a storage medium, which can intelligently and autonomously conclude and form scoring rules according to customer service seat recording texts in different scenes, and automatically identify the similarity between customer service seat recording sentences and rule sentences by using a deep learning algorithm so as to score.
The purpose of the invention can be realized by the following technical scheme:
an intelligent customer service voice quality inspection scoring method comprises the following steps:
step 1: the customer service agent recording is converted into a text and then is exported to a data analysis system;
step 2: setting rules and standards for scoring customer service seat recording in a data analysis system;
and step 3: based on the set rules and standards for scoring the customer service seat recording and the customer service seat recording data after text conversion, scoring is carried out by calculating the similarity between the customer service seat recording data and the customer service seat recording data;
and 4, step 4: and accumulating all the scores to obtain a total score, and outputting the total score which is the recording score of the customer service seat.
Further, the step 1 comprises the following sub-steps:
step 101: connecting a customer service seat recording text data interface, connecting the data interface through an API (application programming interface) and importing data into a data analysis system;
step 102: while storing the data into a workspace of the data analysis system.
Further, the step 2 comprises:
step 201: the scoring rules are set manually in the data analysis system at the beginning, namely corresponding sentences are set;
step 202: and after the data analysis system runs for a set time, the manually set scoring rules are converted into the intelligent autonomous induction scoring rules.
Further, the setting process of the intelligent autonomic induction scoring rule in step 202 includes the following steps:
step 2021: carrying out data cleaning on the customer service seat recording data subjected to text conversion obtained in the step 1 through a regular expression;
step 2022: coding the text subjected to data cleaning through a coding algorithm model;
step 2023: selecting different regression or classification algorithm models for training according to scenes aiming at the coded data;
step 2024: after training, selecting features with larger model weight, and decoding by a decoding algorithm to obtain a corresponding complete sentence;
step 2025: and storing the complete sentence into a scoring rule to serve as an intelligent autonomous inductive scoring rule.
Further, the step 3 comprises the following steps:
step 301: coding the set rules for scoring the customer service seat recording and the rule text corresponding to the standard through a coding algorithm, and then inputting the coded rules and the rule text into a vector representation algorithm;
step 302: carrying out data cleaning on the customer service seat recording data after text conversion through a regular expression, coding through a coding algorithm model, and inputting into a vector representation algorithm;
step 303: pairing sentences which are generated in a vector representation algorithm and correspond to vector representation, and calculating similarity to obtain a similarity score between each pair of sentences;
the vector representation algorithm in step 301 and step 302 includes word2vec, LSTM, or BERT.
Further, the step 4 specifically includes: and taking the maximum value of the similarity scores between all the paired sentences for sequencing and carrying out circular judgment, adding the total score when the maximum value is judged to be the similarity score corresponding to the paired sentence corresponding to the normal expression and is greater than a set threshold value, subtracting the total score when the maximum value is judged to be the similarity score corresponding to the paired sentence corresponding to the prohibited expression and is greater than the set threshold value, and keeping the total score unchanged when the maximum value is less than the set threshold value.
Further, the similarity score in step 303 is calculated by the following formula:
score＝exp(-||h_a-h_b||)
in the formula, score is the similarity score, h _ a is a matrix corresponding to the encoded sound recording text through vector representation, and h _ b is a matrix corresponding to the encoded regular text through vector representation.
Further, the regression algorithm model in step 2023 includes linear regression, logics, SVR, XGBOOST or deep learning, and the classification algorithm model includes bayesian classification, logics, SVM, XGBOOST, random forest, deep random forest or deep learning.
The invention also provides terminal equipment which comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the steps of the intelligent customer service voice quality inspection scoring method when executing the computer program.
The invention also provides a computer readable storage medium, which stores a computer program, and the computer program realizes the steps of the intelligent customer service voice quality inspection scoring method when being executed by a processor.
Compared with the prior art, the invention has the following advantages:
(1) the quality inspection scoring method comprises the following steps: step 1: the customer service agent recording is converted into a text and then is exported to a data analysis system; step 2: setting rules and standards for scoring customer service seat recording in a data analysis system; and step 3: based on the set rules and standards for scoring the customer service seat recording and the customer service seat recording data after text conversion, scoring is carried out by calculating the similarity between the customer service seat recording data and the customer service seat recording data; and 4, step 4: and accumulating all the scores to obtain a total score, outputting the total score which is the recording score of the customer service seat, and intelligently and independently learning and inducing a scoring rule according to different scenes.
(2) The intelligent autonomous induction scoring rule can timely adjust the scoring rule according to new conditions and new situations encountered in business development, and further promote the perfection of dialect.
(3) Based on deep learning, scoring is given through the similarity of scoring rules and customer service seat recording, the keyword search defect is overcome, and the scoring result is more accurate.
(4) And feeding back scoring detailed rules and suggestions according to the scoring of the customer service seat to help the customer service seat to find problems and correct the problems in time.
(5) And the scoring rules are freely set, so that the rules are promoted from word level to sentence level, and the rules are more conveniently set.
Drawings
FIG. 1 is a flowchart illustrating the overall steps of an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the detailed steps of recording text in the steps of the method of the present invention;
FIG. 3 is a flowchart illustrating the detailed steps of the scoring rules in the steps of the method of the present invention;
FIG. 4 is a flowchart of the detailed steps of the intelligent autonomic induction scoring rules algorithm in the scoring rules step of the method embodiment of the present invention;
fig. 5 is a flowchart of the detailed steps of intelligently scoring and outputting the score in the steps of the method of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
The invention provides an intelligent customer service voice quality inspection scoring method, which induces a scoring rule through intelligent autonomous learning and adopts a deep learning algorithm to calculate the similarity between a customer service recording and a set rule sentence and give a score.
The specific implementation step framework is shown in fig. 1, the whole design includes 5 large steps of recording text input, scoring rule, intelligent scoring, scoring output and suggestion output, and the specific method of each step is described in detail below.
Firstly, recording a text:
in this step, the main task is to convert the customer service seat recording in the company system into a text and export the text to the data analysis software or system, so that the recording text of the customer service seat can be further operated and analyzed by the data analysis software or system, as shown in fig. 2:
1. and connecting a customer service agent recording text data interface, namely connecting the customer service agent recording with the data interface through an API (application programming interface), and importing the data into data analysis software or a system.
2. The data is stored in a workspace of the data analysis software or system.
Secondly, scoring rules:
in this step, the scoring rule is a standard for scoring the customer service seat recording, and is divided into a manually set scoring rule and an intelligent autonomous induction scoring rule according to the service requirement, as shown in fig. 3:
1. in the early stage of the project, due to insufficient data, the intelligent autonomous induction scoring rule cannot be trained and perfected sufficiently. Therefore, in the early stage of the project, the scoring needs to be performed by manually setting the scoring rule, and the manually set scoring rule sets a sentence instead of setting a keyword. Including sentences that must be mentioned to the customer, such as coring, introducing products, etc.; forbidden statements that cannot be mentioned, such as guaranteed interest rate, etc.
2. After the project is carried out for a period of time, enough recording data and client feedback information are accumulated, the intelligent autonomous induction scoring rule can complete the setting of the scoring rule, and then the intelligent autonomous induction scoring rule is adopted for scoring.
The specific process of the algorithm of the intelligent autonomous induction scoring rule is shown in fig. 4:
1. after the customer service seat recording is transcribed into a text, words which are not useful for the algorithm, such as special symbols, tone-assisted words, stop words and the like, are removed by a data cleaning method such as a regular expression and the like, so that the effects of reducing the calculation amount and improving the accuracy are achieved.
2. On the basis of removing useless data, the text is converted into digital data capable of being calculated by a computer, namely codes. The coding algorithm includes but is not limited to vector space model, Tf-idf, LDA, word2vec, BERT, XLNET and other models.
3. After encoding is completed, regression or classification models are selected according to scene needs, and different labels are selected. For example, for a business with a main purpose of bargaining, a classification algorithm is adopted, and bargaining/non-bargaining is used as a label for model training; to complete the customer complaints or the business required by the customer, a regression algorithm can be adopted, and the satisfaction degree is used as a label to carry out model training. Wherein, the classification algorithm includes but is not limited to Bayesian classification, logics, SVM, XGBOOST, random forest, deep learning, etc.; regression algorithms include, but are not limited to, linear regression, logistic, SVR, XGBOOST, deep learning, and the like.
4. And setting standard reaching standards of the model according to the training result of the model, wherein the standard reaching standards is more than 60% of accuracy of a classification algorithm or 30% of error of a regression algorithm. After the model training meets the requirements, extracting the features which have larger influence on the model, wherein the features are expressed as a sentence on the text. For example, if a sentence appears repeatedly in a committed sheet and appears less frequently in an uncommitted sheet, the weight of the sentence on the result is automatically increased during model training, the weight is expressed as the weight of a neural network in deep learning, and the weight is expressed as the size of a coefficient in logics, and particularly, which expression form needs to be determined according to an algorithm.
5. After the features which have great influence on the model are selected, the features are still in a digital form, and a complete sentence is obtained after decoding according to the decoding algorithms of models such as but not limited to a vector space model, Tf-idf, LDA, word2vec, BERT, XLNET and the like.
6. After the sentence is formed, the sentence is automatically stored in the scoring rule. In addition, the updating time interval of the intelligent autonomous induction scoring rule algorithm can be determined according to actual needs, and the updating of the intelligent autonomous induction scoring rule algorithm can be determined once in one week or two weeks or even one month so as to update the rule considering that if the scoring rule is changed all the time, the customer service seat is not easy to be familiar with the scoring rule, the work rhythm is disturbed and the thought of the customer service seat is influenced. Therefore, the practical application can be fully considered, the market change can be found in time, the scoring rule is adjusted, the customer service seat is guided to adjust the speech technology, and the deal is promoted or the satisfaction degree is improved.
Thirdly, intelligently scoring and outputting the score:
in this step, since the scoring rule is not a keyword but a sentence, scoring cannot be performed by searching for the keyword, and meanwhile, a situation that corresponding scoring cannot be obtained when the customer service agent expresses the same meaning by using a similar meaning word by using the scoring method of searching for the keyword also occurs. And the similarity between the scoring rule and the customer service seat recording sentences is calculated, and corresponding scoring is given according to the similarity, so that the problems can be avoided. And judging whether the customer service recording text data and the regular sentences have the same meaning according to the similarity, thereby grading.
In the step of outputting the score, the scores calculated by the algorithm are accumulated, and finally the total score is given, as shown in fig. 5:
1. the recording text contains the recording text data of the customer and the customer service seat, and also contains a plurality of tone words, repeated words and the like, so that the recording text of the customer service seat needs to be extracted first, the tone words, the repeated words and the like are removed, data cleaning is generally performed by adopting methods such as a regular expression and the like, and the recording text is divided into sentences.
2. And coding both the sound recording text and the regular text, wherein the coding algorithm comprises but is not limited to a vector space model, Tf-idf, LDA, word2vec, BERT, XLNET and other models.
3. Respectively inputting the encoded sound recording text and the regular text into a vector representation algorithm, wherein the vector representation algorithm comprises but is not limited to word2vec, LSTM, BERT and the like, calculating the vector representation h of each sentence through the vector representation algorithm, if the length of the vector set by a user is 256, the output of one sentence is a vector of 1 × 256, and if 20 regular sentences exist, a matrix h _ a of 20 × 256 can be generated to represent all the regular sentences. Similarly, if there are 50 words of the customer service recording text, the data of the customer service recording text in one recording is represented by a matrix h _ b of 50 × 256.
4. Because two sentences need to be paired to calculate the similarity between each sentence in the regular sentences and each sentence in the customer service seat recording, 20 × 50-1000 vector combinations can be generated, that is, each regular sentence in the regular sentences needs to be paired with all sentences of the customer service seat one by one to calculate the similarity, and the formula for calculating the similarity is as follows:
score＝exp(-||h_a-h_b||)
in the formula, score is a similarity score, h _ a is a matrix corresponding to the encoded sound recording text through vector representation, and h _ b is a matrix corresponding to the encoded regular text through vector representation, and the similarity score between each pair of sentences is calculated.
5. One regular sentence and 50 sentences of the customer service recording text calculate 50 score, and take the maximum score _ max, when score _ max is larger than the set threshold, the two sentences are considered to be the same meaning, and the customer service seat completes the specified task, then add score on the total score SUM, the add score can be freely set, if a certain sentence has high importance degree, a high score value can be set. The score can also be automatically set by an algorithm, and the setting method comprises the following steps: the weights of all sentences in the intelligent autonomous induction scoring rule are standardized to the interval of 0-1, and then the scores are set according to the mode of multiplying the weights by 100. Similarly, if score _ max matched by the prohibited term is also greater than the threshold, indicating that the customer service agent says prohibition and thus needs to be decremented, then the total score SUM is decremented. If score _ max is less than the threshold, then no score is added.
6. And finally, outputting the total score SUM, namely the recording score of the customer service seat.
Fourthly, outputting suggestions:
the output suggestion is to let the customer service seat see the quality control score and the detailed rules of the point of deduction in time, and the system can output detailed rules and improved suggestions according to the recording condition of the customer service seat. If the customer service seat does not say the necessary term, the score of the term can not be obtained, and the customer service seat can be advised to pay secondary attention to the term when the advice is output.
Fifthly, summarizing:
based on the analysis and the implementation steps of each step, the scheme provides a set of intelligent customer service quality inspection scoring method, which can intelligently and autonomously summarize scoring rules according to different application scenes, continuously improve and perfect the rules according to the development of services, and automatically identify the similarity between sentences in the customer service recording and sentences in the rules by applying a deep learning algorithm, thereby scoring.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An intelligent customer service voice quality inspection scoring method is characterized by comprising the following steps:
step 1: the customer service agent recording is converted into a text and then is exported to a data analysis system;
step 2: setting rules and standards for scoring customer service seat recording in a data analysis system;
and step 3: based on the set rules and standards for scoring the customer service seat recording and the customer service seat recording data after text conversion, scoring is carried out by calculating the similarity between the customer service seat recording data and the customer service seat recording data;
and 4, step 4: and accumulating all the scores to obtain a total score, and outputting the total score which is the recording score of the customer service seat.
2. The intelligent customer service voice quality control scoring method according to claim 1, wherein the step 1 comprises the following substeps:
step 101: connecting a customer service seat recording text data interface, connecting the data interface through an API (application programming interface) and importing data into a data analysis system;
step 102: while storing the data into a workspace of the data analysis system.
3. The intelligent customer service voice quality control scoring method according to claim 1, wherein the step 2 comprises:
step 201: the scoring rules are set manually in the data analysis system at the beginning, namely corresponding sentences are set;
step 202: and after the data analysis system runs for a set time, the manually set scoring rules are converted into the intelligent autonomous induction scoring rules.
4. The intelligent customer service voice quality control scoring method according to claim 3, wherein the setting process of the intelligent autonomic induction scoring rules in step 202 comprises the following steps:
step 2021: carrying out data cleaning on the customer service seat recording data subjected to text conversion obtained in the step 1 through a regular expression;
step 2022: coding the text subjected to data cleaning through a coding algorithm model;
step 2023: selecting different regression or classification algorithm models for training according to scenes aiming at the coded data;
step 2024: after training, selecting features with larger model weight, and decoding by a decoding algorithm to obtain a corresponding complete sentence;
step 2025: and storing the complete sentence into a scoring rule to serve as an intelligent autonomous inductive scoring rule.
5. The intelligent customer service voice quality control scoring method according to claim 1, wherein the step 3 comprises the following steps:
step 301: coding the set rules for scoring the customer service seat recording and the rule text corresponding to the standard through a coding algorithm, and then inputting the coded rules and the rule text into a vector representation algorithm;
step 302: carrying out data cleaning on the customer service seat recording data after text conversion through a regular expression, coding through a coding algorithm model, and inputting into a vector representation algorithm;
step 303: pairing sentences which are generated in a vector representation algorithm and correspond to vector representation, and calculating similarity to obtain a similarity score between each pair of sentences;
the vector representation algorithm in step 301 and step 302 includes word2vec, LSTM, or BERT.
6. The intelligent customer service voice quality control scoring method according to claim 5, wherein the step 4 specifically comprises: and taking the maximum value of the similarity scores between all the paired sentences for sequencing and carrying out circular judgment, adding the total score when the maximum value is judged to be the similarity score corresponding to the paired sentence corresponding to the normal expression and is greater than a set threshold value, subtracting the total score when the maximum value is judged to be the similarity score corresponding to the paired sentence corresponding to the prohibited expression and is greater than the set threshold value, and keeping the total score unchanged when the maximum value is less than the set threshold value.
7. The method as claimed in claim 5, wherein the similarity score in step 303 is calculated as:
score＝exp(-||h_a-h_b||)
in the formula, score is the similarity score, h _ a is a matrix corresponding to the encoded sound recording text through vector representation, and h _ b is a matrix corresponding to the encoded regular text through vector representation.
8. The intelligent customer service voice quality inspection scoring method according to claim 4, wherein the regression algorithm model in the step 2023 comprises linear regression, logistic, SVR, XGBOOST or deep learning, and the classification algorithm model comprises Bayesian classification, logistic, SVM, XGBOOST, random forest, deep random forest or deep learning.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and operable on the processor, wherein the processor when executing the computer program implements the steps of an intelligent customer service voice quality inspection scoring method according to any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of an intelligent customer service voice quality inspection scoring method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011131622.XA CN112329437B (en) | 2020-10-21 | 2020-10-21 | Intelligent customer service voice quality inspection scoring method, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011131622.XA CN112329437B (en) | 2020-10-21 | 2020-10-21 | Intelligent customer service voice quality inspection scoring method, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112329437A true CN112329437A (en) | 2021-02-05 |
| CN112329437B CN112329437B (en) | 2024-05-28 |
Family
ID=74311880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011131622.XA Active CN112329437B (en) | 2020-10-21 | 2020-10-21 | Intelligent customer service voice quality inspection scoring method, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112329437B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112687257A (en) * | 2021-03-11 | 2021-04-20 | 北京新唐思创教育科技有限公司 | Sentence similarity judging method and device, electronic equipment and readable storage medium |
| CN113724738A (en) * | 2021-08-31 | 2021-11-30 | 平安普惠企业管理有限公司 | Voice processing method, decision tree model training method, device, equipment and storage medium |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108763362A (en) * | 2018-05-17 | 2018-11-06 | 浙江工业大学 | Method is recommended to the partial model Weighted Fusion Top-N films of selection based on random anchor point |
| CN108876145A (en) * | 2018-06-13 | 2018-11-23 | 广西电网有限责任公司电力科学研究院 | A kind of 95598 customer service worksheet quality classification method and devices |
| CN109327632A (en) * | 2018-11-23 | 2019-02-12 | 深圳前海微众银行股份有限公司 | Intelligent quality inspection system, method and the computer readable storage medium of customer service recording |
| WO2019037382A1 (en) * | 2017-08-24 | 2019-02-28 | 平安科技（深圳）有限公司 | Emotion recognition-based voice quality inspection method and device, equipment and storage medium |
| CN109740155A (en) * | 2018-12-27 | 2019-05-10 | 广州云趣信息科技有限公司 | A kind of customer service system artificial intelligence quality inspection rule self concludes the method and system of model |
| CN109767791A (en) * | 2019-03-21 | 2019-05-17 | 中国—东盟信息港股份有限公司 | A kind of voice mood identification and application system conversed for call center |
| CN109788020A (en) * | 2017-11-13 | 2019-05-21 | 腾讯科技（深圳）有限公司 | One kind is attended a banquet distribution method and relevant device |
| CN109816106A (en) * | 2017-11-20 | 2019-05-28 | 北京智合大方科技有限公司 | One kind carrying out call center's customer service knowledge class response quality evaluation system based on speech recognition and natural language processing technique |
| CN110069784A (en) * | 2019-05-05 | 2019-07-30 | 广东电网有限责任公司 | A kind of voice quality inspection methods of marking, device, terminal and can storage medium |
| TW201935370A (en) * | 2018-02-12 | 2019-09-01 | 中華電信股份有限公司 | System and method for evaluating customer service quality from text content |
| CN110334241A (en) * | 2019-07-10 | 2019-10-15 | 深圳前海微众银行股份有限公司 | Quality detecting method, device, equipment and the computer readable storage medium of customer service recording |
| CN110347823A (en) * | 2019-06-06 | 2019-10-18 | 平安科技（深圳）有限公司 | Voice-based user classification method, device, computer equipment and storage medium |
| CN111128241A (en) * | 2019-12-30 | 2020-05-08 | 上海浩琨信息科技有限公司 | Intelligent quality inspection method and system for voice call |
| CN111325029A (en) * | 2020-02-21 | 2020-06-23 | 河海大学 | Text similarity calculation method based on deep learning integration model |
| KR20200119358A (en) * | 2019-03-15 | 2020-10-20 | (주) 데이터뱅크 | System for providing tofel course recommendation service using artificial intelligence machine learning based automatic evaluation |
-
2020
- 2020-10-21 CN CN202011131622.XA patent/CN112329437B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019037382A1 (en) * | 2017-08-24 | 2019-02-28 | 平安科技（深圳）有限公司 | Emotion recognition-based voice quality inspection method and device, equipment and storage medium |
| CN109788020A (en) * | 2017-11-13 | 2019-05-21 | 腾讯科技（深圳）有限公司 | One kind is attended a banquet distribution method and relevant device |
| CN109816106A (en) * | 2017-11-20 | 2019-05-28 | 北京智合大方科技有限公司 | One kind carrying out call center's customer service knowledge class response quality evaluation system based on speech recognition and natural language processing technique |
| TW201935370A (en) * | 2018-02-12 | 2019-09-01 | 中華電信股份有限公司 | System and method for evaluating customer service quality from text content |
| CN108763362A (en) * | 2018-05-17 | 2018-11-06 | 浙江工业大学 | Method is recommended to the partial model Weighted Fusion Top-N films of selection based on random anchor point |
| CN108876145A (en) * | 2018-06-13 | 2018-11-23 | 广西电网有限责任公司电力科学研究院 | A kind of 95598 customer service worksheet quality classification method and devices |
| CN109327632A (en) * | 2018-11-23 | 2019-02-12 | 深圳前海微众银行股份有限公司 | Intelligent quality inspection system, method and the computer readable storage medium of customer service recording |
| CN109740155A (en) * | 2018-12-27 | 2019-05-10 | 广州云趣信息科技有限公司 | A kind of customer service system artificial intelligence quality inspection rule self concludes the method and system of model |
| KR20200119358A (en) * | 2019-03-15 | 2020-10-20 | (주) 데이터뱅크 | System for providing tofel course recommendation service using artificial intelligence machine learning based automatic evaluation |
| CN109767791A (en) * | 2019-03-21 | 2019-05-17 | 中国—东盟信息港股份有限公司 | A kind of voice mood identification and application system conversed for call center |
| CN110069784A (en) * | 2019-05-05 | 2019-07-30 | 广东电网有限责任公司 | A kind of voice quality inspection methods of marking, device, terminal and can storage medium |
| CN110347823A (en) * | 2019-06-06 | 2019-10-18 | 平安科技（深圳）有限公司 | Voice-based user classification method, device, computer equipment and storage medium |
| CN110334241A (en) * | 2019-07-10 | 2019-10-15 | 深圳前海微众银行股份有限公司 | Quality detecting method, device, equipment and the computer readable storage medium of customer service recording |
| CN111128241A (en) * | 2019-12-30 | 2020-05-08 | 上海浩琨信息科技有限公司 | Intelligent quality inspection method and system for voice call |
| CN111325029A (en) * | 2020-02-21 | 2020-06-23 | 河海大学 | Text similarity calculation method based on deep learning integration model |
Non-Patent Citations (2)
| Title |
|---|
| 严可;胡国平;魏思;戴礼荣;李萌涛;杨晓果;冯国栋;: "面向大规模英语口语机考的复述题自动评分技术", 清华大学学报(自然科学版), no. 1, 15 August 2009 (2009-08-15), pages 1356 - 1362 * |
| 杨杉;宋瑞中;赵占军;: "呼叫中心行业的质培一体化系统应用", 现代信息科技, no. 11, 10 June 2020 (2020-06-10), pages 113 - 115 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112687257A (en) * | 2021-03-11 | 2021-04-20 | 北京新唐思创教育科技有限公司 | Sentence similarity judging method and device, electronic equipment and readable storage medium |
| CN112687257B (en) * | 2021-03-11 | 2021-06-01 | 北京新唐思创教育科技有限公司 | Sentence similarity judging method and device, electronic equipment and readable storage medium |
| CN113724738A (en) * | 2021-08-31 | 2021-11-30 | 平安普惠企业管理有限公司 | Voice processing method, decision tree model training method, device, equipment and storage medium |
| CN113724738B (en) * | 2021-08-31 | 2024-04-23 | 硅基(昆山)智能科技有限公司 | Speech processing method, decision tree model training method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112329437B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109871545B (en) | Named entity identification method and device | |
| CN109146610B (en) | Intelligent insurance recommendation method and device and intelligent insurance robot equipment | |
| CN110717031B (en) | Intelligent conference summary generation method and system | |
| WO2019153737A1 (en) | Comment assessing method, device, equipment and storage medium | |
| CN109509470B (en) | Voice interaction method and device, computer readable storage medium and terminal equipment | |
| CN110619506B (en) | Post image generation method, post image generation device and electronic equipment | |
| US8644488B2 (en) | System and method for automatically generating adaptive interaction logs from customer interaction text | |
| US11481387B2 (en) | Facet-based conversational search | |
| CN107423440B (en) | Question-answer context switching and reinforced selection method based on emotion analysis | |
| CN105654250A (en) | Method and device for automatically assessing satisfaction degree | |
| CN108647191B (en) | Sentiment dictionary construction method based on supervised sentiment text and word vector | |
| CN113934887B (en) | No-proposal time sequence language positioning method based on semantic decoupling | |
| CN110069776B (en) | Customer satisfaction evaluation method and device and computer readable storage medium | |
| WO2012165529A1 (en) | Language model construction support device, method and program | |
| CN111831801B (en) | Man-machine conversation method and system | |
| CN113361266A (en) | Text error correction method, electronic device and storage medium | |
| CN111933127A (en) | Intention recognition method and intention recognition system with self-learning capability | |
| CN112329437A (en) | Intelligent customer service voice quality inspection scoring method, equipment and storage medium | |
| CN110457424A (en) | Generate method, computer readable storage medium and the terminal device of interview report | |
| CN112151019A (en) | Text processing method and device and computing equipment | |
| CN112200674B (en) | Stock market emotion index intelligent calculation information system | |
| Zorrilla et al. | Audio Embedding-Aware Dialogue Policy Learning | |
| CN116502637A (en) | Text keyword extraction method combining context semantics | |
| CN112650838B (en) | Intelligent question-answering method and system based on historical case big data | |
| CN112506405B (en) | Artificial intelligent voice large screen command method based on Internet supervision field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |