CN113723975A - System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service - Google Patents
System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service Download PDFInfo
- Publication number
- CN113723975A CN113723975A CN202111070370.9A CN202111070370A CN113723975A CN 113723975 A CN113723975 A CN 113723975A CN 202111070370 A CN202111070370 A CN 202111070370A CN 113723975 A CN113723975 A CN 113723975A
- Authority
- CN
- China
- Prior art keywords
- quality inspection
- return visit
- intelligent
- model
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 148
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012545 processing Methods 0.000 title claims abstract description 44
- 238000003032 molecular docking Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims description 35
- 238000005070 sampling Methods 0.000 claims description 18
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 12
- 238000013145 classification model Methods 0.000 claims description 10
- 238000012790 confirmation Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000013518 transcription Methods 0.000 claims description 8
- 230000035897 transcription Effects 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 238000000926 separation method Methods 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 239000003550 marker Substances 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 210000001520 comb Anatomy 0.000 description 2
- 230000002354 daily effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q30/016—After-sales
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3343—Query execution using phonetics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
Abstract
The invention relates to a system for realizing intelligent quality inspection processing in an intelligent return visit service, which comprises a data docking module, a quality inspection module and a quality inspection module, wherein the data docking module is used for regularly synchronizing return visit questionnaire information, return visit task information and audio files; the quality inspection task module is used for preprocessing and scheduling a quality inspection task; the manual spot check module is used for randomly spot checking the quality check result; and the data statistics module is used for determining the quality inspection passing rate. The invention also relates to a corresponding method, a device, a processor and a computer readable storage medium for realizing the intelligent quality inspection processing in the intelligent return visit service. By adopting the system, the method, the device, the processor and the computer readable storage medium for realizing the intelligent quality inspection processing in the intelligent return visit business, the high-efficiency and full-coverage quality inspection of the return visit records of the client is realized, the intelligent quality inspection is used, the labor is saved, the records of pre-judgment errors are collected by means of a manual spot inspection mode, the model is trained in a periodical increment mode, the accuracy and the efficiency of the intelligent quality inspection model are improved, the integral service capability is improved, and the service efficiency and the service experience of the client are improved.
Description
Technical Field
The invention relates to the field of quality inspection of telephone return visit business, in particular to the field of secondary intelligent quality inspection, and specifically relates to a system, a method, a device, a processor and a computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit business.
Background
For customer service work, customer return visit is an important link, and with the popularization of the mobile internet, telephone return visit is a main service mode. Every day, the work of the outbound telephone operator is the action of continuously and repeatedly making a call, which not only consumes huge labor cost, but also has the current situations of simplicity, repetition and low value. The intelligent voice return visit application directly adopts the AI robot to automatically configure the external call return visit task at the telephone terminal through various artificial intelligent technologies such as voice recognition, text-to-voice, semantic understanding, voice synthesis and the like through natural voice interaction with the user in the whole process, so that the work of full-automatic external call return visit and the like can be realized, and the integral service level of a customer service center is improved. In the traditional service process, the video frequency shooting, voice and other trace recording are generally used, so that the service quality supervision and evidence storage are facilitated, and the customer service process is comprehensively mastered. Therefore, how to make quality inspection of the return visit quality of the client is a complex problem in a customer service center in which the intelligent voice return visit serves as a main stream.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a system, a method, a device, a processor and a computer readable storage medium thereof for realizing intelligent quality inspection processing in intelligent return visit business, which have the advantages of high accuracy, high efficiency and wide application range.
In order to achieve the above object, the system, method, apparatus, processor and computer readable storage medium for implementing intelligent quality inspection processing in the intelligent return visit service of the present invention are as follows:
the system for realizing intelligent quality inspection processing in the intelligent return visit service is mainly characterized by comprising the following steps:
the data docking module is connected with the intelligent return visit system and used for periodically synchronizing the return visit questionnaire information, the return visit task information and the audio files;
the quality inspection task module is connected with the data docking module and used for preprocessing return visit task flow and scheduling a quality inspection task to perform quality inspection;
the manual spot check module is connected with the quality inspection task module and used for randomly spot checking the quality inspection result in a certain proportion according to the model training accuracy and the manual preset sampling proportion, manually confirming and judging the correctness;
and the data statistics module is connected with the manual spot check module and used for determining the quality inspection passing rate according to the spot check rate and the manual confirmation accuracy rate and displaying the multi-dimensional data statistics result.
Preferably, the system further comprises a model service module for collecting and sorting the intention classifications of all the return access papers, collecting corpora, sampling and balancing, training a FastText classification model, and performing a time-increment corpus enhancement model.
Preferably, the model service module includes:
the corpus collection unit is used for collecting historical revisitation corpuses and summarizing revisitation intention classification;
the corpus labeling unit is connected with the corpus collecting unit and is used for manually classifying and labeling the corpus;
the model training unit is connected with the corpus labeling unit and used for training the model based on a training program;
and the model verification unit is connected with the model training unit and used for comparing the model training result with the manual quality inspection result to verify the model.
Preferably, the model of the model service module includes an input layer, a hidden layer and an output layer, the input layer, the hidden layer and the output layer are connected in sequence, the input layer receives multi-dimensional vectorized text information, the hidden layer is used for performing superposition average processing on word vectors, and the output layer outputs a specific classification tag by adopting a softmax function.
The method for realizing intelligent quality inspection processing in the intelligent return visit service based on the system is mainly characterized by comprising the following steps of:
(1) the data docking service module performs data synchronous docking;
(2) preprocessing the return visit running water, and performing sound channel separation and audio transcription on an audio file;
(3) carrying out model prediction on the client problem of each communication session, and storing the client problem in a database;
(4) judging whether the model prediction result is consistent with the return visit result, if so, continuing the step (5); otherwise, carrying out secondary confirmation through manual intervention, and continuing the step (7);
(5) randomly extracting return visit running water according to the sampling rate of the quality detection result;
(6) manually carrying out secondary quality inspection on the return visit of the marker, and calculating the accuracy of intelligent quality inspection according to the result of manual spot inspection;
(7) and (4) counting the accuracy, error rate and score distribution of the intelligent quality inspection.
Preferably, the step (2) specifically comprises the following steps:
(2.1) preprocessing the return visit running water, matching the return visit task, the client information and the call records, and sorting and storing the return visit task, the client information and the call records;
and (2.2) carrying out sound channel separation and audio transcription on the audio file, and converting the call audio into a text library.
Preferably, the step (3) specifically includes the following steps:
(3.1) transcribing the return visit voice file of the client into a text record;
(3.2) performing intention prediction on the text returned by the customer return visit, segmenting the text and removing stop words;
(3.3) inputting the word segmentation result into a model, performing model prediction through word vector conversion, and outputting intention classification and score of the text;
(3.4) judging whether the score is smaller than a preset score threshold value, if so, indicating that the model result is not advisable, and marking the text by manual intervention; otherwise, continuing the step (3.5);
(3.5) judging whether the intention classification is consistent with the options checked by the questionnaire, if so, matching the questionnaire intention without problems; otherwise, the quality inspection has problems and needs manual intervention.
Preferably, the method further comprises a step of performing model training, specifically comprising the following processing procedures:
(1-1) collecting historical return visit corpora and summarizing the return visit intention classification of the client;
(1-2) manually marking corpora, and classifying and marking the corpora;
(1-3) performing material equalization on different classifications so that data of each classification is in the same magnitude;
(1-4) writing a model training program;
and (1-5) training the model for multiple times to obtain the model with high accuracy.
The device for realizing intelligent quality inspection processing in the intelligent return visit service is mainly characterized by comprising the following components:
a processor configured to execute computer-executable instructions;
and a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of the above-described method for implementing intelligent quality inspection processing in an intelligent return visit service.
The processor for realizing the intelligent quality inspection processing in the intelligent return visit service is mainly characterized in that the processor is configured to execute computer executable instructions, and the computer executable instructions are executed by the processor to realize the steps of the method for realizing the intelligent quality inspection processing in the intelligent return visit service.
The computer-readable storage medium is mainly characterized by storing a computer program thereon, wherein the computer program can be executed by a processor to realize the steps of the method for realizing the intelligent quality inspection processing in the intelligent return visit service.
By adopting the system, the method, the device, the processor and the computer readable storage medium for realizing the intelligent quality inspection processing in the intelligent return visit business, the invention provides a secondary intelligent quality inspection method and a secondary intelligent quality inspection system for identifying the pre-judged intention of the robot in the return visit, aiming at the field of the intelligent return visit business of a customer service center. The classification model based on the fastText algorithm is used for carrying out secondary verification on the return visit reply intention of the user, high-efficiency and full-coverage quality inspection of the return visit record of the user is achieved, intelligent quality inspection can be used for effectively achieving release of seat resources on the aspect of saving manpower, meanwhile, the record of pre-judgment errors is collected by means of an artificial spot inspection mode, the model is trained in a regular incremental mode, and the accuracy and the efficiency of the intelligent quality inspection model are improved. By using the algorithm model to solve massive return visit quality inspection, quality inspection personnel can randomly inspect and train the algorithm model, and intelligent and manual closed-loop service promotion on the return visit quality inspection of clients is realized. By introducing intelligent quality inspection, each communication record of customer service and customers is comprehensively covered, quality inspection scoring is intelligently completed by the system, the return visit quality of the robot is timely tracked, the overall service capacity is improved, and the service efficiency and service experience of the customers are improved.
Drawings
Fig. 1 is a schematic block diagram of a system for implementing intelligent quality inspection processing in an intelligent return visit service according to the present invention.
Fig. 2 is a service flow chart of the method for implementing intelligent quality inspection processing in the intelligent return visit service according to the present invention.
Fig. 3 is a schematic view of the fastText classification model of the system for implementing intelligent quality inspection processing in the intelligent return visit service according to the present invention.
Fig. 4 is a schematic diagram of a model prediction flow of the method for implementing intelligent quality inspection processing in the intelligent return visit service according to the present invention.
FIG. 5 is a schematic diagram of an embodiment of a system, a method, an apparatus, a processor and a computer-readable storage medium for implementing an intelligent quality inspection process in an intelligent return visit service according to the present invention.
Detailed Description
In order to more clearly describe the technical contents of the present invention, the following further description is given in conjunction with specific embodiments.
The system for realizing intelligent quality inspection processing in the intelligent return visit service comprises the following components:
the data docking module is connected with the intelligent return visit system and used for periodically synchronizing the return visit questionnaire information, the return visit task information and the audio files;
the quality inspection task module is connected with the data docking module and used for preprocessing return visit task flow and scheduling a quality inspection task to perform quality inspection;
the manual spot check module is connected with the quality inspection task module and used for randomly spot checking the quality inspection result in a certain proportion according to the model training accuracy and the manual preset sampling proportion, manually confirming and judging the correctness;
and the data statistics module is connected with the manual spot check module and used for determining the quality inspection passing rate according to the spot check rate and the manual confirmation accuracy rate and displaying the multi-dimensional data statistics result.
As a preferred embodiment of the present invention, the system further includes a model service module, which is used for collecting and organizing the intention classifications of all the return access papers, collecting corpora, sampling and balancing, training a FastText classification model, and performing a time-increment corpus enhancement model.
As a preferred embodiment of the present invention, the model service module includes:
the corpus collection unit is used for collecting historical revisitation corpuses and summarizing revisitation intention classification;
the corpus labeling unit is connected with the corpus collecting unit and is used for manually classifying and labeling the corpus;
the model training unit is connected with the corpus labeling unit and used for training the model based on a training program;
and the model verification unit is connected with the model training unit and used for comparing the model training result with the manual quality inspection result to verify the model.
As a preferred embodiment of the present invention, the model of the model service module includes an input layer, a hidden layer, and an output layer, the input layer, the hidden layer, and the output layer are sequentially connected, the input layer receives multi-dimensional vectorized text information, the hidden layer is used to perform an average process on word vectors, and the output layer outputs a specific classification tag using a softmax function.
The method for realizing intelligent quality inspection processing in the intelligent return visit service based on the system comprises the following steps:
(1) the data docking service module performs data synchronous docking;
(2) preprocessing the return visit running water, and performing sound channel separation and audio transcription on an audio file;
(3) carrying out model prediction on the client problem of each communication session, and storing the client problem in a database;
(4) judging whether the model prediction result is consistent with the return visit result, if so, continuing the step (5); otherwise, carrying out secondary confirmation through manual intervention, and continuing the step (7);
(5) randomly extracting return visit running water according to the sampling rate of the quality detection result;
(6) manually carrying out secondary quality inspection on the return visit of the marker, and calculating the accuracy of intelligent quality inspection according to the result of manual spot inspection;
(7) and (4) counting the accuracy, error rate and score distribution of the intelligent quality inspection.
As a preferred embodiment of the present invention, the step (2) specifically comprises the following steps:
(2.1) preprocessing the return visit running water, matching the return visit task, the client information and the call records, and sorting and storing the return visit task, the client information and the call records;
and (2.2) carrying out sound channel separation and audio transcription on the audio file, and converting the call audio into a text library.
As a preferred embodiment of the present invention, the step (3) specifically comprises the following steps:
(3.1) transcribing the return visit voice file of the client into a text record;
(3.2) performing intention prediction on the text returned by the customer return visit, segmenting the text and removing stop words;
(3.3) inputting the word segmentation result into a model, performing model prediction through word vector conversion, and outputting intention classification and score of the text;
(3.4) judging whether the score is smaller than a preset score threshold value, if so, indicating that the model result is not advisable, and marking the text by manual intervention; otherwise, continuing the step (3.5);
(3.5) judging whether the intention classification is consistent with the options checked by the questionnaire, if so, matching the questionnaire intention without problems; otherwise, the quality inspection has problems and needs manual intervention.
As a preferred embodiment of the present invention, the method further includes a step of performing model training, specifically including the following processing procedures:
(1-1) collecting historical return visit corpora and summarizing the return visit intention classification of the client;
(1-2) manually marking corpora, and classifying and marking the corpora;
(1-3) performing material equalization on different classifications so that data of each classification is in the same magnitude;
(1-4) writing a model training program;
and (1-5) training the model for multiple times to obtain the model with high accuracy.
The device for realizing intelligent quality inspection processing in the intelligent return visit service comprises:
a processor configured to execute computer-executable instructions;
and a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of the above-described method for implementing intelligent quality inspection processing in an intelligent return visit service.
The processor for implementing intelligent quality inspection processing in the intelligent return visit service is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the method for implementing intelligent quality inspection processing in the intelligent return visit service are implemented.
The computer-readable storage medium of the present invention has stored thereon a computer program which is executable by a processor to implement the steps of the above-described method of implementing intelligent quality inspection processing in an intelligent return visit service.
In the specific implementation mode of the invention, a secondary intelligent quality inspection method and a secondary intelligent quality inspection system for the purpose identification of robot pre-judgment in return visit are provided for the field of intelligent return visit business daily used by a customer service center. The invention aims to solve the technical problem of providing a quality inspection method for a robot in return visit in the field of intelligent return visit business of a customer service center, greatly reducing labor cost by introducing intelligent quality inspection capability, and more comprehensively covering quality inspection in the customer service process.
As shown in fig. 1, the system includes:
the method comprises the steps that firstly, deep model training service is used for collecting and sorting intention classifications of all return access papers, corpora of a certain magnitude are collected in an initial building period, a FastText classification model is trained through corpus sampling and balancing, and a later-period timed incremental corpus strengthening model is obtained;
a return visit data docking module for synchronizing intelligent return visit questionnaires, return visit running water files and the like;
thirdly, quality inspection task service adopts a T +1 mode, and scoring and judging right and wrong are carried out on return visit running water of the last day every morning;
the manual spot check module samples the return visit quality check record of the last day to generate a spot check list based on the model training accuracy and the manual preset sampling proportion, and performs manual check to determine whether the quality check is correct; and the data statistics module is used for performing data distribution statistics and accuracy statistics by maintaining time, questionnaires and the like.
The model service module is developed based on Python + Django, provides an http rest data interface and provides a prediction interface to a user;
the data docking module is developed by adopting a Java + Springboot framework, and the module mainly comprises the steps of docking an intelligent return visit system, regularly synchronizing questionnaire information, return visit task information, audio files and the like;
the quality inspection task module adopts Vue + Java + Springboot technology stack to realize preprocessing of return visit task flow, quality inspection core logic realization and quality inspection task scheduling;
the manual quality inspection module is also based on Vue + Java + Springboot technology stack, and performs manual confirmation on the quality inspection result by adopting random sampling inspection in a certain proportion;
and the data statistics module comprehensively determines the quality inspection passing rate based on the sampling inspection rate and the manual confirmation accuracy rate, and can display the multi-dimensional data statistics result.
The invention provides a method for a model service module based on a fastText algorithm, which comprises the following steps of S101-S106:
s101: collecting historical revisit corpora, and summarizing the revisit intention classification of the client, wherein the revisit intention classification includes yes, no, refusal, ambiguity, reverse access and the like;
s102: manually marking the corpora, and classifying and marking a large number of corpora;
s103: performing language material balance on different classifications to enable data of each classification to be in the same magnitude;
s104: compiling Python algorithm training codes based on Keras;
s105: training the model for multiple times to finally obtain a model with the accuracy higher than 95%, and obtaining service approval;
s106: based on the Django framework, Web service is written, a classification prediction rest http interface is provided, and intention prediction of a user is supported.
Through the above steps, the release of the core intelligent quality inspection model is completed, and the service data flow of the present invention is described below, as shown in fig. 2, as steps S201 to S210:
s201: the intelligent return visit system is connected, and the data connection service module completes the daily questionnaire content, return visit flow and audio files;
s202: preprocessing return visit running water, and matching, sorting and storing return visit tasks, client information and call records;
s203: audio files are subjected to sound channel separation and audio transfer, and conversation audio is transferred into a text falling library;
s204: carrying out model prediction on the client problem of each communication session, and storing the client problem in a database;
s205: comparing the model prediction result with the return visit result, and if the model prediction result is inconsistent with the return visit result, manually intervening for secondary confirmation;
s206: randomly extracting a series of return visit running water according to the sampling rate of the quality detection result;
s207: manually carrying out secondary quality inspection on the return visit of the mark;
s208: according to the result of the manual sampling inspection, the accuracy rate of the intelligent quality inspection is calculated;
s209: counting the accuracy, error rate, score distribution and the like of the intelligent quality inspection from dimensions such as date, tasks and the like;
s210: and (4) keeping the error prediction session text, synchronizing to the model service, and continuously iterating the training model.
The method adopts a fastText algorithm to carry out intention prediction, the fastText is a text classification algorithm which is open in 2016, and the fastText can obtain powerful advantages in training time and model size and can obtain accuracy rate comparable to that of a deep network at the same time based on the design of a shallow network.
Fig. 3 is a schematic diagram of a fastText algorithm model, where the input layer receives a multi-dimensional vectorized client reply text, the hidden layer averages the word vectors, and the output layer outputs specific classification labels, such as agreement, disagreement, ambiguity, and the like, using a softmax function. The fastText obtains sentence vectors by superposing and averaging the word vectors of the reply text, and performs multi-classification according to the whole sentence vectors, so that the intention of a client in replying the sentences can be learned, and better accuracy can be obtained. The invention combs the questionnaire options of intelligent return visit, summarizes the intention classification of the client, combs and collects a large amount of corpora, avoids over-fitting and under-fitting of the model through pretreatment such as manual marking, corpus balancing, sampling and the like, and finishes the training of the intelligent quality inspection multi-classification model.
The core logic of the quality inspection is shown in figure 4, a client return visit voice file is transcribed into a text record by means of intelligent voice transcription, next, the text intention of the client return visit reply is predicted, firstly, the text is segmented, stop words are removed, the segmentation result is input into a model, a fastText algorithm model outputs a classification model and word vectors, which are reasons for adopting the fastText at this time, and the intention classification and score of the text can be output through word vector transformation and model prediction. Then judging whether model classification is available or not according to a preset score, if the score is low, the model result is not available, marking the text, needing manual intervention marking, and collecting linguistic data to be subjected to subsequent model incremental training; if the score is high, the score is compared with the option selected by the revisit questionnaire, if the score is consistent, the questionnaire intention is matched without problems, otherwise, the questionnaire record is questioned, manual intervention is needed, and the corpus is used as a training corpus database after manual marking.
FIG. 5 is a functional diagram of a system implemented by the present invention, showing the result of the quality inspection with intention of a return visit record. In the method and the system, a classification model based on a fastText algorithm is used for carrying out secondary verification on the return visit reply intention of a user, high-efficiency and full-coverage quality inspection of the return visit records of the user is realized, the release of seat resources can be effectively realized on the aspect of saving manpower by using intelligent quality inspection, meanwhile, by means of a manual sampling inspection mode, the records of pre-judgment errors are collected, the model is trained in a regular incremental mode, and the accuracy and the efficiency of the intelligent quality inspection model are improved. The algorithm model solves massive return visit quality inspection, and quality inspection seat personnel carry out spot inspection training on the algorithm model, so that closed-loop service improvement of intellectualization and manual work on the return visit quality inspection of clients is realized.
According to the invention, through natural language algorithms such as voice transcription and intention classification, intelligent quality inspection in the intelligent return visit service is realized, the problems that the traditional manual quality inspection consumes manpower and cannot be fully covered are solved, through a fastText algorithm, an intention pre-judging model is continuously and incrementally trained, the problem that the intelligent return visit call quality inspection of a customer service center is difficult is solved, the cost of the manual quality inspection is reduced, meanwhile, a timing incremental training model is added on the technical strategy, and the reliability and the accuracy of the intelligent quality inspection are better improved by combining sampling manual inspection. By introducing intelligent quality inspection, each communication record of customer service and customers is comprehensively covered, quality inspection scoring is intelligently completed by the system, the return visit quality of the robot is timely tracked, the overall service capacity is improved, and the service efficiency and service experience of the customers are improved.
For a specific implementation of this embodiment, reference may be made to the relevant description in the above embodiments, which is not described herein again.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that the terms "first," "second," and the like in the description of the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present invention, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by suitable instruction execution devices. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, and the corresponding program may be stored in a computer readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
By adopting the system, the method, the device, the processor and the computer readable storage medium for realizing the intelligent quality inspection processing in the intelligent return visit business, the invention provides a secondary intelligent quality inspection method and a secondary intelligent quality inspection system for identifying the pre-judged intention of the robot in the return visit, aiming at the field of the intelligent return visit business of a customer service center. The classification model based on the fastText algorithm is used for carrying out secondary verification on the return visit reply intention of the user, high-efficiency and full-coverage quality inspection of the return visit record of the user is achieved, intelligent quality inspection can be used for effectively achieving release of seat resources on the aspect of saving manpower, meanwhile, the record of pre-judgment errors is collected by means of an artificial spot inspection mode, the model is trained in a regular incremental mode, and the accuracy and the efficiency of the intelligent quality inspection model are improved. By using the algorithm model to solve massive return visit quality inspection, quality inspection personnel can randomly inspect and train the algorithm model, and intelligent and manual closed-loop service promotion on the return visit quality inspection of clients is realized. By introducing intelligent quality inspection, each communication record of customer service and customers is comprehensively covered, quality inspection scoring is intelligently completed by the system, the return visit quality of the robot is timely tracked, the overall service capacity is improved, and the service efficiency and service experience of the customers are improved.
In this specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Claims (11)
1. A system for realizing intelligent quality inspection processing in intelligent return visit service is characterized by comprising:
the data docking module is connected with the intelligent return visit system and used for periodically synchronizing the return visit questionnaire information, the return visit task information and the audio files;
the quality inspection task module is connected with the data docking module and used for preprocessing return visit task flow and scheduling a quality inspection task to perform quality inspection;
the manual spot check module is connected with the quality inspection task module and used for randomly spot checking the quality inspection result in a certain proportion according to the model training accuracy and the manual preset sampling proportion, manually confirming and judging the correctness;
and the data statistics module is connected with the manual spot check module and used for determining the quality inspection passing rate according to the spot check rate and the manual confirmation accuracy rate and displaying the multi-dimensional data statistics result.
2. The system for implementing intelligent quality inspection in intelligent return visit business as claimed in claim 1, further comprising a model service module for collecting and organizing the intention classification of all return visit papers, collecting corpora, sampling and balancing, training the FastText classification model, and time-increment corpora reinforcement model.
3. The system of claim 2, wherein the model service module comprises:
the corpus collection unit is used for collecting historical revisitation corpuses and summarizing revisitation intention classification;
the corpus labeling unit is connected with the corpus collecting unit and is used for manually classifying and labeling the corpus;
the model training unit is connected with the corpus labeling unit and used for training the model based on a training program;
and the model verification unit is connected with the model training unit and used for comparing the model training result with the manual quality inspection result to verify the model.
4. The system for realizing intelligent quality inspection processing in the intelligent return visit business as claimed in claim 2, wherein the model of the model service module comprises an input layer, a hidden layer and an output layer, the input layer, the hidden layer and the output layer are connected in sequence, the input layer receives multi-dimensional vectorized text information, the hidden layer is used for performing superposition average processing on word vectors, and the output layer outputs a specific classification tag by adopting a softmax function.
5. A method for implementing intelligent quality inspection processing in intelligent return visit service based on the system of claim 1, wherein the method comprises the following steps:
(1) the data docking service module performs data synchronous docking;
(2) preprocessing the return visit running water, and performing sound channel separation and audio transcription on an audio file;
(3) carrying out model prediction on the client problem of each communication session, and storing the client problem in a database;
(4) judging whether the model prediction result is consistent with the return visit result, if so, continuing the step (5); otherwise, carrying out secondary confirmation through manual intervention, and continuing the step (7);
(5) randomly extracting return visit running water according to the sampling rate of the quality detection result;
(6) manually carrying out secondary quality inspection on the return visit of the marker, and calculating the accuracy of intelligent quality inspection according to the result of manual spot inspection;
(7) and (4) counting the accuracy, error rate and score distribution of the intelligent quality inspection.
6. The method for implementing intelligent quality inspection processing in intelligent return visit service according to claim 5, wherein the step (2) specifically comprises the following steps:
(2.1) preprocessing the return visit running water, matching the return visit task, the client information and the call records, and sorting and storing the return visit task, the client information and the call records;
and (2.2) carrying out sound channel separation and audio transcription on the audio file, and converting the call audio into a text library.
7. The method for implementing intelligent quality inspection processing in intelligent return visit service according to claim 5, wherein the step (3) specifically comprises the following steps:
(3.1) transcribing the return visit voice file of the client into a text record;
(3.2) performing intention prediction on the text returned by the customer return visit, segmenting the text and removing stop words;
(3.3) inputting the word segmentation result into a model, performing model prediction through word vector conversion, and outputting intention classification and score of the text;
(3.4) judging whether the score is smaller than a preset score threshold value, if so, indicating that the model result is not advisable, and marking the text by manual intervention; otherwise, continuing the step (3.5);
(3.5) judging whether the intention classification is consistent with the options checked by the questionnaire, if so, matching the questionnaire intention without problems; otherwise, the quality inspection has problems and needs manual intervention.
8. The method for implementing intelligent quality inspection processing in intelligent return visit business as claimed in claim 5, wherein the method further comprises a step of performing model training, specifically comprising the following processing procedures:
(1-1) collecting historical return visit corpora and summarizing the return visit intention classification of the client;
(1-2) manually marking corpora, and classifying and marking the corpora;
(1-3) performing material equalization on different classifications so that data of each classification is in the same magnitude;
(1-4) writing a model training program;
and (1-5) training the model for multiple times to obtain the model with high accuracy.
9. An apparatus for implementing intelligent quality inspection processing in an intelligent return visit service, the apparatus comprising:
a processor configured to execute computer-executable instructions;
a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of the method of implementing intelligent quality inspection processing in an intelligent revisit service of any of claims 5 to 8.
10. A processor for implementing intelligent quality inspection processing in an intelligent return visit service, characterized in that the processor is configured to execute computer-executable instructions which, when executed by the processor, implement the steps of the method of implementing intelligent quality inspection processing in an intelligent return visit service as claimed in any one of claims 5 to 8.
11. A computer-readable storage medium, having stored thereon a computer program executable by a processor for carrying out the steps of the method of carrying out an intelligent quality check process in an intelligent return visit service according to any one of claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111070370.9A CN113723975A (en) | 2021-09-13 | 2021-09-13 | System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111070370.9A CN113723975A (en) | 2021-09-13 | 2021-09-13 | System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113723975A true CN113723975A (en) | 2021-11-30 |
Family
ID=78683590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111070370.9A Pending CN113723975A (en) | 2021-09-13 | 2021-09-13 | System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723975A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116108373A (en) * | 2023-04-17 | 2023-05-12 | 京东科技信息技术有限公司 | Bill data classifying and labeling system, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335692A (en) * | 2014-08-15 | 2016-02-17 | 软控股份有限公司 | Tire X-ray image detection and identification method and system |
| CN110472041A (en) * | 2019-07-01 | 2019-11-19 | 浙江工业大学 | A kind of file classification method towards the online quality inspection of customer service |
| CN110532400A (en) * | 2019-09-04 | 2019-12-03 | 江苏苏宁银行股份有限公司 | Knowledge base maintenance method and device based on text classification prediction |
| CN111090753A (en) * | 2018-10-24 | 2020-05-01 | 马上消费金融股份有限公司 | Training method of classification model, classification method, device and computer storage medium |
| CN111339760A (en) * | 2018-12-18 | 2020-06-26 | 北京京东尚科信息技术有限公司 | Method and device for training lexical analysis model, electronic equipment and storage medium |
| CN111835925A (en) * | 2020-06-16 | 2020-10-27 | 杭州云嘉云计算有限公司 | Off-line voice quality inspection and analysis system for call center |
| US20200349199A1 (en) * | 2019-05-03 | 2020-11-05 | Servicenow, Inc. | Determining semantic content of textual clusters |
| US20210182912A1 (en) * | 2019-12-17 | 2021-06-17 | At&T Intellectual Property I, L.P. | Method and apparatus for labeling data |
| CN113283541A (en) * | 2021-06-15 | 2021-08-20 | 无锡锤头鲨智能科技有限公司 | Automatic floor sorting method |
-
2021
- 2021-09-13 CN CN202111070370.9A patent/CN113723975A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105335692A (en) * | 2014-08-15 | 2016-02-17 | 软控股份有限公司 | Tire X-ray image detection and identification method and system |
| CN111090753A (en) * | 2018-10-24 | 2020-05-01 | 马上消费金融股份有限公司 | Training method of classification model, classification method, device and computer storage medium |
| CN111339760A (en) * | 2018-12-18 | 2020-06-26 | 北京京东尚科信息技术有限公司 | Method and device for training lexical analysis model, electronic equipment and storage medium |
| US20200349199A1 (en) * | 2019-05-03 | 2020-11-05 | Servicenow, Inc. | Determining semantic content of textual clusters |
| CN110472041A (en) * | 2019-07-01 | 2019-11-19 | 浙江工业大学 | A kind of file classification method towards the online quality inspection of customer service |
| CN110532400A (en) * | 2019-09-04 | 2019-12-03 | 江苏苏宁银行股份有限公司 | Knowledge base maintenance method and device based on text classification prediction |
| US20210182912A1 (en) * | 2019-12-17 | 2021-06-17 | At&T Intellectual Property I, L.P. | Method and apparatus for labeling data |
| CN111835925A (en) * | 2020-06-16 | 2020-10-27 | 杭州云嘉云计算有限公司 | Off-line voice quality inspection and analysis system for call center |
| CN113283541A (en) * | 2021-06-15 | 2021-08-20 | 无锡锤头鲨智能科技有限公司 | Automatic floor sorting method |
Non-Patent Citations (1)
| Title |
|---|
| 李梁等: "《万号大学：打造客户中心学习型组织》", 北京邮电大学出版社, pages: 178 - 179 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116108373A (en) * | 2023-04-17 | 2023-05-12 | 京东科技信息技术有限公司 | Bill data classifying and labeling system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109522556B (en) | Intention recognition method and device | |
| WO2018032937A1 (en) | Method and apparatus for classifying text information | |
| US8914285B2 (en) | Predicting a sales success probability score from a distance vector between speech of a customer and speech of an organization representative | |
| CN112184525A (en) | System and method for realizing intelligent matching recommendation through natural semantic analysis | |
| CN109783637A (en) | Electric power overhaul text mining method based on deep neural network | |
| CN109740159B (en) | Processing method and device for named entity recognition | |
| CN111177350A (en) | Method, device and system for forming dialect of intelligent voice robot | |
| CN111177351A (en) | Method, device and system for acquiring natural language expression intention based on rule | |
| CN114925260A (en) | Intelligent bidding information fitting method | |
| CN112966082A (en) | Audio quality inspection method, device, equipment and storage medium | |
| CN111159356A (en) | Knowledge graph construction method based on teaching content | |
| CN111144116B (en) | Document knowledge structured extraction method and device | |
| CN113723975A (en) | System, method, device, processor and computer readable storage medium for realizing intelligent quality inspection processing in intelligent return visit service | |
| CN111160034A (en) | Method and device for labeling entity words, storage medium and equipment | |
| CN112417852B (en) | Method and device for judging importance of code segment | |
| CN113918697A (en) | Optimization method and optimization system of intelligent question-answering system | |
| KR20200127587A (en) | A program for an automatic sentiment information labeling to news articles for providing sentiment information | |
| CN111427996A (en) | Method and device for extracting date and time from human-computer interaction text | |
| US11431472B1 (en) | Automated domain language parsing and data extraction | |
| CN116127011A (en) | Intention recognition method, device, electronic equipment and storage medium | |
| CN115080732A (en) | Complaint work order processing method and device, electronic equipment and storage medium | |
| CN111311197A (en) | Travel data processing method and device | |
| CN115391496B (en) | Legal document case extraction method, system and storage medium | |
| KR20210056085A (en) | Customer Consultation Summary Apparatus and Method | |
| Nwankwo et al. | Text Classification of Digital Forensic Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |