CN113887246A - Method and device for detecting consistency of man-machine schemes in customer service field and storage medium - Google Patents
Method and device for detecting consistency of man-machine schemes in customer service field and storage medium Download PDFInfo
- Publication number
- CN113887246A CN113887246A CN202111217426.9A CN202111217426A CN113887246A CN 113887246 A CN113887246 A CN 113887246A CN 202111217426 A CN202111217426 A CN 202111217426A CN 113887246 A CN113887246 A CN 113887246A
- Authority
- CN
- China
- Prior art keywords
- session
- core
- machine
- manual
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000007781 pre-processing Methods 0.000 claims abstract description 87
- 238000013136 deep learning model Methods 0.000 claims abstract description 70
- 230000011218 segmentation Effects 0.000 claims description 134
- 238000013135 deep learning Methods 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 8
- 230000009467 reduction Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000015654 memory Effects 0.000 description 22
- 238000004891 communication Methods 0.000 description 11
- 239000013598 vector Substances 0.000 description 11
- 238000012545 processing Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 239000003550 marker Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- RWSOTUBLDIXVET-UHFFFAOYSA-N Dihydrogen sulfide Chemical compound S RWSOTUBLDIXVET-UHFFFAOYSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 238000000586 desensitisation Methods 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present disclosure relates to a method, an apparatus and a storage medium for detecting consistency of man-machine schemes in the field of customer service, wherein the method comprises: extracting a machine conversation from a first conversation record of the robot customer service and the client, and extracting a manual conversation from a second conversation record of the manual customer service and the client; carrying out data preprocessing on the machine conversation and the manual conversation; determining a first core scheme of the machine conversation subjected to data preprocessing and a second core scheme of the manual conversation subjected to data preprocessing through a deep learning model; calculating a consistency score of the first core scheme and the second core scheme through a deep learning model; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score.
Description
Technical Field
The present disclosure relates to the field of speech recognition, and in particular, to a method, an apparatus, and a storage medium for human-machine scheme consistency detection in the field of customer service.
Background
In the E-commerce field, a customer is received by adopting manual work or a robot customer service, relevant problems of the customer in the fields before and after sale are answered, the customer is helped to smoothly complete the whole shopping process, the shopping experience of the customer is improved, and the method is a more universal solution in the industry. In the process of receiving customers by the robot customer service, the problem that the customer service cannot be solved necessarily occurs, under the condition, a user can actively request to transfer to the artificial customer service, and the data of the robot customer service transferred to the artificial customer service can just reflect the short board of the robot, so that the response capability of the robot can be well improved by analyzing and optimizing the data, and the method is very important. Meanwhile, the schemes given by the robot customer service and the manual customer service are different, and the scheme of the manual customer service is superior to the scheme of the robot customer service. By comparing whether the two schemes are consistent or not, whether the scheme given by the robot customer service is excellent or not and whether the robot customer service is to be improved or not can be evaluated. The prior art lacks a method of comparing whether the two schemes are consistent.
In the course of implementing the disclosed concept, the inventors found that there are at least the following technical problems in the related art: the prior art lacks a method for comparing whether the schemes of the robot customer service and the manual customer service are consistent.
Disclosure of Invention
In order to solve the technical problem or at least partially solve the technical problem, embodiments of the present disclosure provide a method, an apparatus, and a storage medium for detecting consistency of human-machine solutions in a customer service field, so as to solve at least the problem in the prior art that a method for comparing whether solutions of robot customer service and human-machine service are consistent is absent.
The purpose of the present disclosure is realized by the following technical scheme:
in a first aspect, an embodiment of the present disclosure provides a method for detecting consistency of human-machine solutions in a customer service field, including: extracting a machine conversation from a first conversation record of a robot customer service and a client, and extracting a manual conversation from a second conversation record of a manual customer service and the client; performing data preprocessing on the machine session and the manual session; determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model; calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score.
In an exemplary embodiment, the determining, by the deep learning model, a first core solution of the machine session subjected to the data preprocessing and a second core solution of the human session subjected to the data preprocessing includes: extracting a first session feature of the machine session subjected to the data preprocessing and a second session feature of the manual session subjected to the data preprocessing; determining the first core scheme according to the first session characteristics and determining the second core scheme according to the second session characteristics.
In an exemplary embodiment, the determining the first core scenario according to the first session characteristic and the determining the second core scenario according to the second session characteristic includes: acquiring a core scheme set; determining a first cut session corresponding to the first session feature as the first core scheme under the condition that the similarity between the first session feature and the core scheme set is greater than a first preset threshold, wherein the first cut session is obtained by performing the data preprocessing on the machine session; and determining a second segmentation session corresponding to the second session feature as the second core scheme under the condition that the similarity between the second session feature and the core scheme set is greater than the first preset threshold, wherein the second segmentation session is obtained by performing the data preprocessing on the manual session.
In one exemplary embodiment, the first session feature and the second session feature each comprise at least one of: a first feature, configured to represent a business meaning of a segmentation session, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: a first segmentation session and a second segmentation session; a second feature, configured to indicate whether a customer service answer in the segmentation session is an question; a third feature for representing key information of customer service answers in the segmentation session; a fourth feature for representing a sentence length of the segmentation session; a fifth feature, configured to represent a sequence number of the split session in the machine session or the manual session; a sixth feature for indicating a business meaning of a last one of the slicing sessions.
In an exemplary embodiment, the core solution set includes: a first object, configured to indicate that the segmentation session has a business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions; a second object for indicating that the customer service answer in the segmentation session is not an question; a third object for indicating that the customer service answer in the segmentation session contains key information; the fourth object is used for representing the segmentation, and the sentence length of the conversation is in a first preset interval; the fifth object is used for indicating that the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval; a sixth object for indicating that a last one of the slicing sessions has a business meaning.
In one exemplary embodiment, includes: obtaining historical session data, wherein the historical session data comprises: machine conversation historical data and manual conversation historical data; extracting session features of the historical session data; and performing deep learning training on the deep learning model through the historical conversation data and the conversation features.
In an exemplary embodiment, the pre-processing data of the machine session and the human session includes: filtering the machine session and the manual session according to a noise reduction strategy; and/or segmenting the machine session and the manual session respectively according to segmentation logic to obtain a plurality of segmentation sessions respectively.
In an exemplary embodiment, the respectively segmenting the machine session and the manual session according to a segmentation logic to obtain a plurality of segmentation sessions respectively includes: determining a split code according to the split logic, wherein the split logic comprises: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers; and respectively segmenting the machine session and the manual session according to the segmentation codes to respectively obtain the plurality of segmentation sessions.
In a second aspect, an embodiment of the present disclosure provides an apparatus for detecting consistency of human-machine solutions in a customer service field, including: the extraction module is used for extracting the machine session from a first session record of the robot customer service and the client and extracting the manual session from a second session record of the manual customer service and the client; the preprocessing module is used for preprocessing data of the machine conversation and the manual conversation; a first determination module, configured to determine, through a deep learning model, a first core solution of the machine session subjected to the data preprocessing and a second core solution of the human session subjected to the data preprocessing; a calculation module, configured to calculate a consistency score of the first core solution and the second core solution through the deep learning model, where the deep learning model has learned a correspondence between the machine session and the first core solution, a correspondence between the human session and the second core solution, and a correspondence between the first core solution and the second core solution and the consistency score through deep learning; and the second determining module is used for determining the consistency rate of the core schemes corresponding to the machine session and the manual session according to the consistency score.
In a third aspect, embodiments of the present disclosure provide an electronic device. The electronic equipment comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; a memory for storing a computer program; and the processor is used for realizing the method for detecting the consistency of the man-machine scheme in the customer service field or the method for processing the image when the program stored in the memory is executed.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium. The computer-readable storage medium stores thereon a computer program that, when executed by a processor, implements the method for customer care domain human-machine scenario consistency detection or the method for image processing as described above.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure at least has part or all of the following advantages: extracting a machine conversation from a first conversation record of a robot customer service and a client, and extracting a manual conversation from a second conversation record of a manual customer service and the client; performing data preprocessing on the machine session and the manual session; determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model; calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score. Because the first core scheme of the machine session and the second core scheme of the manual session can be determined through the deep learning model, and the consistency score of the first core scheme and the second core scheme can be calculated through the deep learning model, the technical means can solve the problem that a method for comparing whether the schemes of the robot customer service and the manual customer service are consistent or not is lacked in the prior art, and further a scheme for calculating the consistency rate of the schemes of the robot customer service and the manual customer service is provided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the related art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 schematically illustrates a hardware structure block diagram of a computer terminal of a method for detecting consistency of human-computer schemes in the field of customer service according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a method for customer service domain human-machine scenario conformance detection in an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flowchart of a method for human-machine scenario consistency detection in the customer service domain according to an embodiment of the present disclosure;
FIG. 4 is a block diagram schematically illustrating an apparatus for consistency detection of a human-machine scenario in the field of customer service according to an embodiment of the present disclosure;
fig. 5 schematically shows a block diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments. It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided by the embodiments of the present disclosure may be executed in a computer terminal or a similar computing device. Taking an example of running on a computer terminal, fig. 1 schematically shows a hardware structure block diagram of the computer terminal of the method for detecting human-computer scheme consistency in the customer service field according to the embodiment of the present disclosure. As shown in fig. 1, a computer terminal may include one or more processors 102 (only one is shown in fig. 1), wherein the processors 102 may include but are not limited to a processing device such as a Microprocessor (MPU) or a Programmable Logic Device (PLD) and a memory 104 for storing data, and optionally, the computer terminal may further include a transmission device 106 for communication function and an input/output device 108, it is understood by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not a limitation to the structure of the computer terminal, for example, the computer terminal may further include more or less components than those shown in fig. 1, or have equivalent functions or different configurations than those shown in fig. 1.
The memory 104 can be used for storing computer programs, for example, software programs and modules of application software, such as a computer program corresponding to the method for detecting consistency of solutions in human-machine service field in the embodiments of the present disclosure, and the processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-mentioned methods. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In the embodiment of the present disclosure, a method for detecting consistency of human-machine solutions in a customer service field is provided, and fig. 2 schematically illustrates a flowchart of the method for detecting consistency of human-machine solutions in a customer service field according to the embodiment of the present disclosure, and as shown in fig. 2, the flowchart includes the following steps:
step S202, extracting a machine session from a first session record of a robot customer service and a client, and extracting an artificial session from a second session record of the artificial customer service and the client;
step S204, data preprocessing is carried out on the machine conversation and the manual conversation;
step S206, determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model;
step S208, calculating consistency scores of the first core scheme and the second core scheme through the deep learning model, wherein the deep learning model learns the corresponding relationship between the machine session and the first core scheme, the corresponding relationship between the manual session and the second core scheme, and the corresponding relationship between the first core scheme and the second core scheme and the consistency scores through deep learning;
step S210, determining the consistency rate of the core scheme corresponding to the machine conversation and the manual conversation according to the consistency score.
By the method, the machine session is extracted from a first session record of the robot customer service and the client, and the manual session is extracted from a second session record of the manual customer service and the client; performing data preprocessing on the machine session and the manual session; determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model; calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score. Because the first core scheme of the machine session and the second core scheme of the manual session can be determined through the deep learning model, and the consistency score of the first core scheme and the second core scheme can be calculated through the deep learning model, the technical means can solve the problem that a method for comparing whether the schemes of the robot customer service and the manual customer service are consistent or not is lacked in the prior art, and further a scheme for calculating the consistency rate of the schemes of the robot customer service and the manual customer service is provided.
In step S206, determining a first core solution of the machine session subjected to the data preprocessing and a second core solution of the human session subjected to the data preprocessing by a deep learning model, including: extracting a first session feature of the machine session subjected to the data preprocessing and a second session feature of the manual session subjected to the data preprocessing; determining the first core scheme according to the first session characteristics and determining the second core scheme according to the second session characteristics.
Because there is much useless information in both machine and manual conversations, for example, the conversation of customer service and customer call is useless. According to the method and the device for processing the machine conversation, the first core scheme of the machine conversation and the second core scheme of the manual conversation are determined through the deep learning model, then the consistency score of the first core scheme and the second core scheme is calculated, and the interference of more or less useless information is reduced. It should be noted that there is a difference between the extraction of the core solution of the machine conversation and the manual conversation. The extraction of the core scheme of the machine conversation is different from the extraction of the core scheme of the manual conversation in the data preprocessing, and some keyword filtering of invalid answers of the machine is required to be added.
Machine answers containing, for example, the following keywords can be filtered out: ' to better serve ' for you, ' Hi-happy with you ', ' good at the admirable PLUS member of PLUS ', ' good at the afternoon ', ' good at the morning ', ' good at the evening ', ' i have a question to consult ', ' if you go shopping to a question, you can try to click on the lower button to solve ' the shopping question that he goes, ' Hi-input you meet ', ' ask you describe specifically the question of thinking at a glance ', ' ask you click on the lower button to transfer to the artificial customer service ', ' Hi-, ' buy the good at the kyoto, JIMI is asking for you to relieve himself ', ' receive feedback from you, JIMI goes back to good learning ', ' contact artificial consultation ', ' other questions ', ' ask what can be helped ', ' what can be helped up ', ' what can be called up ', ' what you ' and ' can cause a question of you ', 'guess you are also interested in' … …
Determining the first core scenario according to the first session characteristics and the second core scenario according to the second session characteristics, comprising: acquiring a core scheme set; determining a first cut session corresponding to the first session feature as the first core scheme under the condition that the similarity between the first session feature and the core scheme set is greater than a first preset threshold, wherein the first cut session is obtained by performing the data preprocessing on the machine session; and determining a second segmentation session corresponding to the second session feature as the second core scheme under the condition that the similarity between the second session feature and the core scheme set is greater than the first preset threshold, wherein the second segmentation session is obtained by performing the data preprocessing on the manual session.
The disclosed embodiment adopts the idea similar to divide and conquer, and resolves a more complex problem into a plurality of relatively easy problems. Specifically, a machine session and a manual session are firstly split into a plurality of split sessions, and then whether the split sessions are judged from the dimension of the split sessions is a core scheme or not is judged. The machine session and the manual session are disassembled into a plurality of first segmentation sessions, and the manual session are disassembled into a plurality of second segmentation sessions.
In an alternative embodiment, a set of non-core solutions is obtained; determining a first cut session corresponding to the first session feature as the first non-core scheme under the condition that the similarity between the first session feature and the non-core scheme set is greater than a second preset threshold, wherein the first cut session is obtained by performing data preprocessing on the machine session; and under the condition that the similarity between the second session characteristic and the non-core scheme set is greater than a second preset threshold, determining the segmentation session corresponding to the second session characteristic as the second non-core scheme, wherein the second segmentation session is obtained by performing data preprocessing on the manual session.
The first session feature and the second session feature each comprise at least one of: a first feature, configured to represent a business meaning of a segmentation session, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: a first segmentation session and a second segmentation session; a second feature, configured to indicate whether a customer service answer in the segmentation session is an question; a third feature for representing key information of customer service answers in the segmentation session; a fourth feature for representing a sentence length of the segmentation session; a fifth feature, configured to represent a sequence number of the split session in the machine session or the manual session; a sixth feature for indicating a business meaning of a last one of the slicing sessions.
The segmentation session of the core scheme tends to have the following characteristics: have specific business implications; the customer service answer in the segmentation session is a positive sentence; having key information: such as key action words, such as "check, apply for, transact, process, follow-up, implement, return electricity", and key business words, such as "after sale, insurance, return, change, repair, refund, logistics, compensation"; the sentence length of the segmentation conversation is in a first preset interval, for example, the sentence length is in an interval of 7-25 words; the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval, such as 1 st to 5 th or 3 last segmentation session intervals; the last of the split sessions has a business meaning. The embodiment of the disclosure can judge whether the segmentation session corresponding to the session feature is the core scheme or not by calculating the similarity between the session feature and the core scheme set.
The core scheme set comprises: a first object, configured to indicate that the segmentation session has a business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions; a second object for indicating that the customer service answer in the segmentation session is not an question; a third object for indicating that the customer service answer in the segmentation session contains key information; the fourth object is used for indicating that the sentence length of the segmentation conversation is in a first preset interval; the fifth object is used for indicating that the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval; a sixth object for indicating that a last one of the slicing sessions has a business meaning.
In an optional embodiment, the set of non-core schemes includes: a first object, configured to indicate that the segmentation session has no business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions; a second object for indicating that the customer service answer in the segmentation session is an question; a third object for indicating that the customer service answer in the segmentation session does not contain key information; the fourth object is used for indicating that the sentence length of the segmentation conversation is not in the first preset interval; a fifth object, configured to indicate that a sequence number of the split session in the machine session or the manual session is not in a second preset interval; a sixth object for indicating that a last one of the slicing sessions has no business meaning.
Training the deep learning model, including: obtaining historical session data, wherein the historical session data comprises: machine conversation historical data and manual conversation historical data; extracting session features of the historical session data; and performing deep learning training on the deep learning model through the historical conversation data and the conversation features.
The disclosed embodiments add a layer of LSTM coding network to the deep learning model. The LSTM encoded network input is a conversation and the output is a sentence vector of the conversation. And performing deep learning training on the deep learning model through the historical conversation data and the conversation features, wherein in practice, the deep learning training is performed on the deep learning model through sentence vectors of the historical conversation data and the conversation features. The deep learning model after training can uniformly process sentence vectors and conversation features, and meanwhile, the deep learning model has the capabilities of finding core schemes corresponding to conversations and calculating consistency scores of the first core schemes and the second core schemes. Wherein the LSTM encoding network is a long-term and short-term memory network. The deep learning model learns the corresponding relation between the machine session and the first core scheme and the corresponding relation between the artificial session and the second core scheme through deep learning, and actually learns the corresponding relation between the machine session and the first session characteristic and the corresponding relation between the artificial session and the second session characteristic through deep learning, wherein whether a first segmentation session of the machine session is a first core scheme or not can be judged according to the first session characteristic, and whether a second segmentation session of the artificial session is a second core scheme or not can be judged according to the second session characteristic.
The sentence vector can also be obtained by firstly obtaining a word vector of the session by using word2vec, and then performing a layer of LSTM coding based on the word vector, thereby obtaining the sentence vector information. Based on the above results, the cosine similarity of the two vectors is directly calculated, and a consistency score can be obtained. Or matching can be directly performed by bert, and sentences of manual and machine core schemes are directly input, so that the model gives a consistency score.
In step 204, the machine session and the human session are subjected to data preprocessing, including: filtering the machine session and the manual session according to a noise reduction strategy; and/or segmenting the machine session and the manual session respectively according to segmentation logic to obtain a plurality of segmentation sessions respectively.
And the noise reduction strategy filters the machine conversation and the manual conversation, actually filters noise information, reduces problem complexity and facilitates the extraction of subsequent core schemes. The noise reduction strategy means that the filtering is performed when any 1 of the following conditions is satisfied: the length of the customer question is less than or equal to 3 words; the customer's question does not contain Chinese; the customer's question is associated with any sentence in the following set, the LSC common subsequence score is greater than 0.8. The set mainly comprises some non-service meanings spoken by the user, including' good thank you, good trouble, possible, change to manual, i want to change manual, user initiate change to manual, not so clear, get the best time; the answer length of the customer service is less than or equal to 3; the customer service answer does not contain Chinese; the customer service answer is with any sentence in the following set, the LSC score is greater than 0.8. The set mainly includes some non-business meanings of customer service, including "what can help you, don't have relations, can, please provide your order, trouble provide your order, please feel a little, please wait patiently, be looking at, be processing, sister processing, you see a line, you see you can".
And respectively segmenting the machine session and the manual session according to segmentation logic to respectively obtain a plurality of segmentation sessions, wherein the segmentation sessions comprise: determining a split code according to the split logic, wherein the split logic comprises: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers; and respectively segmenting the machine session and the manual session according to the segmentation codes to respectively obtain the plurality of segmentation sessions.
The division logic adopts the idea of division and treatment or the idea similar to the division and treatment, and simplifies the complex problem. Finding the core solution in a long session is difficult, but finding key answers in a question-answer pair is relatively easy. The segmentation logic may be to segment the session according to the following rules: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers.
Table 1 is a long session, which includes multiple rounds of questions and answers, as follows:
"0" is a marker for the non-core scheme and "1" is a marker for the core scheme.
Table 1 is a resulting split session after splitting a long session, as follows:
pair denotes a slicing session, and Pair1 denotes a slicing session with sequence number 1 in a long session.
It should be noted that before the data preprocessing is performed on the machine session and the human session, a desensitization process may also be performed on the machine session and the human session.
In order to better understand the technical solutions, the embodiments of the present disclosure also provide an alternative embodiment for explaining the technical solutions.
Fig. 3 schematically illustrates a flowchart of a method for detecting consistency of human-machine solutions in the customer service field according to an embodiment of the present disclosure, and as shown in fig. 3:
s302: extracting a machine conversation from a first conversation record of a robot customer service and a client, and extracting a manual conversation from a second conversation record of a manual customer service and the client;
s304: desensitizing the machine session and the human session;
s306: filtering the machine session and the manual session according to a noise reduction strategy;
s308: segmenting the machine session and the manual session respectively according to segmentation logic to obtain a plurality of segmentation session fruits respectively, wherein the machine session is segmented to obtain a plurality of first segmentation sessions, and the manual session is segmented to obtain a plurality of second segmentation sessions;
s310: respectively determining whether the plurality of first segmentation sessions are first core schemes and whether the plurality of second segmentation sessions are second core schemes through a deep learning model;
s312: calculating, by the deep learning model, a consistency score for the first core solution and the second core solution;
s314: and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score.
By the method, the machine session is extracted from a first session record of the robot customer service and the client, and the manual session is extracted from a second session record of the manual customer service and the client; performing data preprocessing on the machine session and the manual session; determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model; calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score. Because the first core scheme of the machine session and the second core scheme of the manual session can be determined through the deep learning model, and the consistency score of the first core scheme and the second core scheme can be calculated through the deep learning model, the technical means can solve the problem that a method for comparing whether the schemes of the robot customer service and the manual customer service are consistent or not is lacked in the prior art, and further a scheme for calculating the consistency rate of the schemes of the robot customer service and the manual customer service is provided.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present disclosure or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, and an optical disk), and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a component server, or a network device) to execute the methods of the embodiments of the present disclosure.
The embodiment further provides a device for detecting consistency of human-machine schemes in the customer service field, where the device for detecting consistency of human-machine schemes in the customer service field is used to implement the foregoing embodiment and preferred embodiments, and the description of the device that has been already made is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a block diagram schematically illustrating a structure of an apparatus for human-machine scenario consistency detection in the customer service field according to an alternative embodiment of the present disclosure, where, as shown in fig. 4, the apparatus includes:
an extracting module 402, configured to extract a machine session from a first session record of a robot service and a customer, and extract an artificial session from a second session record of an artificial service and the customer;
a preprocessing module 404, configured to perform data preprocessing on the machine session and the human session;
a first determining module 406, configured to determine, through a deep learning model, a first core solution of the machine session subjected to the data preprocessing and a second core solution of the human session subjected to the data preprocessing;
a calculating module 408, configured to calculate a consistency score of the first core solution and the second core solution through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core solution, a correspondence between the human session and the second core solution, and a correspondence between the first core solution and the second core solution and the consistency score through deep learning;
a second determining module 410, configured to determine a consistency rate of a core scenario corresponding to the machine session and the human session according to the consistency score.
By the method, the machine session is extracted from a first session record of the robot customer service and the client, and the manual session is extracted from a second session record of the manual customer service and the client; performing data preprocessing on the machine session and the manual session; determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model; calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning; and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score. Because the first core scheme of the machine session and the second core scheme of the manual session can be determined through the deep learning model, and the consistency score of the first core scheme and the second core scheme can be calculated through the deep learning model, the technical means can solve the problem that a method for comparing whether the schemes of the robot customer service and the manual customer service are consistent or not is lacked in the prior art, and further a scheme for calculating the consistency rate of the schemes of the robot customer service and the manual customer service is provided.
Optionally, the first determining module 406 is further configured to extract a first session feature of the machine session subjected to the data preprocessing and a second session feature of the human session subjected to the data preprocessing; determining the first core scheme according to the first session characteristics and determining the second core scheme according to the second session characteristics.
Because there is much useless information in both machine and manual conversations, for example, the conversation of customer service and customer call is useless. According to the method and the device for processing the machine conversation, the first core scheme of the machine conversation and the second core scheme of the manual conversation are determined through the deep learning model, then the consistency score of the first core scheme and the second core scheme is calculated, and the interference of more or less useless information is reduced. It should be noted that there is a difference between the extraction of the core solution of the machine conversation and the manual conversation. The extraction of the core scheme of the machine conversation is different from the extraction of the core scheme of the manual conversation in the data preprocessing, and some keyword filtering of invalid answers of the machine is required to be added.
Machine answers containing, for example, the following keywords can be filtered out: ' to better serve ' for you, ' Hi-happy with you ', ' good at the admirable PLUS member of PLUS ', ' good at the afternoon ', ' good at the morning ', ' good at the evening ', ' i have a question to consult ', ' if you go shopping to a question, you can try to click on the lower button to solve ' the shopping question that he goes, ' Hi-input you meet ', ' ask you describe specifically the question of thinking at a glance ', ' ask you click on the lower button to transfer to the artificial customer service ', ' Hi-, ' buy the good at the kyoto, JIMI is asking for you to relieve himself ', ' receive feedback from you, JIMI goes back to good learning ', ' contact artificial consultation ', ' other questions ', ' ask what can be helped ', ' what can be helped up ', ' what can be called up ', ' what you ' and ' can cause a question of you ', 'guess you are also interested in' … …
Optionally, the first determining module 406 is further configured to obtain a core scheme set; determining a first cut session corresponding to the first session feature as the first core scheme under the condition that the similarity between the first session feature and the core scheme set is greater than a first preset threshold, wherein the first cut session is obtained by performing the data preprocessing on the machine session; and determining a second segmentation session corresponding to the second session feature as the second core scheme under the condition that the similarity between the second session feature and the core scheme set is greater than the first preset threshold, wherein the second segmentation session is obtained by performing the data preprocessing on the manual session.
The disclosed embodiment adopts the idea similar to divide and conquer, and resolves a more complex problem into a plurality of relatively easy problems. Specifically, a machine session and a manual session are firstly split into a plurality of split sessions, and then whether the split sessions are judged from the dimension of the split sessions is a core scheme or not is judged. The machine session and the manual session are disassembled into a plurality of first segmentation sessions, and the manual session are disassembled into a plurality of second segmentation sessions.
Optionally, the first determining module 406 is further configured to obtain a non-core scheme set; determining a first cut session corresponding to the first session feature as the first non-core scheme under the condition that the similarity between the first session feature and the non-core scheme set is greater than a second preset threshold, wherein the first cut session is obtained by performing data preprocessing on the machine session; and under the condition that the similarity between the second session characteristic and the non-core scheme set is greater than a second preset threshold, determining the segmentation session corresponding to the second session characteristic as the second non-core scheme, wherein the second segmentation session is obtained by performing data preprocessing on the manual session.
The first session feature and the second session feature each comprise at least one of: a first feature, configured to represent a business meaning of a segmentation session, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: a first segmentation session and a second segmentation session; a second feature, configured to indicate whether a customer service answer in the segmentation session is an question; a third feature for representing key information of customer service answers in the segmentation session; a fourth feature for representing a sentence length of the segmentation session; a fifth feature, configured to represent a sequence number of the split session in the machine session or the manual session; a sixth feature for indicating a business meaning of a last one of the slicing sessions.
The segmentation session of the core scheme tends to have the following characteristics: have specific business implications; the customer service answer in the segmentation session is a positive sentence; having key information: such as key action words, such as "check, apply for, transact, process, follow-up, implement, return electricity", and key business words, such as "after sale, insurance, return, change, repair, refund, logistics, compensation"; the sentence length of the segmentation conversation is in a first preset interval, for example, the sentence length is in an interval of 7-25 words; the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval, such as 1 st to 5 th or 3 last segmentation session intervals; the last of the split sessions has a business meaning. The embodiment of the disclosure can judge whether the segmentation session corresponding to the session feature is the core scheme or not by calculating the similarity between the session feature and the core scheme set.
The core scheme set comprises: a first object, configured to indicate that the segmentation session has a business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions; a second object for indicating that the customer service answer in the segmentation session is not an question; a third object for indicating that the customer service answer in the segmentation session contains key information; the fourth object is used for indicating that the sentence length of the segmentation conversation is in a first preset interval; the fifth object is used for indicating that the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval; a sixth object for indicating that a last one of the slicing sessions has a business meaning.
In an optional embodiment, the set of non-core schemes includes: a first object, configured to indicate that the segmentation session has no business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions; a second object for indicating that the customer service answer in the segmentation session is an question; a third object for indicating that the customer service answer in the segmentation session does not contain key information; the fourth object is used for indicating that the sentence length of the segmentation conversation is not in the first preset interval; a fifth object, configured to indicate that a sequence number of the split session in the machine session or the manual session is not in a second preset interval; a sixth object for indicating that a last one of the slicing sessions has no business meaning.
Optionally, the calculating module 408 is further configured to obtain historical session data, where the historical session data includes: machine conversation historical data and manual conversation historical data; extracting session features of the historical session data; and performing deep learning training on the deep learning model through the historical conversation data and the conversation features.
The disclosed embodiments add a layer of LSTM coding network to the deep learning model. The LSTM encoded network input is a conversation and the output is a sentence vector of the conversation. And performing deep learning training on the deep learning model through the historical conversation data and the conversation features, wherein in practice, the deep learning training is performed on the deep learning model through sentence vectors of the historical conversation data and the conversation features. The deep learning model after training can uniformly process sentence vectors and conversation features, and meanwhile, the deep learning model has the capabilities of finding core schemes corresponding to conversations and calculating consistency scores of the first core schemes and the second core schemes. Wherein the LSTM encoding network is a long-term and short-term memory network. The deep learning model learns the corresponding relation between the machine session and the first core scheme and the corresponding relation between the artificial session and the second core scheme through deep learning, and actually learns the corresponding relation between the machine session and the first session characteristic and the corresponding relation between the artificial session and the second session characteristic through deep learning, wherein whether a first segmentation session of the machine session is a first core scheme or not can be judged according to the first session characteristic, and whether a second segmentation session of the artificial session is a second core scheme or not can be judged according to the second session characteristic.
Optionally, the preprocessing module 404 is further configured to filter the machine session and the human session according to a denoising policy; and/or segmenting the machine session and the manual session respectively according to segmentation logic to obtain a plurality of segmentation sessions respectively.
And the noise reduction strategy filters the machine conversation and the manual conversation, actually filters noise information, reduces problem complexity and facilitates the extraction of subsequent core schemes. The noise reduction strategy means that the filtering is performed when any 1 of the following conditions is satisfied: the length of the customer question is less than or equal to 3 words; the customer's question does not contain Chinese; the customer's question is associated with any sentence in the following set, the LSC common subsequence score is greater than 0.8. The set mainly comprises some non-service meanings spoken by the user, including' good thank you, good trouble, possible, change to manual, i want to change manual, user initiate change to manual, not so clear, get the best time; the answer length of the customer service is less than or equal to 3; the customer service answer does not contain Chinese; the customer service answer is with any sentence in the following set, the LSC score is greater than 0.8. The set mainly includes some non-business meanings of customer service, including "what can help you, don't have relations, can, please provide your order, trouble provide your order, please feel a little, please wait patiently, be looking at, be processing, sister processing, you see a line, you see you can".
Optionally, the preprocessing module 404 is further configured to determine a splitting code according to the splitting logic, where the splitting logic includes: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers; and respectively segmenting the machine session and the manual session according to the segmentation codes to respectively obtain the plurality of segmentation sessions.
The division logic adopts the idea of division and treatment or the idea similar to the division and treatment, and simplifies the complex problem. Finding the core solution in a long session is difficult, but finding key answers in a question-answer pair is relatively easy. The segmentation logic may be to segment the session according to the following rules: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers.
Table 1 is a long session, which includes multiple rounds of questions and answers, as follows:
"0" is a marker for the non-core scheme and "1" is a marker for the core scheme.
Table 1 is a resulting split session after splitting a long session, as follows:
pair denotes a slicing session, and Pair1 denotes a slicing session with sequence number 1 in a long session.
It should be noted that before the data preprocessing is performed on the machine session and the human session, a desensitization process may also be performed on the machine session and the human session.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Embodiments of the present disclosure provide an electronic device.
Fig. 5 schematically shows a block diagram of an electronic device provided in an embodiment of the present disclosure.
Referring to fig. 5, an electronic device 500 provided in the embodiment of the present disclosure includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, where the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504; a memory 503 for storing a computer program; the processor 501 is configured to implement the steps in any of the above method embodiments when executing the program stored in the memory.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, extracting the machine conversation from the first conversation record of the robot customer service and the client, and extracting the manual conversation from the second conversation record of the manual customer service and the client;
s2, performing data preprocessing on the machine conversation and the manual conversation;
s3, determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model;
s4, calculating consistency scores of the first core scheme and the second core scheme through the deep learning model, wherein the deep learning model learns the corresponding relation between the machine session and the first core scheme, the corresponding relation between the manual session and the second core scheme, and the corresponding relation between the first core scheme and the second core scheme and the consistency scores through deep learning;
s5, determining the consistency rate of the core scheme corresponding to the machine conversation and the manual conversation according to the consistency score.
Embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of any of the method embodiments described above.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, extracting the machine conversation from the first conversation record of the robot customer service and the client, and extracting the manual conversation from the second conversation record of the manual customer service and the client;
s2, performing data preprocessing on the machine conversation and the manual conversation;
s3, determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model;
s4, calculating consistency scores of the first core scheme and the second core scheme through the deep learning model, wherein the deep learning model learns the corresponding relation between the machine session and the first core scheme, the corresponding relation between the manual session and the second core scheme, and the corresponding relation between the first core scheme and the second core scheme and the consistency scores through deep learning;
s5, determining the consistency rate of the core scheme corresponding to the machine conversation and the manual conversation according to the consistency score.
The computer-readable storage medium may be contained in the apparatus/device described in the above embodiments; or may be present alone without being assembled into the device/apparatus. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present disclosure described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. As such, the present disclosure is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the principle of the present disclosure should be included in the protection scope of the present disclosure.
Claims (10)
1. A method for detecting consistency of man-machine schemes in the field of customer service is characterized by comprising the following steps:
extracting a machine conversation from a first conversation record of a robot customer service and a client, and extracting a manual conversation from a second conversation record of a manual customer service and the client;
performing data preprocessing on the machine session and the manual session;
determining a first core scheme of the machine conversation subjected to the data preprocessing and a second core scheme of the manual conversation subjected to the data preprocessing through a deep learning model;
calculating a consistency score of the first core scenario and the second core scenario through the deep learning model, wherein the deep learning model has learned a correspondence between the machine session and the first core scenario, a correspondence between the human session and the second core scenario, and a correspondence between the first core scenario and the second core scenario and the consistency score through deep learning;
and determining the consistency rate of the core schemes corresponding to the machine conversation and the manual conversation according to the consistency score.
2. The method of claim 1, wherein determining a first core scenario of the machine session subjected to the data pre-processing and a second core scenario of the human session subjected to the data pre-processing through a deep learning model comprises:
extracting a first session feature of the machine session subjected to the data preprocessing and a second session feature of the manual session subjected to the data preprocessing;
determining the first core scheme according to the first session characteristics and determining the second core scheme according to the second session characteristics.
3. The method of claim 2, wherein determining the first core scenario according to the first session characteristics and determining the second core scenario according to the second session characteristics comprises:
acquiring a core scheme set;
determining a first cut session corresponding to the first session feature as the first core scheme under the condition that the similarity between the first session feature and the core scheme set is greater than a first preset threshold, wherein the first cut session is obtained by performing the data preprocessing on the machine session;
and determining a second segmentation session corresponding to the second session feature as the second core scheme under the condition that the similarity between the second session feature and the core scheme set is greater than the first preset threshold, wherein the second segmentation session is obtained by performing the data preprocessing on the manual session.
4. The method of claim 2, wherein the first session characteristic and the second session characteristic each comprise at least one of:
a first feature, configured to represent a business meaning of a segmentation session, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: a first segmentation session and a second segmentation session;
a second feature, configured to indicate whether a customer service answer in the segmentation session is an question;
a third feature for representing key information of customer service answers in the segmentation session;
a fourth feature for representing a sentence length of the segmentation session;
a fifth feature, configured to represent a sequence number of the split session in the machine session or the manual session;
a sixth feature for indicating a business meaning of a last one of the slicing sessions.
5. The method of claim 3, wherein the core solution set comprises:
a first object, configured to indicate that the segmentation session has a business meaning, where the segmentation session is obtained by performing the data preprocessing on the machine session and the manual session, and the segmentation session includes: the first and second split sessions;
a second object for indicating that the customer service answer in the segmentation session is not an question;
a third object for indicating that the customer service answer in the segmentation session contains key information;
the fourth object is used for representing the segmentation, and the sentence length of the conversation is in a first preset interval;
the fifth object is used for indicating that the sequence number of the segmentation session in the machine session or the manual session is in a second preset interval;
a sixth object for indicating that a last one of the slicing sessions has a business meaning.
6. The method of claim 1, comprising:
obtaining historical session data, wherein the historical session data comprises: machine conversation historical data and manual conversation historical data;
extracting session features of the historical session data;
and performing deep learning training on the deep learning model through the historical conversation data and the conversation features.
7. The method of claim 1, wherein the pre-processing data for the machine session and the human session comprises:
filtering the machine session and the manual session according to a noise reduction strategy; and/or
And segmenting the machine session and the manual session respectively according to segmentation logic to obtain a plurality of segmentation sessions respectively.
8. The method of claim 7, wherein the segmenting the machine session and the human session according to segmentation logic to obtain a plurality of segmented sessions respectively comprises:
determining a split code according to the split logic, wherein the split logic comprises: one question followed by one answer, multiple questions followed by one answer, one question followed by multiple answers, and multiple questions followed by multiple answers;
and respectively segmenting the machine session and the manual session according to the segmentation codes to respectively obtain the plurality of segmentation sessions.
9. A device for detecting consistency of man-machine schemes in the field of customer service is characterized by comprising:
the extraction module is used for extracting the machine session from a first session record of the robot customer service and the client and extracting the manual session from a second session record of the manual customer service and the client;
the preprocessing module is used for preprocessing data of the machine conversation and the manual conversation;
a first determination module, configured to determine, through a deep learning model, a first core solution of the machine session subjected to the data preprocessing and a second core solution of the human session subjected to the data preprocessing;
a calculation module, configured to calculate a consistency score of the first core solution and the second core solution through the deep learning model, where the deep learning model has learned a correspondence between the machine session and the first core solution, a correspondence between the human session and the second core solution, and a correspondence between the first core solution and the second core solution and the consistency score through deep learning;
and the second determining module is used for determining the consistency rate of the core schemes corresponding to the machine session and the manual session according to the consistency score.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111217426.9A CN113887246A (en) | 2021-10-19 | 2021-10-19 | Method and device for detecting consistency of man-machine schemes in customer service field and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111217426.9A CN113887246A (en) | 2021-10-19 | 2021-10-19 | Method and device for detecting consistency of man-machine schemes in customer service field and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113887246A true CN113887246A (en) | 2022-01-04 |
Family
ID=79003679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111217426.9A Pending CN113887246A (en) | 2021-10-19 | 2021-10-19 | Method and device for detecting consistency of man-machine schemes in customer service field and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113887246A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170195486A1 (en) * | 2015-12-30 | 2017-07-06 | Shanghai Xiaoi Robot Technology Co., Ltd. | Intelligent customer service systems, customer service robots, and methods for providing customer service |
| WO2018001169A1 (en) * | 2016-06-28 | 2018-01-04 | 中兴通讯股份有限公司 | Automatic questioning and answering method and system |
| WO2018006727A1 (en) * | 2016-07-08 | 2018-01-11 | 阿里巴巴集团控股有限公司 | Method and apparatus for transferring from robot customer service to human customer service |
| CN108399169A (en) * | 2017-02-06 | 2018-08-14 | 阿里巴巴集团控股有限公司 | Dialog process methods, devices and systems based on question answering system and mobile device |
| CN109508367A (en) * | 2018-09-30 | 2019-03-22 | 厦门快商通信息技术有限公司 | Automatically extract the method, on-line intelligence customer service system and electronic equipment of question and answer corpus |
| CN110110054A (en) * | 2019-03-22 | 2019-08-09 | 北京中科汇联科技股份有限公司 | A method of obtaining question and answer pair in the slave non-structured text based on deep learning |
| CN111143537A (en) * | 2019-12-30 | 2020-05-12 | 税友软件集团股份有限公司 | Service method, device, equipment and medium based on intelligent customer service system |
| CN111459917A (en) * | 2020-04-17 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Knowledge base management method, device and processing equipment |
-
2021
- 2021-10-19 CN CN202111217426.9A patent/CN113887246A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170195486A1 (en) * | 2015-12-30 | 2017-07-06 | Shanghai Xiaoi Robot Technology Co., Ltd. | Intelligent customer service systems, customer service robots, and methods for providing customer service |
| WO2018001169A1 (en) * | 2016-06-28 | 2018-01-04 | 中兴通讯股份有限公司 | Automatic questioning and answering method and system |
| WO2018006727A1 (en) * | 2016-07-08 | 2018-01-11 | 阿里巴巴集团控股有限公司 | Method and apparatus for transferring from robot customer service to human customer service |
| CN108399169A (en) * | 2017-02-06 | 2018-08-14 | 阿里巴巴集团控股有限公司 | Dialog process methods, devices and systems based on question answering system and mobile device |
| CN109508367A (en) * | 2018-09-30 | 2019-03-22 | 厦门快商通信息技术有限公司 | Automatically extract the method, on-line intelligence customer service system and electronic equipment of question and answer corpus |
| CN110110054A (en) * | 2019-03-22 | 2019-08-09 | 北京中科汇联科技股份有限公司 | A method of obtaining question and answer pair in the slave non-structured text based on deep learning |
| CN111143537A (en) * | 2019-12-30 | 2020-05-12 | 税友软件集团股份有限公司 | Service method, device, equipment and medium based on intelligent customer service system |
| CN111459917A (en) * | 2020-04-17 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Knowledge base management method, device and processing equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112365894B (en) | AI-based composite voice interaction method and device and computer equipment | |
| CN109981910B (en) | Service recommendation method and device | |
| CN111932144B (en) | Customer service agent distribution method and device, server and storage medium | |
| CN110347863A (en) | Talk about art recommended method and device and storage medium | |
| CN112183098B (en) | Session processing method and device, storage medium and electronic device | |
| CN113360622B (en) | User dialogue information processing method and device and computer equipment | |
| CN110399473B (en) | Method and device for determining answers to user questions | |
| CN112256856B (en) | Robot dialogue method, robot dialogue device, electronic equipment and storage medium | |
| CN112182186A (en) | Intelligent customer service operation method, device and system | |
| CN110489519B (en) | Session method based on session prediction model and related products | |
| CN115470318A (en) | Customer service problem processing method and device | |
| CN112836515B (en) | Text analysis method, recommendation method, device, electronic equipment and storage medium | |
| CN114969352A (en) | Text processing method, system, storage medium and electronic equipment | |
| CN113254788A (en) | Big data based recommendation method and system and readable storage medium | |
| CN112331201A (en) | Voice interaction method and device, storage medium and electronic device | |
| CN116525093A (en) | Prediction method, device, equipment and storage medium for session ending | |
| CN113887246A (en) | Method and device for detecting consistency of man-machine schemes in customer service field and storage medium | |
| CN114202363B (en) | Artificial intelligence-based call method, device, computer equipment and medium | |
| CN113468306B (en) | Voice dialogue method and device, electronic equipment and storage medium | |
| CN116127010A (en) | Intention recognition method, device, electronic equipment and storage medium | |
| CN114969295A (en) | Dialog interaction data processing method, device and equipment based on artificial intelligence | |
| CN115510199A (en) | Data processing method, device and system | |
| CN112784032A (en) | Conversation corpus recommendation evaluation method and device, storage medium and electronic equipment | |
| CN111818290A (en) | Online interviewing method and system | |
| CN113722466B (en) | Correction model training method, correction method, device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |