CN110069776B - Customer satisfaction evaluation method and device and computer readable storage medium - Google Patents
Customer satisfaction evaluation method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN110069776B CN110069776B CN201910208064.3A CN201910208064A CN110069776B CN 110069776 B CN110069776 B CN 110069776B CN 201910208064 A CN201910208064 A CN 201910208064A CN 110069776 B CN110069776 B CN 110069776B
- Authority
- CN
- China
- Prior art keywords
- training sample
- semantic
- sentence
- sequence
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Machine Translation (AREA)
Abstract
A customer satisfaction assessment method and device and a computer-readable storage medium, wherein the customer satisfaction assessment method comprises the following steps: acquiring a multi-round dialog text to be evaluated; dividing the sentences in the multi-turn dialog text according to time sequence to obtain a plurality of sentence sequences; respectively extracting the features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set; constructing time sequence characteristic sequences corresponding to the statement sequences one by one according to the effective characteristic set; and inputting the time sequence characteristic sequences corresponding to the sentence sequences one by one into a preset recurrent neural network classifier for customer satisfaction evaluation to obtain a customer satisfaction evaluation result corresponding to the multi-turn dialog text. By adopting the scheme, the evaluation precision of the customer satisfaction can be improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of natural language, in particular to a method and a device for evaluating customer satisfaction and a computer-readable storage medium.
Background
Customer service dialog systems are widely used in business customer service practices. When the client uses the product or consults the product, the client can communicate with the customer service to acquire the information which the client wants to know. The client can have a conversation with the customer service through a text chat interface or a telephone.
In the prior art, in order to know the satisfaction degree of a customer, the satisfaction degree of the customer can be evaluated according to a chat record between the customer and a customer service. However, existing customer satisfaction assessments are less accurate.
Disclosure of Invention
The technical problem solved by the embodiment of the invention is that the evaluation precision of the customer satisfaction degree is low.
To solve the above technical problem, an embodiment of the present invention provides a method for evaluating customer satisfaction, including: acquiring a multi-round dialog text to be evaluated; dividing the sentences in the multi-turn dialogue text according to time sequence to obtain a plurality of sentence sequences; respectively extracting the features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set; constructing time sequence characteristic sequences corresponding to the statement sequences one by one according to the effective characteristic set; and inputting the time sequence characteristic sequences corresponding to the sentence sequences one by one into a preset recurrent neural network classifier for customer satisfaction evaluation to obtain a customer satisfaction evaluation result corresponding to the multi-turn dialog text.
Optionally, the performing feature extraction on the plurality of sentence sequences by using a natural language feature extractor to obtain an effective feature set includes: semantically dividing each sentence in the ith sentence sequence to obtain a semantic sequence, wherein the semantic sequence comprises P semantic units, and P is more than or equal to 1; tagging the source of each semantic unit; counting the occurrence frequency of each semantic unit in the multi-turn dialog text; calculating the importance degree of each semantic unit according to the source and the occurrence frequency of each semantic unit; and screening and determining an effective feature set corresponding to the multi-round dialog text based on the occurrence frequency and the importance degree of each semantic unit, wherein the effective feature set comprises at least one semantic unit, i is less than or equal to M, and M is the total number of the sentence sequence.
Optionally, any one of the following division modes is adopted to semantically divide each sentence in the ith sentence sequence to obtain a semantic sequence: semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence; and semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence.
Optionally, the importance degree of each semantic unit is calculated by using any one of the following algorithms: TF-IDF, information gain, chi-square statistic and mutual information.
Optionally, the recurrent neural network classifier is obtained by training in the following manner: acquiring a training sample set, wherein the training sample set comprises a plurality of multi-turn dialogue training sample texts; dividing sentences in each multi-round dialogue training text sample according to time sequence to obtain a plurality of sentence sequences; performing feature extraction on the sentence sequence corresponding to each multi-turn dialogue training sample text by adopting a natural language feature extractor to obtain an effective feature set corresponding to the training sample set; constructing a time sequence characteristic sequence corresponding to each multi-round dialogue training sample text one by one according to the effective characteristic set corresponding to the training sample set; and inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one by one into a recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
Optionally, the performing feature extraction on the sentence sequence corresponding to each multi-turn dialog training sample text by using a natural language feature extractor to obtain an effective feature set corresponding to the training sample set includes: semantically dividing each sentence in a sentence sequence corresponding to each multi-round dialogue training sample text to obtain a semantic sequence corresponding to each multi-round dialogue training sample text, wherein the semantic sequence corresponding to each multi-round dialogue training sample text comprises Q semantic units, and Q is more than or equal to 1; marking the source of each semantic unit corresponding to each multi-round dialogue training sample text; counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text; calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining an effective feature set corresponding to each multi-round dialogue training sample text based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text, wherein the effective feature set corresponding to each multi-round dialogue training sample text comprises at least one semantic unit.
The embodiment of the invention also provides a customer satisfaction evaluating device, which comprises: the acquisition unit is suitable for acquiring a plurality of rounds of dialog texts to be evaluated; the dividing unit is suitable for dividing the sentences in the multi-round dialogue text according to time sequence to obtain a plurality of sentence sequences; the feature extraction unit is suitable for respectively extracting features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set; the time sequence characteristic sequence construction unit is suitable for constructing time sequence characteristic sequences corresponding to the statement sequences one by one according to the effective characteristic set; and the evaluation unit is suitable for inputting the time sequence characteristic sequences corresponding to the sentence sequences one by one into a preset recurrent neural network classifier to evaluate the customer satisfaction degree, so as to obtain a customer satisfaction degree evaluation result corresponding to the multi-turn dialog text.
Optionally, the feature extraction unit is adapted to semantically divide each sentence in the ith sentence sequence to obtain a semantic sequence, where the semantic sequence includes P semantic units, and P is greater than or equal to 1; tagging the source of each semantic unit; counting the occurrence frequency of each semantic unit in the multi-turn dialog text; calculating the importance degree of each semantic unit according to the source and the occurrence frequency of each semantic unit; and screening and determining an effective feature set corresponding to the multi-turn dialog text based on the occurrence frequency and the importance degree of each semantic unit, wherein the effective feature set comprises at least one semantic unit, i is less than or equal to M, and M is the total number of the sentence sequences.
Optionally, the feature extraction unit is adapted to semantically divide each sentence in the ith sentence sequence by using any one of the following dividing manners to obtain a semantic sequence: semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence; and semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence.
Optionally, the feature extraction unit is adapted to calculate the importance degree of each semantic unit by using any one of the following algorithms: TF-IDF, information gain, chi-square statistic and mutual information.
Optionally, the customer satisfaction evaluating apparatus further includes: the model construction unit is suitable for obtaining the recurrent neural network classifier by training in the following way: acquiring a training sample set, wherein the training sample set comprises a plurality of multi-turn dialogue training sample texts; dividing sentences in each multi-round dialogue training text sample according to time sequence to obtain a plurality of sentence sequences; performing feature extraction on the sentence sequence corresponding to each multi-turn dialogue training sample text by adopting a natural language feature extractor to obtain an effective feature set corresponding to the training sample set; constructing a time sequence characteristic sequence corresponding to each multi-round dialogue training sample text one by one according to the effective characteristic set corresponding to the training sample set; and inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one to a recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
Optionally, the model building unit is adapted to semantically divide each sentence in the sentence sequence corresponding to each multi-turn dialogue training sample text to obtain a semantic sequence corresponding to each multi-turn dialogue training sample text, where the semantic sequence corresponding to each multi-turn dialogue training sample text includes Q semantic units, and Q is greater than or equal to 1; marking the source of each semantic unit corresponding to each multi-round dialogue training sample text; counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text; calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining an effective feature set corresponding to each multi-round dialogue training sample text based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text, wherein the effective feature set corresponding to each multi-round dialogue training sample text comprises at least one semantic unit.
The embodiment of the present invention further provides a customer satisfaction evaluating device, which includes a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the computer instructions to perform any of the steps of the customer satisfaction evaluating method described above.
Embodiments of the present invention further provide a computer-readable storage medium, which is a non-volatile storage medium or a non-transitory storage medium, and has stored thereon computer instructions, where the computer instructions, when executed, perform any of the steps of the customer satisfaction assessment method described above.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
and dividing the acquired multi-turn dialog text according to time sequence to acquire a plurality of sentence sequences. And respectively extracting the features of the plurality of sentence sequences by adopting a natural language feature extractor to obtain time sequence feature sequences corresponding to the plurality of sentence sequences one by one. And performing customer satisfaction evaluation by adopting a recurrent neural network classifier based on the time sequence characteristic sequences corresponding to the statement sequences one to obtain a customer satisfaction evaluation result. When the customer satisfaction is evaluated, because the multi-turn dialog text is adopted and divided according to the time sequence, the obtained sentence sequence has the precedence relationship in the time sequence, the feature extraction and the customer satisfaction evaluation are carried out based on the sentence sequence with the time sequence precedence relationship, and the accuracy of the obtained customer satisfaction evaluation can be improved.
Drawings
FIG. 1 is a flow chart of a customer satisfaction assessment method in an embodiment of the present invention;
FIG. 2 is a flow chart of an efficient feature extraction in an embodiment of the present invention;
FIG. 3 is a flow chart of a recurrent neural network classifier in an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a customer satisfaction evaluating apparatus in an embodiment of the present invention.
Detailed Description
In the prior art, to know customer satisfaction, customer satisfaction can be evaluated generally based on a record of chatting between the customer and the customer service. However, the existing customer satisfaction evaluation is generally performed by subjective judgment of chat records by staff, and the accuracy is low.
In the embodiment of the invention, when the customer satisfaction evaluation is carried out, because the multiple turns of dialogue texts are adopted and divided according to the time sequence, the obtained sentence sequences have the precedence relationship in the time sequence, and the characteristic extraction and the customer satisfaction evaluation are carried out based on the sentence sequences with the time sequence precedence relationship, the accuracy of the obtained customer satisfaction evaluation can be improved.
In order to make the aforementioned objects, features and advantages of the embodiments of the present invention more comprehensible, specific embodiments accompanied with figures are described in detail below.
Referring to fig. 1, a flow chart of a customer satisfaction assessment method in an embodiment of the invention is shown, which may include the following steps.
And step 11, acquiring a plurality of rounds of dialog texts to be evaluated.
In a particular implementation, the multi-turn dialog text is text that includes at least two turns of dialog between the customer and the customer service.
In the embodiment of the invention, a plurality of rounds of dialog texts to be evaluated can be obtained from dialogs generated by customer service and customers through a character chat interface; or converting the dialogue audio of the customer service and the customer into corresponding texts, and acquiring multiple rounds of dialogue texts to be evaluated from the converted texts.
And step 12, dividing the sentences in the multi-turn dialog text according to time sequence to obtain a plurality of sentence sequences.
In a particular implementation, multiple rounds of dialog text may include multiple sentences. After the multi-turn dialog text to be evaluated is obtained, the sentences in the multi-turn dialog text can be divided according to the time sequence to obtain a plurality of sentence sequences.
For example, the multi-turn dialog text includes 10 sentences, and the 10 sentences are divided according to the chronological order of occurrence of the 10 sentences by taking 1 sentence as a unit, so as to obtain a 10-sentence sequence.
For another example, the multi-turn dialog text includes 10 sentences, and 2 sentences are taken as a unit, and the 10 sentences are divided according to the chronological order of occurrence to obtain 5 sentence sequences.
It can be understood that the number of sentence sequences obtained by dividing the multi-turn dialog text may be set according to the actual application requirement, and is not limited herein.
And step 13, respectively performing feature extraction on the plurality of sentence sequences by adopting a natural language feature extractor to obtain an effective feature set.
In a specific implementation, after a plurality of sentence sequences corresponding to a plurality of turns of dialog texts are obtained, the plurality of sentence sequences can be input into the natural language feature extractor. And respectively extracting the features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set.
Referring to fig. 2, a flowchart of effective feature extraction in the embodiment of the present invention is given, and a process of extracting effective features is described below by taking an example in which a multi-turn dialog text includes N sentence sequences.
And step 21, semantically dividing each statement in the ith statement sequence to obtain a semantic sequence.
In the specific implementation, each sentence in the ith sentence sequence is semantically divided to obtain a semantic sequence, and each semantic sequence can comprise P semantic units, wherein P is more than or equal to 1.
In a particular implementation, the presence of features in word granularity, or punctuation dimension in multiple rounds of dialog text can reflect customer satisfaction, such as "nice," good, "" go ahead, "" | in multiple rounds of dialog text! "and the like.
In the embodiment of the invention, each statement in the ith statement sequence can be divided semantically by word granularity to obtain a word dimension semantic sequence; or semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence, wherein i is less than or equal to N.
For example, the sentence sequence is: the Shanghai Pudong New district century park is semantically divided by word granularity to obtain a word dimension semantic sequence [ Shanghai, pudong New district, century park ], wherein the Shanghai, the Pudong New district, and the century park are three semantic units of the semantic sequence respectively.
The source of each semantic unit is tagged 22.
In a specific implementation, since the same semantic unit is derived from different objects, the expressed meanings are different, for example, the semantic unit with the appearance of "good" in the statement of the customer may be different from the meaning represented by the semantic unit with the appearance of "good" in the statement of the customer service. In order to accurately distinguish the meanings represented by the semantic units, the source of each semantic unit is labeled. For example, whether each semantic unit is from a customer service or from a customer.
And step 23, counting the occurrence frequency of each semantic unit in the multi-turn dialog text.
In specific implementation, the frequency of each semantic unit appearing in the multi-turn dialog text is counted and recorded as the word frequency of the semantic unit.
And 24, calculating the importance degree of each semantic unit according to the source and the appearance frequency of each semantic unit.
In a specific implementation, the statistical value of each semantic unit can be calculated by using any one of the following algorithms, and the importance degree of each semantic unit is represented by using the statistical value: term Frequency-Inverse text Frequency index (TF-IDF), information gain, chi-square statistic, mutual information, and the like. It can be understood that other algorithms can be selected to calculate the importance degree of the semantic unit according to the actual application scenario and requirements.
And 25, screening and determining effective feature sets corresponding to the multiple rounds of dialog texts based on the occurrence frequency and the importance degree of each semantic unit.
In one embodiment, a semantic unit may be regarded as a feature, and a semantic unit may be a valid feature or an invalid feature. In the embodiment of the invention, some semantic units can be screened out as effective feature sets of the multi-round dialog text based on the occurrence frequency and the importance degree of each semantic unit. The valid feature set may include one or more semantic units.
For example, statistical values are adopted to represent the importance degree of semantic units, each semantic unit is arranged from high to low according to the importance degree, the semantic units with the occurrence frequency meeting the preset frequency are screened out as effective features, and the screened effective features form an effective feature set.
It can be understood that other screening conditions may be set according to the requirements of the actual application scenario to perform screening of the effective features, and details are not described here.
After the determination of the valid feature set, a plurality of rounds of dialog text can be given a feature expression in a mathematical sense. For example, the valid feature set is [ very good, ok ], and the valid features are "very good" and "ok". When there is a corresponding valid feature, a 1 may be adopted; when there is no corresponding valid feature, 0 is used for this. The statement "may, feel very good" may be expressed as [1,1]; the statement "very good! "may be represented as [1,0].
And 14, constructing a time sequence characteristic sequence corresponding to the statement sequences one by one according to the effective characteristic set.
In a specific implementation, after the valid feature set is obtained, a time sequence feature sequence corresponding to a plurality of sentence sequences one by one can be constructed according to the valid feature set.
In the embodiment of the present invention, the time sequence feature sequence may be a matrix of T × M, where T represents the number of time sequence features in the time sequence feature sequence, and the time sequence features are statements in a multi-turn dialog text; m represents the dimension, related to the number of semantic units comprised by the set of valid features. When the value of T is set, considering that the number of dialog turns of multiple rounds of dialog is different, all information of multiple rounds of dialog needs to be included as much as possible, and invalid information is prevented from being introduced as much as possible. For example, for a multi-turn conversation containing N statements, if T < N, then N-T statements are discarded; if T > N, T-N invalid features are introduced.
And step 15, inputting the time sequence characteristic sequences corresponding to the statement sequences one to one into a preset recurrent neural network classifier to evaluate the customer satisfaction, so as to obtain a customer satisfaction evaluation result corresponding to the multi-turn dialog text.
In specific implementation, the time sequence characteristic sequences corresponding to the sentence sequences one to one are input into a preset recurrent neural network classifier to carry out customer satisfaction evaluation, and customer satisfaction evaluation results corresponding to the multi-turn dialog text are obtained.
In specific implementation, the expression form of the customer satisfaction evaluation result is various: the evaluation result may be satisfied, unsatisfied, or not designated attitude, or may be a satisfaction grade, or may be a satisfaction score, or the like, and the specific expression form of the customer satisfaction evaluation result may be set according to the actual demand, which is not limited herein.
According to the scheme, the acquired multi-turn dialog text is divided according to the time sequence, and a plurality of sentence sequences are acquired. And respectively extracting the features of the plurality of sentence sequences by adopting a natural language feature extractor to obtain time sequence feature sequences corresponding to the plurality of sentence sequences one by one. And performing customer satisfaction evaluation by adopting a recurrent neural network classifier based on the time sequence characteristic sequences corresponding to the sentence sequences one by one to obtain a customer satisfaction evaluation result. When the customer satisfaction is evaluated, because the multi-turn dialog text is adopted and divided according to the time sequence, the obtained sentence sequence has the precedence relationship in the time sequence, the feature extraction and the customer satisfaction evaluation are carried out based on the sentence sequence with the time sequence precedence relationship, and the accuracy of the obtained customer satisfaction evaluation can be improved.
In a specific implementation, the recurrent neural network classifier used can be constructed in the following manner. Referring to fig. 3, a flow chart of constructing a recurrent neural network classifier in an embodiment of the present invention is shown, and may include the following steps.
In implementations, the training sample set may include a plurality of multi-turn dialog training sample texts.
And step 32, dividing the sentences in each multi-round dialogue training sample text according to time sequence to obtain a plurality of sentence sequences.
In specific implementation, the sentences in the multi-turn dialogue training sample text can be divided according to time sequence to obtain a plurality of sentence sequences corresponding to each multi-turn dialogue training sample text.
And step 33, performing feature extraction on the sentence sequences corresponding to the multi-turn dialogue training sample texts by using a natural language feature extractor to obtain effective feature sets corresponding to the training sample sets.
In specific implementation, the sentence sequence corresponding to each multi-turn dialogue training sample text is input into a natural language feature extractor for feature extraction, and an effective feature set corresponding to a training sample set is obtained.
Specifically, semantically dividing each sentence in the sentence sequence corresponding to each multi-round dialogue training sample text to obtain a semantic sequence corresponding to each multi-round dialogue training sample text, wherein the semantic sequence corresponding to each multi-round dialogue training sample text comprises Q semantic units, and Q is more than or equal to 1; marking the source of each semantic unit corresponding to each multi-round dialogue training sample text; counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text; calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining an effective feature set corresponding to the training sample set based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-turn dialogue training sample text, wherein the effective feature set comprises at least one semantic unit.
The process of acquiring the effective feature set of the training sample set may refer to the description of the process of acquiring the effective feature set of the text to be evaluated, which is not described herein again.
And step 34, constructing a time sequence feature sequence corresponding to each multi-round dialogue training sample text one to one according to the effective feature set corresponding to the training sample set.
In one embodiment, the time-series feature sequence corresponding to each of the multi-turn dialog training sample texts may be a T × M matrix.
And step 35, inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one to one into the recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
In a specific implementation, corresponding label information exists in the text of the multi-turn dialogue training sample, such as satisfaction, dissatisfaction or no attitude indication. And inputting the time sequence characteristic sequences of the multi-round dialogue training sample texts into a recurrent neural network model for training, so that the recurrent neural network model continuously learns the multi-round dialogue training sample texts and the corresponding label information thereof, and continuously optimizes the parameters of the recurrent neural network model until the preset parameters of the recurrent neural network model are converged. The preset parameter convergence may be the type of the neural unit in the recurrent neural network, the number of the hidden units, the number of the layers, or the learning rate. The types of Neural units generally include Recurrent Neural Network cells (RNN-cells), long Short-Term Memory Network cells (LSTM-cells), and Gated Recurrent Units (GRUs). The number of hidden units is typically set to 32, 64, 128, 256 or 512, etc. The number of the layers is generally 1 layer, 2 layers or 3 layers and the like; the learning rate is generally set between 0.0001 and 0.1. The optimization method can adopt a Stochastic Gradient Descent method (SGD). In specific implementation, the parameters can be adjusted to achieve a better effect for different tasks and different application scenarios. After the training of the recurrent neural network classifier is finished, the structural parameters and the network weight parameters are fixed and stored as binary files, and the binary files can be used for subsequent multi-round dialogue text to evaluate the satisfaction degree of a client.
In order to facilitate better understanding and implementation of the embodiments of the present invention for those skilled in the art, the embodiments of the present invention further provide a customer satisfaction evaluating apparatus.
Referring to fig. 4, a schematic structural diagram of a customer satisfaction evaluating apparatus according to an embodiment of the present invention is shown. The customer satisfaction evaluating means 40 may include: an obtaining unit 41, a dividing unit 42, a feature extracting unit 43, a time sequence feature constructing unit 44, and an evaluating unit 45, wherein:
an obtaining unit 41 adapted to obtain a plurality of turns of dialog text to be evaluated;
the dividing unit 42 is adapted to divide the sentences in the multi-round dialog text according to a time sequence to obtain a plurality of sentence sequences;
a feature extraction unit 43, adapted to perform feature extraction on the sentence sequences respectively by using a natural language feature extractor to obtain an effective feature set;
a time sequence feature sequence construction unit 44, adapted to construct a time sequence feature sequence corresponding to the plurality of sentence sequences one by one according to the effective feature set;
and the evaluation unit 45 is adapted to input the time sequence feature sequences corresponding to the sentence sequences one to one into a preset recurrent neural network classifier to perform customer satisfaction evaluation, so as to obtain a customer satisfaction evaluation result corresponding to the multi-turn dialog text.
In a specific implementation, the feature extraction unit 43 is adapted to semantically divide each sentence in the ith sentence sequence to obtain a semantic sequence, where the semantic sequence includes P semantic units, and P is greater than or equal to 1; tagging the source of each semantic unit; counting the occurrence frequency of each semantic unit in the multi-turn dialog text; calculating the importance degree of each semantic unit according to the source and the occurrence frequency of each semantic unit; and screening and determining an effective feature set corresponding to the multi-turn dialog text based on the occurrence frequency and the importance degree of each semantic unit, wherein the effective feature set comprises at least one semantic unit, i is less than or equal to M, and M is the total number of the sentence sequences.
In a specific implementation, the feature extraction unit 43 is adapted to semantically divide each sentence in the ith sentence sequence by using any one of the following dividing manners to obtain a semantic sequence: semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence; and semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence.
In a specific implementation, the feature extraction unit 43 is adapted to calculate the importance of each semantic unit by using any one of the following algorithms: TF-IDF, information gain, chi-square statistic and mutual information.
In a specific implementation, the customer satisfaction evaluating device 40 further includes: a model construction unit (not shown in fig. 4) adapted to train the recurrent neural network classifier in the following manner: acquiring a training sample set, wherein the training sample set comprises a plurality of multi-turn dialogue training sample texts; dividing sentences in each multi-round dialogue training text sample according to time sequence to obtain a plurality of sentence sequences; performing feature extraction on the sentence sequence corresponding to each multi-turn dialogue training sample text by adopting a natural language feature extractor to obtain an effective feature set corresponding to the training sample set; according to the effective feature set corresponding to the training sample set, a time sequence feature sequence corresponding to each multi-round dialogue training sample text one to one is constructed; and inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one by one into a recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
In specific implementation, the model construction unit is adapted to semantically divide each sentence in the sentence sequence corresponding to each multi-turn dialogue training sample text to obtain a semantic sequence corresponding to each multi-turn dialogue training sample text, wherein the semantic sequence corresponding to each multi-turn dialogue training sample text comprises Q semantic units, and Q is greater than or equal to 1; marking the source of each semantic unit corresponding to each multi-round dialogue training sample text; counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text; calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining the effective feature set corresponding to each multi-round dialogue training sample text based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text.
In a specific implementation, the working principle and the working process of the customer satisfaction evaluating apparatus 40 may refer to the description of any one of the customer satisfaction evaluating methods provided in the embodiments of the present invention, and are not described herein again.
The embodiment of the present invention further provides a customer satisfaction evaluating apparatus, which includes a memory and a processor, where the memory stores computer instructions that can be executed on the processor, and the processor executes the computer instructions to perform any of the above steps of the customer satisfaction evaluating method provided by the embodiment of the present invention.
The embodiment of the present invention further provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transitory storage medium, and has stored thereon computer instructions, where the computer instructions, when executed, perform any of the above steps of the customer satisfaction assessment method provided by the embodiment of the present invention.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a program, and the program may be stored in any computer-readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, and the like.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (12)
1. A customer satisfaction assessment method, comprising:
acquiring a multi-round dialog text to be evaluated;
dividing the sentences in the multi-turn dialog text according to time sequence to obtain a plurality of sentence sequences;
respectively extracting the features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set;
constructing time sequence characteristic sequences corresponding to the statement sequences one by one according to the effective characteristic set; inputting the time sequence characteristic sequences corresponding to the sentence sequences one by one into a preset recurrent neural network classifier for customer satisfaction evaluation to obtain a customer satisfaction evaluation result corresponding to the multi-turn dialog text;
wherein, the feature extraction is respectively carried out on the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set, and the method comprises the following steps:
semantically dividing each sentence in the ith sentence sequence to obtain a semantic sequence, wherein the semantic sequence comprises P semantic units, and P is more than or equal to 1;
tagging the source of each semantic unit;
counting the occurrence frequency of each semantic unit in the multi-turn dialog text;
calculating the importance degree of each semantic unit according to the source and the occurrence frequency of each semantic unit; based on the occurrence frequency and the importance degree of each semantic unit, screening and determining an effective feature set corresponding to the multi-round dialog text, wherein the effective feature set comprises at least one semantic unit, i is less than or equal to M,
m is the total number of sentence sequences.
2. The customer satisfaction evaluation method of claim 1, wherein each sentence in the ith sentence sequence is semantically divided into semantic sequences by any one of the following dividing methods:
semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence;
and semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence.
3. The customer satisfaction evaluation method of claim 1, wherein the importance of each semantic unit is calculated using any one of the following algorithms:
TF-IDF, information gain, chi-square statistic and mutual information.
4. The customer satisfaction evaluation method of claim 1, wherein said recurrent neural network classifier is trained by:
acquiring a training sample set, wherein the training sample set comprises a plurality of multi-turn dialogue training sample texts;
dividing sentences in each multi-round dialogue training text sample according to time sequence to obtain a plurality of sentence sequences;
performing feature extraction on the sentence sequence corresponding to each multi-turn dialogue training sample text by adopting a natural language feature extractor to obtain an effective feature set corresponding to the training sample set;
constructing a time sequence characteristic sequence corresponding to each multi-round dialogue training sample text one by one according to the effective characteristic set corresponding to the training sample set;
and inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one to a recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
5. The method according to claim 4, wherein the extracting the sentence sequence corresponding to each multi-turn dialogue training sample text by using a natural language feature extractor to obtain the valid feature set corresponding to the training sample set comprises:
semantically dividing each sentence in the sentence sequence corresponding to each multi-round dialogue training sample text to obtain a semantic sequence corresponding to each multi-round dialogue training sample text, wherein the semantic sequence corresponding to each multi-round dialogue training sample text comprises Q semantic units, and Q is more than or equal to 1;
marking the source of each semantic unit corresponding to each multi-round dialogue training sample text;
counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text;
calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining an effective feature set corresponding to each multi-round dialogue training sample text based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text, wherein the effective feature set corresponding to each multi-round dialogue training sample text comprises at least one semantic unit.
6. A customer satisfaction evaluating apparatus characterized by comprising:
the acquisition unit is suitable for acquiring a plurality of rounds of dialog texts to be evaluated;
the dividing unit is suitable for dividing the sentences in the multi-round dialogue text according to time sequence to obtain a plurality of sentence sequences;
the feature extraction unit is suitable for respectively extracting features of the sentence sequences by adopting a natural language feature extractor to obtain an effective feature set;
the time sequence characteristic sequence construction unit is suitable for constructing time sequence characteristic sequences corresponding to the statement sequences one by one according to the effective characteristic set;
the evaluation unit is suitable for inputting the time sequence characteristic sequences corresponding to the sentence sequences one by one into a preset recurrent neural network classifier to evaluate the customer satisfaction degree, so as to obtain a customer satisfaction degree evaluation result corresponding to the multi-turn dialog text;
the feature extraction unit is suitable for semantically dividing each statement in the ith statement sequence to obtain a semantic sequence, wherein the semantic sequence comprises P semantic units, and P is more than or equal to 1; tagging the source of each semantic unit; counting the occurrence frequency of each semantic sequence in the multi-turn dialog text;
calculating the importance degree of each semantic unit according to the source and the occurrence frequency of each semantic unit; based on the occurrence frequency and the importance degree of each semantic unit, screening and determining an effective feature set corresponding to the multi-round dialog text, wherein the effective feature set comprises at least one semantic unit, i is less than or equal to M,
m is the total number of sentence sequences.
7. The customer satisfaction evaluation device according to claim 6, wherein said feature extraction unit is adapted to semantically divide each sentence in the ith sentence sequence into semantic sequences by any one of the following dividing methods: semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence; and semantically dividing each sentence in the ith sentence sequence by word granularity to obtain a word dimension semantic sequence.
8. The customer satisfaction evaluation device according to claim 6, wherein the feature extraction unit is adapted to calculate the importance degree of each semantic unit using any one of the following algorithms: TF-IDF, information gain, chi-square statistic and mutual information.
9. The customer satisfaction evaluating apparatus according to claim 6, further comprising: the model construction unit is suitable for obtaining the recurrent neural network classifier by adopting the following training modes: acquiring a training sample set, wherein the training sample set comprises a plurality of multi-turn dialogue training sample texts; dividing sentences in each multi-round dialogue training text sample according to time sequence to obtain a plurality of sentence sequences; performing feature extraction on the sentence sequences corresponding to the multi-round dialogue training sample texts by adopting a natural language feature extractor to obtain effective feature sets corresponding to the training sample sets; constructing a time sequence characteristic sequence corresponding to each multi-round dialogue training sample text one by one according to the effective characteristic set corresponding to the training sample set; and inputting the time sequence characteristic sequences corresponding to the multi-round dialogue training sample texts one to a recurrent neural network model for training until the preset parameters of the recurrent neural network model are converged to obtain the recurrent neural network classifier.
10. The customer satisfaction evaluation device according to claim 9, wherein the model construction unit is adapted to semantically divide each sentence in the sentence sequence corresponding to each multi-turn dialogue training sample text to obtain a semantic sequence corresponding to each multi-turn dialogue training sample text, wherein the semantic sequence corresponding to each multi-turn dialogue training sample text comprises Q semantic units, and Q is greater than or equal to 1; marking the source of each semantic unit corresponding to each multi-round dialogue training sample text; counting the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text in the multi-round dialogue training sample text; calculating the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text according to the source and the occurrence frequency of each semantic unit corresponding to each multi-round dialogue training sample text; and screening and determining an effective feature set corresponding to each multi-round dialogue training sample text based on the occurrence frequency and the importance degree of each semantic unit corresponding to each multi-round dialogue training sample text, wherein the effective feature set corresponding to each multi-round dialogue training sample text comprises at least one semantic unit.
11. A customer satisfaction evaluation device comprising a memory and a processor, said memory having stored thereon a computer program operable on said processor, wherein said processor executes the steps of the customer satisfaction evaluation method of any of claims 1-5 when executing said computer program.
12. A computer-readable storage medium, being a non-volatile storage medium or a non-transitory storage medium, having a computer program stored thereon, wherein the computer program, when being executed by a processor, is adapted to perform the steps of the customer satisfaction assessment method according to any of the claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910208064.3A CN110069776B (en) | 2019-03-19 | 2019-03-19 | Customer satisfaction evaluation method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910208064.3A CN110069776B (en) | 2019-03-19 | 2019-03-19 | Customer satisfaction evaluation method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110069776A CN110069776A (en) | 2019-07-30 |
| CN110069776B true CN110069776B (en) | 2023-04-18 |
Family
ID=67366375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910208064.3A Active CN110069776B (en) | 2019-03-19 | 2019-03-19 | Customer satisfaction evaluation method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110069776B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909136B (en) * | 2019-10-10 | 2023-05-23 | 百度在线网络技术（北京）有限公司 | Satisfaction degree estimation model training method and device, electronic equipment and storage medium |
| CN111091832B (en) * | 2019-11-28 | 2022-12-30 | 秒针信息技术有限公司 | Intention assessment method and system based on voice recognition |
| CN111754980A (en) * | 2020-05-21 | 2020-10-09 | 华南理工大学 | Intelligent scoring method and device based on semantic recognition and storage medium |
| CN111666396B (en) * | 2020-06-05 | 2023-10-31 | 北京百度网讯科技有限公司 | User intention understanding satisfaction evaluation method, device, equipment and storage medium |
| CN112883180A (en) * | 2021-02-24 | 2021-06-01 | 挂号网（杭州）科技有限公司 | Model training method and device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654250A (en) * | 2016-02-01 | 2016-06-08 | 百度在线网络技术（北京）有限公司 | Method and device for automatically assessing satisfaction degree |
| CN108664473A (en) * | 2018-05-11 | 2018-10-16 | 平安科技（深圳）有限公司 | Recognition methods, electronic device and the readable storage medium storing program for executing of text key message |
-
2019
- 2019-03-19 CN CN201910208064.3A patent/CN110069776B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110069776A (en) | 2019-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110069776B (en) | Customer satisfaction evaluation method and device and computer readable storage medium | |
| CN107818798B (en) | Customer service quality evaluation method, device, equipment and storage medium | |
| CN110619506B (en) | Post image generation method, post image generation device and electronic equipment | |
| US20110282648A1 (en) | Machine Translation with Side Information | |
| CN112328742A (en) | Training method and device based on artificial intelligence, computer equipment and storage medium | |
| EP4170520A1 (en) | Method and device for constructing knowledge graph, computer device, and storage medium | |
| CN110795913B (en) | Text encoding method, device, storage medium and terminal | |
| CN110727778A (en) | Intelligent question-answering system for tax affairs | |
| KR102244938B1 (en) | Artificial intelligence employment system and employing method of thereof | |
| CN113468296B (en) | Model self-iteration type intelligent customer service quality inspection system and method capable of configuring business logic | |
| CN108804526A (en) | Interest determines that system, interest determine method and storage medium | |
| CN108920513A (en) | A kind of multimedia data processing method, device and electronic equipment | |
| CN108039168B (en) | Acoustic model optimization method and device | |
| CN111182162A (en) | Telephone quality inspection method, device, equipment and storage medium based on artificial intelligence | |
| CN111401105B (en) | Video expression recognition method, device and equipment | |
| KR20230013793A (en) | Method and Apparatus for Classifying Document Based on Attension Mechanism and Semantic Analysis | |
| CN113361266A (en) | Text error correction method, electronic device and storage medium | |
| CN111597821B (en) | Method and device for determining response probability | |
| CN115063035A (en) | Customer evaluation method, system, equipment and storage medium based on neural network | |
| CN113342942A (en) | Corpus automatic acquisition method and device, computer equipment and storage medium | |
| CN112329437A (en) | Intelligent customer service voice quality inspection scoring method, equipment and storage medium | |
| CN109658148B (en) | Marketing activity complaint risk prediction method based on natural language processing technology | |
| CN113808709B (en) | Psychological elasticity prediction method and system based on text analysis | |
| CN110580899A (en) | Voice recognition method and device, storage medium and computing equipment | |
| CN113505606B (en) | Training information acquisition method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |