WO2017133165A1 - Method, apparatus and device for automatic evaluation of satisfaction and computer storage medium - Google Patents

Abstract

A method, an apparatus and a device for automatic evaluation of satisfaction and a computer storage medium. The method comprises: acquiring information sent by a user (101); extracting, from the information sent by the user, features regarding the user's satisfaction to determine input parameters of a satisfaction evaluation model (102); determining the user's satisfaction according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model (103). The method is capable of automatically evaluating the user's satisfaction from the information sent by the user by means of analyzing and processing the information sent by the user, not only improving the speed of the satisfaction evaluation and increasing the accuracy of the evaluation, but also saving the labor cost.

Description

Method, device, device and computer storage medium for automatic evaluation of satisfaction

The present application claims the priority of the Chinese patent application whose filing date is February 1, 2016, and whose application number is 201610069429.5, entitled "A Method and Apparatus for Automatic Evaluation of Satisfaction".

Technical field

The invention relates to computer data processing technology, in particular to a method and device for automatic satisfaction evaluation.

Background technique

In the voice customer service system or instant messaging (IM) customer service system, the superior of the customer service needs to know the working status of the customer service under the jurisdiction and the user feedback through the data indicators.

A common method of obtaining user feedback in the prior art is by letting the client complete a satisfaction survey, such as after the bank's customer service completes the service or after the voice customer service completes the service. Most of the existing satisfaction surveys are initiated by the customer service after the customer completes the service, allowing the customer to make evaluation choices. For example, the voice customer service will play the voice after the voice ends, and the voice prompts the customer to input the number keys 1 to 4 to determine whether the service is very satisfied, satisfied, generally or not satisfied, and the customer submits the evaluation according to the voice instruction. The existing communication customer service provides links for users to evaluate.

Regardless of the above-mentioned evaluation mechanism in the prior art, it depends to some extent on the subjective consciousness of the customer, and it takes up extra time of the customer, and sometimes makes the customer feel resentful. If the customer ignores or forgets the evaluation, it may cause certain errors in the evaluation management data of the customer service.

In summary, there is no method and apparatus in the prior art that can automatically measure user satisfaction so that user satisfaction feedback can be obtained quickly and accurately.

Summary of the invention

The invention provides a method, a device, a device and a computer storage medium for automatically measuring satisfaction, which is used for improving the speed of satisfaction evaluation, increasing the accuracy of the evaluation, and saving labor cost.

The specific technical solutions are as follows:

A method for automatic satisfaction assessment, the method comprising:

Obtain the information sent by the user;

Extracting user satisfaction characteristics from information sent by the user to determine input parameters of the satisfaction evaluation model;

The user's satisfaction is determined according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model.

According to a preferred embodiment of the present invention, before acquiring the information sent by the user, the method further includes:

The sample set is trained to obtain a satisfaction evaluation model, wherein the sample set includes samples composed of information that the user has transmitted and user satisfaction results corresponding to information that the user has transmitted.

According to a preferred embodiment of the present invention, the training sample set to obtain a satisfaction evaluation model includes:

Extracting a user satisfaction characteristic from information sent by a user of the sample set to determine an input parameter of the satisfaction evaluation model;

Using the input parameters of the extracted satisfaction evaluation model and the user satisfaction results, the machine learning method is used to train to obtain the satisfaction evaluation model.

According to a preferred embodiment of the present invention, the information sent by the user includes voice information and/or text information sent by the user.

According to a preferred embodiment of the present invention, the extracting user satisfaction characteristics from the information sent by the user includes:

At least one of a voice, a tone, a volume, and a speech rate is extracted from the voice information transmitted by the user as a satisfaction feature.

According to a preferred embodiment of the present invention, the extracting user satisfaction characteristics from the information sent by the user includes:

Converting voice information sent by the user into text information;

Extracting keywords from the converted text information as a satisfaction feature according to the keyword dictionary; or

Semantic features are extracted from the transformed text information as a satisfaction feature according to the semantic model.

According to a preferred embodiment of the present invention, the extracting user satisfaction characteristics from the information sent by the user includes:

Extracting keywords from the text information sent by the user according to the keyword dictionary as a satisfaction feature; or

Semantic features are extracted from the text information sent by the user as a satisfaction feature according to the semantic model.

According to a preferred embodiment of the present invention, extracting a keyword from the text information according to the keyword dictionary as a satisfaction feature includes:

Performing word segmentation processing on the text information to obtain a feature set;

The feature set is matched with the keyword dictionary, and the keyword matching the keyword dictionary is extracted from the feature set as the satisfaction feature.

According to a preferred embodiment of the present invention, the extracting the user's satisfaction characteristic from the information sent by the user and determining the input parameter of the satisfaction evaluation model includes at least one of the following:

After the satisfaction characteristics are quantified, the quantitative values of the satisfaction characteristics are used as input parameters of the satisfaction evaluation model; or

The input parameter of the satisfaction evaluation model is determined according to the attribute of the satisfaction feature, wherein the attribute of the satisfaction feature includes the sound frequency or the sound amplitude of the satisfaction feature; or

The parameter corresponding to the satisfaction feature matching the keyword dictionary is used as an input parameter of the satisfaction evaluation model; or

The parameters corresponding to the satisfaction characteristics matching the semantic model are used as input parameters of the satisfaction evaluation model.

According to a preferred embodiment of the present invention, determining the user satisfaction according to the input parameter of the satisfaction evaluation model and the satisfaction evaluation model includes:

Entering the input parameter of the satisfaction evaluation model into the satisfaction evaluation model, and obtaining the satisfaction of the user output by the satisfaction evaluation model;

The satisfaction evaluation model determines the scores of each satisfaction feature according to the input parameters of the satisfaction evaluation model, so as to determine the user satisfaction according to the scores of the satisfaction features.

An apparatus for automatically measuring satisfaction, the apparatus comprising:

An obtaining unit, configured to obtain information sent by a user;

An extracting unit, configured to extract a user satisfaction characteristic from the information sent by the user to determine an input parameter of the satisfaction evaluation model;

The determining unit is configured to determine the satisfaction of the user according to the input parameter of the satisfaction evaluation model and the satisfaction evaluation model.

According to a preferred embodiment of the present invention, the apparatus further includes a training unit, configured to: before the obtaining unit acquires the information sent by the user, training the sample set to obtain a satisfaction evaluation model, wherein the sample set includes information sent by the user And corresponding to the information that the user has sent A sample of the results of household satisfaction.

According to a preferred embodiment of the present invention, the training unit specifically performs the following operations:

Extracting a user satisfaction characteristic from information sent by a user of the sample set to determine an input parameter of the satisfaction evaluation model;

Using the input parameters of the extracted satisfaction evaluation model and the user satisfaction results, the machine learning method is used to train to obtain the satisfaction evaluation model.

According to a preferred embodiment of the present invention, the information sent by the user includes voice information and/or text information sent by the user.

According to a preferred embodiment of the present invention, if the information sent by the user includes voice information, the extracting unit extracts the satisfaction feature of the user from the information sent by the user by performing the following operations:

At least one of a voice, a tone, a volume, and a speech rate is extracted from the voice information transmitted by the user as a satisfaction feature.

According to a preferred embodiment of the invention, the extraction unit further performs the following operations:

Converting voice information sent by the user into text information;

Extracting keywords from the converted text information as a satisfaction feature according to the keyword dictionary; or

Semantic features are extracted from the transformed text information as a satisfaction feature according to the semantic model.

According to a preferred embodiment of the present invention, if the information sent by the user includes text information, the extracting unit extracts the satisfaction characteristic of the user from the information sent by the user by performing the following operations:

Extracting keywords from the text information sent by the user according to the keyword dictionary as a satisfaction feature; or

Semantic features are extracted from the text information sent by the user as a satisfaction feature according to the semantic model.

According to a preferred embodiment of the present invention, the extracting unit extracts a keyword from the text information as a satisfaction feature according to the keyword dictionary by performing the following operations:

Performing word segmentation processing on the text information to obtain a feature set;

The feature set is matched with the keyword dictionary, and the keyword matching the keyword dictionary is extracted from the feature set as the satisfaction feature.

According to a preferred embodiment of the present invention, the extracting unit specifically performs an operation of at least one of the following:

After the satisfaction characteristics are quantified, the quantitative values of the satisfaction characteristics are used as input parameters of the satisfaction evaluation model; or

The input parameter of the satisfaction evaluation model is determined according to the attribute of the satisfaction feature, wherein the attribute of the satisfaction feature includes the sound frequency or the sound amplitude of the satisfaction feature; or

The parameter corresponding to the satisfaction feature matching the keyword dictionary is used as an input parameter of the satisfaction evaluation model; or

The parameters corresponding to the satisfaction characteristics matching the semantic model are used as input parameters of the satisfaction evaluation model.

According to a preferred embodiment of the present invention, the determining unit specifically performs the following operations:

Entering the input parameter of the satisfaction evaluation model into the satisfaction evaluation model, and obtaining the satisfaction of the user output by the satisfaction evaluation model;

The satisfaction evaluation model determines the scores of each satisfaction feature according to the input parameters of the satisfaction evaluation model, so as to determine the user satisfaction according to the scores of the satisfaction features.

It can be seen from the above technical solution that the present invention analyzes the information sent by the user. And processing, can automatically measure the user's satisfaction from the information sent by the user, not only improves the speed of satisfaction evaluation, increases the accuracy of the evaluation, and saves labor costs.

DRAWINGS

FIG. 1 is a flowchart of a method for automatic satisfaction assessment according to Embodiment 1 of the present invention; FIG.

FIG. 2 is a schematic structural diagram of an apparatus for automatically evaluating satisfaction according to Embodiment 2 of the present invention.

detailed description

The present invention will be described in detail below with reference to the drawings and specific embodiments.

Embodiment 1

FIG. 1 is a flowchart of a method for automatically measuring satisfaction of satisfaction according to Embodiment 1 of the present invention. As shown in FIG. 1 , the method may include the following processes:

101. Acquire information sent by the user.

The process can obtain information sent by the user on which the satisfaction is analyzed.

The information sent by the user may be voice information and/or text information sent by the user. For example, in the voice customer service system, the information sent by the user may be the inquiry information sent by the user when the user communicates with the customer service, or may be the chat information input by the user through the voice function and/or by typing in the instant messaging IM customer service system. Wait.

The information sent by the user may be obtained in real time when the user generates voice or text information; or the user-generated voice or text information may be stored, and the stored information is obtained from the storage device every time interval or as needed. Information sent by the user.

In addition, when the information sent by the user is obtained, if the amount of information sent by the user is too large, the information sent by the user may be extracted, for example, the beginning of the information sent by the user is intercepted. The content of the tail, or the content of the time period in which the user satisfaction feature is most likely to occur, such as in the last 10 minutes.

The extracted rules can be set according to the rules found when training the samples, or according to the experience of the person.

102. Extract user satisfaction characteristics from information sent by the user to determine input parameters of the satisfaction evaluation model.

In the process, for voice information sent by the user, at least one of voice, intonation, volume, and speech rate may be extracted from the voice information sent by the user as a satisfaction feature; or the voice information sent by the user may be converted into text information; The keyword is extracted from the text information as a satisfaction feature according to the keyword dictionary; or the semantic feature is extracted from the text information as a satisfaction feature according to the semantic model.

For the text information sent by the user, the keyword may be extracted from the text information as a satisfaction feature according to the keyword dictionary; or the semantic feature may be extracted from the text information as a satisfaction feature according to the semantic model.

The extracting the keyword from the text information according to the keyword dictionary as the satisfaction feature may include: performing word segmentation processing on the text information to obtain a feature set; matching the feature set with the satisfaction feature dictionary, and extracting from the feature set The keyword matching the satisfaction feature dictionary is used as the satisfaction feature.

The method for determining the input parameters of the satisfaction evaluation model may include, but is not limited to, the following methods:

The first way: the satisfaction characteristics can be quantified, and the quantitative values of each satisfaction feature are used as input parameters of the satisfaction evaluation model.

For example, the higher the tone, the greater the user’s emotional excitement, the dissatisfaction; The faster, the largely because the user is emotional and not satisfied. Therefore, if the intonation is higher, the quantized value corresponding to the tonal feature can be smaller; the faster the speech rate, the smaller the quantized value corresponding to the speech rate feature can be. For another example, when the user inputs text, using an exclamation point indicates that the user is angry and not satisfied, so the text feature containing the exclamation mark can be determined to have a smaller quantization value. The smaller the quantized value, the lower the user satisfaction.

The second way: the input parameter of the satisfaction evaluation model can be determined according to the attribute of the satisfaction feature, and the attribute of the satisfaction feature can include the sound frequency or the sound amplitude of the satisfaction feature.

Specifically, if the user sends the voice information, the voice, the tone, the volume, the speech rate, and the like can be extracted from the voice information sent by the user, and the features can finally reflect the satisfaction result of the user, and thus can be used as the satisfaction for determining Satisfaction characteristics.

Since the features such as voice, intonation, volume, and speech rate have their own sound frequency or sound amplitude, for example, the pitch can be determined by the frequency, and the higher the frequency, the higher the pitch; the volume can be determined by the amplitude, and the amplitude is louder. The larger the value, the attribute value such as the sound frequency or the sound amplitude of the above feature can be used as an input parameter for measuring its own size or characteristic, and the satisfaction characteristics can be quantified by these input parameters.

For example, the input parameter may be determined to be 200 Hz according to the frequency of the acquired tone, and the input parameter may be determined to be 10 dB according to the acquired volume amplitude.

The third way: the parameter corresponding to the satisfaction feature matching the keyword dictionary can be used as the input parameter of the satisfaction evaluation model.

If the user sends the voice information, the voice information sent by the user can also be converted into text information, thereby processing and analyzing the converted text information to further quantify the satisfaction features in the voice information sent by the user. .

The technique of converting speech into text can be implemented by speech recognition technology.

In addition, there is no order for the execution of the voice information to extract the features such as voice, intonation, volume, speech rate, or the conversion of the voice information into text information, which may be performed sequentially or simultaneously, all of which are protected by the present invention. Within the scope.

Whether the text information converted by the voice information or the text information directly transmitted by the user, the above text information can be processed by extracting keywords or semantic features.

Specifically, the text information may be subjected to word segmentation processing to obtain a feature set. The word segmentation processing of the text information can be based on the traditional word segmentation dictionary, that is, the word segmentation dictionary is matched with the text information, and the words in the text information matching the word segment dictionary are taken as the obtained features, thereby forming the word segment set according to the obtained features.

For example, if the text information includes the content "I am very satisfied with the processing result", the following feature set "I/pair/processing/result/very/satisfied" can be obtained after the word segmentation processing.

After the feature set is obtained, the feature set may be matched with the keyword dictionary to extract a keyword matching the keyword dictionary from the feature set as the satisfaction feature.

The keyword dictionary here is not the same as the above-mentioned word segment dictionary. The word segment dictionary can maintain all common or traditional words, the purpose of which is to be able to divide the text information almost completely in vocabulary, and maintain it in the keyword dictionary. The vocabulary is a vocabulary that can indicate the characteristics of the user's satisfaction. The vocabulary is pre-marked by the relevant person. The relevant person can pre-maintain the keyword dictionary based on the keywords generally found to be satisfactory in the user communication.

For example, if the user is very grateful for the communication, it indicates that the satisfaction is very high, and the keyword and similar content can be maintained in the keyword dictionary.

Moreover, the content maintained in the keyword dictionary can be updated by the relevant personnel as needed.

In addition, the keyword dictionary also records parameters corresponding to the respective words, and the parameters indicate The degree of user satisfaction that each vocabulary can express.

For example, the keyword dictionary can maintain the content shown in Table 1 below:

Table 1

Keywords (satisfaction characteristics)	parameter
satisfaction	10
Thank you	5
Not satisfied	-10
difference	-8

Through the keyword dictionary, it is possible to determine which vocabulary is a keyword for determining the satisfaction feature from the feature set, thereby extracting the keywords matching the keyword dictionary from the feature set as the satisfaction feature, and The parameter corresponding to the satisfaction feature matching the keyword dictionary is used as the input parameter of the satisfaction evaluation model.

Preferably, the input parameter of the satisfaction evaluation model may also be related to the number of occurrences of the satisfaction feature. For example, if “satisfaction” occurs two or more times, the input parameter may be based on the parameter corresponding to the original keyword dictionary. A certain value is added, and the magnitude of the increase can be determined by a predetermined algorithm.

Taking the feature set "I/pair/processing/result/very/satisfied" obtained as described above as an example, according to the keyword dictionary, the satisfaction feature that can be extracted from the feature set is "satisfaction", and the satisfaction evaluation model The input parameter is 10.

The fourth way: the parameter corresponding to the satisfaction feature matching the semantic model can be used as the input parameter of the satisfaction evaluation model.

With regard to the extraction of semantic features and their input parameters, text information can be derived from semantic models. The semantic features are extracted, and the parameters corresponding to the satisfaction features matching the semantic model are used as input parameters of the satisfaction evaluation model.

Semantics can refer to the meaning of a sentence, including synonyms, synonyms, interrogative sentences, exclamatory sentences, and so on.

The semantic model may be pre-fetched by training a sample set that may contain known statements that reflect user satisfaction and their corresponding known satisfaction results, based on known statements and satisfaction results. A satisfaction parameter corresponding to the relevant sentence reflecting the user satisfaction can be obtained. The known statement can be a sentence or phrase having the same or similar semantics.

103. Determine the user's satisfaction according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model.

The satisfaction evaluation model is obtained by pre-training the sample set before acquiring the information sent by the user.

That is, before the method of automatic satisfaction evaluation is performed, in the data preparation stage, the user satisfaction characteristics can be extracted from the information sent by the user of the sample set to determine the input parameters of the satisfaction evaluation model; The input parameters of the model and the user satisfaction results are trained by the machine learning method to obtain the satisfaction evaluation model.

The sample set may include samples composed of information that the user has sent and user satisfaction results corresponding to the information that the user has sent.

The manner of extracting the user satisfaction characteristic and the input parameter of the satisfaction evaluation model in the stage of obtaining the satisfaction evaluation model and the manner of extracting the user satisfaction characteristic from the information sent by the user to determine the input parameter of the satisfaction evaluation model, The only difference is that the former is based on the information that the user in the sample set has sent as a sample, while the latter is based on analysis. User satisfaction is based on real data and not sample data. The rest of the extraction methods are the same, so I won't go into details here.

The satisfaction evaluation model may be a multi-dimensional model. The output of the multi-dimensional model of the embodiment is used to represent the satisfaction evaluation result, and each dimension of the multi-dimensional model represents each element that affects the satisfaction evaluation result.

The multidimensional model may include elements such as speech, intonation, volume, speech rate, keywords, semantic features, etc., each element of the multidimensional model corresponding to a satisfaction feature extracted from information sent by the user.

For example, a multidimensional model can be understood as a parameter xyz..., where the speech is x, the intonation is y, the keyword is z..., and the input parameters of the satisfaction evaluation model of various dimensions such as speech, intonation, and keywords can be used. Enter into the model.

Through the satisfaction evaluation model, the scores of the satisfaction characteristics can be calculated by using a predetermined algorithm of the input parameters through the satisfaction evaluation model, thereby further determining the user satisfaction according to the scores of the satisfaction characteristics.

In addition, the voice, the intonation, the volume, and the speech rate are obtained from the voice information input by the user; the keyword or the semantic is the conversion of the voice information input by the user into the text information, and the keyword is obtained from the converted text information. Or semantics can also be obtained directly from the text information entered by the user.

For example, if it is a voice customer service system, the existing satisfaction scoring mechanism can be used as a sample, the obtained known user recording is taken as the information that the user has sent, and the known user satisfaction score is taken as the user has sent the result. The user satisfaction result corresponding to the information; or, if it is the instant messaging IM customer service system, the text chat information including the expression image that the user has input may be used as the information that the user has sent, and the known user satisfaction score result is used as the use. The user satisfaction result corresponding to the information that the user has sent constitutes a training sample. After obtaining a sample set consisting of known recording records or known chat records, and known satisfaction scores, the satisfaction evaluation model can be obtained based on known recordings or texts and known results. The score of each dimension feature.

Since there are a large number of recorded or recorded chat texts, a large number of trainings can be performed using pattern recognition related algorithms, and finally a satisfaction evaluation model is generated.

This approach not only greatly reduces the extra work of the customer, but also reduces the subjective component of the customer evaluation, so that each communication produces an objective and effective evaluation.

Preferably, the method of machine learning may employ, for example, a neural network model, a decision tree, a support vector machine, etc., all of which are within the scope of the present invention.

Step 103 may be based on the satisfaction feature and the input parameter extracted by step 102, and input the satisfaction parameter of the satisfaction evaluation model into the satisfaction evaluation model according to the satisfaction evaluation model obtained in advance; and then determine the satisfaction through the satisfaction evaluation model. The score of the degree feature is used to determine the user's satisfaction based on the score of each satisfaction feature. .

For example, if the information sent by the obtained user is “I am satisfied with the processing result”, the correspondence between the extracted satisfaction characteristics and the input parameters for the information sent by the specific user is as shown in Table 2:

Table 2

Based on the correspondence between the satisfaction characteristics and the elements in the satisfaction evaluation model, input parameters [10,0,200,10,7] of [keyword, semantic, intonation, volume, speech rate] can be input to the satisfaction evaluation model. .

The satisfaction evaluation model can be normalized, and the input parameters are normalized to [1, 0, 0.3, 0.2, 0.6], and the satisfaction evaluation model is further based on the normalized input parameters according to the predetermined algorithm of the model. The scores of each satisfaction feature are determined, and finally a result is output according to the scores of the respective satisfaction features, for example, an output value of 0.9 or 0.1 is obtained.

Among them, it can be set that the closer the result value is to 1, the more satisfactory, the closer to 0, the more unsatisfactory, and the closer to 0.5, the more neutral.

In addition, the above-mentioned normalization, determining the scores of each satisfaction feature according to a predetermined algorithm, and the final score obtained are actually inputting the input parameters into the satisfaction evaluation model, and the satisfaction evaluation model. The processing process is processed by the satisfaction evaluation model, and then the satisfaction evaluation model is output as the output result of the user satisfaction.

Preferably, after obtaining the information sent by the user, the determined satisfaction of the user may be returned to the user, and if the user views the automatically generated satisfaction and adjusts the user satisfaction, the user satisfaction may be further obtained. The adjustment is made and the user's adjusted satisfaction is taken as the satisfaction of the user.

After that, you can use the automatically generated satisfaction evaluation results to further evaluate the satisfaction. The price model is trained to continuously obtain more accurate evaluation results.

Embodiment 2

FIG. 2 is a schematic structural diagram of an apparatus for automatic satisfaction evaluation according to Embodiment 2 of the present invention. As shown in FIG. 2, the apparatus may include:

The obtaining unit 201 is configured to acquire information sent by the user.

The obtaining unit 201 can mainly acquire information transmitted by the user on which the satisfaction is analyzed.

The information sent by the user may be voice information and/or text information sent by the user. For example, in the voice customer service system, the information sent by the user may be the inquiry information sent by the user when the user communicates with the customer service, or may be the chat information input by the user through the voice function and/or by typing in the instant messaging IM customer service system. Wait.

The information sent by the user may be obtained in real time when the user generates voice or text information; or the user-generated voice or text information may be stored, and the stored information is obtained from the storage device every time interval or as needed. Information sent by the user.

In addition, when the information sent by the user is obtained, if the amount of information sent by the user is too large, the information sent by the user may be extracted, for example, intercepting the content at the beginning and end of the information sent by the user, or intercepting the most likely user satisfaction. The content of the time period of the feature, such as in the last 10 minutes.

The extracted rules can be set according to the rules found when training the samples, or according to the experience of the person.

The extracting unit 202 is configured to extract a user satisfaction feature from the information sent by the user to determine an input parameter of the satisfaction evaluation model.

Specifically, for voice information sent by the user, the extracting unit 202 may send the information from the user. Extracting at least one of a voice, a tone, a volume, and a speech rate as a satisfaction feature; and converting the voice information sent by the user into text information; and extracting keywords from the text information as a satisfaction according to the keyword dictionary The feature is extracted; or, according to the semantic model, the semantic feature is extracted from the text information as a satisfaction feature.

For the text information sent by the user, the extracting unit 202 may extract the keyword as the satisfaction feature from the text information according to the keyword dictionary; or extract the semantic feature from the text information as the satisfaction feature according to the semantic model.

The extracting unit 202 may perform the following operations to extract a keyword from the text information according to a keyword dictionary as a satisfaction feature: performing word segmentation processing on the text information to obtain a feature set; and matching the feature set with the satisfaction feature dictionary. A keyword matching the satisfaction feature dictionary is extracted from the feature set as a satisfaction feature.

The extracting unit 202 may determine an input parameter of the satisfaction evaluation model by performing, but not limited to, the following operations:

The first type: the satisfaction characteristics can be quantified, and the quantitative values of each satisfaction feature are used as input parameters of the satisfaction evaluation model.

Second: the input parameter of the satisfaction evaluation model can be determined according to the attribute of the satisfaction feature, and the attribute of the satisfaction feature can include the sound frequency or the sound amplitude of the satisfaction feature.

The third type: the parameter corresponding to the satisfaction feature matching the keyword dictionary can be used as the input parameter of the satisfaction evaluation model.

If the user sends the voice information, the voice information sent by the user can also be converted into text information, thereby processing and analyzing the converted text information to further quantify the satisfaction features in the voice information sent by the user. .

The technique of converting speech into text can be implemented by speech recognition technology.

In addition, there is no order for the execution of the voice information to extract the features such as voice, intonation, volume, speech rate, or the conversion of the voice information into text information, which may be performed sequentially or simultaneously, all of which are protected by the present invention. Within the scope.

Whether the text information converted by the voice information or the text information directly transmitted by the user, the above text information can be processed by extracting keywords or semantic features.

Specifically, the text information may be subjected to word segmentation processing to obtain a feature set. The word segmentation processing of the text information can be based on the traditional word segmentation dictionary, that is, the word segmentation dictionary is matched with the text information, and the words in the text information matching the word segment dictionary are taken as the obtained features, thereby forming the word segment set according to the obtained features.

For example, if the text information includes the content "I am very satisfied with the processing result", the following feature set "I/pair/processing/result/very/satisfied" can be obtained after the word segmentation processing.

After the feature set is obtained, the feature set may be matched with the keyword dictionary to extract a keyword matching the keyword dictionary from the feature set as the satisfaction feature.

The keyword dictionary here is not the same as the above-mentioned word segment dictionary. The word segment dictionary can maintain all common or traditional words, the purpose of which is to be able to divide the text information almost completely in vocabulary, and maintain it in the keyword dictionary. The vocabulary is a vocabulary that can indicate the characteristics of the user's satisfaction. The vocabulary is pre-marked by the relevant person. The relevant person can pre-maintain the keyword dictionary based on the keywords generally found to be satisfactory in the user communication.

Moreover, the content maintained in the keyword dictionary can be updated by the relevant personnel as needed.

In addition, the keyword dictionary also records parameters corresponding to each vocabulary, which indicates the degree of user satisfaction that each vocabulary can express.

Through the keyword dictionary, it is possible to determine which words are used to determine satisfaction from the feature set. The keyword of the degree feature, so as to extract these keywords matching the keyword dictionary from the feature set as the satisfaction feature, and also the parameter corresponding to the satisfaction feature matching the keyword dictionary as the satisfaction evaluation The input parameters of the model.

Preferably, the input parameter of the satisfaction evaluation model may also be related to the number of occurrences of the satisfaction feature. For example, if the satisfaction occurs two or more times, the input parameter may be added to the parameter corresponding to the original keyword dictionary. The magnitude of the increase can be determined by a predetermined algorithm.

The fourth type: the parameter corresponding to the satisfaction feature matching the semantic model can be used as the input parameter of the satisfaction evaluation model.

With regard to the extraction of semantic features and their input parameters, semantic features can be extracted from the text information according to the semantic model, and the parameters corresponding to the satisfaction features matching the semantic model are used as input parameters of the satisfaction evaluation model.

Semantics can refer to the meaning of a sentence, including synonyms, synonyms, interrogative sentences, exclamatory sentences, and so on.

The semantic model may be pre-fetched by training a sample set that may contain known statements that reflect user satisfaction and their corresponding known satisfaction results, based on known statements and satisfaction results. A satisfaction parameter corresponding to the relevant sentence reflecting the user satisfaction can be obtained. The known statement can be a sentence or phrase having the same or similar semantics.

The determining unit 203 is configured to determine the satisfaction of the user according to the input parameter of the satisfaction evaluation model and the satisfaction evaluation model.

The satisfaction evaluation model is obtained by the training unit 204 pre-training the sample set before the obtaining unit 201 acquires the information sent by the user.

That is, before performing the method of automatic satisfaction evaluation, in the data preparation phase, the training unit 204 may extract the user satisfaction feature from the information that the user of the sample set has sent to determine the input parameter of the satisfaction evaluation model; The input parameters of the satisfaction evaluation model and the user satisfaction results are trained by the machine learning method to obtain the satisfaction evaluation model.

The sample set may include samples composed of information that the user has sent and user satisfaction results corresponding to the information that the user has sent.

The manner of extracting the user satisfaction characteristic and the input parameter of the satisfaction evaluation model in the stage of obtaining the satisfaction evaluation model and the manner of extracting the user satisfaction characteristic from the information sent by the user to determine the input parameter of the satisfaction evaluation model, The only difference is that the former is based on the information that the user in the sample set has sent as a sample. The latter is based on the actual data used to analyze the user satisfaction and not the sample data. The rest are extracted in the same way, so here No longer.

The satisfaction evaluation model may be a multi-dimensional model. The output of the multi-dimensional model of the embodiment is used to represent the satisfaction evaluation result, and each dimension of the multi-dimensional model represents each element that affects the satisfaction evaluation result.

The multidimensional model may include elements such as speech, intonation, volume, speech rate, keywords, semantic features, etc., each element of the multidimensional model corresponding to a satisfaction feature extracted from information sent by the user.

The satisfaction evaluation model can calculate the scores of each satisfaction feature according to the input parameters of the satisfaction evaluation model through a predetermined algorithm, thereby further determining the user satisfaction according to the scores of the satisfaction features.

In addition, voice, intonation, volume, and speech rate are obtained from voice information input by the user; The keyword or semantics is to convert the voice information input by the user into text information, and the keyword or the semantics can also be obtained directly from the text information input by the user.

Since there are a large number of recorded or recorded chat texts, the training unit 204 can perform a large amount of training using a pattern recognition related algorithm, and finally generate a satisfaction evaluation model.

This approach not only greatly reduces the extra work of the customer, but also reduces the subjective component of the customer evaluation, so that each communication produces an objective and effective evaluation.

Preferably, the method of machine learning may employ, for example, a neural network model, a decision tree, a support vector machine, etc., all of which are within the scope of the present invention.

The determining unit 203 may substitute the input parameter extracted by the extracting unit 202 and the satisfaction evaluation model obtained by the training unit 204, thereby substituting the input parameter of the satisfaction evaluation model into the satisfaction evaluation model; and further determining each of the satisfaction evaluation models The score of the satisfaction feature is determined by the score of each satisfaction feature to determine the user's satisfaction.

The satisfaction evaluation model can adopt a normalized manner. When the input parameters are normalized, the satisfaction evaluation model can further determine the scores of each satisfaction feature according to the normalized input parameters, thereby obtaining a result.

In addition, the normalization mentioned above, the correction using the correction value, and the final score obtained are actually the processing steps of the satisfaction evaluation model after the input parameters are input into the satisfaction evaluation model. After the degree evaluation model is processed, the satisfaction evaluation model is output as the output result of the user satisfaction.

Preferably, after acquiring the information sent by the user, the determining unit 203 may return the determined satisfaction of the user to the user, and if the user views the automatically generated satisfaction and adjusts it, the determining unit 203 may further Get the user's adjustment to satisfaction, The user's adjusted satisfaction is taken as the satisfaction of the user.

After that, the satisfaction evaluation model can be further trained by using the automatically generated satisfaction evaluation results to continuously obtain more accurate evaluation results.

By implementing the method and device for automatic satisfaction evaluation provided by the present invention, after the communication of the voice or the text of the customer and the customer service is completed, the characteristics of the voice and the semantics included in the communication can be used to perform automatic satisfaction evaluation. The evaluation criteria corresponding to these features can be obtained by machine learning. As a result, automatic satisfaction evaluation can be achieved at any time after the communication is over.

In the several embodiments provided by the present invention, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division, and the actual implementation may have another division manner.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are made within the spirit and principles of the present invention, should be included in the present invention. Within the scope of protection.

Claims (22)

1. A method for automatic evaluation of satisfaction, characterized in that the method comprises:

   Obtain the information sent by the user;

   Extracting user satisfaction characteristics from information sent by the user to determine input parameters of the satisfaction evaluation model;

   The user's satisfaction is determined according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model.
2. The method according to claim 1, wherein before the obtaining the information sent by the user, the method further comprises:

   The sample set is trained to obtain a satisfaction evaluation model, wherein the sample set includes samples composed of information that the user has transmitted and user satisfaction results corresponding to information that the user has transmitted.
3. The method of claim 2 wherein said training sample set to obtain a satisfaction rating model comprises:

   Extracting a user satisfaction characteristic from information sent by a user of the sample set to determine an input parameter of the satisfaction evaluation model;

   Using the input parameters of the extracted satisfaction evaluation model and the user satisfaction results, the machine learning method is used to train to obtain the satisfaction evaluation model.
4. The method according to claim 1, wherein the information sent by the user comprises voice information and/or text information sent by the user.
5. The method according to claim 4, wherein the extracting the satisfaction characteristics of the user from the information sent by the user comprises:

   At least one of a voice, a tone, a volume, and a speech rate is extracted from the voice information transmitted by the user as a satisfaction feature.
6. The method according to claim 5, wherein the extracting the satisfaction characteristics of the user from the information sent by the user comprises:

   Converting voice information sent by the user into text information;

   Extracting keywords from the converted text information as a satisfaction feature according to the keyword dictionary; or

   Semantic features are extracted from the transformed text information as a satisfaction feature according to the semantic model.
7. The method according to claim 4, wherein the extracting the satisfaction characteristics of the user from the information sent by the user comprises:

   Extracting keywords from the text information sent by the user according to the keyword dictionary as a satisfaction feature; or

   Semantic features are extracted from the text information sent by the user as a satisfaction feature according to the semantic model.
8. The method according to claim 6 or 7, wherein extracting the keyword from the text information according to the keyword dictionary as the satisfaction feature comprises:

   Performing word segmentation processing on the text information to obtain a feature set;

   The feature set is matched with the keyword dictionary, and the keyword matching the keyword dictionary is extracted from the feature set as the satisfaction feature.
9. The method according to claim 1, wherein the extracting the user's satisfaction characteristic from the information sent by the user and determining the input parameter of the satisfaction evaluation model comprises at least one of the following:

   After the satisfaction characteristics are quantified, the quantitative values of the satisfaction characteristics are used as input parameters of the satisfaction evaluation model; or

   The input parameters of the satisfaction evaluation model are determined according to the attributes of the satisfaction characteristics, and the satisfaction is satisfied. The attributes of the degree feature include the sound frequency or sound amplitude of the satisfaction feature; or,

   The parameter corresponding to the satisfaction feature matching the keyword dictionary is used as an input parameter of the satisfaction evaluation model; or

   The parameters corresponding to the satisfaction characteristics matching the semantic model are used as input parameters of the satisfaction evaluation model.
10. The method according to claim 1, wherein the determining the user satisfaction according to the input parameter of the satisfaction evaluation model and the satisfaction evaluation model comprises:

    Entering the input parameter of the satisfaction evaluation model into the satisfaction evaluation model, and obtaining the satisfaction of the user output by the satisfaction evaluation model;

    The satisfaction evaluation model determines the scores of each satisfaction feature according to the input parameters of the satisfaction evaluation model, so as to determine the user satisfaction according to the scores of the satisfaction features.
11. An apparatus for automatic evaluation of satisfaction, characterized in that the apparatus comprises:

    An obtaining unit, configured to obtain information sent by a user;

    An extracting unit, configured to extract a user satisfaction characteristic from the information sent by the user to determine an input parameter of the satisfaction evaluation model;

    The determining unit is configured to determine the satisfaction of the user according to the input parameter of the satisfaction evaluation model and the satisfaction evaluation model.
12. The apparatus according to claim 11, wherein the apparatus further comprises: a training unit, configured to: before the obtaining unit acquires the information sent by the user, training the sample set to obtain a satisfaction evaluation model, wherein the sample set includes A sample of the information that the user has sent and the result of the user satisfaction corresponding to the information that the user has sent.
13. The apparatus according to claim 12, wherein the training unit specifically performs the following operations:

    Extracting a user satisfaction characteristic from information sent by a user of the sample set to determine an input parameter of the satisfaction evaluation model;

    Using the input parameters of the extracted satisfaction evaluation model and the user satisfaction results, the machine learning method is used to train to obtain the satisfaction evaluation model.
14. The apparatus according to claim 11, wherein the information sent by the user comprises voice information and/or text information sent by the user.
15. The apparatus according to claim 14, wherein if the information transmitted by the user includes voice information, the extracting unit extracts a satisfaction characteristic of the user from information transmitted by the user by performing the following operations:

    At least one of a voice, a tone, a volume, and a speech rate is extracted from the voice information transmitted by the user as a satisfaction feature.
16. The apparatus according to claim 15, wherein said extracting unit further performs the following operations:

    Converting voice information sent by the user into text information;

    Extracting keywords from the converted text information as a satisfaction feature according to the keyword dictionary; or

    Semantic features are extracted from the transformed text information as a satisfaction feature according to the semantic model.
17. The apparatus according to claim 14, wherein if the information transmitted by the user includes text information, the extracting unit extracts a satisfaction characteristic of the user from information transmitted by the user by performing the following operations:

    Extracting keywords from the text information sent by the user according to the keyword dictionary as a satisfaction feature; or

    Extracting semantic features from text information sent by users according to semantic model as satisfaction Sign.
18. The apparatus according to claim 16 or 17, wherein said extracting unit extracts a keyword from the text information as a satisfaction characteristic according to the keyword dictionary by performing the following operation:

    Performing word segmentation processing on the text information to obtain a feature set;

    The feature set is matched with the keyword dictionary, and the keyword matching the keyword dictionary is extracted from the feature set as the satisfaction feature.
19. The apparatus according to claim 11, wherein the extracting unit specifically performs an operation of at least one of the following:

    After the satisfaction characteristics are quantified, the quantitative values of the satisfaction characteristics are used as input parameters of the satisfaction evaluation model; or

    The input parameter of the satisfaction evaluation model is determined according to the attribute of the satisfaction feature, wherein the attribute of the satisfaction feature includes the sound frequency or the sound amplitude of the satisfaction feature; or

    The parameter corresponding to the satisfaction feature matching the keyword dictionary is used as an input parameter of the satisfaction evaluation model; or

    The parameters corresponding to the satisfaction characteristics matching the semantic model are used as input parameters of the satisfaction evaluation model.
20. The apparatus according to claim 11, wherein the determining unit specifically performs the following operations:

    Entering the input parameter of the satisfaction evaluation model into the satisfaction evaluation model, and obtaining the satisfaction of the user output by the satisfaction evaluation model;

    The satisfaction evaluation model determines the scores of each satisfaction feature according to the input parameters of the satisfaction evaluation model, so as to determine the user satisfaction according to the scores of the satisfaction features.
21. a device, including

    One or more processors;

    Memory

    One or more programs, the one or more programs being stored in the memory, when executed by the one or more processors:

    Obtain the information sent by the user;

    Extracting user satisfaction characteristics from information sent by the user to determine input parameters of the satisfaction evaluation model;

    The user's satisfaction is determined according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model.
22. A computer storage medium encoded with a computer program, when executed by one or more computers, causes the one or more computers to perform the following operations:

    Obtain the information sent by the user;

    Extracting user satisfaction characteristics from information sent by the user to determine input parameters of the satisfaction evaluation model;

    The user's satisfaction is determined according to the input parameters of the satisfaction evaluation model and the satisfaction evaluation model.

Images