CN111311309A - User satisfaction determining method, device, equipment and medium - Google Patents
User satisfaction determining method, device, equipment and medium Download PDFInfo
- Publication number
- CN111311309A CN111311309A CN202010062923.5A CN202010062923A CN111311309A CN 111311309 A CN111311309 A CN 111311309A CN 202010062923 A CN202010062923 A CN 202010062923A CN 111311309 A CN111311309 A CN 111311309A
- Authority
- CN
- China
- Prior art keywords
- target
- sample
- determining
- satisfaction
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q30/016—After-sales
Landscapes
- Business, Economics & Management (AREA)
- Finance (AREA)
- Engineering & Computer Science (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Marketing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- Machine Translation (AREA)
Abstract
The embodiment of the application discloses a method, a device, equipment and a medium for determining user satisfaction, relates to the field of data processing, and particularly relates to an intelligent interaction technology. The specific implementation scheme is as follows: acquiring current interactive text information input by a user; and inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and responding to the user based on the current satisfaction result, wherein the target sample characteristics and the target sample labels of the pre-trained user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics. The embodiment of the application provides a user satisfaction determining method, a user satisfaction determining device and a user satisfaction determining medium, so that the user satisfaction can be pre-estimated, the user can be responded based on the pre-estimated result, and the efficiency of obtaining target demand information and the user interaction experience of the user can be improved.
Description
Technical Field
The embodiment of the application relates to the field of data processing, in particular to an intelligent interaction technology. The embodiment of the application provides a user satisfaction determining method, a user satisfaction determining device, user satisfaction determining equipment and a user satisfaction determining medium.
Background
In this data explosion era, people increasingly attach importance to the efficiency of obtaining target demand information.
However, existing retrieval systems and dialog systems both have a certain proportion of unsatisfied traffic, and the unsatisfied traffic often reduces the efficiency of obtaining target demand information by users, and affects the interactive experience between users and the systems.
Disclosure of Invention
The embodiment of the application provides a user satisfaction determining method, a user satisfaction determining device and a user satisfaction determining medium, so that the user satisfaction can be pre-estimated, the user can be responded based on the pre-estimated result, and the efficiency of obtaining target demand information and the user interaction experience of the user can be improved.
The embodiment of the application provides a user satisfaction determining method, which comprises the following steps:
acquiring current interactive text information input by a user;
and inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and responding to the user based on the current satisfaction result, wherein the target sample characteristics and the target sample labels of the pre-trained user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics.
According to the method and the device, the satisfaction degree judgment rules of different types of data are learned by using the user satisfaction degree model, then the user satisfaction degree of the current interactive text information is estimated based on the learned satisfaction degree judgment rules, so that the user is responded based on the estimation result, and the efficiency of acquiring the target demand information and the interactive experience of the user are improved.
In addition, the target sample characteristics and the target sample labels for pre-training the user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics, so that the determination accuracy of the model training samples can be improved, and the learning accuracy of the satisfaction judgment rule is further improved.
Further, determining the target sample feature and the target sample label according to the historical satisfaction degree of the candidate sample feature comprises the following steps:
determining a first target threshold and a second target threshold;
comparing the historical satisfaction of the candidate sample feature to the first target threshold and the historical satisfaction of the candidate sample feature to the second target threshold, wherein the first target threshold is less than the second target threshold;
and determining the target sample characteristic and the target sample label according to the comparison result.
Based on the technical characteristics, the target sample characteristics are screened from the candidate sample characteristics through the first target threshold and the second target threshold, so that the accuracy of the training sample is improved.
Further, the determining the first target threshold and the second target threshold includes:
obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
according to the label labeling result and a sample label determined according to a first candidate threshold value and a second candidate threshold value, determining the accuracy of the first candidate threshold value and the second candidate threshold value;
based on the determined accuracy, the first target threshold is determined from the first candidate thresholds and the second target threshold is determined from the second candidate thresholds.
Based on the technical characteristics, the accuracy of the first candidate threshold value and the accuracy of the second candidate threshold value are determined by comparing the label labeling result with the target sample label determined according to the first candidate threshold value and the second candidate threshold value; based on the determined accuracy, the first target threshold is determined from the first candidate threshold, and the second target threshold is determined from the second candidate threshold, thereby improving the accuracy of determining the target threshold.
Further, the determining the target sample characteristic and the target sample label according to the comparison result includes:
if the historical satisfaction degree of the candidate sample feature is smaller than the first target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a negative sample label;
and if the historical satisfaction degree of the candidate sample feature is greater than the second target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a positive sample label.
Based on the technical characteristics, the screening of the target sample characteristics and the determination of the target sample label are realized by comparing the historical satisfaction degree of the candidate sample characteristics with the target threshold value.
Further, before determining the target sample feature and the target sample label according to the historical satisfaction of the candidate sample feature, the method further comprises:
acquiring initial sample characteristics, and performing duplicate removal on the initial sample characteristics to obtain candidate sample characteristics;
and determining the historical satisfaction degree of the candidate sample characteristics according to the historical satisfaction degree of the initial sample characteristics and the occurrence time of the initial sample characteristics.
Based on the technical characteristics, the resource waste caused by the same calculation is reduced by carrying out duplicate removal on the initial sample characteristics. And determining the historical satisfaction degree of the candidate sample characteristics according to the historical satisfaction degree of the initial sample characteristics and the occurrence time of the initial sample characteristics, so that the accuracy of determining the historical satisfaction degree of the association of the candidate sample characteristics is improved.
Further, the determining the historical satisfaction of the candidate sample feature according to the historical satisfaction of the initial sample feature and the occurrence time of the initial sample feature includes:
determining a weight of the initial sample feature based on an occurrence time of the initial sample feature;
determining historical satisfaction of the candidate sample feature based on the determined weights and the historical satisfaction of the initial sample feature.
Based on the technical characteristics, the embodiment of the application determines the weight of the initial sample characteristics based on the occurrence time of the initial sample characteristics; and then carrying out weighted summation on the historical satisfaction degrees of the initial sample characteristics based on the weight to obtain the historical satisfaction degrees of the candidate sample characteristics.
Further, before the current interactive text information is input into a pre-trained user satisfaction model and a current satisfaction result of the user is output, the method further includes:
determining a target feature representation based on the target sample features;
determining an output result of the initial model according to the target feature representation;
determining model loss according to the occurrence frequency of the target sample characteristics, the target sample label associated with the target sample characteristics and the output result of the initial model;
continuing to train the initial model based on the determined model loss to obtain the user satisfaction model.
Based on the technical characteristics, the model loss is determined based on the occurrence frequency of the target sample characteristics, so that the model is trained based on the same target sample characteristics.
Further, the determining a model loss according to the frequency of occurrence of the target sample features, the target sample label associated with the target sample features, and the output result of the initial model includes:
comparing the target sample label associated with the target sample characteristic with the output result of the initial model;
determining a single sample loss based on the comparison;
and taking the product of the single sample loss and the occurrence frequency of the target sample characteristic as the model loss.
Based on the technical characteristics, the embodiment of the application compares the target sample label associated with the target sample characteristics with the output result of the initial model; determining a single sample loss based on the comparison; and taking the product of the single sample loss and the occurrence frequency of the target sample characteristic as the model loss, thereby realizing the determination of the model loss.
Further, before determining a model loss according to the frequency of occurrence of the target sample features, the target sample label associated with the target sample features, and the output result of the initial model, the method further includes:
counting the occurrence frequency of the candidate sample characteristics;
and normalizing the statistical result to smooth the occurrence frequency of the candidate sample.
Based on the technical characteristics, the embodiment of the application realizes the smoothing of the occurrence frequency of the candidate sample by normalizing the occurrence frequency of the candidate sample characteristics.
Further, the determining a target feature representation based on the target sample feature comprises:
determining an interactive text feature representation based on historical interactive text data in the target sample feature;
determining an intention feature representation based on the intention of the historical interactive text in the target sample feature;
determining entity feature representation based on the type of the entity in the target sample feature and the value of the entity;
and splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation.
Based on the technical characteristics, the embodiment of the application determines interactive text characteristic representation, intention characteristic representation and entity characteristic representation based on the target sample characteristics; and then splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation, so that the text feature, the intention feature and the entity feature are introduced into the determination of the user satisfaction degree, and the accuracy rate of determining the user satisfaction degree is improved.
Further, after the current interactive text information is input into a pre-trained user satisfaction model and a current satisfaction result of the user is output, the method further comprises:
and if the current satisfaction result is disappointed, pausing the generation of the answer to the text information based on the current interactive text information and guiding the user.
Based on the technical characteristics, when the answer satisfaction degree of the user to the current interactive text is estimated to be disappointed, the generation of the answer to the text information based on the current interactive text information is suspended, and the user is guided, so that the efficiency of the user for acquiring the required information and the user experience are improved.
An embodiment of the present application further provides a device for determining user satisfaction, where the device includes:
the information acquisition module is used for acquiring current interactive text information input by a user;
and the result output module is used for inputting the current interactive text information into a pre-trained user satisfaction model and outputting a current satisfaction result of the user so as to respond to the user based on the current satisfaction result, wherein the target sample characteristics and the target sample labels of the user satisfaction model are pre-trained and are determined according to the historical satisfaction of the candidate sample characteristics.
Further, the apparatus further comprises:
the threshold value determining module is used for determining a first target threshold value and a second target threshold value before the current interactive text information is input into a pre-trained user satisfaction model;
a threshold comparison module for comparing the historical satisfaction of the candidate sample feature to the first target threshold and the historical satisfaction of the candidate sample feature to the second target threshold, wherein the first target threshold is less than the second target threshold;
and the label determining module is used for determining the target sample characteristics and the target sample labels according to the comparison result.
Further, the threshold determination module includes:
the result obtaining unit is used for obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
the accuracy rate determining unit is used for determining the accuracy rates of the first candidate threshold value and the second candidate threshold value according to the label labeling result and the sample label determined according to the first candidate threshold value and the second candidate threshold value;
a threshold determination unit configured to determine the first target threshold from the first candidate thresholds and determine the second target threshold from the second candidate thresholds based on the determined accuracy.
Further, the tag determination module includes:
a negative sample determining unit, configured to determine that the candidate sample feature is the target sample feature and a target sample label of the target sample feature is a negative sample label if the historical satisfaction of the candidate sample feature is less than the first target threshold;
and the positive sample determining unit is used for determining that the candidate sample feature is the target sample feature and the target sample label of the target sample feature is the positive sample label if the historical satisfaction degree of the candidate sample feature is greater than the second target threshold.
Further, the apparatus further comprises:
the sample de-duplication module is used for acquiring initial sample characteristics and carrying out de-duplication on the initial sample characteristics to obtain candidate sample characteristics before determining the target sample characteristics and the target sample label according to the historical satisfaction degree of the candidate sample characteristics;
and the satisfaction determining module is used for determining the historical satisfaction of the candidate sample characteristics according to the historical satisfaction of the initial sample characteristics and the occurrence time of the initial sample characteristics.
Further, the satisfaction determining module comprises:
a weight determination unit for determining a weight of the initial sample feature based on an occurrence time of the initial sample feature;
and the satisfaction determining unit is used for determining the historical satisfaction of the candidate sample characteristics based on the determined weight and the historical satisfaction of the initial sample characteristics.
Further, the apparatus further comprises:
a feature representation determination module for determining a target feature representation based on the target sample features;
the output result determining module is used for determining the output result of the initial model according to the target feature representation;
the model loss determining module is used for determining model loss according to the occurrence frequency of the target sample characteristics, the target sample labels associated with the target sample characteristics and the output result of the initial model;
and the model training module is used for continuing training the initial model based on the determined model loss so as to obtain the user satisfaction model.
Further, the model loss determination module includes:
the result comparison unit is used for comparing the target sample label associated with the target sample characteristic with the output result of the initial model;
a single sample loss determining unit for determining a single sample loss according to the comparison result;
a model loss determination unit, configured to use a product of the single-sample loss and the frequency of occurrence of the target sample feature as the model loss.
Further, the apparatus further comprises:
the frequency counting module is used for counting the occurrence frequency of the candidate sample characteristics before determining the loss of the model according to the occurrence frequency of the target sample characteristics, the target sample labels associated with the target sample characteristics and the output result of the initial model;
and the frequency smoothing module is used for normalizing the statistical result so as to smooth the occurrence frequency of the candidate sample.
Further, the feature representation determination module includes:
the text feature representation unit is used for determining interactive text feature representation based on historical interactive text data in the target sample features;
the intention characteristic representation unit is used for determining intention characteristic representation based on the intention of the historical interactive text in the target sample characteristic;
the entity characteristic representation unit is used for determining entity characteristic representation based on the type of the entity in the target sample characteristic and the value of the entity;
and the feature representation splicing unit is used for splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation.
Further, the apparatus further comprises:
and the user guiding module is used for inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and then suspending answer generation of the text information based on the current interactive text information and guiding the user if the current satisfaction result is disappointed.
An embodiment of the present application further provides an electronic device, where the device includes:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present application.
Embodiments of the present application also provide a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any of the embodiments of the present application.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1 is a flowchart of a user satisfaction determining method according to a first embodiment of the present application;
fig. 2 is a flowchart of a user satisfaction determining method according to a second embodiment of the present application;
fig. 3 is a flowchart of a user satisfaction determining method according to a third embodiment of the present application;
fig. 4 is a flowchart of a user satisfaction determining method according to a fourth embodiment of the present application;
fig. 5 is a flowchart of a sample generation method according to a fifth embodiment of the present application;
FIG. 6 is a schematic view of a mold structure provided in a fifth embodiment of the present application;
fig. 7 is a schematic structural diagram of a user satisfaction determining apparatus according to a sixth embodiment of the present application;
fig. 8 is a block diagram of an electronic device of a user satisfaction determination method according to an embodiment of the application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
First embodiment
Fig. 1 is a flowchart of a user satisfaction determining method according to a first embodiment of the present application. The embodiment can be applied to the interactive system, and the satisfaction degree condition of the user on the system feedback result is estimated based on the current interactive text input by the user. Optionally, the interactive system may be a search system or a man-machine interaction system. The method may be implemented by a user satisfaction determining means, which may be implemented in software and/or hardware. Referring to fig. 1, a method for determining user satisfaction provided by the embodiment of the present application includes:
and S110, acquiring the current interactive text information input by the user.
The current interactive text refers to the interactive text input by the user at the current moment.
Specifically, the interactive text information includes at least one of an interactive text, a user intention associated with the interactive text, and an entity included in the interactive text.
The interactive text may be a text to be retrieved (also called query) input by the user.
And S120, inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and responding to the user based on the current satisfaction result, wherein a target sample characteristic and a target sample label of the user satisfaction model are pre-trained and are determined according to the historical satisfaction of the candidate sample characteristic.
The current satisfaction result of the user refers to the satisfaction of the user on a target feedback result, and the target feedback result is a feedback result of the interactive system based on the current interactive text.
Specifically, the current satisfaction result of the user may be that the user is satisfied with the target feedback result, or that the user is disappointed with the target feedback result.
Optionally, the step of inputting the current interactive text information into a pre-trained user satisfaction model and outputting the current satisfaction result of the user may occur before or after generating the feedback result based on the current interactive text.
The target sample features refer to sample features used for training a user satisfaction model.
The target sample label refers to a sample label used for training a user satisfaction model.
In particular, candidate sample features may be extracted from historical interactive text.
The historical satisfaction of the candidate sample characteristics can be determined according to the operation information of the user on the historical feedback result, and the historical feedback result is the feedback result of the system aiming at the historical text input by the user.
Taking the interactive system as a retrieval system as an example, the historical feedback result is a search result retrieved by the retrieval system based on the historical retrieval text. According to the click information and the viewing time of the user on the search result, the historical satisfaction degree of the user on the search result can be determined.
Specifically, determining the target sample feature and the target sample label according to the historical satisfaction degree of the candidate sample feature comprises the following steps:
if the historical satisfaction degree of the candidate sample characteristic is larger than a set satisfaction degree threshold value, taking the candidate sample as a target sample characteristic, and determining that a target sample label associated with the target sample characteristic is a positive sample label;
and if the historical satisfaction of the candidate sample feature is smaller than a set satisfaction threshold, taking the candidate sample as the target sample feature, and determining that the target sample label associated with the target sample feature is a negative sample label.
According to the method and the device, the satisfaction degree judgment rules of different types of data are learned by using the user satisfaction degree model, then the user satisfaction degree of the current interactive text information is estimated based on the learned satisfaction degree judgment rules, so that the user is responded based on the estimation result, and the efficiency of acquiring the target demand information and the interactive experience of the user are improved.
In addition, the target sample characteristics and the target sample labels for pre-training the user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics, so that the determination accuracy of the model training samples can be improved, and the learning accuracy of the satisfaction judgment rule is further improved.
In order to improve the efficiency of obtaining the demand information and the user experience of the user, after the current interactive text information is input into a pre-trained user satisfaction model and a current satisfaction result of the user is output, the method further comprises the following steps:
and if the current satisfaction result is disappointed, pausing the generation of the answer to the text information based on the current interactive text information and guiding the user.
And continuing taking the interactive system as an example of the retrieval system, after the user inputs the text to be retrieved, if the search result of the user on the system is determined to be disappointed based on the text to be retrieved, pausing the retrieval of the retrieval system based on the text to be retrieved and prompting the user to re-input the text to be retrieved in an expression form.
Second embodiment
Fig. 2 is a flowchart of a user satisfaction determining method according to a second embodiment of the present application. The present embodiment is an alternative proposed on the basis of the above-described embodiments. Referring to fig. 2, a method for determining user satisfaction provided in the embodiment of the present application includes:
s210, determining a first target threshold value and a second target threshold value.
Wherein the first target threshold and the second target threshold are thresholds for screening target sample features from the candidate sample features.
Specifically, the first target threshold and the second target threshold may be set empirically.
Typically, the determining the first target threshold and the second target threshold includes:
obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
according to the label labeling result and a sample label determined according to a first candidate threshold value and a second candidate threshold value, determining the accuracy of the first candidate threshold value and the second candidate threshold value;
based on the determined accuracy, the first target threshold is determined from the first candidate thresholds and the second target threshold is determined from the second candidate thresholds.
Specifically, the label labeling result may be obtained by manual labeling.
According to the label labeling result and the sample label determined according to the first candidate threshold and the second candidate threshold, determining the accuracy of the first candidate threshold and the second candidate threshold, including:
taking the label labeling result as a true value, matching the label labeling result with a sample label determined according to a first candidate threshold and a second candidate threshold;
and determining the accuracy of the first candidate threshold and the second candidate threshold according to the proportion of the matched label labeling results in all the label labeling results.
S220, comparing the historical satisfaction degree of the candidate sample characteristic with the first target threshold value, and comparing the historical satisfaction degree of the candidate sample characteristic with the second target threshold value, wherein the first target threshold value is smaller than the second target threshold value.
And S230, determining the target sample characteristics and the target sample labels according to the comparison result, and training according to the target sample characteristics and the target sample labels to obtain a user satisfaction model.
Specifically, the determining the target sample feature and the target sample label according to the comparison result includes:
if the historical satisfaction degree of the candidate sample feature is smaller than the first target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a negative sample label;
and if the historical satisfaction degree of the candidate sample feature is greater than the second target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a positive sample label.
And S240, acquiring the current interactive text information input by the user.
And S250, inputting the current interactive text information into a pre-trained user satisfaction model, and outputting a current satisfaction result of the user so as to respond to the user based on the current satisfaction result.
In the embodiment of the present application, the execution sequence of the above steps is not limited, and optionally, S240 may be executed before S210.
According to the technical scheme of the embodiment of the application, the first target threshold and the second target threshold are determined, and then the target sample characteristics are screened from the candidate sample characteristics based on the determined first target threshold and the second target threshold, so that the accuracy of determining the training sample is improved.
Third embodiment
Fig. 3 is a flowchart of a user satisfaction determining method according to a third embodiment of the present application. The present embodiment is an alternative proposed on the basis of the above-described embodiments. Referring to fig. 3, a method for determining user satisfaction provided in the embodiment of the present application includes:
s310, collecting initial sample characteristics, and carrying out duplication removal on the initial sample characteristics to obtain candidate sample characteristics.
S320, determining the historical satisfaction degree of the candidate sample characteristics according to the historical satisfaction degree of the initial sample characteristics and the occurrence time of the initial sample characteristics.
Specifically, determining the historical satisfaction of the candidate sample feature according to the historical satisfaction of the initial sample feature and the occurrence time of the initial sample feature includes:
determining a weight of the initial sample feature based on an occurrence time of the initial sample feature;
determining historical satisfaction of the candidate sample feature based on the determined weights and the historical satisfaction of the initial sample feature.
S330, determining target sample characteristics and target sample labels according to the historical satisfaction of the candidate sample characteristics, and training according to the target sample characteristics and the target sample labels to obtain a user satisfaction model.
And S340, acquiring the current interactive text information input by the user.
And S350, inputting the current interactive text information into a pre-trained user satisfaction model, and outputting a current satisfaction result of the user.
In the embodiment of the present application, the execution sequence of the above steps is not limited, and optionally, S340 may be executed before S310.
According to the embodiment of the application, the original sample characteristics are subjected to duplicate removal, so that resource waste caused by the same calculation is reduced. And determining the historical satisfaction degree of the candidate sample characteristics according to the historical satisfaction degree of the initial sample characteristics and the occurrence time of the initial sample characteristics, so that the accuracy of determining the historical satisfaction degree is improved.
Fourth embodiment
Fig. 4 is a flowchart of a user satisfaction determining method according to a fourth embodiment of the present application. The present embodiment is an alternative proposed on the basis of the above-described embodiments. Referring to fig. 4, a method for determining user satisfaction provided in the embodiment of the present application includes:
and S410, determining target characteristic representation based on the target sample characteristics.
Specifically, interactive text feature representation is determined based on historical interactive text data in the target sample feature, and the determined interactive text feature representation is used as target feature representation.
In order to introduce more factors to improve the determination accuracy of the user satisfaction, the determining a target feature representation based on the target sample feature includes:
determining an interactive text feature representation based on historical interactive text data in the target sample feature;
determining an intention feature representation based on the intention of the historical interactive text in the target sample feature;
determining entity feature representation based on the type of the entity in the target sample feature and the value of the entity;
and splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation.
And S420, determining an output result of the initial model according to the target characteristic representation.
Specifically, determining an output result of the initial model according to the target feature representation includes:
encoding the target feature representation;
and processing the coding result by a pooling layer, a full-link layer and a classification layer to obtain a satisfaction degree judgment result.
S430, determining model loss according to the occurrence frequency of the target sample characteristics, the target sample labels associated with the target sample characteristics and the output result of the initial model.
Specifically, the determining a model loss according to the frequency of occurrence of the target sample features, the target sample label associated with the target sample features, and the output result of the initial model includes:
comparing the target sample label associated with the target sample characteristic with the output result of the initial model;
determining a single sample loss based on the comparison;
and taking the product of the single sample loss and the occurrence frequency of the target sample characteristic as the model loss.
S440, based on the determined model loss, continuing to train the initial model to obtain the user satisfaction model.
And S450, acquiring the current interactive text information input by the user.
And S460, inputting the current interactive text information into a pre-trained user satisfaction model, and outputting a current satisfaction result of the user.
According to the method and the device, model loss is determined based on the occurrence frequency of the target sample characteristics, so that the model is trained based on the same target sample characteristics, the accuracy of model training is improved, and resource waste caused by the same calculation is reduced.
In order to realize statistics and smoothing of the occurrence frequency of the candidate samples, before determining the model loss according to the occurrence frequency of the target sample features, the target sample labels associated with the target sample features, and the output result of the initial model, the method further includes:
counting the occurrence frequency of the candidate sample characteristics;
and normalizing the statistical result to smooth the occurrence frequency of the candidate sample.
Fifth embodiment
The present embodiment is an alternative proposed on the basis of the above-described embodiments. The user satisfaction determining method provided by the embodiment of the application comprises the following steps:
a sample data preparation stage:
referring to fig. 5, initial sample features are collected from an offline satisfaction data source and/or other feature data sources, where the initial sample features may include text-type features, and a particular text-type feature may include historical interactive text, intent of the historical interactive text, and entities included in the historical interactive text.
Optionally, the initial sample features may also include other types of features, which is not limited in this embodiment.
Typically, the initial sample features may also include numerical type features.
The initial sample features are pre-processed.
Specifically, the preprocessing of the initial sample features comprises: and performing text cleaning on the historical interactive text, performing slot cleaning on entities included in the historical interactive text, splicing intents, and performing dimensionless processing on numerical features.
And carrying out duplicate removal on the preprocessed initial sample characteristics to obtain candidate sample characteristics.
The weight of the initial sample feature is determined based on the time of occurrence of the initial sample feature.
And determining the historical satisfaction degree of the candidate sample characteristics based on the determined weight and the historical satisfaction degree of the initial sample characteristics.
And counting the occurrence frequency of the candidate sample features based on the vertical class.
And sequencing the statistical results of the occurrence frequency, and grading the sequencing results.
And multiplying the weight associated with different grades by each occurrence frequency in the sorting result to smooth the occurrence frequency of the candidate sample.
And obtaining label labeling results of the candidate sample features with a set number, wherein the set number is smaller than the total number of the candidate sample features.
And determining the accuracy of the first candidate threshold and the second candidate threshold according to the label labeling result and the target sample label determined according to the first candidate threshold and the second candidate threshold.
Based on the determined accuracy, a first target threshold is determined from the first candidate thresholds and a second target threshold is determined from the second candidate thresholds.
The historical satisfaction of the candidate sample feature is compared to a first target threshold, and the historical satisfaction of the candidate sample feature is compared to a second target threshold, wherein the first target threshold is less than the second target threshold.
And if the historical satisfaction degree of the candidate sample feature is smaller than a first target threshold value, determining that the candidate sample feature is a target sample feature, and determining that a target sample label of the target sample feature is a negative sample label.
And if the historical satisfaction degree of the candidate sample feature is greater than a second target threshold, determining that the candidate sample feature is a target sample feature, and determining that a target sample label of the target sample feature is a positive sample label.
And determining a training sample set and a testing sample set from the target sample characteristics based on the set proportion.
A model training stage:
referring to fig. 6, interactive text feature representations are determined based on historical interactive text data in the target sample features.
An intent feature representation is determined based on the intent of the historical interactive text in the target sample feature.
And determining the entity characteristic representation based on the type of the entity in the target sample characteristic and the value of the entity.
And splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain a target sample feature representation.
The target feature representation is encoded.
And processing the coding result by a pooling layer, a full-link layer and a classification layer to obtain a satisfaction judgment result.
And comparing the target sample label associated with the target sample characteristic with the output result of the initial model.
And determining the single sample loss according to the comparison result.
And taking the product of the single sample loss and the occurrence frequency of the target sample characteristic as the model loss.
And based on the determined model loss, continuing training the initial model to obtain a user satisfaction model.
And (3) a model application stage:
and inputting the acquired current interactive text information into a pre-trained satisfaction judging model.
And if the output result of the model is unsatisfactory, guiding the user to improve the user experience.
The beneficial effect of this scheme lies in: the satisfaction degree judgment rule of different types of data can be learned through the model so as to improve the generalization capability. Because the model learns satisfaction judging rules of different data types, the updating frequency of the model is lower than that based on a word list, wherein the word list comprises historical interactive text information and historical satisfaction results. And the storage space occupied by the model is smaller than the storage space occupied by the word list.
Sixth embodiment
Fig. 7 is a schematic structural diagram of a user satisfaction determining apparatus according to a sixth embodiment of the present application. Referring to fig. 7, a user satisfaction determining apparatus 700 provided in an embodiment of the present application includes: an information acquisition module 701 and a result output module 702.
The information acquisition module 701 is used for acquiring current interactive text information input by a user;
a result output module 702, configured to input the current interactive text information into a pre-trained user satisfaction model, and output a current satisfaction result of the user to respond to the user based on the current satisfaction result, where a target sample feature and a target sample label of the pre-trained user satisfaction model are determined according to historical satisfaction of candidate sample features.
According to the method and the device, the satisfaction degree judgment rules of different types of data are learned by using the user satisfaction degree model, then the user satisfaction degree of the current interactive text information is estimated based on the learned satisfaction degree judgment rules, so that the user is responded based on the estimation result, and the efficiency of acquiring the target demand information and the interactive experience of the user are improved.
In addition, the target sample characteristics and the target sample labels for pre-training the user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics, so that the determination accuracy of the model training samples can be improved, and the learning accuracy of the satisfaction judgment rule is further improved.
Further, the apparatus further comprises:
the threshold value determining module is used for determining a first target threshold value and a second target threshold value before the current interactive text information is input into a pre-trained user satisfaction model;
a threshold comparison module for comparing the historical satisfaction of the candidate sample feature to the first target threshold and the historical satisfaction of the candidate sample feature to the second target threshold, wherein the first target threshold is less than the second target threshold;
and the label determining module is used for determining the target sample characteristics and the target sample labels according to the comparison result.
Further, the threshold determination module includes:
the result obtaining unit is used for obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
the accuracy rate determining unit is used for determining the accuracy rates of the first candidate threshold value and the second candidate threshold value according to the label labeling result and the sample label determined according to the first candidate threshold value and the second candidate threshold value;
a threshold determination unit configured to determine the first target threshold from the first candidate thresholds and determine the second target threshold from the second candidate thresholds based on the determined accuracy.
Further, the tag determination module includes:
a negative sample determining unit, configured to determine that the candidate sample feature is the target sample feature and a target sample label of the target sample feature is a negative sample label if the historical satisfaction of the candidate sample feature is less than the first target threshold;
and the positive sample determining unit is used for determining that the candidate sample feature is the target sample feature and the target sample label of the target sample feature is the positive sample label if the historical satisfaction degree of the candidate sample feature is greater than the second target threshold.
Further, the apparatus further comprises:
the sample de-duplication module is used for acquiring initial sample characteristics and carrying out de-duplication on the initial sample characteristics to obtain candidate sample characteristics before determining the target sample characteristics and the target sample label according to the historical satisfaction degree of the candidate sample characteristics;
and the satisfaction determining module is used for determining the historical satisfaction of the candidate sample characteristics according to the historical satisfaction of the initial sample characteristics and the occurrence time of the initial sample characteristics.
Further, the satisfaction determining module comprises:
a weight determination unit for determining a weight of the initial sample feature based on an occurrence time of the initial sample feature;
and the satisfaction determining unit is used for determining the historical satisfaction of the candidate sample characteristics based on the determined weight and the historical satisfaction of the initial sample characteristics.
Further, the apparatus further comprises:
a feature representation determination module for determining a target feature representation based on the target sample features;
the output result determining module is used for determining the output result of the initial model according to the target feature representation;
the model loss determining module is used for determining model loss according to the occurrence frequency of the target sample characteristics, the target sample labels associated with the target sample characteristics and the output result of the initial model;
and the model training module is used for continuing training the initial model based on the determined model loss so as to obtain the user satisfaction model.
Further, the model loss determination module includes:
the result comparison unit is used for comparing the target sample label associated with the target sample characteristic with the output result of the initial model;
a single sample loss determining unit for determining a single sample loss according to the comparison result;
a model loss determination unit, configured to use a product of the single-sample loss and the frequency of occurrence of the target sample feature as the model loss.
Further, the apparatus further comprises:
the frequency counting module is used for counting the occurrence frequency of the candidate sample characteristics before determining the loss of the model according to the occurrence frequency of the target sample characteristics, the target sample labels associated with the target sample characteristics and the output result of the initial model;
and the frequency smoothing module is used for normalizing the statistical result so as to smooth the occurrence frequency of the candidate sample.
Further, the feature representation determination module includes:
the text feature representation unit is used for determining interactive text feature representation based on historical interactive text data in the target sample features;
the intention characteristic representation unit is used for determining intention characteristic representation based on the intention of the historical interactive text in the target sample characteristic;
the entity characteristic representation unit is used for determining entity characteristic representation based on the type of the entity in the target sample characteristic and the value of the entity;
and the feature representation splicing unit is used for splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation.
Further, the apparatus further comprises:
and the user guiding module is used for inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and then suspending answer generation of the text information based on the current interactive text information and guiding the user if the current satisfaction result is disappointed.
Seventh embodiment
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 8 is a block diagram of an electronic device according to the user satisfaction determining method according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 8, the electronic apparatus includes: one or more processors 801, memory 802, and interfaces for connecting the various components, including a high speed interface and a low speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 8 illustrates an example of a processor 801.
The memory 802 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the user satisfaction determination methods provided herein. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to perform the user satisfaction determination method provided by the present application.
The memory 802, as a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (e.g., the information acquisition module 701 and the result output module 702 shown in fig. 7) corresponding to the user satisfaction determination method in the embodiments of the present application. The processor 801 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 802, that is, implements the user satisfaction determination method in the above-described method embodiment.
The memory 802 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by determining use of the electronic device according to user satisfaction, and the like. Further, the memory 802 may include high speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 802 optionally includes memory located remotely from the processor 801, which may be connected to the user satisfaction determining electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, blockchain networks, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the user satisfaction determination method may further include: an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803, and the output device 804 may be connected by a bus or other means, and are exemplified by a bus in fig. 8.
The input device 803 may receive input numeric or character information and generate key signal inputs related to user satisfaction determination of user settings and function control of the electronic device, such as a touch screen, keypad, mouse, track pad, touch pad, pointer stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 804 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (16)
1. A user satisfaction determination method, comprising:
acquiring current interactive text information input by a user;
and inputting the current interactive text information into a pre-trained user satisfaction model, outputting a current satisfaction result of the user, and responding to the user based on the current satisfaction result, wherein the target sample characteristics and the target sample labels of the pre-trained user satisfaction model are determined according to the historical satisfaction of the candidate sample characteristics.
2. The method of claim 1, wherein determining the target sample feature and the target sample label based on historical satisfaction of the candidate sample feature comprises:
determining a first target threshold and a second target threshold;
comparing the historical satisfaction of the candidate sample feature to the first target threshold and the historical satisfaction of the candidate sample feature to the second target threshold, wherein the first target threshold is less than the second target threshold;
and determining the target sample characteristic and the target sample label according to the comparison result.
3. The method of claim 2, wherein determining the first target threshold and the second target threshold comprises:
obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
according to the label labeling result and a target sample label determined according to a first candidate threshold value and a second candidate threshold value, determining the accuracy of the first candidate threshold value and the second candidate threshold value;
based on the determined accuracy, the first target threshold is determined from the first candidate thresholds and the second target threshold is determined from the second candidate thresholds.
4. The method of claim 2, wherein determining the target sample characteristic and the target sample label based on the comparison comprises:
if the historical satisfaction degree of the candidate sample feature is smaller than the first target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a negative sample label;
and if the historical satisfaction degree of the candidate sample feature is greater than the second target threshold, determining that the candidate sample feature is the target sample feature, and the target sample label of the target sample feature is a positive sample label.
5. The method of claim 1, wherein prior to determining the target sample feature and the target sample label based on historical satisfaction of the candidate sample feature, the method further comprises:
acquiring initial sample characteristics, and performing duplicate removal on the initial sample characteristics to obtain candidate sample characteristics;
and determining the historical satisfaction degree of the candidate sample characteristics according to the historical satisfaction degree of the initial sample characteristics and the occurrence time of the initial sample characteristics.
6. The method of claim 5, wherein determining the historical satisfaction of the candidate sample feature based on the historical satisfaction of the initial sample feature and the time of occurrence of the initial sample feature comprises:
determining a weight of the initial sample feature based on an occurrence time of the initial sample feature;
determining historical satisfaction of the candidate sample feature based on the determined weights and the historical satisfaction of the initial sample feature.
7. The method of claim 1, wherein before inputting the current interactive text information into a pre-trained user satisfaction model and outputting a current user satisfaction result, the method further comprises:
determining a target feature representation based on the target sample features;
determining an output result of the initial model according to the target feature representation;
determining model loss according to the occurrence frequency of the target sample characteristics, the target sample label associated with the target sample characteristics and the output result of the initial model;
continuing to train the initial model based on the determined model loss to obtain the user satisfaction model.
8. The method of claim 7, wherein determining a model loss according to the frequency of occurrence of the target sample features, the target sample labels associated with the target sample features, and the output of the initial model comprises:
comparing the target sample label associated with the target sample characteristic with the output result of the initial model;
determining a single sample loss based on the comparison;
and taking the product of the single sample loss and the occurrence frequency of the target sample characteristic as the model loss.
9. The method of claim 7, wherein before determining model loss based on the frequency of occurrence of the target sample features, the target sample labels associated with the target sample features, and the output of the initial model, the method further comprises:
counting the occurrence frequency of the candidate sample characteristics;
and normalizing the statistical result to smooth the occurrence frequency of the candidate sample.
10. The method of claim 7, wherein determining a target feature representation based on the target sample features comprises:
determining an interactive text feature representation based on historical interactive text data in the target sample feature;
determining an intention feature representation based on the intention of the historical interactive text in the target sample feature;
determining entity feature representation based on the type of the entity in the target sample feature and the value of the entity;
and splicing the interactive text feature representation, the intention feature representation and the entity feature representation to obtain the target sample feature representation.
11. The method of claim 1, wherein after inputting the current interactive text information into a pre-trained user satisfaction model and outputting a current user satisfaction result, the method further comprises:
and if the current satisfaction result is disappointed, pausing the generation of the answer to the text information based on the current interactive text information and guiding the user.
12. A user satisfaction determination apparatus, comprising:
the information acquisition module is used for acquiring current interactive text information input by a user;
and the result output module is used for inputting the current interactive text information into a pre-trained user satisfaction model and outputting a current satisfaction result of the user so as to respond to the user based on the current satisfaction result, wherein the target sample characteristics and the target sample labels of the user satisfaction model are pre-trained and are determined according to the historical satisfaction of the candidate sample characteristics.
13. The apparatus of claim 12, further comprising:
the threshold value determining module is used for determining a first target threshold value and a second target threshold value before the current interactive text information is input into a pre-trained user satisfaction model;
a threshold comparison module for comparing the historical satisfaction of the candidate sample feature to the first target threshold and the historical satisfaction of the candidate sample feature to the second target threshold, wherein the first target threshold is less than the second target threshold;
and the label determining module is used for determining the target sample characteristics and the target sample labels according to the comparison result.
14. The apparatus of claim 13, wherein the threshold determination module comprises:
the result obtaining unit is used for obtaining label labeling results of a set number of candidate sample features, wherein the set number is smaller than the total number of the candidate sample features;
the accuracy rate determining unit is used for determining the accuracy rates of the first candidate threshold value and the second candidate threshold value according to the label labeling result and the sample label determined according to the first candidate threshold value and the second candidate threshold value;
a threshold determination unit configured to determine the first target threshold from the first candidate thresholds and determine the second target threshold from the second candidate thresholds based on the determined accuracy.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010062923.5A CN111311309A (en) | 2020-01-19 | 2020-01-19 | User satisfaction determining method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010062923.5A CN111311309A (en) | 2020-01-19 | 2020-01-19 | User satisfaction determining method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111311309A true CN111311309A (en) | 2020-06-19 |
Family
ID=71158226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010062923.5A Pending CN111311309A (en) | 2020-01-19 | 2020-01-19 | User satisfaction determining method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311309A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114880057A (en) * | 2022-04-22 | 2022-08-09 | 北京三快在线科技有限公司 | Image display method, image display device, terminal, server, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090306995A1 (en) * | 2008-06-04 | 2009-12-10 | Robert Bosch Gmbh | System and Method for Automated Testing of Complicated Dialog Systems |
| CN103020289A (en) * | 2012-12-25 | 2013-04-03 | 浙江鸿程计算机系统有限公司 | Method for providing individual needs of search engine user based on log mining |
| CN104573312A (en) * | 2014-10-22 | 2015-04-29 | 浙江中烟工业有限责任公司 | Log-based mobile application user satisfaction evaluation method |
| CN104680428A (en) * | 2015-03-16 | 2015-06-03 | 朗新科技股份有限公司 | Construction method of power grid customer satisfaction model |
| CN107220353A (en) * | 2017-06-01 | 2017-09-29 | 深圳追科技有限公司 | A kind of intelligent customer service robot satisfaction automatic evaluation method and system |
| CN108960110A (en) * | 2018-06-26 | 2018-12-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating information |
| CN108960316A (en) * | 2018-06-27 | 2018-12-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating model |
| CN109816221A (en) * | 2019-01-07 | 2019-05-28 | 平安科技（深圳）有限公司 | Decision of Project Risk method, apparatus, computer equipment and storage medium |
-
2020
- 2020-01-19 CN CN202010062923.5A patent/CN111311309A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090306995A1 (en) * | 2008-06-04 | 2009-12-10 | Robert Bosch Gmbh | System and Method for Automated Testing of Complicated Dialog Systems |
| CN103020289A (en) * | 2012-12-25 | 2013-04-03 | 浙江鸿程计算机系统有限公司 | Method for providing individual needs of search engine user based on log mining |
| CN104573312A (en) * | 2014-10-22 | 2015-04-29 | 浙江中烟工业有限责任公司 | Log-based mobile application user satisfaction evaluation method |
| CN104680428A (en) * | 2015-03-16 | 2015-06-03 | 朗新科技股份有限公司 | Construction method of power grid customer satisfaction model |
| CN107220353A (en) * | 2017-06-01 | 2017-09-29 | 深圳追科技有限公司 | A kind of intelligent customer service robot satisfaction automatic evaluation method and system |
| CN108960110A (en) * | 2018-06-26 | 2018-12-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating information |
| CN108960316A (en) * | 2018-06-27 | 2018-12-07 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating model |
| CN109816221A (en) * | 2019-01-07 | 2019-05-28 | 平安科技（深圳）有限公司 | Decision of Project Risk method, apparatus, computer equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114880057A (en) * | 2022-04-22 | 2022-08-09 | 北京三快在线科技有限公司 | Image display method, image display device, terminal, server, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112560912B (en) | Classification model training method and device, electronic equipment and storage medium | |
| CN111709247B (en) | Data set processing method and device, electronic equipment and storage medium | |
| CN111507104B (en) | Method and device for establishing label labeling model, electronic equipment and readable storage medium | |
| CN111078865B (en) | Text title generation method and device | |
| CN111753914A (en) | Model optimization method and device, electronic equipment and storage medium | |
| US20220067439A1 (en) | Entity linking method, electronic device and storage medium | |
| CN111522967A (en) | Knowledge graph construction method, device, equipment and storage medium | |
| CN112148881B (en) | Method and device for outputting information | |
| CN112241452A (en) | Model training method and device, electronic equipment and storage medium | |
| CN111797216B (en) | Search term rewriting method, apparatus, device and storage medium | |
| CN111078878B (en) | Text processing method, device, equipment and computer readable storage medium | |
| CN111859953B (en) | Training data mining method and device, electronic equipment and storage medium | |
| CN111667056A (en) | Method and apparatus for searching model structure | |
| CN111680517A (en) | Method, apparatus, device and storage medium for training a model | |
| CN111259671A (en) | Semantic description processing method, device and equipment for text entity | |
| CN111832613B (en) | Model training method and device, electronic equipment and storage medium | |
| CN113553412B (en) | Question-answering processing method, question-answering processing device, electronic equipment and storage medium | |
| CN112507090A (en) | Method, apparatus, device and storage medium for outputting information | |
| CN111522944A (en) | Method, apparatus, device and storage medium for outputting information | |
| CN110543558A (en) | question matching method, device, equipment and medium | |
| CN113342946B (en) | Model training method and device for customer service robot, electronic equipment and medium | |
| CN112232089B (en) | Pre-training method, device and storage medium of semantic representation model | |
| CN111325000B (en) | Language generation method and device and electronic equipment | |
| CN111311309A (en) | User satisfaction determining method, device, equipment and medium | |
| CN111125445A (en) | Community theme generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210514 Address after: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant after: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. Applicant after: Shanghai Xiaodu Technology Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200619 |
|
| RJ01 | Rejection of invention patent application after publication |