CN115374253A - Statistical method and device for multiple rounds of conversations, electronic equipment and computer storage medium - Google Patents

Abstract

The disclosure provides a statistical method, a statistical device, an electronic device and a storage medium for multiple rounds of conversations, wherein the method comprises the following steps: predicting dialogue data between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object; performing keyword statistics on the plurality of prediction sequences to obtain a statistical result; wherein the statistical result is at least used for determining the conversation theme of the conversation data, the problems related to the conversation, the conversation satisfaction degree of any one of the two parties of the conversation and the intention of any one of the two parties of the conversation; even for a data set with a small data sample size, a good statistical effect can be obtained.

Description

Statistical method and device for multiple rounds of conversations, electronic equipment and computer storage medium

Technical Field

The present disclosure relates to the field of information technologies, and in particular, to a statistical method and apparatus for multiple sessions, an electronic device, and a computer storage medium.

Background

In the existing prediction of data training of multiple rounds of conversations, a large number of data sets are often needed in order to achieve a commercial degree and obtain stable and good test indexes, but satisfactory test indexes cannot be obtained for medium and small data sets.

Therefore, there is a need for a device that can process a large number of data sets and can also obtain better statistical effect for multiple sessions of small and medium data sets.

Disclosure of Invention

The embodiment of the disclosure provides a statistical method and device for multiple rounds of conversations, electronic equipment and a computer storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a statistical method for multiple rounds of conversations, the method including:

predicting dialogue data between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

performing keyword statistics on the plurality of prediction sequences to obtain a statistical result; wherein the statistical result is used for determining at least a conversation topic of the conversation data, a question related to the conversation, a conversation satisfaction degree of any one of the two parties of the conversation, and an intention of any one of the two parties of the conversation.

Optionally, the predicting dialog data between the first object and the second object by using the machine learning model to obtain a plurality of prediction sequences includes:

predicting a first dialogue data set corresponding to the first object by using a first model in the machine learning model to obtain a first prediction sequence; the first model is obtained by training according to a dialogue data set of the same object as the first class of objects and/or historical dialogue data of the first object;

predicting a second dialogue data set corresponding to the second object by using a second model in the machine learning model to obtain a second prediction sequence; and the second model is obtained by training according to the dialogue data set of the same object as the second class of object and/or the historical dialogue data of the second object.

Optionally, the performing keyword statistics on the plurality of prediction sequences to obtain a statistical result includes:

performing keyword statistics on the first prediction sequence to obtain a first statistical result;

performing keyword statistics on the second prediction sequence to obtain a second statistical result;

and obtaining a statistical result used for determining the conversation topic according to the product of the first statistical result and the first weight and the product of the second statistical result and the second weight.

Optionally, before predicting the dialog between the first object and the second object by using the machine learning model to obtain a plurality of prediction sequences, the method further includes:

collecting dialogues in the dialog data, wherein a speaker is the first object, so as to obtain a first dialog data set;

and collecting the dialogue of which the speaker is the second object in the dialogue data to obtain the second dialogue data set.

Optionally, the method further comprises:

removing the nth keyword in the predicted sequence, wherein the correlation degree of the nth keyword and the mth keyword is lower than a first threshold value; the m-th keyword is any keyword except the n-th keyword in the prediction sequence;

the performing keyword statistics on the plurality of prediction sequences to obtain a statistical result includes:

and carrying out keyword statistics on the plurality of prediction sequences without the nth keyword to obtain the statistical result.

Optionally, before predicting the dialog between the first object and the second object by using the machine learning model to obtain a plurality of prediction sequences, the method further includes:

segmenting a historical dialog between the first object and the second object into a first set of historical dialog data corresponding to the first object and a second set of historical dialog data corresponding to the second object;

carrying out classification training and integrity training in machine learning on the first historical dialogue data set to obtain a first model;

performing classification training in machine learning on the second historical dialogue data set to obtain a second model; wherein the dialogue in predicting a dialogue between a first object and a second object using a machine learning model occurs after the historical dialogue.

Optionally, the statistical result is specifically used to determine at least one of:

the type of product;

problems before the product is sold;

product after-market problems.

Optionally, the statistical result is further used to determine at least one of:

a degree of satisfaction of the first object with the response of the second object;

an intent of the first object to purchase the product;

the first object has a tendency to evaluate the product.

A second aspect of the embodiments of the present disclosure provides a statistical apparatus for multiple rounds of conversations, the apparatus including:

the prediction module is used for predicting dialogue data between the first object and the second object by utilizing the machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

the statistic module is used for carrying out keyword statistics on the plurality of prediction sequences to obtain a statistic result; wherein the statistical result is used for determining at least a topic of the conversation data, a question related to the conversation, a degree of satisfaction of the conversation of either one of the two parties of the conversation, and an intention of either one of the two parties of the conversation.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory;

and the processor is connected with the memory and used for realizing the steps in the statistical method of the multi-turn conversations provided by the first aspect through the computer execution instructions stored in the memory.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer storage medium having executable instructions stored therein; the computer-executable instructions, when executed by the processor, enable the steps of the statistical method for multiple sessions provided in the first aspect.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

compared with the prior art that the machine learning model is used for predicting the conversation data before the first object and the second object, the method and the device have the advantages that the input of the first object and the second object in multiple rounds of conversations is considered simultaneously, and more comprehensive and accurate statistical results are obtained; meanwhile, the machine learning model is used for predicting dialogue data between the first object and the second object to obtain a plurality of prediction sequences, then keyword statistics is carried out on the prediction sequences to obtain statistical results, and the machine learning and statistical modes are integrated.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

FIG. 1 is a flow diagram illustrating a statistical method for multiple sessions in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a statistical method for multiple sessions in accordance with an exemplary embodiment;

FIG. 3 is a flow diagram illustrating a statistical method for multiple sessions in accordance with an exemplary embodiment;

FIG. 4 is a flow diagram illustrating a statistical method for multiple sessions in accordance with an exemplary embodiment;

FIG. 5 is a flow diagram illustrating a statistical method for multiple sessions in accordance with an exemplary embodiment;

FIG. 6 is a diagram illustrating an exemplary embodiment of a statistical apparatus for multiple sessions;

FIG. 7 is a diagram illustrating an exemplary embodiment of a statistical apparatus for multiple sessions;

FIG. 8 is a diagram illustrating a statistical apparatus for multiple sessions in accordance with an exemplary embodiment;

FIG. 9 is a schematic diagram of a statistical apparatus for multiple sessions in accordance with an exemplary embodiment;

FIG. 10 is a schematic diagram of a statistical apparatus for multiple sessions, according to an exemplary embodiment;

fig. 11 is a schematic diagram of a statistical apparatus for multiple sessions according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with embodiments of the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the disclosed embodiments, as detailed in the appended claims.

As shown in fig. 1, an embodiment of the present disclosure provides a statistical method for multiple rounds of conversations, the method including:

step S110, predicting dialogue data between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

step S120, carrying out keyword statistics on the plurality of prediction sequences to obtain a statistical result; wherein the statistical result is used for determining at least a topic of the conversation data, a question related to the conversation, a degree of satisfaction of the conversation of either one of the two parties of the conversation, and an intention of either one of the two parties of the conversation.

In the embodiment of the present disclosure, the multiple rounds of dialogs may occur between two languages and/or voice input parties, or may occur between more than three languages and/or voice input parties.

In some embodiments, on the premise that the two language and/or voice input parties in the multiple rounds of conversations are ensured, the situation that an input party newly joining the conversation replaces an input party exiting the conversation may also occur, for example, when the robot customer service cannot meet the requirements of the user, the robot is exited, and the robot customer service is replaced manually to answer the user questions.

In the embodiment of the disclosure, the first object and the second object are different dialog input parties, the first object may be user input, and the second object may be human service or robot service input. In some embodiments, the second object may be replaced with a human customer service when the second object is a robot customer service whose answer fails to solve the question of the first object.

In some embodiments, the keywords may be: and (5) the words with the largest information entropy in the dialogue data.

In other embodiments, the keywords may be: verbs and/or nouns.

In the embodiment of the present disclosure, the machine learning model may be a training model obtained by training a dialogue data set by a supervised learning model, and for example, the machine learning model may include, but is not limited to: the linear classifier is a training model obtained by training the dialogue data set by any classification model such as LR (logistic regression), support vector machine SVM (support vector machines), deep Neural network DNN (deep Neural Networks), and the like.

In the embodiment of the disclosure, the machine learning model includes an integrity detection model and a classification model, the integrity detection model is used for detecting the integrity of the keyword, and the classification model is used for classifying the keyword or a single sentence where the keyword is located. In some embodiments, the keywords or the sentences in which the keywords are located are classified, including but not limited to: and classifying the content of the single sentence, and classifying according to the content of the single sentence to obtain the theme related to the single sentence and the emotion related to the single sentence. And classifying subjects involved in the single sentence and/or classifying emotions involved in the single sentence.

In the embodiment of the present disclosure, the keyword or the topic type related in the single sentence where the keyword is located includes but is not limited to: product type, product pre-sale issues, and product post-sale issues. The emotions referred to in the sentence include, but are not limited to, positive emotions or negative emotions. For example, the emotion of a single sentence related to a rating may be classified, a rating including a positive emotion may be classified as a positive rating, and a rating including a negative emotion may be classified as a negative rating.

In the embodiment of the present disclosure, a natural language vectorization technology, a one-hot or word vector technology, or the like may be adopted to perform preprocessing and vectorization on the dialogue data between the first object and the second object to obtain a K-dimensional/H-dimensional vector.

In the embodiment of the present disclosure, the keyword is related to a topic of the dialog in the dialog data, a question related to the dialog, a degree of satisfaction of one of both sides of the dialog, and an intention of one of both sides of the dialog, so that the statistical result obtained by performing the keyword statistics on the plurality of prediction sequences has high accuracy.

In the embodiment of the present disclosure, the dialog theme of the dialog data may be a plurality of relatively discrete themes with different categories, for example, a product belongs to one theme, and a question belongs to another theme. The products can comprise electronic products, living goods, foods, clothes and other articles. The questions may include: bad, damaged, overdue, slow logistics and the like. The discrete product themes, plus the problem themes, can be combined into a final composite theme. In one embodiment, the product may be a "computer" and the product problem may be "bad", both of which belong to discrete topics, but these two discrete topics may be combined together to get a composite topic such as: "the computer is bad".

In the embodiment of the disclosure, the manner of performing keyword statistics on the plurality of prediction sequences to obtain the statistical result fuses the dialog input by the first object and the second object, and does not waste the dialog content input by any object, so that the accuracy of the statistical result can be improved.

In the embodiment of the present disclosure, the way of performing keyword statistics on the plurality of prediction sequences includes count statistics and probability statistics, the count statistics and the probability statistics of the plurality of prediction sequences are fused according to weights, and then the count statistics and the probability statistics after the plurality of prediction sequences are fused according to preset parameters, so that compared with the case that a more accurate result cannot be obtained due to under-fitting or over-fitting possibly caused by machine learning on a small sample data set, the accuracy of the statistics obtained by multiple statistics and multiple fusion is higher.

In the embodiment of the disclosure, on the basis of machine learning, a statistical result is obtained by combining a statistical mode, and machine learning and statistical processing can be performed on medium and small data sets while a large number of data sets can be compatible, so that a more accurate statistical result is obtained.

As shown in fig. 2, in the statistical method for multiple rounds of conversations in the embodiment of the present disclosure, the step S110 of predicting conversation data between a first object and a second object by using a machine learning model to obtain multiple prediction sequences includes:

step S1101, predicting a first dialog data set corresponding to the first object by using a first model in the machine learning models to obtain a first prediction sequence; the first model is obtained by training according to a dialogue data set of the same object as the first class of objects and/or historical dialogue data of the first object;

step S1102, predicting a second dialogue data set corresponding to the second object by using a second model in the machine learning model to obtain a second prediction sequence; and the second model is obtained by training according to the dialogue data set of the same object as the second class of object and/or the historical dialogue data of the second object.

In the embodiment of the present disclosure, in a case that the first object is a user, the first model includes a first classification model and an integrity detection model, and the integrity model is configured to detect integrity of a single sentence in the first dialog data set; the first classification model is used for classifying the single sentences in the first dialogue data set. In some embodiments, the single sentences in the first dialog dataset are classified, including but not limited to: and classifying the content of the single sentence, and classifying according to the content of the single sentence to obtain the theme related to the single sentence and the emotion related to the single sentence. And classifying the subjects involved in the single sentence and/or classifying the emotions involved in the single sentence.

In some embodiments, the topic types involved in the sentence include, but are not limited to: product type, product pre-sale issues, and product post-sale issues. The emotions referred to in the sentence include, but are not limited to, positive emotions or negative emotions. For example, the emotion of a single sentence related to a rating may be classified, a rating including a positive emotion may be classified as a positive rating, and a rating including a negative emotion may be classified as a negative rating. In an embodiment of the disclosure, in case the second object is a skilled robot customer service, the second model comprises a classification model for classifying the single sentences in the second dialogue data set.

In an embodiment of the present disclosure, in a case that the second object is an artificial customer service, the second model may also include: a completeness detection model and a classification model.

In the embodiment of the present disclosure, when the second object is a robot customer service, but the robot customer service is not skilled enough at this time, and a single sentence may also be incomplete, the second model may also include: a completeness detection model and a classification model.

In the embodiment of the disclosure, the integrity detection model is used for detecting whether a single sentence is complete, and if the single sentence is complete, the single sentence where the mark is located is complete; if the single sentence is incomplete, the single sentence is marked as incomplete. In one embodiment, the flag is 1 for complete sentences and 0 for incomplete sentences.

In the embodiment of the present disclosure, for an incomplete single sentence, merging processing is performed. For example, incomplete sentences are merged according to the category of the single sentence and the continuity of the context. For example, in some embodiments, the single sentence entered in the previous sentence relates to a product, the sentence is incomplete; the problem of the product related to the single sentence input in the next sentence is that the sentence is incomplete, and the previous sentence and the next sentence are input continuously, so that the incomplete single sentence in the previous sentence and the incomplete single sentence in the next sentence can be merged.

In the embodiment of the present disclosure, the dialog data sets of the same object in the first class are dialog data sets input by different users belonging to the same type. And the historical dialogue data of the first object is historical dialogue data belonging to the same user. The historical dialogue data refers to: dialog data for a dialog that occurred before the current time. Thus, the first model combining different user data and historical dialogue data of the same user can be obtained, so that the utilization rate of the data set is higher, and the accuracy of the final result can be improved.

In the embodiment of the present disclosure, the dialog data sets of the same object in the second class are dialog data sets input by different customer services (e.g., robot customer service and manual customer service) belonging to the same type. And the historical conversation data of the second object belongs to the same customer service, namely a conversation data set input by the same robot customer service or the same manual customer service. Thus, a second model combining different customer service data and historical dialogue data of the same customer service can be obtained, so that the utilization rate of the data set is higher, and the accuracy of the final result can be improved.

In the embodiment of the present disclosure, a first dialog data set corresponding to a first object is predicted to obtain a first prediction sequence, or a second dialog data set corresponding to a second object is predicted to obtain a second prediction sequence, and dialog contents input by the first object and the second object are separated first, and are preprocessed and vectorized, which is beneficial to improving processing efficiency.

As shown in fig. 3, in the statistical method for multiple sessions in the embodiment of the present disclosure, in step S120, performing keyword statistics on the multiple prediction sequences to obtain a statistical result, including:

step S1201, performing keyword statistics on the first prediction sequence to obtain a first statistical result;

step S1202, performing keyword statistics on the second prediction sequence to obtain a second statistical result;

step S1203, obtaining a statistical result for determining the dialog topic according to a product of the first statistical result and the first weight, and a product of the second statistical result and the second weight.

In an embodiment of the present disclosure, the performing keyword statistics on the first prediction sequence includes: and counting the topics reflected by the former N keywords with higher frequency in the first prediction sequence and carrying out probability statistics, and accumulating the counting statistics for normalization and accumulating the probability statistics for normalization to obtain a first counting normalization value L1 and a first probability normalization value PL1. The first statistical result includes: a first degree-of-valuation value L1 and a first probability-of-valuation value PL1.

In the embodiment of the disclosure, the keywords themselves can reflect the topic of the conversation, and the keyword combination can also reflect the topic of the conversation.

Similarly, in this embodiment of the present disclosure, the performing keyword statistics on the second prediction sequence includes: and performing counting statistics and probability statistics on the subjects of the former N keywords with higher frequency in the second prediction sequence, and accumulating the counting statistics for normalization and accumulating the probability statistics for normalization to obtain a second counting normalization value L2 and a second probability normalization value PL2. The second statistical result comprises: a second degree-of-enumeration normalization value L2 and a second probability normalization value PL2.

In the embodiment of the present disclosure, a count statistical result TL1 and a probability statistical result TL2 are obtained according to a product of the first statistical result and the first weight w, and a product of the second statistical result and the second weight 1-w, and a statistical result for determining the conversation topic is obtained according to a product of the first preset parameter w1 and the count statistical result TL1, and a product of the second preset parameter 1-w1 and the probability statistical result TL 2. Wherein the value range of the first weight w and the second weight 1-w is 0 to 1.w =1 represents a statistic that fully values the first object, and w =0 represents a statistic that fully values customer service.

In the embodiment of the present disclosure, the value range of the preset parameter w1 is 0 to 1, w1=1 indicates a thorough deviation counting result, and w1=0 indicates a thorough deviation probability statistical result.

In an embodiment of the present disclosure, the first weight w is obtained by using optimized gaussian fusion according to the mean value and variance of the distribution indexes of the correlation coefficient series group of the first prediction sequence and the second prediction sequence, and the number of effective rounds of dialogue (1/n, o1, n), (1/m, o2, m): the second prediction sequence weight coefficient w = sqrt (n × o 1)/(sqrt (n × o 1) + sqrt (m × o 2)), where sqrt is the meaning of the square root.

In the embodiment of the disclosure, the probability cumulative statistics and count cumulative statistics methods, and the statistical method for fusing the probability cumulative statistics and count cumulative statistics of the first object and the second object, can adapt to various scale data sets, especially medium and small scale data sets. And more accurate statistical results can be obtained.

As shown in fig. 4, before performing step S110, predicting a dialog between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences, a statistical method for a plurality of rounds of dialog in the embodiment of the present disclosure further includes:

step S104, collecting the dialogue of which the speaker is the first object in the dialogue data to obtain the first dialogue data set;

step S105, collecting the dialogue of which the speaker is the second object in the dialogue data to obtain the second dialogue data set.

In the embodiment of the disclosure, the data sets of which the speakers are different objects in the dialogue data are separately and correspondingly collected, and the statistical results are fused after independent prediction and independent statistics in the subsequent process, so that the dialogue data sets input by different objects can be fused, the supplement of user input is realized, complete and multi-round dialogue information can be obtained, and better accuracy can be obtained.

In this embodiment of the disclosure, before performing keyword statistics on the plurality of prediction sequences to obtain a statistical result, the statistical method for multiple rounds of conversations further includes:

obtaining the correlation degree of the keywords in the prediction sequence, and adjusting the keywords in the prediction sequence according to the correlation degree; wherein the obtaining of the relevancy of the keywords in the prediction sequence comprises: obtaining the correlation degree of the key words according to the element values and the number of the elements in the prediction sequence;

and moving the nth keyword in the predicted sequence, wherein the correlation degree of the nth keyword and the mth keyword is lower than a first threshold value, from the predicted sequence to a first related session set.

In an embodiment of the present disclosure, the first relevancy session set is a low relevancy session set.

In an embodiment of the present disclosure, the designing of the correlation includes: and designing the correlation degree according to the obtained first prediction sequence and the second prediction sequence. For example, in one embodiment, after the first and second dialog data sets are trained using a machine learning model, the prediction results are obtained, and the prediction results include the first prediction sequence P1 and the second prediction sequence P2. In one embodiment, a probability prediction result is obtained by using a softmax function, where 0 is restricted pij to 1, pi1+ pi2+.. + piW =1, pij is a predicted probability value of an element in the j-th column in the i-th row, piW is a predicted probability value of an element in the W-th column in the i-th row, i, j, W =1,2,3 · · n, n is a positive integer, and a first pre-sequencing column is obtained as P1= [ [ P11, P12,. P1W ], [ P21, P22,. Page.,. P2W ], [ P31, P32,. P3W ], [ P41, P42.,. P4W ]; the second pre-sequencing column was obtained as P2= [ [ P11, P12,.. P1W ], [ P21, P22.. P2W ] ]. Performing correlation design on the first prediction sequence P1 and the second prediction sequence P2:

where p is _i To predict the probability, W (W)>= 2) number of predicted categories, R ranges from [0,1]The larger R is, the higher the relevance of the single sentence in which the keyword is, 1 represents complete relevance, and 0 represents complete irrelevance.

In the embodiment of the present disclosure, assuming that the obtained relevancy sequence of the first predicted sequence P1 is [0.6,0.52,0.2,0.7], the second pre-sequencing is [0.61,0.76], the relevancy threshold of the first predicted sequence P1 is set to R1=0.5, and the relevancy threshold of the second predicted sequence P2 is set to R2=0.5, the relevancy in the first predicted sequence P1 and the second predicted sequence P2 is compared with the relevancy threshold, if the relevancy is greater than the threshold, the subsequent statistics are performed, and if the relevancy is less than the threshold, the single sentence where the nth keyword is located is placed in the low relevancy conversation set. For example, the ellipsis ". Cndot. Cndot." statement is put into the low-relevance session set.

In the embodiment of the disclosure, the detection of the relevancy of the keywords in the prediction sequence can improve the accuracy of the statistical result of the multi-turn dialog and the controllability among the steps in the multi-turn dialog statistical method.

The statistical method for multiple rounds of conversations in the embodiment of the present disclosure further includes:

step S1103, removing the nth keyword in the prediction sequence, wherein the correlation degree of the nth keyword and the mth keyword is lower than a first threshold value; the m-th keyword is any keyword except the n-th keyword in the prediction sequence;

in step S120, the performing keyword statistics on the plurality of prediction sequences to obtain a statistical result includes:

step S1204, performing keyword statistics on the plurality of prediction sequences without the nth keyword to obtain the statistical result.

In the embodiment of the present disclosure, the mth keyword and the nth keyword belong to different phrases.

In this embodiment of the present disclosure, in the step S1103, the method further includes removing a single sentence with an nth keyword in the prediction sequence, where a relevance of the single sentence with the mth keyword is lower than a first threshold.

In this embodiment of the present disclosure, in step S1204, the method further includes: and carrying out keyword statistics on the plurality of prediction sequences of the single sentence from which the nth keyword is removed to obtain the statistical result.

In this embodiment of the present disclosure, performing keyword statistics on the plurality of prediction sequences from which the nth keyword is removed to obtain the statistical result, includes:

a sequence P2 distribution index (1/n, o1 (variance), n) of the first prediction sequence P1 and the second prediction sequence is calculated, for example, a sequence distribution index of P1 is found to be (1/4,0.1403, 4) and a sequence distribution index of P2 is found to be (1/2,0.076, 2), and based on the sequence distribution index, a first weight w = sqrt (4 x 0.1403)/(sqrt (4 x 0.1403) + sqrt (2 x 0.076)) =0.664 is found.

In the embodiment of the present disclosure, after the first weight w is obtained, step S1201, step S1202, and step S1203 may be executed to obtain a statistical result for determining the dialog topic.

In the embodiment of the disclosure, the keywords with low relevance or the sentences where the keywords are located are removed, so that the execution efficiency of the subsequent steps can be improved, and unnecessary calculation is reduced. And a more accurate weight coefficient can be obtained, the accuracy of the statistical result fusing the first prediction sequence and the second prediction sequence representing the user is improved, the accuracy of the statistical result can be improved for medium and small data sets, and the controllability of the statistical method of the multi-turn conversation is improved.

With reference to fig. 5, before the step S110 is executed to predict dialog data between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences, the method of the embodiment of the present disclosure further includes:

a step S101 of dividing a historical dialogue between the first object and the second object into a first historical dialogue data set corresponding to the first object and a second historical dialogue data set corresponding to the second object;

step S102, performing classification training and integrity training in machine learning on the first historical dialogue data set to obtain a first model;

step S103, performing classification training in machine learning on the second historical dialogue data set to obtain a second model; wherein the dialog in predicting a dialog between a first object and a second object using a machine learning model occurs after the historical dialog.

In the embodiment of the present disclosure, before the step S110 is performed, the step S101, the step S102, and the step S103 may be performed in an online manner.

In the embodiment of the present disclosure, the first object is a user, and the second object is a customer service. The customer service may be a robotic customer service and/or a manual customer service. The robot customer service and/or the artificial customer service are/is used as an additional data source, the robot customer service and/or the artificial customer service are/is trained and learned independently, and the corresponding predicted statistical result is used as supplement of user input, so that more complete multi-turn conversation information can be obtained, and the accuracy of conversation summary can be improved.

In an embodiment of the present disclosure, the first historical dialogue data set is a user historical dialogue data set, and the second historical dialogue data set is a customer service historical dialogue data set. The first historical dialogue data set and the second historical dialogue data set can be small and medium data sets or large data sets.

In the embodiment of the present disclosure, if the historical dialogue occurs only once, the historical dialogue data set generated by the current historical dialogue is collected, and if the number of the dialogues generated by the current historical dialogue is small, the collected historical dialogue data set is a small data set, and the machine learning training may also be performed on the historical dialogue which has occurred only once, for example, the classification training and the integrity training in the machine learning are performed on the first historical dialogue data set, and the classification training in the machine learning is performed on the second historical dialogue data set.

In the embodiment of the present disclosure, after training the first historical dialogue data set and the second historical dialogue data set, a prediction may be directly performed on a dialogue that occurs later. For example, after a first historical session has occurred, the first historical session may be trained directly before predicting a second session. Therefore, the scheme provided by the embodiment of the disclosure can be more friendly to users who have few historical conversations during starting, and has wider applicability compared with the existing training needing a large number of data sets. For example, in one embodiment, after a new user has a first historical dialog with the customer service of the application, when the new user has a dialog with the customer service of the application again, the dialog to be generated again by the new user with the customer service can be predicted according to the machine learning model obtained by the training of machine learning performed by the last historical dialog.

In the embodiment of the disclosure, statistics is combined on the basis of machine learning, so that a plurality of rounds of dialogue summary with few initial historical dialogue are more user-friendly during starting, and a more accurate statistical result can be obtained for better prediction.

In an embodiment of the present disclosure, the statistical result is specifically used to determine at least one of the following:

the type of product;

problems before the product is sold;

the product has after-sale problems.

In embodiments of the present disclosure, product types may include, but are not limited to: electronic products, living goods, food and clothes. The electronic product may include, but is not limited to: computer, cell-phone, earphone, electron bracelet, hard disk. Pre-sale issues for a product may include, but are not limited to: the quality of the product is ensured or not, and the product can be replaced or not. After-market issues for a product may include, but are not limited to: problems with how the product is used, problems with the quality of the product, problems with whether the product needs to be changed back or not.

In the embodiment of the disclosure, the dialog is counted, and the obtained statistical result is used for determining the product type and the pre-sale or post-sale problem of the product, so that more intelligent and accurate service can be provided for the user, the satisfaction degree of the user is improved, and meanwhile, the service efficiency can also be improved.

In the embodiment of the present disclosure, a first dialog data set corresponding to the first object is predicted by using a first model in the machine learning model, so as to obtain a first prediction sequence. The first model is obtained by carrying out classification training and integrity training in machine learning on the first historical dialogue data set. The first model includes: a first classification model and a first integrity detection model.

In the embodiment of the disclosure, when a dialog occurring after a history dialog, that is, a current dialog is predicted, integrity detection is performed on the current dialog through a first integrity detection model, so as to obtain an integrity sequence related to a single sentence. When an incomplete single sentence is detected, setting 0 at the position corresponding to the single sentence in the sequence; when a complete single sentence is detected, the position in the sequence corresponding to this single sentence is set to 1.

In the embodiment of the disclosure, incomplete sentences may be merged according to the category, completeness, and continuity of context of the user input sentence. For example, in one embodiment, the following five words are entered by the user: "my computer", "bad", "kah", "pair", "thank you, i has no problem,/::)", the following sequence of the completeness of the single sentence [0,1] is obtained, and "my computer" and "bad" are detected to belong to two incomplete sentences which are continuously input, "my computer" belongs to the subject matter of the product type, and "bad" belongs to the subject matter of the product problem, so that "my computer" and "bad" can be combined to be "my computer bad".

In the embodiment of the present disclosure, incomplete sentences input by the user may also be merged according to the correlation between keywords. For example, in some embodiments, the product type and the product question are set to be of higher relevance, at this time, "my computer" and "bad" are words and phrases of higher relevance, and "my computer" belongs to the subject of the product type, and "bad" belongs to the subject of the product question, so that "my computer" and "bad" can be merged into "my computer is bad".

In the embodiment of the disclosure, incomplete but related sentences are combined together, so that more accurate subject categories can be obtained, and the accuracy of statistical results can be improved.

In an embodiment of the present disclosure, the statistical result is further used to determine at least one of:

a degree of satisfaction of the first object with the response of the second object;

an intent of the first object to purchase the product;

the first object has an evaluation tendency for the product.

In an embodiment of the present disclosure, the satisfaction degree of the answer of the first object to the second object includes: very unsatisfactory, general, satisfactory, very satisfactory.

In the embodiment of the disclosure, according to the satisfaction degree of the first object to the answer of the second object, that is, the satisfaction degree of the user to the robot customer service and/or the manual customer service, the answer attitude of the customer service is improved, and the service competitiveness is improved.

In an embodiment of the present disclosure, the intention of the first object to purchase the product includes: strong purchasing intention, general purchasing intention, weak purchasing intention, and non-purchasing intention.

In the embodiment of the disclosure, according to the purchase intention of the first object to the product, that is, after the user asks the question of the pre-sale question to the product, the purchase intention of the user to the product is provided for the user through the application, and the possibility of purchasing by the user is improved.

In an embodiment of the present disclosure, the tendency of the first object to evaluate the product includes: a positive evaluation tendency and a negative evaluation tendency. Positive evaluation trends indicate positive evaluations of the product, while negative evaluation trends indicate criticality and non-satisfaction evaluations of the product.

In the embodiment of the disclosure, the performance of the product can be improved, the competitiveness of the product can be improved, and the user market of the product can be expanded according to the first object, namely the evaluation tendency of the user to the product.

Referring to fig. 6, in an embodiment of the present disclosure, an apparatus 200 for statistics of multiple conversations is provided, where the apparatus includes:

the prediction module 210 is configured to predict dialog data between the first object and the second object by using a machine learning model, so as to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

a statistic module 220, configured to perform keyword statistics on the plurality of prediction sequences to obtain a statistical result; wherein the statistical result is used for determining at least a topic of the conversation data, a question related to the conversation, a degree of satisfaction of the conversation of either one of the two parties of the conversation, and an intention of either one of the two parties of the conversation.

In an embodiment of the present disclosure, the prediction module 210 is further configured to:

predicting a first dialogue data set corresponding to the first object by using a first model in the machine learning model to obtain a first prediction sequence; the first model is obtained by training according to a dialogue data set of the same object as the first class of objects and/or historical dialogue data of the first object;

predicting a second dialogue data set corresponding to the second object by using a second model in the machine learning model to obtain a second prediction sequence; and the second model is obtained by training according to the dialogue data set of the same object as the second class of object and/or the historical dialogue data of the second object.

In an embodiment of the present disclosure, the statistics module 220 is further configured to:

performing keyword statistics on the first prediction sequence to obtain a first statistical result;

performing keyword statistics on the second prediction sequence to obtain a second statistical result;

and obtaining a statistical result used for determining the conversation topic according to the product of the first statistical result and the first weight and the product of the second statistical result and the second weight.

Referring to fig. 7, in the embodiment of the present disclosure, the statistical apparatus 200 for multiple sessions further includes:

a first collecting module 230, configured to collect a dialog of a speaker as the first object in the dialog data to obtain the first dialog data set;

a second collecting module 240, configured to collect the dialog of the speaker as the second object in the dialog data to obtain the second dialog data set.

In an embodiment of the present disclosure, the statistical apparatus 200 for multiple sessions further includes:

a relevancy obtaining module, configured to obtain relevancy of the keyword in the prediction sequence;

the adjusting module is used for adjusting the keywords in the prediction sequence according to the correlation degree; the relevancy obtaining module is used for obtaining the relevancy of the keyword according to the element value and the number of the elements in the prediction sequence;

and the putting-in module is used for putting the nth keyword in the prediction sequence, the correlation degree of which with the mth keyword is lower than a first threshold value, into the first correlation degree conversation set from the prediction sequence.

Referring to fig. 8, in an embodiment of the present disclosure, the statistical apparatus 200 for multiple sessions further includes:

a removing module 250, configured to remove an nth keyword in the predicted sequence, where a correlation degree with the mth keyword is lower than a first threshold; the m-th keyword is any keyword except the n-th keyword in the prediction sequence;

the statistic module 220 is configured to perform keyword statistics on the plurality of prediction sequences from which the nth keyword is removed to obtain the statistical result.

Referring to fig. 9, in the embodiment of the present disclosure, the statistical apparatus 200 for multiple rounds of conversations further includes:

a segmentation module 260 for segmenting the historical dialog between the first object and the second object into a first historical dialog data set corresponding to the first object and a second historical dialog data set corresponding to the second object;

a first training module 270, configured to perform classification training and integrity training in machine learning on the first historical dialogue data set to obtain a first model;

a second training module 280, configured to perform classification training in machine learning on the second historical dialogue data set to obtain a second model; wherein the dialogue in predicting a dialogue between a first object and a second object using a machine learning model occurs after the historical dialogue.

In connection with the above embodiments, the following examples are provided:

example 1: a multi-turn dialog statistic device is provided.

FIG. 10 is a diagram of the overall architecture of a multi-turn dialog summary; FIG. 11 is a detailed flow diagram of the statistics module.

The multi-turn dialogue statistic device comprises a machine learning training module and a prediction module, and a single sentence classification model and a statistic module are added.

A machine learning training module: for performing the steps of:

step S301, processing the logs of the multi-turn dialog after labeling, and dividing the dialog logs of the dialog into a user input set A and a customer service (robot) input set B;

step S302, preprocessing (filtering) and vectorizing a data set A (the data set A has 2 labels, one is a classification label, and the other is an integrity 0 and 1 type label), inputting the preprocessed (filtered) and vectorized data into a machine learning training module 1, and performing training evaluation to obtain a user single sentence classification model M1 and an integrity detection model X1;

and step S303, preprocessing (filtering) and vectorizing the data set B, inputting the preprocessed (filtered) and vectorized data set B into a machine learning training module, and performing training evaluation to obtain a customer service (robot) single sentence classification model M2.

A prediction module: for performing the steps of:

step S304, processing the one-pass complete user service record, and dividing the user service record into a user input sequence A1 and a customer service (robot) input sequence B1;

step S305, inputting a user input sequence A1 into an X1 model, detecting the integrity of a single sentence of the model to obtain integrity sequence detection W1, performing correlation combination on the sequence A1 according to the integrity detection W1 to obtain A2, performing pretreatment (filtration) and vectorization (same training M1), inputting the sequence A1 into the model M1, and predicting to obtain a predicted sequence P1;

step S306, preprocessing (filtering) and vectorizing (co-training) the sequence B1, inputting the preprocessed and vectorized sequence B into a model M2, and predicting to obtain a predicted sequence P2;

step S307, inputting the sequence results P1 and P2 obtained in step S305 and step S306 into a statistics module, performing statistics, performing relevant post-processing, obtaining a statistics result T, and taking the statistics result T as a multi-turn dialog summary:

step S307, specifically including the steps of:

step S3071, inputting the prediction sequences P1 and P2 into a statistical module, judging whether the statistics is finished, if so, finishing, and outputting a result; if not, go to step S3072;

step S3072, carrying out correlation evaluation on the P1 sequence to obtain a correlation sequence (R11, R12.. R1 n), and setting a correlation threshold value R1; if the correlation coefficient is larger than the threshold value R1, entering a statistical module of the step S3073, otherwise entering a low correlation degree session set;

performing correlation evaluation on the P2 sequence to obtain a correlation sequence (R21, R22.. R2 m), and setting a correlation threshold R2; if the correlation coefficient is greater than the threshold R2, entering a statistical module of the step S3073, otherwise entering a low correlation degree session set;

step S3073, calculating weight coefficients w of different data sources, calculating the mean value and variance of the distribution indexes of the P1 and P2 related coefficient sequence groups and the number of effective dialogue rounds (1/n, o1, n), (1/m, o2, m), and using optimized Gaussian fusion: p1 weight coefficient w = sqrt (n × o 1)/(sqrt (n × o 1) + sqrt (m × o 2)), where sqrt means a square root;

step S3074, performing counting accumulation (L1, L2) and probability accumulation statistics (PL 1, PL 2) on the filtered P1 and P2 sequences obtained in the step S3072;

step S3075, performing weighted accumulation on the counting accumulation (L1, L2) and the probability accumulation statistics (PL 1, PL 2) obtained in step S3074, and introducing a weight coefficient w, w × (L1, L2) + (1-w) × (PL 1, PL 2) in 4.23 to obtain final statistical results (TL 1, TL 2); the (TL 1, TL 2) results may be used as a result for counting multiple rounds of dialog summary.

Example 2: a multi-turn dialog statistical method is provided.

Step S401, the customer service multi-turn dialog data set D and its label in the existing e-market scene, some examples are as follows:

TABLE 1 customer service data set D

Step S402, training

Step S4021, dividing all e-commerce customer service data sets D in step S401 to obtain a user data set a and customer service (robot) data B, as shown in tables 2 and 3 below:

TABLE 2 user data set A

TABLE 3 customer service (robot) data set B

S4022, filtering irrelevant information such as relevant expression special characters and links, and sensitive information such as identity cards and telephones in the A/B data set through various kinds of preprocessing, and performing conventional desensitization to obtain data sets a1 and B1;

table 4-user data set a1

TABLE 5 customer service (robot) data set b1

Step S4023, vectorizing the a1/b1 data by using a natural language vectorization technology, a one-hot or word vector technology and the like, wherein the K dimension/H dimension vector is as follows:

a1＝[((x11,x12,...x1K),y11),...,((xn1,xn2,...xnK),y1n)]

c1＝[((x11,x12,...x1K),yc1),...,((xn1,xn2,...xnK),ycn)]

b1＝[((x11,x12,...x1H),y21),...((xm1,xm2,...xmH),y2m)]

wherein y has a total of W classifications;

step S4024, training a1, b1 and c1 by using any one of classification models such as a machine learning model, LR (regional regression), SVM (support vector machines), DNN (deep Neural Networks) and the like, and obtaining a user data classification model M1, a customer service (robot) classification model M2 and a user data integrity detection model X1.

Step S403, predicting

Step S4031, predict one-pass complete conversation as in Table 6

TABLE 6 prediction sessions

Step S4032, the user sequence A1 is obtained by dividing as shown in a table 7, and the customer service sequence B1 is obtained by dividing as shown in a table 8

TABLE 7 user sequences A1

TABLE 8 customer service (robot) sequences B1

Step S4033, performs preprocessing-vectorization (co-training) to obtain the following vector sequence:

A1＝[[(x11,x12,...x1K)]，[x21,x22,...x2K]，[x31,x32,...x3K]，[x41,x42,...x4K],[x51,x52,...x5K]]

B1＝[[(x11,x12,...x1H)],[x21,x22,...x2H]]

step S4034, the A1 is sent to an X1 model for integrity prediction, and the following integrity sequence [0,1] is obtained, wherein 0 represents that the sentence is incomplete, 1 represents complete, and the first sentence 'my computer' and the second sentence 'bad' are both incomplete, the first sentence 'my computer' and the second sentence 'bad' are merged, the 'my computer + bad' is obtained, here, the '+' represents a sentence splicing special symbol, and the A1 data set is as follows:

TABLE 9-A1 data set

The resulting dataset A1' vector features are as follows:

A1’＝[[(x'11,x'12,...x'1K)],[x'21,x'22,...x'2K],[x'31,x'32,...x'3K]，[x'41,x'42,...x'4K]]

step S404, according to the flow of FIG. 11, respectively sending A1'/B1 to the M1 and M2 models obtained during training, and obtaining the following P1/P2 as the prediction result:

P1＝[[p11,p12,...，p1W]，[p21,p22,...，p2W]，[p31,p32,...，p3W]，[p41,p42,...，p4W]]

P2＝[[p11,p12,...，p1W]，[p21,p22,...，p2W]]

softmax probability prediction, wherein 0-pi-j-pi-1, pi1+ pi 2. + pi W =1.

Step S405, calculating a correlation index

Step S4051, correlation design

Here, pi is the prediction probability, W (W > = 2) is the number of prediction categories, R has a value range of [0,1], and the larger R represents the higher relevance of the single sentence where the keyword is located, 1 represents complete relevance, and 0 represents complete irrelevance.

Step S4052, here exemplified, sets the correlation threshold values of the P1 and P2 sequences to R1=0.5 and R2=0.5, respectively;

step S4053, calculating the sequences P1 and P2 obtained in step S404 according to the correlation defined in step S4051:

assuming that the P1 correlation sequence is [0.6,0.52,0.2,0.7];

assuming that the P2 correlation sequence is [0.61,0.76];

comparing with the correlation threshold value in the step S4052, if the correlation threshold value is larger than the correlation threshold value, entering the step S406, otherwise, obtaining a correlation statement, and circulating to a low correlation session set O; here ". . . "the statement will go into low relevance O;

step S4054, the P1 and P2 sequence distribution indices (1/n, o1 (variance), n) are (1/4,0.1403,4), (1/2,0.076,2), w = sqrt (4 × 0.1403)/(sqrt (4 × 0.1403) + sqrt (2 × 0.076)) =0.664.

In step S406, the statistical module performs the following steps:

in step S405, the correlation data obtained by taking the probability predictions topN (here, N = 3) and topM (here, M = 3) are counted as in table 10 below. N, M refer to different topics with the first N or M occurring more frequently, e.g. in the following table y1 may be a first class of topic, e.g.: "I computer is out"; y2 may be a second class of topics, for example: "my mobile phone is broken"; y3 may be a third category of subject matter, such as: "i'm earphone is bad". In { y1, y2, y3}, y1 corresponds to the occurrence frequency of the theme category in which y1 is located, and so on; { y1: p11, y2: p12, y3: p13}, y1: p11 corresponds to the probability of occurrence of the topic category in which y1 is located, and so on.

TABLE 10 statistical examples

Step S4061, accumulating P1 and P2 to obtain

Step S40611, P1 top3 is accumulated, counted and normalized to L1:

{y2：3，y3：3，y4：2，y1：1}/max([3，3，2，1])

step S40612, counting and normalizing P1 top3 accumulation times into L1:

{y2：(p12+p21+p41)，y3:(p13+p22+p42)，y4：(p24+p44)，y1：p11}/max([(p12+p21+p41),(p13+p22+p42),(p24+p44),p11])

step S40613, similarly, the P2 top3 accumulated count statistics can be obtained and normalized to PL1, and the P2 top3 accumulated probability statistics can be obtained and normalized to PL2.

Step S4062, fusing the statistical results of the user and the customer service (robot):

the weight w =0.664 of the effective information of the robot obtained in step S4054 is substituted into the following formula:

(TL1，TL2)＝w*(L1，PL1)+(1-w)*(L2，PL2) (1.3)

here, the value range of the weight coefficient w is [0,1], and w =1 represents the statistics of the fully-valued user; w =0 represents statistics that only value customer service completely.

TL1 and TL2 represent counting fusion statistical result and probability fusion statistical result respectively.

Step S4063, integrating probability and counting statistical results:

TL＝w1*TL1+(1-w1)*TL2 (1.4)

a TL statistic summary is obtained, and probability statistics and counting statistics are fused; the value range of w1 is [0,1], w1=1, the probability statistical result is completely biased, w1=0, the number statistical result is completely biased; TL may be the final result of multiple rounds of dialog summary. Adjustable setting parameters.

With reference to fig. 11, the secondary sorting module is a module for executing steps S4062 and S4063, and on the basis of machine learning, the secondary sorting module in the statistical module combines a probability cumulative statistics method, a count cumulative statistics method, and a multisource statistical result fusion technique, and is suitable for various scale data sets, especially medium and small scale data sets.

By combining the above embodiments and examples, the scheme of the present disclosure provides a set of new technical routes for solving multiple rounds of conversations by applying mature machine learning or advanced learning in the front edge and by using process control and mathematical statistics methods in order to solve the actual problems of multiple rounds of conversations in the field of intelligent customer service.

From the aspect of a module, the system comprises an offline training module and an online prediction module; from the technical stack, the method not only integrates machine learning (deep learning), a single sentence integrity detection model and an integrity corpus preprocessing module, but also comprises a machine learning result discrete statistical analysis module and machine learning result flow control, multiple data sources are adaptively integrated, and multiple index methods are self-controlled and integrated.

In the prediction module, an integrity corpus preprocessing algorithm, a single sentence relevancy algorithm, a dependence relevancy index control method and an algorithm carrier program thereof are as follows: firstly, a machine learning or deep learning model is obtained according to training, and a single-sentence prediction result is input as a single-sentence correlation algorithm to obtain a correlation index result; and (4) according to a threshold value set by the actual situation, if the threshold value is smaller than the threshold value, performing truncation processing on the single-sentence prediction result, and if the threshold value is larger than the threshold value, entering a statistical module.

In the secondary sequencing module, probability cumulative statistics, counting cumulative statistics methods, multi-source statistical result fusion technical methods and final algorithm carrier programs thereof are as follows: probability cumulative statistics and counting cumulative statistics methods are 2 different technical methods, reflect multi-turn dialogue nodules from the side surface respectively, obtain a final statistical result through a fusion technology, and can improve the accuracy of the statistical result.

Compared with the prior art, the scheme of the application has the following technical advantages:

as shown in fig. 10 and 11, based on a pipeline control flow architecture, supervised machine learning is integrated, the method is applicable to various scale data sets, especially small and medium scale data sets, and is more user-friendly to zero-start multi-turn dialog summary (for example, machine learning and training are performed on a first turn of historical dialog, and prediction and statistics can be performed on a second turn of historical dialog), and compared with the large scale data set required in the prior art, the method has wider applicability;

because the technical scheme based on accumulation is adopted, the self-adaptive weight algorithm based on filtering is adopted, all the characteristics are highly interpretable, and the self-control multi-scheme technology fusion and the process control technology are adopted, compared with the uncontrollable and unexplainable performances that the whole end-to-end training and prediction are black box sub-modes, the controllability of a certain degree is realized; therefore, the controllability and the interpretability are broken through.

According to the technical scheme and the technical method, on one hand, partial data of a robot (man-machine conversation) or a customer service (man-man) is used as a supplementary characteristic, so that the input of a user is supplemented, and complete multi-turn conversation information is obtained; on the other hand, the technical scheme is used for fusing statistical results of different technical schemes to be used as a summary of multi-turn conversations; in addition, the single sentence integrity detection module can obtain effective context data information, and benefits can be generated to the algorithm effect, so that better accuracy can be obtained.

In an embodiment of the present disclosure, there is provided an electronic device including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to implement the steps of the feedback method when running the computer service.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

In an embodiment of the present disclosure, a storage medium is provided, and the storage medium has computer-executable instructions, which are executed by a processor to implement the steps in the feedback method described above.

Alternatively, the integrated unit according to the embodiment of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present disclosure, and all the changes or substitutions should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims (12)

1. A statistical method for multiple sessions, the method comprising:

predicting dialogue data between a first object and a second object by using a machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

performing keyword statistics on the plurality of prediction sequences to obtain a statistical result; wherein the statistical result is used for determining at least a conversation topic of the conversation data, a question related to the conversation, a conversation satisfaction degree of any one of the two parties of the conversation, and an intention of any one of the two parties of the conversation.

2. The method of claim 1, wherein predicting the dialogue data between the first object and the second object using the machine learning model to obtain a plurality of prediction sequences comprises:

predicting a first dialogue data set corresponding to the first object by using a first model in the machine learning model to obtain a first prediction sequence; the first model is obtained by training according to a dialogue data set of the same object as the first class of objects and/or historical dialogue data of the first object;

predicting a second dialogue data set corresponding to the second object by using a second model in the machine learning model to obtain a second prediction sequence; wherein the second model is trained from the dialog data set of the same object as the second class of objects and/or historical dialog data of the second object.

3. The statistical method for multiple conversations according to claim 2, wherein the performing keyword statistics on the plurality of prediction sequences to obtain a statistical result comprises:

performing keyword statistics on the first prediction sequence to obtain a first statistical result;

performing keyword statistics on the second prediction sequence to obtain a second statistical result;

and obtaining a statistical result for determining the conversation topic according to the product of the first statistical result and the first weight and the product of the second statistical result and the second weight.

4. A statistical method of multiple rounds of dialogue as claimed in claim 2 or 3, wherein before predicting the dialogue between the first object and the second object using the machine learning model to obtain a plurality of prediction sequences, the method further comprises:

collecting a dialog of which a speaker is the first object in the dialog data to obtain the first dialog data set;

and collecting the dialogue of which the speaker is the second object in the dialogue data to obtain the second dialogue data set.

5. A statistical method for multiple conversations according to claim 1, wherein before the keyword statistics of the plurality of prediction sequences to obtain statistical results, the method further comprises:

obtaining the relevancy of the keywords in the prediction sequence, and adjusting the keywords in the prediction sequence according to the relevancy; wherein, the obtaining the relevancy of the keywords in the prediction sequence comprises: obtaining the correlation degree of the key words according to the element values and the number of the elements in the prediction sequence;

and putting the nth keyword in the predicted sequence, wherein the correlation degree of the nth keyword and the mth keyword is lower than a first threshold value, into a first correlation degree conversation set from the predicted sequence.

6. A statistical method of multiple sessions according to claim 1,2 or 3, wherein the method further comprises:

removing the nth keyword in the predicted sequence, wherein the correlation degree of the nth keyword and the mth keyword is lower than a first threshold value; the m-th keyword is any keyword except the n-th keyword in the prediction sequence;

the performing keyword statistics on the plurality of prediction sequences to obtain a statistical result includes:

and carrying out keyword statistics on the plurality of prediction sequences without the nth keyword to obtain the statistical result.

7. A statistical method of multiple conversations according to claim 2, wherein before predicting the conversation between the first and second objects using the machine learning model to obtain multiple prediction sequences, the method further comprises:

segmenting a historical dialog between the first object and the second object into a first historical dialog data set corresponding to the first object and a second historical dialog data set corresponding to the second object;

carrying out classification training and integrity training in machine learning on the first historical dialogue data set to obtain a first model;

performing classification training in machine learning on the second historical dialogue data set to obtain a second model; wherein the dialog in predicting a dialog between a first object and a second object using a machine learning model occurs after the historical dialog.

8. A statistical method of multiple rounds of dialogue as claimed in claim 1, characterized in that said statistical result is specifically used to determine at least one of:

a product type;

problems before the product is sold;

product after-market problems.

9. A statistical method of multiple rounds of dialogue as claimed in claim 1 or 8, characterized in that said statistical results are also used to determine at least one of:

a degree of satisfaction of the first object with the response of the second object;

an intent of the first object to purchase the product;

the first object has a tendency to evaluate the product.

10. A statistical apparatus for multiple sessions, the apparatus comprising:

the prediction module is used for predicting dialogue data between the first object and the second object by utilizing the machine learning model to obtain a plurality of prediction sequences; wherein the prediction sequence comprises: one or more keywords of a conversation between the first object and the second object;

the statistic module is used for carrying out keyword statistics on the plurality of prediction sequences to obtain a statistic result; wherein the statistical result is used for determining at least a conversation topic of the conversation data, a question related to the conversation, a conversation satisfaction degree of any one of the two parties of the conversation, and an intention of any one of the two parties of the conversation.

11. An electronic device, characterized in that the electronic device comprises:

a memory;

a processor coupled to the memory for executing the instructions by a computer stored in the memory, the processor being capable of implementing the method of any of claims 1 to 9.

12. A computer storage medium having stored thereon the computer-executable instructions; the computer-executable instructions, when executed by a processor, are capable of performing the method of any one of claims 1 to 9.

Images