CN117194664A

CN117194664A - Opinion classification model training method, opinion classification device and medium

Info

Publication number: CN117194664A
Application number: CN202311253073.7A
Authority: CN
Inventors: 于向丽; 张媛媛
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2023-09-26
Filing date: 2023-09-26
Publication date: 2023-12-08

Abstract

The disclosure provides an opinion classification model training method, an opinion classification device and a medium, relates to the technical field of data, and solves the problem of how to fully mine effective information in feedback opinions of users, wherein the training method comprises the following steps: acquiring a training set comprising a plurality of pieces of historical opinion classification data, each piece of historical opinion classification data comprising: the historical opinion word segmentation feature vector, the actual question category vector and the actual emotion category vector; and taking the historical opinion segmentation feature vectors in the training set as training input of the iterative opinion classification model to obtain training output problem category vectors and training output emotion category vectors, calculating average losses among the training output problem category vectors and the training output emotion category vectors and between the actual problem category vectors and the actual emotion category vectors by using a loss function, and controlling parameter updating and iteration of the opinion classification model by using an optimization algorithm so as to gradually reduce the average losses until reaching a training ending condition, thereby obtaining the opinion classification model after training.

Description

Opinion classification model training method, opinion classification device and medium

Technical Field

The present disclosure relates to at least the field of data analysis technology, and more particularly, to an opinion classification model training method, an opinion classification model training device, an opinion classification device, and a computer readable storage medium.

Background

For a large number of user feedback opinions received by the customer service system, repeated similar opinions in the feedback opinion need to be classified and arranged so as to know different problem categories, problem frequency degrees and emotional experience of users on the problems of the operator system. Obviously, the opinions can be classified only according to simple category fields, or similar opinions are identified, classified and arranged manually, so that the problems of high labor cost, low working efficiency, incapability of timely mining effective information in the opinions and the like are caused.

Disclosure of Invention

The technical problem to be solved by the present disclosure is to provide an opinion classification model training method, an opinion classification model training device, an opinion classification device and a computer readable storage medium to solve the problem of how to fully mine effective information in feedback opinion of a user.

In a first aspect, the present disclosure provides a method for training an opinion classification model, comprising:

Acquiring a training set comprising a plurality of pieces of historical opinion classification data, each piece of historical opinion classification data comprising: the historical opinion word segmentation feature vector, the actual question category vector and the actual emotion category vector;

the historical opinion word segmentation feature vector in the training set is used as the training input of the iterative opinion classification model to obtain the training output problem category vector and the training output emotion category vector,

calculating average losses between the training output question category vector and the training output emotion category vector and the actual question category vector and the actual emotion category vector with a loss function,

parameter updating and iteration of the opinion classification model are controlled by an optimization algorithm, so that the average loss is gradually reduced,

and obtaining the opinion classification model after training until reaching the training ending condition.

Further, obtaining a training set including a plurality of pieces of historical opinion classification data specifically includes:

labeling the historical opinion text with a question category and an emotion category;

extracting word segmentation characteristics of each historical opinion text to obtain historical opinion word segmentation characteristic vectors;

according to the number n and m of the problem categories and the emotion categories, assigning a to the dimension where the marked problem category and emotion category are located corresponding to each historical opinion word segmentation feature vector, and assigning b to the other dimension, wherein a is not equal to b, so as to obtain an n-dimensional actual problem category vector and an m-dimensional actual emotion category vector;

Combining each historical opinion word segmentation feature vector with a corresponding actual question category vector and an actual emotion category vector to obtain each piece of historical opinion classification data;

all the historical opinion classification data is divided into a training set and a testing set.

Further, before the historical opinion segmentation feature vector in the training set is used as the training input of the iterative opinion classification model, the training method further comprises:

and selecting the gating circulation unit neural network model as an opinion classification model, and setting the weight and the activation function of a reset gate and an update gate of the gating circulation unit neural network model.

Further, the training input of the iterative opinion classification model is the historical opinion segmentation feature vector in the training set, and the training output problem category vector and the training output emotion category vector are obtained, which concretely comprises:

the historical opinion word segmentation feature vectors in the training set are input into an input layer of a gating loop unit neural network model obtained through each iteration;

training the hidden layer of the neural network model of the gating circulating unit, which is obtained by each iteration of the input layer, wherein a reset gate and an update gate respectively calculate gating values according to weights and activation functions in the training, and jointly adjust the update of the hidden layer state of the neural network according to the respective gating values;

And obtaining training output problem category vectors and training output emotion category vectors which reach an output layer of the gate control loop unit neural network model obtained by each iteration after hidden layer training.

Further, calculating average loss between the training output question category vector and the training output emotion category vector and the actual question category vector and the actual emotion category vector by using a loss function, specifically including:

the cross entropy average loss per training is calculated according to:

loss＝-1/(k*(n+m))*∑(y*log(y_pred)+(1–y)*log(1–y_pred))

at this time, a=1, b=0, where: k is the number of historical opinion classification data in the training set, y is each dimension of the actual question category vector and the actual emotion category vector, and y_pred is each dimension of the training output question category vector and the training output emotion category vector corresponding to y.

Further, the parameter updating and iteration of the opinion classification model are controlled by an optimization algorithm so as to gradually reduce the average loss, and the method specifically comprises the following steps:

along the gradient direction that reduces the average loss, the update parameter new_weight of the opinion classification model is obtained according to the following equation:

new_weight＝weight-learning_rate*gradient

wherein: weight is the value of the current weight, gradient is the gradient of the loss function to the weight, learning_rate is the learning rate;

The new weight is used to update the opinion classification model used for the next iteration of training.

Further, after obtaining the trained opinion classification model, the training method further comprises:

the historical opinion word segmentation feature vectors in the test set are input into a trained opinion classification model to obtain a test output problem category vector and a test output emotion category vector,

calculating a first model performance assessment value between the test output question category vector and the actual question category vector, and a second model performance assessment value between the test output emotion category vector and the actual emotion category vector,

retraining the opinion classification model in response to the first model performance assessment value being less than a first preset value and/or the second model performance assessment value being less than a second preset value.

In a second aspect, the present disclosure provides a method of opinion classification comprising:

extracting word segmentation characteristics of the opinion text to be analyzed to obtain word segmentation characteristic vectors of the opinion to be analyzed;

inputting the feature vector of the opinion segmentation to be analyzed into a trained opinion classification model obtained by the opinion classification model training method according to any one of claims 1-7 to obtain a predicted problem category vector and a predicted emotion category vector;

Acquiring a predicted problem category vector and a problem category description and an emotion category description corresponding to the predicted emotion category vector;

classifying the opinion text to be analyzed according to the description of the problem category and the description of the emotion category.

In a third aspect, the present disclosure provides an opinion classification model training apparatus comprising:

the first data processing module is used for acquiring a training set comprising a plurality of pieces of historical opinion classification data, and each piece of historical opinion classification data comprises: the historical opinion word segmentation feature vector, the actual question category vector and the actual emotion category vector;

the model training module is connected with the first data processing module and is used for:

In a fourth aspect, the present disclosure provides an opinion classification device comprising:

the second data processing module is used for extracting word segmentation characteristics of the opinion text to be analyzed so as to obtain word segmentation characteristic vectors of the opinion to be analyzed;

the prediction module is connected with the second data processing module and is used for inputting the opinion segmentation feature vector to be analyzed into the trained opinion classification model obtained according to the opinion classification model training method to obtain a prediction problem category vector and a prediction emotion category vector;

the description module is connected with the prediction module and used for acquiring a problem category description and an emotion category description corresponding to the predicted problem category vector and the predicted emotion category vector;

the classifying module is connected with the describing module and used for classifying the opinion text to be analyzed according to the description of the problem category and the description of the emotion category.

In a fifth aspect, the present disclosure provides a computer readable storage medium having a computer program stored therein, which, when executed by a processor, implements the opinion classification model training method as described above and/or the opinion classification method as described above.

The invention provides an opinion classification model training method, an opinion classification method, a corresponding device and a medium, wherein an opinion classification model is trained on historical opinion word segmentation feature vectors, actual problem category vectors and actual emotion category vectors of historical opinion classification data to obtain a trained opinion classification model capable of simultaneously predicting and obtaining the problem category vectors and the emotion category vectors according to the opinion word segmentation feature vectors, and accuracy of the model in predicting the problem category and the emotion category is improved through a loss function and an optimization algorithm in the training process, so that the obtained opinion classification model can efficiently mine problem category information and emotion category information in opinions at the same time, and efficiency of the model in mining and classifying the opinion information is improved.

Drawings

FIG. 1 is a flow chart of a method of opinion classification model training according to an embodiment of the present disclosure;

FIG. 2 is a flow chart of a method of opinion classification according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of another opinion classification method according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a training device for an opinion classification model according to an embodiment of the present disclosure;

fig. 5 is a schematic structural view of an opinion classification device according to an embodiment of the present disclosure.

Detailed Description

In order for those skilled in the art to better understand the technical solutions of the present disclosure, embodiments of the present disclosure will be described in further detail below with reference to the accompanying drawings.

It is to be understood that the specific embodiments and figures described herein are merely illustrative of the present disclosure, and are not limiting of the present disclosure.

It is to be understood that the various embodiments of the disclosure and features of the embodiments may be combined with one another without conflict.

It is to be understood that for convenience of description, only portions relevant to the present disclosure are shown in the drawings of the present disclosure, and portions irrelevant to the present disclosure are not shown in the drawings.

It should be understood that each unit and module in the embodiments of the present disclosure may correspond to only one physical structure, may be composed of a plurality of physical structures, or may be integrated into one physical structure.

It will be appreciated that, without conflict, the functions and steps noted in the flowcharts and block diagrams of this disclosure may occur out of the order noted in the figures.

It will be appreciated that in the flow charts and block diagrams of the present disclosure, architecture, functionality, and operation of possible implementations of systems, apparatuses, devices, methods according to various embodiments of the present disclosure are shown. Where each block in the flowchart or block diagrams may represent a unit, module, segment, code, or the like, which comprises executable instructions for implementing the specified functions. Moreover, each block or combination of blocks in the block diagrams and flowchart illustrations can be implemented by hardware-based systems that perform the specified functions, or by combinations of hardware and computer instructions.

It is to be understood that the units and modules referred to in the embodiments of the disclosure may be implemented in software or hardware, for example, the units and modules may be located in a processor.

Example 1:

as shown in fig. 1, the present disclosure provides an opinion classification model training method, the training method comprising:

s11, acquiring a training set comprising a plurality of pieces of historical opinion classification data, wherein each piece of historical opinion classification data comprises: the historical opinion word segmentation feature vector, the actual question category vector and the actual emotion category vector;

S12, taking the historical opinion word segmentation feature vector in the training set as training input of an iterative opinion classification model to obtain a training output problem category vector and a training output emotion category vector,

Specifically, in this embodiment, according to the intelligent means, a dimension of viewing data is newly added, that is, the opinion feedback is categorized according to the similarity, so that the method can be used for processing the user feedback opinion received by the telecom customer service system, is convenient for maintenance and management of customer service personnel, reduces labor cost, and improves working efficiency. The method comprises the steps of training an opinion classification model (also called a similarity recognition model, a similarity analysis model and the like) on historical opinion segmentation feature vectors, actual problem category vectors and actual emotion category vectors of historical opinion classification data to obtain a trained opinion classification model capable of simultaneously predicting and obtaining the problem category vectors and the emotion category vectors according to the opinion segmentation feature vectors, and improving accuracy of the model in predicting the problem category and the emotion category through a loss function and an optimization algorithm in the training process, so that the obtained opinion classification model can efficiently mine problem category information and emotion category information in opinions at the same time, and improve information mining and classification efficiency of the model on the opinions.

In a more specific example, the application comprises two parts: one is a part of applying a model to classify opinions, including: reading an opinion feedback database by adopting an intelligent analysis technology, classifying similar data contents through data preparation, feature extraction and model application, marking classification marks on each data, storing the data into the database again, marking the classification marks on the data in the database after analysis, and carrying out data display according to the need in the follow-up classification; the other is part of building and modifying a model (i.e., model training), including: the method comprises the steps of utilizing a classified word stock and opinion feedback sample to train opinion feedback data, constructing a cyclic neural network model, training the model, evaluating and the like to generate a similarity recognition model, and manually examining and processing on the basis of a classification result, wherein if the classification result is found to be incorrect, the opinion feedback data and the classification of manual analysis can be input into the similarity recognition model for correcting the analysis result of the model, so that the follow-up recognition accuracy is improved. The key technologies involved include: the method comprises the steps of performing data processing on opinion texts by using a classified Word stock and Word Embedding (Word Embedding) technology, designing a learning model by using a gate control circulation unit (GRU, gate Recurrent Unit), and controlling a model training process by using a cross entropy loss function and a random gradient descent algorithm.

In one embodiment, obtaining a training set including a plurality of pieces of historical opinion classification data specifically includes:

Specifically, in this embodiment, the training model using annotated historical opinion data includes:

1) Data preparation: firstly, a group of marked telecom customer service opinion feedback data sets are needed to be prepared, the data sets usually comprise a series of comments or feedback texts, and corresponding labels are used for indicating the question type of the feedback and whether the feedback emotion of a user is positive or negative, and the data sets should be preprocessed, including word segmentation, stop word deletion, case conversion and the like. The following is an example telecommunications customer service opinion feedback data set preparation:

Sample example: text data (comment) |tag

To the best, my network is slow all the time, please help me look over the neutral emotion of the network problem

The mobile phone signal is always poor, and the problem of the signal is not normally communicated, namely negative emotion is solved

I want to terminate My contract, ask me specific flow for neutral emotion of refund question

Bill is wrong, deduction amount is incorrect |bill problem|negative emotion

Each recharging will be problematic, and the recharging will not be in charge of the charging problem

In the above example, each row represents opinion feedback of a user, where the "text data" column contains comments or feedback content of the user, and the "tag" column is a tag for the category to which the feedback belongs, and in order to prepare such a data set, opinion feedback of the user may be obtained through different channels, for example: online chat records, feedback forms submitted by users, social media and the like, and then classifying and labeling each feedback according to the content and the context of the feedback.

2) Feature extraction: a feature extraction method is applied to an input text to convert the text into a numerical representation so that the neural network can process the text, word embedding is a technology capable of mapping words to a vector space, word embedding is a common feature extraction method, text data can be converted into the numerical representation, and semantic relations among words are reserved. In this embodiment, feature extraction may be performed using a pre-trained word embedding model, and the following is an example of how feature extraction may be performed using word embedding: selecting a word embedding model: firstly, selecting a proper pre-training Word embedding model, such as Word2Vec, fastText, gloVe or BERT, which is trained on a large-scale text corpus and learns to map words into a high-dimensional vector space; loading a pre-training word embedding model: downloading and loading a selected Word embedding model, which can be used to convert words in text data into Word embedding vectors after loading the model, using a relevant library of Python, such as Gensim (for Word2 Vec), spaCy (for FastText), torchtext (for BERT), etc.; text data preprocessing: for text data in the opinion, preprocessing is needed firstly, including word segmentation, stop word removal, lowercase and the like, which will make the text data more suitable for being input into a word embedding model; text is converted into word embedding vectors: for each word in each text document, the word embedding model is used to convert it into a corresponding word embedding vector, and typically, word embedding vectors of words may be added or averaged to obtain a representation of the entire text document, e.g., for a text document, word embedding vectors of all of its words may be added to obtain a document-level word embedding vector; the feature representation is obtained: eventually each text document will be represented as a vector of values, which can be used as input features to the neural network, which can be used to train a text classification model. The following is a simple example demonstrating how a Gensim library can be used to load a pre-trained Word2Vec Word embedding model and convert text data into Word embedding vectors:

import gensim

from gensim.modles import word2Vec

# load pre-trained word2Vec model

word2vec_model＝word2vec.load(“path_to_word2vec_model”)

# example text data

text= "this application relates to natural language processing tasks"

Preprocessing the text (word segmentation, word deactivation removal, etc.)

Omitting text preprocessing step #

# convert words in text into word-embedded vectors and calculate document vectors

word=text

word_vectors＝[word2vec_model[word]for word in words if word in word2vec_model]

# calculate document vector

document_vector＝sum(word_vectors)/len(word_vectors)if word_vectors else None

Word embedded representation of a document that now contains text

3) Constructing a tag vector: in order to meet the requirement of telecommunication customer service feedback opinion data information mining, two labels, namely a question type label and an emotion type label, are arranged on text data, the question types can comprise network questions, signal questions, refund questions, bill questions, recharging questions and the like, and n kinds of questions can be manually set according to the actual classification requirement of a customer service system; emotion classification is a task of text classification, the goal of which is to determine emotion or emotion contained in a text, in which the text can be classified into m different emotion classes according to specific needs, for example: positive emotion: user feedback on product or service indicating satisfaction, negative emotion: users feed back dissatisfied comments on products or services, neutral emotions, users put general questions or comments, no obvious emotion tendency is expressed, and other emotion categories are shown: other emotion categories may also be defined as desired, such as advice, questions, etc.; because the number of categories is not fixed, for flexibly constructing data, the labels are represented by vectors, such as for the five problem types, the sequence of the five problem types is arranged firstly, then the labels of the specific problem types are assigned according to the sequence, such as the vector is [1, 0] when the network problem exists, the vector is [0,1, 0] when the signal problem exists, similarly, the emotion classifications are sequentially arranged as positive emotion, negative emotion and neutral emotion, vectors are [1, 0] when the emotion belongs to positive emotion, and the like, and all the vectors of the labels are expressed, so that the classification of various different dimensions is conveniently expressed, and the verification of subsequent model training is also convenient.

4) After all the prepared data are subjected to data processing, a data set which can be used for model training is obtained, and the data set is divided into a training set and a testing set in order to be able to timely check the effect of model training.

In one embodiment, before the training set of the historical opinion word segmentation feature vectors is used as the training input of the iterative opinion classification model, the training method further includes:

Specifically, in this embodiment, before training a model, a model structure for training is constructed, a similarity analysis model is constructed by using a recurrent neural network, and a recurrent neural network model improved by a gating circulation unit is used, so that the problems of gradient disappearance and gradient explosion are alleviated by introducing a reset gate and an update gate, and long-term dependency relationship, which is the dependency between time steps with a longer distance in sequence data, is better captured, and when the long-term dependency relationship is processed, the conventional RNN model faces the problems of gradient disappearance and gradient explosion, resulting in difficulty in correctly learning and capturing the dependency relationship. The GRU can better process long-term dependence, effectively transfer and update information, and better capture and model long-term dependence in sequence data in a recurrent neural network by introducing a reset gate and an update gate. Gradient extinction and gradient explosion are training problems commonly encountered in recurrent neural networks (RNN, recurrent Neural Network): gradient vanishing (Vanishing Gradient): when training an RNN model, in an iteration process of time expansion, gradients are multiplied continuously in a counter-propagation process, so that the gradients gradually become smaller and finally approach zero, the dependence relationship in a longer time sequence cannot be effectively learned by a network, and information cannot be transmitted and updated in the network; gradient explosion (Exploding Gradient): in contrast to the disappearance of the gradient, the gradient value becomes very large when the gradient explosion counter propagates, resulting in excessive weight update, unstable model, and difficult convergence. To solve these problems, the gating cycle unit introduces a reset gate and an update gate: reset Gate (Reset Gate): in the gating loop unit, the reset gate determines how much old information should be forgotten, it calculates a value between 0 and 1 by a sigmoid activation function, controlling whether the past state is to be mixed into the current state; update Gate (Update Gate): the update gate determines how much new information should be added to the current state, calculates a value between 0 and 1 by another sigmoid activation function, which determines the weights of the past state and the current input to determine the degree of update.

In one embodiment, the training input of the iterative opinion classification model is a historical opinion word segmentation feature vector in the training set, and the training output problem category vector and the training output emotion category vector are obtained specifically includes:

Specifically, in this embodiment, based on the above-mentioned gating cyclic unit neural network model, the model structure includes an input layer, a hidden layer and an output layer, where the hidden layer may have multiple layers, the output layer includes two nodes of an output problem category vector and an emotion category vector, and in the hidden layer training, the specific usage modes of resetting a gate and updating a gate in the gating cyclic unit are as follows: the input gate controls the influence of input data of the current time step on the hidden state; the reset gate controls the influence of the previous hidden state on the current time step input; the update gate controls whether the hidden state is updated, balancing the input of the previous hidden state and the current time step. Specifically, the reset gate and the update gate use a set of weights in the gating loop to calculate a gating value and adjust the update of the hidden state according to the gating value, so that long-term dependency can be effectively captured and information flow in the input sequence can be better processed.

In one embodiment, the average loss between the training output question category vector and the training output emotion category vector and the actual question category vector and the actual emotion category vector is calculated by using a loss function, and specifically includes:

the cross entropy average loss per training is calculated according to:

loss＝-1/(k*(n+m))*∑(y*log(y_pred)+(1–y)*log(1–y_pred))

Specifically, after the data set is divided into the training set and the test set, the similarity analysis model is trained by using the training set, and in the model training process, a loss function (cross entropy loss function) and an optimization algorithm (random gradient descent algorithm) need to be defined, and parameters of the model are gradually updated according to samples of the training set. A commonly used Loss function is the Cross-Entropy Loss function (Cross-Entropy Loss) which measures the difference between the model output and the sample label as follows: loss = -1/k = (y × log (y_pred) + (1-y) × log (1-y_pred), where "y" is the value of the actual tag (e.g., 0 represents negative emotion, 1 represents positive emotion) and "y_pred" is the predicted output of the model, this formula averages k samples to get the final average cross entropy loss. In this embodiment, the above formula needs to be modified as follows: loss = -1/(k (n+m)) Σ (y × (y_pred) + (1-y) × (1-y_pred)), because the present embodiment uses vector representation output, the difference between the output and the tag is calculated once for each dimension of the vector, and then the overall average is calculated, so that the difference for each dimension can be calculated in balance for the assignments represented by 0 and 1 in accordance with the above formula, and the above formula can be adapted when the assignments use other values.

In one embodiment, the optimization algorithm is used for controlling the parameter updating and iteration of the opinion classification model so as to gradually reduce the average loss, and the method specifically comprises the following steps:

new_weight＝weight-learning_rate*gradient

In particular, optimization algorithms typically use a random gradient descent algorithm (Stochastic Gradient Descent, SGD) or a variation thereof, which is a gradient-based optimization algorithm for adjusting parameters of the model to minimize the loss function. In this embodiment, the basic idea is to adjust the parameters in small steps along the gradient direction of the loss function, with the following formula: new_weight=weight-learning_rate, wherein "weight" is the value of the current weight, "gradient" is the gradient (slope) of the loss function to the weight, and "learning_rate" is the learning rate, which is a super-parameter that controls the step size of each gradient update. In the training process, the parameters of the model are updated by calculating the gradient of the loss function and using an optimization algorithm (such as SGD), and the model gradually reduces the value of the loss function by continuous iteration and parameter updating, so that the prediction accuracy of the model is improved.

In one embodiment, after obtaining the trained opinion classification model, the training method further comprises:

Specifically, in this embodiment, the test set is used to evaluate the performance of the trained model, and common evaluation indexes include accuracy, precision, recall, and F1 value, etc., and the model parameters may be optimized or the model structure may be adjusted according to the evaluation result. In this embodiment, since there are two kinds of prediction results, the evaluation may be performed respectively, and if the evaluation result does not meet the requirement after the evaluation, model improvement training may be performed pertinently, for example, the problem classification result may not reach the expectation that the problem classification part may be retrained separately, the emotion classification may not reach the expectation that the emotion classification part may be retrained separately, and all retrained may not reach the standard.

Common evaluation indexes include: accuracy (Accuracy): the ratio between the number of samples the model predicts correctly and the total number of samples is calculated as follows: accuracy= (tp+tn)/(tp+tn+fp+fn), where TP represents a True case (True posives), TN represents a True negative case (True posives), FP represents a False positive case (False posives), and FN represents a False negative case (False posives). Illustrating: if there are 100 samples in the test set, 80 of which are correctly classified as positive emotions (tp=80), 10 are incorrectly classified as positive emotions (fp=10), 5 are correctly classified as negative emotions (tn=5), and 5 are incorrectly classified as negative emotions (fn=5), the accuracy is (80+5)/100=0.85, i.e. 85%. Precision (Precision): the proportion of real examples in the sample predicted to be positive by the calculation model is calculated as follows: precision = TP/(tp+fp), illustrated: if the model predicts 100 samples as positive emotion (tp=80, fp=10), the accuracy is 80/(80+10) =0.89, i.e. 89%. Recall (Recall): the calculation model correctly predicts the proportion of the number of samples of positive category to the number of samples of true category, and the formula is as follows: recall=tp/(tp+fn), illustrated: if there are 90 positive emotion samples in the test set (tp=80, fn=10), the recall is 80/(80+10) =0.89, i.e. 89%. F1 value: the F1 value is the harmonic average of the precision rate and the recall rate, and comprehensively considers the performances of the precision rate and the recall rate, and the formula is as follows: f1 =2 (precision)/(precision+recovery), illustrating: if the accuracy is 0.89 and the recall is 0.89, the F1 value is 2 x (0.89 x 0.89)/(0.89+0.89) =0.89.

After the model is evaluated, model parameters can be optimized or model structures can be adjusted according to the evaluation result so as to further improve the model performance. Some optimization and adjustment methods include: parameter tuning: adjusting the hyper-parameters of the model, such as learning rate, regularization coefficient, hidden layer size, etc., may use grid search, bayesian optimization, etc. methods to find the optimal combination of parameters. And (3) adjusting a model structure: attempts have been made to improve model performance using deeper network structures, increasing the width and depth of the model, adding regularization or batch normalization layers, etc. Data enhancement: data enhancement techniques (e.g., rotation, translation, scaling, adding noise, etc.) are used to augment the training data, improving the generalization ability of the model. Migration learning: pre-trained models (e.g., BERT, gloVe, etc.) can be used to initialize model parameters and fine tune on the current task to improve model performance. And (3) ensemble learning: attempts have been made to combine the predicted results of multiple models using ensemble learning methods (e.g., voting, averaging, stacking, etc.) to achieve better performance.

Example 2:

as shown in fig. 2, the present disclosure provides an opinion classification method comprising:

S21, extracting word segmentation characteristics of the opinion text to be analyzed to obtain word segmentation characteristic vectors of the opinion to be analyzed;

s22, inputting the feature vectors of the opinion segmentation to be analyzed into a trained opinion classification model obtained according to an opinion classification model training method to obtain a predicted problem category vector and a predicted emotion category vector;

s23, obtaining a predicted problem category vector and a problem category description and an emotion category description corresponding to the predicted emotion category vector;

s24, classifying the opinion text to be analyzed according to the description of the problem category and the description of the emotion category.

Specifically, in this embodiment, the opinion classification model training method is as described in embodiment 1, and when the model training and evaluation of embodiment 1 is completed, the model can be used to predict questions and emotion classifications of unlabeled new feedback text, as described above, and a more specific example of the method is shown in fig. 3, including:

1) Obtaining opinion data to be analyzed, which may be specifically, connecting and reading opinion feedback databases, and obtaining data for classification in batches;

2) The data input opinion classification model specifically can be that the data is transmitted into a training constructed cyclic neural network model after feature extraction, and the text is clustered;

3) Classifying the opinions according to the classification result, namely, classifying and identifying the clustered data according to the classification word stock marks, so that the subsequent customer service personnel can conveniently check in a classified manner; a classified lexicon is a list containing predefined words or phrases, which are usually keywords related to the opinion feedback field, used for describing the subjects or problems of opinion feedback, and a common method for constructing the classified lexicon is to read and analyze some sample data by a professional or field expert in a manual labeling manner, and then extract related subjects or problems according to the keywords or phrases appearing therein, which may include specific product names, common problems, descriptive words, etc. used for describing different classifications of opinion feedback; when the clustered data is marked, corresponding classification identifications can be determined according to words or phrases in a classification word stock, for each cluster, the matching degree of opinion feedback and keywords in the classification word stock can be calculated, the keyword with the highest matching degree is selected to be used as the classification identification of the cluster, and the matching condition of a plurality of keywords can be considered according to requirements to classify more fine granularity; according to the classification identifiers, customer service personnel can easily check and filter opinion feedback belonging to different classifications, and the clustered data can be stored in a specific classification folder by using the classification identifiers, or the classification identifiers are added into the original data as extra columns for subsequent check and analysis;

4) Checking whether the classification is correct or not, specifically, checking the accuracy of clustered data through manual intervention;

5) And adding model training data and carrying out model correction, if the classification result is found to be incorrect, inputting opinion feedback data and the classification of manual analysis into an opinion classification model for correcting the model analysis result and improving the follow-up recognition accuracy.

Example 3:

as shown in fig. 4, the present disclosure provides an opinion classification model training apparatus, the training apparatus comprising:

a first data processing module 11, configured to obtain a training set including a plurality of pieces of historical opinion classification data, each piece of historical opinion classification data including: the historical opinion word segmentation feature vector, the actual question category vector and the actual emotion category vector;

the model training module 12 is connected with the first data processing module 11 and is used for:

In one embodiment, the first data processing module 11 specifically includes:

the marking unit is used for marking the problem category and emotion category for the historical opinion text;

the feature extraction unit is used for extracting word segmentation features of each historical opinion text so as to obtain historical opinion word segmentation feature vectors;

the classification assignment unit is used for assigning a dimension of the marked problem category and emotion category to each history opinion word segmentation feature vector according to the number n and m of the problem category and emotion category, and assigning b to the other dimensions, wherein a is not equal to b, so as to obtain an n-dimensional actual problem category vector and an m-dimensional actual emotion category vector;

the combination data unit is used for combining each historical opinion word segmentation feature vector with the corresponding actual problem category vector and actual emotion category vector to obtain each historical opinion classification data;

the data dividing unit is used for dividing all the historical opinion classification data into a training set and a testing set.

In one embodiment, model training module 12 further includes:

The model setting unit is used for selecting the gating circulation unit neural network model as an opinion classification model and setting the weight and the activation function of a reset gate and an update gate of the gating circulation unit neural network model.

In one embodiment, model training module 12 specifically includes:

the input layer unit is used for inputting the historical opinion word segmentation feature vectors in the training set into the input layer of the gating loop unit neural network model obtained by each iteration;

the hidden layer unit is used for enabling the input layer to be subjected to hidden layer training of the neural network model of the gating circulating unit obtained through each iteration, calculating gating values according to weights and activation functions by a reset gate and an update gate in the training, and jointly adjusting updating of the hidden layer state of the neural network according to the respective gating values;

and the output layer unit is used for acquiring training output problem category vectors and training output emotion category vectors which reach the output layer of the gate control loop unit neural network model obtained by each iteration after the hidden layer training.

In one embodiment, model training module 12 specifically includes:

a loss calculation unit for calculating a cross entropy average loss per training according to the following formula:

loss＝-1/(k*(n+m))*∑(y*log(y_pred)+(1–y)*log(1–y_pred))

In one embodiment, model training module 12 specifically includes:

an optimization iteration unit, configured to obtain an update parameter new_weight of the opinion classification model according to the following formula along a gradient direction that reduces average loss:

new_weight＝weight-learning_rate*gradient

In an embodiment, the training device further includes a model verification module configured to:

Example 4:

as shown in fig. 5, the present disclosure provides an opinion classification apparatus comprising:

the second data processing module 21 is configured to perform word segmentation feature extraction on the opinion text to be analyzed to obtain an opinion word segmentation feature vector to be analyzed;

a prediction module 22 connected to the second data processing module 21, for inputting the feature vector of the opinion segmentation to be analyzed into the trained opinion classification model obtained according to the opinion classification model training method described in embodiment 1, to obtain a predicted question class vector and a predicted emotion class vector;

the description module 23 is connected with the prediction module 22 and is used for acquiring a problem category description and an emotion category description corresponding to the predicted problem category vector and the predicted emotion category vector;

the classifying module 24 is connected with the describing module 23 and is used for classifying the opinion text to be analyzed according to the description of the question category and the description of the emotion category.

Specifically, the opinion classification device according to the present embodiment may be integrally provided with the opinion classification model training device according to embodiment 3, and the second data processing module 21 may be included in the first data processing module 11, and after the opinion classification is completed by the device according to the present embodiment, the opinion classification model may be retrained in conjunction with fig. 3.

Example 5:

embodiment 5 of the present disclosure provides a computer-readable storage medium having a computer program stored therein, which when executed by a processor, implements the opinion classification model training method described in embodiment 1 and/or the opinion classification method described in embodiment 2.

Computer-readable storage media includes volatile or nonvolatile, removable or non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, computer program modules or other data. Computer-readable storage media includes, but is not limited to, RAM (Random Access Memory ), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory, charged erasable programmable Read-Only Memory), flash Memory or other Memory technology, CD-ROM (Compact Disc Read-Only Memory), digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.

In addition, the present disclosure may also provide a computer apparatus including a memory and a processor, the memory having stored therein a computer program, which when executed by the processor performs the opinion classification model training method as described in embodiment 1 and/or the opinion classification method as described in embodiment 2.

The memory is connected with the processor, the memory can be flash memory or read-only memory or other memories, and the processor can be a central processing unit or a singlechip.

The embodiments 1 to 5 of the present disclosure provide an opinion classification model training method, an opinion classification method, a corresponding device and a medium, by performing opinion classification model training on a history opinion word segmentation feature vector, an actual problem category vector and an actual emotion category vector of history opinion classification data, a trained opinion classification model capable of simultaneously predicting and obtaining a problem category vector and an emotion category vector according to the opinion word segmentation feature vector is obtained, and accuracy of model prediction on the problem category and the emotion category is improved through a loss function and an optimization algorithm in the training process, so that the obtained opinion classification model can efficiently mine problem category information and emotion category information in opinions at the same time, and efficiency of information mining and classification of the model on opinions is improved.

It is to be understood that the above embodiments are merely exemplary embodiments employed to illustrate the principles of the present disclosure, however, the present disclosure is not limited thereto. Various modifications and improvements may be made by those skilled in the art without departing from the spirit and substance of the disclosure, and are also considered to be within the scope of the disclosure.

Claims

1. A method for training an opinion classification model, comprising:

2. The opinion classification model training method of claim 1, wherein obtaining a training set comprising a plurality of historical opinion classification data, comprises:

3. The opinion classification model training method of claim 1 or 2, characterized by further comprising, prior to using the historical opinion segmentation feature vectors in the training set as training inputs for the iterative opinion classification model:

4. The opinion classification model training method of claim 3, wherein the training output problem class vector and the training output emotion class vector are obtained by using the historical opinion segmentation feature vectors in the training set as training inputs of the iterative opinion classification model, and specifically comprising:

5. The opinion classification model training method of claim 2, characterized in that calculating average losses between the training output question class vector and the training output emotion class vector and the actual question class vector and the actual emotion class vector with a loss function, specifically comprises:

The cross entropy average loss per training is calculated according to:

loss＝-1/(k*(n+m))*∑(y*log(y_pred)+(1–y)*log(1–y_pred))

6. The opinion classification model training method of claim 2, characterized by controlling parameter updating and iteration of the opinion classification model with an optimization algorithm to gradually reduce average loss, comprising:

new_weight＝weight-learning_rate*gradient

7. The opinion classification model training method of claim 2, further comprising, after obtaining the trained opinion classification model:

8. A method of opinion classification comprising:

9. An opinion classification model training device, comprising:

10. An opinion classification device, comprising:

the prediction module is connected with the second data processing module and is used for inputting the opinion segmentation feature vector to be analyzed into the trained opinion classification model obtained by the opinion classification model training method according to any one of claims 1-7 to obtain a prediction problem category vector and a prediction emotion category vector;

11. A computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, which, when being executed by a processor, implements the opinion classification model training method of any of claims 1-7 and/or the opinion classification method of claim 8.