WO2019228000A1

WO2019228000A1 - Method and device for evaluating value of user review

Info

Publication number: WO2019228000A1
Application number: PCT/CN2019/076937
Authority: WO
Inventors: 陈岑
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2018-05-28
Filing date: 2019-03-05
Publication date: 2019-12-05
Also published as: CN108776864A; TW202004571A

Abstract

A method and device for evaluating the value of a user review. The method for evaluating the user review value comprises: obtaining a user review to be evaluated (S410); inputting the user review to be evaluated into a multi-task prediction model to predict a valuable evaluation value and a non-valuable evaluation value of the user review to be evaluated (S420), the multi-task prediction model being obtained by training a neural network on the basis of a plurality of user review samples having corresponding valuable evaluation values and non-valuable evaluation values; and evaluating the value of the user review to be evaluated according to the valuable evaluation value and the non-valuable evaluation value (S430).

Description

Method and device for evaluating user reviews

Technical field

One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a method and device for evaluating the value of user reviews.

Background technique

With the rapid development of online e-commerce, consumers have formed to judge whether an object (such as a product or a merchant) is good or bad or the service is good or bad according to user reviews. However, user reviews for a certain object are usually massive. Some of these massive user reviews are valuable or informative, while others are not.

Therefore, it is necessary to provide a method for evaluating the value of user reviews so that the value of user reviews can be evaluated as a problem to be solved.

Summary of the Invention

One or more embodiments of the present specification describe a method and device for evaluating the value of user reviews, which can improve the accuracy of evaluating the value of user reviews.

The first aspect provides a method for evaluating the value of user reviews, including:

Get comments from users to be evaluated;

Inputting the user evaluation to be evaluated into a multi-tasking prediction model to predict valuable evaluation values and non-valuation evaluation values of the user evaluation to be evaluated; the multi-task prediction model is based on a plurality of corresponding evaluation evaluation values and Samples of user reviews without value evaluation, obtained after training the neural network;

The value of the user comments to be evaluated is evaluated according to the valuable evaluation value and the non-valuation evaluation value.

In a second aspect, a device for evaluating user reviews is provided, including:

An obtaining unit for obtaining comments from users to be evaluated;

An input unit, configured to input the user comments to be evaluated obtained by the obtaining unit into a multi-tasking prediction model to predict valuable evaluation values and non-valuation evaluation values of the user evaluations to be evaluated; the multi-task prediction model is According to a plurality of user review samples with corresponding valuable evaluation values and non-valuable evaluation values, obtained after training the neural network;

The method and device for evaluating the value of user reviews provided by one or more embodiments of the present specification, obtain the user reviews to be evaluated. The user comments to be evaluated are input into a multi-tasking prediction model to predict the valuable evaluation values and non-valuable evaluation values of the user comments to be evaluated. The multi-task prediction model is obtained by training a neural network based on a plurality of user review samples with corresponding valuable evaluation values and non-valuable evaluation values. Evaluate the value of user reviews based on valuable and unvaluable evaluations. As a result, it is possible to evaluate the value of user reviews.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solutions of the embodiments of the present specification more clearly, the drawings used in the description of the embodiments are briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present specification. Those of ordinary skill in the art can also obtain other drawings according to these drawings without paying creative labor.

FIG. 1 is a flowchart of a method for acquiring a multi-task prediction model provided in this specification;

FIG. 2 is a schematic diagram of user comments provided in this specification;

FIG. 3 is a schematic diagram of a multi-task prediction model provided by this specification;

FIG. 4 is a flowchart of a method for evaluating a user review value provided by an embodiment of the present specification; FIG.

FIG. 5 is a schematic diagram of an apparatus for evaluating user reviews provided by an embodiment of the present specification.

Detailed ways

The solutions provided in this specification are described below with reference to the drawings.

The method for evaluating the value of user reviews provided by an embodiment of the present specification is applicable to a scenario in which the value of a large number of user reviews of an object is evaluated. The object here may refer to a virtual entity on a network such as a product or a merchant.

Before executing the method for evaluating the value of user reviews provided in this specification, a method for obtaining a multi-task prediction model may be performed first. The multi-task prediction model in this specification may be obtained by training a neural network based on a plurality of user review samples having corresponding valuable evaluation values and non-valuable evaluation values. The neural network here may include a recurrent neural network (Recurrent Neural Network, RNN), a convolutional neural network (Convolutional Neural Network, CNN), and the like. Among them, CNN can be TextCNN, which is a convolutional neural network used for text classification. The basic idea of text classification based on TextCNN is: combining CNN with text processing. The core is to use the CNN model in text classification tasks to extract key n-gram-like information in sentences. TextCNN is widely used in sentiment analysis, and the biggest advantage is that it is much faster than RNN, and it is easier to meet the requirements of online services.

Taking the neural network as TextCNN as an example, the above method for obtaining a multi-task prediction model can be shown in FIG. 1. In Figure 1, the method may include the following steps:

In step 110, a plurality of user comment samples are collected in advance.

In an implementation manner, a user can comment on an object on a webpage page, and based on the existing "like" function, a "click" function can be added. The "like" function can be It is used when the user comment is considered to be valuable or informative. The "click" function may be used when the user thinks that the user comment is not valuable. For example, when a user views a balcony cabinet, if there is a user comment describing the advantages and disadvantages of the balcony cabinet and related installation information, such as buying a faucet, a sewer pipe, etc., then the user will I think this user review is valuable or informative.

In one implementation manner, through the "like" function of the user comments, the number of users P (referred to as the first number) who consider the user comments to be valuable or informative can be counted. Through the “click” function of user reviews, the number of users Q (referred to as the second number) who think that the user reviews are not valuable can be counted, where P and Q are positive integers.

Taking the object as a product as an example, after adding a “click” function to the user review of the product, the user review of the product can be shown in FIG. 2. In FIG. 2, the button “useful” corresponds to the existing “like” function, and the button “useless” corresponds to the newly added “click” function. Specifically, when the user thinks that a certain user comment is valuable or informative, he can click a "useful" button corresponding to the user comment. Accordingly, the first number of user reviews is incremented by one. When a user feels that a user comment is worthless, he can click the "useless" button corresponding to the user comment. Accordingly, the second number of user reviews is incremented by one.

Specifically, the above multiple user comment samples may be collected from web pages of the object user comments that have both a “Like” function and a “Thumb” function. Therefore, the collected user review samples can have a corresponding first number and a second number at the same time. Of course, there may be only the first number (or the second number), and the second number (or the first number) may be determined according to a default value (for example, 0), which is not limited in this specification.

It should be noted that after collecting the above multiple user comments, the corresponding valuable evaluation value and non-valuation evaluation value can be determined. Among them, the valuable evaluation value is used to measure whether a user comment is valuable or informative. A non-valuable value is used to measure whether a user comment is worthless.

In an implementation manner, the foregoing determination process may be as follows: when the number of users viewing the multiple user review samples is the same or similar, the first quantity of each user review sample may be used as its valuable evaluation value, and The second quantity of each user review sample is used as its worthless evaluation value. When the number of users browsing the multiple user review samples is relatively large (for example, some user review samples are viewed by 100 users, while other user review samples are viewed by 1,000 users), the first The proportion of the number (that is, P) in the total number determines the valuable evaluation value of the user review sample, where the total number is the sum of P and Q. For example, valuable evaluation value = P / (P + Q). The non-valuable evaluation value of the user review sample can be determined according to the proportion of the second number (ie, Q) in the total number. For example, the valueless evaluation value = Q / (P + Q).

Of course, in practical applications, the above-mentioned valuable evaluation value and non-valuation evaluation value may also be determined according to other methods, for example, assigning corresponding weights to the first quantity and the second quantity, which is not limited in this specification.

Step 120: Preprocess multiple user review samples to obtain corresponding multiple words.

The preprocessing here includes, but is not limited to, word processing. Among them, the word segmentation processing is a traditional and conventional technology, which will not be repeated here.

Step 130: Determine a word vector of each word.

In one example, a word vector of each word may be determined according to a predefined word-to-word vector correspondence table. The predefined word-to-word vector correspondence table is used to record multiple words and word vectors corresponding to multiple words. The word vector here may be a vector used to represent a word, which may include several dimensions, which can reflect the actual relationship between words and words. Commonly used trained word vectors are Global Word Vectors (GloVe).

Step 140: input the word vector of each word into a convolutional neural network to construct a multi-task prediction model.

The process of building a multi-task prediction model here can be understood as the training process of a convolutional neural network. The training process may be: input the word vector of each word into the convolutional neural network to obtain the predicted value of each user review sample, including the predicted valuable evaluation value and the predicted non-valued evaluation value. After that, each parameter in the convolutional neural network is adjusted according to the predicted value and the valuable evaluation value and non-valuable evaluation value (actual value) determined in step 110; until the predicted value is equal to or close to the actual value, the above can be obtained. Multi-task prediction model.

The multi-task prediction model constructed in this specification can be shown in FIG. 3. In FIG. 3, the leftmost side contains multiple matrices, where each matrix corresponds to a user comment. The elements in the matrix can be word vectors of individual words contained in user comments. After that, the word vectors of each word can be convolved to form a convolution network. Finally, two fully-connected networks can be connected, one of which is used to predict the valuable evaluation value (N0) of user reviews and the other of which is used to predict the non-valuable evaluation value (N1) of user reviews.

As can be seen from Figure 3, the multi-task prediction model constructed in this specification promotes mutual learning between tasks and tasks in the neural network through the underlying network sharing (such as matrices and convolutional networks), and then connects multiple tasks The output layer (for example, constructing two fully connected networks) to further learn each individual task is a general technique in multitasking (putting multiple "related" tasks in a network) learning, and will not be repeated here. .

It should be noted that the above-mentioned process of constructing two fully-connected networks can also be referred to as a process of multi-task (predicting N0 and N1) learning. Through the above-mentioned multi-task learning process, the association between the task (prediction N0) and the task (prediction N1) can be better learned.

FIG. 4 is a flowchart of a method for evaluating a user review value provided by an embodiment of the present specification. As shown in FIG. 4, the method may include the following steps:

Step 410: Obtain user comments to be evaluated.

The user evaluation to be evaluated here may refer to a user's new comment on an object (such as a product or a merchant), which does not have corresponding valuable evaluation value and non-valuation evaluation value.

Step 420: Input the user comments to be evaluated into a multi-tasking prediction model to predict the valuable evaluation values and non-valuation evaluation values of the user comments to be evaluated.

For example, user comments to be evaluated may be input into the multi-task prediction model shown in FIG. 3 to output valuable evaluation values and non-valuable evaluation values of the user comments.

Step 430: Evaluate the value of the user comments to be evaluated according to the value evaluation value and the valueless evaluation value.

In one implementation manner, the proportion of valuable evaluation value may be determined according to the valuable evaluation value and the unvaluable evaluation value. According to the proportion of valuable evaluation values, the value of user comments to be evaluated is evaluated. For example, the value of user reviews can be evaluated according to the following formula.

Among them, C is the proportion of valuable evaluation value, which can also be called the audience of user reviews, N0 is the valuable evaluation value of user reviews, N1 is the non-valuable evaluation value of user reviews, and δ is a constant. The meaning of the above audience face may be what percentage of users think that the user comment is valuable or informative.

Specifically, the larger the C of the user comment, the greater the value of the user comment, that is, the wider the audience. Conversely, the less valuable it is.

Of course, the above is just a way to evaluate the value of user reviews. In other implementations, the corresponding weighted value and the non-valued evaluation value may be set with corresponding weight values, and then the value of the user comments to be evaluated is evaluated according to the valued evaluation value, the non-valued evaluation value, and the weight value. . For example, N0 and N1 in the above formula can be multiplied by corresponding weight values, which are not limited in this specification.

It can be understood that after corresponding user faces are calculated according to the above formula 1 for multiple user reviews, the audience faces of the multiple user reviews can be sorted. For example, multiple user reviews can be sorted according to the order of the audience, so that user reviews with a wide audience (that is, a large positive score) can be put in front, which will bring better user experience. Experience.

In summary, the multi-task prediction model of the above embodiment of this description is constructed after performing multi-task learning (valued evaluation value and non-valued evaluation value of user reviews), which can well learn the relationship between tasks and tasks Sex. Finally, since the value of user reviews is evaluated based on both the valued evaluation value and the non-valued evaluation value, the accuracy of the evaluation of the value of user comments can be improved. In addition, the above-mentioned embodiment of the present specification sorts based on the audience facing user reviews, which can conveniently allow users to see valuable or informative user reviews, thereby improving the user experience.

Corresponding to the above method for evaluating user reviews, an embodiment of the present specification also provides a device for evaluating user reviews, as shown in FIG. 5, the device includes:

The obtaining unit 501 is configured to obtain a user comment to be evaluated.

The input unit 502 is configured to input the user comments to be evaluated obtained by the obtaining unit 501 into a multi-tasking prediction model, so as to predict the valuable evaluation values and non-valuation evaluation values of the user comments to be evaluated. The multi-task prediction model is obtained by training a neural network based on a plurality of user review samples with corresponding valuable evaluation values and non-valuable evaluation values.

The neural network here may be a convolutional neural network TextCNN.

The value evaluation value of the user review sample may be determined according to the number P of users who consider the user review sample to be valuable or informative. The unworthy evaluation value of the user review sample may be determined according to the number Q of users who consider the user review sample to be unworthy. or,

The valuable evaluation value of the user review sample may be determined according to the proportion of P in the total number. The valueless evaluation value of the user review sample is determined based on the proportion of Q in the total number. Among them, the total number is the sum of P and Q. P and Q are both positive integers.

The evaluation unit 503 is configured to evaluate the value of the user comments to be evaluated according to the valued evaluation value and the valueless evaluation value.

The evaluation unit 503 may be specifically configured to determine the proportion of the valued evaluation value according to the valued evaluation value and the valueless evaluation value. According to the proportion of valuable evaluation values, the value of user comments to be evaluated is evaluated.

Specifically, the proportion of valuable evaluation value can be determined according to the following formula:

Among them, C is the proportion of valued evaluation value, N0 is the valued evaluation value, N1 is the valueless evaluation value, and δ is a constant.

Optionally, when the number of user comments to be evaluated is multiple, the apparatus may further include:

The sorting unit 504 is configured to sort each of the user comments to be evaluated according to an evaluation result of the value of each of the user comments to be evaluated.

The functions of the functional modules of the device in the foregoing embodiments of this specification can be implemented through the steps of the method embodiments described above. Therefore, the specific working process of the device provided by one embodiment of this specification is not repeated here.

An apparatus for evaluating the value of a user review provided by an embodiment of the present specification, an acquisition unit obtains a user review to be evaluated. The input unit 502 inputs the comments of the user to be evaluated into the multi-tasking prediction model to predict the valuable evaluation value and the non-valuation evaluation value of the user evaluation to be evaluated. The evaluation unit 503 evaluates the value of the user comments to be evaluated according to the valued evaluation value and the valueless evaluation value. As a result, the accuracy of the evaluation of user reviews can be improved.

Those skilled in the art should appreciate that, in one or more of the above examples, the functions described in this specification may be implemented by hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored in or transmitted over as one or more instructions or code on a computer-readable medium.

The specific implementation manners described above further describe the purpose, technical solutions, and beneficial effects of the present specification. It should be understood that the foregoing descriptions are merely specific implementation manners of the present description, and are not intended to limit the scope of the present description. The scope of protection, any modification, equivalent replacement, or improvement made on the basis of the technical solution of this specification shall be included in the scope of protection of this specification.

Claims

A method for evaluating the value of user reviews, which is characterized by:

Get comments from users to be evaluated;

Inputting the user evaluation to be evaluated into a multi-tasking prediction model to predict the valuable evaluation value and non-valuation evaluation value of the user evaluation to be evaluated; the multi-task prediction model is based on a plurality of corresponding Samples of user reviews without value evaluation, obtained after training the neural network;

The value of the user comments to be evaluated is evaluated according to the valuable evaluation value and the non-valuation evaluation value.
The method according to claim 1, wherein when the number of user comments to be evaluated is multiple, the method further comprises:

According to the evaluation results of the values of the user comments to be evaluated, the comments of the users to be evaluated are sorted.
The method according to claim 1 or 2, wherein the neural network is a convolutional neural network TextCNN.
The method according to claim 1, wherein the evaluating the value of the user comments to be evaluated according to the valuable evaluation value and the non-valuation evaluation value comprises:

Determining the proportion of valuable evaluation value according to the valuable evaluation value and the non-valuation evaluation value;

According to the proportion of the valuable evaluation value, the value of the user comments to be evaluated is evaluated.
The method according to claim 4, wherein determining the proportion of valuable evaluation value according to the valuable evaluation value and the non-valuation evaluation value comprises:

Determine the proportion of the valuable evaluation value according to the following formula:

Wherein, C is the proportion of the valued evaluation value, N0 is the valued evaluation value, N1 is the valueless evaluation value, and δ is a constant.
The method according to claim 1, wherein the value evaluation value of the user review sample is determined according to the number P of users who consider the user review sample as valuable or informative; the user review sample The value of the valueless evaluation is determined based on the number Q of users who consider the sample of user reviews to be worthless; or,

The valuable evaluation value of the user review sample is determined according to the proportion of P in the total number; the unvaluable evaluation value of the user review sample is determined according to the proportion of Q in the total number; The total number is the sum of P and Q, and P and Q are positive integers.
A device for evaluating the value of user reviews, which is characterized by:

An obtaining unit for obtaining comments from users to be evaluated;

An input unit, configured to input the user comments to be evaluated obtained by the obtaining unit into a multi-tasking prediction model to predict valuable evaluation values and non-valuation evaluation values of the user evaluations to be evaluated; According to a plurality of user review samples with corresponding valuable evaluation values and non-valuable evaluation values, obtained after training the neural network;

An evaluation unit is configured to evaluate the value of the user evaluation to be evaluated according to the valuable evaluation value and the non-valuation evaluation value.
The device according to claim 7, wherein when the number of the user comments to be evaluated is multiple, further comprising:

The sorting unit is configured to sort the user comments to be evaluated according to an evaluation result of the value of the user comments to be evaluated.
The device according to claim 7 or 8, wherein the neural network is a convolutional neural network TextCNN.
The apparatus according to claim 7, wherein the evaluation unit is specifically configured to:

Determining the proportion of valuable evaluation value according to the valuable evaluation value and the non-valuation evaluation value;

According to the proportion of the valuable evaluation value, the value of the user comments to be evaluated is evaluated.
The apparatus according to claim 10, wherein the evaluation unit is further configured to:

Determine the proportion of the valuable evaluation value according to the following formula:

Wherein, C is the proportion of the valued evaluation value, N0 is the valued evaluation value, N1 is the valueless evaluation value, and δ is a constant.
The device according to claim 7, wherein the value evaluation value of the user review sample is determined according to the number P of users who consider the user review sample as valuable or informative; the user review sample The value of the valueless evaluation is determined based on the number Q of users who consider the sample of user reviews to be worthless; or,

The valuable evaluation value of the user review sample is determined according to the proportion of P in the total number; the unvaluable evaluation value of the user review sample is determined according to the proportion of Q in the total number; The total number is the sum of P and Q, and P and Q are positive integers.