CN110751180B - Spurious comment group division method based on spectral clustering - Google Patents

Abstract

The invention relates to a false comment group division method based on spectral clustering, which comprises the following steps: s1: collecting and cleaning comment data of an electronic commerce platform; s2: calculating 5 similarity indexes of common comment times, scoring similarity of the same commodity, user interaction times, user positive scoring proportion and user negative scoring proportion based on the metadata in the S1, wherein the similarity of the scoring proportions between two users is measured by Euclidean distance; s3: constructing a weighted reviewer graph; s4: dividing the adjacent matrix to obtain a plurality of groups; s5: the classification of the divided groups is further determined manually by selecting reasonable analysis indexes and proper thresholds. The invention reflects the behavior similarity between users more accurately, and improves the accuracy of the subsequent dividing algorithm; the dividing effect on the weighted reviewer graph is better and has universality.

Description

A Fake Review Group Classification Method Based on Spectral Clustering

technical field

The present invention relates to the technical field of data mining, and more specifically, to a method for grouping false comments based on spectral clustering.

Background technique

With the rapid development of the Internet, the emergence of e-commerce platforms has changed people's consumption patterns in various aspects such as shopping, traveling, and dining. In the transaction process of the e-commerce platform, service or product reviews play a key role in the user's purchase decision-making behavior. Products with more positive reviews and more authentic products are more favored by users. The willingness to buy is low. In recent years, with the diversified development of e-commerce platforms and the intensification of market competition, many unscrupulous merchants will use various means to obtain more false positive reviews or give false negative reviews to competitors. Traditional methods such as sellers obtaining higher-than-average positive ratings through the review service provided by intermediaries, or inducing consumers to give false positive ratings by means of "cash back for positive ratings". With the rise of online social platforms, a new type of review mode has become popular. This mode is promoted through Key Opinion Leaders (KOL), and its operation team simultaneously publishes false reviews on products. In this mode, due to the high efficiency of information dissemination of key opinion leaders, their fans will publish a large number of comments after normal consumption in a short period of time, and normal comments and fake comments burst out in a short period of time, making the detection of fake comment groups difficult. The difficulty is greatly increased.

Many existing algorithms for detecting fake review groups cannot meet the new needs, especially in the new review mode, because many normal consumers will buy the same product within a similar period of time, the product reviews will be in the The large increase in a short period of time makes some algorithms that detect a small amount of anomalous behavior or that use comment explosion to perform poorly on this problem. Therefore, it is necessary to find a method with better effect and able to meet new demands.

Contents of the invention

In order to overcome the defect of poor detection effect on false comment groups described in the prior art, the present invention provides a method for dividing false comment groups based on spectral clustering.

The method comprises the steps of:

S1: Collect and clean the comment data of the e-commerce platform; the metadata used includes: user id, comment id, product id, rating, and the number of interaction behaviors of comments (such as: being "liked" by other users, being "liked" by other users) found useful", "interesting" by other users, etc.);

S2: Based on the metadata in S1, calculate the five similarity indicators of "number of common comments", "score similarity of the same product", "user interaction times", "proportion of positive user ratings" and "proportion of negative user ratings". The similarity of rating ratio between two users is measured by Euclidean distance;

S3: Construct a graph of weighted reviewers: each user is a graph node, and users who have commented on a product in pairs are connected by an undirected edge, and the weight of the edge is based on the five indicators calculated in S2 calculated;

S4: Divide the adjacency matrix of the graph constructed by S3 through the spectral clustering algorithm to obtain several groups;

S5: By selecting reasonable analysis indicators and appropriate thresholds, manually make further judgments on the categories of the groups.

Preferably, the calculation formula of the number of common reviews (Co-Reviewing Times, CRT) in S2 is:

CRT(n ₁ ,n ₂ )＝|P ₁ ∩P ₂ |

Among them, n ₁ , n ₂ are two different reviewers, and P ₁ , P ₂ are collections of commodities that n ₁ , n ₂ have commented on respectively.

Preferably, the formula for calculating the rating similarity (Similarity of Rating on SameProduct, SRSP) of the same product in S2 is:

分别为n₁,n₂对商品P发表第i或第j条评价的评分，N₁,N₂分别为n₁,n₂在商品P上发表的评论数。in,

N ₁ , n ₂ are the ratings of the i-th or j-th evaluation on product P, respectively, and N ₁ , N ₂ are the number of comments n ₁ , n ₂ have published on product P.

Preferably, the calculation formula of the user interaction times (Interaction Times, IT) in S2 is:

Among them, C _1i and C _2i represent the times of n ₁ and n ₂ interaction behaviors of the mth kind respectively.

Preferably, the formula for calculating the positive rating ratio (Positive Rating Ratio, PR) of users in S2 is:

Among them, S _i and S represent the rating of a user's comment, ∑S ⁰ is the number of comments published by the user with a score of {1, 2.3.4.5}, and ∑S _i ⁰ is the number of comments published by the user with a score of {4,5} frequency.

Preferably, the formula for calculating the negative rating ratio (Negative Rating Ratio, NR) of users in S2 is:

Preferably, in S2, the Euclidean distance is used to measure the similarity between the proportion of positive ratings and the proportion of negative ratings of two users:

Among them: _n1 and r _n2 are the proportion of positive ratings or negative ratings of reviewers n ₁ , 2 .

Preferably, S3 includes the following steps:

S3.1: Import all users as graph nodes;

S3.2: Using two nodes as a node combination, calculate the number of common comments between two nodes;

S3.3: Determine whether the number of common comments calculated in S3.2 is greater than 0; if so, proceed to S3.4; if not, return to S3.2 for the next node combination until all node combinations are traversed;

S3.4: Calculate the similarity index of the node to the rest; calculate the weight and establish the edge of the two nodes;

S3.5: Complete the construction of the weighted reviewer graph.

Preferably, the formula for calculating the weight of an edge in S3 is as follows:

和/>

分别为用户i和用户j积极评分比例和消极评分比例的近似程度。Among them, ω _ij is the weight on the edge of node i and node j, k is the number of metrics, here is 5, CRT _ij is the number of common comments by user i and user j, SPSR _ij is the number of user i and user j j is the similarity degree of ratings published by the same product, IT _ij is user i and user j,

and />

are the approximate degree of positive rating ratio and negative rating ratio of user i and user j respectively.

Preferably, S4 includes the following steps:

S4.1: Input the weighted reviewer graph G, and divide the number n of clusters;

S4.2: Calculate the adjacency matrix A, degree matrix D and Laplacian matrix L=D-A from the weighted reviewer graph G;

S4.3: Obtain the standardized Laplacian matrix according to the following formula:

NL＝D ^-1/ (-A) ^-1/ ＝ ^-1/ LD ^-1/

S4.4: Calculate the smallest k eigenvalues of NL and their corresponding eigenvectors f, where k is the number n of divided clusters;

S4.5: Form the corresponding feature vector f into a feature matrix f of v×k size and normalize it by row, v is the number of samples, that is, the number of nodes in graph G;

S4.6: Clustering the standardized feature vectors by using the K-Means method to obtain n candidate groups C=(c ₁ ,2...c _n ).

Compared with the prior art, the beneficial effect of the technical solution of the present invention is: the present invention is based on the idea of the weighted reviewer graph, and proposes five similarity indexes for measuring the similarity in behavior between different users, which is better than the existing The algorithm based on the weighted reviewer graph more accurately reflects the behavior similarity between users and improves the accuracy of the subsequent division algorithm.

The present invention uses the spectral clustering algorithm to divide the weighted reviewer graph, which is better than some existing division algorithms, such as: KMeans algorithm, hierarchical clustering algorithm, Louvain community discovery algorithm, etc. fitness.

Description of drawings

FIG. 1 is a flow chart of the method for grouping false comments based on spectral clustering described in Embodiment 1.

Figure 2 is a schematic flow chart of the spectral clustering algorithm.

Detailed ways

The accompanying drawings are for illustrative purposes only and cannot be construed as limiting the patent;

In order to better illustrate this embodiment, some parts in the drawings will be omitted, enlarged or reduced, and do not represent the size of the actual product;

For those skilled in the art, it is understandable that some well-known structures and descriptions thereof may be omitted in the drawings.

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Example 1

The present embodiment provides a method for grouping false comments based on spectral clustering, as shown in Figure 1, the method includes the following steps:

S1: Collect and clean the comment data of the e-commerce platform; the metadata used includes: user id, comment id, product id, rating, and the number of interaction behaviors of comments (such as: being "liked" by other users, being "liked" by other users) found useful", "interesting" by other users, etc.);

Among them, metadata is data used to define data. Review data is data of many dimensions collected from e-commerce platforms, including fields such as user id, comment time, rating, etc. Each item here, such as "user id" and "comment time", is a piece of metadata. Through the description of the metadata, the information of a comment can be restored. In this example, some data items (metadata) of a comment are selected, such as "comment time" is not used.

S2: Based on the metadata in S1, calculate the five similarity indicators of "number of common comments", "score similarity of the same product", "user interaction times", "proportion of positive user ratings" and "proportion of negative user ratings". The similarity of rating ratio between two users is measured by Euclidean distance;

S3: Construct a graph of weighted reviewers: each user is a graph node, and users who have commented on a product in pairs are connected by an undirected edge, and the weight of the edge is based on the five indicators calculated in S2 calculated;

S4: Divide the adjacency matrix of the graph constructed by S3 through the spectral clustering algorithm to obtain several groups;

S5: By selecting reasonable analysis indicators and appropriate thresholds, manually make further judgments on the categories of the groups. In this example, extreme rating ratio, repeated comment ratio and rating deviation are selected as analysis indicators. The calculation formula of Co-Reviewing Times (CRT) in S2 is:

CRT(n ₁ ,n ₂ )＝|P ₁ ∩P ₂ |

Among them, n ₁ , n ₂ are two different reviewers, and P ₁ , P ₂ are collections of commodities that n ₁ , n ₂ have commented on respectively.

The calculation formula of the rating similarity (Similarity of Rating on Same Product, SRSP) of the same product in S2 is:

分别为n₁,n₂对商品P发表第i或第j条评价的评分，N₁,N₂分别为n₁,n₂在商品P上发表的评论数。in,

N ₁ , n ₂ are the ratings of the i-th or j-th evaluation on product P, respectively, and N ₁ , N ₂ are the number of comments n ₁ , n ₂ have published on product P.

The calculation formula of user interaction times (Interaction Times, IT) in S2 is:

Among them, C _1i and C _2i represent the times of n ₁ and n ₂ interaction behaviors of the mth kind respectively.

The formula for calculating the Positive Rating Ratio (PR) of users in S2 is:

Among them, S _i and S represent the rating of a user's comment, ∑S ⁰ is the number of comments published by the user with a score of {1, 2.3.4.5}, and ∑S _i ⁰ is the number of comments published by the user with a score of {4,5} frequency.

The formula for calculating the negative rating ratio (Negative Rating Ratio, NR) of users in S2 is:

In S2, the Euclidean distance is used to measure the similarity between the proportion of positive ratings and the proportion of negative ratings of two users:

Among them: r _n1 and r _n2 are the proportion of positive ratings or negative ratings of reviewers n ₁ and n ₂ .

S3 consists of the following steps:

S3.1: Import all users as graph nodes;

S3.2: Using two nodes as a node combination, calculate the number of common comments between two nodes;

S3.3: Determine whether the number of common comments calculated in S3.2 is greater than 0; if so, proceed to S3.4; if not, return to S3.2 for the next node combination until all node combinations are traversed;

S3.4: Calculate the similarity index of this node to the rest, including: "score similarity of the same product", "user interaction times", "user positive score ratio" and "user negative score ratio"; calculate the weight and Create an edge between two nodes;

S3.5: Complete the construction of the weighted reviewer graph.

The formula for calculating the weight of an edge in S3 is as follows:

和/>

分别为用户i和用户j积极评分比例和消极评分比例的近似程度。Among them, ω _ij is the weight on the edge of node i and node j, k is the number of metrics, here is 5, CRT _ij is the number of common comments by user i and user j, SPSR _ij is the number of user i and user j j is the similarity degree of ratings published by the same product, IT _ij is user i and user j,

and />

are the approximate degree of positive rating ratio and negative rating ratio of user i and user j respectively.

S4 includes the following steps:

S4.1: Input the weighted reviewer graph G, and divide the number n of clusters;

S4.2: Calculate the adjacency matrix A, degree matrix D and Laplacian matrix L=D-A from the weighted reviewer graph G;

S4.3: Obtain the standardized Laplacian matrix according to the following formula:

NL＝D ^-1/2 (D-AD ^-1/2 ＝D ^-1/2 LD ^-1/2

S4.4: Calculate the smallest k eigenvalues of NL and their corresponding eigenvectors f, where k is the number n of divided clusters;

S4.5: Form the corresponding feature vector f into a feature matrix F of v×k size and normalize it by row, v is the number of samples, that is, the number of nodes in graph G;

S4.6: Use the K-Means method to cluster the standardized feature vectors to obtain n candidate groups C=(c ₁ ,c ₂ . . . c _n ).

As a specific embodiment, the similarity index discussed in this embodiment can omit, replace or add other indexes according to the actual situation, and the calculation method of the indexes can be combined and modified.

Group division is to divide a large number of users into several groups. Users in each group have the same or similar behavior. Whether a group belongs to a fake comment group needs to be further judged manually. Since the types of data generated and collected by each e-commerce platform are quite different, in practical applications, the specific implementation of the group segmentation algorithm should be adjusted accordingly according to the different types of metadata given by the dataset. Theoretically, the idea of this embodiment can also be applied to public opinion monitoring, marketing and other fields.

The embodiment of the false comment group division method in this embodiment can be roughly divided into five steps: data collection and cleaning, statistical similarity index, construction of weighted reviewer graph, spectral clustering division of groups and manual judgment of group categories, among which :

Data collection and cleaning include collecting, analyzing and cleaning the original data. There are often many errors and incomplete data in the original data. Some data items with missing data or data with abnormal values can be processed by deleting, filling with the mean value, etc. . The embodiment uses the data set used in the research, so it is only necessary to analyze the data, select the required data type, and delete useless data.

Statistical similarity index: Most of the indexes used in this embodiment are calculated for pairs of users, so the calculation of the similarity index is carried out simultaneously in the process of constructing the weighted reviewer graph. This step is to count and store some statistical indicators that need to be used (such as the ratio of positive/negative ratings of users) in advance, so as to avoid repeated statistics of data in the subsequent calculation process and increase the time complexity of the algorithm.

Build a weighted reviewer graph: the graph uses users as graph nodes. When two users comment on the same product together, an edge is established. The weight of the edge reflects the similarity between the two nodes. The construction of the graph and the calculation process of the similarity index are carried out simultaneously: first, add nodes equal to the number of users in the data set to the graph, and each node is named after the user id; Statistics, if the indicator is greater than or equal to 1, further calculate the remaining indicators, and use all indicators to calculate the weight on the edge of the two nodes, and then build an edge connecting the two nodes; if the indicator is 0, do not perform any Operation, continue to calculate the "number of common comments" of the next node combination, until all pairwise node combinations in the graph are traversed.

Spectral clustering to divide groups: as shown in Figure 2, first select the number n of groups to be divided, and then calculate its adjacency matrix and degree matrix from the weighted reviewer graph, and further, from the adjacency matrix and degree matrix The Laplacian matrix of the graph and the normalized Laplacian matrix can be calculated. The next step is to calculate the smallest n eigenvalues and the corresponding n eigenvectors of the Laplacian matrix, combine the n eigenvectors into a matrix f and normalize it by row. Finally, use the K-Means method to cluster f by row, and divide to obtain n candidate groups.

Manual judgment of group category: the embodiment of the present invention selects the discrimination index and the corresponding threshold according to the existing research in the field. In different fields, the basis for judging the category of the group is different.

In this embodiment, a research data set of Yelp, an American review website, is divided, and other common methods of grouping false reviews are selected for comparative experiments. According to the research status in the field, the experiment selected three representative fake comment group indicators: Extreme Rating Ratio (ERR), Repeated Comment Ratio (RCR) and Rating Deviation (RD). As the basis for judging the division effect, in terms of three indicators, the performance of the algorithm of the present invention is better than that of the K-Means clustering, hierarchical clustering and Louvain community discovery algorithms used as control experiments.

The terms describing the positional relationship in the drawings are only for illustrative purposes and cannot be interpreted as limitations on this patent;

Apparently, the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limiting the implementation of the present invention. For those of ordinary skill in the art, other changes or changes in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaustively list all the implementation manners here. All modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (10)

1. A method for classifying false comment groups based on spectral clustering, the method comprising the steps of:

s1: collecting and cleaning comment data of an electronic commerce platform; the metadata used includes: user id, comment id, commodity id, score and comment interaction times;

s2: calculating 5 similarity indexes of common comment times, scoring similarity of the same commodity, user interaction times, user positive scoring proportion and user negative scoring proportion based on the metadata in the S1, wherein the similarity of the scoring proportions between two users is measured by Euclidean distance;

s3: constructing a weighted reviewer graph: each user is a graph node, the users who comment on a product two by two are connected by a undirected edge, and the weight of the edge is obtained by calculation according to 5 indexes obtained by calculation in the step S2;

s4: dividing an adjacency matrix of the graph constructed in the step S3 by a spectral clustering algorithm to obtain a plurality of groups;

s5: the classification of the divided groups is further determined manually by selecting an analysis index and a threshold.

2. The method for classifying a false comment group based on spectral clustering according to claim 1, wherein the calculation formula of the number of common comments in S2 is:

CRT(n ₁ ,n ₂ )＝|P ₁ ∩P ₂ |

wherein ,n₁ ,n ₂ P for two different reviewers ₁ ,P ₂ Respectively n ₁ ,n ₂ A collection of articles for which comments were posted.

3. The method for classifying a false comment group based on spectral clustering according to claim 2, wherein the calculation formula of the scoring similarity of the same commodity in S2 is:

wherein ,

respectively n ₁ ,n ₂ Scoring the item P to release the ith or jth rating, N ₁ ,N ₂ Respectively n ₁ ,n ₂ Number of reviews posted on commodity P.

4. The method for classifying a false comment group based on spectral clustering according to claim 3, wherein the calculation formula of the number of user interactions in S2 is:

wherein ,C_1m ,C _2m Respectively represent n ₁ ,n ₂ Number of m-th interaction actions.

5. The method for classifying a false comment group based on spectral clustering according to claim 4, wherein the calculation formula of the positive scoring proportion of the user in S2 is:

wherein ,S_i And S represents the score of a comment of the user, ΣS ⁰ Number of comments scored {1,2.3.4.5}, ΣS, posted to the user _i ⁰ The number of times the user was published with a score of {4,5 }.

6. The method for classifying a false comment group based on spectral clustering according to claim 5, wherein the calculation formula of the user negative score ratio in S2 is:

7. the method for classifying a false comment group based on spectral clustering according to claim 6, wherein the similarity of the positive score ratio and the negative score ratio of two users is measured by the euclidean distance in S2:

wherein ：r_n1 and r_n2 For reviewer n ₁ ,n ₂ A user positive score proportion or a user negative score proportion.

8. The method for classifying a false comment group based on spectral clustering according to claim 7, wherein S3 includes the steps of:

s3.1: importing all users as graph nodes;

s3.2: the number of times of common comments among every two nodes is calculated by taking every two nodes as a node combination;

s3.3: judging whether the number of the common comments calculated in the step S3.2 is larger than 0; if yes, S3.4 is carried out; if not, returning to S3.2 to perform the next node combination until all node combinations are traversed;

s3.4: calculating the similarity index of the node to the rest; calculating weight values and establishing edges of two nodes;

s3.5: and (5) completing the construction of the weighted reviewer graph.

9. The method for classifying a false comment group based on spectral clustering according to claim 8, wherein the weight calculation formula of the edge in S3 is as follows:

wherein ,ω_ij For the weights on node i and node j, k is the number of metrics, here 5, CRT _ij SPSR is the number of times user i and user j comment together _ij IT for the similarity of user i and user j scoring the same merchandise _ij For both user i and user j,

and />

The approximation of the positive score proportion and the negative score proportion for user i and user j, respectively.

10. The method for classifying a false comment group based on spectral clustering according to claim 1 or 9, wherein S4 includes the steps of:

s4.1: inputting a weighted reviewer graph G, and dividing the number n of clusters;

s4.2: calculating an adjacency matrix a, a degree matrix D and a laplace matrix l=d-a from the weighted reviewer graph G;

s4.3: a normalized laplacian matrix is obtained according to:

NL＝D ^-1/2 (D-A)D ^-1/2 ＝D ^-1/2 LD ^-1/2

s4.4: calculating k minimum characteristic values of NL and corresponding characteristic vectors f, wherein k is the number n of the partition clusters;

s4.5: forming the corresponding feature vectors F into feature matrixes F with the size of v multiplied by k, and standardizing according to the rows, wherein v is the number of samples, namely the number of nodes of the graph G;

s4.6: clustering the normalized feature vectors by using a K-Means method to obtain n candidate groups C= (C) ₁ ,c ₂ …c _n )。

Images