US20180217977A1 - Computer-implemented methods and systems for clustering user reviews and ranking clusters - Google Patents
Computer-implemented methods and systems for clustering user reviews and ranking clusters Download PDFInfo
- Publication number
- US20180217977A1 US20180217977A1 US15/936,273 US201815936273A US2018217977A1 US 20180217977 A1 US20180217977 A1 US 20180217977A1 US 201815936273 A US201815936273 A US 201815936273A US 2018217977 A1 US2018217977 A1 US 2018217977A1
- Authority
- US
- United States
- Prior art keywords
- reviews
- review
- clusters
- term
- frequency values
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G06F17/277—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/355—Class or cluster creation or modification
-
- G06F17/3071—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0282—Rating or review of business operators or products
Definitions
- the present application relates generally to analysis of user reviews and, more particularly, to computer-implemented methods and systems for organizing such reviews into clusters and ranking the clusters so that the reviews may be more meaningfully analyzed.
- An app is a computer program designed to run on a computer device or system, including, without limitation, mobile devices (such as smartphones and tablet computers), desktop computers, wearable computers, connected devices, etc.
- App stores, from which apps can be downloaded, allow app users to create and publicly share their reviews of apps. This allows users to praise app developers for effective new features, but it also allows users to post complaints on a public stage and to expose undesirable elements of apps.
- a method for organizing user reviews for data analysis.
- the method includes the steps of (a) sampling a plurality of reviews to be analyzed from one or more sources; (b) identifying unique terms in each of the plurality of reviews; (c) determining frequency values indicating the number of times each of the unique terms appears in each of the plurality of reviews; (d) adjusting the frequency values for each unique term to account for the rarity of that unique term across the plurality of reviews; (e) calculating the similarities between the reviews based on the frequency values as adjusted in step (d); and (f) grouping the reviews into clusters based on the similarities of the reviews determined in step (e).
- a computer system including at least one processor, memory associated with the at least one processor, and a program supported in the memory for organizing user reviews for data analysis.
- the program contains a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to: (a) sample a plurality of reviews to be analyzed from one or more sources; (b) identify unique terms in each of the plurality of reviews; (c) determine frequency values indicating the number of times each of the unique terms appears in each of the plurality of reviews; (d) adjust the frequency values for each unique term to account for the rarity of that unique term across the plurality of reviews; (e) calculate the similarities between the reviews based on the frequency values as adjusted in (d); and (f) group the reviews into clusters based on the similarities of the reviews determined in (e).
- FIG. 1 is a simplified flow chart illustrating an exemplary method for organizing app reviews into clusters in accordance with one or more embodiments.
- FIG. 2 is a simplified flow diagram illustrating an exemplary method for ranking clusters of reviews in accordance with one or more embodiments.
- FIG. 3 is a screenshot illustrating an exemplary representation of a review cluster shown to a system user in accordance with one or more embodiments.
- FIG. 4 is a screenshot illustrating additional information on a cluster shown to a system user in accordance with one or more embodiments.
- Various embodiments disclosed herein are directed to computer-implemented methods and systems for organizing app reviews into clusters and ordering the clusters so that user review data can be more effectively analyzed by app developers to improve their apps.
- the methods and systems for clustering reviews may be used for organizing generally any feedback related corpus that includes text feedback and a scaled rating such as, e.g., a star or numerical rating.
- FIG. 1 is a simplified flow chart illustrating an exemplary process for organizing app reviews into clusters in accordance with one or more embodiments.
- the reviews may be obtained from app stores or other sites displaying reviews, e.g., by crawling the site.
- a set of reviews collected for an app are sampled, as desired. For instance, a user of the clustering system may choose to sample only the last N reviews or only reviews posted in the last M days.
- Tokenization refers to the process of breaking a stream of text into a set of words.
- each of the words is then reduced to a stem, i.e., to its root meaning. For instance, the words “crashed” and “crashing” would be stemmed to “crash.”
- the stemmed words (and words already in root form) in reviews are referred to herein as “terms.”
- each row identifies a review and each column identifies a unique term, and the matrix specifies the frequency of terms within each review.
- “tf/idf” (term frequency/inverse document frequency) data is calculated for each review in order to determine how important each term is to a review within the set of reviews to control for the fact that some terms are more commonly used than others.
- tf the raw frequency of a term within a review.
- idf log(total number of reviews/total number of reviews containing the term).
- the calculated tf/idf values are placed in respective row/column cells in the term/review matrix.
- the matrix is then normalized.
- the user of the clustering system selects a similarity threshold, which can be based, e.g., on the total number of reviews posted for the app, the total number posted in the last M days, and the total number selected for the sample.
- similarity thresholds range between 0.20-0.35.
- the reviews are divided up into k number of chunks of equal number of reviews. In one or more embodiments, the reviews are separated into chunks based on when each review was posted.
- the similarity of each review is calculated against every other review in the chunk, cos _sim(ri, rj), using standard cosine similarity between each pair of reviews.
- the transitive property is used for an initial pass through reviews for clustering reviews. For example: If ri ⁇ rj, and rj ⁇ rk, then ri, rj, rk form a cluster.
- Clusters 1 and 2 are merged only if sim_score(ri, rn)>avg(sim_score for cluster1) and>avg(sim_score for cluster2); otherwise the clusters are left unmerged. Thus, even though there is a link between cluster 1 and cluster 2, it may not be strong enough to group the clusters together.
- Clusters can also be merged based on cluster centroids within each chunk.
- a cluster similarity threshold, C is first selected based on samplings of apps across markets and categories. The centroid of each cluster is calculated using a standard calculation that takes each review within the cluster and uses the row that represents the review as a point in n-dimensional space. Centroid calculation generally finds the geometric center of all the points.
- a post merge process is performed to collapse clusters across k chunks.
- Clusters again are merged based on cluster centroids.
- a cluster similarity threshold, C is selected based on sampling of apps across markets and categories, C. The centroid of each cluster is calculated. The cosine similarity between clusters based on their centroids is calculated. Clusters with a cluster similarity threshold>C are merged.
- a final cluster list is prepared of all reviews within X number of arbitrary clusters for an app, based on similarity of text and number of reviews per app.
- methods and system are provided for ranking clusters, which are formed as described above.
- FIG. 2 is a simplified flow diagram illustrating an exemplary process for ranking clusters in accordance with one or more embodiments.
- a time bucket is assigned to each cluster.
- the average rating of each cluster is extracted.
- the number of reviews in each cluster is extracted.
- a weight is assigned to each cluster for each of the features identified above with descending priority. Recent time buckets are assigned higher weights than older time buckets. Negative star ratings are assigned higher weights than positive ratings. Higher number of reviews in a cluster are assigned higher weights than a lower number of reviews. Higher similarity scores are assigned higher weights than lower similarity scores.
- the clusters are ordered based on the weighted scores, built by linear function:
- the result of the process is an ordered list of clustered reviews for one specific app.
- FIG. 3 is an exemplary screenshot of a screen or window shown on a computer display to a user of the clustering system.
- the display shows a representation of a review cluster 300 .
- the cluster includes text 302 that is representative of similar text found in the reviews in the cluster.
- the total number of reviews in the cluster are indicated at 304 .
- the time period over which the reviews were posted is indicated at 306 .
- the average star rating of reviews in the cluster is indicated at 308 .
- a user of the system can zoom in on the cluster 300 in FIG. 3 by clicking on it.
- FIG. 4 is an exemplary screenshot showing further information on a selected cluster, where the user can view each of the posted reviews in the cluster.
- the processes of clustering reviews and ranking clusters described above may be implemented in software, hardware, firmware, or any combination thereof.
- the processes are preferably implemented in one or more computer programs executing on a programmable computer including a processor, a storage medium readable by the processor (including, e.g., volatile and non-volatile memory and/or storage elements), and input and output devices.
- Each computer program can be a set of instructions (program code) in a code module resident in the random access memory of the computer.
- the set of instructions may be stored in another computer memory (e.g., in a hard disk drive, or in a removable memory such as an optical disk, external hard drive, memory card, or flash drive) or stored on another computer system and downloaded via the Internet or other network.
- Program code may be applied to input entered using the input device to perform the functions described and to generate output.
- the output may be provided to one or more output devices.
- the computer system may comprise one or more physical machines, or virtual machines running on one or more physical machines.
- the computer server system may comprise a cluster of computers or numerous distributed computers that are
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Accounting & Taxation (AREA)
- General Engineering & Computer Science (AREA)
- Economics (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Computer-implemented methods and systems are disclosed for organizing user reviews, especially computer app reviews, into clusters and ranking the clusters so that the reviews may be more meaningfully analyzed.
Description
- This Application is a Continuation of U.S. application Ser. No. 14/539,623, filed Nov. 12, 2014, entitled “COMPUTER-IMPLEMENTED METHODS AND SYSTEMS FOR CLUSTERING USER REVIEWS AND RANKING CLUSTERS,” incorporated herein by reference in its entirety.
- The present application relates generally to analysis of user reviews and, more particularly, to computer-implemented methods and systems for organizing such reviews into clusters and ranking the clusters so that the reviews may be more meaningfully analyzed.
- An app is a computer program designed to run on a computer device or system, including, without limitation, mobile devices (such as smartphones and tablet computers), desktop computers, wearable computers, connected devices, etc. App stores, from which apps can be downloaded, allow app users to create and publicly share their reviews of apps. This allows users to praise app developers for effective new features, but it also allows users to post complaints on a public stage and to expose undesirable elements of apps.
- Given the steady influx of review data flowing into app stores, app developers need to constantly monitor the incoming stream of user feedback. However, many apps get hundreds and even thousands of reviews a day, and it can be difficult for app developers to keep up with the issues raised by app users.
- In accordance with one or more embodiments, a method is provided for organizing user reviews for data analysis. The method, which is implemented in a computer system, includes the steps of (a) sampling a plurality of reviews to be analyzed from one or more sources; (b) identifying unique terms in each of the plurality of reviews; (c) determining frequency values indicating the number of times each of the unique terms appears in each of the plurality of reviews; (d) adjusting the frequency values for each unique term to account for the rarity of that unique term across the plurality of reviews; (e) calculating the similarities between the reviews based on the frequency values as adjusted in step (d); and (f) grouping the reviews into clusters based on the similarities of the reviews determined in step (e).
- In accordance with one or more further embodiments, a computer system is provided, including at least one processor, memory associated with the at least one processor, and a program supported in the memory for organizing user reviews for data analysis. The program contains a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to: (a) sample a plurality of reviews to be analyzed from one or more sources; (b) identify unique terms in each of the plurality of reviews; (c) determine frequency values indicating the number of times each of the unique terms appears in each of the plurality of reviews; (d) adjust the frequency values for each unique term to account for the rarity of that unique term across the plurality of reviews; (e) calculate the similarities between the reviews based on the frequency values as adjusted in (d); and (f) group the reviews into clusters based on the similarities of the reviews determined in (e).
-
FIG. 1 is a simplified flow chart illustrating an exemplary method for organizing app reviews into clusters in accordance with one or more embodiments. -
FIG. 2 is a simplified flow diagram illustrating an exemplary method for ranking clusters of reviews in accordance with one or more embodiments. -
FIG. 3 is a screenshot illustrating an exemplary representation of a review cluster shown to a system user in accordance with one or more embodiments. -
FIG. 4 is a screenshot illustrating additional information on a cluster shown to a system user in accordance with one or more embodiments. - Various embodiments disclosed herein are directed to computer-implemented methods and systems for organizing app reviews into clusters and ordering the clusters so that user review data can be more effectively analyzed by app developers to improve their apps.
- The methods and systems for clustering reviews may be used for organizing generally any feedback related corpus that includes text feedback and a scaled rating such as, e.g., a star or numerical rating.
- While the exemplary embodiments disclosed herein refer to user reviews for apps, it should be understood that the clustering techniques described herein can be used for analyzing user reviews involving generally any subject matter.
-
FIG. 1 is a simplified flow chart illustrating an exemplary process for organizing app reviews into clusters in accordance with one or more embodiments. The reviews may be obtained from app stores or other sites displaying reviews, e.g., by crawling the site. - At
step 100, a set of reviews collected for an app are sampled, as desired. For instance, a user of the clustering system may choose to sample only the last N reviews or only reviews posted in the last M days. - At
step 102, the text comments in the sampled reviews are tokenized. Tokenization refers to the process of breaking a stream of text into a set of words. - Each of the words is then reduced to a stem, i.e., to its root meaning. For instance, the words “crashed” and “crashing” would be stemmed to “crash.” The stemmed words (and words already in root form) in reviews are referred to herein as “terms.”
- At
step 104, all unique terms in a review are extracted and placed in a term/review matrix. In an exemplary matrix, each row identifies a review and each column identifies a unique term, and the matrix specifies the frequency of terms within each review. - At
step 106, “tf/idf” (term frequency/inverse document frequency) data is calculated for each review in order to determine how important each term is to a review within the set of reviews to control for the fact that some terms are more commonly used than others. -
tf=the raw frequency of a term within a review. -
idf=log(total number of reviews/total number of reviews containing the term). - The calculated tf/idf values are placed in respective row/column cells in the term/review matrix. The matrix is then normalized.
- The user of the clustering system selects a similarity threshold, which can be based, e.g., on the total number of reviews posted for the app, the total number posted in the last M days, and the total number selected for the sample. In one example, similarity thresholds range between 0.20-0.35.
- The reviews are divided up into k number of chunks of equal number of reviews. In one or more embodiments, the reviews are separated into chunks based on when each review was posted.
- At
step 108, the similarity of each review is calculated against every other review in the chunk, cos _sim(ri, rj), using standard cosine similarity between each pair of reviews. - For each pair of reviews, if cos _sim(ri, rj)>sim_threshold, the reviews are clustered at
step 110. - The transitive property is used for an initial pass through reviews for clustering reviews. For example: If ri˜rj, and rj˜rk, then ri, rj, rk form a cluster.
- A merge list is then created in order to merge clusters with sufficiently similar reviews. For instance, if cluster1=[ri, rj, rk] and cluster2=[rl, rm, rn], and ri˜rn, then the clusters are added to merge list.
- Next, the system passes through the merge list. Clusters 1 and 2 are merged only if sim_score(ri, rn)>avg(sim_score for cluster1) and>avg(sim_score for cluster2); otherwise the clusters are left unmerged. Thus, even though there is a link between cluster 1 and cluster 2, it may not be strong enough to group the clusters together.
- Clusters can also be merged based on cluster centroids within each chunk. A cluster similarity threshold, C, is first selected based on samplings of apps across markets and categories. The centroid of each cluster is calculated using a standard calculation that takes each review within the cluster and uses the row that represents the review as a point in n-dimensional space. Centroid calculation generally finds the geometric center of all the points.
- The cosine similarity between clusters based on their centroids is calculated. Clusters with a cluster similarity threshold>C are merged.
- At
step 112, a post merge process is performed to collapse clusters across k chunks. Clusters again are merged based on cluster centroids. A cluster similarity threshold, C, is selected based on sampling of apps across markets and categories, C. The centroid of each cluster is calculated. The cosine similarity between clusters based on their centroids is calculated. Clusters with a cluster similarity threshold>C are merged. - At step 114, a final cluster list is prepared of all reviews within X number of arbitrary clusters for an app, based on similarity of text and number of reviews per app.
- In accordance with one or more further embodiments, methods and system are provided for ranking clusters, which are formed as described above.
-
FIG. 2 is a simplified flow diagram illustrating an exemplary process for ranking clusters in accordance with one or more embodiments. - At step 200, a time bucket is assigned to each cluster. The time buckets are broken down into N chunks, from the most recent to the least recent review in the sample. For example, if the cluster contains a review within the last 24 hours, assign time_bucket=1, and if the cluster contains a review within the last 72 hours, assign time_bucket =3. In each case, the most recent review in cluster a determines the time bucket for that cluster.
- At step 202, the average rating of each cluster is extracted. The star rating of all reviews within the cluster are averaged. For example, if ri=1 star, rj=1 star, rk=5 star, rl=5 star, the average star rating for cluster=3 star.
- At step 204, the number of reviews in each cluster is extracted.
- At step 206, the average similarity score for each cluster is extracted. The similarity rating
- of all reviews within the cluster is averaged, given every pairwise review within the cluster was clustered based on similarity. For example:
-
cos _sim(ri, rj)=0.50, cos _sim(ri, rk)=0.60, average_similarity score for cluster=0.55. - At
step 208, a weight is assigned to each cluster for each of the features identified above with descending priority. Recent time buckets are assigned higher weights than older time buckets. Negative star ratings are assigned higher weights than positive ratings. Higher number of reviews in a cluster are assigned higher weights than a lower number of reviews. Higher similarity scores are assigned higher weights than lower similarity scores. - At
step 210, the clusters are ordered based on the weighted scores, built by linear function: -
Weighted_score=m1 * x1+m2 * x2+m3 * x3+m4 * x4, where mi=weight of feature xi, and xi=value for that cluster for an app. - The result of the process is an ordered list of clustered reviews for one specific app.
- By way of example,
FIG. 3 is an exemplary screenshot of a screen or window shown on a computer display to a user of the clustering system. The display shows a representation of areview cluster 300. The cluster includestext 302 that is representative of similar text found in the reviews in the cluster. The total number of reviews in the cluster are indicated at 304. The time period over which the reviews were posted is indicated at 306. The average star rating of reviews in the cluster is indicated at 308. - A user of the system can zoom in on the
cluster 300 inFIG. 3 by clicking on it. -
FIG. 4 is an exemplary screenshot showing further information on a selected cluster, where the user can view each of the posted reviews in the cluster. - The processes of clustering reviews and ranking clusters described above may be implemented in software, hardware, firmware, or any combination thereof. The processes are preferably implemented in one or more computer programs executing on a programmable computer including a processor, a storage medium readable by the processor (including, e.g., volatile and non-volatile memory and/or storage elements), and input and output devices. Each computer program can be a set of instructions (program code) in a code module resident in the random access memory of the computer. Until required by the computer, the set of instructions may be stored in another computer memory (e.g., in a hard disk drive, or in a removable memory such as an optical disk, external hard drive, memory card, or flash drive) or stored on another computer system and downloaded via the Internet or other network. Program code may be applied to input entered using the input device to perform the functions described and to generate output. The output may be provided to one or more output devices.
- Having thus described several illustrative embodiments, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to form a part of this disclosure, and are intended to be within the spirit and scope of this disclosure. While some examples presented herein involve specific combinations of functions or structural elements, it should be understood that those functions and elements may be combined in other ways according to the present disclosure to accomplish the same or different objectives. In particular, acts, elements, and features discussed in connection with one embodiment are not intended to be excluded from similar or other roles in other embodiments.
- Additionally, elements and components described herein may be further divided into additional components or joined together to form fewer components for performing the same functions. For example, the computer system may comprise one or more physical machines, or virtual machines running on one or more physical machines. In addition, the computer server system may comprise a cluster of computers or numerous distributed computers that are
- Accordingly, the foregoing description and attached drawings are by way of example only, and are not intended to be limiting.
Claims (20)
1. A method of organizing user reviews for data analysis, comprising the steps, each implemented in a computer system, of:
identifying unique terms in a plurality of reviews from a sample set of reviews;
determining frequency values indicating a number of times the unique terms appears in the plurality of reviews, wherein determining the frequency values includes generating a term/review matrix identifying a respective review and a respective unique term, and wherein the matrix specifies the frequency values for the respective unique terms;
adjusting the frequency values for respective unique terms to account for the rarity of that unique term across the plurality of reviews, wherein adjusting the frequency values includes normalizing the values in the matrix based on values associated with the rarity of the unique term;
calculating similarities between the reviews based at least in part on the frequency values once adjusted; and
grouping the reviews into clusters based at least in part on the similarities of the reviews.
2. The method of claim 1 , wherein identifying includes tokenizing text in the plurality of reviews into a set of words and stemming for words not already in root form to generate a set of terms for each review, from which said unique terms are identified.
3. The method of claim 1 , wherein adjusting includes calculating tf/idf values for each review, wherein tf=the raw frequency of a term within a review, and idf=log (total number of reviews/total number of reviews containing the term), and wherein the tf/idf values comprise the frequency values adjusted.
4. The method of claim 3 , further comprising normalizing the term/review matrix.
5. The method of claim 1 , wherein calculating and grouping includes dividing the reviews into a given number of chunks, calculating the similarity of each review against every other review in each chunk, and for each pair of reviews whose similarity exceeds a given threshold, clustering said reviews.
6. The method of claim 5 , further comprising using the transitive property for clustering reviews.
7. The method of claim 5 , further comprising merging clusters with similarly sufficient reviews.
8. The method of claim 7 , wherein the clusters are merged based on cluster centroids within each chunk.
9. The method of claim 5 , further comprising performing a post merge process to collapse clusters across chunks.
10. The method of claim 1 , further comprising ranking the clusters.
11. The method of claim 10 , wherein the clusters are ranked based on a plurality of factors.
12. A computer system, comprising:
at least one processor;
memory associated with the at least one processor; and
a program supported in the memory for organizing user reviews for data analysis, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to:
identify unique terms in a plurality of reviews from a sample set of reviews;
determine frequency values indicating a number of times the unique terms appears in the plurality of reviews, wherein determining the frequency values includes generating a term/review matrix identifying a respective review and a respective unique term, and wherein the matrix specifies the frequency values for the respective unique terms;
adjust the frequency values for respective unique terms to account for the rarity of that unique term across the plurality of reviews, wherein adjusting the frequency values includes normalizing the values in the matrix based on values associated with the rarity of the unique term;
calculate similarities between the reviews based at least in part on the frequency values once adjusted; and
group the reviews into clusters based at least in part on the similarities of the reviews.
13. The computer system of claim 12 , wherein identifying includes tokenizing text in the plurality of reviews into a set of words and stemming each word not already in root form to generate a set of terms for each review, from which said unique terms are identified.
14. The computer system of claim 12 , wherein adjusting includes calculating tf/idf values for each review, wherein tf =the raw frequency of a term within a review, and idf=log (total number of reviews/total number of reviews containing the term), and wherein the tf/idf values comprise the frequency values adjusted in (d).
15. The computer system of claim 14 , wherein the program further comprises instructions for normalizing the term/review matrix.
16. The computer system of claim 12 , calculating and grouping includes dividing the reviews into a given number of chunks, calculating the similarity of each review against every other review in each chunk, and for each pair of reviews whose similarity exceeds a given threshold, clustering said reviews.
17. The computer system of claim 16 , wherein the program further comprises instructions for using the transitive property for clustering reviews.
18. The computer system of claim 16 , wherein the program further comprises instructions for merging clusters with similarly sufficient reviews.
19. The computer system of claim 18 , wherein the clusters are merged based on cluster centroids within each chunk.
20. The computer system of claim 19 , wherein the program further comprises instructions for performing a post merge process to collapse clusters across chunks.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/936,273 US20180217977A1 (en) | 2014-11-12 | 2018-03-26 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/539,623 US9928233B2 (en) | 2014-11-12 | 2014-11-12 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
US15/936,273 US20180217977A1 (en) | 2014-11-12 | 2018-03-26 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/539,623 Continuation US9928233B2 (en) | 2014-11-12 | 2014-11-12 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180217977A1 true US20180217977A1 (en) | 2018-08-02 |
Family
ID=55912355
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/539,623 Active 2035-10-18 US9928233B2 (en) | 2014-11-12 | 2014-11-12 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
US15/936,273 Abandoned US20180217977A1 (en) | 2014-11-12 | 2018-03-26 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/539,623 Active 2035-10-18 US9928233B2 (en) | 2014-11-12 | 2014-11-12 | Computer-implemented methods and systems for clustering user reviews and ranking clusters |
Country Status (1)
Country | Link |
---|---|
US (2) | US9928233B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021129123A1 (en) * | 2019-12-25 | 2021-07-01 | 华为技术有限公司 | Corpus data processing method and apparatus, server, and storage medium |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160379281A1 (en) * | 2015-06-24 | 2016-12-29 | Bank Of America Corporation | Compliance violation early warning system |
US11366821B2 (en) * | 2018-05-25 | 2022-06-21 | Salesforce.Com, Inc. | Epsilon-closure for frequent pattern analysis |
US20230029643A1 (en) * | 2021-07-26 | 2023-02-02 | International Business Machines Corporation | Mapping of heterogeneous data as matching fields |
CN113704330B (en) * | 2021-09-23 | 2024-06-07 | 北京网界科技有限公司 | Fair assessment system and method thereof |
CN116506680B (en) * | 2023-06-26 | 2023-09-19 | 北京万物镜像数据服务有限公司 | Comment data processing method and device for virtual space and electronic equipment |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150051948A1 (en) * | 2011-12-22 | 2015-02-19 | Hitachi, Ltd. | Behavioral attribute analysis method and device |
Family Cites Families (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6901582B1 (en) | 1999-11-24 | 2005-05-31 | Quest Software, Inc. | Monitoring system for monitoring the performance of an application |
US8185487B2 (en) | 2001-02-12 | 2012-05-22 | Facebook, Inc. | System, process and software arrangement for providing multidimensional recommendations/suggestions |
US6721737B2 (en) | 2001-04-04 | 2004-04-13 | International Business Machines Corporation | Method of ranking items using efficient queries |
US7337124B2 (en) | 2001-08-29 | 2008-02-26 | International Business Machines Corporation | Method and system for a quality software management process |
US7516438B1 (en) | 2001-09-12 | 2009-04-07 | Sun Microsystems, Inc. | Methods and apparatus for tracking problems using a problem tracking system |
US20030065597A1 (en) | 2001-10-01 | 2003-04-03 | Smith Steven B. | Creating and distributing a software application |
AU2003217737A1 (en) | 2002-01-25 | 2003-12-12 | Bdellium Inc. | Method of analyzing investments using overlapping periods |
US8676853B2 (en) | 2003-02-27 | 2014-03-18 | Hewlett-Packard Development Company, L.P. | System and method for software reuse |
US20060149575A1 (en) | 2005-01-04 | 2006-07-06 | Srinivas Varadarajan | Software engineering process monitoring |
US20060195566A1 (en) | 2005-02-25 | 2006-08-31 | Hurley Mark E | Method and system for taking remote inventory in a network |
US7356590B2 (en) | 2005-07-12 | 2008-04-08 | Visible Measures Corp. | Distributed capture and aggregation of dynamic application usage information |
US7849447B1 (en) | 2005-11-21 | 2010-12-07 | Verizon Laboratories Inc. | Application testing and evaluation |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US9367823B1 (en) | 2007-11-09 | 2016-06-14 | Skyword, Inc. | Computer method and system for ranking users in a network community of users |
US8099332B2 (en) | 2008-06-06 | 2012-01-17 | Apple Inc. | User interface for application management for a mobile device |
US9898675B2 (en) | 2009-05-01 | 2018-02-20 | Microsoft Technology Licensing, Llc | User movement tracking feedback to improve tracking |
CN102483753A (en) | 2009-05-27 | 2012-05-30 | 谷歌公司 | Computer Application Data In Search Results |
US8972391B1 (en) | 2009-10-02 | 2015-03-03 | Google Inc. | Recent interest based relevance scoring |
US20110258067A1 (en) | 2010-04-14 | 2011-10-20 | Michael Rowell | Online marketplace for information and intangible goods |
US9183560B2 (en) | 2010-05-28 | 2015-11-10 | Daniel H. Abelow | Reality alternate |
JP4898938B2 (en) | 2010-06-08 | 2012-03-21 | 株式会社ソニー・コンピュータエンタテインメント | Information providing system and information providing method |
US8694779B2 (en) | 2010-08-13 | 2014-04-08 | Bmc Software, Inc. | Monitoring based on client perspective |
US9805022B2 (en) | 2010-12-09 | 2017-10-31 | Apple Inc. | Generation of topic-based language models for an app search engine |
US9733934B2 (en) | 2011-03-08 | 2017-08-15 | Google Inc. | Detecting application similarity |
US20120265814A1 (en) | 2011-04-14 | 2012-10-18 | Stilianos George Roussis | Software Application for Managing Personal Matters and Personal Interactions through a Personal Network |
US20130013492A1 (en) | 2011-05-17 | 2013-01-10 | Nelson Chris R | Method and system for refunds and/or credits responsive to consumer dissatisfaction with software downloads |
GB2511221B (en) | 2011-12-09 | 2020-09-23 | Veracyte Inc | Methods and compositions for classification of samples |
US8626774B2 (en) | 2012-01-23 | 2014-01-07 | Qualcomm Innovation Center, Inc. | Location based apps ranking for mobile wireless computing and communicating devices |
US8543576B1 (en) | 2012-05-23 | 2013-09-24 | Google Inc. | Classification of clustered documents based on similarity scores |
US9704171B2 (en) | 2012-06-05 | 2017-07-11 | Applause App Quality, Inc. | Methods and systems for quantifying and tracking software application quality |
US9672023B2 (en) | 2013-03-15 | 2017-06-06 | Apple Inc. | Providing a unified update center for security software updates and application software updates |
US20140282493A1 (en) | 2013-03-15 | 2014-09-18 | Quixey, Inc | System for replicating apps from an existing device to a new device |
-
2014
- 2014-11-12 US US14/539,623 patent/US9928233B2/en active Active
-
2018
- 2018-03-26 US US15/936,273 patent/US20180217977A1/en not_active Abandoned
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150051948A1 (en) * | 2011-12-22 | 2015-02-19 | Hitachi, Ltd. | Behavioral attribute analysis method and device |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021129123A1 (en) * | 2019-12-25 | 2021-07-01 | 华为技术有限公司 | Corpus data processing method and apparatus, server, and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US9928233B2 (en) | 2018-03-27 |
US20160132504A1 (en) | 2016-05-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180217977A1 (en) | Computer-implemented methods and systems for clustering user reviews and ranking clusters | |
US11526809B2 (en) | Primary key-foreign key relationship determination through machine learning | |
US10592837B2 (en) | Identifying security risks via analysis of multi-level analytical records | |
CN111984689B (en) | Information retrieval method, device, equipment and storage medium | |
Rabosky et al. | FiSSE: a simple nonparametric test for the effects of a binary character on lineage diversification rates | |
US10885056B2 (en) | Data standardization techniques | |
WO2018103718A1 (en) | Application recommendation method and apparatus, and server | |
US20170300575A1 (en) | Methods and systems for quantifying and tracking software application quality | |
CN112365171B (en) | Knowledge graph-based risk prediction method, device, equipment and storage medium | |
US11379466B2 (en) | Data accuracy using natural language processing | |
US10191985B1 (en) | System and method for auto-curation of Q and A websites for search engine optimization | |
US10748166B2 (en) | Method and system for mining churn factor causing user churn for network application | |
US10915537B2 (en) | System and a method for associating contextual structured data with unstructured documents on map-reduce | |
WO2020215675A1 (en) | Method and apparatus for building medical treatment database, and computer device and storage medium | |
US9299035B2 (en) | Iterative refinement of pathways correlated with outcomes | |
Fung et al. | Automation of QIIME2 metagenomic analysis platform | |
Nicolau et al. | Investigating laboratory and everyday typing performance of blind users | |
Abdrabo et al. | Enhancing big data value using knowledge discovery techniques | |
CN104685493A (en) | Dictionary creation device for monitoring text information, dictionary creation method for monitoring text information, and dictionary creation program for monitoring text information | |
US10353927B2 (en) | Categorizing columns in a data table | |
WO2019080419A1 (en) | Method for building standard knowledge base, electronic device, and storage medium | |
CN112395401A (en) | Adaptive negative sample pair sampling method and device, electronic equipment and storage medium | |
CN115048523A (en) | Text classification method, device, equipment and storage medium | |
CN111191089B (en) | Data visualization method, system, equipment and medium based on medical care scene | |
CN112926297A (en) | Method, apparatus, device and storage medium for processing information |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: APPLAUSE APP QUALITY, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YOUNG, HEIDI A.;STREDWICK, JASON M.;MAVIANAKERE, YASHAS;SIGNING DATES FROM 20141119 TO 20141124;REEL/FRAME:045759/0857 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |