US20050034071A1 - System and method for determining quality of written product reviews in an automated manner - Google Patents

System and method for determining quality of written product reviews in an automated manner Download PDF

Info

Publication number
US20050034071A1
US20050034071A1 US10636966 US63696603A US2005034071A1 US 20050034071 A1 US20050034071 A1 US 20050034071A1 US 10636966 US10636966 US 10636966 US 63696603 A US63696603 A US 63696603A US 2005034071 A1 US2005034071 A1 US 2005034071A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
user review
determining
quality
parameter
method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10636966
Other versions
US7363214B2 (en )
Inventor
Timothy Musgrove
Robin Walsh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CBS Interactive Inc
Original Assignee
CNET NETWORKS Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20Handling natural language data
    • G06F17/27Automatic analysis, e.g. parsing
    • G06F17/2765Recognition
    • G06F17/277Lexical analysis, e.g. tokenisation, collocates
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20Handling natural language data
    • G06F17/27Automatic analysis, e.g. parsing
    • G06F17/2785Semantic analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06QDATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce, e.g. shopping or e-commerce

Abstract

A system and method for determining quality of written product reviews to distinguish the user reviews for further use or processing. In one embodiment, the opinion analyzer system includes a processor, a parser module, an prose analyzer module, a characteristic features database, and a language value rules database. In another embodiment, the method comprises the steps of parsing language of a portion of a user review, extracting characteristic feature from the user review, determining a quality parameter based on the extracted characteristic feature, determining a raw score based on the quality parameter, and determining quality of the user review based on the raw score.

Description

    BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • The present invention is directed to a system and method for determining quality of written user reviews or professional reviews of products or services. In particular, the present invention is directed to a system and method that allows determination of the quality of user reviews in an automated manner so that user reviews may be distinguished from each other for further use or processing.
  • 2. Description of Related Art
  • Many different products and services are provided from product and service vendors. For example, manufacturers of a particular category of product offer various models in their product line, each model targeting a particular group of users and/or meeting the specific needs of a market segment. In particular, manufacturers of automobiles, vacuum cleaners, cameras, computers, etc., all generally manufacture a variety of models of their products. In many instances, each model from each manufacturer and models from differing manufacturers have different features and/or attributes.
  • For example, in the product category of vacuum cleaners, various models having different combinations of features and/or attributes are presently available. These features and/or attributes for vacuum cleaners include bag/bagless operation, motor power, attachments, exhaust filtration, price, etc. In another example, for the product category of digital cameras, features and/or attributes include optical and digital zoom capability, pixel count, the presence, type and size of a view screen, flash capacity, memory capacity, price, etc.
  • Of course, this is also applicable to providers of services as well in that they typically provide numerous different services in their field. Such providers of services may include telecommunications companies such as wireless service providers, utilities such as cable and satellite broadcast providers, travel companies including airlines, cruise lines and travel agencies, etc.
  • In the products arena, the vast number of manufacturers and models available for each product category, and the disparity in features and/or attributes between the products of a product category, can make a consumer's purchasing decision very difficult. Of course, analogous difficulties would be encountered by consumers of services as well.
  • Companies such as CNET Networks, Inc. (hereinafter “CNET”) which operates www.cnet.com™ provide information regarding consumer and technology oriented products such as electronics and computer products for buyers, sellers, and suppliers of technology, as well as any other interested user. In addition to providing raw data and information, many different products in a particular product category are evaluated by editors of CNET for various features and/or attributes and rated. Moreover, the editors often provide written narrative product summaries that highlight various features of the particular product, and discuss strengths and weaknesses of the reviewed product in comparison to other comparable products. The information and narrative product summaries provided by CNET and others regarding various products in a product category may be used by consumers to facilitate potential purchase decisions.
  • Due to the large number of products available in the market place, providing information and narrative product summaries for all of the products that are available can be very costly. In addition, actual user's experience with a particular product or service may be different than the experiences of the editors that provide such product reviews and summaries.
  • Various websites provide forums for users to review the various products, merchants from which these products can be purchased, and/or services. For example, the website www.eopinions.com™ allows users of various products to provide reviews and opinions regarding products and services identified in various categories. Moreover, websites such as www.mysimon.com™ and www.yahoo.com™ provide shopping links and search features which allows a consumer to purchase products from one or more on-line merchants. These websites further provide links to allow actual users of the products to provide their review, and allows potential consumers to view these reviews.
  • Consequently, the user reviews that are provided by actual consumers of products and services further provide valuable information which can be used by consumers to facilitate potential purchase decisions. Moreover, the user's opinion information that is provided in such user reviews contain information regarding consumer experience with the product or service being reviewed which is very valuable to the product manufacturers and/or service provider.
  • Article entitled “Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews” by Dave et al. discloses a method for automatically distinguishing between positive and negative reviews regarding a particular product by focusing on the semantic orientation of words in the review. This reference is primarily directed to the determination of whether a particular review of a product is positive or negative, and does not address whether a particular review is of high or low quality and helpfulness.
  • However, the user reviews vary significantly in their quality and their helpfulness, whether the review is positive or negative. Whereas one user review may provide valuable, well phrased opinions of a user regarding a product, other reviews may be poorly written and not be very helpful. Some websites such as www.amazon.com™ further allows the readers of the user reviews to rate the reviews viewed, for example, as being helpful or not helpful. However, there are no known methods or systems for facilitating substantially automated synthesis and/or analysis of the opinion information that is provided in such user reviews so as to allow determination of the quality of user reviews so that user reviews of varying quality can be distinguished from one another for further use or processing.
  • Therefore, there exists an unfulfilled need for methods and systems for facilitating synthesis and/or analysis of the opinion information that is provided in user reviews. In particular, there exists an unfulfilled need for such methods or systems that facilitates determination of the quality of user reviews so that user reviews can be distinguished for further use or processing.
  • SUMMARY OF THE INVENTION
  • In view of the foregoing, one aspect of the present invention is a method for determining quality of user reviews to distinguish the user reviews for further use or processing. In accordance with one embodiment, the method comprises the steps of parsing text of at least a portion of a user review, extracting at least one characteristic feature from the user review that is indicative of quality of the user review, determining at least one quality parameter for the user review based on the extracted characteristic feature, determining a raw score for the at least one quality parameter, and determining quality of the user review based on the raw score.
  • Another aspect of the present invention is a system for determining quality of user reviews to distinguish the user reviews for further use or processing. In one embodiment, the system comprises a processor, a parser module adapted to parse text of at least a portion of a user review and to extract a characteristic feature indicative of the quality of the user review, and a prose analyzer module adapted to determine at least one quality parameter based on the characteristic feature, determine a raw score for the at least one quality parameter, and to determine quality of the user review based on the raw score.
  • Yet another aspect of the present invention is a computer readable medium for determining quality of user reviews to distinguish the user reviews for further use or processing. In one embodiment, the medium comprises instructions for parsing text of at least a portion of a user review, instructions for extracting at least one characteristic feature from the user review that is indicative of quality of the user review, instructions for determining at least one quality parameter for the user review based on the extracted characteristic feature, instructions for determining a raw score for the at least one quality parameter, and instructions for determining quality of the user review based on the raw score.
  • These and other aspects of the present invention will become more apparent from the following detailed description of the embodiments of the present invention when viewed in conjunction with the accompanying drawings.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a schematic illustration of an opinion analyzer system for determining the quality of user reviews in accordance with one embodiment of the present invention.
  • FIG. 2 is a flow diagram illustrating a method for determining the quality of user reviews in accordance with another aspect of the present invention.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
  • FIG. 1 shows a schematic illustration of an opinion analyzer system 10 for determining the quality of written product reviews in accordance with one embodiment of the present invention. As will be evident from the discussions set forth herein below, the opinion analyzer system 10 is adapted to substantially automatically determine the quality of user reviews. The present invention allows high quality user reviews to be distinguished from poor quality user reviews so that further use and/or processing of the user reviews is facilitated.
  • Initially, as used herein and claimed, the term “user review” and variations thereof, should be understood to broadly mean any written product review or prose text with the primary thematic focus is to offer an evaluative opinion on a product, regardless of the status of the reviewer that generated the user review. For example, the user review may be a written product review from the actual user of the product such as the reviews provided in www.eopinions.com™ previously noted. The user review may also be a written product review that are written by professionals whose occupation is to write reviews or opinion columns relating to new products in the marketplace. Furthermore, the user review may be snippets from Forum discussion “threads”, etc.
  • Moreover, it should also be noted that the opinion analyzer system 10 is described herein below as analyzing product user reviews, it should be understood that the opinion analyzer system 10 may be used to analyze user reviews regarding services as well. Thus, the term “product” and variations thereof, should be construed and understood herein to encompass services.
  • The opinion analyzer system 10 of the illustrated embodiment includes a processor 12 that is connected to a parser module 14, prose analyzer module 16, a submitter profiling module 18, a characteristic features database 20, and a language value rules database 22, the functions of each of these components being described in further detail below. The processor 12 of the opinion analyzer system 10 is provided to fulfill processing requirements of the parser module 14, the prose analyzer module 16, and the submitter profiling module 18.
  • The opinion analyzer system 10 of FIG. 1 may be implemented with any type of hardware and/or software, and may be a pre-programmed general purpose computing device. For example, the opinion analyzer system 10 may be implemented using a server, a personal computer, a portable computer, a thin client, or any suitable device or devices. The opinion analyzer system 10 and/or components of the opinion analyzer system 10 may be a single device at a single location or multiple devices at a single, or multiple, locations that are connected together using any appropriate communication protocols over any communication medium such as electric cable, fiber optic cable, or in a wireless manner.
  • It should also be noted that the opinion analyzer system 10 in accordance with one embodiment of the present invention is illustrated and discussed herein as having a plurality of modules and databases which perform particular functions. It should be understood that these modules and databases are merely schematically illustrated based on their function for clarity purposes only, and do not necessary represent specific hardware or software. In this regard, these modules and/or databases may be hardware and/or software implemented to substantially perform the particular functions explained. Moreover, the modules and/or databases may be combined together within the opinion analyzer system 10, or divided into more components based on the particular function desired. Thus, the present invention as schematically embodied in FIG. 1 should not be construed to limit the opinion analyzer system 10 of the present invention.
  • The parser module 14 of the opinion analyzer system 10 in accordance with the illustrated embodiment is adapted to parse through at least portions of one or more user reviews 2. The user reviews 2 may be electronic documents or files containing the opinions of a user regarding a product that is being reviewed. The parser module 14 extracts “characteristic features” from the user reviews 2 which are indicative of quality of the user reviews 2. In this regard, the parser module 14 accesses the characteristic features database 20 that stores the various characteristic features in various files as described in further detail below.
  • The parser module 14 in accordance with one embodiment may be adapted to require particular conditions to be met by the text pattern extracted from the user review before determining that the extracted text pattern is a characteristic feature. For example, the parser module 14 may specify that certain text, or variations thereof, be present within a predetermined number of words of the characteristic feature. The text patterns in the user review 2 that are extracted are analyzed by the prose analyzer module 16 to determine whether, the characteristic features are rule text patterns that is positively or negatively critical, nullifier text patterns, enhancer text patterns, comparative text patterns, experiential text patterns, or jargon text patterns, as previously described, or some other characteristic feature.
  • Characteristic features may be text or character strings and/or patterns (collectively referred to herein as text patterns) that are indicative of the quality of the user's review for the product being reviewed. The text pattern may be a wildcard based pattern having one or more wildcards therein to allow variations and inflections within the text pattern. The characteristic features database 20 may comprise a plurality of files for storing categories of characteristic features that are indicative of the quality of the user's review. For example, the characteristic features database 20 may be provided with a rules file that contains positively critical text patterns indicating that the user is pleased with, or otherwise satisfied with, the product being reviewed. Such rule text patterns may include, for example, “good”, “outstanding”, “better”, etc. The rules file may further contain negatively critical text patterns indicating that the user is displeased with, or otherwise dissatisfied with the product being reviewed, together with negative values associated with the with the text patterns which indicate the degree of displeasure or dissatisfaction. Such rule text patterns may include, for example, “bad”, “poor”, “worse”, etc.
  • The characteristic features database 20 may also include a nullifier file which contains nullifier text patterns that nullify the effect of positively or negatively critical text patterns. For example, the nullifier file may include text patterns such as “not”, “instead of”, etc. The characteristic features database 20 may further include an enhancers file in which enhancer text patterns are stored. The enhancer text patterns magnify the effect of positively or negatively critical text patterns and may include, for example, “very”, “highly”, “especially”, etc. Alternatively, or in addition thereto, the characteristic features database 20 may further include a comparativeness file in which comparative text patterns indicating a comparison between product or service reviewed by the user review 2, and another comparable product or service are stored. Such text patterns may include, for example, “comparison”, “contrast”, etc. The comparativeness file may also include brand names and/or models of the another comparable product.
  • Furthermore, the characteristic features database 20 may also include an experience file in which experience text patterns indicating actual experience in usage of product reviewed by the user review are stored. Such text patterns may include, for example, “use”, “experience”, “application”, etc. Moreover, the characteristic features database 20 may also include a jargon file in which jargon text patterns that are considered to be jargons for the category of products of the user review are stored. For example, the text pattern “Zebra stripes” may be associated with the product category of camcorders and stored in the jargon file.
  • Of course, the above noted and described characteristic features are merely examples thereof, and characteristic features may be any text patterns that are indicative of the quality of the user's opinion. Moreover, not all of the above discussed characteristic features need be provided and/or used in other embodiments of the present invention. However, utilizing the characteristic features noted would enhance accurate determination of the quality of the user review 2. In addition, in the above example, the characteristic features are stored in various files in the characteristic features database 20 that is connected to, and accessed by, the parser module 14 via the processor 12 of the opinion analyzer system 10. However, it should be noted that the characteristic features may stored in any appropriate manner, for example, as a list of text patterns in the characteristic features database 20.
  • In accordance with the present embodiment, the prose analyzer module 16 of the opinion analyzer system 10 analyzes the characteristic feature to allow determination of the quality of the user review 2. For instance, the prose analyzer module 16 determines the number of occurrences for each characteristic feature in the user review. The prose analyzer module 16 also determines various quality parameters that are indicative of the quality of the user reviews 2 being evaluated based on the characteristic features.
  • In particular, in the illustrated embodiment, the prose analyzer module 16 is adapted to determine a “positive parameter” which is the total number of positively critical text patterns, and a “negative parameter” which is negatively critical text patterns in the user review 2 being analyzed.
  • Using the determined positive parameter and negative parameter, the prose analyzer module 16 in accordance with the present embodiment further determines a “balance” parameter for the user review which provides an indication of whether the user review is excessively positive or excessively negative. The balance parameter may be determined by calculating the difference between number of positively critical text patterns of the user review, and number of negatively critical text patterns of the user review. The balance parameter provides an indication of any bias of the review who wrote the user review.
  • The prose analyzer module 16 in accordance with the present embodiment is also be adapted to determine a “depth” parameter which provides an indication of the substantive quality of the user review 2 being evaluated. The depth parameter may be determined by calculating a minimum of the absolute values of the positive parameter of the user review (i.e. the number of positively critical text patterns), and the negative parameter of the user review (i.e. the number of negatively critical text patterns).
  • In addition, the prose analyzer module 16 determines the “comparativeness” parameter by calculating the number of comparative text patterns in the user review that are represented in the comparativeness file of the characteristic features database 20. Thus, the comparativeness parameter would provide an indication of the quantity of text patterns that are in the user review being analyzed as compared to discussion of other products, models, etc.
  • Furthermore, an “experience” parameter is determined by the prose analyzer module 16 by calculating the number of experiential text patterns in the user review that are represented in the experiential file of the characteristic features database 20. Thus, the experience parameter provides an indication of the quantity of text patterns that are in the user review being analyzed which indicate ownership, experience, or use of the product being analyzed.
  • A “richness” parameter may also be determined by the prose analyzer module 16 for the user review being analyzed, this parameter being indicative of how much actual review information is provided in the user review. In one implementation, the richness parameter may be determined by calculating a sum of the positively critical text patterns, negatively critical text patterns, comparative text patterns, and experiential text patterns.
  • Moreover, the prose analyzer module 16 may be further adapted to determine a “verbosity” parameter which provides an indication of the quality of the user review as compared to its length. This verbosity parameter may be determined by calculating ratio of the richness parameter to the total length of the user review.
  • A “conciseness” parameter may also be calculated by the prose analyzer module 16 which provides an indication of whether the user review is concisely reviewing the product in the user review or the review is unnecessarily long. The conciseness parameter may be determined by calculating how close the length of the user review approximates a predetermined ideal length for a user review.
  • In addition, a “focus” parameter may be calculated which provides an indication of the extent to which the user review focuses mainly on the product being reviewed, instead of on other topics, for example, comparative products. The focus parameter may be determined by calculating the length of the user review that actually discusses the product being reviewed in proportion to the length of the user review that does not actually discuss the product being reviewed.
  • Further, the prose analyzer module 16 may be adapted to determine a “grammar” parameter that is indicative of the extent to which the user review avoids grammatical errors. This provides an indication of whether the reviewer was careful or careless during the generation of the user review being analyzed. The grammar parameter may be implemented using any standard grammar checking method known in the art.
  • Finally, the prose analyzer module 16 of the present embodiment is also adapted to determine a “jargon” parameter which is indicative of the extent to which the user review utilizes jargon that is related to the product. The jargon parameter may be simply determined by counting the number of jargon text patterns in the user review that are stored in the jargon file for the category of products of the characteristic features database 20.
  • Based on the above described quality parameters, the prose analyzer module 16 in accordance with the present embodiment calculates a raw score for each of the quality parameters of the user review being analyzed. In particular, the prose analyzer module 16 calculates a raw score for balance, depth, comparativeness, experience, richness, verbosity, conciseness, focus, grammar, and jargon parameters. For certain quality parameters, the raw scores may merely be the actual number of characteristic features that are present in the particular user review being analyzed. For other quality parameters, the raw scores may be a processed value derived from various characteristic features for each of the quality parameters.
  • However, in accordance with one preferred embodiment of the present invention, the raw score for various quality parameters are calculated bases on rule-based point assignments from the language value rule database 22. The language value rule database 22 includes various rules specifying keywords, phrases, and expressions that specify an antecedent condition which, if found, triggers the prose analyzer module 16 to assign a tentative number of points (positive or negative) to each of the characteristic features for each of the quality parameters.
  • For example, regarding the experience parameter, rules of the language value rule database 22 may specify “I ? used |+3” which instructs the prose analyzer module 16 to assign +3 points if the text pattern occurs where “I” and “used” is present, together with any word therein between, as represented by “?”. In another example, regarding the experience parameter, rules of the language value rule database 22 may also specify “my ? experience |+2” which instructs the prose analyzer module 16 to assign +3 points if this text pattern occurs. Typically, many rules would be stored in the language value rule database 22 for each of the characteristic features of each of the quality parameters, rules preferably including verbal stemming and other syntactic options. Of course, the point assignment as described above is merely one example and any other point assignment methods may be used for the various quality parameters. For example, various look up tables and/or equations for deriving the point assignments may be provided.
  • The points which are assigned to the various quality parameters for each of the characteristic features may be diminished or increased by the prose analyzer module 16 based on the presence of a nullifier text pattern or an enhancer text pattern. As previously described, enhancer text patterns and nullifier text patterns indicate the reduction or intensification of degree in the content of positively or negatively critical text patterns. For example, “hardly” may be a nullifier text pattern and “extremely” may be an enhancer text pattern. Thus, if the characteristic feature “fast flash” for a camera results in assignment of +2 points, the presence of the text pattern “extremely” before the characteristic feature may result in an additional +1 point for this characteristic feature. Conversely, the presence of the text pattern “hardly” may result in diminishing the assigned points by −1 or −2.
  • Moreover, a fall-off factor may be used by the prose analyzer module 16 to decrease the raw score for a particular quality parameter if there are repetitive instances of a particular text pattern. This allows the prose analyzer module 16 to take into consideration, whether the reviewer of the user review 2 being analyzed is overly zealous in his/her opinion regarding the product. In particular, the fall-off factor may be used to diminish points assigned to a particular characteristic feature when the user review is repetitive, or uses repetitive nullifier text patterns and/or repetitive enhancer text patters.
  • For example, “very, very, very fast flash” which is a characteristic feature that has three enhancer text patterns, each repetitive instance of the enhancer text pattern “very” enhances the point assigned to the characteristic feature by a reduced amount so that there will be increasing less enhancement effect. By decreasing the raw score for repeated instances of a characteristic feature, the affects of such user bias on the determination of the user review's quality is minimized. In accordance with one implementation, the fall-off value may be between zero and one and multiplied by itself for each repeated instance. Thus, in the above example, if the fall-off value is 0.5, then the second occurrence of the enhancer text pattern “very” will increase the assigned points for the characteristic feature by 50% of the increase caused by the first instance of the same enhancer text pattern. Similarly, the third instance of the enhancer text pattern will increase the assigned points for the characteristic feature by 25% (since 0.5×0.5=0.25).
  • Furthermore, in accordance with one embodiment of the present invention, a predetermined ideal raw score is provided for each of the quality parameters, together with a predetermined high differential value, and a predetermined low differential value. The prose analyzer module 16 determines a functional score for each of the quality parameters based on the raw score and the predetermined ideal raw score, as well as the predetermined high and low differential values.
  • In particular, the functional score is determined in the present embodiment by the prose analyzer module 16 by calculating the difference between the raw score and the predetermined ideal raw score. If the calculated difference is zero (i.e. raw score for the quality parameter of a user review 2 corresponds to the ideal raw score), the prose analyzer module 16 assigns a functional value, for example, one point. The prose analyzer module 16 assigns a zero functional value (zero points) if the calculated difference is below the predetermined low differential value, and also assigns a zero functional value (zero points) if the calculated differential value is above the predetermined high differential value.
  • Furthermore, the prose analyzer module 16 is adapted to calculate and assign an apportioned functional value if the calculated difference between the raw score and the idealized raw score is greater than zero, but is between the predetermined low differential value and the predetermined high differential value. The apportioned functional value may be apportioned in any appropriate manner. For example, in accordance with one implementation, the apportioned functional value is apportioned proportionately based on the raw score relative to the predetermined idealized raw score and the predetermined high differential value or the predetermined low differential value. Of course, this is merely one method of apportioning the functional values, and other methods may be used in other embodiments.
  • Then, the prose analyzer module 16 in accordance with the illustrated embodiment of the opinion analyzer system 10 calculates the final quality score based on the functional value of the various quality parameters. Thus, the final quality score takes in to consideration, all of the various quality parameters discussed above in determining the actual quality of the user review 2 being analyzed. The final quality score may be calculated in any appropriate manner by processing the functional scores for each of the quality parameters described above. For example, in one embodiment, the functional scores of the quality parameters may be simply added or multiplied together to derive the final quality score. The final quality score may also be normalized scores on a scale of 0 to 100 relative to the final quality scores of the other user reviews 2.
  • Furthermore, weighing factors may be provided for each of the functional scores to allow one or more quality parameters to be emphasized or de-emphasized relative to other quality parameters. In such an implementation, the weighing factors may be multiplied together with the corresponding functional scores. The functional scores may then be added together or otherwise processed to derive the final quality score.
  • As can be appreciated, the use of weighing factors may be particularly useful to bias the final quality scores of user reviews based on the intended audience. For example, if a more professional and technology savvy audience is targeted, weighing factors for the jargon parameter can be increased so that the functional score associated with the jargon parameter is increased to thereby increase the final quality score of the user review. Moreover, the conciseness parameter may be emphasized so that more in-depth user reviews are favored over brief user reviews. If a novice audience is targeted, converse weighing factors may be provided. Of course, the above discussed are merely examples of methods for determining the final quality score and any appropriate methods may be used in other embodiments.
  • In accordance with another embodiment, the determined final quality score for the user reviews may be further modified by the submitter profiling module 18 of the opinion analyzer system 10 to increase or decrease the final quality score for the user review based on information associated with the reviewer that provided the user review, i.e. author of the review. For example, if the user providing the user review is considered to be especially knowledgeable regarding a particular category of products and/or services, the final quality score of the user review may be increased.
  • Such information associated with the reviewer may be the reviewer's e-mail address so that if the reviewer has a real e-mail name and address like Bob.Jones@IBM.com, the review provided by this individual may be weighed more so that the review is scored higher as compared to reviews provided by a reviewer with an anonymous e-mail name and/or address such as “Axe-Head@LoveMachine.com”. In another example, the information regarding the reviewer may be obtained directly from the reviewer, for example, through various questions that may be present to the reviewer. These questions may include the duration of ownership of the product, previous experiences with similar products, whether use of the product for professional use rather than casual use, etc.
  • In addition, if the reviewer has provided user reviews previously which were considered to be of high quality, the user reviews from the reviewer may be favored, for example, by increasing the final quality score. Furthermore, if there are more than one user review provided by the reviewer regarding a particular product, the more recent review may be favored over the older user reviews by increasing the final quality score of the recent review. Moreover, collaborative points that provide archival data of other users of the opinion analyzer system 10 may be used to increase or decrease the final quality score for a particular user review. For example, the viewers of the particular user review may be requested to rank the user's review as either being helpful, or unhelpful. The final quality score of user reviews that are indicated by viewers to be helpful may be increased correspondingly.
  • Thus, the final quality score for each user review being analyzed, as determined by the opinion analyzer system 10 in the manner described above, is ultimately dependent on the characteristic features of the user review being analyzed. The determined final quality scores of each of the user reviews 2 will invariably differ from one another depending their characteristic features. This allows various user reviews 2 to be distinguished from one another so that selected user reviews can be used further or otherwise processed.
  • It should be clear that by variously editing and modifying the various source files described above, and/or by changing the idealized parameters, the opinion analyzer system 10 can be variously and alternatively instantiated so as to provide a different “voice” or “style” to the user opinions. For example, for a more professional or business-oriented audience, longer opinions using more jargon and which are submitted by users who are professionals, could easily be made to receive systematically higher final quality scores. For a more general consumer audience, those user reviews that are shorter, simpler, and avoid a lot of jargon, could be preferred.
  • For example, user reviews having a final quality score that is less than a predetermined minimum final quality score can be discarded so that such reviews do not take up resources and waste time of viewers. User reviews having a final quality score that is higher than the minimum final quality score may be further analyzed. For example, the manufacture of the product being reviewed may further analyze the user reviews to determine strengths and weaknesses of the product reviewed for further improvement in future models.
  • Alternatively, as noted above, the final quality scores may be normalized on a scale of 0 to 100 and the user reviews having a normalized final quality score of less than a cut-off value, 50 for example, may be discarded or suppressed while those user reviews having a normalized final quality score more than the cut-off value are further processed, or displayed, for example, to prospective consumers researching a particular product. Of course, the above are merely examples of how the final quality score may be further weighed, and other processing methods may be used in other embodiments as well.
  • Therefore, in view of the above, it should also be evident that the present invention also provides a method for determining quality of user reviews, or any other prose content, such as professional reviews, which contains opinions on products, to thereby distinguish them for further use or processing. FIG. 2 is a flow diagram 30 showing a method in accordance with one embodiment. The method comprises the step 32 in which language of at least a portion of a user review is parsed. In step 34, characteristic features are extracted from the user review that are indicative of quality of the user review. Various quality parameters are determined in step 36. In step 38, raw scores are determined for the quality parameters. Finally, in the illustrated embodiment, the method also includes step 40 in which quality of the user review is determined based on the raw score.
  • Of course, it should be evident that the flow diagram 30 of FIG. 2 merely shows one example method and other methods of the present invention may include additional steps. For example, the method may further include the step of providing ideal raw scores, determining functional scores based on the raw scores, and calculating a final quality score, all of which were described in detail above, and thus, are omitted herein to avoid repetition. Moreover, as also described, various quality parameters may be emphasized or de-emphasized using weighing factors. Furthermore information associated with the reviewer that provides the user review may be utilized to further modify the final quality score.
  • It should now be evident that the opinion analyzer and the method for determining quality of user reviews in accordance with the present invention as described above may be used to analyze user opinions regarding any products or services. In particular, the opinion analyzer of the present invention allows substantially automated analysis of user opinions so that user opinions that are of high quality can be distinguished from poor quality opinions so that further use or processing may be conducted. Moreover, the present invention also allows poor quality user opinions to be removed from websites or databases to thereby reduce system and processing requirements of websites or databases that maintain user opinions. Furthermore, it should also be evident that the present invention may be implemented as a computer software stored on a computer readable medium having instructions for implementing the method described herein.
  • While various embodiments in accordance with the present invention have been shown and described, it is understood that the invention is not limited thereto. The present invention may be changed, modified and further applied by those skilled in the art. Therefore, this invention is not limited to the detail shown and described previously, but also includes all such changes and modifications.

Claims (75)

  1. 1. A method for determining quality of user reviews to distinguish the user reviews for further use or processing, said method comprising the steps of:
    parsing text of at least a portion of a user review;
    extracting at least one characteristic feature from said user review that is indicative of quality of said user review;
    determining at least one quality parameter for said user review based on said extracted characteristic feature;
    determining a raw score for said at least one quality parameter; and
    determining quality of said user review based on said raw score.
  2. 2. The method of claim 1, wherein said at least one characteristic feature is a text pattern.
  3. 3. The method of claim 2, wherein said text pattern is a wildcard based pattern having a wildcard therein.
  4. 4. The method of claim 1, wherein said step of determining at least one quality parameter based on said at least one characteristic feature includes determining a number of occurrences of said at least one characteristic feature.
  5. 5. The method of claim 1, wherein said step of determining said raw score for said at least one quality parameter includes the step of specifying an antecedent condition that is satisfied with a text pattern.
  6. 6. The method of claim 1, wherein said step of determining said raw score for said at least one quality parameter includes the step of counting each occurrence of said at least one characteristic feature in said user review.
  7. 7. The method of claim 1, wherein said step of determining said raw score for said at least one quality parameter includes the step of providing a predetermined ideal raw score.
  8. 8. The method of claim 7, further including the step of determining a functional score for said at least one quality parameter based on said raw score and said predetermined ideal raw score.
  9. 9. The method of claim 8, wherein said step of determining functional score includes the step of determining the difference between said raw score and said predetermined ideal raw score.
  10. 10. The method of claim 9, wherein said step of determining functional score further includes the step of assigning a functional value if said raw score corresponds to said ideal raw score.
  11. 11. The method of claim 8, wherein said step of determining functional score further includes the step of providing a predetermined high differential value, and a predetermined low differential value for said at least one characteristic feature.
  12. 12. The method of claim 11, wherein said step of determining functional score further includes the step of assigning zero value if the difference between said raw score and said predetermined ideal raw score is below said predetermined low differential value.
  13. 13. The method of claim 11, wherein said step of determining functional score further includes the step of assigning zero value if the difference between said raw score and said predetermined ideal raw score is above said predetermined high differential value.
  14. 14. The method of claim 11, wherein said step of determining functional score further includes the step of assigning an apportioned value if the difference between said raw score and said predetermined ideal raw score is above said predetermined low differential value and is below said predetermined high differential value, but is not zero.
  15. 15. The method of claim 14, wherein said apportioned value is apportioned proportionately based on said raw score relative to said predetermined idealized raw score and at least one of said predetermined high differential value and said predetermined low differential value.
  16. 16. The method of claim 8, further including the step of weighing said at least one quality parameter emphasize or de-emphasize said at least one quality parameter.
  17. 17. The method of claim 8, further including the step of determining a final quality score for said user review based on said functional score of each quality parameter of said user review.
  18. 18. The method of claim 17, further including the step of weighing at least one of said at least one quality parameter emphasize or de-emphasize said quality parameter to modify said final quality score.
  19. 19. The method of claim 1, further including the step of providing a fall-off factor that diminishes said raw score of said at least one quality parameter when said at least one characteristic feature is repeated.
  20. 20. The method of claim 1, wherein said at least one characteristic feature of said user review is a positively critical text pattern.
  21. 21. The method of claim 1, wherein said at least one characteristic feature of said user review is a negatively critical text pattern.
  22. 22. The method of claim 1, wherein said at least one characteristic feature of said user review is a nullifier text pattern nullifying effect of at least one of an positively critical text pattern and a negatively critical text pattern.
  23. 23. The method of claim 1, wherein said at least one characteristic feature of said user review is an enhancer text pattern enhancing effect of at least one of an positively critical text pattern and a negatively critical text pattern.
  24. 24. The method of claim 1, wherein said at least one characteristic feature of said user review is a comparative text pattern.
  25. 25. The method of claim 25, wherein said comparative text pattern is at least one of a brand name and a model of said another comparable product.
  26. 26. The method of claim 1, wherein said at least one characteristic feature of said user review is an experiential text pattern indicating real experience in usage of product reviewed by said user review.
  27. 27. The method of claim 1, wherein said at least one characteristic feature is a jargon text pattern associated with a product category.
  28. 28. The method of claim 1, further including the step of determining a balance parameter of said user review by calculating the difference between number of positively critical text patterns of said user review, and number of negatively critical text patterns of said user review.
  29. 29. The method of claim 1, further including the step of determining a depth parameter of said user review by determining a minimum of the number of positively critical text patterns and the number of negatively critical text patterns of said user review.
  30. 30. The method of claim 1, further including the step of determining a comparativeness parameter of said user review by determining number of comparative text patterns.
  31. 31. The method of claim 1, further including the step of determining an experiential parameter of said user review by determining number of experiential text patterns.
  32. 32. The method of claim 1, further including the step of determining a richness parameter of said user review by calculating a sum of positively critical text patterns, negatively critical text patterns, experiential text patterns, and comparative text patterns.
  33. 33. The method of claim 1, further including the step of determining a verbosity parameter of said user review by calculating ratio of length of said user review to sum of positively critical text patterns, negatively critical text patterns, comparative text patterns, and experiential text patterns.
  34. 34. The method of claim 1, further including the step of determining a conciseness parameter of said user review by calculating length of said user review in comparison to a predetermined length.
  35. 35. The method of claim 1, further including the step of determining a focus parameter of said user review by calculating length of text that discuss product of said user review in comparison to length of text that does not discuss product of said user review.
  36. 36. The method of claim 1, further including the step of determining a grammar parameter of said user review indicative of extent to which said user review avoids grammatical errors.
  37. 37. The method of claim 1, further including the step of determining a jargon parameter of said user review indicative of the number of technical jargons used in said user review.
  38. 38. The method of claim 17, further including the step of adjusting said final quality score of said user review based on information associated with a reviewer providing said user review.
  39. 39. The method of claim 38, wherein said information associated with said reviewer is at least one of an e-mail address associated with said user, responses to a questionnaire, recency of said user review, final quality scores of previous user reviews from said reviewer, and collaborative information.
  40. 40. The method of claim 1, wherein said user review is directed to at least one of a product and a service.
  41. 41. A system for determining quality of user reviews to distinguish the user reviews for further use or processing, said system comprising:
    a processor;
    a parser module adapted to parse text of at least a portion of a user review and extract a characteristic feature of said user review that is indicative of quality of said user review; and
    a prose analyzer module adapted to determine at least one quality parameter based on said characteristic feature, determine a raw score for said at least one quality parameter, and to determine quality of said user review based on said raw score.
  42. 42. The system of claim 41, wherein said prose analyzer module determines said raw score for said at least one quality parameter by counting each occurrence of said characteristic feature in said user review.
  43. 43. The system of claim 41, further comprising a character features database having a plurality of character features stored therein.
  44. 44. The system of claim 43, wherein said plurality of character features are a plurality of text patterns.
  45. 45. The system of claim 44, wherein at least one of said plurality of text pattern is a wildcard based pattern having a wildcard therein.
  46. 46. The system of claim 44, further comprising a language value rules database having a plurality of rules specifying point assignment values associated with said plurality of text patterns.
  47. 47. The system of claim 41, wherein said prose analyzer module is further adapted to determine a functional score for said at least one quality parameter based on difference between said raw score and a predetermined ideal raw score.
  48. 48. The system of claim 47, wherein said prose analyzer module is further adapted to assign an apportioned value for said at least one quality parameter if the difference between said raw score and a predetermined ideal raw score is not within a predetermined differential value.
  49. 49. The system of claim 47, wherein said prose analyzer is further adapted to allow weighing of said at least one quality parameter to emphasize or de-emphasize said quality parameter to modify said final quality score.
  50. 50. The system of claim 47, wherein said prose analyzer module is further adapted to determine a final quality score for said user review based on said functional score for each quality parameter of said user review.
  51. 51. The system of claim 50, further comprising a submitter profiling module adapted to adjust said final quality score based on information associated with a reviewer providing said user review.
  52. 52. The system of claim 51, wherein said information associated with said reviewer is at least one of an e-mail address associated with said user, responses to a questionnaire, recency of said user review, final quality scores of previous user reviews from said reviewer, and collaborative information.
  53. 53. The system of claim 41, wherein said characteristic feature of said user review is at least one of a positively critical text pattern and a negatively critical text pattern.
  54. 54. The system of claim 41, wherein said characteristic feature of said user review is at least one of a nullifier text pattern and an enhancer text pattern.
  55. 55. The system of claim 41, wherein said characteristic feature of said user review is a comparative text pattern indicating a comparison between product reviewed by said user review and another comparable product.
  56. 56. The system of claim 41, wherein said characteristic feature of said user review is an experiential text pattern indicating real experience in usage of product reviewed by said user review.
  57. 57. The system of claim 41, wherein said characteristic feature of said user review is a jargon text pattern associated with a product category.
  58. 58. The system of claim 41, wherein said at least one quality parameter is a balance parameter which is the difference between number of positively critical text patterns of said user review, and number of negatively critical text patterns of said user review.
  59. 59. The system of claim 41, wherein said at least one quality parameter is a depth parameter which is a minimum of the number of positively critical text patterns and the number of negatively critical text patterns of said user review.
  60. 60. The system of claim 41, wherein said at least one quality parameter is a comparativeness parameter which is a number of comparative text patterns.
  61. 61. The system of claim 41, wherein said at least one quality parameter is an experiential parameter which is a number of experiential text patterns.
  62. 62. The system of claim 41, wherein said at least one quality parameter is a richness parameter which is a sum of positively critical text patterns, negatively critical text patterns, experiential text patterns, and comparative text patterns.
  63. 63. The system of claim 41, wherein said at least one quality parameter is a verbosity parameter which is a ratio of length of said user review to sum of positively critical text patterns, negatively critical text patterns, comparative text patterns, and experiential text patterns.
  64. 65. The system of claim 41, wherein said at least one quality parameter is a conciseness parameter which is the length of said user review in comparison to a predetermined length.
  65. 66. The system of claim 41, wherein said at least one quality parameter is a focus parameter which is the length of text that discuss product of said user review in comparison to length of text that does not discuss product of said user review.
  66. 67. The system of claim 41, wherein said at least one quality parameter is a grammar parameter indicative of extent to which said user review avoids grammatical errors.
  67. 68. The system of claim 41, wherein said at least one quality parameter is a jargon parameter indicative of the number of technical jargons used in the user review.
  68. 69. The system of claim 41, wherein said user review is directed to at least one of a product and a service.
  69. 70. A computer readable medium for determining quality of user reviews to distinguish the user reviews for further use or processing, said medium comprising:
    instructions for parsing text of at least a portion of a user review;
    instructions for extracting at least one characteristic feature from said user review that is indicative of quality of said user review;
    instructions for determining at least one quality parameter for said user review based on said extracted characteristic feature;
    instructions for determining a raw score for said at least one quality parameter; and
    instructions for determining quality of said user review based on said raw score.
  70. 71. The medium of claim 70, wherein said instructions for determining at least one quality parameter includes instructions for determining a number of occurrences of said at least one characteristic feature.
  71. 72. The medium of claim 70, further including instructions for determining a functional score of said at least one quality parameter based on said raw score.
  72. 73. The medium of claim 72, further including instructions for determining a final quality score for said user review based on said functional score.
  73. 74. The medium of claim 73, further including instructions for weighing at least one of said at least one quality parameter emphasize or de-emphasize said quality parameter to modify said final quality score.
  74. 75. The medium of claim 70, further including instructions for diminishing said raw score of said at least one quality parameter when said at least one characteristic feature is repeated.
  75. 76. The medium of claim 73, further including instructions to allow adjustment of said final quality score of said user review based on information associated with a reviewer providing said user review.
US10636966 2003-08-08 2003-08-08 System and method for determining quality of written product reviews in an automated manner Active 2026-04-12 US7363214B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10636966 US7363214B2 (en) 2003-08-08 2003-08-08 System and method for determining quality of written product reviews in an automated manner

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US10636966 US7363214B2 (en) 2003-08-08 2003-08-08 System and method for determining quality of written product reviews in an automated manner
PCT/US2004/019240 WO2005017656A3 (en) 2003-08-08 2004-06-17 System and method for determining quality of written product reviews in an automated manner
EP20040755417 EP1660964A2 (en) 2003-08-08 2004-06-17 System and method for determining quality of written product reviews in an automated manner

Publications (2)

Publication Number Publication Date
US20050034071A1 true true US20050034071A1 (en) 2005-02-10
US7363214B2 US7363214B2 (en) 2008-04-22

Family

ID=34116500

Family Applications (1)

Application Number Title Priority Date Filing Date
US10636966 Active 2026-04-12 US7363214B2 (en) 2003-08-08 2003-08-08 System and method for determining quality of written product reviews in an automated manner

Country Status (3)

Country Link
US (1) US7363214B2 (en)
EP (1) EP1660964A2 (en)
WO (1) WO2005017656A3 (en)

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060200342A1 (en) * 2005-03-01 2006-09-07 Microsoft Corporation System for processing sentiment-bearing text
US20060200341A1 (en) * 2005-03-01 2006-09-07 Microsoft Corporation Method and apparatus for processing sentiment-bearing text
US20070033092A1 (en) * 2005-08-04 2007-02-08 Iams Anthony L Computer-implemented method and system for collaborative product evaluation
US20070078845A1 (en) * 2005-09-30 2007-04-05 Scott James K Identifying clusters of similar reviews and displaying representative reviews from multiple clusters
US20070078699A1 (en) * 2005-09-30 2007-04-05 Scott James K Systems and methods for reputation management
US20070078670A1 (en) * 2005-09-30 2007-04-05 Dave Kushal B Selecting high quality reviews for display
US20070078671A1 (en) * 2005-09-30 2007-04-05 Dave Kushal B Selecting high quality text within identified reviews for display in review snippets
US20080033781A1 (en) * 2006-07-18 2008-02-07 Jonah Holmes Peretti System and method for online product promotion
US20080109232A1 (en) * 2006-06-07 2008-05-08 Cnet Networks, Inc. Evaluative information system and method
US20080235721A1 (en) * 2007-03-21 2008-09-25 Omar Ismail Methods and Systems for Creating and Providing Collaborative User Reviews of Products and Services
US20080249764A1 (en) * 2007-03-01 2008-10-09 Microsoft Corporation Smart Sentiment Classifier for Product Reviews
US20080313165A1 (en) * 2007-06-15 2008-12-18 Microsoft Corporation Scalable model-based product matching
US20090048823A1 (en) * 2007-08-16 2009-02-19 The Board Of Trustees Of The University Of Illinois System and methods for opinion mining
US20090083096A1 (en) * 2007-09-20 2009-03-26 Microsoft Corporation Handling product reviews
US7519562B1 (en) * 2005-03-31 2009-04-14 Amazon Technologies, Inc. Automatic identification of unreliable user ratings
US20090281870A1 (en) * 2008-05-12 2009-11-12 Microsoft Corporation Ranking products by mining comparison sentiment
US20100217654A1 (en) * 2001-04-24 2010-08-26 Keller Thomas L Creating an incentive to author useful item reviews
US20100235311A1 (en) * 2009-03-13 2010-09-16 Microsoft Corporation Question and answer search
US7822631B1 (en) * 2003-08-22 2010-10-26 Amazon Technologies, Inc. Assessing content based on assessed trust in users
US20100274791A1 (en) * 2009-04-28 2010-10-28 Palo Alto Research Center Incorporated Web-based tool for detecting bias in reviews
US20110041075A1 (en) * 2009-08-12 2011-02-17 Google Inc. Separating reputation of users in different roles
US20110078167A1 (en) * 2009-09-28 2011-03-31 Neelakantan Sundaresan System and method for topic extraction and opinion mining
US20110184726A1 (en) * 2010-01-25 2011-07-28 Connor Robert A Morphing text by splicing end-compatible segments
US20110184727A1 (en) * 2010-01-25 2011-07-28 Connor Robert A Prose style morphing
US20110225203A1 (en) * 2010-03-11 2011-09-15 Board Of Trustees Of Michigan State University Systems and methods for tracking and evaluating review tasks
US20110246533A1 (en) * 2010-03-31 2011-10-06 Salesforce.Com, Inc. System, method and computer program product for rule-based performance of actions on related objects
US20110276406A1 (en) * 2010-05-10 2011-11-10 Nahu BVBA System and method for social product information sharing
US20110313756A1 (en) * 2010-06-21 2011-12-22 Connor Robert A Text sizer (TM)
US8108255B1 (en) * 2007-09-27 2012-01-31 Amazon Technologies, Inc. Methods and systems for obtaining reviews for items lacking reviews
US20120054638A1 (en) * 2010-08-31 2012-03-01 Disney Enterprises, Inc. Automated effort judgement of user generated content
US8290811B1 (en) 2007-09-28 2012-10-16 Amazon Technologies, Inc. Methods and systems for searching for and identifying data repository deficits
US20130006707A1 (en) * 2010-01-07 2013-01-03 Sarkar Ssubhanjan Crm application for analysing and generating actionable reports based on activity of users on a product portal
US20130060760A1 (en) * 2011-09-02 2013-03-07 Microsoft Corporation Determining comprehensive subsets of reviews
US8438469B1 (en) 2005-09-30 2013-05-07 Google Inc. Embedded review and rating information
US20130158978A1 (en) * 2011-12-14 2013-06-20 International Business Machines Corporation Adaptation of Vocabulary Levels for Enhanced Collaboration
US8489438B1 (en) * 2006-03-31 2013-07-16 Intuit Inc. Method and system for providing a voice review
US8554601B1 (en) 2003-08-22 2013-10-08 Amazon Technologies, Inc. Managing content based on reputation
KR101319413B1 (en) 2012-01-10 2013-10-17 한국과학기술연구원 Summary Information Generating System and Method for Review of Product and Service
US8615401B1 (en) * 2011-03-08 2013-12-24 Amazon Technologies, Inc. Identifying individually approved and disapproved segments of content
US8738390B1 (en) * 2011-03-08 2014-05-27 Amazon Technologies, Inc. Facilitating approval or disapproval of individual segments of content
US9009082B1 (en) 2008-06-30 2015-04-14 Amazon Technologies, Inc. Assessing user-supplied evaluations
US20150286977A1 (en) * 2014-04-07 2015-10-08 Homeaway, Inc. Systems and methods to reconcile free-text with structured data
US20160070709A1 (en) * 2014-09-09 2016-03-10 Stc.Unm Online review assessment using multiple sources
US20170068648A1 (en) * 2015-09-04 2017-03-09 Wal-Mart Stores, Inc. System and method for analyzing and displaying reviews
US9646338B1 (en) * 2007-07-06 2017-05-09 Amazon Technologies, Inc. Managing textual descriptions based on quality rating
US10033537B2 (en) * 2012-11-30 2018-07-24 Stephen B. Heppe Promoting learned discourse in online media with consideration of sources and provenance

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007095557A3 (en) 2006-02-13 2008-08-21 Junaid Ali Web-based application or system for managing and coordinating review-enabled content
JP4870448B2 (en) * 2006-02-23 2012-02-08 株式会社日立製作所 The information processing apparatus, customer needs analysis method, and program
US8504410B2 (en) * 2007-03-02 2013-08-06 Poorya Pasta Method for improving customer survey system
US7996252B2 (en) * 2006-03-02 2011-08-09 Global Customer Satisfaction System, Llc Global customer satisfaction system
EP2038743A4 (en) * 2006-07-12 2009-08-05 Arbitron Inc Methods and systems for compliance confirmation and incentives
US9332363B2 (en) 2011-12-30 2016-05-03 The Nielsen Company (Us), Llc System and method for determining meter presence utilizing ambient fingerprints
US7761287B2 (en) * 2006-10-23 2010-07-20 Microsoft Corporation Inferring opinions based on learned probabilities
US8140375B2 (en) * 2007-04-18 2012-03-20 Microsoft Corporation Voting on claims pertaining to a resource
US20090125369A1 (en) * 2007-10-26 2009-05-14 Crowe Horwath Llp System and method for analyzing and dispositioning money laundering suspicious activity alerts
US8463594B2 (en) * 2008-03-21 2013-06-11 Sauriel Llc System and method for analyzing text using emotional intelligence factors
US8195522B1 (en) * 2008-06-30 2012-06-05 Amazon Technologies, Inc. Assessing users who provide content
US20100125484A1 (en) * 2008-11-14 2010-05-20 Microsoft Corporation Review summaries for the most relevant features
US20100235343A1 (en) * 2009-03-13 2010-09-16 Microsoft Corporation Predicting Interestingness of Questions in Community Question Answering
GB0922608D0 (en) 2009-12-23 2010-02-10 Vratskides Alexios Message optimization
US8990124B2 (en) * 2010-01-14 2015-03-24 Microsoft Technology Licensing, Llc Assessing quality of user reviews
US20110184780A1 (en) * 2010-01-21 2011-07-28 Ebay Inc. INTEGRATION OF eCOMMERCE FEATURES INTO SOCIAL NETWORKING PLATFORM
CN102760264A (en) * 2011-04-29 2012-10-31 国际商业机器公司 Computer-implemented method and system for generating extracts of internet comments
US8798995B1 (en) * 2011-09-23 2014-08-05 Amazon Technologies, Inc. Key word determinations from voice data
US9009024B2 (en) * 2011-10-24 2015-04-14 Hewlett-Packard Development Company, L.P. Performing sentiment analysis
US9268769B1 (en) * 2011-12-20 2016-02-23 Persado Intellectual Property Limited System, method, and computer program for identifying message content to send to users based on user language characteristics
US9589535B2 (en) 2013-07-19 2017-03-07 Paypal, Inc. Social mobile game for recommending items
US9400956B2 (en) 2014-11-05 2016-07-26 International Business Machines Corporation Answer interactions in a question-answering environment
US10061842B2 (en) 2014-12-09 2018-08-28 International Business Machines Corporation Displaying answers in accordance with answer classifications

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5361353A (en) * 1991-10-02 1994-11-01 International Business Machines Corporation System for parsing message units from an unstructured message stream of interleaved message units to form structured messages
US5708822A (en) * 1995-05-31 1998-01-13 Oracle Corporation Methods and apparatus for thematic parsing of discourse
US5778397A (en) * 1995-06-28 1998-07-07 Xerox Corporation Automatic method of generating feature probabilities for automatic extracting summarization
US5867799A (en) * 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US5983170A (en) * 1996-06-25 1999-11-09 Continuum Software, Inc System and method for generating semantic analysis of textual information
US6026410A (en) * 1997-02-10 2000-02-15 Actioneer, Inc. Information organization and collaboration tool for processing notes and action requests in computer systems
US6356864B1 (en) * 1997-07-25 2002-03-12 University Technology Corporation Methods for analysis and evaluation of the semantic content of a writing based on vector length
US6366759B1 (en) * 1997-07-22 2002-04-02 Educational Testing Service System and method for computer-based automatic essay scoring
US20030031966A1 (en) * 2001-08-10 2003-02-13 Berry Jonathan D. Anode gas burner for inert gas generation, method and apparatus
US6542888B2 (en) * 1997-11-26 2003-04-01 International Business Machines Corporation Content filtering for electronic documents generated in multiple foreign languages
US6766287B1 (en) * 1999-12-15 2004-07-20 Xerox Corporation System for genre-specific summarization of documents
US20040225651A1 (en) * 2003-05-07 2004-11-11 Musgrove Timothy A. System and method for automatically generating a narrative product summary
US6978274B1 (en) * 2001-08-31 2005-12-20 Attenex Corporation System and method for dynamically evaluating latent concepts in unstructured documents
US7120613B2 (en) * 2002-02-22 2006-10-10 National Institute Of Information And Communications Technology Solution data edit processing apparatus and method, and automatic summarization processing apparatus and method
US7165024B2 (en) * 2002-02-22 2007-01-16 Nec Laboratories America, Inc. Inferring hierarchical descriptions of a set of documents

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1276061A1 (en) 2001-07-09 2003-01-15 Accenture Computer based system and method of determining a satisfaction index of a text
US20030031996A1 (en) * 2001-08-08 2003-02-13 Adam Robinson Method and system for evaluating documents

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5361353A (en) * 1991-10-02 1994-11-01 International Business Machines Corporation System for parsing message units from an unstructured message stream of interleaved message units to form structured messages
US5708822A (en) * 1995-05-31 1998-01-13 Oracle Corporation Methods and apparatus for thematic parsing of discourse
US5778397A (en) * 1995-06-28 1998-07-07 Xerox Corporation Automatic method of generating feature probabilities for automatic extracting summarization
US5867799A (en) * 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US5983170A (en) * 1996-06-25 1999-11-09 Continuum Software, Inc System and method for generating semantic analysis of textual information
US6026410A (en) * 1997-02-10 2000-02-15 Actioneer, Inc. Information organization and collaboration tool for processing notes and action requests in computer systems
US6366759B1 (en) * 1997-07-22 2002-04-02 Educational Testing Service System and method for computer-based automatic essay scoring
US6356864B1 (en) * 1997-07-25 2002-03-12 University Technology Corporation Methods for analysis and evaluation of the semantic content of a writing based on vector length
US6542888B2 (en) * 1997-11-26 2003-04-01 International Business Machines Corporation Content filtering for electronic documents generated in multiple foreign languages
US6766287B1 (en) * 1999-12-15 2004-07-20 Xerox Corporation System for genre-specific summarization of documents
US20030031966A1 (en) * 2001-08-10 2003-02-13 Berry Jonathan D. Anode gas burner for inert gas generation, method and apparatus
US6978274B1 (en) * 2001-08-31 2005-12-20 Attenex Corporation System and method for dynamically evaluating latent concepts in unstructured documents
US7120613B2 (en) * 2002-02-22 2006-10-10 National Institute Of Information And Communications Technology Solution data edit processing apparatus and method, and automatic summarization processing apparatus and method
US7165024B2 (en) * 2002-02-22 2007-01-16 Nec Laboratories America, Inc. Inferring hierarchical descriptions of a set of documents
US20040225651A1 (en) * 2003-05-07 2004-11-11 Musgrove Timothy A. System and method for automatically generating a narrative product summary

Cited By (72)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100217654A1 (en) * 2001-04-24 2010-08-26 Keller Thomas L Creating an incentive to author useful item reviews
US8140380B2 (en) 2001-04-24 2012-03-20 Amazon.Com, Inc. Creating an incentive to author useful item reviews
US8554601B1 (en) 2003-08-22 2013-10-08 Amazon Technologies, Inc. Managing content based on reputation
US7822631B1 (en) * 2003-08-22 2010-10-26 Amazon Technologies, Inc. Assessing content based on assessed trust in users
US20060200341A1 (en) * 2005-03-01 2006-09-07 Microsoft Corporation Method and apparatus for processing sentiment-bearing text
US20060200342A1 (en) * 2005-03-01 2006-09-07 Microsoft Corporation System for processing sentiment-bearing text
US7788086B2 (en) * 2005-03-01 2010-08-31 Microsoft Corporation Method and apparatus for processing sentiment-bearing text
US7788087B2 (en) * 2005-03-01 2010-08-31 Microsoft Corporation System for processing sentiment-bearing text
US7519562B1 (en) * 2005-03-31 2009-04-14 Amazon Technologies, Inc. Automatic identification of unreliable user ratings
US20070033092A1 (en) * 2005-08-04 2007-02-08 Iams Anthony L Computer-implemented method and system for collaborative product evaluation
US8249915B2 (en) * 2005-08-04 2012-08-21 Iams Anthony L Computer-implemented method and system for collaborative product evaluation
US20070078699A1 (en) * 2005-09-30 2007-04-05 Scott James K Systems and methods for reputation management
EP1949332A4 (en) * 2005-09-30 2011-08-24 Google Inc Selecting high quality reviews for display
US20070078845A1 (en) * 2005-09-30 2007-04-05 Scott James K Identifying clusters of similar reviews and displaying representative reviews from multiple clusters
US20110125736A1 (en) * 2005-09-30 2011-05-26 Dave Kushal B Selecting High Quality Reviews for Display
US8010480B2 (en) 2005-09-30 2011-08-30 Google Inc. Selecting high quality text within identified reviews for display in review snippets
US7558769B2 (en) * 2005-09-30 2009-07-07 Google Inc. Identifying clusters of similar reviews and displaying representative reviews from multiple clusters
US8438469B1 (en) 2005-09-30 2013-05-07 Google Inc. Embedded review and rating information
EP1949332A2 (en) * 2005-09-30 2008-07-30 Google Inc. Selecting high quality reviews for display
US20070078671A1 (en) * 2005-09-30 2007-04-05 Dave Kushal B Selecting high quality text within identified reviews for display in review snippets
US20070078670A1 (en) * 2005-09-30 2007-04-05 Dave Kushal B Selecting high quality reviews for display
US7827052B2 (en) * 2005-09-30 2010-11-02 Google Inc. Systems and methods for reputation management
US8489438B1 (en) * 2006-03-31 2013-07-16 Intuit Inc. Method and system for providing a voice review
US20080109232A1 (en) * 2006-06-07 2008-05-08 Cnet Networks, Inc. Evaluative information system and method
US20080033781A1 (en) * 2006-07-18 2008-02-07 Jonah Holmes Peretti System and method for online product promotion
US10043191B2 (en) * 2006-07-18 2018-08-07 Buzzfeed, Inc. System and method for online product promotion
US20080249764A1 (en) * 2007-03-01 2008-10-09 Microsoft Corporation Smart Sentiment Classifier for Product Reviews
US20080235721A1 (en) * 2007-03-21 2008-09-25 Omar Ismail Methods and Systems for Creating and Providing Collaborative User Reviews of Products and Services
US8949876B2 (en) * 2007-03-21 2015-02-03 Productwiki, Inc. Methods and systems for creating and providing collaborative user reviews of products and services
US7979459B2 (en) * 2007-06-15 2011-07-12 Microsoft Corporation Scalable model-based product matching
US20080313165A1 (en) * 2007-06-15 2008-12-18 Microsoft Corporation Scalable model-based product matching
US9646338B1 (en) * 2007-07-06 2017-05-09 Amazon Technologies, Inc. Managing textual descriptions based on quality rating
US20090048823A1 (en) * 2007-08-16 2009-02-19 The Board Of Trustees Of The University Of Illinois System and methods for opinion mining
US20090083096A1 (en) * 2007-09-20 2009-03-26 Microsoft Corporation Handling product reviews
US8108255B1 (en) * 2007-09-27 2012-01-31 Amazon Technologies, Inc. Methods and systems for obtaining reviews for items lacking reviews
US9633388B2 (en) 2007-09-28 2017-04-25 Amazon Technologies, Inc. Methods and systems for searching for and identifying data repository deficits
US8290811B1 (en) 2007-09-28 2012-10-16 Amazon Technologies, Inc. Methods and systems for searching for and identifying data repository deficits
US8566178B1 (en) 2007-09-28 2013-10-22 Amazon Technologies, Inc. Methods and systems for searching for and identifying data repository deficits
US8731995B2 (en) 2008-05-12 2014-05-20 Microsoft Corporation Ranking products by mining comparison sentiment
US20090281870A1 (en) * 2008-05-12 2009-11-12 Microsoft Corporation Ranking products by mining comparison sentiment
US9009082B1 (en) 2008-06-30 2015-04-14 Amazon Technologies, Inc. Assessing user-supplied evaluations
US20100235311A1 (en) * 2009-03-13 2010-09-16 Microsoft Corporation Question and answer search
US20100274791A1 (en) * 2009-04-28 2010-10-28 Palo Alto Research Center Incorporated Web-based tool for detecting bias in reviews
US8732176B2 (en) * 2009-04-28 2014-05-20 Palo Alto Research Center Incorporated Web-based tool for detecting bias in reviews
WO2011019526A3 (en) * 2009-08-12 2011-05-05 Google Inc. Separating reputation of users in different roles
US20110041075A1 (en) * 2009-08-12 2011-02-17 Google Inc. Separating reputation of users in different roles
US20110078167A1 (en) * 2009-09-28 2011-03-31 Neelakantan Sundaresan System and method for topic extraction and opinion mining
US9514156B2 (en) 2009-09-28 2016-12-06 Ebay Inc. System and method for topic extraction and opinion mining
US8533208B2 (en) * 2009-09-28 2013-09-10 Ebay Inc. System and method for topic extraction and opinion mining
US20130006707A1 (en) * 2010-01-07 2013-01-03 Sarkar Ssubhanjan Crm application for analysing and generating actionable reports based on activity of users on a product portal
US20110184726A1 (en) * 2010-01-25 2011-07-28 Connor Robert A Morphing text by splicing end-compatible segments
US8543381B2 (en) * 2010-01-25 2013-09-24 Holovisions LLC Morphing text by splicing end-compatible segments
US20110184727A1 (en) * 2010-01-25 2011-07-28 Connor Robert A Prose style morphing
US8428934B2 (en) * 2010-01-25 2013-04-23 Holovisions LLC Prose style morphing
US20110225203A1 (en) * 2010-03-11 2011-09-15 Board Of Trustees Of Michigan State University Systems and methods for tracking and evaluating review tasks
US8819081B2 (en) * 2010-03-31 2014-08-26 Salesforce.Com, Inc. System, method and computer program product for rule-based performance of actions on related objects
US20110246533A1 (en) * 2010-03-31 2011-10-06 Salesforce.Com, Inc. System, method and computer program product for rule-based performance of actions on related objects
US20110276406A1 (en) * 2010-05-10 2011-11-10 Nahu BVBA System and method for social product information sharing
US8484080B2 (en) * 2010-05-10 2013-07-09 Nahu BVBA System and method for social product information sharing
US20110313756A1 (en) * 2010-06-21 2011-12-22 Connor Robert A Text sizer (TM)
US20120054638A1 (en) * 2010-08-31 2012-03-01 Disney Enterprises, Inc. Automated effort judgement of user generated content
US9652551B2 (en) * 2010-08-31 2017-05-16 Disney Enterprises, Inc. Automated effort judgement of user generated content
US8738390B1 (en) * 2011-03-08 2014-05-27 Amazon Technologies, Inc. Facilitating approval or disapproval of individual segments of content
US8615401B1 (en) * 2011-03-08 2013-12-24 Amazon Technologies, Inc. Identifying individually approved and disapproved segments of content
US20130060760A1 (en) * 2011-09-02 2013-03-07 Microsoft Corporation Determining comprehensive subsets of reviews
US20130158978A1 (en) * 2011-12-14 2013-06-20 International Business Machines Corporation Adaptation of Vocabulary Levels for Enhanced Collaboration
US8738364B2 (en) * 2011-12-14 2014-05-27 International Business Machines Corporation Adaptation of vocabulary levels for enhanced collaboration
KR101319413B1 (en) 2012-01-10 2013-10-17 한국과학기술연구원 Summary Information Generating System and Method for Review of Product and Service
US10033537B2 (en) * 2012-11-30 2018-07-24 Stephen B. Heppe Promoting learned discourse in online media with consideration of sources and provenance
US20150286977A1 (en) * 2014-04-07 2015-10-08 Homeaway, Inc. Systems and methods to reconcile free-text with structured data
US20160070709A1 (en) * 2014-09-09 2016-03-10 Stc.Unm Online review assessment using multiple sources
US20170068648A1 (en) * 2015-09-04 2017-03-09 Wal-Mart Stores, Inc. System and method for analyzing and displaying reviews

Also Published As

Publication number Publication date Type
WO2005017656A3 (en) 2006-10-26 application
US7363214B2 (en) 2008-04-22 grant
EP1660964A2 (en) 2006-05-31 application
WO2005017656A2 (en) 2005-02-24 application

Similar Documents

Publication Publication Date Title
US7734641B2 (en) Recommendation systems and methods using interest correlation
US6636853B1 (en) Method and apparatus for representing and navigating search results
US7962461B2 (en) Method and system for finding and aggregating reviews for a product
US7548929B2 (en) System and method for determining semantically related terms
Zhang et al. An associative classification-based recommendation system for personalization in B2C e-commerce applications
US7966305B2 (en) Relevance-weighted navigation in information access, search and retrieval
US8224773B2 (en) Mining of user event data to identify users with common interests
US6523025B1 (en) Document processing system and recording medium
US7082426B2 (en) Content aggregation method and apparatus for an on-line product catalog
US20070156677A1 (en) Database access system
US20070038600A1 (en) Detecting spam related and biased contexts for programmable search engines
US20100312769A1 (en) Methods, apparatus and software for analyzing the content of micro-blog messages
US20050222901A1 (en) Determining ad targeting information and/or ad creative information using past search queries
US20060143158A1 (en) Method, system and graphical user interface for providing reviews for a product
US20080109285A1 (en) Techniques for determining relevant advertisements in response to queries
US7668821B1 (en) Recommendations based on item tagging activities of users
US20100094673A1 (en) Computer-implemented method and system for keyword bidding
US7231395B2 (en) Method and apparatus for categorizing and presenting documents of a distributed database
Liu et al. Opinion observer: analyzing and comparing opinions on the web
US20060155751A1 (en) System and method for document analysis, processing and information extraction
US7774333B2 (en) System and method for associating queries and documents with contextual advertisements
US6356879B2 (en) Content based method for product-peer filtering
US7451135B2 (en) System and method for retrieving and displaying information relating to electronic documents available from an informational network
US8090724B1 (en) Document analysis and multi-word term detector
US20070192129A1 (en) Method and system for the objective quantification of fame

Legal Events

Date Code Title Description
AS Assignment

Owner name: CNET NETWORKS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MUSGROVE, TIMOTHY A.;WALSH, ROBIN HIROKO;REEL/FRAME:015082/0812;SIGNING DATES FROM 20040108 TO 20040109

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, WA

Free format text: SECURITY AGREEMENT;ASSIGNOR:CNET NETWORKS, INC.;REEL/FRAME:019995/0150

Effective date: 20071012

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT,WAS

Free format text: SECURITY AGREEMENT;ASSIGNOR:CNET NETWORKS, INC.;REEL/FRAME:019995/0150

Effective date: 20071012

AS Assignment

Owner name: CNET NETWORKS, INC., CALIFORNIA

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:021172/0207

Effective date: 20080630

Owner name: CNET NETWORKS, INC.,CALIFORNIA

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:021172/0207

Effective date: 20080630

AS Assignment

Owner name: CBS INTERACTIVE INC., DELAWARE

Free format text: CHANGE OF NAME;ASSIGNOR:CNET NETWORKS, INC;REEL/FRAME:021719/0564

Effective date: 20080930

Owner name: CBS INTERACTIVE INC.,DELAWARE

Free format text: CHANGE OF NAME;ASSIGNOR:CNET NETWORKS, INC;REEL/FRAME:021719/0564

Effective date: 20080930

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8