US20130024464A1 - Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user - Google Patents

Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user Download PDF

Info

Publication number
US20130024464A1
US20130024464A1 US13553599 US201213553599A US2013024464A1 US 20130024464 A1 US20130024464 A1 US 20130024464A1 US 13553599 US13553599 US 13553599 US 201213553599 A US201213553599 A US 201213553599A US 2013024464 A1 US2013024464 A1 US 2013024464A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
relevant
user
users
data
server computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US13553599
Inventor
Christopher Eric Shogo Berner
Jeremy Ryan Schiff
Corey Layne Reese
Paul Kenneth Twohey
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OPENTABLE Inc
Original Assignee
Ness Computing Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/30861Retrieval from the Internet, e.g. browsers
    • G06F17/30864Retrieval from the Internet, e.g. browsers by querying, e.g. search engines or meta-search engines, crawling techniques, push systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30634Querying
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30634Querying
    • G06F17/30657Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30699Filtering based on additional data, e.g. user or group profiles
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30Information retrieval; Database structures therefor ; File system structures therefor
    • G06F17/3061Information retrieval; Database structures therefor ; File system structures therefor of unstructured textual data
    • G06F17/30705Clustering or classification

Abstract

Embodiments of the invention relate to a computer-implemented method for generating explanatory data from a personalized recommendations process for a primary user based at least on stored data about the primary user. The method comprises a server computer obtaining data related to one or more users who are relevant to the primary user, then determining at least one group of users relevant to the primary user. The server computer also obtains data related to one or more entities, determines one or more entities relevant to the primary user, and associates the at least one relevant group of users with the one or more relevant entities. One or more potential candidate factors are generated. A set of factors are selected from the one or more potential candidate factors, wherein the potential candidate factors are used as explanatory data to determine recommendations to the primary user.

Description

    CROSS-REFERENCES TO RELATED APPLICATIONS
  • This application claims priority from co-pending U.S. Provisional Patent Application No. 61/509,999 filed Jul. 20, 2011 entitled “Method and Apparatus for Explaining Recommendations”.
  • The present disclosure may be related to the following commonly assigned U.S. Provisional patent applications:
    • U.S. Provisional Patent Application No. 61/509,995 filed Jul. 20, 2011, entitled “Method and Apparatus for Category Based Navigation”,
    • U.S. Provisional Patent Application No. 61/510,003 filed Jul. 20, 2011 entitled “Method and Apparatus for Allowing Users to Augment Searches”, and
    • U.S. Provisional Patent Application No. 61/510,004 filed Jul. 20, 2011 entitled “Method and Apparatus for Quickly Evaluating Entities”.
  • The respective disclosures of these applications/patents are incorporated herein by reference, as if set forth in full in this document, for all purposes.
  • FIELD OF THE INVENTION
  • The present invention relates to recommendation systems in general and in particular to recommendation system that collect data to sift through and generate personalized recommendations and also explain personalized recommendations for users, via user devices, in order to present recommendation explanation factors used in providing informative and intuitive recommendations.
  • BACKGROUND OF THE INVENTION
  • The Internet has become a useful source of information relating to goods and services, but because so many users and companies are connected to the Internet and providing information of their own, it is difficult to find relevant information among all available data. As individuals and groups are now able to connect to the Internet and retrieve information from websites and via apps, and do so perhaps while in transit towards a geographic area of interest, they come to expect the ability to find information about goods and services being offered.
  • One source of information may be in the form of websites hosted by or for providers. As used herein, “website” refers to a collection of one or more web pages displayable by a browser or other web client, the one or more web pages being hosted on a web server (i.e., a computer coupled to the Internet, or other network over which communication occurs) that responds to requests for pages hosted thereon. An example might be a restaurant that generates information about its hours, pictures of its dining room, contact information, history, menus and other offerings, then puts that information into a suitable form (such as HTML pages, dynamically generated pages, etc.), stores those pages in electronic storage accessible by the web server (hosted by the restaurant or perhaps a company that specializes in web presence that provides such services to the restaurant), and then serves those pages as client devices request pages. The information might also be presented in another form, such as data supplied to an application (i.e., app) on the client device rather than a generic browser on the client device.
  • However, while interested users may get information directly from the restaurant, the users are still limited to information sourced by that restaurant. A number of websites have been set up to collect information from other customers of a vendor of goods or services and provide reviews, comments and other details to other customers or potential customers. For example, there might be a website that focuses on hotels, another on restaurants, and another on home repair, or a general one. However, these websites may have too much information to be useful to a user searching for relevant recommendations.
  • Additionally, with the increase of users participating in social media websites, users are also interested in getting information about their friends, and the opinions of their friends. For example, a user may find out that a number of their friends on a social media website highly recommend a restaurant, which may be more persuasive to that user than the recommendations of strangers or high rankings on general rating websites. Thus, social media websites that provide information to a user regarding the user's friends may also be another source that the user refers to in gathering information and searching for relevant recommendations. However, social media websites allow users to provide a wide variety of information that may or may not be related to what a user is searching for (e.g., status updates unrelated to a vendor of goods or services), and it is also difficult to cross-reference the recommendations of friends through social media websites with general websites providing reviews, or websites for particular businesses, in a user's search for relevant recommendations.
  • An improved recommendations system is desirable.
  • BRIEF SUMMARY OF THE INVENTION
  • Embodiments of the invention relate to an improved computer-implemented method for generating personalized recommendations and providing explanation data related to the personalized recommendations. Existing methods for generating personalized recommendations to a target user may not provide explaining factors and processes used to determine the personalized recommendations. Thus, if the target user is dissatisfied with the personalized recommendations, it is difficult for the target user to feed back to a personalized recommendations system how to improve the generation of personalized recommendations for the target user. Further, providing the target user with explanation data also informs the target user of a source from which the personalized recommendation was determined, which may influence the target user's decision to select the personalized recommendation or not. For example, a target user may receive a personalized recommendation to go to Restaurant C and also receive explanation data that Restaurant C was recommended to the target user because a “friend” of the target user, User C, goes there regularly. However, if User C is an ex-boyfriend of the target user, the target user may not want to go to Restaurant C for fear of running into User C, even though it may be likely that the target user would have enjoyed the food at Restaurant C.
  • Thus, providing explanation data of personalized recommendations to the target user is advantageous and allows users to understand the logic and processes behind generating their personalized recommendations. Additionally, personalized recommendation systems are improved in this manner by becoming more intelligent about generating personalized recommendations to its users.
  • Embodiments of the invention relate to a method for providing explanation data from a personalized recommendations process for a primary user based at least on stored data about the primary user. The method comprises a server computer obtaining data related to one or more users who are relevant to the primary user, wherein the one or more users relevant to the primary user are similar to the primary user. The server computer then determines at least one group of users relevant to the primary user based on the data related to the one or more users, each relevant group including one or more relevant users to the primary user. Then, the server computer also obtains data related to one or more entities. The method further comprises determining, at the server computer, one or more entities relevant to the primary user based on the data related to the one or more entities, and then associating the at least one relevant group of users with the one or more relevant entities. The server computer then generates one or more potential candidate factors based on at least the relevant group of users or the one or more relevant entities.
  • The method may further comprise selecting, by the server computer, a set of factors from the one or more potential candidate factors and displaying the set of selected factors to the primary user, wherein the potential candidate factors are used to determine recommendations or explanations to the primary user.
  • Another embodiment of the invention relates to a server computer comprising a processor and a non-transitory computer readable medium, the non-transitory computer readable medium comprising code executable by the processor to implement a method for generating personalized recommendations for a primary user based at least on stored data about the primary user. The method comprises a server computer obtaining data related to one or more users who are relevant to the primary user, wherein the one or more users relevant to the primary user are similar to the primary user. The server computer then determines at least one group of users relevant to the primary user based on the data related to the one or more users, each relevant group including one or more relevant users to the primary user. Then, the server computer also obtains data related to one or more entities. The method further comprises determining, at the server computer, one or more entities relevant to the primary user based on the data related to the one or more entities, and then associating the at least one relevant group of users with the one or more relevant entities. The server computer then generates one or more potential candidate factors based on at least the relevant group of users or the one or more relevant entities.
  • The method performed by the server computer may further comprise selecting, by the server computer, a set of factors from the one or more potential candidate factors and displaying the set of selected factors to the primary user, wherein the potential candidate factors are used to determine recommendations to the primary user.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • A further understanding of the nature and advantages of the present invention may be realized by reference to the following drawings. In the appended figures, similar components or features may have the same reference label, with distinct instances of the component or feature distinguished by parenthetical indices. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.
  • FIG. 1 is a block diagram of a personalized recommendation system according to aspects of the present invention.
  • FIG. 2 is a block diagram of an exemplary flow that may be performed by the personalized recommendation system to explain recommendations according to aspects of the present invention.
  • FIG. 3 is a block diagram of a modularized personalized recommendations system according to aspects of the present invention.
  • FIG. 4 is an illustration of a user display of selected factors to explain recommendations according to aspects of the present invention.
  • FIG. 5 is an illustration of a user display of selected factors and user input to explain and provide recommendations according to aspects of the present invention.
  • DETAILED DESCRIPTION OF THE INVENTION
  • An improved recommendation system is described herein. In order to provide a context for describing embodiments of the present invention, embodiments of the invention will be described herein with reference to a user device that receives user recommendations. Those skilled in the art will recognize, however, that the present invention is not limited to such embodiments.
  • It should be understood that, given the complexity of the operations performed, the invention is implemented using computing elements and may also include communication elements that convey data between disparate locations.
  • In various embodiments, systems which recommend entities may use a variety of algorithmic approaches to produce predicted evaluations or recommendations, hereafter referred to as recommendations. Prior to discussing embodiments of the invention, some terms used herein are described below:
  • “Entities” according to embodiments of the invention may be anything that can be represented or referred to in a computer system. In some embodiments, entities are providers of goods and services, such as restaurants or nightclubs. In other embodiments, entities are particular goods or services or events, such as a particular dish at a restaurant or a one-time or repeating event such as a concert. In others entities are documents or sections of documents, such as text pages or musical compositions, to be presented to a user. In other embodiments, an entity may be a relationship or potential relationship between two people. For example, a dating recommendation system may recommend in favor of or against a potential relationship between one user and another. In another example, entities may be individuals.
  • “Recommendations” according to embodiments of the invention may be along a single or multiple dimensions, and/or over a discrete or continuous domain. Recommendations may be presented in a numerical, textual, pictorial, auditory, video, or tactile fashion. Recommendations may be include any of these characteristics, either alone or in combination. For example, a user may rate a restaurant on a scale of 1, 2, 3, 4, or 5 stars, with 1 star being the lowest rating, and 5 stars being the highest rating. In another realization, a user may instead rate a restaurant on a 1-5 star scale for taste rating, 1-4 “$” scale for price rating, and add an optional one word description.
  • In some implementations, recommendations might be called “evaluations” or used in a similar manner. For example, an evaluation might be done by a team of doctors choosing a picture that best represents the mood or situation of a clinical patient, in effect, “recommending” which is the “best” image to used.
  • Another example of an evaluation or recommendation may include citizen journalists recording a five second video to make a review or recommendation, or an audio summary of a protest or event. In another example, recommendations may use three dimensional sensing technologies to have dancers record their expressions in response to listening to a composition. Another example may include a user operating an application on a mobile device, and dragging pictorial representations of dishes into different groups or piles categorized as “tasty”, “yucky”, and “only sometimes”. In another example, a recommendation may be inferred from aspects of a user's behavior such as whether the user visits a particular web page, looks in one direction or another, or whether the user walks or turns in one way or another.
  • An “affinity graph” according to embodiments of the invention is a model of the affinity between users or entities and users or entities, either in general or with respect to a given attribute or cluster of attributes of the users or entities, as a graph. Affinity graphs may represent relationships between users and entities, users and other users, or entities and other entities, and may be used to determine recommendations. An example of an affinity graph is a social graph wherein nodes represent users of a social interaction system and edges between nodes represent “friend” links. It should be understood, however, that some embodiments may not represent users explicitly and might not require that a person interacting with the application be identified or authenticated.
  • Attributes provide additional information about a node in the graph. For example, an attribute may be the zoom level of a camcorder type or the primary cuisine type of a restaurant. Entities may be represented by nodes, with individual users being one type of entity. Edges represent relationships between nodes. Directional edges can model unidirectional relationships, such as if person A follows person B on Twitter. A symmetric relationship, like friends on Facebook, may be represented as two directional edges, or a unidirectional edge.
  • “Evaluation data” for a user may be, according to embodiments of the invention, inputs into a recommendation service provided by a personalized recommendation system described herein. For example, evaluation data may include ratings on a discrete star scale for the quality of a restaurant for a restaurant discovery website/product or answers to personality questions for a match making website/service. Evaluation data may be associated with one entity or a set of entities. For example, a user may rate Jane's Sushi 90/100 and may rate sushi restaurants in general 82/100.
  • A “recommendations provider” is a service that generates recommendations and predictions of evaluation data of entities for users. The recommendations provider may be on the same device or different devices from the device displaying the recommendations. The recommendations provider may be operated by the same system collecting the evaluation data, or may be operated on a separate system as the system collecting the evaluation data.
  • An “explanations provider” is a service that generates explanations of the recommendations (i.e., recommendation explanations) provided by the recommendations provider. The explanations provider may be on the same device or different devices from the device displaying explanations. The explanations provider and the recommendations provider may be operated on the same or on different systems from each other. The explanations provider may be operated on the same system or a separate system as the system collecting the evaluation data.
  • As used herein, an “unknown user” is a user of the recommendations provider or other service known to the recommendations provider for which the service has access to evaluation data but is not connected to a primary user or group of users viewing a recommendation by an appropriate link or links in the affinity graph. For example, if the affinity graph is a family tree, then Jane and her mother, Joan, would be connected, but Jane's best friend Becky would be an unknown user in evaluating recommendations for Jane and Joan as primary users. Depending on the system or the primary user, a user may be known or unknown in different contexts. For example, in a system with an affinity graph linking users with their favorite celebrities and an affinity graph linking users and their friends, the celebrity linkage might be appropriate for movie or fashion recommendation explanations for a primary user, but not appropriate for dentist recommendations. The celebrity's dentist may not be relevant to the primary user, but the dentist of the primary user's coworker may be more relevant and useful to the primary user.
  • As used herein, a “known user” is a user of the recommendations provider or another service known to the recommendations provider for which the recommender knows evaluation data and which is connected to the user (group of users) viewing a recommendation by an appropriate link or links in the affinity graph. For example Joan is a known user to Jane from the example above.
  • An “explained entity” is a specific entity being recommended to a user or group of users for which recommendations and recommendation explanations are being generated. The explained entity may or may not have associated evaluation data. A “known entity” is an entity with sufficient associated evaluation data to show a user how the known entity has been evaluated. For example, in a system for recommending theater performances, a user may have given the category “Musicals” two thumbs up to indicate approval, thereby making the category a known entity (because the thumbs up are evaluation data that can be shown to a user). As a further example, when viewing a page about “Joseph and the Amazing Technicolor Dreamcoat,” where the user has not given a thumbs up or thumbs down to that musical, that entity would not be a known entity because the system would not have associated evaluation data (the thumbs up or down) that could be shown to the user. Nevertheless, the user's previous thumbs down ratings for two other Andrew Lloyd Webber plays could indicate that “Joseph and the Amazing Technicolor Dreamcoat” is an explained entity (because the previous ratings could be used to create a recommendation for “Joseph and the Amazing Technicolor Dreamcoat”).
  • An “evaluation description” is the language or messaging used to describe evaluation data for an entity. For example, in a system with 1 to 5 star ratings, 1 star might be “I hated it,” 2 stars might be “I disliked it,” 3 stars might be “I′m okay with it,” 4 stars might be “I liked it,” and 5 stars might be “I loved it.” The descriptions need not be a direct mapping from an evaluation to prose and may be dependent on the context in which the evaluation description is viewed. From example, in a system with 1 to 5 star ratings where various entities are being compared, those with 4 and 5 stars may be described and categorized as “enjoyed” and 1 and 2 stars as “Did not care for.” Evaluation descriptions may include textual information as well as richer media sources in the same or similar manner as recommendation explanations.
  • A “predicted evaluation description” is the language or messaging used to make predictions about what known or unknown users would evaluate an explained entity. For example, when a recommendations provider for kitchen tiles has an affinity graph that models followers and a primary user following Joe, then when the primary user is viewing an entity describing marble tile that the recommendations provider believes Joe would rate 89/90 on beauty and 75/80 on long term value, a predicted evaluation description might be, “Joe would probably love the marble tile.” Predicted evaluation descriptions may include textual information as well as richer media sources in the same or similar manner as recommendation explanations. For example, with “Joe would probably love the marble tile,” a picture of Joe may appear above his name and both the picture and his name may be hyperlinked to a page describing his preferences.
  • Recommendations
  • In FIG. 1, an exemplary personalized recommendations system according to an embodiment of the invention is shown. The personalized recommendations system 160 may be coupled to a plurality of users, such as a primary user 100, secondary user 102, or any number of users 104. There may be relationships between specific users, for example, the primary user 100 and secondary user 102 may be friends, family members, or co-workers. The plurality of users may be coupled to the personalized recommendations system 160 through associated user devices 110, 112, and 114. User devices may include mobile phones (e.g., a Blackberry™ phone, an iPhone™ device, an Android™ device), tablets (e.g., an iPad™ tablet), or other electronic devices.
  • The personalized recommendations system 160 may obtain inputs from any number of users 100, 102, 104, registered with the personalized recommendations system 160. The users 100, 102, and 104 may be interconnected and have relationships with one another. The inputs may be transmitted through the user devices 110, 112, 114 via an application. The user devices 110, 112, 114 may be connected to the personalized recommendations system 160 through the internet, a wireless network, a telecommunications network, or any suitable communications network. For example, the primary user 100 may use the primary user device 110 to download and install an application issued by the personalized recommendations system 160. Through the application, the primary user 100 may edit personal ratings for entities, such as venues and restaurants, and other personal preferences, such as cuisine, type of food, price range, location, distance from current location, etc. The primary user 100 may also provide feedback items or data through the application to respond to recommendations, to add or remove entities, and/or rate an experience at an entity. In other embodiments of the invention, entities may include merchants and service providers.
  • Additional inputs to the personalized recommendations system 160 may come from external data sources, for example, a social network computer system such as the Facebook™ computer system 140(A), a public messaging system such as the Twitter™ messaging system 140(B), and/or any number of other data sources 140(N) via the user accounts 150(A), 150(B), 152(A), 152(B) with the data sources and/or user devices 110, 112. The external data sources may also provide raw location data of the users, for example, through a “check-in” through system 140(A), system 140(B), or other social networking provider system capable of providing raw location data. The other data sources 140(N) may themselves have associated user accounts, which may map directly or indirectly to user accounts of other systems.
  • The personalized recommendations system or provider may use affinity graphs as models of the affinity between users or entities and users or entities, either in general or with respect to a given attribute or cluster of attributes of the users or entities, as a graph. Affinity graphs may represent relationships between users and entities, users and other users, or entities and other entities, and may be used to determine recommendations. Attributes provide additional information about a node in the graph. For example, an attribute may be the zoom level of a camcorder type or the primary cuisine type of a restaurant. Entities may be represented by nodes, with individual users being one type of entity. Edges represent relationships between nodes. Directional edges can model unidirectional relationships, such as if person A follows person B on Twitter. A symmetric relationship, like friends on Facebook, may be represented as two directional edges, or a unidirectional edge.
  • These relationships may be about different associations other than social affiliations, such as an expertise graph where an edge from user A to user B indicates that user A believes user B to be an expert. There may also be weights on these edges, which for example, might represent the confidence in another user's expertise, or represent how familiar two users are with each other. Affinity graph nodes may be further augmented with additional information such as short form textual messages, pictures, check-ins, other explicit or implicit annotations that are used to denote a user has visited or prefers a location, or other explicitly tagged content. The affinity graph may be constructed by users choosing to link themselves to other nodes, or by aggregation and analysis methods which infer the links, weights, and/or nodes. In embodiments, a link may be inferred from who sends messages to whom in a network, or links may be taken as given in the form of user-determined friendship relationships and weights on the links inferred from messages, or information may be inferred by who or what a user looks at.
  • The recommendations provider operated on the personalized recommendations system may also use evaluation data, such as inputs into the recommendation service provided by the personalized recommendation system described herein. For example, evaluation data may include ratings on a discrete star scale for the quality of a restaurant for a restaurant discovery website/product or answers to personality questions for a match making website/service. Evaluation data may be associated with one entity or a set of entities. For example, a user may rate Jane's Sushi 90/100 and may rate sushi restaurants in general 82/100.
  • Using affinity graphs, evaluation data, and/or other algorithms and data, the recommendations provider operated by personalized recommendations system may generate recommendations along a single or multiple dimensions, and/or over a discrete or continuous domain. Recommendations may be numerical, textual, pictorial, auditory, or tactile in fashion. Recommendations may be include any of these characteristics, either alone or in combination. For example, a user may rate a restaurant on a scale of 1, 2, 3, 4, or 5 stars, with 1 star being the lowest rating, and 5 stars being the highest rating. In another realization, a user may instead rate a restaurant on a 1-5 star scale for taste rating, 1-4 “$” scale for price rating, and add an optional one word description. Another example of a recommendation (e.g., evaluation) involves having a team of doctors choosing a picture that best represents the mood of a clinical patient. Other examples may include recording a five second video to make a review or recommendation, an audio summary of a protest or event, where a user looks or how longs the user looks at something, or in a mobile application, dragging pictorial representations of entities into rating categories.
  • In embodiments, a recommendations provider may construct recommendations using statistical techniques that are known in the literatures of statistics, artificial intelligence, and machine learning. An example of such a technique is using cosine similarity in an application where the ratings are on a 1-5 star rating scale to determine how similar one entity's set of ratings is to another entity's set of ratings, and to provide recommendations of the first entity based on the similarity. Another example of such a technique is using latent Dirichlet allocation in an application where the ratings are combinations of one or more words like “yucky,” “tasty,” etc., to model the ratings as words associated with topics that an entity is associated with in a topic model. Another example of such a technique is using natural language processing to extract sentiment information from free-form textual statements related to an entity and using a collaborative filtering technique with the information thus extracted.
  • Example recommendations may be, but are not limited to being, textual (e.g., a short 20 word comment), continuous (e.g., a ratings slider), discrete as in star ratings, pictorial (e.g., a representative picture, drawing, or photograph which most closely resembles the entity), auditory (e.g., a volume measurement or review), gestural (e.g., a swipe up or down), visual (e.g., an interpretive dance for the entity), or some combination thereof. Recommendations may involve adding edges or weights, either individually or some combination to one or more affinity graphs.
  • Some recommendations may not be intuitive to the primary user viewing the recommendation. The personalized recommendations system according to embodiments of the invention presents one or many explanations for why an entity has been included in a recommendation to a primary user or set of users. An explanation of the recommendation may or may not reflect computations performed to generate the recommendation or factors used to perform the computations. Recommendation explanations may be considered as a set of plausible reasons that a given entity or entities would be a good choice for a specific user or group of users.
  • For example, if a recommendation is given to a user that is derived from a statistical technique that incorporates large quantities of information, and that uses features of the information that are learned by machine and are not human-interpretable, no aspect of the decision to make the recommendation may be clearly explainable to the user. Instead, the system might construct an explanation by, for example, calculating a similarity score between an entity being recommended and another entity the user has already rated, and reporting on the similarity.
  • Recommendation Explanations
  • A conceptual architecture of an exemplary system according to an embodiment of an invention is diagrammed in FIG. 2. There may be a primary user or primary group of users 202 in which a recommendation is to be provided to. The personalized recommendations system may have a user account database storing data (e.g., evaluation data) related to all users, including the primary user. The recommendation for the primary user includes recommendations to entities within a group of explained entities 204. For example, the personalized recommendation system may have an entity database of stored data for explained entities 204, such as restaurants, hotels, and other vendors providing goods and/or services to users 202.
  • Explained entities 204 may be entities that are recommended to a primary user or group of users 202 for which recommendations and recommendation explanations are being generated. The explained entity may or may not have associated evaluation data, in other words, it may not be a known entity, which is an entity with sufficient associated evaluation data to show a user how the known entity has been evaluated.
  • In step S1, the personalized recommendations system may perform similar user and entity selection. Data related to primary user or primary user group 202 and data related to explained entities 204 may be collected, and the personalized recommendations system may use this data to determine similar users 206 and similar entities 208. Similar users and similar entities are identified as relevant groups of users and entities, respectively, relative to the primary user or primary group of users 202 and the entities to be explained 204. In some embodiments, only relevant entities are determined and used to generate personalized recommendations and/or explanations, and data related to similar or relevant users is not used.
  • Next, in step S2, the primary user 202, the entities to be explained 204, the similar users 206, and similar entities 208 are used alone or in combination in a process to generate candidate factors for recommendation explanations. From these potential explanation factors the system may determine and select, in step S3, a subset to show to the user. Finally, in step S4, the system crafts a display of the explanation factors or recommendation explanations to present to the primary user.
  • Any number or part of the steps may be elided or combined. Such combinations may be necessary to improve the speed of computation, ease the maintainability of an implementation, or allow customization by a primary user. For example, an implementation for a horse recommendation in Montana, the personalized recommendations explanation system may decide to generate and process the English language HTML explanations simultaneously while generating explanation factors, so that there is no need for display generation after factor selection is done. Thus, step S4 may occur simultaneously with step S2 instead of chronologically after step S3. This may be done to improve performance (e.g., performing steps in parallel to improve speed) or to simplify the implementation, among other possible reasons.
  • As depicted in FIG. 2, in step S1, locating similar users 206 and similar entities 208 may produce many subsets of all the users and entities known to the personalized recommendations explanation system. For example, it may produce “user and user,” “user and entity,” and “entity and entity” pairings with optional subsets or supersets of any combination, such as pairing a group of users with a set of groups of users. Locating similar users and entities may take into account attributes of the entities or users previously stored in the entity database and user account database, respectively. Attributes may also be data collected in real-time at the time of a recommendation request of the primary user, such as the time of day, and location information of the primary user, which may be used either alone or in combination with each other or with previously stored data. The user database and entity database could be NoSQL databases, an SQL database, another information retrieval system such as a computer filesystem, or an in-memory data store. In other embodiments, the system could have a temporary piece of data obtained from a third-party on demand, such that data does not need to be statically or locally stored. A database could consist of multiple storage facilities, such as a cache backed by a database or a normalized authoritative database and denormalized in-memory stores.
  • Exemplary attributes may include demographic information, user gender, user height, user weight, user income, entity revenues, the cost of items, cuisine type, previous evaluations done by the user, check-ins, other explicit or implicit annotations that are used to denote whether a user has visited or prefers a location or entity, or relations in an affinity graph(s). Demographic data may include gender, height, weight, income, or pre-determined attributes. Additional user attributes may also include explicitly stated preferences. The personalized recommendations explanation system recommending restaurants may allow a user to declare they are one or more of pescetarian, vegetarian, vegan, kosher, lactose intolerant, and wheat gluten intolerant. In an example of a restaurant recommendation system, two restaurants might be similar if they both serve the same or related cuisines, e.g., the system may consider cupcakes and desserts being similar cuisines, while Japanese and Chinese are not. Extending the example, a restaurant recommendation system may also deem two restaurants similar if their respective head chefs both apprenticed at the same restaurant, or if both restaurants are in a zip code with the same median family income and have been open for more than 3 years. All these forms of information may be learned or explicit. For example, user attributes such as gender might be inferred from user behavior; whether a restaurant is Japanese may be inferred or may be received from an external database, and whether Japanese or Chinese should be considered similar or dissimilar may be learned or preprogrammed.
  • In steps S2 and S3, the generation and selection of recommendation explanation factors may be computed using a variety of metrics incorporating different spatial, temporal, social, demographic, and user interaction based signals, either alone or in combination. Other examples of data user in selecting recommendation explanation factors include, but are not limited to, explicit user suggestions, demographic data, location data at the time a user or group of users is seeing an explanation, the time of day a primary user or group of users is requesting a recommendation and viewing recommendation explanation, explicit user suggestion of an explanation or part of an explanation, voting or other group selection of explicit user suggestions or system generated suggestions of an explanation or part of an explanation, table driven matching. Additionally, historical data, such as query history including click through and other interest actions to a search or discovery service, browsing history, purchase history, the location of the primary user or group of users over time, and information in third party services from previous visits or evaluations, may be used in generating and selecting recommendation explanation factors. The selection of factors may also involve psychological models of behavior which may or may not be culturally dependent. For example, it could be the case that American users respond to explanations better when referred to as “users,” while British prefer to be referred to as “patrons.” The selection of factors may also involve functions to be maximized such as a prediction of profitability, user retention, or conversion rate (of a number viewers or visitors into the number of users who take some action, such as making a purchase). A function to be maximized may be predicted for this purpose using statistical techniques (for example, regression analysis).
  • In generation and selection of recommendation explanation factors for recommendations of explained entities 204 for primary user or primary user groups 202, the system may access evaluation descriptions of the explained entities 204 and/or similar entities 208. Evaluation descriptions describe evaluation data for an entity, relating to a user's experience or relationship with that entity. For example, in a system with 1 to 5 star ratings for a restaurant, 1 star indicate that the user “hates” it, 2 stars might indicate that the user “dislikes” it, 3 stars might indicate that the user is “okay” or indifferent to it, 4 stars might indicate that the user “likes” it, and 5 stars might indicate that the user “loves” it. The descriptions may not be a direct mapping from an evaluation to prose and may be dependent on the context in which the evaluation description is viewed. For example, in a system with 1 to 5 star ratings where various entities are being compared, those with 4 or 5 stars may be grouped together and categorized as “Enjoyed,” and 1 or 2 stars as “Did not care for.” Evaluation descriptions may include textual information as well as richer media sources in the same or similar manner as recommendation explanations.
  • Predicted evaluation descriptions may also be used to make predictions about what known or unknown users would evaluate an explained entity. For example, a recommendations provider for kitchen tiles may have an affinity graph modeling followers and a primary user who is following Joe. When the primary user is viewing an entity describing marble tile which the recommendations provider believes Joe would rate 89/90 on beauty and 75/80 on long term value, a predicted evaluation description might be, “Joe would probably love the marble tile.” Predicted evaluation descriptions may include textual information as well as richer media sources in the same or similar manner as recommendation explanations. For example, with “Joe would probably love the marble tile,” a picture of Joe may appear above his name and both the picture and his name may be hyperlinked to a page describing his preferences.
  • As another example, consider a system in which bars are evaluated on a 1 to 5 star rating scale, which has an affinity graph which models friendships, in which Abby is viewing a bar Formosa. When the recommendations provider predicts a rating for Abby's friend, Sarah, the predicted evaluation description might be: “would probably hate” for a 1 star prediction, “would dislike” for a 2 star prediction, “would be okay with” for a 3 star prediction, “would like” for a 4 star prediction and “would love” for a 5 star prediction. There are many other ways in which such a description could be phrased; for instance, a 1 star prediction could give a description of “would probably rate 1 star.” Note that Abby's and Sarah's predicted ratings may be the same or different.
  • FIG. 3 shows a block diagram of an exemplary modular personalized recommendations system 160 according to an embodiment of the invention, where the personalized recommendations system 160 is configured to provide explanations of the personalized recommendations. In other embodiments of the invention, an explanations provider may be separate from the recommendations provider, and may be operated by different systems. In the example shown in FIG. 3, the personalized recommendations system 160 operates as the recommendations provider and explanation (i.e., recommendations explanation) provider.
  • The personalized recommendations system 160 may be a server computer 700, which may be a single computer, a cluster of computers, a server, or a super computer. Processes and methods implemented by or at the server computer could also be implemented by or at a client, in various embodiments. The server computer 700 may comprise a processor 740, and a non-transitory computer readable medium 750, on which code is stored. There may be several databases to store data, for example, an entity database 730(a), a factor database 730(b), a user database 730(c), and a media object database 730(d). Stored in the entity database 730(a) may be evaluation data relating to entities, including entity meta-data (e.g., location, hours of operation, specialty, rating). Entity meta-data may include any sub-category of data related to the entity. The factor database 730(b) may store previously determined and selected factors from a plurality of users. The user database 730(c) may store user data obtained from a plurality of users, such as real-time data from social networking sites (e.g., tweets, status updates), and stored data related to a plurality of users, such as user preferences and settings (e.g., gender, dietary restrictions). The real-time data may be explicit or implicit, and may be inferred or its significance or relevance may be inferred, and may include text, images, video, audio, or other media. The user data may be directly obtained from the user or may be obtained from external data sources, such as a social networking site associated with the user. The personalized recommendations system 160 may also include a media object database 730(d), storing media objects, such as high resolution photos. The media objects may be associated with entities, recommendations, and/or the target user, and may be used in the display of recommendations and/or recommendation explanations. The association between media objects and entities, recommendations, or a user may be inferred or may be preprogrammed or derived from a preexisting database.
  • The code stored on the non-transitory computer readable medium 750 may be in modules, and is executable by the processor 740 to perform functions, which are implemented as processing steps described above by the personalized recommendations system providing recommendation explanations. The non-transitory computer-readable medium 750 may comprise software modules, such as a similar user selection module 701, a similar entity selection module 702, a factor generation module 703, a factor selection module 704, a factor display module 705, and an explanation module 706. The functions of these modules are described in the process flows described above and in FIG. 2.
  • The personalized recommendations system 160, to communicate with external data sources, social networking sites, and users, may include a network interface 720. The network interface 720 includes hardware and software that enables the personalized recommendations system 160 to transmit and receive data over the Internet or any other communications network (e.g., telecommunications network, cable, radio).
  • Displaying Recommendations and Recommendation Explanations
  • Once recommendations have been determined and their explaining factors have been selected, the recommendations and recommendation explanations may be displayed. The display of recommendation explanations performed in step S4 may be in the form of textual information on a screen of a mobile device. However, the means by which information is communicated is not limited to textual information or to a device screen, but may include instead or in addition richer media sources or forms of communication, such as two or three dimensional models, pictures, a heads-up display, video, hyperlinked text, web pages, styled text, recorded sounds, computer generated sounds, vibrations of a tactile device, pressure exerted on the user by haptic technology, electric shocks or pulses, the control of remote devices, and the generation of scents. As an example of a restaurant recommendation system, the system may explain a recommendation for an Italian restaurant by indicating that five of a primary user's friends go to the restaurant and then providing pictures of the five friends, where each picture is linked to more information from each friend, such as possibly including the friend's evaluation (e.g., review) of the restaurant. The display of recommendation explanations may be visible or not depending on external context, for example, not showing recommendations about user's cuisine preferences to other users until that primary user has rated at least ten places on a 1-5 scale.
  • Similar to the generation and selection of recommendation explanation factors, the explanation of display generation may be computed using a variety of metrics incorporating different spatial, temporal, social, demographic, and user interaction based signals, either alone or in combination. Other examples of data used in explaining display generation include, but are not limited to, explicit user suggestions, demographic data, location data at the time a user or group of users is seeing an explanation, the time of day a primary user or group of users is requesting a recommendation and viewing recommendation explanation, explicit user suggestion of an explanation or part of an explanation, voting or other group selection of explicit user suggestions or system generated suggestions of an explanation or part of an explanation, table driven matching. Additionally, historical data, such as query history (including click through and other interest actions to a search or discovery service), browsing history, purchase history, the location of the primary user or group of users over time, and information in third party services from previous visits or evaluations, may be used in generating and selecting recommendation explanation factors. The selection of factors may also involve psychological models of behavior which may or may not be culturally dependent. For example, it could be the case that American users respond to explanations better when referred to as “users,” while British prefer to be referred to as “patrons.”
  • In explaining a predicted evaluation, the personalized recommendations system may count or aggregate a number of users with a similar actual or predicted evaluation. When generating explanations, incorporating this number the number may be shown directly to the user(s) or a description or summary may be shown instead. The language or messaging used is known as a numeric descriptor and may include textual information as well as richer media sources in the same or similar manner as discussed earlier in regards to recommendation explanations.
  • FIG. 4 shows an illustration of a user display 400 in which a recommendation system for restaurants is displayed explained entities. The predicted rating 402 is 3.5 stars, and shows a percentage 404 of 84%, indicating to the primary user, “You'll love it.” The text in 406 describes the explained entity, Kaygetsu (a sushi restaurant), and also indicates that there exists sufficient evaluation data for Kirala (a different sushi restaurant), and there are over 10 people who like Kirala and Kaygetsu. Therefore, the explanation may be: “Many people who like Kirala also like Kaygetsu,” where hovering over “many” brings up a slideshow of photographs 408 of a subset of the people's faces who like both restaurants where the photos are selected according to a model of the user(s)'s affinity for facial features. The slide of photographs of relevant users to the primary user or textual description may indicate users who have similar tastes to the primary user, thus predicting and suggesting that the primary user may like the same restaurants as the relevant user (e.g., Gordo Taqueria, The Counter, Bouchon), or dislike others (e.g., McDonald's).
  • In an embodiment of the invention shown in FIG. 4, one packaging of explanation factors may incorporate a numeric descriptor of unknown users, an evaluation description for a known entity A, a known entity A, an evaluation description for an unknown entity B, and an unknown entity B. For example, “A few people who like Kirala also like Kaygetsu” 406 or “People who love Bi Rite Creamery also love Mitchell's Ice Cream”.
  • In another embodiment of the invention shown in FIG. 4, another packaging of explanation factors may incorporate an objective statistic and a relation on the objective statistic (possibly involving one or more affinity graphs). For example, “Several people who have similar tastes to you love Bouchon” 406, “People who have similar tastes to you hate Plutos”, and “Many people with similar taste preferences to you are okay with Il Fornaio” may all be such explanation factors. Examples of objective statistics may include counting, or other forms of aggregation and cosine similarity in a vector space, either alone or in combination.
  • Explicit or implicit annotations, such as a check-in or status update on a social media website, may indicate that a user has visited or prefers a location or entity. These explicit or implicit annotations may be used by the system to determine and specify narrowing relationships between users and entities. A check-in on Facebook is an example of such an annotation, and may be used to determine a number of a primary user's relevant users (e.g., Facebook friends, Twitter follows) that have gone there in the past and/or the frequency at which the primary user's relevant users visit the entity. For example, “A few people you follow came here last month” 410, “Abby (one of your followers) who also likes Plutos checked in here”, “Michael Preysman (a person you follow) checked in at Baretta and has also checked in here” 412, “Many people who have checked in at Delfina have checked in here” 412, “You checked in at Café Borrone six months ago” 412, “You checked in at Plutos six times in the last two weeks”, and “Paul came here recently and loved it” 414 are all example explanations generated from such a narrowing.
  • Narrowing relationships may involve location data that may be obtained implicitly through a user's mobile device or other means (e.g., GPS, location services). For example, “Koi is the best match for your tastes in San Francisco”, “Denny's is the best match for your tastes in Sunnyvale that is open now” 416, “The Counter is the best Burger Joint for you in Palo Alto”, and “Burger King is the best Burger Joint for you in Cupertino open now” all explain a restaurant choice in terms of location with optional extra descriptions based on preferences or restaurant attributes like hours of operation cross-referenced with the current time. Location data may also be used to compute statistics for geographic areas which can be explanation factors. For example, “Berkeley is known for sushi, try Kirala” 416 and “New Orleans is known for Po Boys, try them at Mahony's”. Location may also include facts of general interest or interest to the user, for example: “Cafe Du Monde is the home of the Beignet, try one!”
  • While the preceding examples and FIG. 4 are related to restaurants, the preceding explanation factors may also be extended to narrowing relationships which may involve one or more affinity graphs, and in other areas. For example, in a song recommendation system, “Many people you follow hate Madonna”, “Abby (one of your followers) who has similar music preferences to you loves Pink”, “Sting would love Meatloaf”, and “Eminem who has similar music preferences to you would like Mozart” all provide more specific information than that computed by an objective statistic alone. One class of narrowing relations involves selecting a subset of users who have each rated a subset of entities and showing them as explanation factors. For example, in a restaurant recommendation system, there may be 40 ratings from 40 users about a given dish, with 6 users rating it 5 stars, and 34 rating it 1 star. If the primary user whose recommendations are being explained to are more similar to the 6 users than the other 34, according to an objective statistic, like cosine similarity, the recommendations explanation may call attention to the 6 and elide the other 34. In other words, the evaluation data related to users that are more similar to the primary user may be weighted more heavily than evaluation data from users that are not as similar to the primary user.
  • Recommendation explanations may be displayed in a variety of contexts which may or may not include the recommendation being explained. For example, FIG. 5 shows an exemplary user display 500 which presents a list of cosmetic surgery options (e.g., nose job, face lift, calf implants, liposuction), along with a percentage ranking of how much the system recommends the given surgery to the primary user, see section 506. When the primary user selects surgery options from the list, they are presented with a textual description of why the surgery was ranked that way for the primary user, thus explaining its ranking recommendation. The display may also include other descriptions (either textual or other rich media) of relevant information 502, such as the primary user's insurance or other information that the primary user may find interesting. The display may also include photographs 504 or other media.
  • Recommendation explanations may also be used to explain the ranking of entities as part of a browsing, search, or other information retrieval experience. For example, the recommendation explanations associated with a list of bars near a group of users could include text to explain why one bar was ranked higher than another, examples being “15 Romelo has the best cocktails in the city” and “Cocktails at BarOne are second only to those at 15 Romelo”.
  • The system may monitor and track various metrics, and optionally incorporate those while selecting recommendation explanation factors for their display. For example, explicit collections or lists of entities a user has annotated, frequency at which users select an entity with a given type of explanation optionally adjusting for user location, the time of day the user is viewing the recommendation explanation, present and past purchase history for the user or group of users, and comparisons between the primary user and other users; may all be used either alone or in combination, in generating and selecting recommendation explanations, and the display of the selected recommendation explanations.
  • Embodiments of the invention may be extended such that an explanation of a recommendation relates to a target user's friend (e.g., on Facebook) or follower/followee (e.g., Twitter). Location information may be used in the context of an affinity graph to determine recommendations. Explanations of recommendations may also relate to a prediction of what the target user might do, or how they might think or evaluate an entity. Then, the per-user data in the affinity graph about the user (e.g., a user's followees/friends may lead to an inference about the user) to determine what explanation is likely to be most useful, valuable, and relevant. The explanation may involve similarity between users. The explanations may be offered to the user as a set of selections, which involves what explanations have been helpful in the past for the same or similar users. In some embodiments, there may be multiple simultaneous target users, or there may be one target user and several associated users for whom searches should also be relevant. Input data from such multiple users can be presented synchronously or asynchronously. Some embodiments may provide methods of operation in which they do not need to know who a user is; for example, there may be a kiosk mode in which members of the public can use the system without identifying themselves.
  • The description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the invention. Rather, the description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment of the invention. It is to be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention as set forth in the appended claims.
  • Specific details are given in the description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, systems may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
  • Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations may be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
  • Moreover, as disclosed herein, the term “storage medium” may represent one or more devices for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term “computer-readable medium” includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and various other mediums capable of storing, containing or carrying instruction(s) and/or data.
  • Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as storage medium. A processor(s) may perform the necessary tasks. A code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
  • Further embodiments may be envisioned to one of ordinary skill in the art after reading this disclosure. In other embodiments, combinations or sub-combinations of the above disclosed invention may be advantageously made. The example arrangements of components are shown for purposes of illustration and it should be understood that combinations, additions, re-arrangements, and the like are contemplated in alternative embodiments of the present invention. Thus, while the invention has been described with respect to exemplary embodiments, one skilled in the art will recognize that numerous modifications are possible.
  • For example, the processes described herein may be implemented using hardware components, software components, and/or any combination thereof. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the claims and that the invention is intended to cover all modifications and equivalents within the scope of the following claims.

Claims (30)

  1. 1. A computer-implemented method for generating explanatory data from a personalized recommendations process for a primary user based at least on stored data about the primary user, the method comprising:
    determining the primary user for whom explanatory data is to be generated, the primary user being one of a plurality of users over which recommendations are considered;
    obtaining data related to a set of relevant users, wherein the set of relevant users comprises one or more users in the plurality of users that are deemed relevant to the primary user, wherein relevancy of a user in the set of relevant users to the primary user is based, at least in part, on similarities, according to stored user data, between the relevant user and the primary user;
    obtaining, at the server computer, entity data from a plurality of data sources, wherein the entity data is associated with an entity in a plurality of entities;
    storing the entity data at the server computer, wherein the entity data associated with the plurality of entities is stored in an entity database;
    determining, at the server computer, one or more entities relevant to the primary user based on the data related to the one or more entities;
    determining, by the server computer, one or more potential candidate factors based on at least the one or more relevant entities; and
    generating, by the server computer, explanation data based on the one or more potential candidate factors used in the personalized recommendations process.
  2. 2. The method of claim 1, further comprising:
    selecting, by the server computer, a set of factors from the one or more potential candidate factors; and
    communicating, by the server computer, the set of selected factors to the primary user, wherein the potential candidate factors are used to determine recommendations to the primary user.
  3. 3. The method of claim 1, further comprising:
    determining, at the server computer, at least one group of users relevant to the primary user based on the data related to the one or more users, each relevant group including one or more relevant users to the primary user;
    associating, by the server computer, the at least one relevant group of users with the one or more relevant entities;
    determining, by the server computer, one or more potential candidate factors based on at least the relevant group of users; and
    generating, by the server computer, explanation data based on the one or more potential candidate factors used in the personalized recommendations process.
  4. 4. The method of claim 1, wherein associating the at least one relevant group of users with the one or more relevant entities includes determining a relationship between a relevant user with a relevant entity, a relationship between a relevant entity with another relevant entity, or a relationship between a relevant user with another relevant user.
  5. 5. The method of claim 1, wherein data related to one or more relevant users includes account information stored in a database of users, wherein the account information includes demographic data.
  6. 6. The method of claim 1, wherein the data related to one or more relevant users includes current location information of the user.
  7. 7. The method of claim 5, wherein the pre-determined attributes includes cuisine preferences or food allergies.
  8. 8. The method of claim 1, wherein the current location information is associated with data related to one or more relevant entities, wherein the data related to one of relevant entities includes previous data from the one or more relevant users related to the one or more relevant entities.
  9. 9. The method of claim 3, wherein selecting the set of factors is based on at least one of, or combinations of, spatial, temporal, social, demographic, or user history data previously stored in a database.
  10. 10. The method of claim 9, wherein selecting the set of factors is based on at least one of, or combinations of, spatial, temporal, social, demographic, or user current data collected in real-time.
  11. 11. The method of claim 9, wherein the user history data includes at least one of, or combinations of, query history, browsing history, or purchase history of the primary user.
  12. 12. The method of claim 9, wherein selecting the set of factors is based on table driven matching, wherein the tables are stored in the database.
  13. 13. The method of claim 10, wherein selecting the set of factors is based on psychological models of behavior of a demographic of the primary user.
  14. 14. The method of claim 3, further comprising:
    determining a set of media objects associated with the set of selected factors, wherein the set of media objects includes at least one of, or combinations of, textual information, pictures, video, hyperlinked text, webpages, styled text, and sounds; and
    communicating the set of media objects with the set of selected factors for the recommendation explanation.
  15. 15. The method of claim 3, wherein communicating the set selected factors includes multi-dimensional models, vibrations, electric shocks, pulses, or scents.
  16. 16. A server computer comprising a processor and a non-transitory computer readable medium, the non-transitory computer readable medium comprising code executable by the processor to implement a computer-implemented method for generating explanatory data from a personalized recommendations process for a primary user based at least on stored data about the primary user, the method comprising:
    determining the primary user for whom explanatory data is to be generated, the primary user being one of a plurality of users over which recommendations are considered;
    obtaining, at a server computer, data related to a set of relevant users, wherein the set of relevant users comprises one or more users in the plurality of users that are deemed relevant to the primary user, wherein relevancy of a user in the set of relevant users to the primary user is based, at least on part, on similarities, according to stored user data, between the relevant user and the primary user;
    obtaining, at the server computer, entity data from a plurality of data sources, wherein the entity data is associated with an entity in a plurality of entities;
    storing the entity data at the server computer, wherein the entity data associated with the plurality of entities is stored in an entity database;
    determining, at the server computer, one or more entities relevant to the primary user based on the data related to the one or more entities;
    determining, by the server computer, one or more potential candidate factors based on at least the one or more relevant entities; and
    generating, by the server computer, explanation data based on the one or more potential candidate factors used in the personalized recommendations process.
  17. 17. The server computer of claim 16, the method further comprising:
    selecting, by the server computer, a set of factors from the one or more potential candidate factors; and
    communicating, by the server computer, the set of selected factors to the primary user, wherein the potential candidate factors are used to determine recommendations to the primary user.
  18. 18. The server computer of claim 16, the method further comprising:
    determining, at the server computer, at least one group of users relevant to the primary user based on the data related to the one or more users, each relevant group including one or more relevant users to the primary user;
    associating, by the server computer, the at least one relevant group of users with the one or more relevant entities;
    determining, by the server computer, one or more potential candidate factors based on at least the relevant group of users; and
    generating, by the server computer, explanation data based on the one or more potential candidate factors used in the personalized recommendations process.
  19. 19. The server computer of claim 16, wherein associating the at least one relevant group of users with the one or more relevant entities includes determining a relationship between a relevant user with a relevant entity, a relationship between a relevant entity with another relevant entity, or a relationship between a relevant user with another relevant user.
  20. 20. The server computer of claim 16, wherein data related to one or more relevant users includes account information stored in a database of users, wherein the account information includes demographic data.
  21. 21. The server computer of claim 16, wherein the data related to one or more relevant users includes current location information of the user.
  22. 22. The server computer of claim 20, wherein the pre-determined attributes includes cuisine preferences or food allergies.
  23. 23. The server computer of claim 16, wherein the current location information is associated with data related to one or more relevant entities, wherein the data related to one of relevant entities includes previous data from the one or more relevant users related to the one or more relevant entities.
  24. 24. The method of claim 18, wherein selecting the set of factors is based on at least one of, or combinations of, spatial, temporal, social, demographic, or user history data previously stored in a database.
  25. 25. The method of claim 24, wherein selecting the set of factors is based on at least one of, or combinations of, spatial, temporal, social, demographic, or user current data collected in real-time.
  26. 26. The method of claim 24, wherein the user history data includes at least one of, or combinations of, query history, browsing history, or purchase history of the primary user.
  27. 27. The method of claim 24, wherein selecting the set of factors is based on table driven matching, wherein the tables are stored in the database.
  28. 28. The method of claim 25, wherein selecting the set of factors is based on psychological models of behavior of a demographic of the primary user.
  29. 29. The method of claim 18, further comprising:
    determining a set of media objects associated with the set of selected factors, wherein the set of media objects includes at least one of, or combinations of, textual information, pictures, video, hyperlinked text, webpages, styled text, and sounds; and
    communicating the set of media objects with the set of selected factors for the recommendation explanation.
  30. 30. The method of claim 18, wherein communicating the set selected factors includes multi-dimensional models, vibrations, electric shocks, pulses, or scents.
US13553599 2011-07-20 2012-07-19 Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user Pending US20130024464A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
US201161510003 true 2011-07-20 2011-07-20
US201161509995 true 2011-07-20 2011-07-20
US201161510004 true 2011-07-20 2011-07-20
US201161509999 true 2011-07-20 2011-07-20
US13553599 US20130024464A1 (en) 2011-07-20 2012-07-19 Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13553599 US20130024464A1 (en) 2011-07-20 2012-07-19 Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user

Publications (1)

Publication Number Publication Date
US20130024464A1 true true US20130024464A1 (en) 2013-01-24

Family

ID=47556530

Family Applications (3)

Application Number Title Priority Date Filing Date
US13553599 Pending US20130024464A1 (en) 2011-07-20 2012-07-19 Recommendation engine that processes data including user data to provide recommendations and explanations for the recommendations to a user
US13553684 Abandoned US20130024449A1 (en) 2011-07-20 2012-07-19 Method and apparatus for allowing users to augment searches
US13553646 Active 2033-10-17 US9189551B2 (en) 2011-07-20 2012-07-19 Method and apparatus for category based navigation

Family Applications After (2)

Application Number Title Priority Date Filing Date
US13553684 Abandoned US20130024449A1 (en) 2011-07-20 2012-07-19 Method and apparatus for allowing users to augment searches
US13553646 Active 2033-10-17 US9189551B2 (en) 2011-07-20 2012-07-19 Method and apparatus for category based navigation

Country Status (1)

Country Link
US (3) US20130024464A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8732101B1 (en) 2013-03-15 2014-05-20 Nara Logics, Inc. Apparatus and method for providing harmonized recommendations based on an integrated user profile
US8756187B2 (en) 2011-09-28 2014-06-17 Nara Logics, Inc. Systems and methods for providing recommendations based on collaborative and/or content-based nodal interrelationships
US20150025995A1 (en) * 2013-07-19 2015-01-22 Ebay Inc. Generating recommendations based on transaction data
US20150095278A1 (en) * 2013-09-30 2015-04-02 Manyworlds, Inc. Adaptive Probabilistic Semantic System and Method
US20150106367A1 (en) * 2013-10-14 2015-04-16 International Business Machines Corporation Utilizing social information for recommending an application
CN104850641A (en) * 2015-05-26 2015-08-19 无线生活(杭州)信息科技有限公司 Information recommendation method and device
US20150324434A1 (en) * 2014-05-09 2015-11-12 Paul Greenwood User-Trained Searching Application System and Method
US9509772B1 (en) 2014-02-13 2016-11-29 Google Inc. Visualization and control of ongoing ingress actions
US9507791B2 (en) 2014-06-12 2016-11-29 Google Inc. Storage system user interface with floating file collection
US9531722B1 (en) 2013-10-31 2016-12-27 Google Inc. Methods for generating an activity stream
US9536199B1 (en) 2014-06-09 2017-01-03 Google Inc. Recommendations based on device usage
US9542457B1 (en) 2013-11-07 2017-01-10 Google Inc. Methods for displaying object history information
US9614880B1 (en) 2013-11-12 2017-04-04 Google Inc. Methods for real-time notifications in an activity stream
US9870420B2 (en) 2015-01-19 2018-01-16 Google Llc Classification and storage of documents
US10078781B2 (en) 2014-06-13 2018-09-18 Google Llc Automatically organizing images

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8549407B2 (en) * 2007-12-05 2013-10-01 Ebay Inc. Multi-dimensional dynamic visual browsing
US20140108197A1 (en) * 2010-07-19 2014-04-17 Payme, Inc. Mobile system and method for payments and non-financial transactions
WO2012026095A1 (en) * 2010-08-26 2012-03-01 京セラ株式会社 Character string retrieval device
WO2012170589A1 (en) 2011-06-06 2012-12-13 Nfluence Media, Inc. Consumer driven advertising system
US9619567B2 (en) 2011-06-06 2017-04-11 Nfluence Media, Inc. Consumer self-profiling GUI, analysis and rapid information presentation tools
CN103425659B (en) * 2012-05-15 2017-06-09 阿里巴巴集团控股有限公司 Location-based information search method and server
US20140278992A1 (en) * 2013-03-15 2014-09-18 Nfluence Media, Inc. Ad blocking tools for interest-graph driven personalization
US9501503B2 (en) 2013-05-09 2016-11-22 Microsoft Technology Licensing, Llc Inferring entity attribute values
US9430476B2 (en) * 2014-03-13 2016-08-30 GM Global Technology Operations LLC Method and apparatus of user recommendation system for in-vehicle apps
US9743243B1 (en) * 2016-03-16 2017-08-22 International Business Machines Corporation Location context inference based on user mobile data with uncertainty

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010044759A1 (en) * 2000-05-17 2001-11-22 Matsushita Electric Industrial Co., Ltd. Information recommendation apparatus and information recommendation system
US6370513B1 (en) * 1997-08-08 2002-04-09 Parasoft Corporation Method and apparatus for automated selection, organization, and recommendation of items
WO2003054760A2 (en) * 2001-12-21 2003-07-03 Accenture Global Services Gmbh Evaluation of a plurality of alternatives based on a plurality of individual preferences
US20040103092A1 (en) * 2001-02-12 2004-05-27 Alexander Tuzhilin System, process and software arrangement for providing multidimensional recommendations/suggestions
US20060250935A1 (en) * 2003-02-28 2006-11-09 Ntt Docomo, Inc Radio communication system and radio communication method
US20090234664A1 (en) * 2008-03-11 2009-09-17 Schaffnit Wayne A System and method for recommending entertainment venues for specific occasions
US20110270774A1 (en) * 2010-04-30 2011-11-03 Microsoft Corporation Group Recommendations in Social Networks
US20120233191A1 (en) * 2010-11-22 2012-09-13 Salesforce.Com, Inc. Method and system for making content-based recommendations

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5758257A (en) 1994-11-29 1998-05-26 Herz; Frederick System and method for scheduling broadcast of and access to video programs and other data using customer profiles
WO2004003705A3 (en) * 2002-06-27 2004-08-26 Small World Productions Inc System and method for locating and notifying a user of a person, place or thing having attributes matching the user's stated prefernces
US7584181B2 (en) 2003-09-30 2009-09-01 Microsoft Corporation Implicit links search enhancement system and method for search engines using implicit links generated by mining user access patterns
US7606772B2 (en) 2004-11-04 2009-10-20 Manyworlds, Inc. Adaptive social computing methods
KR20060103909A (en) 2003-12-03 2006-10-04 코닌클리케 필립스 일렉트로닉스 엔.브이. Enhanced collaborative filtering technique for recommendation
US7818394B1 (en) 2004-04-07 2010-10-19 Cisco Techology, Inc. Social network augmentation of search results methods and apparatus
US7707220B2 (en) 2004-07-06 2010-04-27 Icosystem Corporation Methods and apparatus for interactive searching techniques
WO2007002820A3 (en) 2005-06-28 2007-05-18 Yahoo Inc Search engine with augmented relevance ranking by community participation
US8290810B2 (en) 2005-09-14 2012-10-16 Jumptap, Inc. Realtime surveying within mobile sponsored content
US7853485B2 (en) 2005-11-22 2010-12-14 Nec Laboratories America, Inc. Methods and systems for utilizing content, dynamic patterns, and/or relational information for data analysis
US20070150342A1 (en) 2005-12-22 2007-06-28 Law Justin M Dynamic selection of blended content from multiple media sources
US7653761B2 (en) 2006-03-15 2010-01-26 Microsoft Corporation Automatic delivery of personalized content to a portable media player with feedback
US7634474B2 (en) 2006-03-30 2009-12-15 Microsoft Corporation Using connectivity distance for relevance feedback in search
US20070282621A1 (en) 2006-06-01 2007-12-06 Flipt, Inc Mobile dating system incorporating user location information
US20080104059A1 (en) 2006-11-01 2008-05-01 Dininginfo Llc Restaurant review search system and method for finding links to relevant reviews of selected restaurants through the internet by use of an automatically configured, sophisticated search algorithm
US7849104B2 (en) 2007-03-01 2010-12-07 Microsoft Corporation Searching heterogeneous interrelated entities
US8176440B2 (en) * 2007-03-30 2012-05-08 Silicon Laboratories, Inc. System and method of presenting search results
US9053195B2 (en) 2007-07-19 2015-06-09 Grant Chieh-Hsiang Yang Method and system for user and reference ranking in a database
US20090076928A1 (en) * 2007-08-28 2009-03-19 Needish, Inc. System and method for automating RFP process and matching RFP requests to relevant vendors
US9626685B2 (en) 2008-01-04 2017-04-18 Excalibur Ip, Llc Systems and methods of mapping attention
US8145619B2 (en) * 2008-02-11 2012-03-27 International Business Machines Corporation Method and system for identifying companies with specific business objectives
US8306921B2 (en) 2008-02-13 2012-11-06 Toyota Motor Engineering & Manufacturing North America, Inc. Mobile recommendation and reservation system
US8538811B2 (en) 2008-03-03 2013-09-17 Yahoo! Inc. Method and apparatus for social network marketing with advocate referral
US20090293019A1 (en) * 2008-05-22 2009-11-26 Keith Raffel User interface having slider controls for weighted parameters in searching or decision making processes
US9646025B2 (en) * 2008-05-27 2017-05-09 Qualcomm Incorporated Method and apparatus for aggregating and presenting data associated with geographic locations
US8095432B1 (en) 2009-01-30 2012-01-10 Intuit Inc. Recommendation engine for social networks
US8244564B2 (en) 2009-03-31 2012-08-14 Richrelevance, Inc. Multi-strategy generation of product recommendations
WO2010141429A1 (en) 2009-06-01 2010-12-09 Sean Christopher Timm Providing suggested web search queries based on click data of stored search queries
US8359285B1 (en) 2009-09-18 2013-01-22 Amazon Technologies, Inc. Generating item recommendations
US20110125759A1 (en) * 2009-11-17 2011-05-26 Yoono, Inc Method and system to contextualize information being displayed to a user
US8285840B2 (en) 2010-03-15 2012-10-09 Cellco Partnership Using social networking to help users purchase mobile applications and devices
WO2011117737A1 (en) * 2010-03-22 2011-09-29 Heystaks Technologies Limited Systems and methods for user interactive social metasearching
US8719198B2 (en) 2010-05-04 2014-05-06 Microsoft Corporation Collaborative location and activity recommendations
US8347211B1 (en) 2010-06-22 2013-01-01 Amazon Technologies, Inc. Immersive multimedia views for items
US20120209839A1 (en) * 2011-02-15 2012-08-16 Microsoft Corporation Providing applications with personalized and contextually relevant content
US8751472B2 (en) * 2011-05-19 2014-06-10 Microsoft Corporation User behavior model for contextual personalized recommendation
US9449106B2 (en) 2013-03-08 2016-09-20 Opentable, Inc. Context-based queryless presentation of recommendations

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6370513B1 (en) * 1997-08-08 2002-04-09 Parasoft Corporation Method and apparatus for automated selection, organization, and recommendation of items
US20010044759A1 (en) * 2000-05-17 2001-11-22 Matsushita Electric Industrial Co., Ltd. Information recommendation apparatus and information recommendation system
US20040103092A1 (en) * 2001-02-12 2004-05-27 Alexander Tuzhilin System, process and software arrangement for providing multidimensional recommendations/suggestions
WO2003054760A2 (en) * 2001-12-21 2003-07-03 Accenture Global Services Gmbh Evaluation of a plurality of alternatives based on a plurality of individual preferences
US20060250935A1 (en) * 2003-02-28 2006-11-09 Ntt Docomo, Inc Radio communication system and radio communication method
US20090234664A1 (en) * 2008-03-11 2009-09-17 Schaffnit Wayne A System and method for recommending entertainment venues for specific occasions
US20110270774A1 (en) * 2010-04-30 2011-11-03 Microsoft Corporation Group Recommendations in Social Networks
US20120233191A1 (en) * 2010-11-22 2012-09-13 Salesforce.Com, Inc. Method and system for making content-based recommendations

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8756187B2 (en) 2011-09-28 2014-06-17 Nara Logics, Inc. Systems and methods for providing recommendations based on collaborative and/or content-based nodal interrelationships
US8909583B2 (en) 2011-09-28 2014-12-09 Nara Logics, Inc. Systems and methods for providing recommendations based on collaborative and/or content-based nodal interrelationships
US9449336B2 (en) 2011-09-28 2016-09-20 Nara Logics, Inc. Apparatus and method for providing harmonized recommendations based on an integrated user profile
US9009088B2 (en) 2011-09-28 2015-04-14 Nara Logics, Inc. Apparatus and method for providing harmonized recommendations based on an integrated user profile
US8732101B1 (en) 2013-03-15 2014-05-20 Nara Logics, Inc. Apparatus and method for providing harmonized recommendations based on an integrated user profile
US20150025995A1 (en) * 2013-07-19 2015-01-22 Ebay Inc. Generating recommendations based on transaction data
US20150095278A1 (en) * 2013-09-30 2015-04-02 Manyworlds, Inc. Adaptive Probabilistic Semantic System and Method
US20150106367A1 (en) * 2013-10-14 2015-04-16 International Business Machines Corporation Utilizing social information for recommending an application
US9531722B1 (en) 2013-10-31 2016-12-27 Google Inc. Methods for generating an activity stream
US9542457B1 (en) 2013-11-07 2017-01-10 Google Inc. Methods for displaying object history information
US9614880B1 (en) 2013-11-12 2017-04-04 Google Inc. Methods for real-time notifications in an activity stream
US9509772B1 (en) 2014-02-13 2016-11-29 Google Inc. Visualization and control of ongoing ingress actions
US20150324434A1 (en) * 2014-05-09 2015-11-12 Paul Greenwood User-Trained Searching Application System and Method
US9626361B2 (en) * 2014-05-09 2017-04-18 Webusal Llc User-trained searching application system and method
US20170220692A1 (en) * 2014-05-09 2017-08-03 Paul Greenwood User-Trained Searching Application System and Method
US9536199B1 (en) 2014-06-09 2017-01-03 Google Inc. Recommendations based on device usage
US9507791B2 (en) 2014-06-12 2016-11-29 Google Inc. Storage system user interface with floating file collection
US10078781B2 (en) 2014-06-13 2018-09-18 Google Llc Automatically organizing images
US9870420B2 (en) 2015-01-19 2018-01-16 Google Llc Classification and storage of documents
CN104850641A (en) * 2015-05-26 2015-08-19 无线生活(杭州)信息科技有限公司 Information recommendation method and device

Also Published As

Publication number Publication date Type
US9189551B2 (en) 2015-11-17 grant
US20130024449A1 (en) 2013-01-24 application
US20130024456A1 (en) 2013-01-24 application

Similar Documents

Publication Publication Date Title
Ma et al. Recommender systems with social regularization
Konstan et al. Recommender systems: from algorithms to user experience
Shi et al. Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges
Tang et al. Social recommendation: a review
Isinkaye et al. Recommendation systems: Principles, methods and evaluation
Bobadilla et al. Recommender systems survey
US8032480B2 (en) Interactive computing advice facility with learning based on user feedback
US20110196863A1 (en) Tagged favorites from social network site for use in search request on a separate site
US8055673B2 (en) Friendly search and socially augmented search query assistance layer
US20120296991A1 (en) Adaptive system architecture for identifying popular topics from messages
US20110225043A1 (en) Emotional targeting
Ricci et al. Recommender systems: introduction and challenges
US20110066507A1 (en) Context Enhanced Marketing of Content and Targeted Advertising to Mobile Device Users
US20140074629A1 (en) Method and system for customized, contextual, dynamic & unified communication, zero click advertisement, dynamic e-commerce and prospective customers search engine
Qian et al. Personalized recommendation combining user interest and social circle
US20100250578A1 (en) System and method for conducting a profile based search
US8732101B1 (en) Apparatus and method for providing harmonized recommendations based on an integrated user profile
US20060004713A1 (en) Methods and systems for endorsing local search results
US20120030228A1 (en) Method and system for need fulfillment
Colombo-Mendoza et al. RecomMetz: A context-aware knowledge-based mobile recommender system for movie showtimes
US20130036117A1 (en) System and method for metadata capture, extraction and analysis
US20140067535A1 (en) Concept-level User Intent Profile Extraction and Applications
US20110252121A1 (en) Recommendation ranking system with distrust
US20120290434A1 (en) Method For Providing a Recommendation Such as a Personalized Recommendation, Recommender System, and Computer Program Product Comprising a Recommender Computer Program
US20080288573A1 (en) Method and apparatus for filtering virtual content

Legal Events

Date Code Title Description
AS Assignment

Owner name: NESS COMPUTING, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BERNER, CHRISTOPHER ERIC SHOGO;SCHIFF, JEREMY RYAN;REESE, COREY LAYNE;AND OTHERS;SIGNING DATES FROM 20120723 TO 20120903;REEL/FRAME:029048/0584

AS Assignment

Owner name: NESS COMPUTING, LLC, CALIFORNIA

Free format text: CONVERSION OF CORPORATION TO LLC;ASSIGNOR:NESS COMPUTING, INC.;REEL/FRAME:032387/0975

Effective date: 20140205

AS Assignment

Owner name: OPENTABLE. INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NESS COMPUTING LLC;REEL/FRAME:032969/0450

Effective date: 20140422