EP2272015A1 - Open framework for integrating, associating and interacting with content objects - Google Patents

Abstract

Information regarding a mobile user's context including but not limited to current mobile activity, social relations and associations history, and past mobile, search and browsing history is identified and converted to metadata. Metadata is also applied to content sources delivering content to a search engine or personalized content engine. The metadata is used in part to determine the relative display of content objects delivered to the mobile user as search results or a personalized aggregated information resource, e.g., home page. The user may select information, from one or more entities or search results or as presented to the user in other contexts, to be automatically delivered to the user's home page as a content feed including multiple content objects or content feeds associated with an entity. Information regarding mobile user activity permits publishers and advertisers to identify target candidates to receive advertisements or marketing materials.

Description

OPEN FRAMEWORK FOR INTEGRATING, ASSOCIATING AND INTERACTING

WITH CONTENT OBJECTS

FIELD OF THE INVENTION

[0001] The present invention relates generally to search queries over a network and user configuration of feeds and clusters in conjunction with a mobile device.

BACKGROUND

[0002] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, the approaches described in this section may not be prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

[0003] Tremendous changes have been occurring in the Internet that influence our everyday lives. For example, in today's society, mobile computing devices are becoming ubiquitous. Many mobile computing devices, such as personal digital assistants, cellular phones, and the like, may be employed to communicate voice messages, emails, text messages, and so forth, as well as to search for information over the Internet. It is not uncommon to see a person on a bus, train, or boat to be using their mobile devices to search for merchants, restaurants, music, businesses, or the like. It is conceivable these devices will eventually be allowed on planes, at least for wireless Internet access. [0004] Translating the traditional web page browsing interaction and content consumption experience to the mobile environment has generally resulted in less-than- satisfactory experiences for the user due to a variety of reasons. In a mobile device, screen real estate is at a minimum. Navigation of any presented information may be dependent upon, or subject to the quirks of, the particular mobile device. Typically, the user is not in an environment conducive to casual exploration of the World Wide Web. In fact, the user may be in a changing environment that itself demands the lion's share of the user's attention. Thus the tailoring of search engine results and content consumption to the mobile experience is of prime importance to the user.

[0005] Along with the desire to perform relevant active information searches, some mobile device users may desire delivery of content in a more automatic manner, possibly with the content tailored to a unique set of interests, such as hobbies or social network groups. Even if a user has the time, energy, and technical skills necessary to manually configure a mobile device, the user will likely be forced to remain content to receive a narrow set of pre-defined information sources, such as news headlines, weather and sports. Yet other users may desire delivery of particular content depending on the current user environment.

[0006] Similarly, advertisers desire to provide information that is tailored to each user and enhances the user experience. An advertiser can approximate this goal through delivery of advertising material as part of delivery of search engine results, because of the likely correlation between the user search terms and the user's actual interests. However, in the mobile environment when the user is not searching, advertisers typically have limited ability to reach a target audience that is likely on the move and in need of information relevant to the environment they are in or going to.

[0007] BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 is a system diagram of one embodiment of an environment in which the invention may be practiced;

[0009] FIG. 2 illustrates a sample search and clustered search results on a mobile device;

[0010] FIG. 3 illustrates the clustering achieved through the use of metadata wrappers applied in a social networking context;

[0011] FIG. 4 illustrates the generation of results based on a search yielding no named cluster results;

[0012] FIG. 5 illustrates a mobile user search system performing a federated query to multiple data sources in which third parties provide plug in modules to interpret their data into a format usable by the search;

[0013] FIG. 6 illustrates a sample framework for the plug-ins shown in FIG. 5;

[0014] FIG. 7 illustrates a schematic of an Add Anything Engine;

[0015] FIG. 8 illustrates a schematic of a Mobile Homepage Targeting Engine;

[0016] FIG. 9 illustrates a portion of a sample use of the Add Anything Engine;

[0017] FIG. 10 illustrates a portion of a sample use of the Add Anything Engine;

[0018] FIG. 11 illustrates a portion of a sample use of the Add Anything Engine;

[0019] FIG. 12 illustrates a complete screen shot (with portions omitted) of a mobile device resulting from the performance operations illustrated in FIGS. 9-11;

[0020] FIG. 13 illustrates information associated with the entity "Boston Red Sox";

[0021] FIG. 14 illustrates user configuration screens used in conjunction with various embodiments of the invention; [0022] FIG. 15 illustrates a subset of the oneSearch entity types organized by hierarchical type-subtype relations; and

[0023] FIG. 16 is a block diagram of a computer system on which embodiments of the invention may be implemented.

DETAILED DESCRIPTION

[0024] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

OVERVIEW

[0025] Techniques are disclosed herein to enrich a mobile user's personalized content consumption, web browsing and search experience by incorporating the user's current mobile activity, social relations and association histories as well as historical search and browsing data to personalize many aspects of the user's mix of mobile content and configuration of information displayed on the user's mobile device or aggregated on a personalized home page. These techniques may be automatically applied, or performed by the user. A content object may be any form or format of content that is consumable by users such as, but not limited to, any one or more of the following: text, numerical data, images, animation, audio, video or any combination thereof including content encountered as search results, through browsing or from communications with others through voice, email, text, image, animation, gesture, audio or video. Content may come from any data processing source or device.

[0026] For example, in an embodiment, contextual information related to the user's current activity is derived from the mobile device or other sources, and passed to a search engine as metadata in order to reorder generic search results into a personalized result set including a system and method for personalizing mobile user content requests, i.e., searches or dynamic page assembly requests by augmenting a mobile user's input with a first set of metadata derived from the context of the input, including location information to match the request and rank or re-rank the results

[0027] In an embodiment, one or more content objects are displayed on a mobile device. For each content object selected by a user, one or more entities or entity types are determined to correlate to the selected content object. Based at least in part on the determined entity or entity type, an information feed is created. The user may incorporate the information feed into the user's home page or device and automatically receive updates based on the web page or information corresponding to the content object, entity or associated information sources.

[0028] In an embodiment, a set of search results is displayed on a mobile device. For each search result selected by a user, an entity or entity type is determined for the selected search result. Based at least in part on the determined entity or entity type, an information feed is created. The user may incorporate the information feed into the user's home page or device and automatically receive updates based on the web page or information corresponding to the search result or information resource when users annotate results to cause the future inclusion of that source, result, or search type in their mobile "home page."

[0029] In an embodiment, a set of users are ranked against a selected advertiser or product. From the set of users, candidate targets are predicted for a set of one or more advertisers and conditions, based at least in part on the context of the user, the advertisement copy and a pre-selected time for refining, qualifying and scoring users for an advertiser based upon the actual hierarchical content configurations of the user's mobile home page and the interaction and usage data of the user with the home page over time.

[0030] In an embodiment, (1) a user's content consumption and communications on the user's mobile device(s), along with (2) the preferences and activities that the user performs in configuring, consuming, interacting with, and sharing content, as well as (3) the contexts in which and the people with whom the user performs these actions, provide information to advertisers about the user's interests and intentions.

EXAMPLE OPERATING ENVIRONMENTS

[0031] Except where indicated, illustrated mobile and network operating environments are unnecessary for a full appreciation of various embodiments of the invention, and will not be repeated here. Such illustrated mobile and network operating environments can be found in, for example, Application Ser. No. 11/651,102 entitled "Clustered Search Processing." FIG. 1 illustrates a high-level schematic of a mobile device having a web browser, with the mobile device in communication with a server. The server is connected to a network and includes a personalized content engine configured to aggregate and display disparate types and sources of content, a search engine that is configured to perform a search of the network and a user data tracking engine for instrumenting and recording user's behavior. [0032] In FIG. 1 , networked system 100 includes a mobile device 110 in communication with a server 130 using network 104. In an embodiment, the network 104 is the Internet, but it could also be a non-public network such as a telecommunications carrier. Server 130 includes a set of one or more search and data applications 134. Search and data applications 134 include personalized content engine 140, configured to generate and maintain a "home page" of personalized content types and sources in one centralized display, and search engine 150 configured to perform searches of data accessible via the network and the Internet. In an embodiment, personalized content engine 140 corresponds to is the Yahoo! onePlace mobile home page and search engine 150 is configured to search a private network of user data such as Yahoo! or Google. [0033] Search and data applications 134 also include one or more of the following modules: User Data Store (UDS) 152, Mobile Search Refine Manager 154, Search Module 156, and Search Clustering Module 158. These particular naming conventions are not necessary from any technical standpoint to understand the methods and systems disclosed herein, but are provided for the convenience of the reader. These modules will not be described in a serial fashion, but will be introduced as certain features and embodiments are illustrated. Although each module is described separately from search engine 150, each module could be incorporated into search engine 150. [0034] Mobile device 110 may include a mobile telephone, gaming or reading device or another wireless device (such as Wi-Fi or Bluetooth, for example) that provides access to one or more interconnected networks 104. Mobile device 110 includes an interactive application 120 that permits browsing of content and data using display 126. [0035] Before describing an illustrated operating environment in which various embodiments may be practiced, concepts including "social relations and associations" and "metadata" are introduced and described below. Later, during discussion of various embodiments, additional refinements to these concepts may be described. [0036] Social Relations and Associations

[0037] Social relations and associations connect people to activities and one another. A social relations and associations graph may be constructed from nearly any activity, from dating to professional organizations, with some networks not focused on a particular activity, but instead a set of personal or familial relationships.

[0038] Social relations and associations have expanded from interpersonal activities such as bowling leagues and now encompass any manner of offline or online activity and connections, typically made through the Internet and intertwined into the World Wide Web. [0039] An individual's social relations and associations graph is not usually static, as movements and interactions among an individual and his or her social relations and associations may be a function of one or more of the following non-limiting factors: (1) where the individual is; (2) the time; (3) who the individual is interacting with, either physically or via online social relations and associations; and (4) what the individual is doing.

[0040] Many variations, combinations, and/or refinements of these factors can be relevant; for example, the individual may be planning and coordinating a family reunion involving a variety of direct and indirect personal relationships that change through time. Information related to these factors may be captured by the mobile device or by a network server. Some methods and systems for the collection and use of social relations and associations information is disclosed in both Application Ser. No. 12/069,731 entitled "Identifying and Employing Latent Social Network Relationships," and Application Ser. No. 12/182,756 entitled "Social Aspects of Content Aggregation, Syndication, Sharing, and Updating." [0041] Metadata

[0042] As described above with respect to an individual's social relations and associations, a description representing an individual's current activities may include some or all of the factors relating to the "Where," the "When," the "Who," and the "What," which provide a continuous path/state data stream on the user and provide exact context for any user-originating requests such as search or loading a mobile home page. When used to describe particular data about an individual, these factors are themselves metadata called for example a user's W4 metadata including one or more of spatial metadata, temporal metadata, social metadata, object metadata and/or topical metadata. [0043] As described herein, each of the (1) operating environment, (2) mobile client environment, (3) server environment and (4) operation of mobile search queries in conjunction with location modifiers, can be further extended with reference to metadata wrappers, including user W4 metadata, placed around data that is exchanged among mobile clients, servers, and (optionally) administrators or third party service providers. [0044] User W4 metadata includes contextual metadata which relate to the "Where," the "When," the "Who," and/or the "What" of any given event, e.g., a text message or voice communication. That is, W4 metadata may include information which is spatial or geographic in nature (i.e., the "Where"), temporal (i.e., the "When"), social (i.e., the "Who"), and/or related to physical objects or topical interests (i.e., the "What"). In addition, the relevance of at least some of these aspects may be determined by analyzing the similarity of these aspects among user groups, as well as patterns of these similarities within and among the respective spatial, temporal, social, and topical aspects. [0045] As described further herein, the context provided by user W4 metadata aids in the disambiguation of mobile search queries, which in turn permits improved clustering of search terms and search results.

[0046] For example, a user enters the query "apple." In addition to text in or associated with web pages and documents, the term "apple" can be mapped to a number of abstract ideas, real world objects, and digital objects and media. Collectively, these things are referred to herein as "entities." In this example, the disambiguation is performed on the string "apple" to determine at least some of the possible entities "apple" might represent, e.g., the company, the fruit, the record label, etc., each of which may then be mapped to one or more result types. In addition, disambiguation can preference recent real- world or online behavior data to preference certain results. For example, if the user was in the Apple Mac Store yesterday and made a purchase, then the disambiguation might preference local receipts or product information pages the user recently accessed in conjunction with the user's prior purchase or research.

[0047] For example, if the entity is Apple® the company, the different result types might include geographic results (e.g., company or store locations), stock price, news stories (e.g., text and video), as well as conventional web and mobile web links. As described further herein, clusters of result types based on the entity or entities identified in the disambiguation phase and their respective mappings to result types may be rendered or manipulated. Additional information regarding disambiguation of search queries can be found in Application Ser. No. 11/651,102, entitled "Clustered Search Processing."

[0048] For example, FIG. 15 represents a subset of the oneSearch entity types organized by hierarchical type-subtype relations. Additional relations among entities and entity types may include numeric and symbolic relationships, as well as composite entity types such as "events" which link other entity types such as places, times, people, etc. [0049] In FIG. 15, for example, row 1510 references the entity "Apple Computer" (corresponding to the discussion above) and row 1520 references the entity "Pizza Restaurant."

[0050] Entities may also exist in multiple categories. For example, the television series Lost could exist in the TV shows category of Entertainment Information Objects yet also be referenced in the Actor category of Entertainment People. Likewise, Pizza could be Business Listing, Venue, Place as well as a recipe entry in Wikipedia, Reference, or Information Object.

[0051] Incorporation of entities and entity types represents an important shift away from the mere bookmarking of web pages, as it permits saving pointers to entities that may have real time information feeds or for which real time information feeds may be created. For example, consider a static bookmark to the website "www.examplemovie.com", as compared to bookmarking the entity "examplemovie," from which it is possible to receive automatic updates regarding reviews, local playing times, DVD release notices, etc. relating to "examplemovie." [0052] Sample Mobile User and Network Configurations

[0053] Various embodiments of the networked system above are now described, in which the capture and transmission of metadata is then used to enhance the mobile user browsing interaction and consumption experience.

[0054] With reference to FIG. 1, storage of metadata and other user information such as social relations and associations information may be accomplished by User Data Store (UDS) 152 that is part of search and data applications 134 on server 130. UDS 152 may include a database, text, spreadsheet, folder, file, or the like, and may be configured to maintain and store social relations and associations information as collected from network 104 as well as associated networks or user data sources. For example, a four dimensional map of a mobile device may be used as a proxy for the user's historical and current location, state and association with other nearby or related users.

[0055] In an embodiment, social relations and associations information stored in UDS 152 for a mobile user includes one or more of the following: (1) the user's identification information for social networking websites to which the user belongs, such as MySpace, Facebook, or Linkedln; (2) contact information maintained by the user, such as for voice contact, electronic mail, instant messaging, and text messaging; (3) membership groups to which the user belongs; (4) subscriber information of individuals that subscribe to user- created content, such as subscribers to the user's Flickr account or the user's blog and (5) any other accessible source of data on the user's location, association, actions, interests or transactions.

[0056] In an embodiment, social relations and associations information stored in UDS 152 for a mobile user includes electronic communications sent or received by the mobile user, such as electronic mail, instant messaging, and text messaging, while other embodiments combine information from multiple devices, mobile and stationary to include all known data, communications and transactions for the user. [0057] In an embodiment, location information based both on a user's actual and past path data as well as on a user's physical residence(s), account registration information, locations of interests the user has implicitly or explicitly shared with the system, or the like, may also be stored within UDS 152. Such information may indicate that the user resides in a particular town, neighborhood, city, or the like, within a particular county, state, country, or the like, or is interested in a given location (e.g. venue, district, city, region, etc.), or the like. Such location information may also be configured and arranged, in one embodiment, in a hierarchical relationship.

[0058] In an embodiment, mobile status information that is related to the user may also be stored within UDS 152. Such mobile status information may include one or more of following: (1) the name, operating system, and CPU of mobile device 110; (2) the current status, location and/or usage history of mobile device 110, such as when and how long it has been powered on and accessed as well as to which cell towers or network nodes it has been communicatively connected with; (3) status information of applications 242 loaded on or currently operating on mobile device 110, such as information related interactive application 120; (4) contents of data storage of mobile device 110; and (5) information such as coordinate history from GPS of mobile device 110, from which a user's mobile status (such as that the user is located in a slow-moving transportation device on a particular freeway) may be derived. Usage history includes both active interaction data created by the user's actions, as well as passive data such as location of the user, proximity to known landmarks or Points of Interest (POI), and association other co-present users, devices, networks, vehicles or appliances.

[0059] In an embodiment, social networking information may be obtained once a user has provided account access to the user's social network or electronic mail accounts. [0060] In an embodiment, social relations and associations information may be obtained indirectly through monitoring various movements, actions and interactions of a user. Interactions may be monitored though use of a browser toolbar activated by the user, in which some or all data sent to mobile device 110 is forwarded to UDS 152 as well as any other known method of instrumenting a mobile device to track user interaction including orientation, biometric and motion sensing. Thus, for example, the implicit social relations and associations data and/or contact information may be extracted based on data from the users' respective devices, environmental sensors or data feeds and/or the content within a message between the user and another user whether or not they are both members of an explicit social network. [0061] Information such as social networking information, communications information, location information, and mobile path or status information, is stored within UDS 152, and may be passed with other data as a "metadata wrapper". As described below, a metadata wrapper may provide contextual information or content information to the data in which it accompanies. A non-limiting example of a vocabulary in which metadata may be propagated, including use of properties, objects, and datatypes, is provided below in the Section entitled "Sample Metadata Vocabulary." [0062] Information stored within UDS 152 may be passed to Mobile Search Refine Manager (MSRM) 154. MSRM 154 is configured to receive the search query from a user, parse the search query into one or more primary search terms, and provide contextual modifiers based upon information regarding the user stored within UDS 152. Additional features of MSRM 154, including processing of contextual and concept-based searches, are described in Application Ser. No. 11/651,102, entitled "Clustered Search Processing."

[0063] In an embodiment, contextual modifiers are provided in a metadata wrapper passed to MSRM 154 from mobile device 110; for example, the metadata wrapper may comprise data accompanying or part of a search query sent from mobile device 110. In an embodiment, contextual modifiers are provided to MSRM 154 by UDS 152. [0064] In an embodiment, MSRM 154 identifies any concept-specific data sources to be searched. One embodiment of a method in which third-parties may provide customizable data sources will be described later with respect to FIGS. 5-6. [0065] Using the information stored in UDS 152, MSRM 154 then fashions a modified search query to be processed by Search Module 156, along with an indication of any concept-specific data sources to be searched.

[0066] Search results provided by Search Module 156 may be clustered into groups, and optionally ranked, as disclosed in Application Ser. No. 11/651,102, entitled "Clustered Search Processing." In an embodiment, search results from Search Module 156 are then passed to Search Clustering Module 158. In an embodiment, metadata passed to Search Module 156 is also passed to Search Clustering Module 158. [0067] Additionally, search results as determined with respect to mobile user metadata may also be modified based on metadata derived from one or both of concept- specific and content-specific data sources. The corresponding search results may be clustered by source, terms, concept or content.

[0068] For example, metadata identified by one or both UDS 152 and MSRM 154 may be matched to search results based on a matching concept-specific data source. [0069] In an embodiment, search results are combined with other known data sources to create a "virtual cluster" that may be presented to the user in the same or a similar manner in which actual clustered results are presented to the user.

[0070] In an embodiment, concepts identified by Search Clustering Module 158 used to cluster results are attached to the results as metadata. In an embodiment, the identified concepts are used to determine a result type or relative rank based on the entity or entities identified in the disambiguation phase.

[0071] In an embodiment, metadata representing the user's mobile status information may be used to construct the layout of search results appearing on display 126 of mobile device 100.

[0072] In this manner, metadata from many different sources regarding the user and context is used to personalize the current mobile user's search or dynamic page assembly requests. Metadata describing the user's (1) current activities, (2) current environment,

(3) social relations and associations, (4) third party evaluations, reputations, rankings or other quality scoring metric, e.g. credit score, etc., and (4) search query is combined with metadata generated by concept-specific, context-specific, or content-specific data sources, may be used to deliver relevant information to a mobile user.

[0073] Illustrative User Example

[0074] Examples of clustering (a) through use of metadata related to mobile device usage, (b) in conjunction with disambiguation of strings into entities, are illustrated in

FIGS. 2-4 displaying sample searches and search results on a mobile device.

[0075] With respect to FIG. 2, in panel 210, the mobile user performs a simple search using the term "sushi." Previously obtained metadata such as the user's location has been captured and is displayed below the search box; in this example, the user is located "near

3000 Paradise Rd, Las Vegas."

[0076] This location metadata permits the identification and display of nearby businesses, such as restaurants, identified as relevant to the search term. The restaurants form a cluster of particular instances of the abstract entity type "restaurant." These results can be further refined by distance, as shown in panel 210.

[0077] Advantageously, the metadata wrapper can be employed by other content providers, such as Yelp, Zagat, and OpenTable, and shown under the heading "More business results" in panel 210. These business results could be generated based on a combination of the mobile user's location, the search term, and an abstract entity type such as "restaurant," and pushed to the user. [0078] In panel 220, the user has selected the result for "Sushi Roku in the Forum

Shops," and has landed on the informational display page regarding this restaurant.

Although the user is viewing details regarding a particular instance of the abstract entity

"restaurant," the mobile device's display is configured to display links to other related information, such as a map, user reviews, and more tools, via the link to "Business details."

[0079] Panels 231 , 232, and 233 display the links available from the landing page for

"Sushi Roku." For the information shown in each panel, metadata wrappers have been applied to describe the data types of the content sources (such as "review," "review author," "photo," "rating," etc.)

[0080] FIG. 3 illustrates the clustering achieved through the use of metadata wrappers applied in a social networking context. Panel 310 illustrates a search performed by the user for a person named "Will Decker." This search may again being performed with reference to the user's current location, as shown in the location information ("near 3000

Paradise Rd, Las Vegas") below the search box. The search generates a cluster of profile information using the abstract class "Contact" with regard to three applications,

Facebook, Linkedln, and MySpace. In this example, data (including metadata wrappers) from these three applications is pulled from each through an open API framework and assembled by the server.

[0081] Additionally shown in panel 310, each of the instances of the "Contact" class type possess properties that can be used to further refine the results and aid the user in locating the individual named "Will Decker" that he or she actually knows.

[0082] In panel 320, the user has selected the profiles found in Facebook and views a summary of the various people named "Will Decker" (or a slight variant thereof) along with additional information, such as a thumbnail photo and an encapsulated summary about the individual, such as "Berkeley Alum • UChicago Alum," corresponding to the class "Overview."

[0083] In panel 330, the user has navigated to a representation of Facebook photo gallery for a particular instance of "Will Decker." Metadata wrappers including photograph information may be applied by the API framework, and combined with information regarding the mobile device physical features, to present the various

Facebook photographs in an attractive manner.

[0084] FIG. 4 illustrates the generation of results based on a search yielding no named cluster results, in which the user is presented with one or more unnamed, virtual clusters, which themselves may comprise one or more third-party open abstracts. In panel 410, a mobile user enters the search term "Audi tt." While none of the search results yield a named cluster, a virtual cluster entitled "2008 Audi TT" is created using content supplied from MotorTrend and Wikipedia conforming to the open abstracts model. [0085] In panel 420, the user has selected the MotorTrend Research icon and is presented with a landing page assembled from data and metadata that may be (1) supplied from results obtained from general data sources, or (2) supplied from a MotorTrend API plug-in. Other possibilities for generation of results include one or more of the following: (1) generation of a virtual cluster; (2) generation of a series of virtual clusters; and (3) a list of OpenAbstracts that is not clustered but meets at least one relevancy criterion. [0086] Integrating Content Sources into the Mobile Search Framework [0087] In the illustration provided above in FIGS. 2-4, cluster data from Facebook, Linkedln, and My Space may have been: (1) provided to the server via a browser toolbar, or (2) pulled from each website through an open API framework and assembled by the server. A framework for this integration of content sources and simple schematics for the cluster generation via third-party content sources are illustrated in FIGS. 5-6 and described below.

[0088] FIG. 5 illustrates a mobile search system 500 performing a federated query to multiple data sources in which third parties provide plug in modules to interpret their data into a format usable by the search or personalized content processes. The search retrieves data (and associated metadata) in data stores 511, 512, 513 that comprise a "Content Index." Data is passed to the Content Index from content plug-ins 521-524 described below with reference to FIG. 6. Results in mobile search system 500 are also pulled directly from content plug in 526.

[0089] Each of plug-ins 521-525 either pull data from, or are pushed data by, content sources 531-536. Content sources may comprise application program interfaces (APIs) and/or content feeds. Additionally, results in mobile search system 500 are also pulled directly by content source 537, which comprises an application program interface (API). As illustrated, data may be pulled from an API or content feed from a third party provider to the plug-in; or data may be pushed to the plug via a third party provider's content feed. [0090] FIG. 6 illustrates a sample framework for a plug-in shown in FIG. 5. Plug- in 600 comprises parser 610, an association module 620, and renderer 630. Parser 610 may comprise a metadata parser or API Requestor, one or both communicating with one or more of the "Content Sources" shown on FIG. 5. Parsed data, including optional metadata wrappers, are passed from parser 610 to association module 620. Explicit and inferred cluster result mapping is performed by association module 620, with results processed by renderer 630. Results in the renderer may be processed through an XML schema or a template / user interface mapping, providing metadata wrappers (such as clusterlDs, entitylDs, W4 metadata, and content type metadata) that permit the system to determine the appropriate format, rendering, and user actions for the content shown on the mobile device. Results are then sent to one or more data stores that comprise the "Content Index" of FIG. 5.

[0091] Incorporation of Search Sources or Results into a Home Page [0092] In the description above, various forms of metadata are used to derive search results from a corresponding search query. The search results may represent various types of content, such as text, numerical data, image, animation, audio or video and formats of content such as an RSS feed, a link to a newspaper article, a Wikipedia entry, or portions of a web page. The user may desire to incorporate one or more search results into the user's home page or mobile home page that automatically appears when a browser or application is first invoked, or is invoked when the user activates a "home" command of some type.

[0093] Additionally, a search result, or portion of a search result, may itself be considered an entity or entity type as previously described; for example, a web link or RSS feed may be added to, and treated as, an entity or entity type on the user's home page. Items which may be added include: (1) web links; (2) RSS feeds; (3) web links for which RSS feeds are created based on periodically revisiting the link to check for updates; (4) "entity" or "entity type"; and (5) entities or entity types added through the parsing of a a web link or RSS feed.

[0094] As a non-limiting example, a user may select a search result or any online web page or resource for future inclusion, or the user may also add the entity or entities represented within that content to the user's personalized content engine. At the time of the user's selection, the personalized content engine may automatically or interactively determine whether the user wishes to add that content, resource, or feed, among one or multiple associated entities.

[0095] Such desirable features may be accomplished via display of search results or content objects with an associated icon, checkbox, user gesture, or associated spoken command that when selected or spoken, determines a content type for the selected results and creates a corresponding information feed to be incorporated in the user's home page or consumption device, e.g., electronic book reading device, or gaming device. The ability for the user to add any search results as an information feed is now described with reference to FIG. 7. [0096] FIG. 7 illustrates four components of Add Anything Engine 700 that may be used to permit the user to add search results to the user's home page or other information aggregation location or device. It should be understood that the elements in FIG. 7 may be easily incorporated in server 130 shown in Fig. 1. For example, Add Anything Engine

700 may be incorporated into search and data applications 134 on server 130. For simplicity, the description below will be made with reference to FIG. 7.

[0097] Add Anything Engine 700 includes Query Manager 704, Display

Instrumentation Manager 708, Content Analyzer 712, and Registration Manager 716.

[0098] Query Manager 704 tracks search query information and related metadata of one or more usage data sets or search histories of the user as well as across any number of devices or networks associated with the user.

[0099] In an embodiment, Query Manager 704 receives information from UDS 152 and MSRM 154.

[0100] Information from Query Manager 704 is supplied to Content Analyzer 712 for use in determination of the particular content type or format to be associated with the corresponding search result. Content Analyzer 712 parses received web links and RSS feeds for determination of the corresponding entity and entity type.

[0101] Some search results, such as RSS feeds, have established protocols for updates, whereas other search results, such as to a static web page, may not have an established update protocols. Content Analyzer 712 forwards results corresponding to such a static content type to Registration Manager 716.

[0102] Registration Manager 716 may create a recurring information feed out of a non-feed enabled information object so that the user receives updated information from the corresponding link at various intervals. For example, the user may desire to view web page announcements made by government agencies, such as the United States Patent and

Trademark Office, that do not provide web feeds.

[0103] In an embodiment, Registration Manager 716 includes instructions to periodically fetch the data source corresponding to the non-feed enabled information object selected by the user, parse the resulting data, and apply metadata wrappers.

[0104] Registration Manager 716 forwards any recurring information feeds to Mobile

Homepage Manager 720 for display on mobile device 110.

[0105] Display Instrumentation Manager 708 controls both the actual display of the

"add this" icons and receipt of corresponding user input. Display Instrumentation

Manager 708 forwards selected known information feeds to Mobile Homepage Manager

720 and forwards other user selections to Registration Manager 716 for feed creation. Display Instrumentation Manager 708 may also be configured to sense spoken commands and/or user gestures for selecting selected known information feeds to Mobile Homepage Manager 720 and Registration Manager 716 for feed creation.

[0106] FIGS. 9-13 illustrate an illustrative mobile home page environment and one manner in which a user may add an entity to a mobile device home page. Panel 910 displays a portion a mobile device screen from which the user may add content to the user's home page, first by activating the entry entitled "Add Anything to Y! onePlace." After activation, the user is presented with panel 920. Panel 920 includes area 921 for the user to input text for a personalized search, with entries at area 921 corresponding to a default set of locations from which to add content. Therefore, the "Add Anything" capability may be applied to search queries entered by the user and/or to preselected or customized sets of content objects (entities, links, RSS feeds, etc.) presented to the user without requiring the user to enter a search query, but which may utilize contextual and other information about the user, the available entities, and the device, to present one or more customized and/or personalized content objects for possible addition to the user's mobile home page.

[0107] Continuing to FIG. 10, the user viewing panel 1010 has input the text "Red Sox" at area 1011. After activation of the "oneSearch" button in panel 1010₅ the user is presented with panel 1020, offering the user a variety of choices from which to select one or more content objects to add content to the user's mobile home page. Some choices, such as area 1022 correspond to an RSS feed that the user may select. Other choices correspond to news articles and links to web sites, for example "www.redsox.com." Area 1021 corresponds to the entity "Boston Red Sox," which may be associated with one or more information owners, publishers or authors clustered together as described in more detail in Ser. No. 11/651,102, filed January 5, 2007, entitled "Clustered Search Processing" and previously incorporated by reference. [0108] In FIG. 11 , the user in panel 1110 has selected the first entry 1111 corresponding to the entity "Boston Red Sox" to be added to the user's home page. In an embodiment, a default set of content and sources is automatically generated by default. In an embodiment, the user may specifically select or unselect content sources or objects from those associated with the user's subscription to the entity "Boston Red Sox" in order to assemble an individualized set of aggregated content and sources associated with both the entity and the user. In an embodiment, content associated with entities may be free or fee-based, anonymous or registered depending on the topic and standards of practice. [0109] A screenshot illustrating the beginning and ending of the user's home page is shown in FIG. 12. As the screenshot is too long to be adequately displayed herein, panel 1210 represents the 'top' of the home page. The page scrolls down until panel 1220, representing the 'bottom' of the home page, is reached. As indicated in area 1230, an icon representing the entity "Boston Red Sox" has been added to the user's home page, providing the user with an easily accessible manner in which to view the system's latest personalized set of content corresponding to the Boston Red Sox. [0110] In FIG. 13 , panel 1310 illustrates content assembled by the system corresponding to the entity "Boston Red Sox." In FIG. 13, the system has added content sources corresponding to score and schedules, upcoming games, news, photos, and a roster. Unlike mere bookmarks, each of these content sources is dynamic, and information from each source may be automatically retrieved and assembled for the user's consumption. The selection and presentation of the associated content objects and feeds for a given entity may also be dynamically altered (i.e., removed, reordered, transcoded into different media types, etc.) based on preferences or actions of the system, the content publisher, the user, an advertiser, the network operator, the device manufacturer, or any combination thereof, as well as related to the various aspects of context described above (i.e., spatial, temporal, social, topical, etc.) in which the associated content objects and feeds for a given entity may be accessed by a user. [0111] FIG 14 illustrates two views in which the user may reconfigure the user's mobile home page, including the relative order in which items appear. Entry 1430 "Boston Red Sox" demonstrates the new entity added to the user's current content ordering of their mobile home page and the associated arrows allow the user to interactively remove, raise or lower the order of any entity or content on the list. In an embodiment, the user may group, name groups, and delete or reorder groups of any entity or content on the list. [0112] Advertisement Personalization

[0113] The dynamic mobile user web browsing interaction and content consumption experience differs greatly from the traditional, static methods of web browsing typically performed on a personal computer tied to home or work. Attempts to squeeze the traditional web browsing interaction and content consumption experience onto a small mobile screen fail to offer the benefits afforded by the methods and systems disclosed above. [0114] Similarly, use of traditional techniques to squeeze advertising material onto the mobile screen lead to a negative user experience. The traditional techniques lead to 'one-size-fϊts-air advertisements that intrude on user experience rather than accentuate it. [0115] Significant improvements are afforded in which actual mobile user history is used to (a) rank mobile users against brand and/or advertiser, and (b) predict specific users for specific advertisers at a specific campaign and time and/or space combination for delivery. A method and system for matching advertisements with mobile users is described below with reference to FIG. 8.

[0116] FIG. 8 illustrates a Mobile Homepage Targeting Engine (MHTE) 800, a fully customizable personal advertising system for matching specific ad copy or specialized content to specific ad or content delivery opportunities based upon at least the content, format, configuration and interaction data created by users and their mobile home pages, such as Yahoo's onePlace. It should be understood that the elements in FIG. 8 may be easily incorporated in search and data applications on server 130. [0117] MHTE 800 may include modules for: (a) managing interaction with advertisers and publishers; and (b) configuring a user's mobile home page with ad serving requests generated by one or more of the following: (1) network; (2) users; and (3) third parties.

[0118] In an embodiment, MHTE 800 includes Ad Copy Manager 804, Mobile Home Page Manager 808, Matching Manager 812, and Ad Display Manager 816. [0119] Ad Copy Manager 804 interacts with advertisers or publishers, and may permit advertisers or publishers to perform one or more of the following: (1) register and manage their accounts; (2) submit ad copy or specialized content; and (3) specify desired actions in the form of targeting models. Specialized content may be any form of new, updated or exclusive content marketed to the user by an owner, publisher or content retailer. A targeting model may be a function of one or more of the following user characteristics: (1) type of mobile access device; (2) access network; (3) demographic; (4) current user activity; (5) location or proximity to one or more known persons, places or things, (6) co-presence of other users, devices or POI, and (7) user history, preferences, or profile, among others.

[0120] Ad Display Manager 816 responds to specific ad requests whether from the user, the network or a third party service, e.g., carrier or ISP to populate Ad copy or specialized content received from the Ad Copy Manager 804 as instructed by the Matching Manager 812 and returns the instrumented copy for display by the requesting party. Such an ad request may include information indicating the desired target user criteria, the amount or maximum amount the advertiser is willing to pay, limitations or conditions of display, and the duration of the offer.

[0121] Mobile Homepage Manager 808 instruments and tracks user content, configuration and interaction with the user's mobile home page or other selected web page including spoken, tactile and sensed inputs.

[0122] Matching Manager 812 applies targeting models to user profiles to qualify, score and select ad copy or specialized content for serving in response to ad requests handled by Ad Display Manager 816. Data from Mobile Homepage Manager 808 may be forwarded to Matching Manager 812 for refinement or construction of various ad targeting models.

[0123] The following non-limiting examples illustrate some of the benefits afforded by MHTE 800. In a first example, the user has added "Pizza Restaurant" as an entity to onePlace, in order to receive updates such as special promotions, recent reviews, grand openings or any other content, feed or source of information relevant to the entity in the vicinity of the potential target user's current location, home, work or other locations of interest, e.g., regularly visited cities, locations designated of interest by the user, etc. Such an entity "Pizza Restaurant" appears as row 1520 in the non-limiting sample subset of oneSearch entity types illustrated in FIG. 15. The user would be then considered by MHTE 800 as a potential target eligible to receive customized advertising materials or specialized content.

[0124] After the user has added "Pizza Restaurant" as an entity on onePlace, the user would then receive updates when any local pizza restaurants publish new specials or content, when a review is written about a pizza restaurant and/or when a new pizza restaurant opens up near a location of interest. Additional qualification may be achieved by noting the user's other entities on onePlace, and the placement of these and related entities on the home page relative to each other as well as discarded, rejected or unchosen content. The level and type of user activity associated with each entity over time may be used to determine each entity's relative value in targeting advertising or specialized content to that user. Content targeting from publishers, as well as ad targeting from advertisers, may be incorporated.

[0125] Information regarding the same user may arrive into MHTE 800 through additional interactions by the user. For example, the user may have also added "vegetarian recipes" to onePlace which can be combined with the targeting information derived from the user's interactions with the "Pizza Restaurant" entity to develop a targeting profile that the user would likely prefer content, advertisements, or promotions related to "vegetarian pizzas" as opposed to "nonvegetarian pizzas." [0126] An illustration of the benefits afforded by MHTE 800 might be as follows: a pizza maker seeks new customers in a particular area. The pizza maker supplies ad copy or specialized content to Ad Copy Manager 804, along with target data supplied to Ad Display Manager 816.

[0127] The target data seeks males aged 18-34 who tend to eat out. Matching Manager 812 identifies a set of potential targets, including the user described above. [0128] Matching Manager 812 is alerted, reviews the placement of entities on the user's home page and considers the level and type of user activity associated with the entities in the determination of targeted advertisements and content for the user. In this example, Matching Manager 812 configures an advertisement for a meatless pizza special, served by the pizza maker at its nearby establishment. This advertisement is delivered to the user by Mobile Homepage Manager 808.

[0129] In a related example, the user enters a search request for "restaurant" received by Mobile Home Page Manager 808. Matching Manager 812 reviews the user's metadata and search history and identifies the user as a potential target.

[0130] In addition to the existence of the "Vegetarian Recipes" entity in the user's Yahoo! onePlace in the user's Yahoo! Mobile Home Page, additional data and metadata may further enhance the user targeting profile, such as the potential target's prior search history including several searches using the term "vegetarian" in Yahoo! oneSearch, and additionally, the target having posted status updates to MySpace from Yahoo! oneConnect regarding vegetarian recipes.

[0131] Matching Manager 812 is alerted, reviews the user's metadata including social relations and associations history, and configures an advertisement for the meatless pizza special, served by the pizza maker at its nearby establishment. This advertisement is delivered to the user by Mobile Homepage Manager 808 along with search results. [0132] FIG. 14 illustrates two manners in which a user's placement of entities on the user's home page may be used by Matching Manager 812 as part of the advertisement personalization process. Panel 1410 illustrates a user setting panel to coordinate the user's various electronic mail accounts, instant messagers, social networks, contacts, and calendar. When listing information of these kinds, it is customary for the user to place the user's favorite entries near the top of each list. Thus, in panel 1410, it appears the user is more actively uses the user's Facebook account than the user's MySpace account, as evidenced by the relative ranking of the two entries. [0133] As explained above, Matching Manager 812 may use these types of metrics, such as, ranking within group or between groups, as part of the advertisement personalization process increasing the weight of a particular user to an entity or brand based upon a higher order on their mobile home page or strong associations through past, current or predicted activities.

[0134] Similarly, panel 1420 corresponds to the user selected ordering of entities, such as those illustrated in FIGS. 9-13. For example, the entity ordering shown in panel 1420 corresponds to all entities shown in FIG. 12; note the final entity of "Boston Red Sox." In panel 1420, the user is able to reorder the user's entries as described above. Matching Manager 812 may use this ordering, for example, to select a particular advertisement from a pool of eligible advertisements, with the selected advertisement displayed to the user whereby content and entities associated with a mobile home page influence the selection of the advertisement by increasing the rank of potential advertisements related to content or entities to which the user is subscribed. In addition, a user's deselection of content or advertisers in the past may also be used to influence the ranking.

HARDWARE OVERVIEW

[0135] According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, mobile telephones, browsing or gaming devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques. [0136] For example, FIG. 16 is a block diagram that illustrates a computer system 1600 upon which an embodiment of the invention may be implemented. Computer system 1600 includes a bus 1602 or other communication mechanism for communicating information, and a hardware processor 1604 coupled with bus 1602 for processing information. Hardware processor 1604 may be, for example, a general purpose microprocessor.

[0137] Computer system 1600 also includes a main memory 1606, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 1602 for storing information and instructions to be executed by processor 1604. Main memory 1606 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 1604. Such instructions, when stored in storage media accessible to processor 1604, render computer system 1600 into a special-purpose machine that is customized to perform the operations specified in the instructions.

[0138] Computer system 1600 further includes a read only memory (ROM) 1608 or other static storage device coupled to bus 1602 for storing static information and instructions for processor 1604. A storage device 1610, such as a magnetic disk or optical disk, is provided and coupled to bus 1602 for storing information and instructions. [0139] Computer system 1600 may be coupled via bus 1602 to a display 1612, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 1614, including alphanumeric and other keys, is coupled to bus 1602 for communicating information and command selections to processor 1604. Another type of user input device is cursor control 1616, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 1604 and for controlling cursor movement on display 1612. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. Input device 1614 may also be a mobile telephone, gaming, reading, browsing or communication device that may include speech and recording (audio), visual (video and image), tactile and gestural input or sensing input capabilities as well as audio, visual and tactile output capabilities and multi-channel communication capabilities that is interactively connected through one or more networks such as to report its state and location in real-time over time and facilitate access to content and users.

[0140] Computer system 1600 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 1600 to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 1600 in response to processor 1604 executing one or more sequences of one or more instructions contained in main memory 1606. Such instructions may be read into main memory 1606 from another storage medium, such as storage device 1610. Execution of the sequences of instructions contained in main memory 1606 causes processor 1604 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.

[0141] The term "storage media" as used herein refers to any media that store data and/or instructions that cause a machine to operation in a specific fashion. Such storage media may comprise non- volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 1610. Volatile media includes dynamic memory, such as main memory 1606. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM₅ a FLASH-EPROM, NVRAM, any other memory chip or cartridge. [0142] Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 1602. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.

[0143] Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 1604 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 1600 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 1602. Bus 1602 carries the data to main memory 1606, from which processor 1604 retrieves and executes the instructions. The instructions received by main memory 1606 may optionally be stored on storage device 1610 either before or after execution by processor 1604 as well as on mobile device 1614 where it may utilize wireless data transmission, wireless client- server communications, and wireless peer-to-peer communications to deliver mobile applications. [0144] Computer system 1600 also includes a communication interface 1618 coupled to bus 1602. Communication interface 1618 provides a two-way data communication coupling to a network link 1620 that is connected to a local network 1622. For example, communication interface 1618 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 1618 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 1618 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0145] Network link 1620 typically provides data communication through one or more networks to other data devices. For example, network link 1620 may provide a connection through local network 1622 to a host computer 1624 or to data equipment operated by an Internet Service Provider (ISP) 1626. ISP 1626 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 1628. Local network 1622 and Internet 1628 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 1620 and through communication interface 1618, which carry the digital data to and from computer system 1600, are example forms of transmission media.

[0146] Computer system 1600 can send messages and receive data, including program code, through the network(s), network link 1620 and communication interface 1618. In the Internet example, a server 1630 might transmit a requested code for an application program through Internet 1628, ISP 1626, local network 1622 and communication interface 1618.

[0147] The received code may be executed by processor 1604 as it is received, and/or stored in storage device 1610, or other non- volatile storage for later execution. [0148] Embodiments of the invention may also be directed to computer program products comprising software stored on any computer readable medium including mobile telephones or computing devices. Such software, when executed in one or more data processing devices, causes a data processing device(s) to operate as described herein. Embodiments of the present invention employ any computer readable medium, known now or in the future. Examples of computer readable mediums include, but are not limited to. primary storage devices (e.g., any type of random access memory) and secondary storage devices (e.g., hard drives, floppy disks, CD ROMS, zip disks, tapes, magnetic storage devices, optical storage devices, MEMs, nanotechnology-based storage device, etc.).

[0149] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Sample Metadata Vocabulary

The searchmonkey-profile Vocabulary

This specification is on a periodic release schedule to improve conformance to industry standard vocabularies and enable common use cases to be accomplished in a consistent manner. It will be refreshed approximately weekly during the alpha and beta periods, while we work closely with partners.

Introduction to the searchmonkey-profile Vocabulary

The searchmonkey-profile vocabulary can be used in DataRSS feeds, and with pages embedded with RDFa and eRDF.

The whole of searchmonkey-profile consists of a number of vocabulary definitions, including industry-standard ones, as well as a simple way to declare all the prefixes. When using with DataRSS, add the following processing instruction to the beginning of a feed to declare the searchmonkey-profile vocabulary definitions:

<?profile http : //search . yahoo . com/searchmonkey-profile ? >

Normally this would require more than a dozen lines of xmlns declarations, but the Yahoo! open search platform, and any other software that recognizes searchmonkey-profile, will accept this single line instead.

The following sections describe the individual vocabularies. Feel free to browse around the sections below, especially the ones that apply to your field. But even other examples might be helpful to at least glance through, to get a feeling for how to best model your specific metadata.

Since the large tables of possible values can be intimidating, each section highlights specific properties to pay attention to, along with usage hints and examples.

The re) Vocabularies

The rel vocabulary defines values to use in the rel attribute of item. The naming convention for these properties is to always start with the rel: prefix, followed by a short term representing a person, place, thing, idea, or concept. While these are not RDF classes in a formal sense, in practice they are used much the same way and thus share the convention of starting with an initial capital letter, not counting the prefix.

The most general relationship possible is rel : Resource. All other relationships in this vocabulary directly or indirectly "subclass" from rel : Resource. In the table that lists all the rel values, a "Parent Class" column indicates the next-more-general relationship. For example from more specific to more general: a rel .- Image is a rel : Media which is a rel : Resource.

What if there is no good match for the kind of item for which you wish to extract metadata? Let's say you are extracting information about Seattle garage bands. There is no rel : SeattleGarageBand or rel : GarageBand or even rel : Band property. The most specific applicable property that's on the table is rel : Organization, which you should use in this case. Additionally, it would be a good idea to use the feedback email address on the specification to request adding new values to the vocabulary.

To use this vocabulary in feeds or with RDFa/eRDF in web pages, you need to map as closely as possible the items you have identified with classes of items from this vocabulary. If you are knowledgeable about existing RDF vocabularies, you can populate rel attributes with existing properties so long as the property takes a resource, not a literal string, as its domain. Some commonly occurring properties in this category include f oaf : knows and f oaf : homepage.

Common rels

• rel : Agent to stand for any person, organization, club, "things that do stuff

• rel : Photo, rel : Image, rel : Video, rel -.Audio to represent embedded multimedia elements

• rel : Photoset, rel : Videoset for collections of audio/video

• rel : Thumbnail for audio or video, a smaller thumbnail can be represented

• rel : Review to represent a review rating

• rel : Cluster to represent a page with a whole bunch of something on it, for example a search result page, or a list of hotels in a city

Also of particular note are the Task family of rel properties. These are intentionally defined in a fairly generic way, so pick the closest one that matches. Here are a few of the more commonly used ones;

• rel : Append to write a new review, comment on a blog, or generally create new content on the page

• rel : CheckAvailability to perform a non-binding check of a product or service availability

• rel : Edit to edit a wiki page for instance. For wiki sites, the rel : Edit property is particularly useful.

• rel : Procure to place an order, bid on or reserve something

• rel : SendEmail to email this page (or something on it)

• rel : SendToPhone to send this page (or something on it) to a phone

• The entire review: vocabulary. Note that the default scale is 1 to 5, but upper and lower can be changed with review .-best and review : worst

Encyclopedia/Reference/Info

• Common rels

• rel : Section to mark off specific sections. In general not every section should be included, just notable ones

• rel : Discussion to link to a separate discussion page

• rel : History to link to a separate history page

Business/Commercial/Directory

• Common rels

• rel : Listing to represent an overall listing

• rel : Business to represent a physical business (the business has a street adress, not the listing)

• rel : Neighborhood (usually with dc: description child) to indicate a particular neighborhood

News/Events/Gossip

• Common rels

• rel :NewsItem

Shopping/Commerce/Product pages

• Common rels

• rel : Listing to represent an overall listing

• rel : Product to represent the product itself (the product has a shippingWeight, not the listing) • rel : Specification Social Networking

• Common rels

• rel : Section to mark off specific sections. In general not every section should be included, just notable ones

• rel : Discussion to link to a separate discussion page

• rel : History to link to a separate history page

• rel .- WorkExperience for career-related items

• rel : CurrentWorkExperience for a presently-held job position

• rel : Degree to refer to specific educational credentials (not only "degrees", but also diplomas, doctorates, certifications, or any other kind of educational achievement). Further details of the item can be specified in dc : description or other properties.

Travel/Destination

• Common rels

• rel .- Business to link to a particular business, but

• rel : Hotel to link to a particular hotel

• rel : Restaurant to link to a particular restaurant

• rel : Flight to link to information about a particular flight

• rel : Map to link to a map of an area

• rel : Discussion to link to a separate discussion page

• rel : History to link to a separate history page

The property Vocabularies

The property attribute, contained within a meta tag, associates a literal string value with a resource. Official and unofficial industry standards in wide use make up the bulk of the list. Where no existing vocabulary is in wide use, certain key properties are defined in this profile. The following sections give guidance on which vocabularies to use. All properties are listed in the searchmonkey-profile vocabulary specification.

Associated with properties are a domain as well as range. The domain specifies which kinds of items the property applies to. For instance, media : width makes sense only on rel : Media items; it makes no sense applied to rel : Person, for example. The range specifies what kind of string values to expect in this property. For example, the dc : date property should accept a date in a certain format, namely YYYY-MM-DD, also known as ISO-8601 format.

Common properties

• dc : title for a general title

• dc : date for a general date

• dc : identif ier this is often a URL but can also be an ISBN or other identifier

• dc : creator for the creator of a resource

• dc : rights for a copyright statement

• dc .- description for a short, paragraph-length description

• reference : summary for a very short one-line description when dc:description is already present

• tagspace : tags as a space-separated tag list,

• tagspace : tag for a single tag (which might include spaces)

• atom: updated if you specifically need to call out the last— updated timestamp

• media : height and media : width for audio/video resources, it is good to specify the size in pixels with these attributes

• geo : location for a general, human-readable location like "San Francisco" or "South Bay Area" • geo -.point for a specific latitude/longitude (space-separated)

• geo : f loor for the floor number of a building

• f inance : stockTickers for a list of stock tickers relevant (semicolon-separated list, following NY Times metadata)

Business/Commercial/Directory

• Common properties

• vcard : adr for representing the street address in semicolon-separated pieces

• vcard : tel for the telephone number

• vcard : url for the website

• The entire commerce: vocabulary

News/Events/Gossip

• Common properties

• The entire news: vocabulary

Shopping/Commerce/Product

• Common properties

• The entire product: vocabulary

Social Networking

• Common properties Travel/Destination

• Common properties

• The entire travel:* vocabulary

Datatypes

Datatype vocabularies have two main uses. Some properties, like dc .- identifier can have many different possible values, say a URL, or an ISBN number. In this case the range of this property is only "string", however, if you want to specify explicitly that the value you've put in there is more specific than what the range will allow, you can specify a datatype.

For example:

<y :meta property="dc : identifier" datatype="use:url">http: //yahoo. com</meta>

Another use of datatype is for disambiguation. Let's say you are extracting information about a photo, and you have two different ways to represent the creator: by username and URL. The datatype sets apart the otherwise duplicate dc: creator statements.

Example:

<y:item rel="rel : Photo">

<y:meta property="dc:creator">The Nameless One</meta>

<y:meta property="dc : creator" datatype="use:url">http: //photosite .com/users/thenamelessone</meta> </y:item> When the property you need isn't in the list...

There will probably be times when the kinds of metadata you'd like to extract isn't found in the searchmonkey-profile vocabulary reference. It might be an existing RDF vocabulary, or something you needed to make up by yourself.

For example, let's say your metadata is the number of megapixels a digital camera can handle. As a minimum, you should try to define the property prefix, name, domain (what kinds of objects it applies to) and the range (what kind of values the data can be). In short, it should look something like another row in the property table. Of course, a more formal RDF vocabulary definition is great too.

The prefix should be something reasonably short, descriptive, and not likely to used for an unrelated purpose by someone else. As is the case with all CURIEs, the prefix stands for a longer URL, for which you need to define. It should be something underneath a domain name over which you have control.

The actual definition happens in an xmlns : _pref ix_ attribute. If our prefix is digicam, our property is megapixels, our URL is http : / /example , com/vocab/digi cam, our domain is rel : Product, and our range is decimal, then the theoretical table entry would look like this:

The declaration and use of the prefix would look like this:

<y:adjunct id="1.0" name= "piisting" xmlns :digicam="http: //example. com/vocab/digicam"> <y:item rel="rel :Product">

<y : meta prop= "digicam:megapixels " >7.2</y :meta> </y:item> </y:adjunct>

Examples

Encyclopedia/Reference/Info DataRSS Example

<?profile http://search.yahoo.com/searchmonkey-profile ?>

<feed xmlns="http: //www.w3.org/2005/Atom" xmlns :y= "http: //search. yahoo .com/datarss/" >

<id>http.- //articlesite.org/feedspec</id>

<authorxname>articlesite .org</namex/author>

<title>Articlesite Metacontent Feed for Yahoo ! </title>

<updated>2007-ll-14T04 : 05 : 06+07 : 00</updated>

<entry>

<title>Jerry Yang</title>

<id>http : //en. articlesite . org/wiki/Jerry_Yang</id>

<y:adjunct version="l.0" name="article "> <y:item rel="rel :Article">

<y:meta property="dc: title" >Jerry Yang</y:meta>

<y:item rel="rel :Photoset" resource="http://articlesite.org/slideshow/WKJzi4gl0GplnW_6ZJD_rQ"/> <y:item rel="rel :Thumbnail" resource="http://en.articlesite.org/commons/thumb/4/49/Jerry_Yang. jpg"> <y : meta property= "media : width" >225</y : meta> <y : meta property="media : height" >272</y :meta> </y : item>

<y:item rel="rel :Section"> <y:iteτn rel="rel : Section" resource="http : //en . articlesite . org/wiki/Jerry_Yang#Early_life " > <y:meta property= " dc :title">Early life</y :tneta> </y: item>

<y:item rel="rel : Section" resource="http :/ /en . articlesite . org/wiki/Jerry_Yang#Career"> <y:meta property="dc :title">Career</y:meta> </y: item>

<y:item rel="rel : Section" resource="http: //en. articlesite. org/wiki/Jerry_Yang#Personal_life"> <y:meta property= " dc .- title" >Personal life</y:meta> </y: item>

<y:meta property=

¹¹ rdfs : seeAlso" _>http ://en. articlesite . org/wiki/Jerry_Yang#See_also</y: meta> <y:meta property=

"reference: references ">http: //en. articlesite. org/wiki/Jerry_Yang#References</y : re <y:meta property=

" reference : externalLinks " >http : //en . articlesite . org/wiki/Jerry_Yang#External_lin

</y:item> <! — Section —>

<y:item rel="rel :Edit" resource="http: //en. articlesite . org/w/index. php?titIe=Jerry_Yang&amp; action=edit "/> <y:item rel="rel :SendEmail" resource="http: //en. articlesite. org/biz_share?bizid=WKJzi4glOGplnW_6ZJD_rQ"/> <y:item rel="rel :SendToPhone" resource="http://en.articlesite.org/biz_phone?bizid=WKJzi4glOGplnW_6ZJD_rQ"/>

<y.-item rel= "rel :Discussion" resource="http: //en. articlesite. org/wiki/Talk: Jerry_Yang"/> <y:item rel="rel :History" resource="http: //en.articlesite.org/w/index.php?title=Jerry_Yang&amp;action=history

</y:item> <! — Article —> </y:adjunct>

</entry> </feed>

Encyclopedia/Reference/lnfo RDFa Example

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http : //www . w3. org/TR/xhtmll/DTD/xhtmll-strict . dtd" > <html xtnlns="http: //www.w3.org/1999/xhtml" xmlns : dc= "http : //purl . org/dc/elements/1.1/ " xmlns :media= "http : / /search .yahoo . com/rarss/ " xmlns: rdfs= "http://www.w3.org/2000/0l/rdf-schema#" xmlns : rel="http : //search . yahoo . com/searchmonkey-relation/ " xmlns :reference= "http: //search.yahoo.com/searchmonkey/reference/" lang="en" xml :lang="en"> <head>

<title>RDFa marked up encyclopedia article</title> </head> <body>

<div rel="rel :Article">

<hl property="dc: title" >Jerry Yang</hl> <p>...

<a rel="rel :Photoset" href="http: //articlesite. org/slideshow/WKJzi4glOGplnW_6ZJD_rQ">Photos</a> <a rel= " rel . Thumbnail " href = "http : //en . articlesite . org/commons/thumb/4/49/Jerry_Yang ]pg" >

<span property= "media : width" content= "225 " /> <span property_^ "media - height " content=" 272 " / >

<img src= "http : / /en . articlesite org/commons/thumb/4/49/Jerry_Yang opg" /> </a>

</p>

<div class="mainentry" rel="rel :Section">

<div rel="rel Section" resource="http.//en articlesite.org/wiki/Jerry_Yang#Early_life"> <h2 property="dc:title">Early Iife</h2>

<div rel="rel : Section" resource="http_: //en. articlesite. org/wiki/Jerry_Yang#Career"> <h2 property="dc: title">Career</h2>

<div rel="rel: Section" resource="http: //en. articlesite. org/wiki/Jerry_Yang#Personal__life"> <h2 property="dc: title">Personal Iife</h2>

<div class="links"> <ul> href="http: //en. articlesite. org/w/index.php?title=Jerry_Yan.g&amp;action=edit"/x/Ii> <lixa rel="rel :SendMail" href="http: //en. articlesite. org/biz_share?bizid=WKJzi4glOGplnW_6ZJD_rQ"/x/li> <lixa rel="rel :SendToPhone" href="http- //en. articlesite. org/biz_jhone?bizid=WKJzi4glOGplnW_6ZJD_rQ"/x/Ii> < lixa rel="rel -Discussion" href= "http : //en . articlesite . org/wiki/Talk : Jerry_Yang" /></li> <lixa rel="rel :History" href= "http: //en. artlclesite.org/w/index.php?title=Jerry_Yang&aτnp;action=history"/x/li> </body> </html>

Vocabulary Tables

When including structured data, choose from the object types below

Object table

These are the recommended values for use in the rel attribute of item, which represent Nouns; people, places, or things that can have additional metadata properties:

Property table

These are recommended property values for use in the property attribute of meta

string string string string string string decimal decimal string string decimal string decimal decimal decimal decimal comma-list comma-list

comma— list

string

string boolean comma-list string string comma-list

comma— list

decimal

decimal

decimal comma-list comma-list boolean boolean string comma-list comma— list

semicolon-list boolean date decimal float integer nonempty integer integer uri integer

The Domain column indicates which types of objects can have the listed property. For example: the media : uri applies to objects of class rel : Media, which includes child classes rel : Image, rel : Thumbnail, and others.

A blank value in the Domain column indicates that the listed property applies to all objects.

The Range column indicates what kinds of values are appropriate for the listed property. Integer, float, decimal, and duration all map to the XML Schema datatypes of the same name. Date maps to XML Schema dateTime, which uses ISO 8601 format. URI maps to XML Schema anyURI and must be an absolute URL, not starting with a slash. Email maps to an email address defined by RFC 2822. List maps to a space-separated list.

A blank value in the Range column indicate that the listed property can accept values in any format.

Examples: integer

42 boolean

1 boolean

0 boolean true boolean false float

3.14159 decimal

2.00 duration

P3D [3 days] duration

PT8H [8 hours] date 2007-12-12T03:04:05+06:00 date

2007-12-12 uri http://search.yahoo.com/mrss email address@email.com space— list

2007 Croatia photos vacation fun [a whitespace-separated list] comma— list

2007, Croatia photos, vacation, fun [a comma-separated list, individual list items may have insignificant leading or trailing whitespace, but internal whitespace is significant] semicolon— list

622 9th Ave W;Williston,ND; 58801 [a semicolon-separated list] nonempty

"OK"

Datatype table

These are the recommended property values for use in the datatype attribute of meta

units:bytes Information size in octets

For values found in the rel and property attributes, note that the following preferred vocabularies and associated prefixes are automatically provided: atom

A vocabulary for Atom concepts <http://www.w3.org/2005/Atom> cc

Creative Commons <http://creativecommons.org/licenses/> dc

Dublin Core Metadata <http://purl.org/dc/elements/Ll/> /oaf

Friend-of-a-friend <http://xmlns.eom/foaf/0. l/> geo a vocabulary for geographic metadata <http://www.georss.org/georss> media

RSS-Media <http://search.yahoo.com/mrss/> rdf

RDF <http://www.w3.org/1999/02/22-rdf-syntax-ns#> rdfs

RDF Schema <http://www.w3.org/2000/01/rdf-scherna*> review

RDF review <http://www.purl.org/stuff/rev#> veal

RDF calendar <http://www.w3.org/2002/12/calS> vcard vcard <http://www.w3.org/2006/vcard/ns>

...and... rel object relation vocabulary <http://search.yahoo.com/searchmonkey-relation/>

...and... assert a vocabulary for error-detection and flagging in metadata

<http://search.yahoo.com/searchmonkey/assert/> commerce a vocabulary for commerce-related metadata <http://search.yahoo.com/searchmonkey/commerce/> context a vocabulary for transient contextual information <http://search.yahoo.com/searchmonkey/context/> finance a vocabulary for financial metadata <http://search.yahoo.com/searchmonkey/finance/> job a vocabulary for job-related metadata <http://search.yahoo.com/searchmonkey/job/> news a vocabulary for news metadata <http://search.yahoo.com/searchmonkey/news/> product a vocabulary for product-related metadata <http://search.yahoo.com/searchmonlcey/product/> reference a vocabulary for references and links <http://search.yahoo.com/searchmonkey/reference/> social a vocabulary for social graph connections and information <http://search.yahoo.com/searchmonkeysocial/> tagspace a vocabulary for tagsets <http://search.yahoo.com/searchmonkey/tagspace/>

...and... country a datatype used to indicate a particular country

<http://search.yahoo.com/searchmonkey-datatype/country/> currency a datatype used to indicate a particular currency

<http://search.yahoo.com/searchmonkey-datatype/currency/> use a datatype used to differentiate use of a property

^ttpV/search.yahoo.com/searchmonkey-datatype/use^

Claims

CLAIMSWhat is claimed is:

1. A method comprising: receiving user input; associating first metadata with the user input, the first metadata describing a user environment and providing context for the user input; associating second metadata with one or more content sources; and sending, to the user, content that is based at least in part on the user input data, the first metadata, and the second metadata, wherein the method is performed by one or more special-purpose computing devices.

2. The method of Claim 1 , wherein the first metadata includes entity information derived from performing a disambiguation of the user input.

3. The method of Claim 2, wherein the entity information is at least one of a person, organization, association, location, business, event, object, vehicle, device, appliance, tag, label, link, time, or time period.

4. The method of Claim 1 , wherein the first metadata includes at least one of spatial data, temporal data, social data, physical object-related data, or topical data.

5. The method of Claim 1 , wherein the first metadata includes social relations and association information.

6. The method of Claim 1 , wherein the one or more content sources comprises one or more of the following: a content index ,a feed, file, ftp site, webdrive, network cloud, and mobile device.

7. The method of Claim 1 , wherein each of the one or more content sources comprises one or both of a content feed and an application program interface.

8. One or more storage media storing instructions which, when executed by one or more computing devices, cause performance of the method recited in any one of Claims 1 through 7.

9. A server comprising: one or more processors; memory operatively coupled to the one or more processors; a storage device having stored thereon a user data store, the user data store associating first metadata with input from a user, the first metadata describing a user environment and providing context for the user input; a mobile search refine manager, the mobile search refine manager parsing a search query contained in the user input to form a modified search query; a search module, the search module performing a search using the modified search query in combination with second metadata associated with one or more content sources; and a search clustering module, the search clustering module sending content to the user, the content based at least in part on the user input data, the first metadata, and the second metadata.

10. The server of Claim 9, wherein the first metadata includes entity information derived from performing a disambiguation of the user input.

11. The server of Claim 10, wherein the entity information is at least one of a person, organization, association, location, business, event, object, vehicle, device, appliance, tag, label, link, time, or time period.

12. The server of Claim9, wherein the first metadata includes at least one of spatial data, temporal data, social data, physical object-related data, or topical data.

13. The server of Claim9, wherein the first metadata includes social relations and association information.

14. The server of Claim9, wherein the one or more content sources comprises one or more of the following: a content index ,a feed, file, ftp site, webdrive, network cloud, and mobile device.

15. The server of Claim9, wherein each of the one or more content sources comprises one or both of a content feed and an application program interface.