Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/90—Details of database functions independent of the retrieved data types
  • G06F16/95—Retrieval from the web
  • G06F16/953—Querying, e.g. by the use of web search engines
  • G06F16/9535—Search customisation based on user profiles and personalisation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/90—Details of database functions independent of the retrieved data types
  • G06F16/95—Retrieval from the web
  • G06F16/953—Querying, e.g. by the use of web search engines
  • G06F16/9536—Search customisation based on social or collaborative filtering
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/90—Details of database functions independent of the retrieved data types
  • G06F16/95—Retrieval from the web
  • G06F16/953—Querying, e.g. by the use of web search engines
  • G06F16/9538—Presentation of query results

Definitions

• Embodiments of the invention relate, generally, to the field of conceptually organizing information, and more specifically to the analysis of the use of conceptually related information to effect organization of information.
• the Internet is an extensive network of computer networks through which information is exchanged by methods well known to those in the art (e.g. the use of TCP and IP protocols, etc) r
• the Internet permits users to send and receive data between computers connected to this network.
• This data may include web sites, home pages, databases, text collections, audio, video or any other type of information made available over the
• This information may be referred to as articles or documents, and may include, a web page, data on a web page, attachments to a web page, or other data contained in a storage device (e.g., database).
• a storage device e.g., database
• the articles may include key terms representing selected portions of the information contained in the article. These key terms are available over the Internet to other computers and permit these other computers to locate the article.
• a search engine is programs that allow the remote user to type in one or more search terms. The search engine then compares the search query with the key terms from the articles and retrieves at least a portion of the articles having key terms that match the search query.
• search engine will then display to the user the portion of the article such as the title. The user can then scroll through these retrieved portions of the articles and select a desired article.
• the document provider may use as many search terms as are possibly related to the article.
• some articles or search engines use every word in the article as key terms.
• search engines will retrieve many articles that are unrelated, or only peripherally related, to the subject matter that the user desires to find through a combination of search terms.
• many users of such search engines are not skilled in formulating key-term search queries and produce extremely broad searches that often retrieve thousands of articles.
• Such schemes typically base relevancy, at least in part, on the number of "clicks” the document received (i.e., the number of times a document was selected).
• Such schemes known generally as “popularity ranking schemes” or “click popularity schemes”, provide a search result list in which the highest ranked documents are those that attracted and satisfied the greatest number of previous users.
• click popularity schemes generate results that reflect search context. For example, previous search schemes would return documents containing all of the query terms, but would not automatically exclude words that are not part of the query.
• search engines that organize information provided in response to queries using numerous factors including time-based and use-based factors.
• a scheme may use the activity of previous users in response to particular queries to adjust the relevancy of the query response documents.
• user activity may include the number of clicks in conjunction with the timing of prior users' selections or use of particular information.
• Such schemes may also consider where in a prior results listing a particular document was ranked when prior users selected it, actual versus expected use frequency of a document, and how the selected documents were used.
• An embodiment of the invention provides a method in which a set of conceptually related information is provided to a plurality of users. One or more responses regarding the presentation of the conceptually related information is received from each of two or more users. The response from two or more users is then correlated and the presentation of the conceptually related information is modified based upon the correlated responses.
• Figure 1 is a flow chart that illustrates a process in which the organization and presentation of a set of conceptually related information is modified based upon a correlation of responses from multiple users in accordance with one embodiment of the invention
• Figure 2 is a flow chart that illustrates a process by which an association of stored content with a user query is effected based upon the correlation of user activity and/or user information obtained during a search session with similar information of other users in accordance with one embodiment of the invention;
• Figure 3 is a block diagram that illustrates a system for organizing and presenting conceptually related information and associating particular stored content with respective user queries in accordance with one embodiment of the invention;
• Figure 4 illustrates the search log for each of three independent users in accordance with one embodiment of the invention;
• Figure 5 is a flow chart that illustrates a process for providing more relevant search results in accordance with one embodiment of the invention;
• Figure 6 is a flow chart that illustrates a process for providing relevant query suggestions in accordance with one embodiment of the invention;
• Figure 6A is a flow chart that illustrates a process for providing query spelling correction suggestions in accordance with one embodiment of the invention
• Figure 7 is a flow chart that illustrates a process in which suggested queries are provided in accordance with one embodiment of the invention.
• Figure 8 is a flow chart that illustrates a process in which similar results are provided in accordance with one embodiment of the invention.
• Figure 9 is a flow chart that illustrates a process in which information is correlated to provide more relevant search results in accordance with one embodiment of the invention.
• Figure 10 is a flow chart that illustrates a process by which more relevant search results are provided to a user based upon the user's location in relation to a location associated with a pick in accordance with one embodiment of the invention
• Figure 11 is a flow chart that illustrates a process by which more relevant search results are provided to a user based upon the user's location in accordance with one embodiment of the invention
• Figure 12 is a block diagram that illustrates a digital processing system in accordance with one embodiment of the invention. DETAILED DESCRIPTION OVERVIEW
• Embodiments of the invention provide methods and systems for conceptually organizing and presenting information in which a correlation of users' responses to the organization and presentation of information is used to determine an optimal organization and presentation of the information.
• the user activity and/or user information of multiple users, during a search session is correlated with queries to effect an evolving association between queries and the organization and presentation of documents.
• Systems, in accordance with such embodiments employ the ability to store users' activity over the entire search session, thus, making possible the correlation of a number of different types of user activity and user information.
• the use of correlated user input allows such systems to provide relevant search results without the limitations imposed by the key- word-based systems of the prior art.
• Figure 1 illustrates a process by which the organization and presentation of a set of conceptually related information is modified based upon a correlation of responses from multiple users in accordance with one embodiment of the invention.
• Process 100 begins with operation 105 in which a set of conceptually related information is organized and presented to multiple users by an information provider.
• the set of conceptually related information is a gathering page containing information related to a general concept of interest to a particular group of users.
• Such a gathering page may contain a number of related sub-concepts of any sort, including links to documents, directories, databases, spreadsheets, news items, audio, video, images, applications, advertisements, product descriptions, and reference information, as well as links to lists, tables, trees, or catalogs of any of the above items, and links to other gathering pages, all of which may be collected from any number of sources.
• multiple, distinctly formulated, gathering pages are created that vary the organization and presentation of the set of information, including the number, type, arrangement, and prominence of the related sub-concepts. In this way, the information provider attempts to anticipate an optimal way of organizing and presenting the information.
• the information provider receives responses from the users in regard to the organization and presentation of the information.
• the users' responses may be in the form of selecting (or refraining from selecting) portions of the information. For example, some users may select information of particular sub-concepts of a given gathering page, while not selecting others.
• the received responses from multiple users are correlated. That is, a determination is made as to the extent to which the same statistically significant associations have been made by independent users.
• the correlation of responses may take many forms. Exemplary correlations for various user responses are described in more detail below.
• the organization and presentation of the set of conceptually related information is modified based upon the correlated responses.
• a gathering page may be reformulated to more adequately meet the needs of the an individual user, a group or class of users, or all users.
• reformulation may include adding or deleting information.
• one or more sub-concepts may be added to or deleted from a gathering page.
• embodiments of the invention may be used to effect the association of user search engine queries to stored content (e.g., one or more documents). That is, embodiments of the invention may be used to determine search results in response to a particular query that are more relevant (i.e., either generally more relevant or more relevant to a particular user).
• Figure 2 illustrates a process by which an association of stored content with a user query is effected, based upon the correlation of user search engine (USE) activity and/or user information obtained during a search session with similar information of other users in accordance with one embodiment of the invention.
• Process 200 begins at operation 205 in which USE activity information and or user information, during a search session, is recorded for several independent users.
• a search session includes any sequence of search engine actions (activities that can be recorded by the search engine) of a given user.
• the USE activity may include the issuing of queries, the clicking of links on the search page leading to internal or external data, the clicking of links on subsequent internal pages leading to internal or external data, and a return to the search page or any internal page subsequent to clicking an internal or external link.
• the USE activity may be continuous or occur within a practical duration period. That is, a time period may be specified that indicates termination of a search session. For example, if an interruption in recorded user activity exceeded a specified time, it may be practical to assume the search session was terminated. Subsequent user activity may be viewed as a new search session.
• the recorded USE activity information and/or user information for several independent users is correlated.
• the information which reflects entire search sessions for each user, may be correlated in various ways in accordance with alternative embodiments of the invention. Exemplary correlations for various USE activity and/or user information are described in more detail below.
• the results of the correlation of the recorded USE activity information and or user information for several independent users obtained from operation 210 are used to effect an association between particular stored content and corresponding user queries. Such association may provide a search result for a given query having greater relevance, in general, or to one or more particular users.
• SYSTEM [0040] Embodiments of the invention may be implemented in a network environment. Figure 3 illustrates a system for organizing and presenting conceptually related information and associating particular stored content with respective user queries in accordance with one embodiment of the invention.
• System 300 illustrated in Figure 3, illustrates a network of digital processing systems (DPS) 300 that include one or more server DPSs, shown as server DPS 320 and a plurality of client DPSs, shown as client DPSs 305 - 308.
• the DPSs of system 300 are coupled one to another and are configured to communicate a plurality of various types of stored content including documents, such as, web pages, content stored on web pages, including text, graphics, and audio and video content.
• the stored content may be audio/video files, such as programs with moving images and sound.
• Information may be communicated between the DPSs through any type of communications network through which a plurality of different devices may communicate, such as, for example, but not limited to, the Internet, a wide area network (WAN) not shown, a local area network (LAN), an intranet, or the like.
• the DPSs are interconnected one to another through Internet 310, which is a network of networks having a method of communicating data, as described above, and is well known to those skilled in the art.
• the communication links coupling the server DPS to client DPSs need not be a direct link, but may be indirect links including, but not limited to, broadcasted wireless signals, network communications, or the like.
• a set of conceptually related information such as a gathering page, is presented to several users via client DPSs 305 - 308.
• the conceptually related information may also be a search result in response to a user query, communicated from one or more of the client DPSs 305 - 308.
• the information may take many forms, and may be, for example, a list of web-page URL addresses provided by a web-page developer.
• the user makes some response to the presentation. For example, the user may conduct USE activity as described above.
• the user response and other user information, related to the presentation of the information to the user is recorded and communicated to the user response/information correlation application 321.
• the application 321 correlates the user responses and information of several users and modifies the organization and presentation of the conceptually related information based upon the correlation results.
• DATA ANALYSIS [0043] Embodiments of the invention obtain USE activity information and/or user information and correlate this information to aid in defining relevancy through the use of consensus choices of multiple users. This correlation analysis includes the process of evaluating common actions, or information of multiple users, to identify statistically significant associations.
• association and “statistically significant association”, as used in regard to such embodiments, are defined as follows.
• An “association” is any pairing of queries, terms, concepts, articles or other web data, or combinations thereof, which are made explicitly or implicitly, consciously or unconsciously, by a user during a search session.
• An association may be indicated through the issuing of queries and/or the selecting of links (e.g., hypertext links) to queries, terms, concepts, articles, or other web data.
• links e.g., hypertext links
• Embodiments of the invention provide search engine results with far greater relevancy than prior art schemes by recording much more information and more specific information (including USE activity information and/or user information), and effecting a more in-depth analysis of the information.
• One embodiment of the invention provides a system that creates and operates on data files that maintain all USE activity information and user information as shown in Table 1. (Tables used to describe various embodiments of the invention are exemplary and do necessarily represent the actual data structures of embodiments of the invention.)
• Table 1 shows a data file containing a number of data elements that record the queries for a number of users at various times and the URL (pick) that was selected (clicked) subsequent to each respective query for each respective user.
• Such a data file may contain numerous other data elements representing USE activity information and/or user information.
• Such data elements may represent, for example, the display rank of the result selected, the order the result was clicked by the user during the session, the user IP address, geo-location of the IP address, etc.
• Q2RP simplified query-to result pick
• Table 1 A is a simplified example of prior art data structure, which may include score adjustment fields, as known in the prior art. For example, a score, which could be something as simple as the sum of the picks, could also be the result of a more sophisticated adjustment algorithm; the scores and adjustments could also be stored. [0049] The prior art data structure of Table 1 A was adequate for producing ranked search results, but it represented the loss of a great deal of raw information. As exemplified in Table 1 A, prior art schemes did not record or analyze much of the USE activity information and user information available from a search session. This was due to a number of reasons including limited storage, lack of any practical use for the information, and lack of appreciation of the application to which such information could be applied.
• the need for such score adjustment fields is obviated by the extent of the recorded and stored information. That is, for such embodiments, no score adjustment information is stored because any score needed can be calculated parametrically based on the stored information. Moreover, parameters and algorithms for calculating scores can be altered, as necessary, without affecting stored data. [0051] In accordance with various embodiments of the invention, the in-depth analysis of the information depends upon the ability to recognize and record correlative data (including USE activity information and user information) between independent users. That is, by recording and analyzing much more of the information, including up to all of the information of a user session, search results having a much higher degree of relevancy can be obtained.
• correlative data including USE activity information and user information
• a requirement that a URL, following a subsequent query, must be picked at least twice to be correlated to an original query may be imposed. Such a requirement, will eliminate many of the picks of the B queries erroneously associated with query A.
• a correlation analysis is provided that relies on statistically large samples to recognize multiple relevant associations. For such embodiments, prior art restrictions on the associations analyzed, and arbitrary compartmentalization of the data, are reduced to increase relevancy. That is, recording and analyzing much more of the user session information also allows for many more relevant types of associations to be analyzed.
• a system uses one or more of a small set of basic correlations, and combinations thereof.
• any number of correlations may be determined and used to effect increased relevancy of search results, or other objectives, in accordance with various embodiments of the invention.
• Some exemplary correlations will be discussed, in detail, below.
• a query-to-pick (Q2P) correlation associates a query with a pick.
• Q2P query-to-pick
• the Q2P correlation associates a query with all picks in a user session. This is in contrast to prior art schemes that terminated association of a given query with picks upon issuance of a subsequent query.
• each association's score can be adjusted, based upon well- known factors, including rank of the pick in the result list at the time of association, duration of the pick (interval until next known user action), age or order of the association (relative to older or newer associations), and age of the first known instance of association.
• Each user session can be of infinite duration. In a practical application, a reasonable time limit, or limit on intervening actions, should be imposed beyond which no relationship between picks and queries will be assigned. Alternatively or additionally, an interruption of sufficient duration can indicate a break in sessions.
• a search log excerpt in accordance with one embodiment of the invention, is shown below as Table 2. In various alternative embodiments, any other items could be captured in the search log, but are excluded here for clarity.
• Figure 4 illustrates the search log for each of three independent users in accordance with one embodiment of the invention.
• Search log excerpt 400 shown in Figure 4 contains search logs 410, 420, and 430 depicting the search information for three independent users, Ul, U2, and U3, respectively, as described above in reference to Table 2.
• the dashed boxes in each search log represent the Q2RP portion of the Q2P search information.
• search log 410 includes Q2RP portion 411 in which a query Ql has resulted in a pick P5.
• Search log 410 also includes Q2RP portion 412 in which a query Q2 has resulted in picks PI, P2, and P3.
• Search log 420 includes an interruption 421 of 48 hours duration.
• search log 430 includes an interruption 431 of 2 hours duration.
• such interruption may not delineate two separate user sessions. That is, all of the search activity of search log 430 may be viewed as a single user session and the information correlated accordingly.
• Table 2 A illustrates a tabulation of the click information contained in Table 2 in accordance with an embodiment of the invention.
• Table 2B illustrates a tabulation of the click information contained in Table 2 in accordance with a typical prior art scheme employing a Q2RP correlation.
• a single, uncorrelated non-Q2RP click (such as Q3P1 in the table) may not inspire enough confidence to release the result to users, whereas for a single, uncorrelated Q2RP click, the association is reinforced by the fact that the search engine presented the result for the original search.
• a pick-to-query (P2Q) correlation associates all queries recorded during a user session that are correlated with a given pick issued during that user session.
• the search log excerpt of Table 2 illustrates the output of P2Q correlation. That is, the same data generated for Q2P can be re-indexed for P2Q
• a query-to-query (Q2Q) correlation associates all queries issued during a user session with all other queries issued during that session. For one embodiment, a score may be assigned to each association based upon various factors, including the time between queries, the number of intervening queries and/or picks, age or order of the association (relative to older or newer associations), whether or not the query results generated picks, and the pair-wise order of the associated queries, among others. [0070] Determining if the query results generated picks, as well as the pair- wise order of the associated queries, can be particularly informative, as they can indicate whether one query is a "correction" of another. For any practical application, it is useful to know which of two associated queries is an error, and which, a correction.
• a search log excerpt in accordance with one embodiment of the invention, is shown below as Table 3. Only the query portion of the search log is required to create a Q2Q table.
• Table 3 A illustrates a tabulation of the click information contained in Table 3 in accordance with an embodiment of the invention (assuming the order of queries issued is ignored).
• the lower triangular area of Table 3 A can be used to retain the pair- wise query order information, avoiding double-booking cases like rows 301-303.
• a scoring scheme may be employed in which numerous factors can vary or penalize the score. For example, duplicates (e.g., association in rows 101 and 102 and associations made in rows 401 and 402) could be penalized. Or, for example, an uncorrelated Q2Q association, like Q2Q3, would not inspire enough confidence to release the result to users.
• a pick-to-pick (P2P) correlation associates all picks issued during a user session with all other picks issued during that session, and as such, is analogous to the Q2Q correlation described above.
• a score may be assigned to each association based upon various factors, including the time between picks, the number of intervening queries and/or picks, age or order of the association (relative to older or newer associations), and the pair- wise order of the associated picks, among others.
• a search log excerpt in accordance with one embodiment of the invention, is shown below as Table 4. Only the pick portion of the search log is required to create a P2P table.
• Table 4A illustrates a tabulation of the click information contained in Table 4 in accordance with an embodiment of the invention (assuming the order of picks recorded is ignored).
• two or more correlations can be concatenated to provide more relevant search results.
• two or more basic correlations may be concatenated so as to mock the basic correlations in order to augment their results, especially in cases of sparse data, or generate additional products requiring broad matching.
• Concatenating correlations can introduce erroneous relationships; therefore, for one embodiment of the invention, the correlations are correlated. For example, if a Q2Q correlation requires two independent users, a QQQ correlation string should require two queries that link the original and final queries. In QQQ, a mutually associated query (Q3) associates a query (Ql) with another query (Q2). If two or more independent, mutually associated queries make the same association, that is a correlation. [0084] As discussed above in reference to basic correlations, every association between the original pick or query and the output pick or query, was made by at least two individuals (or by a search engine plus one individual). With concatenated correlations, it is possible that no individual user (or search engine) has ever associated the original pick or query with any of output picks or queries. An indirect correlation would still require a minimum of two independent users.
• Combinations of correlations require correlation through multiple intermediate picks or queries.
• the number of distinct picks and/or queries linking the original and related picks or queries is more important than how closely the intermediate picks and/or queries are linked to the original and related picks and/or queries.
• For a correlation there must be at least two distinct link paths, regardless of how many users have established those links. That is, associating Ql and Q2, via one intermediate node Q3, even if multiple users have established that association, does not constitute a correlation between Ql and Q2.
• a number of factors contribute to the strength of a correlation, including the number of link paths, the strength of each direct correlation component link, and the distinctiveness of each intermediate node.
• Table 6A and Table 6B illustrate the results of multiplying the intersecting QP scores of Tables 2A and 2B, respectively, and summing the results over the queries to determine the combined association scores. (This is not necessarily an optimal algorithm, but is used for exemplary purposes).
• Table 6C and Table 6D illustrate the equivalent Q2Q correlation tables for Table 6A and Table 6B, respectively.
• the QPQ correlation results in accordance with one embodiment of the invention, generate considerably more correlations than the narrower QRPQ correlation of a more specific alternative embodiment of the invention, and considerably more than the Q2Q correlation in accordance with still another alternative embodiment of the invention. Furthermore, using the QPQ correlation allows for providing suggestions for poorly formed queries that do not generate search results. This was not possible with the prior art schemes.
• USER-TO-USER Just as queries and/or picks can be correlated by users, users can be correlated by queries and/or picks.
• the general process which is an indirect correlation analogous to a QPQ correlation, will be referred to as a user-to-user (U2U).
• URPU user-to-result pick-to-user
• a score may be assigned to each association based upon various factors. For example, assuming a time threshold, the click in row 103 is penalized due to the user spending a very short time at the URL.
• Table 7 A illustrates a tabulation of the click information contained in Table 7 in accordance with an embodiment of the invention.
• an affinity algorithm in accordance with one embodiment, might account for differences in frequency of search and award different weights to queries, picks, and query-picks.
• Table 7B illustrates the calculated affinity information for the tabulated click information of Table 7 A.
• the pick scores for corresponding users are adjusted according to the affinity information. For example, results for a future query of user Ul, would adjust scores for picks generated by user U2 by a function of (0.67) and scores for picks generated by user U3 by a function of (0.33). Picks generated by a user that had no affinity would be assigned some default value.
• Embodiments of the invention that correlate the Q2P and/or Q2P- equivalent combination USE activity information allow a user to obtain more relevant search results in a number of ways. For example, a user may refine a search and associate some portion of the revised results with the original search. That is, picked documents need not have a textual relationship to the original search, but only a conceptual relationship. The conceptual relationship may provide a better response to the original search. Search results based upon a correlation of Q2P USE activity information circumvents the limitation of text- based retrieval found in the' prior art schemes. Embodiments of the invention have the capability to preserve and exploit the user research process. This capability can be used to effect systems with a number of distinct advantages over prior art schemes.
• FIG. 5 illustrates a process for providing more relevant search results in accordance with one embodiment of the invention.
• Process 500 begins at operation 505 in which a query is received from a user. Such a query may have one or more particular characteristics, which, once discerned, may be the basis for providing more relevant search results in accordance with various alternative embodiments of the invention.
• the Q2P USE activity information for a number of users is correlated in regard to the received query. Each query may have various particular characteristics that may be determined through correlation of Q2P USE activity information.
• the query may be subject to different meanings for different users, the query may be misspelled, the query may have equivalent phrasings, the query may have more relevant portions and less relevant portions, the query may be associated with a specific result product or result product mix, and the query may be subject to broader or narrower search results.
• the query may be subject to different meanings for different users, the query may be misspelled, the query may have equivalent phrasings, the query may have more relevant portions and less relevant portions, the query may be associated with a specific result product or result product mix, and the query may be subject to broader or narrower search results.
• search results are provided to the user in response to the query.
• the search results provided may be based upon one or more characteristics of the query. For example, where it is determined that the query has different meanings, results based upon a more popular meaning may be provided in response to the query.
• results obtained by process 500 in comparison to results obtained through typical prior art methods are shown below in relation to the particular characteristics of queries discussed above.
• Table 8 illustrates exemplary search results for several queries having more than one meaning. As shown, embodiments of the invention allow the search results to relate to the more popular meaning of the particular query in comparison to a typical prior art scheme.
• Table 9 illustrates exemplary search results for a misspelled query
• embodiments of the invention allow the search results to relate to the probable correctly spelled query. In this way, embodiments of the invention can determine the correct spelling for a misspelled query.
• Prior art schemes typically produce no search results, or poor search results, for such misspellings.
• such a spelling correction is a "soft" correction. That is, in accordance with one embodiment of the invention, the results provided in response to the query are the results that were preferred by a majority of the users who entered the exact query. If most users consider the query a misspelling, then many results will contain the corrected query. If most users consider the query intentional, then most results will contain the query unchanged. If both interpretations are legitimate, the results may be a combination. For such embodiments, it becomes highly unlikely that an erroneous spelling correction will be provided because all corrections are conceptually relevant, not just textually similar.
• prior art spelling corrections are typically "hard” corrections. That is, such schemes identify a misspelled query, attempt to correct it and then search for results based upon the correction. Such schemes provide irrelevant results where legitimate queries are misdiagnosed as misspelled, or if the query is in fact misspelled, but the algorithmic correction was also not the intended query. An "active" correction would require a user to click a link to search again on a suggested query, which again might be an e ⁇ oneous correction.
• Table 10 illustrates exemplary search results for a query having two or more representations or equivalent phrasings (e.g., "Burma and Vietnamese").
• embodiments of the invention provide search results for each equivalent phrasing with a high degree of commonality. For the example provided, five of the top nine search result URLs are common to each of the equivalent phrasings. This is much higher similarity than prior art schemes. Exemplary; Embodiment ' ⁇ >
• Example query “burma history”
• Example query “myanmar history”
• Table 11 illustrates exemplary search results for a query including a redundant portion. As shown, embodiments of the invention allow redundant, non-essential, and non-relevant portions of the query to be ignored, where doing so provides a more relevant, simpler query.
• search result picks on various result products are treated equally. That is, users need not specify which type of result they are seeking. Results issued can reflect prior preferences exhibited by the user or the preferences of previous independent users. For example, if a query precipitates an image search so frequently that a particular image is the highest scoring pick, that image may be provided as a search result in accordance with its score. For one embodiment, the various top-scoring results of different result products need not be interleaved in score order, but may be grouped by product for presentation.
• Table 12 illustrates exemplary search results for a query resulting in a search result product mix (e.g., including graphics results). As shown, embodiments of the invention allow search results that include a product mix.
• Example', query ' Exemplary Embodiment search-xesults > Y « "who is homer 1. www.simpsonslOO .com simpson?" 2. www. thesimpsons . com 3. www.snpp.com 4. members .aol.com/jynelms/tv/simpsons .html
• Embodiments of the invention provide the ability to vary the breadth of the search results by adjusting the weighting of the Q2P vs. Q2RP pick scores.
• Table 13 illustrates narrow and broad search results for the query "Stanford”.
• Column 1 of Table 13 contains narrow results related to picks on the query.
• Column 2 of Table 13 shows broad results (e.g., only picks that were not made in response to the results of the original query are shown).
• the column 2 results represent the broad scope of the query topic, whereas the column 1 results help the user explore the depth of the query topic.
• a combination of the narrow and broad results may provide the most relevant search results. For one embodiment of the invention, a number of combinations having varying breadths are created, allowing the user to select a "broaden the results" or "focus the results” link, or vary a control to adjust the mixture.
• FIG. 6 illustrates a process for providing relevant query suggestions in accordance with one embodiment of the invention.
• Process 600 shown in Figure 6, begins at operation 605 in which a query is received from a user.
• the Q2Q (and/or Q2Q-equivalent combination) USE activity information is correlated. Correlating the Q2Q USE activity information directly tends to produce results that are predominantly refinements of the query.
• Table 14 illustrates query suggestions provided in response to the original query, "electronic eavesdropping devices," in accordance with one embodiment of the invention.
• Column 1 of Table 14 contains query suggestions based upon the correlation of Q2Q USE activity information directly, while column 2 of Table 14 contains query suggestions based upon the correlation of Q2Q equivalent combination (i.e., QPQ) USE activity information.
• one or more query suggestions are provided to the user.
• query suggestions may be presented on the result page.
• a link may be provided to a page of query suggestions.
• query suggestions can be sorted into refinements (containing all the original search terms) and related searches, prior to presentation.
• correlation of Q2Q (and/or Q2Q-equivalent combination) USE activity information is used to effect an active query spelling correction scheme.
• An active query spelling correction scheme is one in which the user selects a suggested query correction to obtain search results based upon the query correction.
• FIG. 6A illustrates a process for providing query spelling correction suggestions in accordance with one embodiment of the invention.
• Process 600A shown in Figure 6A, begins at operation 605A in which a query is received from a user. Such a query may be a misspelling of the user's intended query.
• the Q2Q (and/or Q2Q-equivalent combination) USE activity information is correlated, as discussed above, in reference to operation 610 of process 600.
• one or more query suggestions are determined based upon the co ⁇ elated Q2Q (and/or Q2Q-equivalent combination) USE activity information.
• one or more of the query suggestions are determined as spelling corrections of the original query (i.e., the query received at operation 605 A).
• the determination of the query suggestion is effected in various different manners depending upon the query received. For example, for previously observed queries, where Q2Q information exists, conceptually related queries that are also textually similar can be identified using the Q2Q correlation information and an edit distance computation algorithm. In such a case, various criteria may be used to effect the determination of a query suggestion as a spelling correction of the original (i.e., received) query.
• the determined query suggestion may be determined as a spelling correction of the received query.
• more criteria may be considered to bolster confidence in the determination of the query suggestion as a spelling correction of the received query. For example, where the query suggestion tends to be issued subsequent to the received query more often than prior to the received query or where the query suggestion tends to result in more user picks than the received query, these criteria would bolster confidence in the determination of the query suggestion as a spelling correction of the received query.
• the query received at operation 605A is a previously unobserved query, then no Q2Q information exists. In such cases, in accordance with one embodiment of the invention, if a word in a query is suspected of being an error, it is evaluated in one of two ways depending upon whether or not the suspect word has been previously observed or not.
• the suspect word has not been previously observed, other queries in which all the other keywords in the received query appear can be identified.
• the queries with the most distinctive (low frequency) words in common with the received query can be identified.
• the identified queries can be weighted based on textual similarity with the received query and the most common query with high textual similarity to the received query identified.
• the correlated Q2Q USE activity information is used in conjunction with a conventional n-gram-type model.
• word association frequencies are extracted from the query frequency data to determine the common partners for known keywords or phrases. This data, in combination with edit distance, is used to determine a query as a likely spelling correction suggestion for an unknown word in a multi-word query.
• one or more query suggestions that have been determined as a spelling correction of the received query are provided to the user as likely spelling corrections of the received query.
• the query suggestions are presented as described above in operation 615 of process 600 in reference to Figure 6.
• FIG. 7 illustrates a process in which suggested queries are provided in accordance with one embodiment of the invention.
• Process 700 begins at operation 705 in which a query and corresponding search result, providing a number of result URLs, is received.
• the P2Q (or P2Q-equivalent combination) USE activity information is correlated for each result URL.
• suggested queries are provided for each result URL based upon the correlated P2Q USE activity information.
• Table 15 illustrates an exemplary search result page for the query "rangers" and exemplary corresponding suggested queries in accordance with one embodiment of the invention.
• one or more results that are similar to a particular result, provided in response to a received query are provided using P2P or a P2P-equivalent combination.
• a query is received and a result corresponding to the query is evaluated.
• similar results are also provided. That is, for example, a few related pages could be specified along with the result and/or a link could lead to a new result page with additional similar results.
• similar results for an image will yield mostly other images
• similar results for a web page will yield mostly other web pages, etc.
• FIG. 8 illustrates a process in which similar results are provided in accordance with one embodiment of the invention.
• Process 800 shown in Figure 8, begins at operation 805 in which a query is received and a corresponding search result determined.
• the P2P (or P2P-equivalent combination) USE activity information is correlated for the corresponding search result.
• one or more similar results are provided based upon the correlated P2P USE activity information. That is, a list of results that are closely related to the received result is provided. Similar results can constitute search picks, images, news items, etc.
• Table 16 illustrates exemplary similar results for a result provided in response to the query "phoenix" in accordance with one embodiment of the invention.
• each user is viewed as a community of one, with a degree of affinity to other users.
• Figure 9 illustrates a process in which U2U information is correlated to provide more relevant search results in accordance with one embodiment of the invention.
• Process 900 shown in Figure 9, begins at operation 905 in which an affinity matrix is calculated for every user who demonstrates at least a minimum amount of search activity. Calculation of such a matrix could proceed as follows for one embodiment of the invention. First, all queries and picks by a given user, Ul, are extracted. Then users, U2, that duplicated at least N of Ul 's queries and/or picks are identified, and U2's remaining queries and picks are identified.
• weights are assigned to each query and pick and an affinity score calculated.
• the weights may be inversely proportional to how common among all users that query and/or pick was, and the affinity score is calculated from a comparison of the weighted picks/queries in common with the total weighted picks/queries.
• the search results, resulting from a new query of Ul are analyzed, and the score of each result is increased if any previous picks were due to users with a non-zero affinity score.
• the results are reordered based upon the score adjustments of operation 910 and the reordered results are presented to the user.
• a process similar to process 900 can be used to increase the relevancy of search suggestions.
• user information includes geo- location information for users and, by extension, their queries and picks.
• the geo-location information may include latitude and longitude as well as city, state and country name.
• such user information is used to provide search results based upon a user's geographic location.
• Figure 10 illustrates a process by which more relevant search results are provided to a user based upon the user's location in relation to a location associated with a pick in accordance with one embodiment of the invention.
• Process 1000 begins at operation 1005 in which a URL that has acquired more than a specified number of picks is assigned a latitude- longitude "centroid" and a "sphere of influence.”
• the centroid is the position where some function of the sum of the distances to each of the statistically significant logged user-picks is minimized.
• the centroid will represent the location of the actual brick-and-mortar incarnation of the URL (the URL exhibiting local bias) to a high degree of accuracy. Where no local bias exists, the centroid location has little or no meaning. For example, a URL without a brick-and-mortar incarnation may exhibit no local bias on the national scale.
• the sphere of influence is a geographic radius within which the URL is expected to be highly appealing to users and outside of which it is expected to be largely unappealing.
• the radius of the sphere of influence is inversely proportional to some measure of local bias. For example, a highly localized URL like www.canariesbaseball.com forms a very tight sphere of influence centered on Sioux Falls, SD (i.e., the hometown of the Sioux Falls Canaries).
• the sphere of influence is adjusted based upon statistical data. For example, some locations generate many more clicks than other locations.
• Raw click score data is adjusted to reflect the disparity.
• the user's geographic location is compared with a set of pre-computed location data for responses to a particular query and the search result scores are adjusted based upon the geographic location of the user. The scores of those URLs that are nearby and have a high degree of localization are increased, while the scores of distant URLs are decreased (scores of URLs without much localization bias are left unchanged).
• the user is provided with search results that are based, at least in-part, upon the geographic location of the user. This will allow users to receive subjectively relevant results within the initial several results. For example, in contrast to prior art schemes, a user in Wyoming who enters "state tax forms" will be provided with state tax forms for Wyoming more readily. LOCALIZATION BY USER
• Localization can be based on picks, as was described above, or can be based on the user in accordance with alternative embodiments.
• localization based upon the user allows results for popular queries to be cached by jurisdiction and then issued to users based on location without any real-time calculations.
• FIG 11 illustrates a process by which more relevant search results are provided to a user based upon the user's location in accordance with one embodiment of the invention.
• Process 1100 begins at operation 1105 in which jurisdictions that account for a specified portion of the clicks for a given query are determined, starting with the highest-level jurisdictions. For example, the query "car insurance" yields a mix of US and UK sites, with a majority of clicks on the UK sites from users located in the UK and a majority of clicks on the US sites from users located in the US.
• the click score for jurisdictions accounting for a specified threshold portion of the clicks is increased, and a separate result list, tagged to that jurisdiction, is created.
• the click scores for users outside of that jurisdiction are decreased, creating a general list for users outside the jurisdictions that account for a specified threshold portion of the clicks.
• real-time blended lists are created based on user proximity to special jurisdictions.
• Embodiments of the invention provide methods and systems for conceptually organizing and presenting information in which a correlation of users' responses to the organization and presentation of information is used to determine an optimal organization and presentation of the information. Though described above in terms of several exemplary embodiments, many additional applications exist for alternative embodiments of the invention. [00161]
• the invention includes various operations. Many of the methods are described in their most basic form, but operations can be added to or deleted from any of the methods without departing from the basic scope of the invention.
• the operations of the invention may be performed by hardware components or may be embodied in machine-executable instructions as described above. Alternatively, the steps may be performed by a combination of hardware and software.
• FIG. 12 illustrates an embodiment of a digital processing system that may be used for the server DPS 320, or client DPSs 305 -308, as discussed above in reference to Figure 3, in accordance with an embodiment of the invention.
• processing system 1201 maybe a computer or a set top box that includes a processor 1203 coupled to a bus 1207.
• memory 1205, storage 1211, display controller 1209, communications interface 1213, and input/output controller 1215 are also coupled to bus 1207.
• Processing system 1201 interfaces to external systems through communications interface 1213.
• Communications interface 1213 may include an analog modem, Integrated Services Digital Network (ISDN) modem, cable modem, Digital Subscriber Line (DSL) modem, a T-l line interface, a T-3 line interface, an optical carrier interface (e.g. OC-3), token ring interface, satellite transmission interface, a wireless interface or other interfaces for coupling a device to other devices.
• Communications interface 1213 may also include a radio transceiver or wireless telephone signals, or the like.
• communication signal 1225 is received/transmitted between communications interface 1213 and the cloud 1230.
• a communication signal 1225 may be used to interface processing system 1201 with another computer system, a network hub, router, or the like.
• communication signal 1225 is considered to be machine readable media, which may be transmitted through wires, cables, optical fibers or through the atmosphere, or the like.
• processor 1203 may be a conventional microprocessor, such as, for example, but not limited to, an Intel Pentium family microprocessor, a Motorola family microprocessor, or the like.
• Memory 1205 may be a machine-readable medium such as dynamic random access memory (DRAM) and may include static random access memory (SRAM).
• Display controller 1209 controls, in a conventional manner, a display 1219, which in one embodiment of the invention, may be a cathode ray tube (CRT), a liquid crystal display (LCD), an active matrix display, a television monitor, or the like.
• the input/output device 1217, coupled to input/output controller 1215, may be a keyboard, disk drive, printer, scanner and other input and output devices, including a mouse, trackball, trackpad, or the like.
• Storage 1211 may include machine-readable media such as, for example, but not limited to, a magnetic hard disk, a floppy disk, an optical disk, a smart card or another form of storage for data.
• storage 1211 may include removable media, read-only media, readable/writable media, or the like. Some of the data may be written by a direct memory access process, into memory 1205 during execution of software in computer system 1201. It is appreciated that software may reside in storage 1211, memory 1205, or may be transmitted or received via modem or communications interface 1213.
• machine readable medium shall be taken to include any medium that is capable of storing data, information or encoding a sequence of instructions for execution by processor 1203 to cause processor 1203 to perform the methodologies of the present invention.
• machine readable medium shall be taken to include, but is not limited to, solid-state memories, optical and magnetic disks, carrier wave signals, and the like.

Landscapes

• Engineering & Computer Science (AREA)
• Databases & Information Systems (AREA)
• Theoretical Computer Science (AREA)
• Data Mining & Analysis (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Information Transfer Between Computers (AREA)

Applications Claiming Priority (6)

Publications (2)

Family

ID=34682419

Family Applications (2)

Family Applications Before (1)

Country Status (7)

Families Citing this family (8)

Citations (2)

Family Cites Families (9)

• 2004
  • 2004-12-07 ES ES04813565T patent/ES2288455T1/es active Pending
  • 2004-12-07 DE DE04813564T patent/DE04813564T1/de active Pending
  • 2004-12-07 CA CA2546492A patent/CA2546492C/en not_active Expired - Fee Related
  • 2004-12-07 CN CNB2004800358406A patent/CN100511223C/zh not_active Expired - Fee Related
  • 2004-12-07 CA CA2546494A patent/CA2546494C/en not_active Expired - Fee Related
  • 2004-12-07 EP EP04813565A patent/EP1706816A4/de not_active Withdrawn
  • 2004-12-07 DE DE04813565T patent/DE04813565T1/de active Pending
  • 2004-12-07 JP JP2006543980A patent/JP2007513440A/ja active Pending
  • 2004-12-07 WO PCT/US2004/041089 patent/WO2005057366A2/en active Application Filing
  • 2004-12-07 CN CNA2004800358389A patent/CN1898638A/zh active Pending
  • 2004-12-07 WO PCT/US2004/041252 patent/WO2005057367A2/en active Application Filing
  • 2004-12-07 WO PCT/US2004/041254 patent/WO2005057368A2/en active Application Filing
  • 2004-12-07 EP EP04813564A patent/EP1697865A4/de not_active Ceased
  • 2004-12-07 WO PCT/US2004/041255 patent/WO2005057369A2/en active Application Filing
  • 2004-12-07 ES ES04813564T patent/ES2288454T1/es active Pending
  • 2004-12-07 JP JP2006543979A patent/JP2007513439A/ja active Pending

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events