US8065316B1 - Systems and methods for providing search query refinements - Google Patents

Systems and methods for providing search query refinements Download PDF

Info

Publication number
US8065316B1
US8065316B1 US10/953,117 US95311704A US8065316B1 US 8065316 B1 US8065316 B1 US 8065316B1 US 95311704 A US95311704 A US 95311704A US 8065316 B1 US8065316 B1 US 8065316B1
Authority
US
United States
Prior art keywords
refinement
query
particular
plurality
suggestions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US10/953,117
Inventor
Steven Baker
Paul Haahr
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Priority to US10/953,117 priority Critical patent/US8065316B1/en
Assigned to GOOGLE INC. reassignment GOOGLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HAAHR, PAUL, BAKER, STEVEN
Application granted granted Critical
Publication of US8065316B1 publication Critical patent/US8065316B1/en
Assigned to GOOGLE LLC reassignment GOOGLE LLC CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: GOOGLE INC.
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/332Query formulation
    • G06F16/3325Reformulation based on results of preceding query
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/332Query formulation
    • G06F16/3322Query formulation using system suggestions
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/93Document management systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation

Abstract

A system and method for generating query refinement suggestions may include collecting refinement data for at least one received source query. The collected refinement data is then clustered to form at least one cluster. At least one potential refinement query suggestion is identified from the refinement data within the at least one cluster.

Description

FIELD OF THE INVENTION

The present invention relates in general to query processing and, in particular, to a system and method for providing search query refinements.

BACKGROUND OF THE INVENTION

Although the Internet traces back to the late 1960s, the widespread availability and acceptance of personal computing and internetworking have resulted in the explosive growth and unprecedented advances in information sharing technologies. In particular, the Worldwide Web (“Web”) has revolutionized accessibility to untold volumes of information in stored electronic form to a worldwide audience, including written, spoken (audio) and visual (imagery and video) information, both in archived and real-time formats. In short, the Web has provided desktop access to every connected user to a virtually unlimited library of information in almost every language worldwide.

Search engines have evolved in tempo with the increased usage of the Web to enable users to find and retrieve relevant Web content in an efficient and timely manner. As the amount and types of Web content have increased, the sophistication and accuracy of search engines have likewise improved. Generally, search engines strive to provide the highest quality results in response to a search query. However, determining quality is difficult, as the relevance of retrieved Web content is inherently subjective and dependent upon the interests, knowledge and attitudes of the user.

Existing methods used by search engines are based on matching search query terms to terms indexed from Web pages. More advanced methods determine the importance of retrieved Web content using, for example, a hyperlink structure-based analysis, such as described in S. Brin and L. Page, “The Anatomy of a Large-Scale Hypertextual Search Engine,” (1998) and in U.S. Pat. No. 6,285,999, issued Sep. 4, 2001 to Page.

A typical search query scenario begins with a search query being submitted to a search engine. The search engine executes a search against a data repository of potentially retrievable Web content and identifies the candidate Web pages. Searches can often return thousands or even millions of results, so most search engines typically rank or score the results to obtain only the most promising results. The top Web pages are then presented to the user, usually in the form of Web content titles, hyperlinks, and other descriptive information, such as snippets of text taken from the Web pages.

Providing quality search results can be complicated by the literal and implicit scope of the search query itself. A poorly-framed search query could be ambiguous or be too general or specific to yield responsive and high quality search results. For instance, terms within a search query can be ambiguous at a syntactic or semantic level. A syntactic ambiguity can be the result of an inadvertent homonym, which specifies an incorrect word having the same sound and possibly same spelling, but different meaning from the word actually meant. For example, the word “bear” can mean to carry or can refer to an animal or an absence of clothing. A semantic ambiguity can be the result of improper context. For example, the word “jaguar” can refer to an animal, a version of the MACINTOSH operating system, or a brand of automobile. Similarly, search terms that are too general result in overly broad search results while search terms that are too narrow result in unduly restrictive and non-responsive search results.

Accordingly, there is a need for an approach to providing suggestions for search query refinements that will resolve ambiguities, over generalities, or over specificities occurring in search queries. Preferably, such an approach would provide refined search queries that, when issued, result in search results closely related to the actual topic underlying the intent of the original search query and provide suggestions that reflect conceptual independence and clear meanings as potential search terms.

SUMMARY OF THE INVENTION

According to one aspect, a method for generating query refinement suggestions may include collecting refinement data for at least one received source query; clustering the collected refinement data to form at least one cluster; and identifying at least one potential refinement query suggestion from the refinement data within the at least one cluster.

According to still another aspect a method includes reviewing historical query data to identify source queries and associated refinement queries to generate refinement data. The refinement data includes a number of source query/refinement query associations. The method further includes clustering the refinement data to form at least one cluster and identifying at least one potential refinement query suggestion from the refinement data within the at least one cluster.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, explain the invention. In the drawings,

FIG. 1 is an exemplary diagram of a network in which systems and methods consistent with the principles of the invention may be implemented;

FIG. 2 is an exemplary diagram of a client or server of FIG. 1 according to an implementation consistent with the principles of the invention;

FIG. 3 is a flow diagram of exemplary processing for providing search query refinements in response to a received user query according to an implementation consistent with the principles of the invention;

FIG. 4 is a flow diagram illustrating exemplary processing for collecting manual refinement data according to one implementation consistent with the principles of the invention;

FIG. 5 is a flow diagram illustrating exemplary processing for collecting manual refinement data according to another implementation consistent with the principles of the invention;

FIG. 6 is a flow diagram illustrating exemplary processing for characterizing refinement data according to one implementation consistent with the principles of the invention;

FIG. 7 is a flow diagram illustrating exemplary processing for characterizing refinement data according to another implementation consistent with the principles of the invention;

FIG. 8 is a flow diagram illustrating exemplary processing for clustering refinement queries according to one implementation consistent with the principles of the invention;

FIG. 9 is a flow diagram illustrating exemplary processing for selecting representative refinements according to one implementation consistent with the principles of the invention;

FIGS. 10 and 11 are flow diagrams illustrating exemplary processing for scoring and ranking the clusters and their representative refinement query according to implementations consistent with the principles of the invention; and

FIG. 12 is an exemplary diagram of a graphical user interface containing refinement query suggestions that may be presented according to one implementation consistent with principles of the invention.

DETAILED DESCRIPTION

The following detailed description of the invention refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the invention.

Overview

The quantity of documents becoming searchable via search engines is substantially increasing. Accordingly, search queries which may be submitted to locate relevant documents may more easily suffer from potential ambiguities or generalities. It is beneficial to identify and provide search query refinements which may remedy the initial query deficiencies. As described herein, search query refinements may be generated and suggested to assist the user in more quickly and more accurately identifying desirable search results. More specifically, refinement data may be collected based on a received source query. The collected refinement data may be grouped and scored to identify potential refinement suggestions.

Exemplary Network Configuration

FIG. 1 is a block diagram showing a system 100 for providing search query refinements, in accordance with the present invention. System 100 may include multiple individual clients 104 communicatively interfaced to a server 102 via a network 106, such as the Internet, or other form of communications network, as would be recognized by one skilled in the art. Individual clients 104 may be operated by users 118 who transact requests for Web content and other operations through their respective client 104. Three clients 104 and one server 102 have been illustrated as connected to network 106 for simplicity. In practice, there may be more or fewer clients and servers. Also, in some instances, a client may perform the functions of a server and a server may perform the functions of a client.

In general, each client 104 can be any form of computing platform connectable to a network, such as network 106, and capable of interacting with application programs. Exemplary examples of individual clients include, without limitation, personal computers, digital assistances, “smart” cellular telephones and pagers, lightweight clients, workstations, “dumb” terminals interfaced to an application server, and various arrangements and configurations thereof, as would be recognized by one skilled in the art. Network 106 may include various topologies, configurations, and arrangements of network interconnectivity components arranged to interoperatively couple with enterprise, wide area and local area networks and include, without limitation, conventionally wired, wireless, satellite, optical, and equivalent network technologies, as would be recognized by one skilled in the art. Server 102 may include server entities that gather, process, search, and/or maintain documents in a manner consistent with the principles of the invention.

For Web content exchange and, in particular, to transact searches, each client 104 may execute a Web browser application 116 (“Web browser”), which preferably implements a graphical user interface and through which search queries are sent to a Web server 120 executing on the server 102. Typically, each search query describes or identifies information, generally in the form of Web content, which is potentially retrievable via the Web server 102. The search query provides search characteristics, typically expressed as individual terms, such as keywords and the like, and attributes, such as language, character encoding and so forth, which enables a search engine 122, also executing on the server 102, to identify and send back search result documents, generally in the form of Web pages. Other styles, forms or definitions of search queries and characteristics are feasible, as would be recognized by one skilled in the art.

In response to search engine execution, the Web pages may be sent back to Web browser 116 for presentation, usually in the form of Web content titles, hyperlinks, and other descriptive information, such as snippets of text taken from the Web pages. The user may then view or access the Web pages on the graphical user interface and can input selections and responses in the form of typed text, clicks, or both. In one implementation, server 102 maintains a search database 110 in which Web content 124 is maintained. In an alternative embodiment, Web content 124 may also be maintained remotely on other Web servers (not shown) interconnected either directly or indirectly via the network 106 and which are preferably accessible by each client 104. In a further embodiment, server 102 may maintain a cache 126 in which cached documents 128 and cached queries 130 are maintained. More specifically, cache 126 associates each cached document 128 with one or more cached queries 130 to improve searching performance, as is known in the art. Finally, in a still further embodiment, search engine 122 may maintain a query log 132 in which records of previous search queries 134 are tracked.

In one embodiment consistent with principles of the invention, search engine 122 preferably identifies Web content 124 best matching the search characteristics to provide high quality Web pages. In identifying matching Web content 124, search engine 122 operates on information characteristics describing potentially retrievable Web content. Note the functionality provided by server 120, including Web server 120 and search engine 122, could be provided by a loosely- or tightly-coupled distributed or parallelized computing configuration, in addition to a uniprocessing environment.

A “document,” as the term is used herein, is to be broadly interpreted to include any traditional printed work of authorship, such as books, magazines, catalogs, newspapers, articles, etc. A “web document,” as the term is used herein, is to be broadly interpreted to include any machine-readable and machine-storable work product available via a network, such as network 106. A web document may include, for example, a web site, a file, a combination of files, one or more files with embedded links to other files, a news group posting, a blog, a web advertisement, etc. In the context of the Internet, a common web document is a web page. Web pages often include textual information and may include embedded information (such as meta information, images, hyperlinks, etc.) and/or embedded instructions (such as JAVASCRIPT, etc.). A “link,” as the term is used herein, is to be broadly interpreted to include any reference to or from a web document.

Search queries can potentially be ambiguous or lack generality or specificity. Such poorly-framed search queries can be remedied through search query refinements, which can be provided in response to search query issuances. In accordance with one aspect of the invention, search query refinements may be generated and suggested to assist the user in more quickly and more accurately identifying desirable Web content 124.

FIG. 12 is an exemplary diagram of a graphical user interface 1200 containing results of a search performed on an initial source query 1202 according to an implementation consistent with the present invention. In this example, a search on initial source query 1202 has resulted in a number of refinement suggestions 1204 being initially presented to the user. By clicking on any of the underlined query refinement suggestions 1204, the user may perform a search on the selected suggestion, thereby refining their search. In addition to refinement suggestions 1204, conventional search results 1206 indicative of web content 124 may also be provided.

The individual computer systems, including server 102 and clients 104, include general purpose, programmed digital computing devices consisting of a central processing unit (processors 107 and 112, respectively), random access memory (memories 108 and 114, respectively), non-volatile secondary storage, such as a hard drive or CD ROM drive, network or wireless interfaces, and peripheral devices, including user interfacing means, such as a keyboard and display. Program code, including software programs, and data is loaded into the RAM for execution and processing by the CPU and results are generated for display, output, transmittal, or storage. Web browser 116 may be an HTTP-compatible Web browser, such as the INTERNET EXPLORER, licensed by Microsoft Corporation, Redmond, Wash.; NETSCAPE NAVIGATOR, licensed by Netscape Corporation, Mountain View, Calif.; or other forms of Web browsers, as are known in the art.

Exemplary Client/Server Architecture

FIG. 2 is an exemplary diagram of a client or server entity (hereinafter called “client/server entity”), which may correspond to one or more of clients 104 and server 102, according to an implementation consistent with the principles of the invention. The client/server entity may include a bus 210, a processor 220, a main memory 230, a read only memory (ROM) 240, a storage device 250, an input device 260, an output device 270, and a communication interface 280. Bus 210 may include a path that permits communication among the elements of the client/server entity.

Processor 220 may include a conventional processor, microprocessor, or processing logic that interprets and executes instructions. Main memory 230 may include a random access memory (RAM) or another type of dynamic storage device that may store information and instructions for execution by processor 220. ROM 240 may include a conventional ROM device or another type of static storage device that may store static information and instructions for use by processor 220. Storage device 250 may include a magnetic and/or optical recording medium and its corresponding drive.

Input device 260 may include a conventional mechanism that permits an operator to input information to the client/server entity, such as a keyboard, a mouse, a pen, voice recognition and/or biometric mechanisms, etc. Output device 270 may include a conventional mechanism that outputs information to the operator, including a display, a printer, a speaker, etc. Communication interface 280 may include any transceiver-like mechanism that enables the client/server entity to communicate with other devices and/or systems. For example, communication interface 280 may include mechanisms for communicating with another device or system via a network, such as network 106.

As will be described in detail below, the client/server entity, consistent with the principles of the invention, may perform certain searching-related operations. The client/server entity may perform these operations in response to processor 220 executing software instructions contained in a computer-readable medium, such as memory 230. A computer-readable medium may be defined as a physical or logical memory device and/or carrier wave.

The software instructions may be read into memory 230 from another computer-readable medium, such as data storage device 250, or from another device via communication interface 280. The software instructions contained in memory 230 may cause processor 220 to perform processes that will be described later. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes consistent with the principles of the invention. Thus, implementations consistent with the principles of the invention are not limited to any specific combination of hardware circuitry and software.

Exemplary Processing

FIG. 3 is a flow diagram of exemplary processing for providing search query refinements in response to a received user query according to an implementation consistent with the principles of the invention. Initially, refinement data is collected for plurality of source queries (act 300). Additional details regarding the refinement data collection process will be set forth in detail below with respect to FIG. 4. Following collection, the collected refinement data is characterized for each source query (act 302). In one implementation consistent with principles of the invention, refinement data may be characterized by creating composite term vectors representing documents returned by refinement queries. Additional details regarding the characterization process will be set forth in detail below with respect to FIGS. 5-6.

Once the refinement data for each source query has been characterized, potential refinement queries identified from the refinement data may be clustered based upon the composite term vectors of act 302 (act 304). In this manner, potential refinement queries are grouped into related clusters. Additional details regarding the clustering process will be set forth in detail below with respect to FIG. 7. Next, the most representative refinement from each cluster identified in act 304 is identified (act 306). Additional details regarding the clustering process will be set forth in detail below with respect to FIG. 8. In one exemplary implementation, selecting representative refinements for the query clusters may include choosing the refinement query from each cluster which occurs most frequently as identified during refinement data collection. Once selected, the clusters and their representative refinements are scored and ranked for presentation to users upon entry of the respective source query (act 308). Additional details regarding the scoring and ranking process will be set forth in detail below with respect to FIGS. 9 and 10. By characterizing, clustering, and scoring previously received manual refinement queries, the above-described system is able to identify the most accurate and helpful refinement queries for display to users in response to received source queries.

FIG. 4 is a flow diagram illustrating exemplary processing for collecting manual refinement data initially described in act 300 set forth above. Initially, processing may begin when search engine 122 receives a search term (or a group of search terms) from a user as an initial source query for searching a document repository (act 400). In one implementation, the document repository includes documents available from the Internet and/or a database (or set of databases) and the vehicle for searching this repository is a search engine, such as search engine 122 (FIG. 1). In one embodiment, search engine 122 may receive the source query via web browser software 116 on a client, such as client 104 (FIG. 1). Upon receipt of an initial source query, search engine 122 logs the source query in query log 132 (act 402) and identifies Web content 124 that best matches the query (act 404). Server 102 then returns the identified Web content 124 to client 104 as an initial results set (act 405).

As described above, providing quality search results can be complicated by the literal and implicit scope of the search query. Accordingly, users may wish to refine an initial source query in order resolve ambiguities, over-generalities, or over specificities occurring in properly framed search queries. Preferably, such an approach would provide refined search queries that lead to better search results for the user. Ideally, the refined search queries should be conceptually independent and have clear meaning as potential search terms.

In accordance with this manual query refinement process, search engine 122 may receive an additional query (“refinement query”) from the user upon viewing the initially returned results set (act 406). It may next be determined whether a predetermined criteria has been satisfied indicating that the additional received query is likely a refinement query to the initial source query (act 408). In one implementation consistent with principles of the invention, one such criteria may be include a predetermined time limit based upon the receipt of the initial source query. One such exemplary time limit may be two minutes, however any suitable time limit may be used, such that queries received within the time limit are likely to correspond to refinements of the initial source query.

If the criteria has been satisfied, search engine 122 next determines whether the initial source query has been previously received (act 410). If not, an association is created between the received refinement query and the initially received source query (act 412). Next, a counter associated with the refinement query is initialized to 1 indicating a first instance of a source query and subsequent refinement query (act 414). Next, the associated source and refinement queries are stored in database 110 for subsequent usage described below (act 416). However, if the initial source query had been previously received, search engine 122 determines whether the refinement query has been previously received as a refinement to the initial source query (act 418). If so, a counter associated with the refinement query is incremented to indicate the additionally received refinement query (act 420). If the refinement query has not been previously received as a refinement to the initial source query the counter associated with the refinement query is initialized to 1 indicating a first instance of a received refinement query (act 422). The process then returns to act 406 for receipt of the next refinement query (if any). The results of the repeated steps thereby populate database 110 with a plurality of source query/refinement query associations for subsequent usage in generating accurate suggested refinement queries. In one implementation consistent with principles of the invention, multiple receipts of the same initial source query result in incremented count totals for the respective refinement queries. These count totals may be subsequently used in clustering and scoring the potential refinement queries to identify refinement queries suitable for presentation to users.

FIG. 5 is a flow diagram illustrating an alternative implementation of processing for collecting manual refinement data. Initially, processing may begin when search engine 122 receives a search term (or a group of search terms) from a user as an initial source query for searching a document repository (act 500). Upon receipt of an initial source query, search engine 122 logs the source query in query log 132 (act 502) and identifies Web content 124 that best matches the query (act 504). Search engine 122 may also identify user profile or interface information that may be used to further refine returned results (act 505). In one implementation consistent with principles of the invention, user profile or interface information may include language preference (e.g., English, Spanish, etc.) or country/region interface (e.g., USA, Japan, China, etc.) variables. Server 102 then returns the identified Web content 124 to client 104 as an initial results set (act 506).

As described above, search engine 122 may then receive an additional query (“refinement query”) from the user upon viewing the initially returned results set (act 507). It may next be determined whether a predetermined criteria has been satisfied indicating that the additional received query is likely a refinement query to the initial source query (act 508).

If the criteria has been satisfied, search engine 122 next determines whether the initial source query, including its associated user profile or country/region interface variables, has been previously received (act 510). If not, an association is created between the received refinement query and the initially received source query based on the received source query, refinement query, and user profile or country/region interface variables (act 512). Next, a counter associated with the refinement query is initialized to 1 indicating a first instance of a source query and subsequent refinement query (act 514). Next, the associated source and refinement queries are stored in database 110 for subsequent usage described below (act 516).

If the initial source query had been previously received, search engine 122 then determines whether the refinement query has been previously received as a refinement to the initial source query having matching user profile and/or country interface variables (act 518). If so, a counter associated with the refinement query is incremented to indicate the additionally received refinement query (act 520). If the refinement query has not been previously received as a refinement to the initial source query the counter associated with the refinement query is initialized to 1 indicating a first instance of a received refinement query (act 522). The process then returns to act 506 for receipt of the next refinement query (if any). By basing stored associations on user profile and/or country interface variables, subsequent distinctions to refinement data may be made which enable data more specific to a source query to be used in generating refinement suggestions.

FIG. 6 is a flow diagram illustrating one exemplary implementation of processing for characterizing refinement data described briefly above with respect to act 302. The characterization process may begin by performing searches for each refinement query collected in act 300 and more specifically illustrated in FIG. 4 to generate refinement query result sets including one or more documents (act 600). As described above with respect to FIG. 5, the received refinement queries may be stored based on user profile and country/region variables to facilitate additional specificity in generating refinement suggestions. Next, for each document returned in a refinement query result set, a weight is identified for each term appearing in the document (act 602). Term weights may be calculated using a number of possible techniques. One exemplary technique for generating weights bases the weights on the number of occurrences of each term within each returned document. That is, for a term occurring three times within a returned document, a weight of 3 may be assigned. More advanced weighting techniques, such as a weight based on the inverse document frequency (idf), may also be used.

The inverse document frequency (idf) of a term may be defined as a function of the number f of documents in the collection in which the term occurs and the number J of documents in the collection. In the context of a web search engine, the collection may refer to the set of web documents (e.g., web pages) indexed by the search engine. More specifically, the idf may be defined as log.

( J f + 1 ) .
Higher idf values indicate that a term is relatively more important than a term with a lower idf value. For a web search engine, J may be on the order of 1 billion documents.

If tf is the frequency (i.e., the number of occurrences) of the term in a document, then w(tf) may be the tf weight of the term in that document. More specifically, w(tf) may be just the tf itself, or 1+log(tf), or tf/(1+tf), or any other formulation. The weight of a term for any particular document of the M documents may be defined as:
w(tf)·idf,  (1)

With this weighting technique, terms used multiple times or in multiple documents are given more weight than terms used once. Also, terms that are relatively less common are given more weight.

Following term weighting, a term vector is generated for each document returned in act 600 (act 604). In accordance with one implementation consistent with principles of the invention, term vectors may be defined as normalized vectors projected into multi-dimensional space, with each dimension corresponding to a distinct term found in a document returned in a result set associated with one of the refinement queries. Terms may be individual words or word combination. The length of each term vector in each dimension equals the sum of the weights of the corresponding term in each returned document.

Next, the term vectors are normalized (act 606). In one embodiment this may be a L2 norm, however, any suitable normalization methodology, such as an L1 norm may also be implemented. In one exemplary embodiment, the term vectors may be length normalized to a length of one, although other normalizations are possible, as would be recognized by one skilled in the art.

In accordance with one implementation consistent with principles of the invention, the generation of term vectors may be limited to a predetermined number of search results within each result set, for example, the top 10 or top 50 results for each query. In this manner, only the most relevant results are used to represent each query.

Once normalized, a composite term vector is generated for each result set returned in act 600 (act 608). In one implementation consistent with principles of the invention, the composite term vector may be generated by adding the individual term-vectors from all the documents in the result set for the query. This may be accomplished, for example, by summing the weights from identical terms in the each document term vector. These weights from the various results are not necessarily summed equally. Rather, each document term vector may be scaled according to the relevancy of that result to the query, as estimated by search engine 122. This enables the term vectors of highly relevant results to contribute more to the composite term vector for the query than the term vectors of less relevant results. In one implementation consistent with principles of the invention, such scaling can be accomplished by multiplying the weights in the term vector in each result by the search engine's relevancy score for the respective document. It should be understood that the specifics of generating a relevancy score may include any number of methodologies. In an alternative embodiment, the weights of each result may be divided by that result's position a result set ranking. For example, weights for terms within the first result would be divided by one, weights for terms within the second result would be divided by two, etc.

In one implementation consistent with principles of the invention, once the composite term vector has been generated it may be compressed and pruned by excluding those terms that have a low relative weight within the term vector. This removes what is known statistically as the “tail” of the term vector distribution.

FIG. 7 is a flow diagram illustrating another exemplary implementation of processing for characterizing refinement data described briefly above with respect to act 302. The characterization process may begin by performing searches for each refinement query collected in act 300 to generate refinement query result sets including one or more documents (act 700). Next, for each search document returned in a refinement query result set, a listing of discrete search queries is identified which reference the document (act 702). For example, although document X may be returned in response to source query Y, it may also be returned in response to queries Q-U. The terms within each referencing query may also be used to characterize the collected refinement data.

Following identification of each search query referencing each returned document, a term vector is created for each document based upon the terms in the identified queries which reference the document (act 704). As with the embodiment of FIG. 5, term vectors may be defined as normalized vectors projected into multi-dimensional space, with each dimension corresponding to a distinct term found in a referencing query. Terms may be individual words or word combinations. The length of each term vector in each dimension equals the sum of the weights of the corresponding term in the referencing queries. Query term weighting may be performed in a similar manner as that described above with respect to FIG. 5.

Next, the term vectors are normalized (act 706) and a composite term vector is generated for each result set returned in act 700 (act 708). As above, with respect to FIG. 5, the composite term vector may be compressed and pruned by excluding those terms that have a low relative weight within the term vector.

FIG. 8 is a flow diagram illustrating one exemplary embodiment of processing for clustering refinement queries based on the composite term vectors generated in act 302 and described in FIGS. 6-7 above. Initially, upon generation of the composite term vectors for each refinement query identified in act 300, clusters are formed based on the distances (e.g., Euclidian distances) of each composite term vector from each other term vector (act 800). The generated clusters are then ranked based on a count total for each refinement query in the cluster (act 802). The highest ranking clusters are then selected as potential refinement clusters (act 804). In one exemplary implementation, the potential refinement clusters are selected based on a predefined threshold value, although other cluster selection criteria are possible, as would be recognized by one skilled in the art. Additionally, in one implementation consistent with principles of the invention, the clusters are formed using a hierarchical agglomerative clustering algorithm, although other known forms of clustering could also be applied, as would be recognized by one skilled in the art. By clustering the identified refinement queries, several distinct groups (clusters) of queries are produced that are topically related. Additionally, the clustering process also identifies clusters that, when compared to each other, are topically/semantically different.

Different clustering techniques may also be used to cluster the representations of the refinement queries. For example, “group average” and “single link” cluster distance metrics may be implemented. Moreover, alternative clustering algorithms may also be implemented, including k-means clustering, spectral clustering, and clustering based on pairwise document compressibility. Multiple clustering methods may be combined in multiple stages: for example, by running hierarchical agglomerative clustering and then running k-means clustering on the output and number of clusters generated by the hierarchical agglomerative clustering.

FIG. 9 is a flow diagram illustrating one exemplary implementation of processing for selecting representative refinements for the query clusters generated in act 304 and described more specifically in FIG. 8 above. As described above, each generated cluster potentially contains many refinements that are very closely related. Additionally, as described above with respect to FIG. 4, each refinement query is associated with a count, which is proportional to the probability that the user would refine the original query to that specific refinement. As discussed above with respect to FIG. 3, one exemplary implementation for selecting representative refinements for the query clusters includes choosing the refinement from each cluster having the maximum count. However, some clusters may comprise looser collections of queries than others. In these instances, the overlap of the queries isn't as tight, so a refinement that is closest in meaning to the majority of the queries in the cluster may be a better representative query than a refinement that simply has the highest probability of occurring as a refinement of the original query.

In order to account for this possibility, a centroid may be computed for each cluster (act 900). Each centroid represents the weighted center of the term vectors for each cluster, as a normalized sum of the product of the term vectors for each refinement query within the cluster and a relevance score assigned to the search documents generated in response to the refinement queries. Other approaches to computing centroids could also be used, including using unweighted values and by varying the forms of weighting and averaging, as would be recognized by one skilled in the art. Once cluster centroids have been generated, a score is computed for each refinement query in a cluster (act 902). In one implementation, the score may be based on the refinement query's count divided by the distance of its associated term vector from the centroid of the query cluster term vector. The refinement query in each cluster having the highest score is then selected as the representative refinement query (act 904).

In one alternative embodiment, various statistical measurements for the “center” or “average” of the cluster may be implemented rather than the cluster centroid defined above. More specifically, a cluster medioid may be implemented. A cluster medioid may be defined as the median of the values along all the dimensions of the term vector. Alternatively, the prior probability for each refinement query in the cluster may be combined with that refinement query's distance from the centroid using a function other than division, while still maintaining the same effect of increasing the score of a refinement that is close to the center (and more representative) of the cluster. Furthermore, a different distance function may be implemented to calculate the distance of the refinement query from the cluster center, rather than Euclidian distance as described above.

FIGS. 10 and 11 are flow diagrams illustrating exemplary implementations of processing for scoring and ranking the clusters and their representative refinement query as briefly described above in act 308. Ranking and scoring of the clusters is then used to decide which refinement queries should be displayed to the user, since even after the selecting a representative refinement query for each cluster, there may exist more refinement queries than it is possible or useful to display.

In one exemplary implementation consistent with principles of the invention, refinement query counts for all refinement queries within a cluster are identified (act 1000). Next, a cluster score is generated by summing the identified refinement query counts (act 1002). In this embodiment, the resultant number is proportional to the total probability that after typing the original query, the user would manually refine the source query to a query in that query cluster. All clusters generated may then be ranked according to their respective cluster scores (act 1004). Representative queries from a predetermined number of clusters may then be selected for display to users based on their respective rankings (act 1006).

In the implementation of FIG. 11, refinement query counts for all refinement queries within a cluster are also initially identified (act 1100). Next, a cluster score is generating by summing the identified refinement query counts and dividing by the cluster's compactness (act 1102). In one implementation consistent with principles of the invention, compactness may be defined as the standard deviation of the Euclidian distance of each refinement query term vector in the cluster from the cluster centroid. In practice, the standard deviation of refinement queries in the cluster from the centroid will be low if the cluster is a very tight cluster. Conversely, if the cluster is extremely disparate, the standard deviation will be quite high. Consequently, by dividing the score by the standard deviation, the score for clusters that are extremely tight around their centroids is elevated. Accordingly, the likelihood is increased that the higher scoring clusters have a tight coherent meaning, and are therefore more beneficial to display to users. Conversely, lower scoring clusters are more likely to contain a number of queries having high counts but which are ill-defined. The clusters are then ranked according to their respective cluster scores (act 1104). Representative queries from a predetermined number of clusters may then be selected for display to users based on their respective rankings (act 1106).

Although the standard deviation has been described above to define the compactness of a cluster, alternative methods for measuring or defining the cohesiveness or compactness of a cluster may also be implemented in accordance with principles of the invention. Additionally, prior probabilities of the refinement queries may be combined in a different fashion, so long as in general more refinements with greater probabilities yields a higher cluster score. In addition, a function may be implemented that integrates the distances of the queries in the cluster from the center and the individual queries' importance. Additionally, a count (or probability) divided by its distance from the cluster center may be computed for each refinement query in the cluster. The sum of these values for all refinement queries may then yield a similar cluster importance metric. It still has the effect that disparate clusters are downweighted and that clusters with queries with high counts are, in general, upweighted.

It should be understood that all of the above steps described with respect to FIGS. 3-11 may be performed online as a source query comes into the search engine. However, the steps may also be performed as off-line processing and the selected refinements for a given query stored in a database. In this implementation, upon receipt of a source query, selected refinements may be looked up from the database and presented to the user.

Query Refinement Example

By way of example, a user 118 might submit a search query, which includes the individual word, “bikes.” This query is ambiguous because the user could have meant either bicycles or motor bikes, or various in-between variants of an unpowered or powered two wheeled vehicle. Table 1 is a listing of one example of refinement queries which may have been received following an initial source query of “bikes”. Table 1 includes indicating the refinement queries received and a count representing the number of times the particular refinement was received within a predetermined timeframe (e.g., 1 day).

TABLE 1
Refinement Query Count
motorbikes 22
motorcycles 14
bmx bikes 12
mountain bikes 8
motor bikes 8
lowrider bikes 6
bicycles 5
bikes chicks 5
dirt bikes 4
cars 4
croch rockets 4
honda 4
bicicletas 4
como beijar 4
baracuda bikes 3
bikes bmx 3
bikes vancouver 3
bicycle 3
diodon holocanthus 3
name brand rollerblades 3
the history of the bicycle 3
kona bikes 3
sport bikes 3
kona 3
super motard 3
customized bikes 3
superbikes 3
bike sales 3
downhill biking 3
megamo 3
remote control plans 3
merlin 3
indianbikes 2
biker 2
burley bikes 2
city bikes 2
fastest bike in the world 2
costum bikes 2
mountain bikes parts 2
mopeds 2
bikes wheels 2
cycles 2
bikes movies 2
bycicles 2

As shown, there is a great deal of topical overlap in many of these refinements listed in Table 1. For other examples, there may even be more distinct overlap (in particular with celebrity-based queries where there is typically at least 10 variations on “<celebrity name> photos” in the top 50 user refinements. Furthermore, user entered query refinements in general also tend to be noisy, particularly at the lower count portion of the list. As described above, refinements below a predetermined threshold may be removed from the tail of the distribution, since they are unlikely to add anything useful in accurate generating refinement queries. In one implementation, the system may use the top-N refinements (where N is 50 to 100), and also prune out refinements with extremely low counts.

Once identified, each refinement query in Table 1 is characterized by issuing the refinement query and analyzing the returned results set. So, for example, the first refinement, “motorbikes” is issued and the result set retrieved. In this instance, the result set is composed primarily of pages about motorcycles. For each of the pages or documents returned, the terms or a subset of the terms on the page are then used to create term vectors. The term vectors for all the top-N results are summed, possibly with weighting the different results, to obtain a composite term vector representative of the refinement query as whole.

Because most of the pages returned in response to the first refinement query are about motorcycles, high-weight terms in the resultant vector for the query “motorbikes” will contain terms like “motorbikes”, “motor”, “engine”, “gasoline”, “mph”, etc. Similarly, the second query, “motorcycles”, is issued. This also returns many pages about motorcycles, and so after collecting the result set and creating a term vector for the query, the resultant vector will have many high-weight terms that are the same as the high-weight terms for “motorbikes”. However, for the third query in Table 1 “bmx bikes”, results turn out slightly different. The results for bmx bikes are almost all about small pedal bicycles that are used to perform tricks. Thus, when the result set vectors are created and summed to form the vector for the query, the high-weight terms will be different from those for “motorbikes” and “motorcycles”. They will include terms like “bicycle”, “pedaling”, “pedal”, “stunts”, etc.

After the composite term vectors are generated for each refinement query, the queries are clustered, which iteratively collapses the most similar composite term vectors into similar clusters. In one implementation, a threshold may be heuristically set to indicate where the clustering is to stop. Beyond this threshold the clusters may be deemed too dissimilar to be combined. In one embodiment, the threshold may be set to be the same for all clusterings. Table 2 is a listing of the top 5 clusters generated from a fairly aggressive clustering of the refinement queries of Table 1.

TABLE 2
Cluster 1: total weight: 38
motorbikes 22
motor bikes 8
bike sales 3
sport bikes 3
bikes motor 2
Cluster 2: total weight: 24
mountain bikes 8
bicycles 5
bicycle 3
bike 2
road bikes 2
mountain bikes 2
Cluster 3: total weight: 18
motorcycles 14
honda 4
Cluster 4: total weight: 17
bmx bikes 12
bikes bmx 3
bikes wheels 2
Cluster 5: total weight: 8
lowrider bikes 6
lowrider 2

The remaining query clusters tend to be low-weight clusters of only one or two queries and can be removed from the analysis. As shown in Table 2, the generated clusters tend to group together topically similar refinement queries. After the clusters are generated, the best refinement from each cluster is then selected. For the above 5 clusters, if we simply choose the highest-weight (highest prior probability) refinement query, these would be: motorbikes (38); mountain bikes (24); motorcycles (18); bmx bikes (17); and lowrider bikes (8).

Note that, in some instances, highest-weight scoring doesn't always pick the most representative refinement query for each cluster. For example, for the general bicycle cluster (cluster 2), the highest-weight refinement is “mountain bikes”, but “bicycles” is a close second. This is why in the description of the invention given above, it is indicated that cluster weights may be scored by dividing them by the query's distance from the centroid of the cluster. In the present example, “mountain bikes” is farther from the centroid than “bicycles” since the cluster is more about bicycles in general. This has the effect of boosting the more general query “bicycles”, even though it has a slightly lower weight. In this alternate system “bicycles” would be selected as the most representative refinement query rather than “mountain bikes”.

Once the query refinement clusters are named, they are ranked based on the total weight of each cluster, not the weight of the selected query. In one implementation, this involves ranking the representative queries for each cluster based on the total weight of the cluster in which they are contained. As described above, in an alternative embodiment the cohesiveness of the cluster may be accounted for by dividing the total weight by the standard deviation from the centroid. The examples provided here illustrate only a few of the many techniques known to those of ordinary skill in the art that may be employed to obtain related words for a term.

Conclusion

Systems and methods consistent with the principles of the invention may provide refinement query suggestions in response to a received source query.

The foregoing description of preferred embodiments of the present invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention.

For example, while series of acts have been described with regard to FIGS. 3-10, the order of the acts may be modified in other implementations consistent with the principles of the invention. Further, non-dependent acts may be performed in parallel.

It will be apparent to one of ordinary skill in the art that aspects of the invention, as described above, may be implemented in many different forms of software, firmware, and hardware in the implementations illustrated in the figures. The actual software code or specialized control hardware used to implement aspects consistent with the principles of the invention is not limiting of the present invention. Thus, the operation and behavior of the aspects were described without reference to the specific software code—it being understood that one of ordinary skill in the art would be able to design software and control hardware to implement the aspects based on the description herein.

No element, act, or instruction used in the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise.

Claims (34)

1. A method performed by one or more server devices, the method comprising:
receiving, by the one or more server devices, a first search query;
receiving, by the one or more server devices, after receiving the first search query, a second search query that is different from the first search query;
identifying, by one or more processors of the one or more server devices, based on a set of predetermined criteria, that the second search query is a refinement query of the first search query;
collecting, by the one or more processors of the one or more server devices, a plurality of refinement query suggestions for the first search query, where the plurality of refinement query suggestions includes the second search query;
counting, by the one or more processors of the one or more server devices, a quantity of times that each refinement query suggestion, of the plurality of refinement query suggestions, was previously received and identified, by the one or more server devices, as a refinement of a query that matches the first search query;
storing the plurality of refinement query suggestions in a memory associated with the one or more server devices;
clustering, by the one or more processors of the one or more server devices, the stored plurality of refinement query suggestions to form a plurality of clusters;
identifying, by the one or more processors of the one or more server devices, a particular cluster, of the formed plurality of clusters, based on the plurality of refinement query suggestions within the particular cluster;
selecting, by the one or more processors of the one or more server devices, from within the identified particular cluster, at least one potential refinement query suggestion, where the selecting includes:
identifying that a quantity of times, that the at least one potential refinement query suggestion was received and identified as a refinement of the first search query, is higher than quantities of times that other refinement query suggestions, within the identified particular cluster, were received and identified as refinements of the first search query; and
transmitting, by a communication interface of the one or more server devices, the selected at least one potential refinement query suggestion to a client associated with a user, the transmitting occurring in response to receiving, from the user, a third search query that matches the first search query.
2. The method of claim 1, further comprising:
determining whether the second search query is received within a predetermined time period from receipt of the first search query.
3. The method of claim 2, where the predetermined time period is approximately two minutes.
4. The method of claim 1, further comprising:
identifying at least one of a user profile or a country/region information associated with the first search query; and
associating the second search query with the first search query based on the identified at least one of the user profile or the country/region information.
5. The method of claim 4, where the at least one of the user profile or the country/region information includes information identifying a language.
6. The method of claim 1, further comprising:
incrementing an association counter for a second search query/first search query association by one (1) upon receiving the second search query; and
storing the second search query/first search query association and the incremented association counter.
7. The method of claim 6, where the selecting the at least one potential refinement query suggestion further comprises:
identifying the at least one potential refinement query suggestion within the clustered stored plurality of refinement query suggestions as a refinement query suggestion having a maximum association counter value.
8. The method of claim 1, further comprising:
performing searches for each of the plurality of refinement query suggestions to identify at least one document based on the first search query;
identifying terms within the identified at least one document;
assigning weights to the identified terms; and
generating a term vector for the identified at least one document based on the assigned weights to the identified terms,
where the clustering is based on the generated term vector.
9. The method of claim 8, further comprising:
normalizing the generated term vector for the identified at least one document.
10. The method of claim 8, further comprising:
generating a composite term vector combining generated term vectors for each refinement query suggestion associated with the first search query.
11. The method of claim 1, further comprising:
generating a plurality of term vectors representing each of the plurality of refinement query suggestions associated with the first search query.
12. The method of claim 11, where the clustering the stored plurality of refinement query suggestions further comprises:
generating clusters based on the generated plurality of term vectors.
13. The method of claim 12, where the generating the clusters further comprises:
generating clusters based on distances between each term vector from each other term vector.
14. The method of claim 12, where the selecting the at least one potential refinement query suggestion further comprises:
generating a centroid for each generated cluster;
scoring each refinement query suggestion in each generated cluster based on the distance of each term vector from the generated centroid; and
selecting the at least one potential refinement query suggestion as the refinement query suggestion having a highest score.
15. The method of claim 1, further comprising:
scoring the selected at least one potential refinement query suggestion; and
ranking the scored selected at least one potential refinement query suggestion with regard to other potential refinement query suggestions.
16. The method of claim 1, further comprising:
assigning a score to each clustered stored plurality of refinement query suggestions based on the identified quantity of times that each refinement query suggestion, within each clustered stored plurality of refinement query suggestions, was received as the refinement of the query that matches the first search query.
17. The method of claim 16, where the identifying the particular cluster includes:
identifying a cluster, of the formed plurality of clusters, having a highest score, of the assigned score to each clustered stored plurality of refinement query suggestions, as the particular cluster.
18. A system, comprising:
one or more processors; and
a memory to store instructions that, when executed by the one or more processors, cause the one or more processors to implement:
means for receiving, from one or more client devices, a plurality of refinement query suggestions that are refinements of a particular search query, where each refinement query suggestion, of the plurality of refinement query suggestions, has been received as a refinement of the particular search query a particular quantity of times;
means for clustering the plurality of refinement query suggestions to form a plurality of clusters;
means for identifying, for each refinement query suggestion, of the plurality of refinement query suggestions, the particular quantity of times that the refinement query suggestion was received as the refinement of the particular search query;
means for identifying, for each clustered plurality of refinement query suggestions, of the formed plurality of clusters, a particular refinement query suggestion that corresponds to a highest identified quantity, from the identified particular quantity of times that the refinement query suggestion, of refinement query suggestions of the clustered plurality of refinement query suggestions, has been received as the refinement of the particular search query; and
means for presenting the identified particular refinement query suggestion from each of the clustered plurality of refinement query suggestions in response to receiving the particular search query.
19. The system of claim 18, further comprising:
means for generating term vectors representative of each of the refinement query suggestions.
20. The system of claim 19, where the means for generating the term vectors further comprises:
means for performing a search to identify a document based on the identified particular refinement query suggestion; and
means for generating the term vectors representing the document.
21. The system of claim 19, further comprising:
means for generating term vectors for each of the refinement query suggestions included within the clustered plurality of refinement query suggestions; and
where the means for the clustering includes means for clustering based on distances between each of the generated term vectors and a common origin.
22. The system of claim 18, further comprising:
means for calculating a score for each clustered plurality of refinement query suggestions, of the formed plurality of clusters, based on the identified particular quantity of times that the refinement query suggestion, within the clustered plurality of refinement query suggestions, was received as the refinement of the received particular search query, and
where the means for the presenting includes means for presenting the identified particular refinement query suggestion in an order that is based on the calculated score for each clustered plurality of refinement query suggestions, of the formed plurality of clusters.
23. The system of claim 18, further comprising:
means for identifying refinement query suggestions, of the clustered plurality of refinement query suggestions, that are topically related to each other, where the formed plurality of clusters are each related to a respective topic.
24. The system of claim 23, where each refinement query suggestion, within a particular clustered plurality of refinement query suggestions, is related to the respective topic with which the particular clustered plurality of refinement query suggestions is related.
25. The system of claim 18, further comprising:
means for identifying that each of the refinement query suggestions are refinements of the particular search query, where the means for the identifying the particular refinement query suggestion as the refinement of the particular search query include:
means for receiving the particular search query;
means for identifying that the particular refinement query suggestion has been received within a predetermined amount of time after receiving the particular search query; and
means for identifying that the particular refinement query suggestion is the refinement of the particular search query upon identifying that the particular refinement query suggestion has been received within the predetermined amount of time after receiving the particular search query.
26. A system, comprising:
a memory to store instructions; and
a processor to execute the stored instructions to:
receive, from one or more client devices, a plurality of refinement query suggestions that are refinements of a particular search query, where each refinement query suggestion, of the plurality of refinement query suggestions, has been received as a refinement of the particular search query a particular quantity of times;
cluster the received plurality of refinement query suggestions to form a plurality of clusters;
identify, for each refinement query suggestion, of the plurality of refinement query suggestions, the particular quantity of times that the refinement query suggestion was received as a refinement of the particular search query;
identify, for each clustered received plurality of refinement query suggestions, of the formed plurality of clusters, a particular refinement query suggestion that corresponds to a highest identified quantity, from the identified particular quantity of times that the refinement query suggestion, of the refinement query suggestions of the cluster, has been received as the refinement of the particular search query; and
present the identified particular refinement query suggestion from each of the clusters in response to receiving the particular search query.
27. The system of claim 26, where the processor is further configured to:
obtain search results in response to receiving the particular search query; and
where, when presenting the identified particular refinement query suggestion from each of the clusters, the processor is configured to:
present the identified particular refinement query suggestion from each of the clusters on a same graphical user interface as search results.
28. The system of claim 26, where when receiving the plurality of refinement query suggestions, the processor is configured to:
receive at least one refinement query associated with the particular search query, and
store information indicating an association between the received at least one refinement query and the particular search query,
where the information includes an indication of how often the received at least one refinement query is received in association with the particular search query.
29. A method performed by a server, the method comprising:
receiving, by of one or more processors of the server, historical query data, to identify refinement queries, that are refinements of a particular search query, where the refinement queries have each been received as a refinement of the particular search query a particular quantity of times;
determining, for each one of the refinement queries and by the one or more processors of the server, a quantity of times that the refinement query was received as the refinement of the particular search query;
clustering, by the one or more processors of the server, a plurality of refinement queries to form a plurality of clusters;
receiving, by the server, a query from one or more client devices, where the received query matches the particular search query;
identifying, by the one or more processors of the server a refinement query from each of at least one of the clusters from the formed plurality of clusters, where the identifying includes:
identifying, from a particular one of the clusters, a particular refinement query, associated with the particular one of the clusters, that corresponds to a highest determined quantity, from the determined quantity of times that the particular refinement query, of refinement queries associated with the particular one of the clusters, has been received as the refinement of the particular search query; and
sending, in response to the received query and by the server, the identified particular refinement query, from each of the at least one of the clusters from the formed plurality of clusters, to the one or more client devices.
30. The method of claim 29, further comprising:
obtaining search results in response to receiving the query from the one or more client devices, and
where the presenting sending includes:
presenting a graphical user interface, the graphical user interface including the obtained search results and the identified particular refinement query from each of the clusters from the formed plurality of clusters.
31. The method of claim 29, further comprising:
storing a plurality of query/refinement query associations prior to receiving the query from the one or more client devices.
32. A non-transitory computer-readable memory device incorporating one or more instructions, comprising:
one or more instructions to receive, from one or more client devices, a plurality of refinement query suggestions that are refinements of a particular search query, where each refinement query suggestion, of the plurality of refinement query suggestions, has been received as a refinement of the particular search query a particular quantity of times;
one or more instructions to cluster the plurality of refinement query suggestions to form a plurality of clusters;
one or more instructions to identify, for each refinement query suggestion, of the plurality of refinement query suggestions, the particular quantity of times that the refinement query suggestion was received as the refinement of the particular search query;
one or more instructions to identify, for each clustered plurality of refinement query suggestions, of the formed plurality of clusters, a particular refinement query suggestion that corresponds to a highest identified quantity, from the identified particular quantity of times that the refinement query suggestion, of the clustered plurality of refinement query suggestions, has been received as the refinement of the particular search query; and
one or more instructions to present the identified particular refinement query suggestion from each of the clustered plurality of refinement query suggestions in response to receiving the particular search query.
33. The non-transitory computer-readable memory device of claim 32, further comprising:
one or more instructions to receive an initial query;
one or more instructions to receive a second query following receipt of the initial query;
one or more instructions to determine whether the received second query is a refinement of the received initial query; and
one or more instructions to associate the received second query with the received initial query upon determining that the received second query is the refinement of the received initial query.
34. The non-transitory computer-readable memory device of claim 33, further comprising:
one or more instructions to perform searches for each identified particular refinement query suggestion, associated with the particular search query, to identify at least one document based on each identified particular refinement query suggestion;
one or more instructions to identify terms within the identified at least one document;
one or more instructions to generate weights for the identified terms; and
one or more instructions to generate a term vector for the identified at least one document based on the generated weights.
US10/953,117 2004-09-30 2004-09-30 Systems and methods for providing search query refinements Active 2024-11-14 US8065316B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10/953,117 US8065316B1 (en) 2004-09-30 2004-09-30 Systems and methods for providing search query refinements

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US10/953,117 US8065316B1 (en) 2004-09-30 2004-09-30 Systems and methods for providing search query refinements
US13/270,928 US8504584B1 (en) 2004-09-30 2011-10-11 Systems and methods for providing search query refinements
US13/955,119 US9495443B1 (en) 2004-09-30 2013-07-31 Systems and methods for providing search query refinements
US15/348,314 US10223439B1 (en) 2004-09-30 2016-11-10 Systems and methods for providing search query refinements

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/270,928 Continuation US8504584B1 (en) 2004-09-30 2011-10-11 Systems and methods for providing search query refinements

Publications (1)

Publication Number Publication Date
US8065316B1 true US8065316B1 (en) 2011-11-22

Family

ID=44936893

Family Applications (4)

Application Number Title Priority Date Filing Date
US10/953,117 Active 2024-11-14 US8065316B1 (en) 2004-09-30 2004-09-30 Systems and methods for providing search query refinements
US13/270,928 Active US8504584B1 (en) 2004-09-30 2011-10-11 Systems and methods for providing search query refinements
US13/955,119 Active 2026-02-06 US9495443B1 (en) 2004-09-30 2013-07-31 Systems and methods for providing search query refinements
US15/348,314 Active 2024-10-23 US10223439B1 (en) 2004-09-30 2016-11-10 Systems and methods for providing search query refinements

Family Applications After (3)

Application Number Title Priority Date Filing Date
US13/270,928 Active US8504584B1 (en) 2004-09-30 2011-10-11 Systems and methods for providing search query refinements
US13/955,119 Active 2026-02-06 US9495443B1 (en) 2004-09-30 2013-07-31 Systems and methods for providing search query refinements
US15/348,314 Active 2024-10-23 US10223439B1 (en) 2004-09-30 2016-11-10 Systems and methods for providing search query refinements

Country Status (1)

Country Link
US (4) US8065316B1 (en)

Cited By (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090049020A1 (en) * 2006-04-06 2009-02-19 Nhn Corporation System and method for providing personalized recommended word and computer readable recording medium recording program for implementing the method
US20110295897A1 (en) * 2010-06-01 2011-12-01 Microsoft Corporation Query correction probability based on query-correction pairs
US20120047159A1 (en) * 2010-08-19 2012-02-23 Fuji Xerox Co., Ltd. Speculative query expansion for relevance feedback
US20120089627A1 (en) * 2006-10-31 2012-04-12 Aol Inc. System and methods for performing machine-implemented tasks
US20120239653A1 (en) * 2007-06-28 2012-09-20 Microsoft Corporation Machine Assisted Query Formulation
US8412727B1 (en) * 2009-06-05 2013-04-02 Google Inc. Generating query refinements from user preference data
US8504584B1 (en) 2004-09-30 2013-08-06 Google Inc. Systems and methods for providing search query refinements
US8583675B1 (en) 2009-08-28 2013-11-12 Google Inc. Providing result-based query suggestions
US20140074812A1 (en) * 2012-06-25 2014-03-13 Rediff.Com India Limited Method and apparatus for generating a suggestion list
US8676803B1 (en) 2009-11-04 2014-03-18 Google Inc. Clustering images
US8756218B1 (en) * 2011-08-16 2014-06-17 Google Inc. Query classification based on search engine results
US8862609B2 (en) 2012-09-28 2014-10-14 International Business Machines Corporation Expanding high level queries
WO2014193698A1 (en) * 2013-05-31 2014-12-04 Google Inc. Query suggestions based on search data
US8909627B1 (en) 2011-11-30 2014-12-09 Google Inc. Fake skip evaluation of synonym rules
US20140372410A1 (en) * 2013-06-14 2014-12-18 Microsoft Corporation Upper bounds for matching
US8930350B1 (en) 2009-03-23 2015-01-06 Google Inc. Autocompletion using previously submitted query data
US8959103B1 (en) 2012-05-25 2015-02-17 Google Inc. Click or skip evaluation of reordering rules
US8965882B1 (en) 2011-07-13 2015-02-24 Google Inc. Click or skip evaluation of synonym rules
US8965875B1 (en) 2012-01-03 2015-02-24 Google Inc. Removing substitution rules based on user interactions
CN104428762A (en) * 2012-08-17 2015-03-18 英特尔公司 Traversing data utilizing data relationships
US20150120711A1 (en) * 2012-05-15 2015-04-30 Microsoft Technology Licensing, Llc Scenario based insights into structure data
US9043248B2 (en) 2012-03-29 2015-05-26 International Business Machines Corporation Learning rewrite rules for search database systems using query logs
US9069750B2 (en) 2006-10-10 2015-06-30 Abbyy Infopoisk Llc Method and system for semantic searching of natural language texts
US20150186394A1 (en) * 2013-12-30 2015-07-02 Yahoo! Inc. Query suggestions
US9075864B2 (en) 2006-10-10 2015-07-07 Abbyy Infopoisk Llc Method and system for semantic searching using syntactic and semantic analysis
US9092504B2 (en) 2012-04-09 2015-07-28 Vivek Ventures, LLC Clustered information processing and searching with structured-unstructured database bridge
US9098489B2 (en) 2006-10-10 2015-08-04 Abbyy Infopoisk Llc Method and system for semantic searching
US9116952B1 (en) 2013-05-31 2015-08-25 Google Inc. Query refinements using search data
US9135307B1 (en) * 2012-12-27 2015-09-15 Google Inc. Selectively generating alternative queries
US9141672B1 (en) 2012-01-25 2015-09-22 Google Inc. Click or skip evaluation of query term optionalization rule
US9146966B1 (en) 2012-10-04 2015-09-29 Google Inc. Click or skip evaluation of proximity rules
US9152698B1 (en) 2012-01-03 2015-10-06 Google Inc. Substitute term identification based on over-represented terms identification
US9189482B2 (en) 2012-10-10 2015-11-17 Abbyy Infopoisk Llc Similar document search
US9223872B2 (en) 2013-06-13 2015-12-29 Microsoft Technology Licensing, Llc Dynamic query resolution using accuracy profiles
US9336302B1 (en) 2012-07-20 2016-05-10 Zuci Realty Llc Insight and algorithmic clustering for automated synthesis
US9495358B2 (en) 2006-10-10 2016-11-15 Abbyy Infopoisk Llc Cross-language text clustering
US9858313B2 (en) 2011-12-22 2018-01-02 Excalibur Ip, Llc Method and system for generating query-related suggestions
US9892111B2 (en) 2006-10-10 2018-02-13 Abbyy Production Llc Method and device to estimate similarity between documents having multiple segments
EP3149729A4 (en) * 2014-03-27 2018-03-28 Yandex Europe AG Method and system for processing a voice-based user-input
US10229212B2 (en) 2016-04-08 2019-03-12 Microsoft Technology Licensing, Llc Identifying Abandonment Using Gesture Movement
US10318586B1 (en) 2014-08-19 2019-06-11 Google Llc Systems and methods for editing and replaying natural language queries

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8650031B1 (en) 2011-07-31 2014-02-11 Nuance Communications, Inc. Accuracy improvement of spoken queries transcription using co-occurrence information
US9594851B1 (en) * 2012-02-07 2017-03-14 Google Inc. Determining query suggestions
US9892198B2 (en) * 2012-06-07 2018-02-13 Oath Inc. Page personalization performed by an edge server
US9336211B1 (en) 2013-03-13 2016-05-10 Google Inc. Associating an entity with a search query
US20180060438A1 (en) * 2016-08-25 2018-03-01 Linkedin Corporation Prioritizing locations for people search
US20180060432A1 (en) * 2016-08-25 2018-03-01 Linkedln Corporation Prioritizing people search results

Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5790121A (en) * 1996-09-06 1998-08-04 Sklar; Peter Clustering user interface
US5907840A (en) * 1997-07-25 1999-05-25 Claritech Corporation Overlapping subdocuments in a vector space search process
US5940825A (en) * 1996-10-04 1999-08-17 International Business Machines Corporation Adaptive similarity searching in sequence databases
US6006225A (en) * 1998-06-15 1999-12-21 Amazon.Com Refining search queries by the suggestion of correlated terms from prior searches
US6014665A (en) * 1997-08-01 2000-01-11 Culliss; Gary Method for organizing information
US6026388A (en) * 1995-08-16 2000-02-15 Textwise, Llc User interface and other enhancements for natural language information retrieval system and method
US20020169764A1 (en) * 2001-05-09 2002-11-14 Robert Kincaid Domain specific knowledge-based metasearch system and methods of using
US6484162B1 (en) * 1999-06-29 2002-11-19 International Business Machines Corporation Labeling and describing search queries for reuse
US6564202B1 (en) * 1999-01-26 2003-05-13 Xerox Corporation System and method for visually representing the contents of a multiple data object cluster
US20030110181A1 (en) * 1999-01-26 2003-06-12 Hinrich Schuetze System and method for clustering data objects in a collection
US20030187837A1 (en) * 1997-08-01 2003-10-02 Ask Jeeves, Inc. Personalized search method
US6668251B1 (en) * 2000-11-01 2003-12-23 Tacit Knowledge Systems, Inc. Rendering discriminator members from an initial set of result data
US6671681B1 (en) * 2000-05-31 2003-12-30 International Business Machines Corporation System and technique for suggesting alternate query expressions based on prior user selections and their query strings
US6678679B1 (en) * 2000-10-10 2004-01-13 Science Applications International Corporation Method and system for facilitating the refinement of data queries
US20040019588A1 (en) * 2002-07-23 2004-01-29 Doganata Yurdaer N. Method and apparatus for search optimization based on generation of context focused queries
US6701309B1 (en) * 2000-04-21 2004-03-02 Lycos, Inc. Method and system for collecting related queries
US6772150B1 (en) * 1999-12-10 2004-08-03 Amazon.Com, Inc. Search query refinement using related search phrases
US20040186827A1 (en) * 2003-03-21 2004-09-23 Anick Peter G. Systems and methods for interactive search query refinement
US20040249808A1 (en) * 2003-06-06 2004-12-09 Microsoft Corporation Query expansion using query logs
US20050055341A1 (en) * 2003-09-05 2005-03-10 Paul Haahr System and method for providing search query refinements
US6882994B2 (en) * 2000-06-12 2005-04-19 Hitachi, Ltd. Method and system for querying database, as well as a recording medium for storing a database querying program
US20050216446A1 (en) * 2004-03-29 2005-09-29 Hall Karl E Technical process to deliver pre-populated search suggestions using the intelli-match search methodology
US20050234972A1 (en) * 2004-04-15 2005-10-20 Microsoft Corporation Reinforced clustering of multi-type data objects for search term suggestion
US20050278321A1 (en) * 2001-05-09 2005-12-15 Aditya Vailaya Systems, methods and computer readable media for performing a domain-specific metasearch, and visualizing search results therefrom
US7010521B2 (en) * 2002-05-13 2006-03-07 Netezza Corporation Optimized database appliance
US7028020B1 (en) * 2000-06-30 2006-04-11 Intel Corporation Interactive technique to automatically find and organize items similar to example items
US7051023B2 (en) * 2003-04-04 2006-05-23 Yahoo! Inc. Systems and methods for generating concept units from search queries
US7058622B1 (en) * 2001-12-26 2006-06-06 Tedesco Michael A Method, apparatus and system for screening database queries prior to submission to a database
US7092936B1 (en) * 2001-08-22 2006-08-15 Oracle International Corporation System and method for search and recommendation based on usage mining
US7739274B2 (en) * 2003-12-08 2010-06-15 Iac Search & Media, Inc. Methods and systems for providing a response to a query
US7814096B1 (en) * 2004-06-08 2010-10-12 Yahoo! Inc. Query based search engine
US7890526B1 (en) * 2003-12-30 2011-02-15 Microsoft Corporation Incremental query refinement

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5263167A (en) * 1991-11-22 1993-11-16 International Business Machines Corporation User interface for a relational database using a task object for defining search queries in response to a profile object which describes user proficiency
US5675788A (en) * 1995-09-15 1997-10-07 Infonautics Corp. Method and apparatus for generating a composite document on a selected topic from a plurality of information sources
US5864845A (en) * 1996-06-28 1999-01-26 Siemens Corporate Research, Inc. Facilitating world wide web searches utilizing a multiple search engine query clustering fusion strategy
US20030154072A1 (en) * 1998-03-31 2003-08-14 Scansoft, Inc., A Delaware Corporation Call analysis
US6144958A (en) * 1998-07-15 2000-11-07 Amazon.Com, Inc. System and method for correcting spelling errors in search queries
US6411950B1 (en) * 1998-11-30 2002-06-25 Compaq Information Technologies Group, Lp Dynamic query expansion
US6751606B1 (en) * 1998-12-23 2004-06-15 Microsoft Corporation System for enhancing a query interface
US9076448B2 (en) * 1999-11-12 2015-07-07 Nuance Communications, Inc. Distributed real time speech recognition system
US6560597B1 (en) * 2000-03-21 2003-05-06 International Business Machines Corporation Concept decomposition using clustering
US6574624B1 (en) * 2000-08-18 2003-06-03 International Business Machines Corporation Automatic topic identification and switch for natural language search of textual document collections
US7430717B1 (en) * 2000-09-26 2008-09-30 International Business Machines Corporation Method for adapting a K-means text clustering to emerging data
US20020103798A1 (en) * 2001-02-01 2002-08-01 Abrol Mani S. Adaptive document ranking method based on user behavior
US7096218B2 (en) * 2002-01-14 2006-08-22 International Business Machines Corporation Search refinement graphical user interface
US6829599B2 (en) * 2002-10-02 2004-12-07 Xerox Corporation System and method for improving answer relevance in meta-search engines
US7421660B2 (en) * 2003-02-04 2008-09-02 Cataphora, Inc. Method and apparatus to visually present discussions for data mining purposes
US7729913B1 (en) * 2003-03-18 2010-06-01 A9.Com, Inc. Generation and selection of voice recognition grammars for conducting database searches
US8082264B2 (en) * 2004-04-07 2011-12-20 Inquira, Inc. Automated scheme for identifying user intent in real-time
US8392453B2 (en) * 2004-06-25 2013-03-05 Google Inc. Nonstandard text entry
US20060036438A1 (en) * 2004-07-13 2006-02-16 Microsoft Corporation Efficient multimodal method to provide input to a computing device
US20060064411A1 (en) * 2004-09-22 2006-03-23 William Gross Search engine using user intent
US8065316B1 (en) 2004-09-30 2011-11-22 Google Inc. Systems and methods for providing search query refinements

Patent Citations (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6026388A (en) * 1995-08-16 2000-02-15 Textwise, Llc User interface and other enhancements for natural language information retrieval system and method
US5790121A (en) * 1996-09-06 1998-08-04 Sklar; Peter Clustering user interface
US5940825A (en) * 1996-10-04 1999-08-17 International Business Machines Corporation Adaptive similarity searching in sequence databases
US5907840A (en) * 1997-07-25 1999-05-25 Claritech Corporation Overlapping subdocuments in a vector space search process
US20030187837A1 (en) * 1997-08-01 2003-10-02 Ask Jeeves, Inc. Personalized search method
US6014665A (en) * 1997-08-01 2000-01-11 Culliss; Gary Method for organizing information
US6006225A (en) * 1998-06-15 1999-12-21 Amazon.Com Refining search queries by the suggestion of correlated terms from prior searches
US6169986B1 (en) * 1998-06-15 2001-01-02 Amazon.Com, Inc. System and method for refining search queries
US6564202B1 (en) * 1999-01-26 2003-05-13 Xerox Corporation System and method for visually representing the contents of a multiple data object cluster
US20030110181A1 (en) * 1999-01-26 2003-06-12 Hinrich Schuetze System and method for clustering data objects in a collection
US6484162B1 (en) * 1999-06-29 2002-11-19 International Business Machines Corporation Labeling and describing search queries for reuse
US6772150B1 (en) * 1999-12-10 2004-08-03 Amazon.Com, Inc. Search query refinement using related search phrases
US20070239671A1 (en) * 1999-12-10 2007-10-11 Whitman Ronald M Selection of search phrases to suggest to users in view of actions performed by prior users
US20040236736A1 (en) * 1999-12-10 2004-11-25 Whitman Ronald M. Selection of search phrases to suggest to users in view of actions performed by prior users
US6701309B1 (en) * 2000-04-21 2004-03-02 Lycos, Inc. Method and system for collecting related queries
US6671681B1 (en) * 2000-05-31 2003-12-30 International Business Machines Corporation System and technique for suggesting alternate query expressions based on prior user selections and their query strings
US6882994B2 (en) * 2000-06-12 2005-04-19 Hitachi, Ltd. Method and system for querying database, as well as a recording medium for storing a database querying program
US7028020B1 (en) * 2000-06-30 2006-04-11 Intel Corporation Interactive technique to automatically find and organize items similar to example items
US6678679B1 (en) * 2000-10-10 2004-01-13 Science Applications International Corporation Method and system for facilitating the refinement of data queries
US6668251B1 (en) * 2000-11-01 2003-12-23 Tacit Knowledge Systems, Inc. Rendering discriminator members from an initial set of result data
US7519605B2 (en) * 2001-05-09 2009-04-14 Agilent Technologies, Inc. Systems, methods and computer readable media for performing a domain-specific metasearch, and visualizing search results therefrom
US20020169764A1 (en) * 2001-05-09 2002-11-14 Robert Kincaid Domain specific knowledge-based metasearch system and methods of using
US20050278321A1 (en) * 2001-05-09 2005-12-15 Aditya Vailaya Systems, methods and computer readable media for performing a domain-specific metasearch, and visualizing search results therefrom
US7092936B1 (en) * 2001-08-22 2006-08-15 Oracle International Corporation System and method for search and recommendation based on usage mining
US7058622B1 (en) * 2001-12-26 2006-06-06 Tedesco Michael A Method, apparatus and system for screening database queries prior to submission to a database
US7010521B2 (en) * 2002-05-13 2006-03-07 Netezza Corporation Optimized database appliance
US20040019588A1 (en) * 2002-07-23 2004-01-29 Doganata Yurdaer N. Method and apparatus for search optimization based on generation of context focused queries
US20040186827A1 (en) * 2003-03-21 2004-09-23 Anick Peter G. Systems and methods for interactive search query refinement
US7051023B2 (en) * 2003-04-04 2006-05-23 Yahoo! Inc. Systems and methods for generating concept units from search queries
US20040249808A1 (en) * 2003-06-06 2004-12-09 Microsoft Corporation Query expansion using query logs
US20050055341A1 (en) * 2003-09-05 2005-03-10 Paul Haahr System and method for providing search query refinements
US7739274B2 (en) * 2003-12-08 2010-06-15 Iac Search & Media, Inc. Methods and systems for providing a response to a query
US7890526B1 (en) * 2003-12-30 2011-02-15 Microsoft Corporation Incremental query refinement
US20110087686A1 (en) * 2003-12-30 2011-04-14 Microsoft Corporation Incremental query refinement
US20050216446A1 (en) * 2004-03-29 2005-09-29 Hall Karl E Technical process to deliver pre-populated search suggestions using the intelli-match search methodology
US20050234972A1 (en) * 2004-04-15 2005-10-20 Microsoft Corporation Reinforced clustering of multi-type data objects for search term suggestion
US7814096B1 (en) * 2004-06-08 2010-10-12 Yahoo! Inc. Query based search engine

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Edle Rasmussen; "Chapter 16: Clustering Algorithms", University of Pittsburgh, Information Retrieval, 1992, pp. 419-441.
Falk Scholer et al., "Query Association for Effective Retrieval", School of Computer Science and Information Technology, RMIT University, Melbourne, Australia, 2002, pp. 324-331.
Lau, et al., "Patterns of Search: Analyzing and Modeling Web Query Refinement," 7th International Conference on User Modeling, Jun. 20-24, 1999. *

Cited By (66)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9495443B1 (en) 2004-09-30 2016-11-15 Google Inc. Systems and methods for providing search query refinements
US10223439B1 (en) 2004-09-30 2019-03-05 Google Llc Systems and methods for providing search query refinements
US8504584B1 (en) 2004-09-30 2013-08-06 Google Inc. Systems and methods for providing search query refinements
US20090049020A1 (en) * 2006-04-06 2009-02-19 Nhn Corporation System and method for providing personalized recommended word and computer readable recording medium recording program for implementing the method
US8793270B2 (en) * 2006-04-06 2014-07-29 Nhn Corporation System and method for providing personalized recommended word and computer readable recording medium recording program for implementing the method
US9075864B2 (en) 2006-10-10 2015-07-07 Abbyy Infopoisk Llc Method and system for semantic searching using syntactic and semantic analysis
US9495358B2 (en) 2006-10-10 2016-11-15 Abbyy Infopoisk Llc Cross-language text clustering
US9098489B2 (en) 2006-10-10 2015-08-04 Abbyy Infopoisk Llc Method and system for semantic searching
US9892111B2 (en) 2006-10-10 2018-02-13 Abbyy Production Llc Method and device to estimate similarity between documents having multiple segments
US9069750B2 (en) 2006-10-10 2015-06-30 Abbyy Infopoisk Llc Method and system for semantic searching of natural language texts
US20120089627A1 (en) * 2006-10-31 2012-04-12 Aol Inc. System and methods for performing machine-implemented tasks
US8997100B2 (en) * 2006-10-31 2015-03-31 Mercury Kingdom Assets Limited Systems and method for performing machine-implemented tasks of sending substitute keyword to advertisement supplier
US20120239653A1 (en) * 2007-06-28 2012-09-20 Microsoft Corporation Machine Assisted Query Formulation
US8812534B2 (en) * 2007-06-28 2014-08-19 Microsoft Corporation Machine assisted query formulation
US9740780B1 (en) 2009-03-23 2017-08-22 Google Inc. Autocompletion using previously submitted query data
US8930350B1 (en) 2009-03-23 2015-01-06 Google Inc. Autocompletion using previously submitted query data
US9378247B1 (en) 2009-06-05 2016-06-28 Google Inc. Generating query refinements from user preference data
US8412727B1 (en) * 2009-06-05 2013-04-02 Google Inc. Generating query refinements from user preference data
US8918417B1 (en) 2009-06-05 2014-12-23 Google Inc. Generating query refinements from user preference data
US9092528B1 (en) 2009-08-28 2015-07-28 Google Inc. Providing result-based query suggestions
US9563692B1 (en) 2009-08-28 2017-02-07 Google Inc. Providing result-based query suggestions
US8583675B1 (en) 2009-08-28 2013-11-12 Google Inc. Providing result-based query suggestions
US8996527B1 (en) 2009-11-04 2015-03-31 Google Inc. Clustering images
US8676803B1 (en) 2009-11-04 2014-03-18 Google Inc. Clustering images
US20110295897A1 (en) * 2010-06-01 2011-12-01 Microsoft Corporation Query correction probability based on query-correction pairs
US8280900B2 (en) * 2010-08-19 2012-10-02 Fuji Xerox Co., Ltd. Speculative query expansion for relevance feedback
US20120047159A1 (en) * 2010-08-19 2012-02-23 Fuji Xerox Co., Ltd. Speculative query expansion for relevance feedback
US8965882B1 (en) 2011-07-13 2015-02-24 Google Inc. Click or skip evaluation of synonym rules
US8756218B1 (en) * 2011-08-16 2014-06-17 Google Inc. Query classification based on search engine results
US8909627B1 (en) 2011-11-30 2014-12-09 Google Inc. Fake skip evaluation of synonym rules
US9858313B2 (en) 2011-12-22 2018-01-02 Excalibur Ip, Llc Method and system for generating query-related suggestions
US8965875B1 (en) 2012-01-03 2015-02-24 Google Inc. Removing substitution rules based on user interactions
US9152698B1 (en) 2012-01-03 2015-10-06 Google Inc. Substitute term identification based on over-represented terms identification
US9141672B1 (en) 2012-01-25 2015-09-22 Google Inc. Click or skip evaluation of query term optionalization rule
US9043248B2 (en) 2012-03-29 2015-05-26 International Business Machines Corporation Learning rewrite rules for search database systems using query logs
US9298671B2 (en) 2012-03-29 2016-03-29 International Business Machines Corporation Learning rewrite rules for search database systems using query logs
US9092504B2 (en) 2012-04-09 2015-07-28 Vivek Ventures, LLC Clustered information processing and searching with structured-unstructured database bridge
US20150120711A1 (en) * 2012-05-15 2015-04-30 Microsoft Technology Licensing, Llc Scenario based insights into structure data
US8959103B1 (en) 2012-05-25 2015-02-17 Google Inc. Click or skip evaluation of reordering rules
US20140074812A1 (en) * 2012-06-25 2014-03-13 Rediff.Com India Limited Method and apparatus for generating a suggestion list
US9607023B1 (en) 2012-07-20 2017-03-28 Ool Llc Insight and algorithmic clustering for automated synthesis
US9336302B1 (en) 2012-07-20 2016-05-10 Zuci Realty Llc Insight and algorithmic clustering for automated synthesis
US10318503B1 (en) 2012-07-20 2019-06-11 Ool Llc Insight and algorithmic clustering for automated synthesis
CN104428762B (en) * 2012-08-17 2017-10-17 英特尔公司 Use of data relationships through the data
CN104428762A (en) * 2012-08-17 2015-03-18 英特尔公司 Traversing data utilizing data relationships
EP2885721A4 (en) * 2012-08-17 2016-04-20 Intel Corp Traversing data utilizing data relationships
US9208194B2 (en) 2012-09-28 2015-12-08 International Business Machines Corporation Expanding high level queries
US8862609B2 (en) 2012-09-28 2014-10-14 International Business Machines Corporation Expanding high level queries
US9146966B1 (en) 2012-10-04 2015-09-29 Google Inc. Click or skip evaluation of proximity rules
US9189482B2 (en) 2012-10-10 2015-11-17 Abbyy Infopoisk Llc Similar document search
US9135307B1 (en) * 2012-12-27 2015-09-15 Google Inc. Selectively generating alternative queries
CN105247516A (en) * 2013-05-31 2016-01-13 谷歌公司 Query suggestions based on search data
WO2014193698A1 (en) * 2013-05-31 2014-12-04 Google Inc. Query suggestions based on search data
US9336277B2 (en) 2013-05-31 2016-05-10 Google Inc. Query suggestions based on search data
US9116952B1 (en) 2013-05-31 2015-08-25 Google Inc. Query refinements using search data
US9727603B1 (en) 2013-05-31 2017-08-08 Google Inc. Query refinements using search data
RU2638728C2 (en) * 2013-05-31 2017-12-15 Гугл Инк. Request proposal based on search data
US9223872B2 (en) 2013-06-13 2015-12-29 Microsoft Technology Licensing, Llc Dynamic query resolution using accuracy profiles
US9460149B2 (en) 2013-06-13 2016-10-04 Microsoft Technology Licensing, Llc Dynamic query resolution using accuracy profiles
US20140372410A1 (en) * 2013-06-14 2014-12-18 Microsoft Corporation Upper bounds for matching
US20150186394A1 (en) * 2013-12-30 2015-07-02 Yahoo! Inc. Query suggestions
US9672288B2 (en) * 2013-12-30 2017-06-06 Yahoo! Inc. Query suggestions
US10255240B2 (en) 2014-03-27 2019-04-09 Yandex Europe Ag Method and system for processing a voice-based user-input
EP3149729A4 (en) * 2014-03-27 2018-03-28 Yandex Europe AG Method and system for processing a voice-based user-input
US10318586B1 (en) 2014-08-19 2019-06-11 Google Llc Systems and methods for editing and replaying natural language queries
US10229212B2 (en) 2016-04-08 2019-03-12 Microsoft Technology Licensing, Llc Identifying Abandonment Using Gesture Movement

Also Published As

Publication number Publication date
US9495443B1 (en) 2016-11-15
US10223439B1 (en) 2019-03-05
US8504584B1 (en) 2013-08-06

Similar Documents

Publication Publication Date Title
Bollacker et al. CiteSeer: An autonomous web agent for automatic retrieval and identification of interesting publications
Baeza-Yates et al. Query recommendation using query logs in search engines
Nallapati Discriminative models for information retrieval
Liu et al. Identifying meaningful return information for XML keyword search
Si et al. A language modeling framework for resource selection and results merging
Jones et al. Generating query substitutions
JP5632124B2 (en) Rating method, the search result sorting method, rating systems and search results Sort system
KR100857049B1 (en) Automatically targeting web-based advertisements
RU2375747C2 (en) Checking relevance between key words and website content
US7689585B2 (en) Reinforced clustering of multi-type data objects for search term suggestion
EP0996899B1 (en) Apparatus and methods for an information retrieval system that employs natural language processing of search results to improve overall precision
US6167397A (en) Method of clustering electronic documents in response to a search query
US8082278B2 (en) Generating query suggestions from semantic relationships in content
US8725732B1 (en) Classifying text into hierarchical categories
US8429173B1 (en) Method, system, and computer readable medium for identifying result images based on an image query
US7028026B1 (en) Relevancy-based database retrieval and display techniques
US7289985B2 (en) Enhanced document retrieval
Liu et al. Mining topic-specific concepts and definitions on the web
US7321892B2 (en) Identifying alternative spellings of search strings by analyzing self-corrective searching behaviors of users
US7113944B2 (en) Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (CBIR).
US7657507B2 (en) Pseudo-anchor text extraction for vertical search
CA2526386C (en) Serving advertisements using user request information and user information
US8359282B2 (en) Supervised semantic indexing and its extensions
US7451124B2 (en) Method of analyzing documents
US7698267B2 (en) Searching digital information and databases

Legal Events

Date Code Title Description
AS Assignment

Owner name: GOOGLE INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAKER, STEVEN;HAAHR, PAUL;SIGNING DATES FROM 20041119 TO 20041122;REEL/FRAME:016035/0890

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: GOOGLE LLC, CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:GOOGLE INC.;REEL/FRAME:044101/0405

Effective date: 20170929

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8