WO2008016494A1 - Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier - Google Patents
Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier Download PDFInfo
- Publication number
- WO2008016494A1 WO2008016494A1 PCT/US2007/016447 US2007016447W WO2008016494A1 WO 2008016494 A1 WO2008016494 A1 WO 2008016494A1 US 2007016447 W US2007016447 W US 2007016447W WO 2008016494 A1 WO2008016494 A1 WO 2008016494A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- asset
- query
- search
- user
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99937—Sorting
Definitions
- Ranking functions that rank documents according to their relevance to a given search query are known.
- query classifiers are used to boost the search results of the ranking function.
- Known query classifiers utilize machine learning techniques such as Maximum Entropy, Naive Bayes, Conditional Random Fields, and Support.
- Vector Machines to model user performance.
- the process used for building and deploying models is to gather a lot of data, perform off-line data processing over the entire range of data, build the models, and then deploy the models.
- These known processes can be computationally expensive. Further, in these processes, there is typically a delay from the time data is gathered to when the data is actually utilized in the deployed models.
- the incrementally-updatable, query classifier does not require computationally expensive, off-line data processing, but simply requires periodic updating of a minimum number of query classifier variables.
- the incrementally- updatable, query classifier utilizes real-time user search query and response data, and may also comprise a time-decay component that weighs more recent user events greater than past user events.
- the incrementally-updatable, query classifier alone or in combination with other ranking function components, may be used by a search engine to rank multiple documents in order (typically, in descending order) based on document relevance relative to query search terms.
- FIG. 1 represents an exemplary logic flow diagram showing exemplary steps in a method of producing ranked search results in response to a search query inputted by a user
- FIG. 2 is a block diagram of some of the primary components of an exemplary operating environment for implementation of the methods and processes disclosed herein;
- FIGS. 3A-3B represent a logic flow diagram showing exemplary steps in an exemplary method of producing ranked search results using an incrementally updatable model
- FIGS. 4A-4B represent a logic flow diagram showing exemplary steps in a method of producing ranked search results using an incrementally updatable model that further comprises a time-decay component.
- FIG. 1 depicts exemplary search process 10, which starts with process step 80, wherein a user inputs a search query. From step 80, exemplary search process 10 proceeds to step 200, wherein a search engine searches all documents within a network space for one or more terms of the search query.
- exemplary search process 10 proceeds to step 300, wherein a ranking function of the search engine ranks the documents within the network space based on the relevance of each document to the search query, the document relevance being based on the probability of retrieving the document given the search query based on a modified Na ⁇ ve Bayes model as described below. From step 300, exemplary search process 10 proceeds to step 400, wherein ranked search results are presented to the user, typically in descending order, identifying documents within the network space that are most relevant to the search query (i.e., have the highest probability of retrieval given a particular search query).
- the methods of ranking documents comprises utilizing real-time "user response data” or "document usage data” related to actual usage (e.g., viewing) of one or more documents within a network space by one or more users.
- the document usage data is generated and stored by application code on a web server, which is separate from a given search engine.
- document usage data may be maintained by a web site so that each time a user requests a URL, the server updates a usage counter or count value.
- the usage counter may maintain document-related data obtained for a given time interval, such as last week, last month, last year, or the lifetime of a given document or set of documents.
- Application code may be used to obtain the usage data from the web site via (i) a special application programming interface (API), (ii) a web service request, or (iii) by requesting an administration web page that returns usage data for every URL on the web site.
- API application programming interface
- the document relevance may be used to rank documents within a network space, and provide the ranked search results to a user.
- a method of ranking documents on a network may comprise the steps of determining a document relevance for each document on the network using the incrementally-updatable query classifier; and ranking the documents in a desired order (typically, in descending order) based on the document relevance of each document relative to a user inputted search query.
- Application programs using the methods disclosed herein may be loaded and executed on a variety of computer systems comprising a variety of hardware components.
- An exemplary computer system and exemplary operating environment for practicing the methods disclosed herein is described below.
- FIG. 2 illustrates an example of a suitable computing system environment 100 on which the methods disclosed herein may be implemented.
- the computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the methods disclosed herein. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment 100.
- the methods disclosed herein are operational with numerous other general purpose or special purpose computing system environments or configurations.
- Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the methods disclosed herein include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
- program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
- the methods and processes disclosed herein may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote computer storage media including memory storage devices.
- an exemplary system for implementing the methods and processes disclosed herein includes a general purpose computing device in the form of a computer 110.
- Components of computer 110 may include, but are not limited to, a processing unit 120, a system memory 130, and a system bus 121 that couples various system components including, but not limited to, system memory 130 to processing unit 120.
- System bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- Computer 110 typically includes a variety of computer readable media.
- Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media.
- computer readable media may comprise computer storage media and communication media.
- Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
- Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium, which can be used to store the desired information and which can be accessed by computer 110.
- Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
- communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media as used herein.
- System memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132.
- a basic input/output system 133 (BIOS) containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131.
- RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120.
- FIG. 2 illustrates operating system 134, application programs 135, other program modules 136, and program data 137.
- Computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media.
- FIG. 2 illustrates a hard disk drive 140 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes to a removable, nonvolatile magnetic disk 152, and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156 such as a CD ROM or other optical media.
- Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like.
- Hard disk drive 141 is typically connected to system bus 121 through a non-removable memory interface such as interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to system bus 121 by a removable memory interface, such as interface
- hard disk drive 141 is illustrated as storing operating system 144, application programs 145, other program modules 146, and program data 147. Note that these components can either be the same as or different from operating system 134, application programs 135, other program modules 136, and program data 137. Operating system 144, application programs 145, other program modules 146, and program data 147 are given different numbers here to illustrate that, at a minimum, they are different copies.
- a user may enter commands and information (e.g., one or more search query terms or words) into computer 110 through input devices such as a keyboard 162 and pointing device 161, commonly referred to as a mouse, trackball or touch pad.
- Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like.
- processing unit 120 may be connected by a user input interface 160 that is coupled to system bus 121, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).
- a monitor 191 or other type of display device is also connected to system bus 121 via an interface, such as a video interface 190.
- computer 110 may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 195.
- Computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180.
- Remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer 110, although only a memory storage device 181 has been illustrated in FIG. 2.
- the logical connections depicted in FIG. 2 include a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks.
- LAN local area network
- WAN wide area network
- computer 110 When used in a LAN networking environment, computer 110 is connected to LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, computer 110 typically includes a modem
- Modem 172 or other means for establishing communications over WAN 173, such as the Internet.
- Modem 172 which may be internal or external, may be connected to system bus 121 via user input interface 160, or other appropriate mechanism.
- program modules depicted relative to computer 110, or portions thereof may be stored in the remote memory storage device.
- FIG. 2 illustrates remote application programs 185 as residing on memory device 181. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
- Methods and processes disclosed herein may be implemented using one or more application programs including, but not limited to, a server system software application (e.g., WINDOWS SERVER SYSTEMTM software application), a search ranking application, and an application for generating, maintaining and storing usage data of documents within a network space (e.g., WINDOWS ® SHAREPOINT ® Services application), any one of which could be one of numerous application programs designated as application programs 135, application programs 145 and remote application programs 185 in exemplary system 100.
- a server system software application e.g., WINDOWS SERVER SYSTEMTM software application
- search ranking application e.g., WINDOWS ® SHAREPOINT ® Services application
- an application for generating, maintaining and storing usage data of documents within a network space e.g., WINDOWS ® SHAREPOINT ® Services application
- the disclosed methods of ranking documents given a particular search query may be implemented in other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, and the like.
- the disclosed methods of ranking documents may also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote memory storage devices.
- methods of ranking documents on a network using an incrementally-updatable query classifier are provided.
- the disclosed methods of ranking documents on a network utilize a modified Naive Bayesian model as described below.
- Bayes' rule states that the probability of A given B, P(A
- the query classifier model utilizes a similar formulation, which predicts the occurrence of an "Asset" (e.g., a chosen document or “hit") given a search "Query,” where the features are the terms or words in the search query.
- An "Asset” e.g., a chosen document or “hit”
- Query e.g., a search "Query”
- the Query Classifier model used herein is shown by the formula:
- Asset is the asset or task (e.g.,. document) being returned
- Query is the user's query containing one or more search query terms
- NQ is the number of terms in the search query
- V-NQ is the number of terms not seen in the search query where V is the size of the vocabulary
- P(Wi e Q I Asset) is the probability of seeing a term in the search query
- P(Wi $ ⁇ Q I Asset) is the probability of not seeing a term in the search query
- log[P(Asset I Query)] log[P(Asset)] + 2JlOg[P(W 1 . e Q
- Asset)] + ⁇ ⁇
- log[P(Asset I Query)] log[#(Asset)] - log[#(r)] +
- #(Asset) is the number of times a given "Asset” (e.g., a given document on the network) is selected for viewing by any user;
- #(T) is the total number of search queries that have been logged by the system, and
- #(w,; Asset) is the number of times a given "Asset" (e.g., a given document on the network) and a search query term, w b of a search query have been matched by any user.
- log[P(Asset I Query)] ⁇ og[#(Asset)]- ⁇ og[#(T)] + C 1 ) wherein: ⁇ #(w,-, Asset) is the sum of the number of times that a given "Asset" (e.g., a given document on the network) and any search query term, W 1 , of the search query have been matched by any user.
- ⁇ #(w,-, Asset) is the sum of the number of times that a given "Asset" (e.g., a given document on the network) and any search query term, W 1 , of the search query have been matched by any user.
- (I) may be used to provide document relevance scores (i.e., probabilities of retrieving a given asset or document based on a given search query) for documents within a network space.
- document relevance scores i.e., probabilities of retrieving a given asset or document based on a given search query
- the actual usage or user response data (e.g., the values of the terms #(Asset) and #(w,-, Asset)) may be obtained from one or more files that store actual usage data for one or more documents on a network.
- the actual usage data may be stored, along with the document, as a document component, or may be stored in a data storage file separate from the actual document. Data storage and retrieval may be accomplished using any known method including, but not limited to, web service calls, Structured Query
- FIGS. 3A-3B represent a logic flow diagram showing exemplary steps in an exemplary method of producing ranked search results in response to a search query inputted by a user, wherein the method utilizes user response data in an incrementally-updatable system.
- exemplary method 30 starts at block 201 and proceeds to step 202, wherein a user inputs a search query such as a single word or a string of words or terms.
- exemplary method 30 proceeds to step 203, wherein the search query component(s) are "tokenized.”
- tokenized refers to a process in which a word-breaker is used to create a list of "tokens" from a string of words or terms.
- search query "printing problems color” would be tokenized to form the set of tokens/search query components ⁇ "printing", "problems", "color” ⁇ .
- Each "token” in the list represents each search query component, w,-, used in the query classifier formulation described above.
- exemplary method 30 proceeds to step 205, wherein the pre-calculated, stored values #(wt, Asset) and log[#(w / , Asset)] are retrieved from a database for each query component, w,- (e.g., "printing", "problems", and "color”). From step 205, exemplary method 30 proceeds to step 208, wherein the pre-calculated, stored values #(Assef), log[#(A$se ⁇ ], and ⁇ #(w,-, Asset) are retrieved from the data storage database.
- exemplary method 30 proceeds to step 209, wherein the usage data from steps 205 and 208, namely (1) values #(w,-, Asset) and log[#(w,-, Asset)] for each query term/asset combination, and (2) values #(Asset), log[#C4_?_?et)], and ⁇ #(w,-, Asset) for each asset, are utilized in formula (I) provided above to determine the document relevance of each document in the network space relative to the search query.
- exemplary method 30 proceeds to step 210 shown in FIG. 3B, wherein all documents are ranked in descending order of relevance. Exemplary method 30 then proceeds to step 211, wherein at least a portion of the search results are displayed to the user. From step 211, exemplary method 30 proceeds to step 212, wherein the user selects (e.g., clicks on) a given search result (e.g., asset) for viewing.
- a given search result e.g., asset
- exemplary method 30 proceeds to step 213, wherein the count values #(Asset), #(w,-, Asset), and ⁇ #(Wi, Asset) are updated to reflect the user response. It should be noted that the process of updating count values
- #(Asset), #(w h Asset), and ⁇ #(w/, Asset) may occur at any time after the user selects a given search result (e.g., asset).
- the count values may be updated immediately or may be updated at some time following the actual selection of a given search result.
- the count values may be updated, for example, every hour or a single time during a 24 hour period. If the values are not updated immediately, a log is maintained over a given time period (e.g., for a 24 hour time period) to temporarily store user response data during the time period. At the end of each time period, application code is used to update each count value.
- exemplary method 30 proceeds to step 214, wherein the values log[#(Asset)] and log[#(w ⁇ , Asset)] are calculated. From step 214, exemplary method 30 proceeds to step 215, wherein the updated values from steps 213 and 214 (i.e., #(Asset), #(w,-, Asset), ⁇ #(w h Asset), log[#(Asset)] and log[#(w/, Asset)]) are stored in the database, replacing the previously stored values.
- step 215 exemplary method 30 proceeds to decision block
- step 216 wherein a determination is made by the user whether to view other search results provided in step 211. If a determination is made by the user to view other search results provided in step 211, exemplary method 30 proceeds to step 216, wherein the user returns to the search results provided in step 211. From step
- exemplary method 30 returns to step 212, and proceeds as described above.
- exemplary method 30 proceeds to decision block 218.
- decision block 218 a determination is made by the user whether to perform a new search. If a determination is made by the user to perform a new search, exemplary method 30 proceeds to step 219, wherein exemplary method 30 returns to step 202 and proceeds as described above. If a determination is made by the user at decision block 216 not to perform a new search, exemplary method 30 proceeds to step 220, wherein exemplary method 30 ends.
- the above-described method of providing ranked search results to a user in an incrementally-updatable system may further comprise a time-decay component, wherein more recent query/asset data is given more weight when computing values #(Asset), #(vi>,-, Asset), ⁇ #(w;, Asset), log[#(Asset)] and log[#(wv, Asset)] compared to older query/asset data.
- a time-decay component wherein more recent query/asset data is given more weight when computing values #(Asset), #(vi>,-, Asset), ⁇ #(w;, Asset), log[#(Asset)] and log[#(wv, Asset)] compared to older query/asset data.
- the query term "Trojan” might have meant something very different 2000 years ago, as oppose to what the term meant several years ago when a sports team named "Trojan” won a college football title, and what the term meant recently when the query/asset data.
- a weighing multiplier is used to weight more recent events to a greater extent than past events.
- the above-described count values used in Formula (I) may be set to equal the following values:
- #(Asset) #(Asset)(0) + ⁇ [#( ⁇ L ⁇ w?/)(l)] + X 2 [#(Asset)(2) ⁇ + ⁇ 3 [#(Asset)O)] + ...
- #(Asset)(nsw), #(w,-, Asset)(neW) and ⁇ #(w,, Asset)(new) represent the incrementally updated values for the counts #(Assei), #(w,, Asset) and ⁇ #(w,-,
- #(Assef)(Q), #(w / , Asset)(0) and ⁇ #(w,-, Asset)(0) represent the number of occurrences within the last time period (e.g., the last 24 hour period) for the counts #(Asset), #(w,-. Asset) and ⁇ #( ⁇ > / , Asset) respectively; and #(Asset)(old), #(w,- s represent the cumulative count values prior to the last time period (e.g., the count values more than 24 hours ago) for the counts #(Asset), #(w,-, Asset) and ⁇ #(w,-, Asset) respectively.
- exemplary method 40 starts at start block
- step 301 a user selects a weighing multiplier, ⁇ ', wherein ⁇ is a number less than 1.0 and t is an integer representing the age of a query and/or asset count value component.
- step 302 exemplary method 40 proceeds to step 303, wherein a user selects a frequency in which all count values #(Asset), #(w,, Asset), and ⁇ #(w,, Asset) are updated.
- all count values are updated once during a 24 hour period (i.e., each unit of time represented by t is a 24 hour time period); however, a user may choose any desired frequency, such as a count value update every hour (i.e., each unit of time represented by t is an hour of time).
- exemplary method 40 proceeds to step 305, wherein the time and date is monitored so that a given query and user response may be dated. From step 305, exemplary method 40 proceeds to step 306, wherein the count values are incrementally updated (e.g., updated at the end of a given frequency period) as described above, hi this step, all that is required to incrementally update each count value is to add new query/asset combination data (i.e., #(Assef)(0), #(w,, Asset)(fi) and ⁇ #(w,, Asset)(0) data) with a weight of 1.0 to the previously stored count value (i.e., #(Asset)(old), #(w, s ⁇ t ⁇ .sefXoId) and ⁇ #(w,-, Asset)(old)) time-decayed by a factor of ⁇ . As discussed above, the updated count values equal:
- #(Asset)(nQw) #(Asset)(0) + ⁇ [#(Asset)(old)];
- exemplary method 40 proceeds to step 307, wherein the values of ⁇ og[#(Asset)] and log[#(w/, Asset) ⁇ are calculated, and all updated values are stored in the database. From step 307, exemplary method 40 proceeds to decision block 308 as shown in FIG. 4B, wherein a determination is made by a user whether to perform a search.
- exemplary method 40 proceeds to step 318, wherein the above-described steps 202 to 211 of exemplary method 30 (shown in FIGS. 3A-3B) are performed. From step 318, exemplary method 40 proceeds to step 309, wherein the user selects (e.g., clicks on) a given search result (e.g., asset) for viewing.
- a given search result e.g., asset
- exemplary method 40 proceeds to step 310, wherein the values #(Assei)(G), # ⁇ w h Asset) ⁇ ), and ⁇ #(w,-, Asset)(0) are updated to reflect the user response.
- the #(Asset)(Q), #(w,-, Asset)(0), and ⁇ #(w,-, Asset)(0) values are temporarily stored until the end of a frequency period (e.g., the end of a 24 hour period).
- the frequency period may be any desired length of time, but is typically a 24 hour period.
- application code is used to update each count value, #(Asset), #(w,-, Asset), and ⁇ #(w,-, Asset) so as to provide incrementally updated count values #(Asset)(new),
- exemplary method 40 proceeds to decision block 311, wherein a determination is made by the user whether to view other search results provided in 318 (i.e., step 211 of exemplary method 30, which is a component of step 318 of exemplary method 40). If a determination is made by the user to view other search results provided in step 318, exemplary method 40 proceeds to step 312, wherein the user returns to the search results provided in step 318. From step 312, exemplary method 40 returns to step 309, and proceeds as described above. Returning to decision block 311, if a determination is made by the user at decision block 311 not to view other search results provided in step 318, exemplary method 40 proceeds to decision block 313.
- exemplary method 40 returns to step 306 and proceeds as described above. If at decision block 315, a determination is made by application code that the count values do not need to be updated, exemplary method 40 returns to step 308 and proceeds as described above.
- exemplary method 40 proceeds to step 319, wherein exemplary method 40 returns to decision block
- any known ranking function component or components may be used.
- Known ranking function components may include, but are not limited to, (i) a field weighted ranking function component as described in U.S. Patent Application Serial No. 10/804,326 entitled "FIELD WEIGHTING IN TEXT DOCUMENT
- An exemplary computing system contains at least one application module usable on the computing system, wherein the at least one application module comprises application code loaded thereon, wherein the application code performs a method of ranking documents on a network using the above-described incrementally- updatable query classifier with or without a time-decay component.
- the application code may be loaded onto the computing system using any of the above-described computer readable medium having thereon computer- executable instructions for ranking documents on a network using the above- described incrementally-updatable query classifier with or without a time-decay component as described above.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Priority Applications (6)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2009522778A JP2009545809A (ja) | 2006-07-31 | 2007-07-20 | インクリメンタルに更新可能な変形ナイーブベイズクエリ分類器を使用したランク付け関数 |
| MX2009000584A MX2009000584A (es) | 2006-07-31 | 2007-07-20 | Funciones de clasificacion utilizando un clasificador de consulta bayesiana sencillo modificado, crecientemente actualizable. |
| BRPI0714938-7A BRPI0714938A2 (pt) | 2006-07-31 | 2007-07-20 | funÇÕes de ordenaÇço usando um classificador de consulta naÏve bayesiano modificado incrementalmente atualizÁvel |
| AU2007281641A AU2007281641B2 (en) | 2006-07-31 | 2007-07-20 | Ranking functions using an incrementally-updatable, modified naive bayesian query classifier |
| EP07796956A EP2047388A4 (en) | 2006-07-31 | 2007-07-20 | CLASSIFICATION FUNCTIONS USING A BAYESIAN NATIVE CLASSIFIER OF CONSULTATIONS, WHICH CAN BE UPDATED INCREMENTALLY |
| NO20085386A NO20085386L (no) | 2006-07-31 | 2008-12-29 | Rangeringsfunksjoner som anvender en inkrementelt oppdaterbar, modifisert Naïve Bayesian sporre-klassifiserer |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/496,373 | 2006-07-31 | ||
| US11/496,373 US7620634B2 (en) | 2006-07-31 | 2006-07-31 | Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2008016494A1 true WO2008016494A1 (en) | 2008-02-07 |
Family
ID=38987659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2007/016447 Ceased WO2008016494A1 (en) | 2006-07-31 | 2007-07-20 | Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier |
Country Status (10)
| Country | Link |
|---|---|
| US (1) | US7620634B2 (enExample) |
| EP (1) | EP2047388A4 (enExample) |
| JP (1) | JP2009545809A (enExample) |
| CN (1) | CN101496010A (enExample) |
| AU (1) | AU2007281641B2 (enExample) |
| BR (1) | BRPI0714938A2 (enExample) |
| MX (1) | MX2009000584A (enExample) |
| NO (1) | NO20085386L (enExample) |
| RU (1) | RU2443015C2 (enExample) |
| WO (1) | WO2008016494A1 (enExample) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9305279B1 (en) | 2014-11-06 | 2016-04-05 | Semmle Limited | Ranking source code developers |
Families Citing this family (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7860886B2 (en) * | 2006-09-29 | 2010-12-28 | A9.Com, Inc. | Strategy for providing query results based on analysis of user intent |
| US8161045B2 (en) * | 2007-02-01 | 2012-04-17 | The Boeing Company | Use of associate memory learning agent technology to identify interchangeable parts in parts catalogs |
| US7428522B1 (en) * | 2007-09-27 | 2008-09-23 | Yahoo! Inc. | Real-time search term popularity determination, by search origin geographic location |
| US20100042613A1 (en) * | 2008-08-14 | 2010-02-18 | Yield Software, Inc. | Method and system for automated search engine optimization |
| US8195669B2 (en) * | 2008-09-22 | 2012-06-05 | Microsoft Corporation | Optimizing ranking of documents using continuous conditional random fields |
| US20100169356A1 (en) * | 2008-12-30 | 2010-07-01 | Yield Software, Inc. | Method and System for Negative Keyword Recommendations |
| US20100185661A1 (en) * | 2008-12-30 | 2010-07-22 | Yield Software, Inc. | Method and System for Negative Keyword Recommendations |
| US20100169312A1 (en) * | 2008-12-30 | 2010-07-01 | Yield Software, Inc. | Method and System for Negative Keyword Recommendations |
| US9081857B1 (en) * | 2009-09-21 | 2015-07-14 | A9.Com, Inc. | Freshness and seasonality-based content determinations |
| US8311792B1 (en) * | 2009-12-23 | 2012-11-13 | Intuit Inc. | System and method for ranking a posting |
| US20110270815A1 (en) * | 2010-04-30 | 2011-11-03 | Microsoft Corporation | Extracting structured data from web queries |
| US9424351B2 (en) | 2010-11-22 | 2016-08-23 | Microsoft Technology Licensing, Llc | Hybrid-distribution model for search engine indexes |
| US9342582B2 (en) | 2010-11-22 | 2016-05-17 | Microsoft Technology Licensing, Llc | Selection of atoms for search engine retrieval |
| US9195745B2 (en) | 2010-11-22 | 2015-11-24 | Microsoft Technology Licensing, Llc | Dynamic query master agent for query execution |
| US8713024B2 (en) * | 2010-11-22 | 2014-04-29 | Microsoft Corporation | Efficient forward ranking in a search engine |
| US8620907B2 (en) | 2010-11-22 | 2013-12-31 | Microsoft Corporation | Matching funnel for large document index |
| US8478704B2 (en) | 2010-11-22 | 2013-07-02 | Microsoft Corporation | Decomposable ranking for efficient precomputing that selects preliminary ranking features comprising static ranking features and dynamic atom-isolated components |
| US9529908B2 (en) | 2010-11-22 | 2016-12-27 | Microsoft Technology Licensing, Llc | Tiering of posting lists in search engine index |
| US8407211B1 (en) | 2010-12-16 | 2013-03-26 | Google Inc. | Determining relevance scores for locations |
| US8751488B2 (en) * | 2011-08-24 | 2014-06-10 | Waypart, Inc. | Part number search method and system |
| JP2014032532A (ja) * | 2012-08-03 | 2014-02-20 | Advanced Media Inc | オペレータ支援システム |
| WO2014143000A1 (en) * | 2013-03-15 | 2014-09-18 | Mcafee, Inc. | Server-assisted anti-malware |
| US9614865B2 (en) | 2013-03-15 | 2017-04-04 | Mcafee, Inc. | Server-assisted anti-malware client |
| WO2014143012A1 (en) | 2013-03-15 | 2014-09-18 | Mcafee, Inc. | Remote malware remediation |
| RU2543315C2 (ru) | 2013-03-22 | 2015-02-27 | Федеральное государственное автономное образовательное учреждение высшего профессионального образования "Национальный исследовательский университет "Высшая школа экономики" | Способ отбора эффективных вариантов в поисковых и рекомендательных системах (варианты) |
| CN103235815A (zh) * | 2013-04-25 | 2013-08-07 | 北京小米科技有限责任公司 | 一种应用软件的显示方法和设备 |
| US9268823B2 (en) * | 2013-05-10 | 2016-02-23 | International Business Machines Corporation | Partial match derivation using text analysis |
| EP2851808A3 (en) * | 2013-09-19 | 2015-04-15 | Maluuba Inc. | Hybrid natural language processor |
| CN103646207A (zh) * | 2013-12-02 | 2014-03-19 | 北京奇虎科技有限公司 | 一种应用程序安全属性的管理方法和装置 |
| RU2608886C2 (ru) * | 2014-06-30 | 2017-01-25 | Общество С Ограниченной Ответственностью "Яндекс" | Ранжиратор результатов поиска |
| CN106339383B (zh) * | 2015-07-07 | 2019-10-18 | 阿里巴巴集团控股有限公司 | 一种搜索排序方法及系统 |
| CN106919298B (zh) * | 2015-12-24 | 2020-06-23 | 中移(杭州)信息技术有限公司 | 一种显示方法及终端 |
| CN106933882B (zh) * | 2015-12-31 | 2020-09-29 | 华为技术有限公司 | 一种大数据增量计算方法和装置 |
| CN107045504A (zh) * | 2016-02-05 | 2017-08-15 | 百度在线网络技术(北京)有限公司 | 提供与功能组件相关联的搜索建议项的方法与装置 |
| CN107483420B (zh) * | 2017-07-31 | 2019-12-24 | 北京百悟科技有限公司 | 信息审核装置及方法 |
| CN109814976A (zh) * | 2019-02-01 | 2019-05-28 | 中国银行股份有限公司 | 一种功能模块排布方法及装置 |
| CN115129556B (zh) * | 2022-07-25 | 2025-05-16 | 济南浪潮数据技术有限公司 | 一种数据存储方法及相关设备 |
Family Cites Families (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7010508B1 (en) * | 1984-05-24 | 2006-03-07 | Lockwood Lawrence B | Automated multimedia data processing network |
| US6460034B1 (en) * | 1997-05-21 | 2002-10-01 | Oracle Corporation | Document knowledge base research and retrieval system |
| US6185550B1 (en) * | 1997-06-13 | 2001-02-06 | Sun Microsystems, Inc. | Method and apparatus for classifying documents within a class hierarchy creating term vector, term file and relevance ranking |
| US6711585B1 (en) | 1999-06-15 | 2004-03-23 | Kanisa Inc. | System and method for implementing a knowledge management system |
| CN100426284C (zh) * | 2000-11-20 | 2008-10-15 | 英国电讯有限公司 | 管理资源的方法和装置以及服务器计算机 |
| US6853998B2 (en) | 2001-02-07 | 2005-02-08 | International Business Machines Corporation | Customer self service subsystem for classifying user contexts |
| US6748398B2 (en) | 2001-03-30 | 2004-06-08 | Microsoft Corporation | Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (CBIR) |
| US7673234B2 (en) | 2002-03-11 | 2010-03-02 | The Boeing Company | Knowledge management using text classification |
| US7266559B2 (en) | 2002-12-05 | 2007-09-04 | Microsoft Corporation | Method and apparatus for adapting a search classifier based on user queries |
| JP2004206517A (ja) * | 2002-12-26 | 2004-07-22 | Nifty Corp | ホットキーワード提示方法及びホットサイト提示方法 |
| RU2236699C1 (ru) * | 2003-02-25 | 2004-09-20 | Открытое акционерное общество "Телепортал. Ру" | Способ поиска и выборки информации с повышенной релевантностью |
| US20040249796A1 (en) | 2003-06-06 | 2004-12-09 | Microsoft Corporation | Query classification |
| US7685296B2 (en) * | 2003-09-25 | 2010-03-23 | Microsoft Corporation | Systems and methods for client-based web crawling |
| US7231375B2 (en) | 2003-10-10 | 2007-06-12 | Microsoft Corporation | Computer aided query to task mapping |
| US20050131872A1 (en) | 2003-12-16 | 2005-06-16 | Microsoft Corporation | Query recognizer |
| US7424469B2 (en) | 2004-01-07 | 2008-09-09 | Microsoft Corporation | System and method for blending the results of a classifier and a search engine |
| US7359898B1 (en) * | 2004-02-26 | 2008-04-15 | Yahoo! Inc. | Scoring mechanism selection along multiple dimensions |
| WO2006034544A1 (en) | 2004-09-29 | 2006-04-06 | Panscient Pty Ltd | Machine learning system |
-
2006
- 2006-07-31 US US11/496,373 patent/US7620634B2/en not_active Expired - Fee Related
-
2007
- 2007-07-20 RU RU2009103140/08A patent/RU2443015C2/ru not_active IP Right Cessation
- 2007-07-20 JP JP2009522778A patent/JP2009545809A/ja active Pending
- 2007-07-20 WO PCT/US2007/016447 patent/WO2008016494A1/en not_active Ceased
- 2007-07-20 EP EP07796956A patent/EP2047388A4/en not_active Withdrawn
- 2007-07-20 MX MX2009000584A patent/MX2009000584A/es not_active Application Discontinuation
- 2007-07-20 CN CNA200780027961XA patent/CN101496010A/zh active Pending
- 2007-07-20 BR BRPI0714938-7A patent/BRPI0714938A2/pt not_active IP Right Cessation
- 2007-07-20 AU AU2007281641A patent/AU2007281641B2/en not_active Ceased
-
2008
- 2008-12-29 NO NO20085386A patent/NO20085386L/no not_active Application Discontinuation
Non-Patent Citations (5)
| Title |
|---|
| LIANGXIAO JIANG AND YUANYUAN GUO: "Learning lazy naive Bayesian classifiers for ranking", TOOLS WITH ARTIFICIAL INTELLIGENCE, 2005. ICTAI 05. 17TH IEEE INTERNATIONAL CONFERENCE, 14 November 2005 (2005-11-14) - 16 November 2005 (2005-11-16), XP010868005 * |
| See also references of EP2047388A4 * |
| ZHANG H. AND JIANG SU: "Naive Bayesian Classifiers for Ranking", EUROPEAN CONFERENCE ON MANCHINE LEARNING (ECML), 2004, XP008101795 * |
| ZHANG H. AND SHENGLI SHENG: "Learning Weighted Naive Bayes with Accurate Ranking", PROCEEDINGS OF THE FOURTH IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM'04), XP010778968 * |
| ZHANG H., JIANG L., SU J.: "Augmenting Naive Bayes for Ranking", PROCEEDINGS OF THE 22ND INTERNATIONAL CONFERENCE ON MACHINE LEARNING, BONN, GERMANY, 2005, XP008101796 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9305279B1 (en) | 2014-11-06 | 2016-04-05 | Semmle Limited | Ranking source code developers |
Also Published As
| Publication number | Publication date |
|---|---|
| RU2009103140A (ru) | 2010-08-10 |
| AU2007281641B2 (en) | 2011-09-08 |
| BRPI0714938A2 (pt) | 2013-05-21 |
| EP2047388A4 (en) | 2012-07-04 |
| EP2047388A1 (en) | 2009-04-15 |
| JP2009545809A (ja) | 2009-12-24 |
| CN101496010A (zh) | 2009-07-29 |
| US20080028010A1 (en) | 2008-01-31 |
| AU2007281641A1 (en) | 2008-02-07 |
| MX2009000584A (es) | 2009-01-27 |
| US7620634B2 (en) | 2009-11-17 |
| NO20085386L (no) | 2009-02-24 |
| RU2443015C2 (ru) | 2012-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7620634B2 (en) | Ranking functions using an incrementally-updatable, modified naïve bayesian query classifier | |
| CN101460949B (zh) | 索引文档以供信息检索 | |
| JP5174931B2 (ja) | ドキュメント利用統計を用いたランキング関数 | |
| US8209317B2 (en) | Method and apparatus for reconstructing a search query | |
| JP5736469B2 (ja) | ユーザ意図の有無に基づく検索キーワードの推薦 | |
| US8515950B2 (en) | Combining log-based rankers and document-based rankers for searching | |
| US7783626B2 (en) | Pipelined architecture for global analysis and index building | |
| Zamani et al. | Situational context for ranking in personal search | |
| RU2501078C2 (ru) | Ранжирование результатов поиска с использованием расстояния редактирования и информации о документе | |
| CN1728141B (zh) | 信息检索系统中基于短语的搜索 | |
| US20060106784A1 (en) | Linguistically aware link analysis method and system | |
| US9275128B2 (en) | Method and system for document indexing and data querying | |
| US20060129533A1 (en) | Personalized web search method | |
| IL188902A (en) | Rating functions that use a biased click of a document on the grid | |
| JP2011044170A (ja) | 異なるソースからのサーチエンジン結果を1つのサーチ結果へと混合する方法及びシステム | |
| US20110184940A1 (en) | System and method for detecting changes in the relevance of past search results | |
| US7765204B2 (en) | Method of finding candidate sub-queries from longer queries | |
| EP2017752A1 (en) | Information processing apparatus, information processing method and program | |
| GB2418038A (en) | Information handling by manipulating the space forming an information array | |
| CN112445973B (zh) | 一种寻找项目的方法、装置、存储介质和计算机设备 | |
| JP2005322098A (ja) | 情報検索装置、情報検索方法、情報検索プログラムおよび情報検索プログラムが記録された記録媒体 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| WWE | Wipo information: entry into national phase |
Ref document number: 200780027961.X Country of ref document: CN |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 07796956 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2007281641 Country of ref document: AU |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2007796956 Country of ref document: EP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 16/CHENP/2009 Country of ref document: IN |
|
| ENP | Entry into the national phase |
Ref document number: 2007281641 Country of ref document: AU Date of ref document: 20070720 Kind code of ref document: A |
|
| WWE | Wipo information: entry into national phase |
Ref document number: MX/A/2009/000584 Country of ref document: MX |
|
| ENP | Entry into the national phase |
Ref document number: 2009103140 Country of ref document: RU Kind code of ref document: A |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2009522778 Country of ref document: JP |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: PI0714938 Country of ref document: BR Kind code of ref document: A2 Effective date: 20090127 |