WO2001042981A2 - Systeme de recherche et de recuperation de donnees en langage naturel en anglais - Google Patents
Systeme de recherche et de recuperation de donnees en langage naturel en anglais Download PDFInfo
- Publication number
- WO2001042981A2 WO2001042981A2 PCT/IB2000/002009 IB0002009W WO0142981A2 WO 2001042981 A2 WO2001042981 A2 WO 2001042981A2 IB 0002009 W IB0002009 W IB 0002009W WO 0142981 A2 WO0142981 A2 WO 0142981A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- word
- new
- vector
- elementat
- temp
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/3332—Query translation
- G06F16/3334—Selection or weighting of terms from queries, including natural language queries
Definitions
- the present invention relates generally to the field of computer searching and retrieval, and more particularly to the field of computer searching and retrieval using natural English language input into the search system.
- Search and retrieval systems using natural English language input are known in this art. These systems, however, are typically very complex, cumbersome, and costly to implement. Thus, the applicability of these systems to general search and retrieval tasks has been limited. More specifically, these known search and retrieval systems have had very little penetration into the Internet space because of these disadvantages. The known systems do not have a less complex, streamlined, and cost effective search and retrieval system and method that process natural English language inputs.
- a computer-implemented method and system for searching and retrieving using natural language.
- the method and system receive a text string having words. At least one of the words is identified as a topic word. Remaining words are classified either as a prefix description or a postfix description.
- a data store is searched based upon the identified topic word, prefix description, and postfix description. Results from the searching are scored based upon occurrence of the identified topic word, prefix description, and postfix description in the results.
- FIG. 1 is a flow chart of the preferred natural English language search and retrieval methodology according to the present invention
- FIG. 2 is a block diagram depicting the computer-implemented components of the present invention.
- FIG. 1 sets forth a flow chart 10 of the preferred search and retrieval methodology of the present invention.
- the method begins at step 12, where the user of the system inputs an English sentence or keywords in the form of a text string.
- the first stage of the system 14 then extracts words from the text string by using spaces as delimiters. Each word is then found in a dictionary 18 to obtain its properties. If the word is not found in the dictionary 18 it is assumed to be a noun.
- the dictionary 18 contains over 50,000 words with each word associated with one or more properties. These part of speech properties include noun, adjective, adverb, verb, conjunction, determiner (e.g., an article, and preposition).
- the extracted includes noun, adjective, adverb, verb, conjunction, determiner (e.g., an article, and preposition).
- the next stage 16 of the system determines a single property for each
- the rule schema 22 uses the word in question as a pivot and examines the properties of the word before and the properties of the word after the word being analyzed. A decision can only be made when the word before and/or the word after has a single property. If the pivot word's properties cannot be determined because the word before and after has multiple properties, the algorithm proceeds to the next word as the pivot. This process is repeated twice to find a single property for each word. If the rule schema 22 cannot find a single property for a word the default is the first property. The last word of the text string is forced to be a noun.
- the last stage 26 of the system is an interpreter that cleaves the input sentence into phrases based upon the singular properties of the words as identified in step 16.
- the delimiter of each phrase is a conjunction, preposition or a comma.
- the last noun of the first phrase is taken to be the topic (TP).
- the nouns and adjectives before the topic in the first phrase is termed the Prefix Description (Pre).
- the nouns and adjectives contained in the following phrases are termed the Postfix Description (Post).
- Post Postfix Description
- the topic, Prefix Description and N Postfix Description(s) are stored 28 for use in the search stages 30-36.
- the input into the search stages 30-36 include a topic containing a single word, a prefix description containing a collection a words, and a postfix description containing a collection a words.
- the system feeds one or more permutations of TP, Pre and Posts into one or more data miner applications.
- the data miner applications use data miner domain information 32 in order to apply the search permutations to various Internet domains.
- Each of the data miner applications then returns its top M search results for the particular Internet domain searched.
- the system provides the ability to customize the search and retrieval process by specifying what domains to search, and hence what data miners to execute.
- All of the M search results from the selected data miners are then combined and scored based on the occurrence of TP, Pre, and Posts within the search results at step 34.
- the score is calculated by the occurrence of each word contained in the topic, prefix and postfix descriptions. Additional points are give if an exact match is made using the same order of words found in the
- appendices A-G Attached to this application as appendices A-G are the Java source code files that reflect the preferred embodiment of the methodology depicted in FIG. 1. These appendices include: (A) Parser module (which extracts words and find properties); (B) Words Manipulator module (which cleaves sentences into phrases, and associated files); (C) One Subject data structure; (D) One Word data structure; (E) Word Grouping List data structure; (F) Word List data structure; and (G) Filter module (which ranks results according to topic, prefix description, postfix descriptions).
- FIG. 2 describes the Java source code modules set forth in Appendices (A) - (G).
- the Parser module 50 receives a user input text string 52.
- the Parser module 50 reads in dictionary 18 that in this example contains 50,000 words and their associated property codes.
- the Parser module 50 takes the user input text string 52 and tokenizes it into a data structure using spaces as delimiters.
- the Parser module 50 uses a binary search algorithm to find each word in the dictionary 18 and determine its
- Property codes include noun, adjective, adverb, verb, conjunction, determiner, and preposition. If the word is not found in the dictionary 18 it is assumed to be a noun.
- the Parser module 50 uses the properties rules base 22 to determine a single property code for each word.
- the rule schema uses the word in question as a
- pivot examines the properties of the word before and the properties of the word after. The decision is made when the word before and/or the word after has a single property. If the pivot word's properties cannot be determined because the word before and after has multiple properties the algorithm proceeds to the next word as the pivot. The process is repeated twice to find a single property for each word. If the rule schema cannot find a single property for a word the default is the first property. Moreover, the last word of the text string is forced to be a noun.
- the Words Manipulator module 54 takes each set of words and property codes and places it into the One Word data structure 56. Each group
- One Word data structure 56 is then cleaved using conjunctions, prepositions, and commas as delimiters into phrases that are stored in the Word List data structure 58. Each entry in the Word List data structure 58 is added to the Word Grouping List data structure 60.
- the Word Grouping List data structure 60 is decomposed into the One Subject data structure 62 containing topic, prefix description, and postfix descriptions.
- the last noun of the first phrase of the Word List data structure 58 is taken to be the topic.
- Nouns and adjectives before the topic in the first phrase of the Word Grouping List data structure 60 form the prefix description.
- Nouns and adjectives contained in the following phrases in the Word Grouping List data structure 60 are taken as the postfix description.
- the One Word data structure 56 contains a word and its property code.
- the Word List data structure 58 contains a phrase of nouns and adjectives.
- the Word Grouping List data structure 60 contains a group of phrases.
- the One Subject data structure 62 contains topic, prefix description, postfix descriptions.
- the Filter module 64 generates permutations of topic, prefix and postfix descriptions.
- the data miner domain information 32 which may include Internet information uses the permutations to search a domain and return the top results. Results are ranked according to topic, prefix description, postfix descriptions. Points are scored highest for exact matches.
- a Topic match is scored high, then prefix description and the least points are given to a postfix description match.
- the ranked best search results 66 are returned to the user.
- temp compareTof 0
- temp b ⁇ narySearch(Words, sentence elementAt( ⁇ ) toStnngO toLowerCase(), Codes), coding addElement(temp t ⁇ m()),
- ⁇ tok new St ⁇ ngToken ⁇ zer(coding.elementAt( ⁇ ).toS r ⁇ ng(), ","); coding. setElementAt( ⁇ ew String(tok.nextToken()), ⁇ );
- WordList wordList new WordL ⁇ st()
- Vector list new Vector()
- groupinglist new WordGroup ⁇ ngL ⁇ st()
- ⁇ word new Str ⁇ ng(subject getL ⁇ st() elementAt(j) toStnngO), if ( ⁇ sMoney(word))
- ⁇ ⁇ ⁇ queryStnng new OneSubject(ma ⁇ nSubject, precede, description), return queryStnng,
- WordGroupList new Vector(), ⁇ public void addGroup(OneSubject subject) ⁇ WordGroupList. addElement(subject);
- frontText new Stringf
- frontText new Str ⁇ ng(frontText + " " + prec.elementAt(j).toStr ⁇ ng() toLowerCase())
- DPOINTS PPOINTS
- Object tempPoint points. elementAt(i); points. setElementAt(po ⁇ nts.elementAt(j), i), points. setElementAt(tempPoint, j);
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU22128/01A AU2212801A (en) | 1999-12-07 | 2000-12-06 | Natural english language search and retrieval system and method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16941499P | 1999-12-07 | 1999-12-07 | |
US60/169,414 | 1999-12-07 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2001042981A2 true WO2001042981A2 (fr) | 2001-06-14 |
WO2001042981A3 WO2001042981A3 (fr) | 2003-12-24 |
Family
ID=22615581
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2000/002009 WO2001042981A2 (fr) | 1999-12-07 | 2000-12-06 | Systeme de recherche et de recuperation de donnees en langage naturel en anglais |
Country Status (3)
Country | Link |
---|---|
US (1) | US20010044720A1 (fr) |
AU (1) | AU2212801A (fr) |
WO (1) | WO2001042981A2 (fr) |
Families Citing this family (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6859800B1 (en) * | 2000-04-26 | 2005-02-22 | Global Information Research And Technologies Llc | System for fulfilling an information need |
US20020123994A1 (en) * | 2000-04-26 | 2002-09-05 | Yves Schabes | System for fulfilling an information need using extended matching techniques |
US7120627B1 (en) * | 2000-04-26 | 2006-10-10 | Global Information Research And Technologies, Llc | Method for detecting and fulfilling an information need corresponding to simple queries |
US7409336B2 (en) * | 2003-06-19 | 2008-08-05 | Siebel Systems, Inc. | Method and system for searching data based on identified subset of categories and relevance-scored text representation-category combinations |
US20050071328A1 (en) * | 2003-09-30 | 2005-03-31 | Lawrence Stephen R. | Personalization of web search |
US8176041B1 (en) * | 2005-06-29 | 2012-05-08 | Kosmix Corporation | Delivering search results |
US7512596B2 (en) * | 2005-08-01 | 2009-03-31 | Business Objects Americas | Processor for fast phrase searching |
US10884585B2 (en) | 2006-06-22 | 2021-01-05 | Rohit Chandra | User widget displaying portions of content |
US20140149378A1 (en) * | 2006-06-22 | 2014-05-29 | Rohit Chandra | Method and apparatus for determining rank of web pages based upon past content portion selections |
US11301532B2 (en) | 2006-06-22 | 2022-04-12 | Rohit Chandra | Searching for user selected portions of content |
US11763344B2 (en) | 2006-06-22 | 2023-09-19 | Rohit Chandra | SaaS for content curation without a browser add-on |
US8910060B2 (en) * | 2006-06-22 | 2014-12-09 | Rohit Chandra | Method and apparatus for highlighting a portion of an internet document for collaboration and subsequent retrieval |
US10866713B2 (en) | 2006-06-22 | 2020-12-15 | Rohit Chandra | Highlighting on a personal digital assistant, mobile handset, eBook, or handheld device |
US9292617B2 (en) | 2013-03-14 | 2016-03-22 | Rohit Chandra | Method and apparatus for enabling content portion selection services for visitors to web pages |
US11288686B2 (en) | 2006-06-22 | 2022-03-29 | Rohit Chandra | Identifying micro users interests: at a finer level of granularity |
US11853374B2 (en) | 2006-06-22 | 2023-12-26 | Rohit Chandra | Directly, automatically embedding a content portion |
US10289294B2 (en) | 2006-06-22 | 2019-05-14 | Rohit Chandra | Content selection widget for visitors of web pages |
US8661031B2 (en) * | 2006-06-23 | 2014-02-25 | Rohit Chandra | Method and apparatus for determining the significance and relevance of a web page, or a portion thereof |
US10909197B2 (en) | 2006-06-22 | 2021-02-02 | Rohit Chandra | Curation rank: content portion search |
US11429685B2 (en) | 2006-06-22 | 2022-08-30 | Rohit Chandra | Sharing only a part of a web page—the part selected by a user |
US9043197B1 (en) * | 2006-07-14 | 2015-05-26 | Google Inc. | Extracting information from unstructured text using generalized extraction patterns |
US8280877B2 (en) * | 2007-02-22 | 2012-10-02 | Microsoft Corporation | Diverse topic phrase extraction |
US7860885B2 (en) * | 2007-12-05 | 2010-12-28 | Palo Alto Research Center Incorporated | Inbound content filtering via automated inference detection |
JP5702551B2 (ja) * | 2009-07-02 | 2015-04-15 | 株式会社東芝 | 読影レポート検索支援装置及び読影レポート検索装置 |
WO2019070954A1 (fr) * | 2017-10-05 | 2019-04-11 | Liveramp, Inc. | Extraction et optimisation de termes de recherche dans des fichiers de texte en langage naturel |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0631244A2 (fr) * | 1993-06-24 | 1994-12-28 | Xerox Corporation | Un procédé et système pour le recouvrement d'informations |
US5592668A (en) * | 1993-08-25 | 1997-01-07 | Asymetrix Corporation | Method and apparatus for specifying a query to an information system using natural language-like constructs |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5488725A (en) * | 1991-10-08 | 1996-01-30 | West Publishing Company | System of document representation retrieval by successive iterated probability sampling |
GB9220404D0 (en) * | 1992-08-20 | 1992-11-11 | Nat Security Agency | Method of identifying,retrieving and sorting documents |
US5454106A (en) * | 1993-05-17 | 1995-09-26 | International Business Machines Corporation | Database retrieval system using natural language for presenting understood components of an ambiguous query on a user interface |
US5715468A (en) * | 1994-09-30 | 1998-02-03 | Budzinski; Robert Lucius | Memory system for storing and retrieving experience and knowledge with natural language |
US5963940A (en) * | 1995-08-16 | 1999-10-05 | Syracuse University | Natural language information retrieval system and method |
US5852820A (en) * | 1996-08-09 | 1998-12-22 | Digital Equipment Corporation | Method for optimizing entries for searching an index |
US5895464A (en) * | 1997-04-30 | 1999-04-20 | Eastman Kodak Company | Computer program product and a method for using natural language for the description, search and retrieval of multi-media objects |
US5933822A (en) * | 1997-07-22 | 1999-08-03 | Microsoft Corporation | Apparatus and methods for an information retrieval system that employs natural language processing of search results to improve overall precision |
US6263328B1 (en) * | 1999-04-09 | 2001-07-17 | International Business Machines Corporation | Object oriented query model and process for complex heterogeneous database queries |
-
2000
- 2000-12-06 AU AU22128/01A patent/AU2212801A/en not_active Abandoned
- 2000-12-06 WO PCT/IB2000/002009 patent/WO2001042981A2/fr active Application Filing
-
2001
- 2001-02-26 US US09/732,190 patent/US20010044720A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0631244A2 (fr) * | 1993-06-24 | 1994-12-28 | Xerox Corporation | Un procédé et système pour le recouvrement d'informations |
US5592668A (en) * | 1993-08-25 | 1997-01-07 | Asymetrix Corporation | Method and apparatus for specifying a query to an information system using natural language-like constructs |
Also Published As
Publication number | Publication date |
---|---|
US20010044720A1 (en) | 2001-11-22 |
WO2001042981A3 (fr) | 2003-12-24 |
AU2212801A (en) | 2001-06-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2001042981A2 (fr) | Systeme de recherche et de recuperation de donnees en langage naturel en anglais | |
US11321312B2 (en) | Vector-based contextual text searching | |
US6721697B1 (en) | Method and system for reducing lexical ambiguity | |
US7283951B2 (en) | Method and system for enhanced data searching | |
Soderland | Learning information extraction rules for semi-structured and free text | |
Muslea | Extraction patterns for information extraction tasks: A survey | |
Miller et al. | BBN: Description of the SIFT system as used for MUC-7 | |
Arampatzis et al. | Phase-based information retrieval | |
Kim et al. | Acquisition of semantic patterns for information extraction from corpora | |
EP0805404A1 (fr) | Méthode et système pour le traîtement lexicographique de textes en lettres majuscules et non accentuées | |
US20040078190A1 (en) | Method and system for describing and identifying concepts in natural language text for information retrieval and processing | |
WO2004114163A2 (fr) | Procede et systeme pour une recherche amelioree de donnees | |
WO2001084376A2 (fr) | Systeme pour repondre a des questions formulees en langage naturel | |
US6535886B1 (en) | Method to compress linguistic structures | |
Fu et al. | Towards indonesian part-of-speech tagging: Corpus and models | |
US6907562B1 (en) | Hypertext concordance | |
Ruprecht et al. | Supertagging-based parsing with linear context-free rewriting systems | |
Lehmann et al. | BNCweb | |
Zahariev | A linguistic approach to extracting acronym expansions from text | |
Simov et al. | Cascaded regular grammars over XML documents | |
Xiao et al. | A global rule induction approach to information extraction | |
POPOVIČ et al. | Processing of documents and queries in a Slovene language free text retrieval system | |
Đorđević et al. | Different approaches in serbian language parsing using context-free grammars | |
Ando et al. | Unsupervised statistical segmentation of Japanese kanji strings | |
Rodphon et al. | Thai OCR error correction using token passing algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: JP |