WO2009029922A2 - Indexation factuelle servant à une recherche en langage naturel - Google Patents

Indexation factuelle servant à une recherche en langage naturel Download PDF

Info

Publication number
WO2009029922A2
WO2009029922A2 PCT/US2008/074981 US2008074981W WO2009029922A2 WO 2009029922 A2 WO2009029922 A2 WO 2009029922A2 US 2008074981 W US2008074981 W US 2008074981W WO 2009029922 A2 WO2009029922 A2 WO 2009029922A2
Authority
WO
WIPO (PCT)
Prior art keywords
fact
semantic
computer
readable media
terms
Prior art date
Application number
PCT/US2008/074981
Other languages
English (en)
Other versions
WO2009029922A3 (fr
Inventor
Martin Henk Van Den Berg
Daniel Bobrow
Robert D. Cheslow
Barney Pell
Giovanni Lorenzo Thione
Chad Waters
Original Assignee
Powerset, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US12/201,596 external-priority patent/US8639708B2/en
Application filed by Powerset, Inc. filed Critical Powerset, Inc.
Publication of WO2009029922A2 publication Critical patent/WO2009029922A2/fr
Publication of WO2009029922A3 publication Critical patent/WO2009029922A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis

Definitions

  • Online search engines have become an increasingly important tool for conducting research or navigating documents accessible via the Internet. Often, the online search engines perform a matching process for detecting possible documents, or text within those documents, that utilizes a query submitted by a user. Initially, the matching process, offered by conventional online search engines such as those maintained by Google or Yahoo, allow the user to specify one or more words in the query to describe information that s/he is looking for. Next, the conventional online search engine proceeds to find all documents that contain exact matches of the words and typically presents the result for each document as a block of text that includes one or more of the words provided by the user therein.
  • the matching process offered by conventional online search engines such as those maintained by Google or Yahoo, allow the user to specify one or more words in the query to describe information that s/he is looking for.
  • the conventional online search engine proceeds to find all documents that contain exact matches of the words and typically presents the result for each document as a block of text that includes one or more of the words provided by the user therein.
  • Embodiments of the present invention generally relate to computer- readable media and a computer system for implementing a natural language search using fact-based structures. More specifically, this invention relates to generating fact-based structures. Such a fact-based structure is generated using a semantic structure, which represents information, such as the meaning of a piece of text, from a document, such as a web page. Typically, a natural language parser is used to create a syntactic structure of the information, and the parser identifies terms, as well as the relationship between the terms.
  • the syntactic structure associated with this sentence would include the information that "Mary” is the subject of the verb "wash” and "the cat” is the object of that verb.
  • the syntactic representation is then further semantically analyzed to create a semantic representation of the information associating with this sentence the semantic relationship between "Mary", the agent of the action, "the cat", the patient of the action, and "wash” the action, or relation, itself.
  • agent and patient are chosen here only as an example, and other choices for semantic relations are explicitly meant to be included in this invention.
  • a fact- based structure of a semantic structure allows for a linear structure of these terms and their relationships to be created, while also maintaining identifiers of the terms to convey the dependency of one fact-based structure on another fact-based structure.
  • FIG. 1 is a block diagram of an exemplary computing environment suitable for use in implementing embodiments of the present invention
  • FIG. 2 is a schematic diagram of an exemplary system architecture suitable for use in implementing embodiments of the present invention
  • FIG. 3 is a schematic diagram of a subsystem within the exemplary system architecture, in accordance with an embodiment of the present invention.
  • FIG. 4 is illustrative of a semantic structure, in accordance with an embodiment of the present invention.
  • FIGS. 5-6 are illustrative of fact-based structures, in accordance with an embodiment of the present invention.
  • FIG. 7 is a flow diagram illustrating an overall method for implementing a natural language search using fact-based structures, in accordance with an embodiment of the present invention.
  • the present invention is generally related to computer-readable media and a computer system for implementing a natural language search using fact-based structures. More specifically, this invention relates to generating fact-based structures. Such a fact- based structure is generated using a semantic structure, which represents information, such as the meaning of a piece of text, from a document, such as a web page. Typically, a natural language parser is used to create a syntactic representation of the information, and the parser identifies terms, as well as the relationship between the terms. For example, given the sentence "Mary washes the cat", the syntactic representation associated with this sentence would include the syntactic relationship between the object, subject, and verb in the sentence.
  • a natural language parser is used to create a syntactic representation of the information, and the parser identifies terms, as well as the relationship between the terms. For example, given the sentence "Mary washes the cat", the syntactic representation associated with this sentence would include the syntactic relationship between
  • the syntactic representation is then further semantically analyzed to create a semantic representation of the information associating with this sentence the semantic relationship between "Mary”, the agent of the action, "the cat", the patient of the action, and “wash” the action, or relation, itself.
  • a fact-based structure of a semantic structure allows for a linear structure of these terms and their relationships to be created, while also maintaining identifiers of the terms to convey the dependency of one fact-based structure on another fact-based structure.
  • one or more computer-readable media having computer-executable instructions embodied thereon that, when executed, perform a method for generating a fact-based index system for a natural language search.
  • the method includes first generating one or more semantic structures using information from documents, including information about the documents' textual content and possibly information about the document itself, and then, determining the facts within the semantic structures (because facts are the derived core relations between the entities mentioned in a sentence or a text, each semantic structure will contain at least one). And finally, the method includes generating a fact-based structure from each fact found within the semantic structure.
  • one or more computer-readable media having computer-executable instructions embodied thereon that, when executed, perform a method for implementing a natural language search using a fact-based index system.
  • the method comprises, in part, generating one or more semantic structures and assigning exactly one identifier for each term in the semantic representation to mark terms that are shared between facts. For example, the sentence "Mary washes the red cat.” corresponds to two facts: one expressing that Mary washes the cat, and one expressing that the cat is red. In this case, the term the cat will be assigned exactly one identifier in the semantic representation. As a result, in the two resulting fact-structures, the cat will have the same identifier.
  • a computer system embodied on one or more computer storage-media having computer-executable instructions provided thereon for performing a method for generating a fact-based structure.
  • the system comprises, in part, a server that invokes a semantic index procedure configured for generating a fact-based structure, wherein the procedure is implemented on, at least, the following components: a fact-based structure generating component for generating a fact-based structure using a functional structure associated with information from documents, the documents containing terms, wherein the functional structure contains one or more facts; and an identifying component for identifying and assigning an unique identifier to each node within the fact-based structure.
  • computing device 100 an exemplary operating environment for implementing embodiments of the present invention is shown and designated generally as computing device 100.
  • Computing device 100 is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing device 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.
  • the invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device.
  • program components including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types.
  • Embodiments of the present invention may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, specialty computing devices, etc.
  • Embodiments of the invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
  • computing device 100 includes a bus 110 that directly or indirectly couples the following devices: memory 112, one or more processors 114, one or more presentation components 116, input/output (I/O) ports 118, I/O components 120, and an illustrative power supply 122.
  • Bus 110 represents what may be one or more busses (such as an address bus, data bus, or combination thereof).
  • Computing device 100 typically includes a variety of computer-readable media.
  • computer-readable media may comprise Random Access Memory (RAM); Read Only Memory (ROM); Electronically Erasable Programmable Read Only Memory (EEPROM); flash memory or other memory technologies; CDROM, digital versatile disks (DVDs) or other optical or holographic media; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices; or any other medium that can be used to encode desired information and be accessed by computing device 100.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • EEPROM Electronically Erasable Programmable Read Only Memory
  • flash memory or other memory technologies
  • CDROM compact discs
  • DVDs digital versatile disks
  • magnetic cassettes magnetic tape
  • magnetic disk storage magnetic disk storage devices
  • Memory 112 includes computer-storage media in the form of volatile and/or nonvolatile memory.
  • the memory may be removable, nonremovable, or a combination thereof.
  • Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc.
  • Computing device 100 includes one or more processors that read data from various entities such as memory 112 or I/O components 120.
  • Presentation component(s) 116 present data indications to a user or other device.
  • Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
  • I/O ports 118 allow computing device 100 to be logically coupled to other devices including I/O components 120, some of which may be built in.
  • Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
  • FIG. 2 a schematic diagram of an exemplary system architecture 200 suitable for use in implementing embodiments of the present invention is shown, in accordance with an embodiment of the present invention.
  • the exemplary system architecture 200 shown in FIG. 2 is merely an example of one suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the present invention. Neither should the exemplary system architecture 200 be interpreted as having any dependency or requirement related to any single component or combination of components illustrated therein.
  • the system architecture 200 may include a distributed computing environment, where a client device 215 is operably coupled to a natural language engine 290, which, in turn, is operably coupled to a data store 220.
  • the operable coupling refers to linking the client device 215 and the data store 220 to the natural language engine 290, and other online components through appropriate connections.
  • These connections may be wired or wireless. Examples of particular wired embodiments, within the scope of the present invention, include USB connections and cable connections over a network (not shown). Examples of particular wireless embodiments, within the scope of the present invention, include a near-range wireless network and radio-frequency technology.
  • near-range wireless network is not meant to be limiting, and should be interpreted broadly to include at least the following technologies: negotiated wireless peripheral (NWP) devices; short- range wireless air interference networks (e.g., wireless personal area network (wPAN), wireless local area network (wLAN), wireless wide area network (wWAN), BluetoothTM, and the like); wireless peer-to-peer communication (e.g., Ultra Wideband); and any protocol that supports wireless communication of data between devices. Additionally, persons familiar with the field of the invention will realize that a near-range wireless network may be practiced by various data-transfer methods (e.g., satellite transmission, telecommunications network, etc.).
  • connections between the client device 215, the data store 220 and the natural language engine 290 are not limited by the examples described, but embrace a wide variety of methods of communications.
  • the computing device may internally accommodate the functionality of the server 250, thereby alleviating dependence on wireless or wired connections.
  • Exemplary system architecture 200 includes the client device 215 for, in part, supporting operation of the presentation device 275.
  • the presentation device e.g., a touchscreen display
  • the client device 215 may take the form of various types of computing devices.
  • the client device 215 may be a personal computing device (e.g., computing device 100 of FIG. 1), handheld device (e.g., personal digital assistant), a mobile device (e.g., laptop computer, cell phone, media player), consumer electronic device, various servers, and the like.
  • the computing device may comprise two or more electronic devices configured to share information therebetween.
  • the client device 215 includes, or is operably coupled to the presentation device 275, which is configured to present a UI display 295 on the presentation device 275.
  • the presentation device 275 may be configured as any display device that is capable of presenting information to a user, such as a monitor, electronic display panel, touch-screen, liquid crystal display (LCD), plasma screen, one or more light-emitting diodes (LED), incandescent bulbs, a laser, an electroluminescent light source, a chemical light, a flexible light wire, and/or fluorescent light, or any other display type, or may comprise a reflective surface upon which the visual information is projected.
  • LCD liquid crystal display
  • LED light-emitting diodes
  • presentation device 275 Although several differing configurations of the presentation device 275 have been described above, it should be understood and appreciated by those of ordinary skill in the art that various types of presentation devices that present information may be employed as the presentation device 275, and that embodiments of the present invention are not limited to those presentation devices 275 that are shown and described.
  • the UI display 295 rendered by the presentation device 275 is configured to surface a web page (not shown) that is associated with natural language engine 290 and/or a content publisher.
  • the web page may reveal a search-entry area that receives a query and search results that are discovered by searching the Internet with the query.
  • the query may be manually provided by a user at the search-entry area, or may be automatically generated by software.
  • the query may include one or more words that, when submitted, invokes the natural language engine 290 to identify appropriate search results that are most responsive to the words in a query.
  • the natural language engine 290 may take the form of various types of computing devices, such as, for example, the computing device 100 described above with reference to FIG. 1.
  • the natural language engine 290 may be a personal computer, desktop computer, laptop computer, consumer electronic device, handheld device (e.g., personal digital assistant), various remote servers (e.g., online server cloud), processing equipment, and the like. It should be noted, however, that the invention is not limited to implementation on such computing devices but may be implemented on any of a variety of different types of computing devices within the scope of embodiments of the present invention.
  • the natural language engine 290 is configured as a search engine designed for searching for information on the Internet and/or the data store 220, and for gathering search results from the information, within the scope of the search, in response to submission of the query via the client device 215.
  • the search engine includes one or more web crawlers that mine available data (e.g., newsgroups, databases, open directories, the data store 220, and the like) accessible via the Internet and build a semantic index 260 containing web addresses along with the subject matter of web pages or other documents stored in a meaningful format.
  • the search engine is operable to facilitate identifying and retrieving the search results (e.g., listing, table, ranked order of web addresses, and the like) from the semantic index that are relevant to search terms within the submitted query.
  • the search engine may be accessed by Internet users through a web-browser application disposed on the client device 215. Accordingly, the users may conduct an Internet search by submitting search terms at the search-entry area (e.g., surfaced on the UI display 295 generated by the web-browser application associated with the search engine).
  • the data store 220 is generally configured to store information associated with online items and/or materials that have searchable content associated therewith (e.g., documents that comprise the Wikipedia website). In various embodiments, such information may include, without limitation, documents, content of a web page/site, electronic materials accessible via the Internet or a local intranet, and other typical resources available to a search engine.
  • the data store 220 may be configured to be searchable for suitable access of the stored information. For instance, the data store 220 may be searchable for one or more documents selected for processing by the natural language engine 290.
  • the natural language engine 290 is allowed to freely inspect the data store for documents that have been recently added or amended in order to update the semantic index.
  • the process of inspection may be carried out continuously, in predefined intervals, or upon an indication that a change has occurred to one or more documents aggregated at the data store 220.
  • the information stored in the data store 220 may be configurable and may include any information within a scope of an online search. The content and volume of such information are not intended to limit the scope of embodiments of the present invention in any way.
  • the data store 220 may, in fact, be a plurality of databases, for instance, a database cluster, portions of which may reside on the client device 215, the natural language engine 290, another external computing device (not shown), and/or any combination thereof.
  • the natural language engine 290 provides a tool to assist users aspiring to explore and find information online.
  • this tool operates by applying natural language processing technology to compute the meanings of passages in sets of documents, such as documents drawn from the data store 220. These meanings are stored in the semantic index 260 that is referenced upon executing a search.
  • a query search pipeline 205 analyzes the user's query (e.g., a character string, complete words, phrases, alphanumeric compositions, symbols, or questions) and translates the query into a structural representation utilizing semantic relationships.
  • semantic index 260 This representation, referred to hereinafter as a "proposition,” may be utilized to interrogate information stored in the semantic index 260 to arrive upon relevant search results.
  • the information stored in the semantic index 260 includes representations extracted from the documents maintained at the data store 220, or any other materials encompassed within the scope of an online search.
  • This representation referred to hereinafter as a "semantic structure,” relates to the intuitive meaning of content distilled from common text and may be stored in the semantic index 260.
  • the architecture of the semantic index 260 allows for rapid comparison of the stored semantic structures against the derived propositions in order to find semantic structures that match the propositions and to retrieve documents mapped to the semantic structures that are relevant to the submitted query.
  • the natural language engine 290 can determine the meaning of a user's query requirements from the words submitted into a search interface (e.g., the search-entry area surfaced on the UI display 295), and then to sift through a large amount of information to find corresponding search results that satisfy those needs.
  • a search interface e.g., the search-entry area surfaced on the UI display 295.
  • the process above may be implemented by various functional elements that carry out one or more steps for discovering relevant search results. These functional elements include a query parsing component 235, a document parsing component 240, a semantic interpretation component 245, a semantic interpretation component 250, a syntactic application component 255, the semantic index 260, a matching component 265, and a ranking component 270.
  • These functional components 235, 240, 245, 250, 255, 260, 265, and 270 generally refer to individual modular software routines, and their associated hardware that are dynamically linked and ready to use with other components or devices.
  • the syntactic application component 255 can apply specific grammar rules for English.
  • the grammar is a lexical functional grammar (LFG).
  • LFG lexical functional grammar
  • any other suitable parser can be used, including those based on Head-Driven Phrase Structure Grammar (HPSG), Combinatory Categorical Grammar (CCG), Probabilistic Context-free Grammar (PCFG), or any other grammar formalism. Grammar may be provided for various languages.
  • the data store 220, the document parsing component 240, and the semantic interpretation component 250 comprise an indexing pipeline 210.
  • the indexing pipeline 210 serves to distill the semantic structure from content within documents 230 accessed at the data store 220, and to construct the semantic index 260 upon gathering the semantic structures.
  • the semantic structures may retain a mapping to the documents 230, and/or location of content within the documents 230, from which they were derived.
  • the document parsing component 240 is configured to gather data that is available to the natural language engine 290.
  • gathering data includes inspecting the data store 220 to scan content of documents 230, or other information, stored therein. Because, the information within the data store 220 may be constantly updated, the process of gathering data may be executed at a regular interval, continuously, or upon notification that an update is made to one or more of the documents 230.
  • the document parsing component 240 Upon gathering the content from the documents 230 and other available sources, the document parsing component 240 performs various procedures to prepare the content for semantic analysis. These procedures may include text extraction, entity recognition, and syntactic parsing.
  • the text extraction procedure substantially involves extracting tables, images, templates, and textual sections of data from the content of the documents 230 and to converting them from a raw online format to a format usable for the syntactic application component 255 (e.g., HyperText Markup Language (HTML), plain text, extensible Markup Language (XML), etc.), while saving links to documents 230 from which they are extracted in order to facilitate mapping.
  • the usable format of the content may then be split up into sentences.
  • breaking content into sentences involves assembling a string of characters as an input, applying a set of rules to test the character string for specific properties, and, based on the specific properties, dividing the content into sentences.
  • the specific properties of the content being tested may include punctuation and capitalization in order to determine the beginning and end of a sentence.
  • each individual sentence is examined to detect words therein and to potentially recognize each word as an entity (e.g., "The Hindenburg"), an event (e.g., "World War II”), a time (e.g., "September”), or any other category of word that may be utilized for promoting distinctions between words or for understanding the meaning of the subject sentence.
  • the entity recognition procedure assists in recognizing which words are names, as they provide specific answers to question-related words of a query (e.g., who, where, when).
  • recognizing words includes identifying words as names and annotating the word with a tag to facilitate retrieval when interrogating the semantic index 260.
  • identifying words as names includes looking up the words in predefined lists of names to determine if there is a match. If no match exists, statistical or structural information may be used to guess whether the word is a name. For example, statistical information may assist in recognizing a variation of a complex name, such as "USS Enterprise," which may have several common variations in spelling. Structural information may be used to recognize "John R. Walters" as a name, based on the knowledge that "John” is a first name, "Walters" a possible last name, and "R.” a possible initial, combined with knowledge about how names combine.
  • the parsing procedure when implemented, provides insights into the structure of the sentences identified above.
  • these insights are provided by applying rules maintained in a framework of the syntactic application component 255.
  • these rules, or grammars expedite analyzing the sentences to distill representations of the relationships among the words in the sentences.
  • these representations are referred to as syntactic structures, and allow the semantic interpretation component 250 to capture critical information about the structure of the sentence (e.g., verb, subject, object, and the like). In cases where more than one possible set of relationships between the words is possible, the output will be ambiguous.
  • the result will be one ambiguous syntactic structure. If the parser does not handle ambiguity directly, the resulting output may be a set of syntactic structures.
  • the semantic interpretation component 250 is generally configured to diagnose the role of each word in the syntactic structure by recognizing a semantic relationship between the words. Initially, diagnosing may include analyzing the grammatical organization of the syntactic structure and separating the syntactic structure into logical assertions that each express a discrete idea and particular facts. These logical assertions may be further analyzed to determine a function of each of a sequence of words that comprises the assertion. If appropriate, based on the function or role of each word, one or more of the sequence of words may be expanded to include synonyms (i.e., linking to other words that correspond to the expanded word's specific meaning) or hypernyms (i.e., linking to other words that generally relate to the expanded word's general meaning).
  • synonyms i.e., linking to other words that correspond to the expanded word's specific meaning
  • hypernyms i.e., linking to other words that generally relate to the expanded word's general meaning
  • a grammatical relationship of each of the sequence of words, and any other information about the semantic structure, recognized by the semantic interpretation component 250 may be stored at the semantic index 260.
  • this may involve either preserving the ambiguity, resulting in an ambiguous semantic structure, or in the case of a set of input syntactic structures, there may be a set of resulting semantic structures.
  • one syntactic structure may give rise to a semantic ambiguity, resulting in an ambiguous semantic structure in the case of an ambiguity enabled system like the XLE, or more than one semantic structure in non- ambiguity enabled systems.
  • the semantic index 260 serves to store the information about the semantic structure derived by the indexing pipeline 210 and may be configured in any manner known in the relevant field.
  • the semantic index may be configured as an inverted index that is structurally similar to conventional search engine indexes.
  • the inverted index is a rapidly searchable database whose entries are words with pointers to the documents 230, and locations therein, on which those words occur. Accordingly, when writing the information about the semantic structures to the semantic index 260, each word and associated function is indexed along with the pointers to the sentences in documents in which the semantic word appeared. Additionally, in the case of ambiguity, all possible semantic structures are stored in the semantic index.
  • This framework of the semantic index 260 allows the matching component 265 to efficiently access, navigate, and match stored information to recover meaningful search results that correspond with the submitted query.
  • the client device 215, the query parsing component 235, and the semantic interpretation component 245 comprise a query conditioning pipeline 205. Similar to the indexing pipeline 210, the query conditioning pipeline 205 distills meaningful information from a sequence of words. However, in contrast to processing passages within documents 230, the query conditioning pipeline 205 processes words submitted within a query 225. For instance, the query parsing component 235 receives the query 225 and performs various procedures to prepare the words for semantic analysis. These procedures may be similar to the procedures employed by the document parsing component 240 such as text extraction, entity recognition, and parsing. In addition, the structure of the query 225 may be identified by applying rules maintained in a framework of the LFG application component 255, thus, deriving a meaningful representation, or proposition, of the query 215.
  • the semantic interpretation component 245 may process the proposition in a substantially comparable manner as the semantic interpretation component 250 interprets the function structure derived from a passage of text in a document 230.
  • the semantic interpretation component 245 may identify a grammatical relationship of the words within the string of words that comprise the query 225.
  • identifying the grammatical relationship includes identifying whether a word functions as the subject (agent of an action), object, predicate, indirect object, or temporal location of the proposition of the query 225.
  • the proposition is evaluated to identify a logical language structure associated with each of the words.
  • evaluation may include one or more of the following steps: determining a function of at least one of the words; based on the function, replacing the words with a logical variable that encompasses a plurality of meanings; and writing those meanings to the proposition of the query.
  • This proposition of the query 225, the words, and the information distilled from the proposition and/or words are then sent to the matching component 265 for comparison against the structural functions extracted from the documents 230 and stored at the semantic index 260.
  • the matching component 265 compares the propositions of the queries 225 against the structural functions at the semantic index 260 to ascertain matching structural functions.
  • These matching structural functions may be mapped back to the documents 230 from which they were extracted utilizing the tags appended to the structural functions.
  • These documents 230, targeted by the tags, are collected and sorted by the ranking component 270. Sorting may be performed in any known method within the relevant field, and may include without limitation, ranking according to closeness of match, listing based on popularity of the returned documents 230, or sorting based on attributes of the user submitting the query 225.
  • These ranked documents 230 comprise the search result 285 and are conveyed to the presentation device 275 for surfacing in an appropriate format on the UI display 295. [0045] With continued reference to FIG.
  • this exemplary system architecture 200 is but one example of a suitable environment that may be implemented to carry out aspects of the present invention and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the illustrated exemplary system architecture 200, or the natural language engine 290, be interpreted as having any dependency or requirement relating to any one or combination of the components 235, 240, 245, 250, 255, 260, 265, and 270 as illustrated. In some embodiments, one or more of the components 235, 240, 245, 250, 255, 260, 265, and 270 may be implemented as stand-alone devices.
  • one or more of the components 235, 240, 245, 250, 255, 260, 265, and 270 may be integrated directly into the client device 215. It will be understood by those of ordinary skill in the art that the components 235, 240, 245, 250, 255, 260, 265, and 270 illustrated in FIG. 2 are exemplary in nature and in number and should not be construed as limiting.
  • FIG. 2 a diagram 300 representing a semantic index (e.g., the semantic index 260 from FIG. 2) is provided.
  • various components may be used therein to generate a fact-based index, thereby providing a method of storing the indexed information. Storing the information in such a fact-based index allows for more efficient retrieval of the information, for example, by the matching component 265 of FIG. 2.
  • Exemplary components within the semantic index 260 include a fact-based structure generating component 310 and an identifying component 320.
  • the fact-based structure generating component 310 is configured to generate a fact-based structure using, for example, information provided from the indexing pipeline 210 from FIG. 2. Such information has been parsed and the semantic relationship between the terms has been determined before being received at the semantic index 260. In embodiments of the present invention, as discussed above, this information is in the form of a semantic structure.
  • the identifying component 320 is configured to provide an identifier to each node, and will be discussed further below with respect to FIGS. 5 and 6.
  • the fact-based structure generating component 310 uses the semantic structure to generate fact-based structures.
  • a fact-based structure refers to a structure associated with each core element, or fact, of the semantic structure. As illustrated in FIGS. 4-6, a fact-based structure contains various elements, including nodes and edges. One skilled in the art, however, will appreciate that a fact-based structure is not limited to this specific structure.
  • Each node in a fact-based structure, as used herein represents the elements of the semantic structure, where the edges of the structure connect the nodes and represent the relationships between those elements. In embodiments, the edges may be directed and labeled, with these labels representing the roles of each node.
  • FIG. 4 illustrates a structure 400 in accordance with an embodiment of the present invention.
  • This illustrated structure is not yet a fact-based structure generated by the fact-based structure generating component 310, but represents an interim structure that the component utilizes to generate such a fact-based structure.
  • the original sentence is "Mary washes a red tabby cat.”
  • the indexing pipeline 210 in FIG. 2 has identified the words or terms and the relationship between these words or terms. In one example, these relationships for the sentence may be represented as: agent (wash, Mary) theme (wash, cat) mod (cat, red) mod (cat, tabby)
  • agent describes the relationship between Mary and wash.
  • edge 410 connecting the nodes Mary and wash is labeled as "agent.”
  • theme describes the relationship between wash and cat, and edge 420 is labeled accordingly.
  • mod indicates that the terms red and tabby modify cat. These roles are then used to label edges 430 and 440. It will be understood that these labels are merely examples, and are not intended to limit the present invention.
  • a structure is generated for each node that is the target of one or more edges.
  • cat illustrated as node 450, is referred to herein as a head node.
  • a head node is a node that is the target of more than one edge.
  • cat relates to three other nodes (e.g., wash, red, and tabby), and thus, would be a head node.
  • the structure 400 contains two facts, one around the head node wash and one around the head node cat.
  • the semantic structure illustrated by structure 400 allows the dependency between the nodes or words within the sentence to be displayed.
  • FIG. 5 the structure 400 of FIG. 4 is divided such that, with cat as a head node, only one fact within the semantic structure is illustrated as structure 500.
  • This fact-based structure illustrates the first fact in the semantic structure, one that revolves around the wash node.
  • FIG. 6 illustrates structure 600, a fact-based structure that revolves around the second fact in the semantic structure, or the cat node.
  • exactly one identifier may be assigned to each node, for example, by utilizing the identifying component 320 in FIG. 3. In embodiments of the invention, this identifier is referred to as a skolem identifier.
  • One identifier is assigned to one term, regardless of whether the term is included on more than one fact-based structure.
  • the Mary node is assigned identifier 510, as "1”.
  • the wash node is assigned identifier 515, as "2”.
  • the cat node is assigned identifier 520, as "3". Because the cat node is also included in the fact-based structure 600 in FIG. 6, it is assigned the same identifier 520.
  • Red and tabby are assigned identifiers 610 and 620, respectively.
  • each term is assigned the same identifier, but each entity is assigned the identifier.
  • An entity as referred to herein, describes different terms that represent the same thing. For example, if the sentence were "Mary washes her red tabby cat.” Her would be illustrated as a node, and although it is a different term than Mary, it still represents the same entity as Mary. Thus, in a fact-based structure of this sentence, the Mary and her node would be assigned the same identifier.
  • a method 700 for implementing a natural language search by generating fact-based structures is provided, in accordance with embodiments of the present invention.
  • a query is received having search terms therein.
  • the query may be received from any source, including, for example, an input query for a search engine that is inputted from a client device, such as client device 215 in FIG. 2.
  • a proposition is distilled from the search terms received at block 705.
  • a semantic structure is generated from passages within documents accessible to a search engine. In embodiments, this step is performed by the indexing pipeline 210 in FIG. 2.
  • a fact-based structure is generated. More than one fact-based structure may be generated from one semantic structure, as each semantic structure may comprise one or more facts.
  • a semantic structure is analyzed to determine each fact within the semantic structure.
  • a fact serves as a fundamental unit for indexing.
  • each determined fact is separated to create a fact-based structure for each fact.
  • each term or entity within each fact-based structure is assigned exactly one identifier, as indicated at block 750.
  • the proposition distilled at block 710 is compared against the fact-based structures generated at block 720. In doing so, a matching set is determined, for example, utilizing the matching component 265 in FIG. 2. When a match has been determined, passages related to those fact-based structures are identified. Because the semantic structure has been used to construct a fact-based structure, these passages will be identified and retrieved in a more efficient manner. These mapped passages are then presented at block 735, for example, utilizing the presentation device 275 in FIG. 2. [0060] The present invention has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

L'invention concerne des supports lisibles par ordinateur et un système informatique pour mettre en œuvre une recherche en langage naturel utilisant des structures factuelles, et pour générer lesdites structures factuelles. Une structure factuelle est générée au moyen d'une structure sémantique qui représente des informations, telles que du texte extrait d'un document tel qu'une page web. Un analyseur de langage naturel est généralement utilisé pour créer une structure sémantique des informations, et l'analyseur identifie des termes, ainsi que les relations entre les termes. Une structure factuelle d'une structure sémantique permet de créer une structure linéaire de ces termes et leurs relations, et de faire en sorte que des identifiants des termes maintiennent la dépendance d'une structure factuelle à une autre structure factuelle. De plus, des synonymes et des hyperonymes sont identifiés pendant la génération de la structure factuelle afin d'améliorer l'exactitude de la recherche.
PCT/US2008/074981 2007-08-31 2008-09-02 Indexation factuelle servant à une recherche en langage naturel WO2009029922A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US96948607P 2007-08-31 2007-08-31
US60/969,486 2007-08-31
US12/201,596 2008-08-29
US12/201,596 US8639708B2 (en) 2007-08-31 2008-08-29 Fact-based indexing for natural language search

Publications (2)

Publication Number Publication Date
WO2009029922A2 true WO2009029922A2 (fr) 2009-03-05
WO2009029922A3 WO2009029922A3 (fr) 2009-06-11

Family

ID=40388162

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/074981 WO2009029922A2 (fr) 2007-08-31 2008-09-02 Indexation factuelle servant à une recherche en langage naturel

Country Status (1)

Country Link
WO (1) WO2009029922A2 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6366908B1 (en) * 1999-06-28 2002-04-02 Electronics And Telecommunications Research Institute Keyfact-based text retrieval system, keyfact-based text index method, and retrieval method
US20060047632A1 (en) * 2004-08-12 2006-03-02 Guoming Zhang Method using ontology and user query processing to solve inventor problems and user problems
US7225121B2 (en) * 2002-02-20 2007-05-29 Palo Alto Research Center Incorporated Generating with Lexical Functional Grammars

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6366908B1 (en) * 1999-06-28 2002-04-02 Electronics And Telecommunications Research Institute Keyfact-based text retrieval system, keyfact-based text index method, and retrieval method
US7225121B2 (en) * 2002-02-20 2007-05-29 Palo Alto Research Center Incorporated Generating with Lexical Functional Grammars
US20060047632A1 (en) * 2004-08-12 2006-03-02 Guoming Zhang Method using ontology and user query processing to solve inventor problems and user problems

Also Published As

Publication number Publication date
WO2009029922A3 (fr) 2009-06-11

Similar Documents

Publication Publication Date Title
US8639708B2 (en) Fact-based indexing for natural language search
US8229730B2 (en) Indexing role hierarchies for words in a search index
US9449081B2 (en) Identification of semantic relationships within reported speech
CA2698105C (fr) Identification de relations semantiques dans un discours rapporte
US20090070322A1 (en) Browsing knowledge on the basis of semantic relations
US9875299B2 (en) System and method for identifying relevant search results via an index
US8291319B2 (en) Intelligent self-enabled solution discovery
US7882097B1 (en) Search tools and techniques
US20160189029A1 (en) Displaying Quality of Question Being Asked a Question Answering System
US20150254230A1 (en) Method and system for monitoring social media and analyzing text to automate classification of user posts using a facet based relevance assessment model
WO2005124599A2 (fr) Recherche de contenu dans une langue complexe telle que le japonais
Armentano et al. NLP-based faceted search: Experience in the development of a science and technology search engine
Gregorowicz et al. Mining a large-scale term-concept network from Wikipedia
WO2009035871A1 (fr) Connaissances de navigation sur la base de relations sémantiques
EP2181403B1 (fr) Indexation d'arborescence des fonctions pour des mots dans un index de recherche
CA2914398A1 (fr) Identification de relations semantiques dans un discours rapporte
WO2009029922A2 (fr) Indexation factuelle servant à une recherche en langage naturel
EP2185999A2 (fr) Mise en évidence de résultats de recherche en fonction d'une signification conceptuelle
Panchasara et al. AI Based Job Recommedation System using BERT
El idrissi esserhrouchni et al. OntologyLine: A New Framework for Learning Non-taxonomic Relations of Domain Ontology
Jeon A Reference Comments Crawler for Assisting Research Paper Writing.
CN115827829A (zh) 一种基于本体的搜索意图优化方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08828672

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08828672

Country of ref document: EP

Kind code of ref document: A2