JPH0320866A - Text base retrieval system - Google Patents

Abstract

PURPOSE:To perform retrieval with high accuracy by extracting one of the word of a questionaire, the synonym of the word, and the word or the synonym with similar coupling relation and the word having the coupling relation as a text coinciding with the content of the questionaire as a retrieval request. CONSTITUTION:At synonym developing step 5, retrieval structure is reinforced by referring to a synonym dictionary 6 for the word in structure(retrieval structure) which becomes reference to be used in the retrieval generated at structure generating step 4, and selecting the word representing meaning similar to the word. A text retrieval part 7 retrieves a text base 8 setting the retrieval structure generated up to the synonym developing step 5 as a sample, and outputs the word coinciding with the retrieval structure that is the sample as a retrieval result. At this time, morpheme analysis and syntax analysis are performed by using a word dictionary 3. In such a way, it is possible to perform the retrieval with high accuracy from the questionaire by natural language for the data base 8 in which text data of natural language is accumulated as a character code string.

Description

[Detailed description of the invention]

[Industrial Application Field] The present invention enables high-precision searches from natural language makeshift sentences against a database that stores natural language text data as character code strings (hereinafter referred to as "text base"). Regarding the text-based search power formula. [Conventional technology] As a conventional technology of this kind, for example, "
Information retrieval system IRISJ based on natural language understanding (Information Processing Society of Japan Natural Language Processing Study Group Materials NL -58- 8
．． 1986), for each text as data, keywords (
The content or characteristics of each text is expressed by assigning keywords (field name or words, etc.) to the content of the text that the user is looking for, and their logical connection relationship (AND, There is a known system that is configured to specify a search condition (OR, etc.) and extract text that satisfies the search condition. Even when the interface has an interface that accepts question sentences written in natural language, as explained in the above-mentioned literature, the user's search request is expanded into corresponding keywords by analyzing the question sentence, and the logical connection relationship between those keywords is developed. Decide and search. In other words, the text-based search is a keyword search, regardless of whether it has a natural language interface or not. In addition, as a method aimed at improving search accuracy, Kinukawa et al. (1) Automatic indexing method using Japanese sentence structure analysis, Information Processing Society of Japan Transactions Vol. 21, No. 3, 1980
), each keyword has a semantic role (
Although methods have been proposed to add the subject, object, etc. in the text, keywords are still used as clues during inspection. [Problems to be Solved by the Invention] All of the above-mentioned conventional technologies perform a search using keywords contained in the text as clues, so the accuracy of the search, that is, how accurately the text desired by the user can be retrieved, is important. There was a problem that the value was not high. Here, as a measure of search accuracy, recall rate (proportion of related text searched among the text related to the user's search request) and precision rate (proportion of the searched text out of all the text searched) are generally used. (occupied by related texts) are used. In other words, assigning keywords appropriate to the content of the text means assigning words that express the theme, gist, etc. of the text, or words that represent the main related parts, as words that represent the text. However, adding all the phrases that come to mind when a user makes a search request as keywords will result in outputting a lot of unnecessary text when searching, making it difficult to accommodate a variety of expressions while ensuring high search accuracy. It's difficult to do. Furthermore, even when it is desired to search for information in supplementary descriptions, keywords are generally not assigned to supplementary parts, so it is impossible to search by keyword search. The present invention has been made in view of the above circumstances, and its purpose is to solve the above-mentioned problems in the conventional technology, to replace keyword searches with high search accuracy, and to provide supplementary information. The object of the present invention is to provide a text-based search method that can also search written matters. [Means for Solving the Problems] The above-mentioned object of the present invention is to provide heading words and their part-of-speech information,
A word dictionary that stores grammatical information, a text base that stores documents written in natural language, an input section that inputs sentences using natural language, and divides input sentences into words (
a sentence analysis unit that analyzes the grammatical structure (syntactic analysis) of the input sentence from the part of speech information and grammatical information of the divided words; and a sentence analysis unit that analyzes the text base based on the analysis results of the sentence analysis unit. a text-based search system comprising: a synonym dictionary storing words having the same meaning or similar meaning to the word in the heading; and a text-based analysis unit that performs morphological and syntactic analysis of the text-based sentence. , a collation unit is provided that collates the sentence analysis result by the text base analysis unit and the input sentence analysis result by the sentence analysis unit, and selects one or more words to be targeted at the time of search from the input sentence; A structure generation step that generates a structure (search structure) to be a sample for a search based on the case relationship between words, and a sentence and analysis by the text base analysis unit using the search structure created in the structure generation step as a sample. A text-based search method characterized by comprising a text search step of searching the text base by comparing the result with the analysis result of the input sentence by the sentence analysis unit by the collation unit. ．． [Operation] In the text-based search method according to the present invention, a search request for a text-based search, for example, a question sentence in Japanese, is analyzed and the content of the search request is selected from among all sentences in the text base. The feature is that it extracts matching items, and instead of keyword searching, it checks whether all sentences in the text base match the search request. Also,
The reason why conventional text-based search methods have had to rely on keyword searches is that it is extremely difficult to analyze the semantic content of text during a search, and that For example, it is impossible to achieve this within a practical response time. On the other hand, in the text-based search method according to the present invention, the meaning is not extracted from the text, but the words of the question sentence, their synonyms, and their connection relationships are extracted from the text that matches the content of the question sentence as a search request. By extracting words or one of the synonyms and their combinations similar to (case relation), processing speed is increased and practical response speed is achieved. [Example] Hereinafter, an example of the present invention will be described in detail based on the drawings. FIG. 1 is a schematic flowchart of a text-based search method showing an embodiment of the present invention. In the figure, l is the input section, 1
0 indicates an analysis processing section, 3 indicates a word dictionary, 6 indicates a synonym dictionary, 7 indicates a text search section, and 8 indicates a text base. Note that the analysis processing unit IO performs sentence analysis step 2, which will be described later.
, structure generation step 4, and synonym expansion step 5. The word dictionary 3 stores information used for morphological analysis and syntactic analysis in the sentence analysis 1s2. An example of the word dictionary 3 is shown in FIG. 2, and its contents consist of a word heading, the word's part of speech, and grammatical information necessary for syntactic analysis. In the example in Figure 2, the grammatical information indicates the type of word to which the adjunct can be connected (for case particles, ``body 17!J, etc.''), and ``To the right of :j, The case information of the word to which the adjunct is connected is shown. However,
Here, it is shown using superficial case information. In addition, the similar ja word dictionary 6 stores words expressing similar meanings, and is referred to in the synonym expansion step 5, which will be described later. As shown in Figure 3, the content consists of a word heading and a collection of words with similar meanings to that word. The text base 8 is a collection of sentences to be searched, and is stored in a state that can be directly handled by a computer by some means, such as on a magnetic disk or magnetic tape. The input unit l is for inputting a search request (question) for searching text in the form of sentences in natural language. The language text is imported into the device. Sentence analysis step 2 analyzes the sentence input through the input section 1 and determines the grammatical structure of the sentence. This includes morphological analysis, which identifies and decomposes each word that makes up a sentence, and syntactic analysis, which determines the structure of a sentence based on how these words are connected. The syntactic analysis performed in step 2 extracts the case structure corresponding to each predicate in the sentence. Note that the above-mentioned syntactic analysis can be performed by preparing a case structure (case frame) corresponding to a case grammar and extracting the content of the sentence, for example, by using Fill+*ore. For an overview of this process, see rit1! by Nagao, for example. Engineering” (Shokodo, published in 1982)
The description is helpful. In the structure generation step 4, words to be used in a search are extracted based on the results of the sentence analysis step 2 described above, and based on the relationships between these words, a structure (hereinafter referred to as a "search structure") that should be a standard for use in a search is created. ) is generated. At this time, if multiple natural language expressions expressing the same content are possible, conversion to the expression that should be representative is performed as described later. The synonym expansion step 5 is the same as the structure generation step described above. Regarding the words in the search structure generated in step 4, the search structure is reinforced by referring to the synonym dictionary 6 and selecting words expressing similar meanings to the words, as described later. The text base 8 is searched using the search structure generated up to the synonym expansion step 5 as a sample, and those that match the sample search structure are output as search results.At this time, in the same way as in the sentence analysis step 2 described above. Then, morphological analysis and syntactic analysis are performed using the word dictionary 3.The operation of the text-based search method of this embodiment configured as described above will be described below when the input unit l inputs the input sentence "Search text" as follows. An example will be explained in which the input sentence is sent to later processing.In sentence analysis step 2, the input sentence is subjected to morphological analysis and syntactic analysis, and the input sentence is converted into ``text/name r4j ``wo l case particle'' ``search l Furthermore, the action of this input sentence is "search",
``Determine that the target of search J is ``text.'' At this time, syntactic analysis results regarding multiple expressions expressing substantially the same content, such as expressions using active voice and passive voice, are output for each input sentence, as shown in FIG. 4, for example. In the structure generation step 4, a "search structure" indicating one or more words to be searched and the relationships between these words is generated from the output results of the sentence analysis step 2 described above.In other words, a ``search structure'' is generated that indicates the relationship between the words. If it is shown, it will be as follows (search target text)). As mentioned above, if there are multiple expressions that substantially represent the same content, the conversion is performed to the representative structure. In other words, As shown in Figure 4, from each sentence ``Search for text'', ``Text is searched'', and ``Search for text'', I! By the process shown in 5rM, the structure of (search target text)) is generated. In the synonym expansion step 5, the words included in the above-mentioned search structure are expanded into similar words with reference to the synonym dictionary 6. For example, in the thesaurus dictionary 6, the synonym of "search" is "search", the synonym of "text" is "document",
If there is a “sentence”, the above-mentioned search structure is reinforced as ((search search) (target (text document writer))). Next, the text search unit 7 executes the text base analysis step 71 Then, sentence analysis is performed for each Bunryu in the text base 8, and in a matching step 72, the word is found to have a synonym JM open step 5.
Documents that appear in the same relationship as the search structure passed from are output as matching documents. Therefore, in the above example, ``Search for one document'' and ``J, where a document is searched'' are determined to be a match, but ``Search by text'' is determined to be a non-match. According to the above embodiment, a certain text base is searched from a natural language document, the text to be searched in natural language is specified, and the words used in the input sentence are searched using the relationship between each word in the input sentence. It is now possible to search by expanding it to its synonyms, resulting in the following effects. (1) Pre-processing of the text base is no longer necessary, and information loss due to this can be avoided. (2)
It can be used without any special knowledge. (3) You can search for items that are semantically similar. (4) It becomes possible to handle a variety of input sentences. It should be noted that the above-mentioned embodiment is shown as an example, and it goes without saying that the present invention is not limited thereto. For example, the text base analysis step 7■ and the sentence analysis step 72 realize similar functions, and can be shared in the same block (module). Furthermore, between the text base analysis step 7l and the sentence analysis step 72, the analysis processing section 1
It may be configured to have a structure generation step similar to the structure generation step fi4 of 0.

【Effect of the invention】

As explained in detail above, according to the present invention, rather than extracting the meaning from the text, the words of the question sentence, their synonyms, and their combination relationships are extracted from the text that matches the content of the question sentence as a search request. (case relation), by extracting words or one of the above-mentioned class a words and those having the combination relation, instead of a keyword search,
This has the remarkable effect of realizing a text-based search method that has high search accuracy and can also search for supplementary descriptions.

[Brief explanation of drawings]

FIG. 1 is a flowchart of a text-based search method showing an embodiment of the present invention, FIG. 2 is a diagram showing an example of the contents of a word dictionary, FIG. 3 is a diagram showing an example of the contents of a synonym dictionary, and FIG. 4 is a diagram showing an example of the contents of a synonym dictionary. is a diagram showing an example of a syntax analysis result, and FIG. 5 is a flowchart showing details of the processing of the structure generation step. 1: Input unit, lO: Analysis processing unit, 3: Word dictionary, 6: Thesaurus dictionary, 7: Text search unit, 8: Text base,
2: Sentence analysis step, 4: Structure generation step, 5: Synonym development step, 7l: Text-based analysis step,
72: Matching step - Figure 2 [Heading] [Part of speech] [Grammar information] Figure 3 [Heading] [Synonyms] Search for documents and sentences

Claims (1)

[Claims] (1) A word dictionary that stores heading words, their part-of-speech information, grammatical information, etc.; a text base that stores documents written in natural language; and an input section that inputs sentences using natural language; Divide the text into words (morphological analysis),
a sentence analysis unit that analyzes the grammatical structure (syntactic analysis) of an input sentence based on part-of-speech information and grammatical information of the divided words; and means for searching the text base based on the analysis result of the sentence analysis unit. A text-based search system comprising: a thesaurus dictionary storing words having the same meaning or similar meaning to the words in the heading; a text-based analysis unit that performs morphological analysis and syntactic analysis of the text-based sentence; and the text-based analysis unit. A collation unit is provided to compare the result of the sentence analysis by the unit with the result of the analysis of the input sentence by the sentence analysis unit, select one or more words to be searched from the input sentence, and determine the case relationship between the words. a structure generation step that generates a structure (search structure) that should be a sample for search based on the search structure; 1. A text-based search method comprising: a text search step of searching the text base by comparing the analysis result of the input sentence by the matching section with the analysis result of the input sentence by the matching section.