JPH01185724A - Retriever - Google Patents

Abstract

PURPOSE:To improve the accuracy and the efficiency of retrieval by extracting a key word used in the retrieval from a query text inputted in a natural language format, and performing the retrieval after performing the analysis processing of information targeted to be retrieved. CONSTITUTION:The query text inputted to a retriever in the natural language format by a query text input means 1 is analyzed by a query text analysis means 2 provided in the retriever. In other words, a word set as a key word is extracted from the query text, and it is judged whether or not the word is defined by a dictionary. And for the word defined in the dictionary, the existence of an unrequired word which makes a noise for a retrieval object is decided, and a processing to eliminate it from the key word for retrieval is performed, and the word having relationship is drawn out by deciding the word such as a word with many meanings, a synonym, and a word with different morpheme, etc., based on the variety of a natural language, and a related word is set as the key word for retrieval. After that, the retrieval is performed on storage information in which character information set as the object to be retrieved is accumulated by only a required key word. In such a way, the efficiency of retrieval accuracy can be improved.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a search device that searches for necessary information from a storage device storing character information by inputting a question in natural language format. In particular, the present invention relates to a search device for a storage device storing document information in natural language format. The search device of the present invention can be used not only to search a database storing character information, but also to search character information stored in a storage device such as a general word processor or an office computer.

〔overview〕

The present invention provides a search device that searches text information to be searched stored in a storage device from a question text input in a natural language format, and a keyword to be used for searching from a question text input in a natural language format. Search in natural language format by extracting keywords and analyzing whether the keyword is a non-term in the search target information, whether it has a synonym, synonym, or a different form, etc., and then perform a search. The purpose is to improve the accuracy and efficiency of information processing, and also to easily search information created and stored in a normal OA information processing device or the like.

[Conventional technology]

A database system has been constructed in which character information including character information or graphic information is accumulated to form a database, and required data is retrieved by inputting a search formula from a terminal device connected to a host computer. .

In this database system, when searching for necessary information from a database, instead of inputting a search formula consisting of a logical formula directly from a terminal device, the query formula is input in natural language format. There is also a natural language information search processing method that performs a search by extracting words that serve as necessary keywords from the question expression.

Furthermore, in offices, various documents are created by various information processing devices and stored in the storage devices in the form of character information.

[Problem that the invention seeks to solve]

However, conventional database searches using natural language questions have the following problems.

First, you can only search the database using words that have already been registered in the database, called controlled words.
When a searcher creates and inputs a question, there are a limited number of keywords that can be used, and there is a problem in that it is difficult to search using unregistered words called uncontrolled words. Especially in natural language, there are many different words such as synonyms, synonyms, and paraphrases, so if these words exist in the information to be questioned, they should be replaced with controlled words specified in the database, or It is necessary to create a query formula after searching to see if it corresponds to the controlled term, and the person conducting the search is required to have specialized knowledge and skills, making it difficult to conduct the search easily and efficiently. There wasn't.

Also, if you create a question text in natural language format, keywords that are unnecessary for the search target will be included in the question formula, so if you search using the entered natural language format question text, unnecessary keywords will be included in the question format. Information is also searched and output, resulting in lower search accuracy, more time occupied by the host computer, and higher costs.

Furthermore, office combi coaters, word processors, and personal computers are now commonplace in offices, and these information processing devices are used to create various documents such as letters to branch offices and customers, and reports. , but in order to search the contents of the document stored in the storage device, anyone other than the creator of the document can only search by indexing the file name; However, it is not possible to easily search for what kind of document content has been created and stored by inputting a question in natural language format.

Furthermore, even if you try to search documents stored in these OA devices using questions in natural language format, as with the above-mentioned database search, due to the presence of unused terms, polysemous words, synonyms, etc., it is difficult to efficiently accumulate document information. could not be searched.

The present invention solves the above-mentioned problems, and enables easy searching of character information stored in various storage devices by inputting a question sentence in natural language format, and improves the efficiency of the search accuracy. The purpose is to provide a search device.

[Means for solving problems]

The present invention provides a search device that searches text information to be searched stored in a storage device using a question text input in natural language format, which extracts words constituting the question text from the input question text. , a dictionary matching determination means for determining whether the extracted word exists in the dictionary of this search device, and a dictionary matching determination means for determining whether or not the extracted word exists in the dictionary of the search device, and a dictionary matching determination means for determining whether or not the extracted word exists in the dictionary of the search device, and a dictionary matching determination means for determining whether or not the extracted word exists in the dictionary of the search device, and a dictionary matching determination means for determining whether or not the extracted word exists in the dictionary of the search device. A processing means for determining non-words to be excluded, and a consideration word determination method that considers whether the search word contains related words such as polysemous words, synonyms, or allomorphs, and uses these related words as keywords for the search. processing means, and means for performing a search based on the words constituting the question sentence processed by the dictionary matching determination means, the non-word determination processing means, or the consideration word determination processing means.

Further, this search device can be implemented as part of a word processor, office computer, personal computer, or workstation.

Further, the non-word determination processing means extracts non-word candidates from the extracted words, determines whether the word of the non-word candidate matches a usage pattern using the conjugated form, and then determines whether the non-word candidate word matches a usage pattern using the conjugated form. Preferably, it includes means for defining terms.

Further, the consideration word determination processing means determines whether polysemous words, synonyms, heteromorphic words with different written forms, and semantically related words exist in the words extracted from the question sentence, and determines the words to be examined. Preferably, it includes a means.

Further, it is preferable to include means for prompting the user to input the question again based on the result of analyzing the question to improve search accuracy.

Furthermore, it is preferable to analyze and search the question text in an interactive format.In particular, the means for performing the search is a screen that displays search results in a multi-window format and displays where the search keyword appears in the search target information. It is preferable to include means for displaying: and a screen on which search target information is displayed and the contents thereof can be scrolled by the operator.

Preferably, the means for performing a search includes means for displaying search results in a multi-window format, adding or modifying search keywords based on the displayed results, and storing the search results.

[Effect]

A question text input into the search device of the present invention in a natural language format is analyzed by a question text analysis means included in the search device.

First, a word serving as a keyword is extracted from the question sentence, and it is determined whether this word is defined by a dictionary.

The following two processes are performed for words defined in the dictionary.

One is to determine unnecessary noise that becomes noise for the search target,
This is the process of deleting keywords for search, and the other process is based on the polysemic diversity of natural language, and the process of extracting related words by determining polysemous words, synonyms, allomorphic words, etc., and extracting related words. This is a process in which words with specific characteristics are used as search keywords.

After completing this image processing, a search is performed on the stored information in which character information to be searched is stored using only the necessary keywords.

〔Example〕

The present invention will be described in detail below with reference to the drawings.

FIG. 1 is a diagram showing processing means included in the search device of the present invention.

Namely, a question text input means 1 inputs a question text in a natural language format, a question text analysis means 2 analyzes the input question text and extracts and determines a necessary search keyword, and a question text input means 1 inputs a question text in a natural language format. A search processing means 3 that can access a storage device in which characters and document information to be searched are stored based on the keywords of the question text analyzed by the analysis means 2, perform a search, and store the results; and the search results. and a search output means 4 for displaying or printing out. In this search device, it is possible to evaluate the results of the search output means 4, add or modify search keywords based on the evaluation, input the question again, and furthermore, it is possible to input the question again. The previous processing can be changed depending on the processing results of each means, such as prompting the user to input a question again.

Next, the configuration of the search device will be explained using the block diagram shown in FIG. 2 showing the system configuration of the search device according to the embodiment of the present invention.

The search device of this embodiment includes a microprocessor 12,
A common bus 11 to which this microprocessor 12 is connected
This includes a ROM 13 for storing kanji, etc., a RAM 14 for storing programs to be executed, etc., and a floppy disk control circuit 15 for connecting and controlling a floppy disk device 16 connected to this search device. This hard disk device is connected to a hard disk device 18 that can store information including dictionaries to be used, word utilization information used by the question text analysis means, and information including candidate words for unused music sheets, as well as other document information. A hard disk control circuit 17 that controls the hard disk, a line interface circuit 19 that complies with the R3232C standard and interfaces with an external line, a keyboard interface circuit 20 that is connected to the keyboard 21 and controls data transfer with the keyboard 21, and a text screen. T-VRAM 22, which is a memory that stores displayed character information, and G-VRAM 2, which is a memory that stores graphic display data.
5, a CRT control circuit 23 to which these T-VRAM 22 and G-VRAM 25 are connected, controls text display and graphic display on a dedicated high-resolution display device 24, and stores mixed control data thereof, and a printer 27. A printer interface circuit 26 is connected to the printer 27 to control and provide an interface to the printer 27, and an additional RAM board 28 for storing dictionaries and the like used in the search device are connected.

This search device can be connected to a host computer 29 via an external line via a line interface circuit 19, and when searching the database of the host computer 29, it accesses the host computer via the line interface circuit 19 and searches the database. and download data as needed. For example, when searching for a document on a floppy disk that stores a document created with another word processor, the user inserts the target floppy disk into the floppy disk device and executes the search. Furthermore, when searching for document information stored in the hard disk drive, the search is performed on the information stored in the hard disk drive. The configuration of these search devices themselves can be constructed by a system using a well-known microprocessor, and can be implemented as a part of a word processor, office computer, personal computer, or workstation.

Next, a search processing operation using this search device will be described as an example based on FIGS. 3 to 6.

FIG. 3 shows a flowchart showing the overall flow of processing when performing a search using this search device.

First, a question text is input using the above-mentioned question text input means 1 (step 31). Enter this question using the keyboard 21
is input by the searcher. When inputting this question text, a kana-kanji conversion processor can be used to input it in a natural language format as a Japanese sentence, or it can be done in the form of kanji input using the kana-kanji conversion means provided in this search terminal device. .

Next, the input question text is analyzed. This means of analyzing the question text is a feature of the present invention, and in the flowchart of FIG. .

That is, dictionary matching is performed to extract words from the input question and determine whether the words constituting the question match the dictionary used by the search device (step 32).

As a result of this dictionary matching determination, if a word constituting the question sentence is not found in the dictionary, it is determined as an unknown word (unknown word determination 37). The determined unknown word can be used as a search keyword as it is.

It is examined whether or not words included in the dictionary are unused words, and a determination process is performed (steps 33 and 34).

A word that is determined to be an unused word based on the unused word determination result is determined as an unused word (unused word determination 39).

For words that are determined not to be unused terms, a study word determination process is performed for related words such as synonyms, synonyms, allomorphs, semantically related words, or transformations by conjugation (step 35). 36). The study word determined based on this determination result is confirmed as a study word (consideration word confirmation 38). The unconfirmed study words are confirmed as unknown words (unknown words).

Through the above processing, it is determined whether the constituent words of the question sentence have been examined and analyzed to a predetermined level (step 40). For questions that have not reached a certain level, the process returns to the matching process 32 with the single Kogo dictionary that forms the question.

The reason why analysis up to a certain level is sufficient is because if the question text is long, the necessary search information cannot be obtained unless all words are considered, so the judgment is made for all words. This indicates that it is not necessary and that the search process may be performed by analyzing and determining, for example, about 90% of the words.

If the above-mentioned analysis judgment has partially reached the level in this judgment processing step 40, the search processing means 3 performs search processing for the search target information based on the analysis result (step 41).

As a result of the search processing, the corresponding information is output as the search output means 4 (step 42).

Note that the processing results of the above-mentioned non-word determination processing (steps 33 and 34) and consideration word determination processing (step 35 and 36) do not change even if the processing order is changed, so the examination word determination processing is performed first. The non-word determination process may be performed after this.

Next, the procedure for determining non-words will be explained based on FIG.

First, it is determined whether each word is an unnecessary word candidate for the search target information, that is, it is a noise candidate (
Step 50). This is to determine whether the word matches a headword, that is, a keyword, registered in the dictionary as a word that is a candidate word. If there is no unnecessary word in this judgment, the processing moves to the next word, and after examining words of sequential nature, the process enters step 35 of the consideration word judgment processing as a non-unnecessary word.

For words that are unused word candidates, we examine whether there is usage information that indicates the existence of other conjugations depending on the conjugation form. This usage information is used to determine keywords from the usage form of the word, and if usage information is available, the usage information of the word is used to determine whether or not it matches the main usage pattern specified as the usage information. Matching processing is performed based on. Based on the processing result, if the word does not match, it is determined as a non-word (non-word confirmation result 58), and if it matches, the process enters steps 54 to 56 for the non-word determination process.

If there is no usage information, the process similarly enters non-word determination processing steps 54-56.

In the unword determination process, first, it is examined whether or not there is unword determination information, and if there is, the unword determination process 55 is performed, and if there is no unword determination information, it is determined that the unword is determined (unword determination processing 55). Judgment result 58). Various conditions are set for this non-word determination information. For example, the part of speech of the word preceding the non-word candidate word is an auxiliary verb or a verb, and the word following the non-word candidate word is "ga" or "wo". , “wa”, “ni”, ’da”, and “de” are judged as non-words.Specifically, “...
"When you do...", "toki" is determined to be a non-word.

As a result of the judgment in this unused word determination process, if the word is unused, it is determined as unused, and if it is unused, it is determined as unused.

As a result, those determined to be non-words are removed from the search keywords so that the non-words are not used as search words.

Next, the study word determination process will be explained using FIG. 5.

This is because this study word determination process is a search using question sentences in natural language format, and therefore it is necessary to determine the words to be used based on the characteristics of the natural language. In other words, due to the ambiguity of natural language, one word may be used for multiple meanings, so it is necessary to decide which meaning to use, and there are synonyms that can be used for the same meaning, and these It is necessary to consider whether there are synonyms for , and to consider whether to use it in a search, and there are different forms of expression that are written as words, such as ``terigae'' and ``terigae.'' Related expressions, e.g. "the other party"
This is because there are expressions such as ``customer'' and ``customer'', and it is necessary to determine the word to be used depending on the conjugation of the word.

In this study word determination process, it is first determined whether or not there is utilization information such as word variants, synonymous forms, synonymous forms, related expression forms, and inflection forms for the selected study word candidates. Then, in step 63, it is determined whether or not it matches the main usage pattern indicated by the usage information.
．． 64), the study word is determined (result 65) or the non-examination word is determined (result 66). This is done for a certain level of words to complete the question sentence analysis.

Information such as usage information or non-word determination information, etc.
RAM 28 for expansion of search device, hard disk device 18
etc., along with the dictionary of this search device, and the above-mentioned question sentence analysis process is performed based on that information.

Furthermore, with reference to FIG. 6, a search processing step 41 for search target information will be explained.

As a result of the analysis by the question sentence analysis means, the extracted unknown words,
The study words are evaluated (step 71). If the evaluation result is insufficient to create a search formula, a question text re-input request 83 is made. If "sufficient", a search expression is generated (step 73). This search expression is a search expression composed of keywords.

, :, A search is performed on the database downloaded from the host computer and other search target information using the generated search formula (step 74). This search is performed in principle by using AND conditions on the keywords obtained from the above-mentioned analysis of the question text, and extracting search target information that satisfies the conditions, but it is not necessarily limited to AND conditions. do not have. For example, it is also possible to search for synonyms using multiple synonyms as an OR condition.

The search results are output and displayed on the CRT 24 or the printer 27, and the search results are evaluated. If the searcher evaluates that the information is not the search target information, the process returns to the question re-input request 83. To continue your search, click
Furthermore, it is evaluated whether the search formula constituent keywords are suitable for the search (step 76). Here, the information to be searched is searched using the extracted keywords, and the terms that are hit are called relevant words, and the keywords that are not hit are called non-relevant words.

In the evaluation of this search expression keyword (steps 76 and 77), if it is a matching word, the search expression is regenerated,
Perform a search.

If a non-conforming word is included in the evaluation result judgment (step 77) and the non-conforming word is a prohibited word from related word development, or if there is no related word for the non-conforming word, a search is performed that does not use the non-conforming word as a keyword. Regenerates the expression, and if the non-compatible word is not a prohibited word from related word expansion and there is a related word for the non-compatible word, regenerates a search formula that uses the related word as a keyword instead of the non-compatible word. to perform further searches.

Note that related words here refer to the concepts that include synonyms and synonyms and are subject to consideration word determination, and related words that are prohibited from being expanded into related words in the question text are words that are prohibited from being expanded into related words in the question text. , refers to the keyword when exact match is specified.

Therefore, if the searcher specifies an exact match for the keyword and it is a prohibited word, the search expression will be regenerated by specifying another □ keyword, and if the searcher did not specify an exact match. If there is no hit, if another related term exists, it will be replaced, the search formula will be regenerated, and the search will continue.

In this way, a query expression input in natural language format is analyzed, a search expression is generated, and after a search is performed, re-evaluation is performed to perform a more accurate search.

Note that the above-described question text analysis and search are executed in an interactive manner in which the analysis and search results are displayed and the processing is performed while the operator (searcher) confirms the results.

In this interactive method, a question text is input and analyzed, extracted and analyzed, and when a search is executed based on the analysis results, it is displayed in which part of the search target information the keyword used for the search appears, and the operator Displays the information as a reduced screen for understanding, and partially enlarges the information to be searched.
The content is displayed simultaneously in a multi-window format as partially enlarged screens that the operator can freely scroll through, and the searcher can search while viewing these screens while adding, subtracting, or modifying keywords.

For example, when searching the contents of a document such as a letter created using a word processor, a question text consisting of a certain number of characters entered is displayed in the bottom column of the screen of this search device, and the question text using the above-mentioned analysis means,
After extracting, deleting unnecessary words, and finalizing the study words,
With the AND condition of that keyword, this search target document,
For example, you can search for a keyword in the content of a document on a Fronbi disk, and display where the keyword appears in the document you are searching as a thumbnail on the screen, while scrolling through the entire document as necessary. The document to be searched is displayed as a partially enlarged screen on another part of the screen so that the person can see it.

Further, the above-mentioned search keywords can be added or modified as needed while viewing the display of the search results, and the search results can also be stored in a storage device such as a hard disk drive.

Next, an actual search example using this question text analysis will be explained. In addition, in this example, the maximum number of characters in the question text is approximately 3.
It is limited to about 0 characters.

First, we will discuss the effects of searching using non-word processing.

For example, if the required search information is ``Due to the extremely unstable exchange rates of major currencies this week, remittances to Europe will be temporarily suspended.'', and the search question is ``This week, remittances to all European stores except Paris will be temporarily suspended.'' What is the content of the remittance notice?'', the conventional search would search for ``Paris,'' ``All stores,'' ``
For example, "Notification for all stores: Hotels in Paris are fully booked this week due to the International Air Show, so if you are planning a business trip to the same area during this period." Please contact the general affairs department of the head office as soon as possible.''The unnecessary information was output in the search, but with this unnecessary word processing, ``all stores'' and ``notification'' were treated as unnecessary words, and ``Europe'' was changed as a synonym. and "Europe" can be processed and the information to be searched can be searched and output.

Next, we will discuss the effects of searching by identifying polysemous words.

The question is ``What is the total amount of investment in rice in our country this year?''
In this case, ``rice'' may also mean ``United States,'' so after confirming that ``rice'' is ``rice'' using the consideration word determination process, perform a search such as ``Our country's relationship with the United States.'' This prevents the search from outputting unnecessary information such as ``The total investment amounted to $X billion.'', and prevents the search from outputting unnecessary information such as ``The total investment amount is $X billion.'', instead of searching for the original search target information, ``The investment to improve rice productivity in our country this year is Y billion yen.'' ” can be searched.

Next, to explain the case of searching using related expression paths and synonyms, for example, if the information to be searched is rANA, 198
If the search question is ``When did All Nippon Airways fly into the United States since around 1997?'', there are no matching keywords in conventional searches. Therefore, the search cannot be performed, but
In this example, "about when" and "r1987" are related expressions, and "America" and "United States" and "All Nippon Airways" and r
You can search for ANAJ as a synonym.

Furthermore, a case will be described in which a search is performed using variants, synonyms, etc. For example, the necessary search target information is ``Power plant for Iraq: L/Cl8ODA in response to customer request.
I made a bid at YS. We made an agreement with the manufacturer that our company would arrange the financing and that we would bear the entire cost collection risk. ”, and the question is, “Among the business negotiations our company has with Iraq, which ones have financial conditions attached at the request of the other party?”
Traditional search: "Iraq", "request", "conditions"
However, in this example, the unnecessary information ``Japan has promised conditional emergency material aid of 10 million dollars in response to the request of the Iraqi government'' is output based on a complete match of the keyword. We treat "request" and "offer", "finance" and "finance" as synonyms, "our company" and "our company" as different forms, and "party" and "customer" as related expressions. Since the above-mentioned unnecessary information is excluded, necessary search target information can be searched.

Note that if the number of characters in the question text for performing a search is limited to a certain limit, the number of man-hours required for analysis will be reduced. The effects of the present invention can be achieved without limiting the number of characters in a sentence.

In addition, in the above embodiment, keywords are extracted and analyzed from a question text input in natural language format, and then a search expression is generated, or a search is performed using only the keyword AND condition. However, database system searches are not always logical. There is no need to use a search formula consisting of formulas (If the database is searched by parsing the sentences on the host computer side, there is no need to use a search formula consisting of expressions). However, the present invention can also be applied to a method in which a search process is performed on the search sentence in the form of a sentence after processing on the relevance of synonyms, similar words, etc.

〔Effect of the invention〕

As explained above, the present invention analyzes documents stored in a database or storage devices of various information processing devices in a natural language format, and then executes a search. Therefore, it is possible to improve search accuracy and search for necessary information. In particular, since document information stored in natural language format can be searched in the form of queries using natural language, and no specialized knowledge or skills are required for searching, anyone can easily and easily retrieve the information they need. This has the effect of allowing you to search without having to do anything.

Furthermore, when searching a database by performing grammar analysis on the host computer side, the time that the host computer is occupied can be shortened, making the search cost economical.

Furthermore, it is possible to provide a search device that can easily search documents created and stored in office automation equipment such as word processors, office computers, personal computers, and workstations by simply inputting questions in natural language format. It has become possible for anyone to easily search for documents such as letters at the office.

[Brief explanation of the drawing]

FIG. 1 is a diagram illustrating processing in an apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram of the configuration of the embodiment device. FIG. 3 is a processing flowchart of the embodiment. FIG. 4 is a flowchart for processing unused words. Figure 5 is a flowchart of the process of consideration. FIG. 6 is a search processing flowchart. 1... Question text input means, 2... Question text analysis means, 3
...Search processing means, 4...Search output means.

Claims (1)

[Claims] 1. In a search device that searches text information to be searched stored in a storage device using a question text input in natural language format, the question text is constructed from the input question text. Dictionary matching determining means for extracting a word and determining whether the extracted word exists in the dictionary of this search device; A processing means for determining non-words to be removed from the search words, and considering whether the search words contain related words such as polysemous words, synonyms, or allomorphs, and selecting these related words as search keywords. and means for performing a search based on the words constituting the question sentence processed by the dictionary matching determining means, the non-word determining processing means, or the examined word determining processing means. Characteristic search device. 2. The search device according to claim 1, implemented as part of a word processor, office computer, personal computer, or workstation. 3. The non-word determination processing means extracts non-word candidates from the extracted words, determines whether the word of the non-word candidate matches a usage pattern using the conjugated form, and then identifies non-word candidates based on the non-word determination information. Claim 1 including means for determining the term
Or the search device described in 2. 4. The consideration word determination processing means determines whether polysemous words, synonyms, heteromorphic words with different written forms, and semantically related words exist in the words extracted from the question text, and determines keywords for consideration. The search device according to claim 1 or 2, further comprising means. 5. The means for extracting words from a question sentence and determining whether the extracted words exist in the dictionary, the non-word determination processing means, the consideration word determination processing means, and the means for performing a search are all operated by the operator. 5. The retrieval device according to claim 1, wherein the retrieval device is of an interactive type, in which the retrieval device executes the processing while checking the retrieval device. 6. The means for performing a search is a screen that displays search results in a multi-window format and displays where the search keyword appears in the search target information, and a screen that displays the search target information and allows the operator to scroll its contents. 6. The search device according to claim 5, further comprising means for displaying. 7. The search device according to claim 6, wherein the means for performing the search includes means for displaying the search results in a multi-window format and adding or modifying search keywords based on the displayed results. 8. The search device according to claim 7, wherein the means for performing the search includes means for storing search results.