JPH03248272A - Document retrieving device - Google Patents

Abstract

PURPOSE:To reduce the retrieval noise for document information retrieval by subjecting the result of character string collation to morpheme analysis and discriminating whether a character string coinciding with a collation character string is included in morphemes or not and deleting sentences which do not include it. CONSTITUTION:Retrieval of documents related to fire is inputted to a retrieval condition input device 1 as a retrieval condition, and the collation character string at this time is defined as 'fire'. Two documents, namely, an example 1 'The fire at last night is immediately extinguished.' and an example 2 'A series of cases of fire due to arson are solved.' are stored in a storage device. When receiving the collation character string 'fire' from the device 1, a retrieving device 3 collates it with character strings of documents in the device 2. Collation results in success with respect to both of examples 1 and 2. Collation results are sent to a storage device 4, and sentences which succeed in collation are stored. A morpheme analyzing device 5 subjects documents stored in the device 4 to morpheme analysis to recognize that the collation character string 'fire' is included in morphemes, and the results are stored in a secondary result storage device 7, and retrieval results are displayed on a display device 8.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to a document retrieval device using a computer.

2. Description of the Related Art In recent years, with the spread of word brochures and personal converters and the practical use of character recognition using converters, an increasing number of electronic documents have been created using these converters. For this reason, there is a growing interest in document databases that store a large amount of document information and search for document information as needed.

In conventional document databases, when searching for documents, a keyword search using keywords attached to each document is common. However, adding keywords does not keep up with the increase in the number of accumulated documents, keywords become obsolete over time, and it is not possible to handle searches using keywords that exceed the expectations of database administrators, resulting in missed searches. There were problems such as an increase in the number of people.

Against this background, document databases called full-text databases have recently attracted attention.

In other words, in a full-text database, the search conditions given by the user are compared with all the information of the stored documents, and the documents that meet the search conditions are output. There is an advantage that character strings such as sentences can be used in addition to words such as .

Problem to be Solved by the Invention However, in conventional full-text databases, search conditions are set so that matching is successful no matter where the string to be matched is located in the document. and the search conditions are successfully matched, and as a result, unnecessary documents are often output as search noise. This search noise is, for example, noise such as ``arson incident'' in response to the search condition ``fire,'' or ``article culture'' in response to the search condition ``culture.'' The shorter the length, the more it increases.

In view of the increase in search noise in full-text databases as described above, an object of the present invention is to reduce search noise when searching document information.

Means for Solving the Problems The present invention provides a search condition input device for inputting search conditions including a matching character string, a document data storage device for storing document data, and a search for the document data based on the input search conditions. a string matching device; a primary result storage device that stores the matching results of the string matching device as a primary result;
A morphological analysis device that performs morphological analysis from the primary results of the next result storage device, a dictionary storage device that stores a dictionary necessary for performing morphological analysis, and 2 that stores valid documents obtained by the morphological analysis of the morphological analysis device. It is composed of a next result storage device, and a search result display device that displays the secondary results of this secondary result storage device, and performs morphological analysis on the primary results,
Only documents in which a matching character string is included in the analyzed morphemes are outputted to the secondary result storage device. That is, in the present invention, the accuracy of the search is improved by further performing morphological analysis on the result of character string matching and deleting documents that are not included in the analyzed morphemes. In other words, in the present invention, the range in which morphological analysis is performed is reduced, and the time for morphological analysis is reduced.

Operation of the present invention With the configuration described above, a character string necessary for verification is extracted from the search conditions input by the user, and a full text search is performed from a document database using the character string to obtain a primary result. This primary result includes documents that match across words or match parts of words, which occurs because Japanese is not separated (words are not separated by spaces like in English). (1) Perform morphological analysis of these documents, determine whether the matching string is included in the morphemes, and delete those that do not. Accurate search becomes possible.

(2) Morphological analysis requires a large amount of processing, so instead of performing parsing and then matching the matching string, assume that the matching string is a morpheme, then perform morphological analysis, and then If morpheme analysis fails in the vicinity, the document is deleted because the matching character string cannot be used as a morpheme, thereby reducing processing time.

(3) In order to reduce the amount of processing required for morphological analysis, instead of searching the entire document, only the sentences that include the matching string are extracted from the punctuation marks before and after the matching string, and only that sentence is analyzed. , processing time can be shortened.

(4) In order to further reduce the amount of processing for morphological analysis,
In the present invention, the positions before and after the collated character string where the type of plain name, katakana, kanji, or alphabet differ are extracted, and the place that is likely to be a morpheme break is extracted, and only that part is analyzed. Furthermore, processing time can be reduced.

Example Hereinafter, the present invention and the details of the male side will be explained using the drawings.

FIG. 1 is a schematic diagram showing the configuration of a document search device according to the present invention, in which reference numeral 1 denotes a search condition input device for extracting matching character strings from search conditions input by the user, and 2 denotes a document storing document data. 3 receives the collated character string from the search condition input device 1 and the document data storage device 2;
A string matching device that matches strings from document data.
Reference numeral 4 denotes a primary result storage device that stores documents that have been successfully matched with strings by the character string matching device 3, and 5 designates a primary result storage device that morphologically analyzes the documents in the primary result storage device 4 so that the matched strings are included in the morphemes. Each shows a morphological analysis device that determines whether

Reference numeral 6 denotes a dictionary storage device that stores a dictionary necessary for the morphological analysis device 5 to perform morphological analysis; 7 a secondary result storage device that stores documents determined by the morphological analysis device 5; 8;
is a document search result display device of the secondary result storage device 7, and 9 is a central processing unit that performs overall control.

Next, the operation of the document search device shown in FIG. 1 will be explained using an example showing the process flow in FIG. The first match string is "fire." It is assumed that the document data storage device 2 still contains two documents: Example Sentence 1: "Yesterday's fire was extinguished quickly." and Example Sentence 2: "The series of arson incidents have been solved." . Therefore, when the character string retrieval device 3I'i receives the collated character string "fire" from the search condition input device 1, the character string is collated from the document in the document data storage device 2. In this example, example sentence 1
・Verification of both is successful. The results of the comparison are sent to the primary result storage device 4, and the sentences successfully matched by the character string search device 3 (ie, example sentences 1 and 2 in this example) are stored.

Next, the morphological analysis device 5 morphologically analyzes the document stored in the primary result storage device 4 and determines whether or not the matching character string is included in the morphemes. As shown in Figure 2, the example sentences ``1il'' are ``yesterday'', ``no'', ``fire'', ``ha'', ``soon'', ``ni'', and ``extinguishing the fire''.
Since the morphemes have been analyzed as ``LJ rta'' and ``.'', it is recognized that the matching character string ``fire'' is included in the morphemes. In addition, this morphological analysis device 5 converts example sentence 2 into ``series'', ``no'', ``arson'', ``incident'', ``wa'', ``resolved J r Ll r ta'', and ``.'' as shown in Figure '2.
The morphemes are analyzed as follows, but it is recognized that the matching character string ``fire'' is not included in these morphemes.

The dictionary that this morphological analysis device 5 requires for morphological analysis is the dictionary stored in the dictionary storage device 6.

In this manner, the sentence is morphologically analyzed by the morphological analysis device 2, and documents whose morphemes include matching character strings are stored in the secondary result storage device 7. In other words, in the above example, the secondary result is example sentence 1 that contains the morpheme.
Although only the results are stored in the secondary result storage 7,
The documents in the secondary result storage device 7 are displayed on the search result storage device 8.

As described above, according to the first embodiment of the present invention, search noise can be reduced by performing morphological analysis on the result of character string matching.

Next, FIG. 3 is an explanatory diagram of the morphological analysis method of the morphological analysis device 5 according to the second embodiment of the present invention, and the reference numeral 31 is 1
The following is an example of the resulting document, which contains the sentence ``A series of arson incidents have been resolved.'' The matching character string is ``fire.'' In this case, when the morphological analysis device 5 morphologically analyzes the same document, it considers the matching character string as a morpheme and then
Perform morphological analysis. In other words, code 32 is the collation string "
This is the result of morphological analysis when ``fire'' is regarded as a morpheme.In this case, morphological analysis is possible up to ``series'', but the morphological analysis fails at ``fire'', and the process ends there. In other words, instead of morphologically analyzing the entire sentence,
By performing morphological analysis before and after the match string, it is possible to determine whether the match string is a morpheme.

According to this second embodiment, since morphological analysis is performed assuming that the collated character string is a morpheme, a determination can be made without morphological analysis of the entire text, thereby reducing processing time.

Next, FIG. 4 shows the preprocessing of morphological analysis by the morphological analysis device 5 according to the third embodiment of the present invention, and reference numeral 41 is an example of a document of the primary result. , has solved a series of arson cases this year. ] is assumed to be included.

Before performing morphological analysis, the morphological analysis device 5 extracts a portion that includes the matching character string and is surrounded by punctuation marks, and performs morphological analysis only on that portion. Reference numeral 42 in FIG. 4 is a document after pre-processing, which reads ``A series of arson cases were solved.''

Therefore, in the morphological analysis device 5 according to the third embodiment,
Since morphological analysis is performed only on sentences surrounded by punctuation marks that include the matching character string, it is possible to make a determination without morphologically analyzing the entire text, thereby reducing processing time.

Further, FIG. 5 is an explanatory diagram of preprocessing for morphological analysis by the morphological analysis device 5 according to the fourth embodiment of the present invention, and reference numeral 51 is an example of a document as a primary result. Assume that the text contains the following sentence: ``The investigation headquarters has solved a series of arson cases this year.'' Before performing morphological analysis, the morphological analysis device 5 searches for parts where the character type (plain name, kadakana, kanji, alphabet, etc.) changes before and after the collated character string, and performs morphological analysis only on that range. In this example, regarding the front of the word, ``fire'' and ``ho'' are both kanji characters, but ``ho'' and ``no'' have different kanji, plain name, and character type, so we start the morphological analysis from ``ho''. Regarding the back part of the word, ``Koto'' and ``Kate'' are the same kanji, and ``Koto'' and ``wo'' have different character types depending on the kanji and plain name, so ``Koto'' is included in the morphological analysis. Reference numeral 52 is an example after performing such preprocessing, and the morphological analysis result is "arson incident."

This takes advantage of the fact that there are many breaks in morphemes in different character types, but since there is a possibility of confusion between kanji and plain names, it is necessary to limit this to particles only.

Therefore, in the fourth embodiment described above, morphological analysis is performed only on the range in which the character type before and after the collation character string changes, so that judgment can be made without morphological analysis of the entire text, and the processing time is shortened. .

Effects of the Invention As described above, the present invention performs morphological analysis on a document for which character string matching has been successfully performed, determines whether or not the matching string is included in the morphemes, and determines whether or not the matching string is included. By deleting things, accurate searches become possible. In addition, in the present invention, morphological analysis is performed after assuming that the matching string is a morpheme, so if morphological analysis fails, the matching string cannot be a morpheme, so the document is deleted and the morphological analysis processing amount is reduced. Can be reduced. Furthermore, in the present invention,
Processing time can be reduced by performing morphological analysis only on the part that includes the matching character string and is surrounded by the preceding and following punctuation marks. Furthermore, according to the present invention, positions where the character types (plain name, katakana, kanji, alphabet, etc.) before and after the collation string are different are searched for, and morphological analysis is performed only for that range, reducing processing time. It becomes possible.

[Brief explanation of drawings]

FIG. 1 is a schematic configuration diagram of a document retrieval device according to a first embodiment of the present invention, FIG. 2 is a processing flow diagram of the document retrieval device, and FIG.
The figure is a processing example diagram of the morphological analysis device according to the second embodiment of the present invention, FIG. 4 is a processing example diagram of the morphological analysis device according to the third embodiment of the present invention, and FIG. It is a processing example diagram of the morphological analysis device according to the embodiment. ■...Search condition input device, 2...Document data storage device, 3...Character string matching device, 4...Primary result storage device, 5...Morphological analysis device, 6...Secondary Result storage device, 7... Secondary result storage device, 8... Search result display device, 9... Central control device, 31 Primary result example, 32
Morphological processing example, 41.51・Primary result example, 42.52
Pretreatment example.

Claims (4)

[Claims] (1) A search condition input device for inputting search conditions including a matching character string, a document data storage device for storing document data, a string matching device for searching the document data based on the input search conditions, and a character string matching device for searching the document data based on the input search conditions; The collation result of the column collation device is 1
a morphological analysis device that performs morphological analysis from the primary result of the primary result storage device; a dictionary storage device that stores a dictionary necessary for performing morphological analysis; It consists of a secondary result storage device that stores valid documents obtained by the morphological analysis of the morphological analysis device, and a search result display device that displays the secondary results of the secondary result storage device, and the primary result is morphologically analyzed. A document retrieval device characterized in that the secondary result storage device outputs only documents in which a matching character string is included in the analyzed morphemes. (2) The morphological analysis device performs morphological analysis assuming that the collated character string is a morpheme, and outputs only documents for which the analysis is successful to the secondary result storage device. Document search device. (3) The morphological analysis device does not perform morphological analysis on the entire document in the primary result storage device, but performs morphological analysis only on sentences that include the matching character string and are surrounded by punctuation marks. 2. The document retrieval device according to claim 1, wherein the document retrieval device performs the following steps. (4) The morphological analysis device does not perform morphological analysis on the entire document in the primary result storage device, but uses character types (plain name, katakana, kanji, alphabet, etc.) before and after the collated character string. 2. The document retrieval device according to claim 1, wherein the device searches for different positions of the character strings and performs morphological analysis only on character strings within that range.