JPH0954781A - Document retrieving system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To retrieve a character string to be a retrieval object at a high speed by extracting only a meaningful phrase to be the retrieval object as a main index word from the output of a morpheme analyzing part and generating the index file making each main index word and the appearance positional information in a document correspond to each other. SOLUTION: An electronized document to be a retrieval object is inputted in a morpheme analyzing part 2 from a retrieval object document input part 1, the information on the phrase composing a document is obtained in the morpheme analyzing part 2 and only a meaningful phrase which can be a retrieval object is extracted as a main index word from the output of this morpheme analyzing part 2 in a main index word extraction part 3. An auxiliary index word is generated for the main index word in an auxiliary index word generation part 4. Further, the index file 21 making each index word and the appearance positional information in the document correspond to each other is generated by a leading character hash table generation/addition part 5 and an index item table generation/addition part 6. Thus, a character string to be the retrieval object can be retrieved at a high speed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document retrieval system for rapidly retrieving a designated character string from a large amount of digitized documents.

[0002]

2. Description of the Related Art With the remarkable spread of electronic media, digitization of a large amount of documents is progressing in various fields such as news reports, publications and patent applications. The following techniques have been introduced as a search method for searching such a large number of documents simply and quickly as needed. One of them is a keyword search method. This is a method in which a keyword is added in advance when a document is registered and the document is searched using the keyword. In this method, a document can be searched at high speed by using a transposed file that subtracts the relevant document from the keyword. In addition, since a certain degree of language analysis or subject analysis is performed at the time of assigning keywords, the search accuracy is good. However, manual keyword assignment requires specialized knowledge and labor, and the assignment standard varies from worker to worker, so the quality is not constant and maintenance is troublesome. Automatic extraction of keywords has been attempted, but there is a problem that a high-quality dictionary for language analysis is necessary to obtain high-quality keywords.

Then, the full-text search method has been attracting attention in place of the keyword search method. What is the full-text search method?
Keywords freely specified by the searcher by directly referencing the entire text (primary information), not the data (called secondary information) created based on the text data of the document, such as bibliographic information and keywords It is a method of searching based on. However, the full-text search method searches by scanning the entire text, so it is effective for small-scale documents, but the search time becomes a problem for large-scale documents. Therefore, although dedicated hardware has been developed, it takes time to transfer the document stored in the secondary storage to the memory, and it is difficult to bring out the full performance of the hardware. And the hardware itself is expensive and difficult to introduce easily.

Therefore, a software-based high-speed full-text search method, which is less expensive and does not depend on the hardware model, is drawing attention. Many of these methods automatically generate an index file in advance in order to speed up the search, and various methods have been developed such as storing appearance position information for each character appearing in the text. .

[0005]

By the way, in the conventional full-text search method as described above, the size of the index file is limited to the size of the text so that an arbitrary character string designated by the user can be searched at high speed. In comparison, it tends to be significantly larger. The documents to be searched are dramatically increasing and increasing in capacity, and it is impossible to deal with them if the size of the index file is too large compared to the text.

On the other hand, if the capacity of the index file is kept small, there is a risk that the search speed becomes insufficient, or search omission occurs. An object of the present invention is to store more meaningful information in a smaller-capacity index file, sufficiently cover a character string to be searched by a user in general, and enable it to be searched at high speed. To do.

[0007]

The present invention employs the following structure to solve the above problems. (Structure 1) A morpheme analysis unit that receives a digitized document that is a search target and obtains information about the words and phrases that make up the document, and outputs only the meaningful words and phrases that can be the search target from the output of this morpheme analysis unit A main index word extraction unit that extracts the main index word and an index file generation unit that generates an index file that associates each main index word with the appearance position information in the document are provided. (Explanation) An electronic document is a character code,
A document in a format that can be processed by an information processing device. The morphological analysis unit obtains information including a part of speech of a phrase by analysis. Meaningful phrases are, for example, particles alone.
It is a term that excludes words that are not meaningful by itself and that cannot normally be searched. As a result, it is possible to prevent the index file from being registered except for words and phrases that may actually be used, and to reduce the size of the index file. The appearance position information is data representing the position of the index word in the document.

When the user actually searches, not all the terms in the document to be searched are targeted for search with equal probability. There are documents that are easily searched by users and documents that are not. For example, it is rare that a search is performed using only adjuncts such as particles and conjunctions as keys. Therefore, words or phrases that are to be searched by the user are cut out from the text by appropriate processing, an index file storing the appearance position information of them is generated, and these words and phrases can be searched at high speed.

Further, by storing the appearance position information in units of characters instead of storing the information in units of characters, the capacity of the index file can be kept small while storing a lot of information. When the partial words generated from each word are stored in the index file, the capacity of the index file tends to be large, but unnecessary words are deleted by using morphological analysis when cutting out the word.

(Structure 2) Accepting an electronic document to be searched, paying attention to the character type of the characters forming the document, and a word cutting section for cutting out the words forming the document as main index words; An index file generation unit that generates an index file in which the index word and the appearance position information in the document are associated with each other. (Explanation) Focusing on the character type includes a process of detecting a special character type such as “,”, “.”, “→”, which is a delimiter in a document. Further, for example, a process of cutting out a phrase using a Chinese character that appears immediately after a hiragana as a delimiter is included. For example, in “in computer technology, ...”, “in computer” and “in technology” are cut out. The process of detecting such delimiters is simpler and faster than the morphological analysis process, and an index file can be quickly generated from a large number of documents. The main index word is cut out at the delimiter character as a boundary, and an index word including a particle or the like may be generated.

(Structure 3) A partial character string, which starts from the second or more characters of the main index word and is formed by a part of the main index word, is used as an auxiliary index word, and the auxiliary index word is associated with the appearance position information in the sentence. It is preferable to store the data in the index file. (Explanation) By making a partial character string searchable by a part of the main index word, when the main index word is a compound word including a prefix or the like, the search for that part can be speeded up.

(Structure 4) When the main index word and the auxiliary index word are composed of non-hiragana and hiragana character strings, the word length of the hiragana character string at the end of the main index word or the auxiliary index word is reduced. It is preferable to compress it. (Explanation) The word length of the search word is limited by compressing the ending part of the hiragana part which is less important in the search, and the index file can be realized with a small capacity.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to specific examples. <Specific Example 1> FIG. 1 is a block diagram of an index file generation / addition processing device of the document search system of the present invention. Here, as a first specific example, a process when morphological analysis is used to extract an index word when generating an index file will be described. The apparatus shown in FIG. 1 includes a search target document input unit 1, a morpheme analysis unit 2, a main index word extraction unit 3, and an auxiliary index word generation unit 4.
And a leading character hash table generation / addition unit 5 and an index item table generation / addition unit 6. An electronic document is input to the search target document input unit 1. A memory or the like (not shown) is provided here, and a document to be searched is temporarily stored. The morphological analysis unit 2 is a unit that divides a document into character units and collates it with a dictionary prepared in advance, analyzes the words used in the document and the syntax of the document, and obtains the degree of each word as data. It has exactly the same configuration as the device conventionally used for automatic parsing of documents. That is, the search target document input unit 1
The morpheme analysis unit 2 performs a predetermined morpheme analysis on the target document input from, and outputs the result to the main index word extraction unit 3.

The main index word extraction section 3 is a section for extracting only words of a part of speech that are easily used for retrieval, based on word information obtained as a result of morphological analysis. The attached words and punctuation marks are
Since it is rarely a search target by the user on its own, it is truncated as an unnecessary word and excluded from the index generation target. Here, for example, an independent word, an unknown word, and an alphanumeric character are subjected to index generation. Here, the word extracted as the index generation target will be referred to as a "main index word" for convenience.

The auxiliary index word generating section 4 is a part for generating auxiliary index words for all main index words. The auxiliary index word refers to a partial character string consisting of the i-th character to the end of W for the given main index word W (word length L) (0 <i <L). When the word length is L for a certain main index word, at most L-1 auxiliary index words are generated. Hereinafter, for convenience, the main index word and the auxiliary index word are collectively referred to as an index word. The description will be given with reference to FIG. The first character hash table generation / addition unit 5 and the index item table generation / addition unit 6 are based on the main index word extracted by the main index word extraction unit 3 and the auxiliary index word generated by the auxiliary index word generation unit 4. Is a part for generating the index file 21.

FIG. 2 is a block diagram of a search processing device for searching a search pattern character string designated by the user using the generated index file. This device is
A search pattern character string input unit 11, a morphological analysis unit 12,
The index word extracting unit 13, the independent index word searching unit 14, the unknown word searching unit 15, the text matching unit 16, and the result output unit 17 are included. The search pattern character string is input by an operator or the like using a keyboard or the like for document search.
The search pattern character string input unit 11 is composed of such a keyboard and other input devices. Morphological analyzer 1
Reference numeral 2 has the same configuration as that of the morphological analysis unit 2 used in FIG. 1, and the same portion can be used for generating an index file and for searching processing. The morpheme analysis unit 12 performs a morpheme analysis of the search pattern character string input for document data search.

The index word extraction unit 13 removes unnecessary words from the word list obtained as a result of the morpheme analysis, and extracts index words that are actually indexed. The independent index word search unit 14, the unknown word search unit 15, and the body text matching unit 16 are
This is the part where the actual search is performed. First, the independent index word search unit 14 selects one of the index words extracted by the index word extraction unit 13.
The index file is searched for all the parts of speech that are independent words and alphanumeric characters, and candidates for appearance position information are obtained. Next, the unknown word search unit 15 searches the index file for any of the index words determined by the index word extraction unit 13 as unknown words, and obtains candidates for the appearance position. The unknown word is a word that is not in the word dictionary used by the morphological analysis unit 12. Finally, if the search pattern character string includes unnecessary words, the body matching unit 16 uses the body to match the unnecessary words, and the result output unit 17 outputs the obtained search result. That is, for example, if a search pattern character string “computer technology” is input, “computer” and “technology” are searched in the index file. At this time, "no" is an unnecessary word. Therefore, the search result is obtained by directly looking at the text of the target document for the character string "computer technology" near the appearance position of "computer" and the appearance position of "technology".

FIG. 3 is a diagram showing the structure of the index file. The index file 21 shown in FIG. 1 includes one head character hash table 22 and N index item tables 23. N is
It is the total number of character types that appear at the beginning of all index words. The index item table 23 exists for each leading character of the index word. Each index item table stores appearance position information regarding all index words starting with a certain character, and is configured to search the appearance position information using the index word as a key.

FIG. 4 shows a storage data format of the leading character hash table 22. The first character hash table 22 is a hash table for searching the corresponding index item table using the first character of the index word as a key. Each data in the leading character hash table 22 is represented by a set of three data items. Item 31 shown in FIG. 4 is one character that appears at the beginning of the index word, item 32.
Are pointers to the corresponding index item tables. Item 33 is a pointer to the next bucket in the first character hash table. If there is no next bucket, it is null (invalid data).

FIG. 5 shows the storage data format of the index item table. Each data is represented by a set of four data items. Item 41 is a fixed length character string representing an index word, item 42 is item 41
This is a pointer to the remaining character string when the index word cannot be stored in. It is null if there is no residual character string. Item 43 represents the total number of appearance position information corresponding to this index word. The appearance position information has two types of expression formats.
One is a case where it is a value indicating a specific appearance position in the text, and the other is a case where it is a pointer to another index item in the index item table. When the appearance position information corresponding to a certain index word is stored only in the latter format, the total number of items 43 is represented by a negative number. If not, express it as a positive number. Item 44 is a pointer to a list of appearance position information.

<Operation> A specific operation of the character retrieval system of the present invention will be described below with reference to a flowchart. First, an index file is generated in advance. 6 and 7 are flowcharts showing the index file generation processing. First, in step S1, the search target document is input, and then in step S
Morphological analysis is performed in 2. Next, of the words obtained as a result of the morphological analysis, unnecessary words are deleted and main index words are extracted (step S3). The process from step S4 is repeated for each extracted main index word.

First, let W be the main index word (step S
5), the leading character hash table is searched for by the leading character a of W (step S6). If there is an index item table corresponding to the character a (step S7), the appearance position information of W is registered in the index item table (step S9). In this step S9, the index item table is searched with the key W, and if there is no data of W, the appearance position information of W is newly stored with W as a key. If there is already W data, the appearance position information of W is added to the head of the appearance position information list. Step S
If there is no index item table in step 7, a new index item table corresponding to the character a is generated in step S8, and then step S9.
I do.

Next, the auxiliary index word registration process is performed. First, in step S10, L is set to a value obtained by subtracting 1 from the word length of W, and X is set.
0 is set to W (step S11). Furthermore, L is L> 0
While the above conditions are satisfied (step S12), the processing from step S13 onward is repeated. In step S13, from the end of W to L
A character is taken to generate a partial character string X, which is used as an auxiliary index word. Then, information about X is stored in the index file. First, the head character hash table is searched with the head character a of X (step S14), and if there is an item table corresponding to the character a (step S15), the index item table is searched with X (step S16). If there is the corresponding data (step S17), the pointer to X0 is additionally registered at the end of the appearance position information list corresponding to X (step S1).
8). Since the auxiliary index words after this are already registered in the index file, the auxiliary index word corresponding to the main index word W is aborted here, and the next main index word is registered (step S4).

If there is no X data in step S17, the pointer to X0 is stored in the appearance position information list using X as a key (step S20). That is, X0: = X
(Step S21), L: = L-1 (Step S22)
Then, the process returns to step S12.

If there is no index item table corresponding to a in step S15, a new index item table corresponding to the character a is newly generated (step S19), data corresponding to X is newly generated, and its appearance position is generated. A pointer to X0 is stored as information (step S20). Further, X0: = X (step S21) and L: = L-1 (step S22) are set, and the process of generating and registering the auxiliary index word after step S12 is repeated. When the registration process for all the main index words and auxiliary index words is completed as described above, the process ends (step S
4).

FIG. 8 and FIG. 9 are flow charts showing a search process when a search is performed using the generated index file. First, a search pattern character string is input (step S1), and a morphological analysis of the character string is performed (step S).
2). Next, in step S3, the index word Wi (0 ≦ i <n) is extracted from the word list obtained as a result of the morphological analysis.
At this time, if there is a word that is truncated as an unnecessary word (step S4), flag: = in step S5.
1, otherwise flag: = 0 in step S6
And Next, the index file is searched for the obtained index word. First, the candidate set H of the appearance position information is first set to be empty (step S7).

Only independent words and alphanumeric characters are extracted from Wi and set as Xj (0≤j <m) (step S8), Xj
The distance dj between them is calculated (step S9). For example,
In the case of the search pattern character string “computer technology”, “computer” and “technology” are independent words, and “no” exists between them. In this case, the distance between them is the number of bytes of the character code of "no", that is, 2 bytes.
The appearance position information may be the place where the first character of the character string of the word appears. At this time, if the appearance position information of “computer” in “computer technology” is, for example, 301 (byte), the appearance position information of “technology” is 315 which is obtained by adding 14 bytes for 7 characters.
That is, the distance between “computer” and “technology” can be expressed as 14 bytes. Next, step S10
Search for these index terms in ~ 17. The processing is performed as follows. In step S10, j: = 0, and j
During <m (step S11), the following process is repeated. X
Process A is executed to search the index file by j (step S12). Process A will be described later with reference to FIG. If there is corresponding data (step S13), the appearance position information of Xj is registered as an element of H0 (step S14),
With reference to the elements of H and H0, only the element satisfying the distance dj is newly set as the element of H (step S15). For example, in the case of a search pattern character string "computer technology", this processing is performed in order to exclude objects whose distance between the word "computer" and "technology" is 2 bytes or more. In step S16, j: = j + 1 is set, and the process returns to step S11.

If there is no corresponding data in step S13, the search pattern character string is not found, so H is emptied in step S17, and the process proceeds to process D. After the above processing is completed for all Xj, next, Wi
If there are unknown words, search for them. In step S18 of FIG. 9, unknown words in Wi are denoted by Uj (0 ≦
j <r) and the distance dj with W adjacent to Uj is calculated in step S19. In step S20, j: = 0 is set, and in step S21, the following process is repeated while j <r. Process C is executed to search the index file by Uj (step S22). The process C is, for example, the process A.
It suffices to search the index file with Uj in the same manner as above. This will also be described in detail with reference to FIG. If there is corresponding data (step S23), the appearance position information of Uj is registered as an element of H0 (step S24), and only the elements satisfying the distance dj are newly referred to by referring to the elements of H and H0. (Step S25).

In step S26, j: = j + 1 is set, and the process returns to step S21. If there is no corresponding data in step S23, the search pattern character string is not found, so H is emptied in step S27 and the process proceeds to process D. After the above processing is completed for all Uj, finally, if there is an unnecessary word when Wi is extracted, the text matching including them is performed. Flag in step S28
If 1 is 1, it means that there is an unnecessary word. Therefore, the text of each element of H is collated, and only the element to be collated is made the element of H (step S29). Otherwise, text matching need not be done. Finally, H is output in step S30, and the process ends (step S31).

FIG. 10 shows the case where the index word is an independent word.
9 is a flowchart showing an index file search process (process A). First, let W be an index word (step S
1) The candidate set M of appearance positions is set to empty (step S2). Next, the first character hash table is searched for by the first character a of W (step S3), and if there is an index entry table corresponding to the character a (step S4), the index entry table is searched by prefix match with W ( Step S5). If there is the corresponding data (step S6), the following processing is performed.

If matching is performed by prefix matching (step S
7), using the subsequent index word of W, it is checked whether or not to further collate, and if it collates (step S8), the appearance position information is added to M (step S9), and step S5.
Return to. In this way, all the items that satisfy the index item table are extracted and the process is repeated. When all the relevant data have been taken out, M is output (step S11) and the process ends. If there is no index item table corresponding to the character a in step S4, M is emptied and the search is terminated.

<Effect of Concrete Example 1> In Concrete Example 1 described above,
The main index word is extracted from the target document data using morphological analysis. At this time, using the part-of-speech information, it is possible to actively extract words that are easy for the user to search, such as independent words, and positively exclude words that are unlikely to be search targets by themselves. This ensures that only meaningful character sequences are targeted for index generation, and that appearance position information regarding other character sequences is not stored in the index file, which significantly reduces the size of the index file. Moreover, much meaningful information can be included in the index file. Also, assuming that the user's search is often performed for noun phrases, in the first specific example, a solution for these searches can be obtained extremely quickly. Although searching for a compound word or an unknown word is somewhat inferior in speed to the above-mentioned search, it is possible to perform a sufficiently practical search by storing information about auxiliary index words.

<Specific Example 2> As a specific example 2, a process in the case where the extraction of the index word is realized by the simple phrase analysis will be described. FIG. 11 is a block diagram of the index file generation / addition processing device of the document search system of the second specific example. FIG.
The device of FIG. 1 includes a search target document input unit 51, an index word simple cutout unit 52, an auxiliary index word compression / expansion unit 53, a leading character hash table generation / addition unit 54, and an index item table generation /
It is composed of an adding unit 55. A digitized document is input to the search target document input unit 1. A memory or the like (not shown) is provided here, and a document to be searched is temporarily stored. The index word simple cutout unit 52 is a unit for extracting a phrase by appropriately and simply dividing a document. The words and phrases cut out here are called index words for convenience.

There are various methods for simply dividing a Japanese sentence into words and phrases, but here, for example, a simple cut-out by a character type is considered in which a character type such as punctuation marks, kanji, alphabets, katakana, etc. is used as a delimiter. Punctuation marks represent word breaks. In addition, by using the rule of thumb that the first character of Japanese independent words is often written in kanji, and most nouns are displayed in non-hiragana strings, simple cutout using the character type You can figure out the beginning of a phrase to some extent correctly.

Next, the auxiliary index word compression / decompression unit 53 generates an auxiliary index word for each index word cut out by the simple cut-out unit 52 and compresses it for storage in the index file.
The generation method is the same as in the case of the first specific example. There are at most L-1 auxiliary index words of a certain index word W (let's assume a word length L), each of which is a partial character string (0 <i <L) consisting of the characters from the i-th to the end of the index word. . The original index word and auxiliary index word are compressed and stored in the index file. Although various methods can be considered as a method of compressing a character string, here, for example, the following method is considered. That is, when the ending part of the index word and the auxiliary index word is composed of hiragana characters, the hiragana character part of the ending part is compressed by folding it by a certain suitable number of characters. For example, assuming that the number of compression unit characters is 4, a hiragana subsequence consisting of 9 characters is folded by 4 characters, and the compression result becomes 4 characters. Various compression methods based on folding have been known in the past. For example, compressed data is obtained by taking the logical sum of corresponding bits of a plurality of character codes. By doing this, it is possible to reduce the size of the index file by compressing the end of the auxiliary index word, and at the same time reduce the number of auxiliary index words generated from one index word, making it possible to further reduce the file size. it can.

The leading character hash table generation / addition unit 54 and the index item table generation / addition unit 55 generate an index file based on the main index word and the auxiliary index word. The structure of the index file is almost the same as that of the first embodiment. However, the index words stored in the index item table are stored in a compressed form.

FIG. 12 is a block diagram of a search processing device for searching a search pattern character string designated by the user using the generated index file. This device is
Search pattern character string input unit 61, index word simple cutout unit 62, index word compression unit 63, index file search unit 6
4, a text collation unit 65 and a result output unit 66. The search pattern character string is input by an operator or the like using a keyboard or the like for document search. The search pattern character string input unit 61 uses such a keyboard,
It is composed of other input devices. The index word simple cutout unit 62 cuts out the index word. Next, the index word compression unit 63 compresses the obtained index word. The compression method is the same as when the index file was created. The index file search unit 64 searches the index file with the compressed index word. Since the index words are compressed and searched, the solution set obtained here may include incorrect solutions. Therefore, finally, the text collation unit 65 collates the text and the result output unit 66 outputs the result.

<Operation> First, an index file is generated in advance. FIG. 13 is a flowchart showing the index file generation processing. First, in step S1, a document to be searched is input, and in step S2, an index word is simply cut out. The process after step S3 is repeated for each of the cut out index words. First, let W be an index word (step S4),
The word length of W is set to L (step S5). Further, the compression start position of the character string with respect to W is set to p (step S6).
Specifically, for example, when the first few characters of W are Chinese characters and the rest are Hiragana characters, p is the position where the Hiragana characters first appear. If p <L (step S7), the p-th and subsequent characters of W are folded and compressed to be W1 (step S8). Then, a value obtained by adding the number of compression unit characters to p is set to q (step S9).

Q is used as a generation inhibition parameter when generating the auxiliary index word. Theoretically, for an index word W with a given word length L, there are L-1 auxiliary index words, and L-1 partial characters each consisting of the i-th character to the last character of W. It is a column. However, since the method of folding and compressing the character sequence is adopted here, once the auxiliary index words for the pth and subsequent auxiliary index words have been generated for the number of compression unit characters, the rest need not be generated.
If p ≧ L in step S7, the latter half of W is a non-hiragana character, and thus compression is not performed. Therefore, in step S10, W
1: = W, and q: = L in step S11.

Next, the compression format W1 of the index word W is registered in the index file. First, the head character hash table is searched for the head character a of W1 (step S12). If there is an index item table corresponding to the character a (step S13), the data of W1 is registered in the index item table (step S15). First, the index item table is searched with the key W1, and if there is no data of W1, the appearance position information of W is newly stored with W1 as a key. If there is already W1 data, the appearance position information of W is added to the head of the appearance position information list. If there is no index item table in step S13, an index item table corresponding to the character a is newly generated (step S14) and step S15 is executed.

Next, an auxiliary index word corresponding to the index word W is generated and registered in the index file. First, step S16
Then, X0: = W1 is set, and i: = 1 is set in step S17. Moving to FIG. 14, the following processing is repeated while i ≦ q (step S18). First, a partial character string X including the characters from the i-th character to the last character of W is generated (step S1
9), X1 is obtained by compressing the p-th and subsequent hiragana strings of X
(Step S20). The head character hash table is searched with the head character a of X1 (step S21), and if there is an index entry table corresponding to the character a (step S22), the index entry table is searched with the key X1 (step S23). If data of X1 already exists (step S24), X0
A pointer to is added to the end of the appearance position information list (step S25). Since the auxiliary index words after this are already registered in the index file, the generation of the auxiliary index word corresponding to the index word W is terminated here, and the process returns to step S3 to perform the registration processing of the next index word (step S3). ).

If there is no X1 data in step S24, the pointer to X0 is stored in the appearance position information list using X1 as a key (step S27). Step S28
Is set to X0: = X1, i: = i + 1 is set in step S29, and the process returns to step S18. If there is no index item table corresponding to the character a in step S22, the character a is newly added.
After generating the index item table corresponding to (step S2
6), step S27 and subsequent steps are executed. In this way, when the registration process for all index words in the index file is completed, the process ends (step S30).

FIG. 15 is a flow chart showing the processing when a search is performed using the generated index file.
First, enter the search pattern character string (step S1),
Simple cutout of the index word is performed (step S2). First, the candidate set H of the corresponding appearance position information is set to be empty (step S3). Then, the search process from step S4 is repeated for all the index words cut out in step S2 to narrow down the candidate set of solutions.

First, let W be an index word (step S5),
The compressed character string of W is set to W1 (step S6). The compression method is the same as when the index file was created. Next, the head character hash table is searched for with the head character a of W1 (step S7), and if there is an index entry table corresponding to the character a (step S8), the index entry table is searched with the prefix match of the key W1. (Step 9). If the key W1 has data that prefix-matches (step S10), the appearance position information list of all the data is taken out and set to H0 (step S11), and H: = H∩H0 (step S12). In step S13, i: = i + 1 is set, and the process returns to step S4. In this way, when the search process is completed for all index words, each appearance position information included in H is collated with the text (step S15), a final solution is output (step S16), and the process is terminated. (Step S
17). If there is no index item table corresponding to the character a in step S8, or if there is no data that prefix matches the key W1 in step S10, H is emptied (step S
14) End the search.

<Effect of Specific Example 2> In this specific example 2, the retrieval word is cut out by using the simple cutout using the character type.
Therefore, one of the features is that the index word can be extracted by a process that is much lighter than in the first example. And yet
Since the extraction is performed using a known rule of thumb for Japanese sentences, the beginning of a word can be correctly identified with a high probability. A supplementary index word is further generated for the word cut out in this way and stored in the index file. The conventional technique of generating an index file based on the extracted words has a drawback that a character sequence that does not often form a word cannot be searched at high speed. However, according to this example, the search in such a case is also sufficient. The search can be realized at high speed.

In the second specific example, the corresponding appearance position information is stored in units of words generated by appropriately dividing the body,
Further, in addition to this, information about auxiliary index terms is stored. Therefore, as compared with the first specific example, it is possible to uniformly and quickly search any character string except for special characters such as punctuation marks. On the other hand, as compared with the specific example 1, although the auxiliary words such as auxiliary verbs are hardly searched, these pieces of information cannot be omitted. Therefore, it is necessary to access a plurality of data items by the prefix matching operation, and the index file capacity tends to be large. However, in this second specific example, an efficient compression method can be adopted for each index word, which has a suppressing effect on the generation of the auxiliary index word by compressing the word end, etc., and as a result, the size of the index file can be reduced. Can be suppressed.

[Brief description of drawings]

FIG. 1 is a block diagram of an index file generation / addition processing device.

FIG. 2 is a block diagram of a search processing device.

FIG. 3 is an explanatory diagram of a structure of an index file.

FIG. 4 is an explanatory diagram of a storage data format of a leading character hash table.

FIG. 5 is an explanatory diagram of a storage data format of an index item table.

FIG. 6 is an index file generation processing operation flowchart (No. 1).

FIG. 7 is an index file generation processing operation flowchart (No. 2).

FIG. 8 is a search processing operation flowchart (No. 1).

FIG. 9 is a search processing operation flowchart (No. 2).

10 is a flowchart of an independent word index file search operation of process A. FIG.

FIG. 11 is a block diagram of an index file generation / addition processing device.

FIG. 12 is a block diagram of a search processing device.

FIG. 13 is an index file generation processing operation flowchart (No. 1).

FIG. 14 is an index file generation processing operation flowchart (No. 2).

FIG. 15 is a search processing operation flowchart.

[Explanation of symbols]

 1 Search target document input unit 2 Morphological analysis unit 3 Main index word extraction unit 4 Auxiliary index word generation unit 5 First character hash table generation / addition unit 6 Index item table generation / addition unit

Claims (4)

[Claims] 1. A morphological analysis unit that accepts a digitized document to be searched and obtains information about the words and phrases that make up the document, and outputs only the meaningful words and phrases that can be searched from the output of this morphological analysis unit. A document including a main index word extraction unit for extracting as a main index word, and an index file generation unit for generating an index file in which each main index word is associated with appearance position information in the document. Search system. 2. An electronic document to be searched is accepted, and attention is paid to the character type of the characters forming the document,
A phrase cutout unit that cuts out a phrase forming a document as a main index word, and an index file generation unit that generates an index file in which the main index word and the appearance position information in the document are associated with each other. Document search system to do. 3. A sub-character string starting from the second and subsequent characters of the main index word, which is a partial character string of the main index word, is used as an auxiliary index word, and the auxiliary index word is associated with appearance position information in the document. 3. The document search system according to claim 1, wherein the document search system is stored in an index file. 4. When the main index word and the auxiliary index word are composed of non-hiragana and hiragana character strings, the hiragana character string at the end is compressed so that the word length of the main index word or the auxiliary index word is reduced. The document search system according to claim 2, wherein