JPH08147328A - Method and device for retrieving document - Google Patents

Abstract

PURPOSE: To quicken registering processing to a document data base by collectively executing the additional processing of plural indexes on a memory. CONSTITUTION: Key words are extracted from all the documents and key words are assigned to each block so as to equalize the index size of each block of prescribed number of blocks on an index file 8000. At the time of registering the document, when the pertinent extracted key words are larger than a prescribed number corresponding to each block storing the indexes, the block is read from a magnetic disk 105 into a memory 104 and the additional processing of the indexes for the corresponding key word are collectively executed by the memory 4000. Writing this to the disk 105 reduces the number of accessing times to the disk 105 and when smaller than the prescribed number, only indexes corresponding to the pertinent key word are read onto the memory 4000 and the additional processing of the indexes is executed to write into the disk 105 so that an access data quantity is minimized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document search method and apparatus using an index, and a database, a document filing system, and a DTP (Desk Top).
It is applied to a publishing system or the like.

[0002]

2. Description of the Related Art In an information processing system, a searcher searches for all documents that include a specific word, that is, a query word, that a searcher wants to search from among documents that are a collection of character string data stored in a database. Is one important process. An index search method using an index is well known as a method for searching such a document. This method is "information retrieval" (Nakahara, published by The Institute of Electronics, Information and Communication Engineers, 1974) pp.203-
207 (hereinafter referred to as known example 1) or "DOCUMENT"
DATABASE "(G. James, Van N)
eastern Reinhold Co. , 198
5) Specifically described in pp.87-94. The index taken up here is composed of the document number of the document in which the keyword appears. In these index search methods, only by referring to the index of the keyword that matches the query word, the document including the keyword can be found, so that high-speed search is possible.

FIG. 2 shows an example of the index described in the known example 1. The index is configured to store the keyword number and the document number of the document in which the keyword appears, corresponding to the keyword extracted from the document. In this example, the keywords "core", "disk",
It is assumed that indexes corresponding to "computer" and "IR" are created and stored in files on the magnetic disk. When searching, the query words are "core", "disk", "computer" and "I".
Only when "R" is specified, the document number of the document in which each keyword in this index appears is read from the file in which the index is stored (hereinafter referred to as the index file). Document numbers 1 and 4 in the case of "core", document number 4 in the case where the question word is "disk", and document numbers 1, 2 and 4 in the case where the question word is "computer" are "IR"
In the case of, the document number is output as the search result like the document number 2. When registering a new document in the database, the keyword that appears in the document is extracted, and the document number of the document in which this keyword appears is additionally registered in the index corresponding to that keyword. The technology for extracting keywords from documents is described in "Trends in Automatic Indexing Research" (Morohashi, IPSJ, Vol.
5, No. 9, 1984) and "DOCUMENT DA
TABASE ”(G. James, Van Nostr
and Reinhold Co. , 1985) pp. 8
7-94. FIG. 3 shows an example of processing for adding an index corresponding to a keyword extracted using these keyword extraction techniques. In this example, it is assumed that the document number of the document to be registered is 5, and "core", "computer", and "IR" have been extracted from this document. In this index, "core", "computer"
Document number 5 is added to the indexes corresponding to "IR" and "IR", respectively. In this way, the process of adding the index corresponding to the extracted keyword is performed, whereby the document registration process is realized.

[0004]

[Problems to be Solved by the Invention] "Information Retrieval" (Nakahara, IEICE Press, 1974) pp.120-
According to 128, a secondary storage device such as a magnetic disk, which can perform random access and is large-capacity and inexpensive, is generally used as a storage device for storing an index. In the process of adding the index stored in the magnetic disk, the index corresponding to the keyword to be added is independently accessed. That is, in the index addition process, a plurality of indexes on the magnetic disk are randomly accessed (hereinafter referred to as random access). In a general operating system of a workstation, access to a magnetic disk is performed in units called logical blocks. Here, the size of the logical block is 8,192 used in the above operating system.
A byte (hereinafter abbreviated as 8 KB) is assumed. However, writing to the magnetic disk may not be performed in units of 8 KB. For example, assuming that the size of the document number to be written in the index addition process is 4 bytes per document number, data of less than 8 KB, which is smaller than the logical block, is written when registering one document. In such a case, the operating system once reads the logical block to be written from the magnetic disk into the buffer area on the main memory of the operating system. Next, the logical block is updated by writing the data to be written in a predetermined location of the logical block in the buffer area. Then, this logical block is written again on the magnetic disk. In this way, writing of less data than the logical block to the magnetic disk is realized.

An example of the operation of the magnetic disk at this time will be described with reference to FIG. First, the magnetic disk head,
The seek processing and the rotation waiting processing of the head for positioning the head of the logical block to be read are performed, and then the reading processing of the logical block to be read is performed. After that, the logical block is updated (document number addition processing), and head seek processing and rotation waiting processing for positioning the head of the magnetic disk at the position of the logical block to be read again are performed. Then, the writing process of the logical block to the magnetic disk is performed.

The value of each processing time in this figure is for a general 3.5-inch magnetic disk, and the average seek time (hereinafter referred to as the average seek time) is about 14 ms.
It is assumed that the average rotation waiting time (hereinafter referred to as the average rotation waiting time) is about 17 ms, and the read time and the write time per logical block is about 4 ms. Also, the time to write the 4-byte document number to the logical block in the buffer area is 0.001 ms.
Is assumed. In this example, it takes a total of 70 ms to write the data smaller than the logical block to the magnetic disk. From the time chart in this figure, the time spent for reading and writing from the magnetic disk is 8m.
It can be seen that the time required for the seek process for positioning the head and the rotation waiting process is 62 ms, which is about eight times as long as s. That is, the processing speed of reading and writing to the magnetic disk is 2 MB.
On the other hand, the total effective processing speed including the seek processing time and the rotation waiting processing time is about 0.13 MB / s, which means that the read / write performance of the magnetic disk cannot be obtained.

For this reason, in the index addition process performed when a document is registered, random access to the magnetic disk occurs by the number of keywords for which the addition process has occurred, so that the effective processing speed of the magnetic disk decreases and the index Will take time to add. In other words, the document search method using the index has a problem that it takes time to register the document. An object of the present invention is to speed up the index addition process performed at the time of registration and shorten the registration time.

[0008]

In order to solve the above problems, the present invention extracts a keyword from a document, creates an index based on the extracted keyword, and refers to the index corresponding to the keyword matching the query word at the time of search. In the document search method for performing a search, the index is stored in a file on a secondary storage device, the file is divided into a predetermined number of blocks, and the size of the index stored in each block is substantially equal. When a document is registered and the number of indexes added or updated to the block is equal to or more than a predetermined number, the block is read from the secondary storage device to the main storage so that At the same time, for the keyword corresponding to the index in the block, the process of adding the index is mainly stored. The additional processed block is stored in the secondary storage device, and when the number is less than the predetermined number, the index is read from the secondary storage device into the main storage and the index corresponding to the index Is added on the main memory, and the added index is stored in the secondary storage device. In addition, when assigning keywords to blocks, regarding the keywords assigned to blocks,
Of all the documents for which an index is created, the number of appearing documents of the keyword is calculated, and the sum of the number of appearing documents calculated for each keyword is calculated for each block, and the sum of the number of appearing documents in each block is calculated for each block. In, the keyword blocks are allocated so that they are almost even. Further, in a document search device for extracting a keyword from a document, creating an index based on the extracted keyword, and performing a search by referring to an index corresponding to a keyword matching a query word at the time of search, the index is stored in a secondary storage device. And a means for dividing the file into a predetermined number of blocks, an allocating means for allocating the keywords to the blocks so that the sizes of the indexes stored in the blocks are substantially equal, and a document registration process. Means for determining whether the number of indexes added or updated to the block is a predetermined number or more, and when the determination result is a predetermined number or more, the block is read from the secondary storage device to the main storage, The index addition process is performed on the main memory for the keyword corresponding to the index of Means for storing the added processed block in the secondary storage device, and when the number is less than a predetermined number, the index is read from the secondary storage device into the main storage, and the addition process of the index is mainly performed for the keyword corresponding to the index. A means for storing the index which is added to the memory and stored in the secondary storage device is provided. Further, the allocating means calculates the number of appearing documents of the keyword among all the documents for which indexes are created for the keywords assigned to the blocks, and sums up the number of appearing documents calculated for each keyword for each block. The keyword allocation is performed so that the total sum of the number of appearing documents in each block is approximately equal in each block.

[0009]

By the above means, the keywords can be extracted from all the documents, and the keywords can be assigned to the blocks so that the index size of each block of the predetermined number of blocks on the secondary storage device becomes substantially equal. When the number of the corresponding extracted keywords is greater than the specified number for each block that stores the index at the time of registration, that block is read from the magnetic disk into the memory and the index is added to the corresponding extracted keywords collectively in the memory. By writing this to the magnetic disk, the number of accesses to the magnetic disk is reduced, and when the number is less than the specified number, only the index corresponding to the relevant extracted keyword is read into the memory and the process of adding the index is performed. By writing this to a magnetic disk, It is possible to minimize the access data amount to the gas disc, very allows fast indexing addition processing, it is possible to realize a high-speed process of registration in the document database.

[0010]

First, the principle of the present invention will be described below. As an initial setting, a predetermined size area is secured on the magnetic disk as an area of an index file for storing an index, and this is divided into a predetermined number of blocks (not logical blocks). Next, the keywords corresponding to the indexes are allocated to the blocks so that the capacities of the indexes (index sizes) stored in the blocks are almost equal. When registering a document, first
A keyword is extracted from the registered document, and the keyword and the document number of the document in which it appears are stored in the main memory. Next, it is checked how many keywords assigned to each of the blocks are extracted. If the number is less than the predetermined number, the index corresponding to the keyword extracted for each block is read from the magnetic disk into the main memory as in the conventional case. Next, the document number of the document in which the keyword appears is added to the end of the index read in the main memory and stored again on the magnetic disk. If the number is equal to or more than the predetermined number, first, the corresponding block is read from the magnetic disk to the main memory. Next, only the extracted keywords assigned to the block are
The document number of the document in which the keyword appears is added to the end of the index corresponding to the keyword in the block read in the main memory. Then, this block is stored again on the magnetic disk. This series of processing,
By performing the process on all the blocks to which the extracted keywords are assigned, the process of adding the keywords extracted from the registered document to the index is performed.

As described above, with respect to a block containing a predetermined number or more of extracted keywords, the block on the magnetic disk is read into the main memory only once, and an addition process for a new keyword is performed, and then this is magnetically processed. Since it is possible to add multiple indexes just by writing to the disk, it is necessary to read the corresponding index from the magnetic disk for each keyword, perform the additional processing for that keyword, and then write this to the magnetic disk as in the past. Compared with this, it is possible to significantly reduce the time required for the process of adding an index.

The principle described above will be further described using a concrete example. An example of the index file used in this example is shown in FIG. This index file is obtained by dividing the index file shown in FIG. 2 into two blocks, block 1 and block 2, and storing them on a magnetic disk. further,
The keywords "core", "disk" and "IR" are assigned to block 1 and the keyword "computer" is assigned to block 2 so that the indexes included in the blocks are approximately equal in size, and the corresponding blocks are indexed. Is stored.

This index is subjected to a search process in the same manner as in the conventional case, without being aware of blocks at the time of search. Only when “core”, “disk”, “computer” and “IR” are designated as the query words, the document numbers of the documents in which the respective keywords appear are read from the index file. That is, when the question word is “core”, the document numbers 1, 4 and the question word are “disk”
In the case of, the document number is output as the search result, and in the case of the query word being "computer", the document numbers 1, 2, 4 and the query word being "IR", the document number is output as the search result. It

When registering a new document in the database, the above-mentioned block-conscious processing is performed as shown in FIG. The procedure will be described in detail below. First, "core" and "disk" as keywords from the document to be registered
And "IR" are extracted, and document number 5 is assumed as the document number of this document. As for the allocation of the extracted keyword blocks, the keywords “core”, “disk” and “IR” are assigned to block 1.
However, “computer” is assigned to block 2. Among these keywords, the keywords “core”, “disk” and “IR” in block 1 correspond to the keywords extracted from the document 5.
Therefore, first, block 1 is read into the main memory. Next, the keyword "core" stored in block 1
Document numbers 5 of documents in which these keywords appear are added to the end of the indexes corresponding to "disk" and "IR", respectively. After that, the block 1 stored in the main memory is written in the index file on the magnetic disk. For the block 2, since the keyword corresponding to the block 2 is not extracted from the document 5, the index addition process is not performed.
In this way, the document registration process is performed.

FIG. 7 shows a time chart of the index addition processing of the block 1. In this example, a general 3.5-inch magnetic disk is used as the magnetic disk,
It is assumed that 8 KB used in a general operating system of a workstation is used as a size of a logical block and 8 logical blocks are used as a size of a block for storing an index. Further, as the seek time and the rotation waiting time of the magnetic disk, the average seek time and the average rotation waiting time are assumed. Further, the average seek time, the average rotation waiting time, the read time of one logical block, and the write time of one logical block on the magnetic disk are assumed to be about 14 ms, about 17 ms, about 4 ms, and about 4 ms, respectively. The flow of the time chart of this figure will be described using the above values.
In this example, first, when reading the block 1 into the buffer area, access to the magnetic disk occurs, and the seek of the block and the rotation waiting process position the head of the magnetic disk at the head of the block 1. During this period, an average seek time of 14 ms and an average rotation waiting time of 17 ms are consumed. Next, the logical blocks that make up block 1, which are eight logical blocks in this case, are read into the buffer area. At this time, 32 ms, which is eight times as long as the read time for one logical block, is spent. Here, 5 is the document number of the document in which these keywords appear at the end of the index corresponding to the keywords “core”, “disk” and “IR” stored in the block 1 read in the buffer area. Each will be added. Therefore, the document number is added three times in block 1. It is assumed that the time for writing the 4-byte document number into one index on the buffer area at this time is 0.001 ms. In this case, the document number is added three times, so 3
It takes twice as long as 0.003 ms, but it is negligibly small compared to other processing times. After that, the block 1 on which the index addition processing has been performed is written to the magnetic disk. At this time, the head of the magnetic disk is positioned at a predetermined position by the seek process and the rotation waiting process. During that time, an average seek time of 14 ms and an average rotation waiting time of 17 ms are consumed. After this, eight logical blocks forming the block 1 are written on the magnetic disk, and 32 ms, which is eight times as long as one block, is consumed. With the above processing, a total of 126 ms is spent to add the index of this example. This averages 42 ms per keyword. When adding processing to the index on the magnetic disk for each keyword as in the conventional art, as shown in FIG.
It takes about s, but it is possible to shorten the document registration time by about 40% by collectively performing the index addition processing in block units as in the present invention. When registering a general document, the number of registered keywords is several tens of times as large as that in this example, so that the effect of the index addition processing in block units according to the present invention is further enhanced.

As described above, according to the present invention, the blocks on the magnetic disk are once stored in the main memory by collectively performing the process of adding the indexes in the blocks containing the predetermined number or more of additional registration keywords. Since a process of adding a plurality of indexes can be executed only by reading, a high-speed registration process in the document database can be realized.

Examples of the present invention will be described below. The configuration of the document search system to which the present invention is applied will be described with reference to FIG. This system includes a display 101, a keyboard 102, a CPU 103, a memory 104, a magnetic disk 105, and a floppy disk drive (FD).
D) 106. The display 101, keyboard 102, memory 104, magnetic disk 105, and FDD 106 are accessed by the CPU 103 via a bus. An index file 8000 is stored on the magnetic disk 105. A system control program 5000, a search interface program 6000, a registration control program 2000, a search control program 3000, a keyword assignment program 2100, an index creation registration program 2200, and an index search program 3100 are loaded in the memory 104, and a work area 4000 is secured. It The document registered in the document database of this document search system is stored in the floppy disk 107 and accessed by the CPU 103 via the FDD 106.

In this system, when the power is turned on, the CPU 103
The system control program 5000 is started by the registration control program 2000 and the search control program 3000 under the control of the system control program 5000.
Is started. First, an outline of a document registration process in the present system having such a configuration will be described. The system control program 5000 executes the registration control program 2000 according to the instruction input by the user from the keyboard 102.
To start. In the registration control program 2000,
Before registering the document, the keyword assignment program 21
00 to initialize the index file. First, according to an instruction input by the user from the keyboard 102, an index file 8000 for storing an index is secured on the magnetic disk 105 by a predetermined number of blocks and is divided into a specified number of blocks. Then, a predetermined number of documents (hereinafter referred to as seed documents) for assigning a keyword to each block are FDD10.
From floppy disk 107 to memory 104
Is read into the work area 4000 of. As a seed document, for example, when creating a newspaper article DB of 100,000 documents, several hundred to several thousand documents of the same type, that is, newspaper articles are registered. Next, the words necessary for the search are extracted as keywords from the seed document read in the work area 4000, and the number of documents that appear for that keyword is calculated. The keywords are assigned to the blocks so that the total number of documents in which the keywords appear is approximately equal among the blocks. Thereafter, the registration control program 2000 activates the index creation registration program 2200.

Indexing registration program 2200
Then, in accordance with an instruction input by the user from the keyboard 102, the registration target document stored in the floppy disk 107 is read into the work area 4000 of the memory 104 via the FDD 106. Words necessary for a search are extracted as keywords from this registered document, and the keywords and document numbers are stored in the corresponding blocks of the index file 8000.
Alternatively, the document number is registered.

Next, an outline of the document search operation in this system will be described. The system control program 500 is executed according to the instruction input by the user from the keyboard 102.
0 starts the search control program 3000 and the search interface program 6000. Thereafter, the query word input by the user from the keyboard 102 is input to the search interface program 6000 and sent to the search control program 3000. The search control program 3000 activates the index search program 3100 and sends the query word to this program. The index search program 3100 reads the document number from the index corresponding to the received query word and sends it as a search result to the search control program 3000. This search result is sent to the search interface program 6000 and displayed on the display 101 as a search result document number.

Next, the keyword assignment program 21
The configuration of 00 and the keyword allocation process will be described with reference to FIG. Keyword allocation program 2100
Is an index dividing step 2105, a repeating step 2110 for the number of seed documents, and a seed document reading step 212.
0, a keyword extraction step 2130, an appearance document number counting step 2140, and a keyword allocation step 2150. First, in the index division step 2105, the number of blocks designated by the user is read from the keyboard 102. Next, an area for the designated number of blocks is secured as the index file 8000, and this is equally divided into the designated number of blocks. Next, in the seed document reading step 2120, the FD
One seed document is read via D106 and stored in the work area 4000. Further, in the keyword extraction step 2130, words that are keywords are extracted from the read seed document, and the extracted keywords are stored in the work area 4000. The technique for extracting keywords from this document is "Trend of automatic indexing research" for Japanese documents (Morohashi, IPSJ, Vol. 2).
5, No. 9, 1984), and for English documents "DOCUMENT DATABASE"
(G. James, Van Nostrand Rei
nhold Co. , 1985) pp. 87-94. In this embodiment, these keyword extraction techniques are used as they are. In the appearing document number counting step 2140, the number of documents in which the extracted keywords appear is counted and stored in the work area 4000 in association with the keywords. Repeat step 211 for the number of seed documents
At 0, the series of processing from step 2120 to step 2140 is repeated for all seed documents. After all seed documents have been processed, keyword allocation step 21
In 50, the extracted keywords are assigned to each block so that the sum of the numbers of appearing documents is almost equal, and the block number corresponds to the keyword in the work area 400.
Store in 0.

In this way, the index size of the document database can be predicted by calculating the number of documents in which the keywords extracted from the seed document appear. In other words, the size of the index corresponding to the keyword extracted from the seed document can be calculated by multiplying the number of documents that appear in the keyword extracted from the seed document by the document number size, and the number of registered documents in the document database and the number of seed documents (sampled This is because the index size in the document database can be predicted by multiplying the ratio by the number of seed documents).
Also, if the number of documents registered in the document database increases and the sum of the sizes of all indexes assigned to blocks does not become almost equal among blocks, then from among all the documents registered in this document database The candidates for seed documents are extracted by using a method such as random number extraction, and about several percent of the total number of documents is extracted. The keywords are again assigned to the blocks based on this kind of document. By re-registering all the documents registered in the document database based on the allocation of the keywords, it becomes possible to make the sum of the sizes of the indexes allocated to the blocks substantially equal among the blocks. As another method, for blocks that have a larger sum of index sizes assigned to blocks than other blocks, reallocate the keywords assigned to these blocks to smaller blocks to make the sum of index sizes larger. It is also possible to make the blocks substantially equal. As an index when assigning a keyword to a block, in addition to the number of occurrences of the document of the keyword, the number of appearances of the keyword itself is used,
It is also possible to use the number of types of keywords. By performing the above processing, the keywords can be extracted from all the seed documents, and the keywords can be assigned to the blocks so that the index sizes of the blocks are almost equal.

Further, the keyword assigning process in the keyword assigning step 2150 will be described in detail with reference to FIG. The keyword allocation step 2150 is the extracted keyword sorting step 215.
2, block number initial setting step 2153, extracted keyword repeating step 2154, block repeating step 2155, block size determining step 2156, keyword setting step 2157, block number counting step 2158 and jump step 2159. First, in the extraction keyword sorting step 2152, the extraction keywords stored in the work area 4000 are sorted in descending order of the number of documents in which the extraction keywords stored corresponding to the extraction keywords appear.
Next, in the block number initial setting step 2153, the first block number 1 is set as the block number to be processed first. Further, in the block size determination step 2156, assuming that the extracted keyword is assigned to the block, the sum of the sizes of the indexes assigned to this block is calculated, and it is determined whether or not the predetermined block size is exceeded. Only when it does not exceed, the keyword setting step 2157 is first executed to assign the extracted keyword to this block. Next, the block number counting step 2158 is executed, 1 is added to the block number of the block to be processed, and the block number of the block to be processed next is set. In this count-up, when the block number is counted up to the final number, the first block number 1 is returned to. Furthermore, the jump step 2159 terminates the iterative processing in the block iterative step 2155, and the steps after L1 are executed. Here, the extracted keyword repeating step 2154 is executed. In the block repeating step 2155, the processing from step 2156 to step 2159 is repeated for all blocks in order from the block number. In the extracted keyword repeating step 2154, the processing from step 2155 to step 2159 is repeated for all the extracted keywords. By the series of processes described above, the extracted keywords having the largest number of appearing documents can be allocated to each block in order, so that the index size can be allocated to each block almost evenly.

Next, the index creation registration program 2
The configuration of 200 and the document registration process will be described with reference to FIG. The indexing registration program 2200 is
Document number acquisition step 2205, document number repetition step 2210, registered document number reading step 2220, keyword extraction step 2230, block number correspondence step 2240, document number counting step 2245, block number repetition step 2250, extraction keyword number determination step 2260. , Block unit index addition step 2270, keyword number repeated step 22
80 and keyword unit index addition step 2
290. First, the document number acquisition step 2
At 205, the first document number and the number of registered documents of the registered document input by the user from the keyboard 102 are read. Next, in a registered document reading step 2220, the document to be registered is stored in the work area 4000 for reading one document via the FDD 106. Then, in the keyword extraction step 2230, the keyword assignment program 21
Similar to the keyword extraction step 2130 in 00, the keyword word is extracted from the read registration document. This extracted keyword is the work area 4000
To be stored. In the block number correspondence step 2240,
By checking the block number stored in the work area 4000 in the form corresponding to the keyword in the keyword allocation step 2150, the block number of the block to which the extraction keyword is allocated is acquired, and the block number is associated with the extraction keyword, Store in 4000. If the extracted keyword is not assigned to any block, the keyword is assigned to the block having the smallest sum of the sizes of the stored indexes, and the work area 4000 is assigned with the block number corresponding to the extracted keyword. To store. afterwards,
In the document number counting step 2245, the document number is incremented to prepare for the processing of the next registered document. In the step 2210 for repeating the number of documents, the keyword extraction processing of steps 2220 to 2245 is repeated for the number of registered documents. After that, the number of extracted keywords determination step 2
At 260, with respect to the block 1 having the first block number, the number of the extracted keywords assigned to the block 1 is counted, and it is checked whether or not the predetermined number N or more. The predetermined number N is specified by the user in consideration of the performance of the magnetic disk used, the computer used, and the like. If the number of counts is equal to or greater than the predetermined number N, the next block unit index addition processing 2270 is executed. In the block unit index addition processing 2270, block 1
Is read into the work area 4000, and the index addition processing is collectively performed for all the extracted keywords assigned to the block 1. If the predetermined number N is not reached, the keyword-based index addition step 229
Execute 0. In the keyword-based index addition step 2290, only the indexes corresponding to the extracted keywords existing in the block 1 stored in the index file 8000 are added to the work area 4000.
, And the document number of the document in which the extracted keyword corresponding to the index appears is added to the index and is written again in the index file 8000. Further, repeat step 2280 for the number of keywords.
Then, the keyword unit index addition step 2 is executed until the index addition processing is completed for all the extracted keywords assigned to the block 1.
290 is repeatedly executed. In the block repetition step 2250, step 226 is performed for all blocks.
Steps 2290 to 0 are repeatedly executed to perform index addition processing. In this embodiment, additional registration of a new document is realized in this way.

Further, the above-mentioned block unit index addition processing step 2270 will be described in detail with reference to FIG. The block unit index addition processing step 2270 includes a block reading step 2272,
It comprises a keyword number repeating step 2274, an in-block index adding step 2276, and a block storing step 2278. First, in the block reading step 2272, the blocks designated by the block number repeating step 2250 are read from the index file 8000 and stored in the work area 4000. Next, in the in-block index adding step 2276, the relevant keyword is added to the end of the index corresponding to the keyword assigned to the update target block read in the work area 4000 in the block reading step 2272 among the extracted keywords. Add the document number of the appearing document. Further, in the keyword number repeating step 2274, the index adding process of step 2276 is repeated for all the corresponding keywords in the read block. After that, in the block storing step 2278, the above block for which the index addition processing has been completed is stored again in the index file 8000. As described above, in the block unit index addition processing step 2270, the index addition processing is performed in block units. in this way,
By performing the keyword index addition processing collectively for each block, the index addition processing can be performed in a short time even when the number of added keywords is large.

The keyword allocation program 21 described above
A storage example of information related to keywords stored in the work area 00 will be described with reference to FIG. In the keyword extraction step 2130, words that serve as keywords are extracted from the seed document and stored in the work area 4000. As shown in the figure, the keyword number is stored together with the keyword. In this example, the extracted keywords are “core”, “disk”,
It is assumed that "computer" and "IR" were extracted from the seed document. In this step, serial numbers are assigned as the keyword numbers in the order in which these extracted keywords are extracted, and stored in the work area 4000 in the form of this example. In this example, the keywords "core", "disk", "computer" and "IR"
Are assigned 1, 2, 3 and 4 as keyword numbers. Next, in the appearing document number counting step 2140, the number of document appearances in the seed document is counted for each keyword. At this time, the number of appearing documents is stored in a form corresponding to the keyword as shown in the figure. In this example, the numbers of documents appearing in the keywords “core”, “disk”, “computer” and “IR” are 2, 1, 3 and 2, respectively. After that, in the keyword allocation step 2150, the keywords are allocated to the blocks storing the indexes based on the number of appearing documents. The block number assigned here is associated with the keyword as shown in this figure and stored in the work area 4000. In this example, the keywords "core", "disk", "computer" and "IR"
The block numbers of the blocks to which are assigned are 1, 1, 2 and 1, respectively. In this form,
By storing the information about the keywords in the work area 4000, the information about the keywords extracted from the seed document can be managed, so that the keyword allocation process to the blocks can be realized.

The index creation registration program 2 described above
A storage example of information about keywords to be stored in the work area 200 will be described with reference to FIG. In this example, it is assumed that “core”, “disk”, and “IR” have been extracted as keywords from the registered document of document number 5. In the keyword extraction step 2230,
The keyword words are extracted from the registered document and stored in the work area 4000. At this time,
The keyword number corresponding to the extracted keyword and the block number of the allocated block are acquired from the keyword information shown in FIG. At this time, as shown in this figure, the number of the acquired allocation block and the number of the registered document that is the appearing document are stored in correspondence with the keyword. In this example, the keywords “core”, “disk”, “computer” and “IR” store 1, 2, 3 and 4 respectively as keyword numbers, and the allocation block number and the registered document number are respectively set. The block number 1 and the document number 5 are stored. Based on this information, step 22
From 50 to step 2290, index addition processing is performed. By storing the information on the keyword in the work area 4000 in such a format, it is possible to manage the information on the keyword extracted from the registered document, so that it is possible to implement the process of adding an index corresponding thereto.

As described above, according to the present invention, for each block storing an index, when the number of corresponding extraction keywords is a predetermined number or more, the block is read from the magnetic disk onto the memory and the corresponding extraction is performed. Addition of indexes to keywords Batch processing on the memory and writing this to the magnetic disk reduces the number of times the magnetic disk is accessed. When the number is less than the specified number, only the index corresponding to the relevant extracted keyword is stored in the memory. By reading the data on the top, adding the index to it, and writing it to the magnetic disk, the amount of access data to the magnetic disk can be minimized, making it possible to perform extremely high-speed index addition processing, and the document database To realize high-speed registration processing to Kill.

[0029]

As described above, according to the present invention, the number of times of access to the magnetic disk can be reduced by collectively executing the process of adding a plurality of indexes on the memory, which enables a high-speed index addition process. The registration process in the document database can be performed at high speed.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a document search system to which the present invention is applied.

FIG. 2 is a diagram showing a configuration example of an index.

FIG. 3 is a diagram showing a configuration example of an index when a keyword included in a new document is added.

FIG. 4 is a diagram showing a time chart during index addition processing.

FIG. 5 is a diagram showing a configuration example of an index divided into blocks.

FIG. 6 is a diagram showing a configuration example of an index divided into blocks when a keyword included in a new document is added.

FIG. 7 is a diagram showing a time chart at the time of index addition processing when the logical block is read eight times in a row and written eight times in a row;

8 is a diagram showing a processing procedure of a keyword assignment program 2100. FIG.

9 is a diagram showing a processing procedure of a keyword allocation step 2150. FIG.

FIG. 10 is a diagram showing a processing procedure of an index creation registration program 2200.

FIG. 11 is a diagram showing a processing procedure of an index addition processing step 2270 in block units.

FIG. 12 is a diagram showing a configuration example of a keyword management table.

FIG. 13 is a diagram showing a configuration example of a work table.

[Explanation of symbols]

 101 display 102 keyboard 103 CPU 104 memory 105 magnetic disk 106 FDD 107 floppy disk 2000 registration control program 3000 search control program 4000 work area 5000 system control program 6000 search interface program 8000 index file

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Katsumi Tada 1099, Ozenji, Aso-ku, Kawasaki-shi, Kanagawa Inside the Hitachi, Ltd. Systems Development Laboratory (72) Inventor, Kanji Kato 1099, Ozen-ji, Aso-ku, Kawasaki, Kanagawa Hitachi, Ltd. System Development Laboratory (72) Inventor Satoshi Asagawa 5030 Totsuka-cho, Totsuka-ku, Yokohama, Kanagawa Prefecture Hitachi Ltd. Software Development Division

Claims (4)

[Claims] 1. A document search method in which a keyword is extracted from a document, an index is created based on the keyword, and a search is performed by referring to an index corresponding to a keyword that matches a query word at the time of search. The file is stored in a file on the storage device, the file is divided into a predetermined number of blocks, and keywords are assigned to the blocks so that the sizes of the indexes stored in the blocks are approximately equal. When the number of indexes added or updated to the block is equal to or larger than a predetermined number, the block is read from the secondary storage device to the main memory, and the index of the keyword corresponding to the index in the block Perform additional processing on main memory,
The added block is stored in the secondary storage device, and if the number is less than a predetermined number, the index is read from the secondary storage device into the main storage, and the index addition process is performed for the keyword corresponding to the index. Is performed on the main memory, and the added index is stored in the secondary storage device. 2. The document search method according to claim 1, wherein when allocating a block of a keyword, the number of documents appearing with the keyword is calculated for all keywords indexed to the block. , Document retrieval characterized by calculating the sum of the number of appearing documents calculated for each keyword for each block, and performing the keyword block allocation so that the sum of the number of appearing documents in each block is almost equal in each block Method. 3. A document search device that extracts a keyword from a document, creates an index based on the keyword, and refers to the index corresponding to the keyword that matches the query word at the time of search to perform a search. A means for storing the file in a file on the storage device, dividing the file into a predetermined number of blocks, an allocating means for allocating the keywords to the blocks so that the sizes of the indexes stored in the blocks are substantially equal, and a document At the time of registration, means for determining whether the number of indexes added or updated to the block is a predetermined number or more, and when the determination result is a predetermined number or more, read the block from the secondary storage device to the main storage, On the main memory, the process of adding the index for the keyword corresponding to the index in the block is performed. Means for storing the added processed block in the secondary storage device, and reading the index from the secondary storage device into the main storage when the number is less than a predetermined number, and adding the index for the keyword corresponding to the index Is performed on the main memory and the added index is stored in the secondary storage device. 4. The document search device according to claim 3, wherein the allocating unit calculates the number of documents that appear for the keyword allocated to the block, out of all the documents for which an index is created, Calculate the sum of the number of appearing documents calculated for each keyword for each block,
A document search device characterized by performing keyword block allocation so that the total number of documents appearing in each block is approximately equal in each block.