JP2002132789A - Document retrieving method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To inexpensively provide a system capable of retrieving all sentences at high speed even when a phrase containing a high frequency word is designated as a retrieval term concerning a large scale document database described in English, German or French phonograms. SOLUTION: A list (1210) of high frequency words ('a', 'the' and 'of' or the like) is provided and at the time of registering a document on a database, a word is extracted (1120). Concerning this word, an index for retrieval having a document identifier and the position of the word in the document is prepared (1140) and while referring to the list concerning the extracted word, when the word is a high frequency word (1130), the index for retrieval is prepared in respect to the set of one word following this word (1140). At the time of retrieval, when there is a high frequency word in the words in the retrieval term, the set of one word following the word, which is a high frequency word, is handled as a word in the retrieval term and retrieving processing is performed by the retrieval term containing the set of this word.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a full-text search method for searching a document containing a specified character string in a large-scale document database, and particularly relates to phonetic characters such as English, German and French. It is applied to a database, a document management system, a document filing system, a DTP (Desk Top Publishing) system, and the like for the documents described in.

[0002]

2. Description of the Related Art In recent years, with the rapid progress of the information society,
Electronic document information created using a word processor, a personal computer, or the like is also increasing explosively. Under such circumstances, there is an increasing demand for searching a document containing necessary information at high speed and with high accuracy from a huge group of stored electronic documents. There is a full-text search as a technique to meet such a demand. In a full-text search, when a document is registered, the entire text in the document to be registered is input to a computer system and is converted into a database. At the time of search, all documents including a character string specified by a user (hereinafter referred to as a search term) from the database are searched. Without searching for keywords during registration,
It is possible to search for a target document without omission.

In general, a method called a word index method is used for full-text search of a document described in phonetic characters such as English, German, and French. An outline of the word index method will be briefly described. In documents written in phonetic characters such as English, German, and French, words are generally separated by characters such as "" (space), "," (comma), and "." (Period). That is, by extracting these characters as word breaks, words can be extracted from the registration target document. Then, with respect to these words, document identification information for identifying the relevant document in the document database and the position of the appearing word in the relevant document are extracted and stored as search index data. At the time of the search, the search index is referred to for each of the phrases specified in the search condition, the word group specified as the search term is in the same document, and the sequence of each word is the same as the sequence of words in the search condition. By extracting identical documents, full-text search without search noise is realized.

[0004] However, in the full-text search system according to this method, when a search term includes a word having a high appearance frequency in a registered document (hereinafter, referred to as a high-frequency word), the search response is reduced. There is. That is, since index data for high-frequency words is significantly larger than ordinary words, the time required to refer to index data stored on secondary storage such as a magnetic disk becomes longer, and the search takes longer. I will. Frequently used words in English are connectives such as "a" and "the";
There are connectives such as “d” and “but”, and prepositions such as “of” and “to.” These words are included in most of the registered documents and have few meanings as search terms. Generally, `` Information Retrieval '' (PRENTICE
HALL issued, William B. Frakes, Ricardo Baeza-Yates
As shown on page 113 et seq. (Hereinafter referred to as well-known example 1), a method has been proposed in which a corresponding word is defined as a stop word (excluded word) and a search index is not created for these words. ing.

[0005]

However, the English sentence search method in the prior art example 1 has a problem that it is not possible to search for a phrase including a stop word. That is, definite articles such as "a" and "the", and "and"
Although few connectives such as "t" and prepositions such as "of" and "to" have meaning as search terms, there are many cases where phrases containing them have meaning as search terms. For example, definite articles " As for "the", "the Times" which represents "Times" means the original meaning starting with "the", and "Times" without "the" has only meaning as "number of times" etc. In other words, if you can't find a phrase that contains "the", you can only search for "Times"
There is a problem that unnecessary documents used in the meaning of “number of times” are searched as noise. “The Whit
e House ”(White House),“ the West ”(Western),“ T
“He East” is an example of a word that has a predetermined meaning only when accompanied by “the”.

An example of the conjunction "and" is, for example, a phrase "br" representing "bread with butter".
ead and butter ”cannot be searched. Therefore, a search is performed using the AND condition of“ bread ”and“ butter ”, but“ bread ”and“ butter ”are included, but“ bread and
Unwanted documents that are not used in the phrase "butter" are searched for as noise.
Similarly, for "f", for example, the phrase "Bank of America" representing "American Bank" cannot be searched, so the search is performed using the AND condition of "Bank" and "America". And “America” but “Banko
There is a problem that unnecessary documents not used as the phrase "f America" are searched for as noise.

In order to solve this problem, in the conventional method, definite articles such as “a” and “the”, conjunctions such as “and” and “but”, and prepositions such as “of” and “to” are also used. To create a search index, for example, when searching for the phrase "the East", it is necessary to read out index data relating to "the", which has a very large capacity, from a secondary storage such as a magnetic disk. Then, it is necessary to compare the index data relating to “East” with the word position and to extract those that are in the same document and have the word position appearing one word adjacent thereto.
The search response is significantly reduced as compared with the case of searching for one word. That is, the problem to be solved by the present invention is that, for a large-scale document database described in phonetic characters such as English, German, and French, definite articles such as "a" and "the" and "and" If a search term contains a phrase that contains a conjunction, such as や or “but,” and a frequent word, such as a preposition, such as ofof ’or toto’
An object of the present invention is to provide an inexpensive high-speed full-text search system.

[0008]

In order to solve the above-mentioned problems, a document search method according to the present invention includes the following steps. That is, in the document search method according to the present invention, the document registration process includes a registration word extraction step of extracting words by identifying a delimiter such as a space, a comma, or a period for the registration target document; A high-frequency word determining step of determining whether or not the word is a word defined as a high-frequency word; and if the determination result in the high-frequency word determining step is a high-frequency word, One word connected to a word is extracted, search index data is created for the word string composed of the high-frequency word and the extracted one word, and the determination result in the high-frequency word determination step is a high-frequency word. If not, a search index creation / registration step of creating and registering search index data for the extracted word is provided. In the document search method according to the present invention, the registered document search process includes a search word extraction step of extracting a word by identifying a delimiter such as a space, a comma or a period with respect to a search term; A high-frequency word determining step of determining whether or not the word is a word defined as a high-frequency word; and if the determination result in the high-frequency word determining step is a high-frequency word, When one word connected to a word is extracted, search index data is used for the word string composed of the high-frequency word and the extracted one word, and the result of the determination in the high-frequency word determination step is not a high-frequency word Has a search execution step of searching for a document including a specified search term by referring to the search index data related to the extracted word.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment to which the present invention is applied will be described with reference to the drawings. First, a system configuration of the present embodiment will be described. FIG. 1 is a diagram showing an overall configuration of a first embodiment of a document search system according to the present invention. As shown in FIG. 1, the document search system according to the present embodiment includes a registration subsystem 1000, a document search server 2000, search clients 3000 and 4000,
It comprises a network 5000. The document registration subsystem 1000 receives a document to be registered as an input,
Create search indexes required for searching. The index data is transferred to the document search server 2000 via the network 5000, and later transferred to the document search server 2
000 is used when performing full-text search processing. The document search server 2000 includes search clients 3000 and 4
000, and retrieves a document that meets the conditions specified by the retrieval command using the retrieval index created by the document registration subsystem 1000, and sends the retrieval result data back to the requesting retrieval client. The search clients 3000 and 4000 display a screen on which a user interactively specifies search conditions on a display, and the search conditions specified by the user on this screen are converted to search commands that can be interpreted by the document search server 2000. And convert this command to network 5000
To the document search server 2000 via the. When the document search server 2000 performs the search processing corresponding to the search command as described above and returns the search result data,
The search clients 3000 and 4000 present the received search result data to the user as a search result screen. Although FIG. 1 shows a configuration example in which two computers 3000 and 4000 are connected as search clients, a configuration in which only one search client is used may be adopted. Also, a configuration of three or more units can be adopted. Finally, the network 5000 can be a local area network (LAN) or a wide area network (WAN)
The document registration subsystem 1000, the document search server 2000, and the search clients 3000 and 4000 are used to exchange various data and commands.

In FIG. 1, the document registration subsystem 10
00, the network 5000 is used to transfer the search index data to the document search server 2000. Instead, a floppy (registered trademark) is used.
A configuration using a portable medium such as a disk or a magneto-optical disk may be adopted. Alternatively, a configuration in which the document registration subsystem 1000 and the document search server 2000 are mounted on one computer and data transfer is not performed may be adopted. Further, in FIG.
Although 0 and 4000 are configured on a separate computer from the document search server 2000, it is also possible to adopt a configuration in which one or more search clients are executed on the same computer as the document search server 2000. The above is the description of the system configuration in the present embodiment.

Next, the document registration subsystem in this embodiment, that is, the reference numeral 1000 in FIG. 1 will be described with reference to the drawings. FIG. 2 is a diagram illustrating a configuration of the document registration subsystem 1000 according to the present embodiment. The document registration subsystem 1000 shown in the figure includes a display 1010 for displaying the execution status of the processing, a keyboard 1020 for inputting a registration command and the like, a central processing unit CPU 1030 for executing the registration processing, and reading data from a floppy disk. A floppy disk driver 1040, a floppy disk 1050 storing document data to be registered in the database, a main memory 1060 temporarily storing a registration program and data, and a magnetic disk 1070 storing various data and programs
And a bus 1080 connecting them. The main memory 1060 has a system control program 11
00, registration control program 1110, registration word extraction program 1120, registration high frequency word determination program 11
30, the search index creation / registration program 1140 is read from the magnetic disk 1070, and the work area 1150 is secured. The magnetic disk 1
In 070, a delimiter table storage area 1200, a high-frequency word list storage area 1210, a search index storage area 1220, and various program storage areas 1230 are secured. In the present embodiment, these storage areas are secured on the magnetic disk 1070, but may be another secondary storage device such as a magneto-optical disk device. The above is the configuration of the document registration subsystem 1000.

Next, the procedure of the document registration process in this embodiment will be described. First, the system control program 1100 activates the registration control program 1110 in response to a registration command input from the keyboard 1020, and starts document registration processing. Hereinafter, a process at the time of document registration will be described using a PAD (Problem Analysis Diagram) shown in FIG. The registration control program 1110 repeatedly executes a series of processes shown from Step 1310 to Step 1350 for all the registration target documents stored on the floppy disk 1050 (Step 130).
0). First, in step 1310, one unprocessed document is selected from the registration target document group stored in the floppy disk 1050 through the floppy disk driver 1040, and is read out to the work area 1150 on the main memory 1060. Next, step 1320 is executed, and step 1 is executed.
A document identifier, which is a number for uniquely identifying the document in the document database, is assigned to the registration target document read in 310. Further, in step 1330, the registration word extraction program 1120 is executed to extract words from the registration target document on the main memory 1060. In other words, from the beginning to the end of the registration target document, the delimiter table 1 on the magnetic disk 1070 is assigned to each character.
By referring to 200, a delimiter is identified from the registration target document. Then, a series of character strings enclosed between delimiters is extracted as a word. Next, in step 1340, the registration high-frequency word determination program 1130 is executed, and it is determined whether or not each word extracted in step 1330 is a high-frequency word. That is, the high-frequency word list storage area 121 on the magnetic disk 1070
It is determined whether or not each word is a frequent word by determining whether each word is defined in the frequent word list while referring to the frequent word list stored in 0. . Finally, in step 1350, the search index creation / registration program 1140 is executed.
Execute the search index data creation registration process for the word extracted in step.

Further, the processing contents in step 1350, that is, the search index creation registration program 11
The contents of the process 40 will be described in more detail with reference to FIG. Index creation registration program 114 for registration
At 0, a series of processing shown in steps 1410 to 1430 is performed on all the words extracted in step 1330 (step 1400). First, in step 1410, the process of creating the search index is divided into cases based on the determination result of the high frequency word determination program for registration 1130 in step 1340. That is, if the word is a high-frequency word, the process of creating a search index for the high-frequency word is performed by executing step 1420. If the word is not a high-frequency word, the process of step 1430 is executed to create a search index as a general word other than the high-frequency word. In step 1420, as a process of creating a search index for a high-frequency word, a work set in the main memory 1060 is set as a set of a document identifier of the corresponding document and a word position in the corresponding document for a set of one word following the corresponding word. By storing the index data in the area 1150, index data for search is created. In step 1430, as a search index creation process as a general word other than the high-frequency word, a set of a document identifier of the corresponding document and a word position in the corresponding document is set for the corresponding word.
Index data for search is created by storing it in the work area 1150 on the main memory 1060. When the above process is completed for all the words extracted in step 1330, a process of updating the search index is performed in step 1440. That is, the search index data for each word stored in the work area 1150 on the main memory 1060 is added to and updated in the search index storage area on the magnetic disk 1070. The above is the document registration processing procedure in the present embodiment.

Next, regarding the document registration processing in the present embodiment, “The president of Bank of America has decide
First, in the repetition processing in step 1300, first, in step 1310, “the president of Bank of America has d” corresponding to the document to be registered is described.
"ecided..." is read from the floppy disk 1050 and stored in the work area 1150 on the main memory 1060. Then, in step 1320, an initial value 1 is set as a document identifier for the relevant document. Word extraction processing is performed with reference to the delimiter table stored in the delimiter table storage area 1200 on the column 1070. The configuration of the delimiter table in this embodiment is shown in FIG.
Shown in In the delimiter table, the character code of each character is used as an entry, and "1" is set as a delimiter character indicating a delimiter character indicating a break between words, and "0" is set as a delimiter character for other characters. . That is, in the example shown in FIG. 5, "" (space), "!"
(Explanation mark) and "/" are defined as delimiters. Then, the text in the document to be registered, “The president of Bank of America has decid
For each character of “ed...”, the delimiter table is referred to, and a character whose value is 1 (in this example, “” (space)) is identified as a boundary, thereby obtaining “ The, president, of, bank, o
Words such as “f”, “America”, “has”, and “decided” are extracted.

In step 1340, step 1
For each word extracted in 330, the magnetic disk 1
The high-frequency word determination process is performed with reference to the high-frequency word list stored in the high-frequency word list storage area 1210 on the 070. In the present embodiment, it is assumed that a word corresponding to the stop list described in the publicly known example 1 is registered as a high-frequency word list, and for example, words such as those illustrated in FIG. 7 are stored as high-frequency words. If you have
“1” is added as a high-frequency word flag for “the”, “of”, and “has” and output. Finally, in step 1350, search index data is generated for the word to which the high frequency word flag has been added in step 1340. That is, it is determined whether the value of the high-frequency word flag is “1” or “0”. A set of word positions from the head of the document is generated as search index data. In addition, in the case of “0”, as a general word other than the high-frequency word, a set of a relevant document identifier and a word position of the relevant word from the head of the document is generated as search index data for one relevant word.

That is, in the example shown in FIG. 8, "The" is input first, but since the term frequency flag relating to the word is "1", one word following "The" as a high frequency word is input. Index data for search is generated for “The president” combined with “President”. In other words, D1 is set as the document identifier ("D" indicates a document position), and the word position is set to the initial value P1 ("P" indicates the word position) because it is at the beginning of the document. Then, with this combination, search index data is generated for “The president”. Next, “president” is input, and the term frequency flag for the corresponding word is set to “0”.
Therefore, as a general word other than the high-frequency word,
Generate search index data for “esident”. That is, set D1 as the document identifier and P2 as the word position, and generate search index data for “president” with this combination. As a result, search index data for the corresponding document is generated in the work area 1150 of the main memory 1060, and when processing of all input data is completed, this is added to the search index storage area 1220 on the magnetic disk 1070. Then, step 1310 is stored.
In, the next document to be registered is read into the work area 1150 of the main memory 1060, and the same processing is repeated for all the documents to be registered (step 1300). More than,
5 is an example of a document registration process according to the embodiment. In this embodiment, the unit of processing for generating the search index data in the 1060 work area 1150 on the main memory and updating the search index storage area 1220 on the magnetic disk 1070 is set to each registration target document. It is also possible to process this as a unit of a plurality of cases.

Next, the document search server in the present embodiment
That is, 2000 in FIG. 1 will be described with reference to the drawings. FIG. 9 illustrates a document search server 200 according to the present embodiment.
FIG. 3 is a diagram showing a configuration of a zero. Document search server 2 shown in FIG.
Reference numeral 000 denotes a display 2010 for displaying the execution status of the processing, a keyboard 2020 for inputting commands such as start and stop of the server, a central processing unit CPU 2030 for executing a search process, and temporarily stores a search program and data. Main memory 2040 for storing data and magnetic disk 205 for storing various data and programs
0 and a bus 2060 connecting them.
The main memory 2040 has a system control program 2
100, search control program 2110, search word extraction program 2120, search high frequency word determination program 2
130, the search execution program 2140 is the magnetic disk 2
050 is read and stored, and a work area 2150 is secured. The extracted word program for search 2120 and the high-frequency word determination program for search 2
130 is described as a separate program from the registration word extraction program (1120 in FIG. 2) and the registration high-frequency word determination program 1130, however, the processing to be executed in the present embodiment is the same. The magnetic disk 2050 secures a delimiter table storage area 2200, a high-frequency word list storage area 2210, a search index storage area 2220, and various program storage areas 2230.

As described in the description of the outline of the entire system, the delimiter table storage area 2200, the high-frequency word list storage area 2210, and the search index storage area 2220 are transmitted from the document registration subsystem 1000 via the network 5000. In the present embodiment, the delimiter table storage area 1200, the frequent word list storage area 1210, and the search index storage area 1220 in FIG.
Will be stored. In this embodiment, these storage areas are stored on the magnetic disk in 2050.
Although secured above, another secondary storage device such as a magneto-optical disk device may be used. The above is the document search server 200
0.

Next, the procedure of the document search process in the embodiment will be described. First, when a search term is input as a search command from the document search client 3000 or 4000 in FIG. 1, the search command is transmitted to the document search server 2000 via the network 5000. When the document search server 2000 receives the search command, the system control program 2100
Starts the search control program 2110 and starts the document search process. Hereinafter, processing at the time of document search will be described with reference to FIG.
This will be described using the PAD shown in FIG. Search control program 21
10 executes the search word extraction program 2120 in step 2300, and refers to the delimiter table 2200 on the magnetic disk 2050 for each character from the beginning to the end of the search term, thereby obtaining the delimiter from the search term. Identify. Then, a series of character strings enclosed between delimiters is extracted as a word. Next, in step 2310, the search high frequency word determination program 2130 is executed, and it is determined whether or not each word extracted in step 2300 is a high frequency word. That is, while referring to the high-frequency word list stored in the high-frequency word list storage area 2210 on the magnetic disk 2050, it is determined whether or not each word is defined in the high-frequency word list. It is determined whether or not the word is a frequent word. Finally, in step 2320, the search execution program 2140 is executed, and the
Reference is made to the search index stored in the search index storage area 2220 on 50 to execute document search processing.

Further, the processing contents in step 2320, that is, the processing contents of the search execution program 2140 will be described in more detail with reference to FIG. In the search execution program 2140, first, in step 24,
At 00, it is determined whether or not the first word is a high-frequency word based on the determination result at step 2310. If the result of the determination in step 2400 is a high-frequency word, reference is made to search index data relating to a set of the relevant word and one subsequent word in step 2410, and in step 2420 the adjacency determination processing between subsequent indexes 2 is set as the number of adjacent judgment words to be used at times. If the result of the determination in step 2400 is a general word other than a high-frequency word, step 243 is executed.
At 0, reference is made to the search index data for the word, and at step 2440, 1 is set as the number of adjacent determination words. Further, in step 2450, it is determined whether or not the focused word is the end of the search term. If the result is not the end, step 246 is executed.
A series of processes from 0 to step 2540 are performed, and the search term is collated while determining the adjacency with the index data of the word following the word in the search term. That is, in step 2460, as preparation for the adjacentness determination process between indexes, step 24
10 or the search index data extracted in step 2430 in work area 2 on main memory 2040.
Move to the adjacent determination target area secured at 150. Then, in step 2470, a series of processes in steps 2480 to 2530 is repeated until the end of the search term is reached. First, in step 2480,
It is determined whether the focused word is a high-frequency word.
If the word is a high-frequency word, in step 2490, reference is made to the index data for the relevant word and the following one word, and the adjacent data is subjected to the adjacent data determination processing with the index data stored in the adjacent determination target area. At this time, since the number of words of the term as the result of the adjacent determination increases by two words, 2 is added as the number of adjacent determination words in step 2500. If the result of the determination processing in step 2480 is a general word that is not a high-frequency word, reference is made to the search index data for the word in step 2510, and the index data stored in the adjacent determination target area and the adjacent determination Perform processing. At this time, since the number of words of the term as the result of the adjacent determination increases by one word, 1 is added as the number of adjacent determination words in step 2520. Then, as a preparation for performing the process of determining the adjacency with the next word, the result of the adjacency determination is moved to an adjacent determination target area stored in the work area 2150 on the main memory 2040 in step 2530. Such processing is repeated to the end of the search term. When the repetition processing in step 2470 is completed, in step 2540, the data in the adjacent determination target area is output as a search result related to the relevant search term, and the processing ends. If the result of the determination processing in step 2450 is that the search term is at the end, step 2550 is executed, and step 2550 is executed.
410 Also, the search index data extracted in step 2430 is output as a search result related to the search term, and the process ends. The above is the document search processing procedure in the present embodiment.

Next, the document search process in the present embodiment will be described with reference to the drawings, taking as an example a case where a search term "Bank of America" is designated. First, in the repetition processing in step 2300 in FIG. 10, a word extraction processing is performed for the search term “Bank of America” while referring to the delimiter table. Note that the delimiter table in the present embodiment has the configuration shown in FIG. 5, similarly to that used at the time of registration. In other words, from the search term “Bank of America”, as shown in FIG.
k, “of” and “America.” Then, in step 2310, the high-frequency words stored in the high-frequency word list storage area 1210 on the magnetic disk 1070 for each word extracted in step 2300. The high-frequency word determination process is performed with reference to the list, and the same high-frequency word list (FIG. 7) used at the time of registration as in the delimiter table is referred to. As a result, “1” is added to “of” and “0” is added to “Bank” and “America” as high-frequency word flags as shown in Fig. 13. Finally, in step 2320, According to the processing procedure shown in Fig. 11, a search process is executed for the word string to which the high-frequency word flag has been added in step 2310. An example of this process is shown in Figs. It will be described with reference to the 5.
First, in FIG. 14, attention is paid to "Bank" which is the first word in the word strings "Bank", "of" and "America". Then, as a result of the determination processing in step 2400 in FIG. 11, the value of the high-frequency word flag related to “Bank” is “0”, so that it is processed as a general word other than high-frequency words. That is, in step 2430, search index data relating to “Bank” is read from the search index storage area 2220 on the magnetic disk 2050, and in step 2440, “1” is set as the number of adjacent determination words. Next, in step 2450 in FIG.
It is determined whether or not it is the last word of the “nk” search term. In this example, since the following word “of America” exists and is not the last word, a series of processes after step 2460 are executed. At 2460, the search index data relating to “Bank” is moved to the adjacent determination target area secured in the work area 2150 on the main memory 2040. Next, as shown in FIG.
At 0, it is determined whether or not “of”, which is a word following “Bank”, is a high-frequency word. As a result, the value of the high-frequency word flag for “of” is “1”, and is determined to be a high-frequency word. As a result, step 2490 is executed, and a set of the corresponding word and the following one word, that is, “of B
The search index data related to “ank” is referred to.On the other hand, the search index data of “Bank” stored in the adjacent determination target area as the number of adjacent determination words “1” is determined to be adjacent (that is, “ Bank ”and“ of A
merica ”exists and the word position of“ Bank ”differs from the word position of“ of America ”by 1).
nk of America ”. In step 2500, by adding“ 2 ”as the number of adjacent determination words to the number of adjacent determination words“ 1 ”at this time, the number of adjacent determination words becomes three (at this time, , Which is not actually used because it ends the search term.)
Since “America” following “of” is the end of the search term, the repetition processing in step 2470
It will end here. Finally, step 2540
In the adjacency determination result, that is, "Bank of America"
The search result regarding the search term is output as the search result regarding the search term, and the search process ends. The above is an example of the search processing in this embodiment.

In this embodiment, when one word "Bank" is designated as a search term, the result of the determination in step 2450 in the above processing becomes the end of the search term. That is, in this case, step 2550 is executed, and the search index data itself referred to for “Bank” is output as a search result. In this embodiment, for example, “Bank of Am
erica has decided ”as a search term“ B
If some words follow ank of America, the process in step 2480 is further repeated to make “has decided” following “Bank of America”.
For example, the search index can be referred to and the adjacency can be determined to obtain a search result. The above is the processing content at the time of document search in this embodiment.

As described above, according to the present invention, at the time of registering a document, for a high-frequency word in a document to be registered, a search index is created by combining one word following the high-frequency word. At the time of a search, for a phrase including a high-frequency word, by referring to the search index data for a set of one word following the high-frequency word, the capacity of the search index data to be read at the time of the search is reduced. The search performance of phrase search can be greatly improved.
In the present embodiment, a method for creating index data for search by combining one word following a high-frequency word has been described. However, by combining this with a continuous word string of two or more words, search performance can be further improved. It can be improved. In the present embodiment, the high-frequency words registered in the high-frequency word list are defined as stop words (excluded words) listed as a stop list in, for example, the known example 1. It may be defined based on statistical information such as appearance frequency information in a group. Further, the search term information may be selected and defined from search terms requiring time for a search within the search terms input by the end user. Further, in the present embodiment, the case where English data is targeted as a search target has been described. However, the present invention can be applied to other documents including German and French.

As described above, in the first embodiment of the present invention, it is assumed that words corresponding to the stop list described in the well-known example 1 are registered as a high-frequency word list, The method of extracting a high-frequency word from a registration target document while referring to a predefined high-frequency word list has been described. However, the document search method according to the present embodiment has a problem that sufficient search performance cannot be obtained for a phrase including a high-frequency word unique to a database. That is, in the database storing the English-language patent specifications, it is considered that the appearance frequency of “step”, “metod”, “apparatus”, etc. is high. However, the high frequency words in the stoplist do not define these words as high frequency words.
ket ”(multi-stage rocket) and“ step function ”(step function)
For such a phrase, it is necessary to read the index data of the high-frequency word “step” and the index data of the word connected to the word from a secondary storage such as a magnetic disk, and determine the adjacency of these. For this reason, there is a problem that the search time becomes extremely long. Therefore, in the second embodiment of the present invention, the document registration subsystem determines whether or not the index data of each word has exceeded a predetermined index capacity at the time of creating and registering the search index. Then, for a word exceeding a predetermined index capacity, an index for two consecutive words, including one word connected to the corresponding word, is generated and registered, and the corresponding word is registered in a high frequency list. This makes it possible to realize a document search system that can dynamically register words that frequently appear in a registered document in a high-frequency word list and can quickly search for a phrase that includes the word.

FIG. 16 shows the configuration of the document registration subsystem according to the second embodiment of the present invention. The document registration subsystem in the present embodiment has substantially the same configuration as the document registration subsystem in the first embodiment shown in FIG. 2, but the search index creation registration program 114 shown in FIG.
0 is replaced by a high-frequency word extraction type search index creation registration program 1141, and a search index extension program 1160 is newly added. Next, the procedure of the document registration process in this embodiment is shown in FIG.
This will be described using the PAD shown in FIG. Note that the document registration processing procedure in the present embodiment is almost the same as the processing procedure in the first embodiment shown in FIG. 3, and here, supplementary description of steps 1351 and 1360 different from the first embodiment. . First, in step 1351, the search index creation / registration program 1141 is executed, and the same as step 1350 in FIG.
Execute the search index data creation registration process for the word extracted in step. Further, it is determined whether or not the capacity of the index data for each registered word is larger than a predetermined value. If it is determined that the capacity is larger, the corresponding word is extracted as a high-frequency word. In the work area 1150. And step 13
At 60, the work area 1150 on the main memory 1060 is referred to, and it is determined whether or not a newly extracted high-frequency word exists. If it does, steps 1370 and 1380 are executed. That is, step 137
At 0, a high-frequency word stored in the work area 1150 on the main memory 1060 is searched for a registered text (not shown). Then, one word connected to the high-frequency word obtained in step 1351 is extracted, and processing for creating full-text search index data is performed for the two consecutive words that have been combined (the search index data here is used). Is used for the creation process of (1) in FIG. 4). Finally, in step 1380, step 13
The high-frequency words obtained in 51 are additionally stored in the high-frequency word list storage area 1210, and the document registration processing is completed. The above is the outline of the registration processing in the second embodiment of the present invention.

As described above, according to the present embodiment, it is determined whether or not the index capacity for each word is larger than a predetermined index capacity at the time of document registration. Then, a word exceeding a predetermined index capacity is regarded as a high-frequency word and is connected to the corresponding word from the registered text.
Create and register index data for two consecutive words, including words. The corresponding word is additionally registered in the high-frequency word list. As a result, for a phrase including the corresponding word, the search index for the two words including the phrase is referred to at the time of the search, and a phrase search including the corresponding high-frequency word can be realized for any high-frequency word at high speed. become.

In step 1351, the reference index capacity as a criterion for determining whether a word is a high-frequency word is:
In the present embodiment, predetermined values are described, but a method of setting these values as configuration information in the system may be used. Further, in this embodiment, every time a document is registered, extraction of a high-frequency word in step 1351, expansion of the search index in steps 1360 to 1380, and registration processing as a high-frequency word are executed. However, it is also possible to suspend the processes in steps 1360 to 1380 for a while and execute these processes at a timing suitable for performing database maintenance. Further, in this embodiment, a series of processing in steps 1360 to 1380 is executed for a high-frequency word extracted at the time of document registration. However, by adopting the configuration shown in FIG. It is also possible to execute for a word input and designated as.

[0028]

According to the present invention, when a phrase including a high-frequency word is designated as a search term, a search index for a set of one word following the high-frequency word, which has been created in advance at the time of document registration. By referring to the data, it is possible to reduce the capacity of the search index data read at the time of the search, and it is possible to greatly improve the search performance of the phrase search.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of a document registration subsystem according to the first embodiment.

FIG. 3 is a diagram showing a processing flow at the time of document registration in the first embodiment.

FIG. 4 is a diagram showing a processing flow of a search index creation registration program in the first embodiment.

FIG. 5 is a diagram showing a configuration of a delimiter table according to the first embodiment.

FIG. 6 is a diagram showing an outline of a word extraction process at the time of document registration in the first embodiment.

FIG. 7 is a diagram illustrating an outline of a high-frequency word determination process at the time of document registration in the first embodiment.

FIG. 8 is a diagram showing an outline of search index creation registration processing at the time of document registration in the first embodiment.

FIG. 9 is a diagram illustrating a configuration of a search server according to the first embodiment.

FIG. 10 is a diagram showing a processing flow at the time of document search in the first embodiment.

FIG. 11 is a diagram illustrating a processing flow of a search execution program in the first embodiment.

FIG. 12 is a diagram showing an outline of a word extraction process at the time of a search in the first embodiment.

FIG. 13 is a diagram illustrating an outline of a high-frequency word determination process at the time of search in the first embodiment;

FIG. 14 is a diagram showing processing contents at the time of retrieval in the first embodiment.

FIG. 15 is a diagram showing processing contents at the time of retrieval in the first embodiment.

FIG. 16 is a diagram illustrating a configuration of a document registration subsystem according to a second embodiment of the present invention.

FIG. 17 is a diagram showing a processing flow at the time of document registration in the second embodiment.

FIG. 18 is a diagram showing a configuration of a document registration subsystem for realizing addition of a high-frequency word by input designation from outside.

[Explanation of symbols]

1000 Document registration subsystem 2000 Document search server 3000 Search client 1 4000 Search client 2 5000 Network 1010 Display 1020 Keyboard 1030 Central processing unit (CPU) 1040 Floppy disk driver 1050 Floppy disk 1060 Main memory 1070 Magnetic disk 1100 System control program 1110 Registration Control program 1120 Registration word extraction program 1130 Registration high frequency word determination program 1140 Search index creation registration program 1141 High frequency word extraction type search index creation registration program 1150 Work area 1160 Search index expansion program 1170 High frequency word input designation Program 1200 delimiter table storage area 1210 High-frequency word list storage area 1220 Search index storage area 1230 Various program storage areas 2010 Display 2020 Keyboard 2030 Central processing unit (CPU) 2040 Main memory 2050 Magnetic disk 2100 System control program 2110 Search control program 2120 Search word extraction program 2130 Search high frequency word determination program 2140 Search execution program 2150 Work area 2200 Delimiter table storage area 2210 High frequency word list storage area 2220 Search index storage area 2230 Various program storage areas

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Natsuko Sugaya 890 Kashimada, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture, Japan Business Solution Development Headquarters, Hitachi, Ltd. (72) Inventor Tadataka Matsubayashi 890, Kashimada, Sai-ku, Kawasaki-shi, Kanagawa Prefecture, Japan Hitachi, Ltd. Business Solution Development Headquarters (72) Inventor Yasuhiko Inaba 890 Kashimada, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Hitachi, Ltd. Business Solution Development Headquarters (72) Inventor Yasushi Kawashita Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture 5030 F-term in the Software Division of Hitachi, Ltd. (Reference) 5B075 NK02 NK12 NK24 NK32 5B082 EA05 GC04

Claims (9)

[Claims] In a document search method in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance, the process of registering a document includes the steps of: A registration word extraction step of extracting a sequence of more than one character as a word; a high-frequency word determination step of determining whether the extracted word is a word defined as a high-frequency word; If the determination result in the frequent word determination step is a high-frequency word, one word connected to the high-frequency word is extracted, and a word string composed of the high-frequency word and the extracted one word is searched for. A document search method comprising a search index creation registration step of creating and registering index data. 2. A document search method in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance, wherein a process of searching for a document is designated as a search term. A search word extraction step of extracting a sequence of one or more characters as a word from a character string, and a high-frequency word determination step of determining whether the extracted word is a word defined as a high-frequency word If the determination result in the high-frequency word determination step is a high-frequency word, one word connected to the high-frequency word is extracted, and a word string including the high-frequency word and the extracted one word is extracted. A document search method comprising a search execution step of performing a search by referring to search index data. 3. The document search method according to claim 1, wherein a word group that is considered to have a high frequency of appearance in a document database of the document search system is defined in advance, and the high-frequency word determination is performed. In the document search method, it is determined whether the extracted word is a word included in the word group. 4. The document retrieval method according to claim 1, wherein the appearance frequency is considered high based on statistical information such as appearance frequency information on each word registered in a document database of the document retrieval system. A word group to be extracted is defined in advance, and the high-frequency word determining step determines whether or not the extracted word is a word included in the word group. 5. In a document search method in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance, the process of registering a document includes the steps of: A registration word extraction step of extracting a sequence of characters or more as words, and determining whether the word extracted in the registration word extraction step is a high-frequency word that appears frequently in a set of registered documents. If the determination result in the high-frequency word determination step is a high-frequency word, one word connected to the high-frequency word is extracted,
Index data for search is created and registered for the word string consisting of the high-frequency word and the extracted one word. If the high-frequency word is not a high-frequency word, the index capacity of the index data related to the word is set to a predetermined index. It is determined whether or not the capacity is larger than the capacity. If the capacity is larger, the corresponding word is newly determined as a high-frequency word, and an index creation and registration step for high-frequency word extraction search is performed. For a high-frequency word newly extracted in the index creation registration step, refer to a set of registered documents, and connect the word to the word.
Extracting a word string composed of words, creating and registering search index data for the word string, and further including a search index expansion step of newly registering the high-frequency word as a high-frequency word. The document search method you want. 6. The document search method according to claim 5, wherein
A document search method comprising configuration information for setting an index capacity serving as a criterion for determining a high-frequency word. 7. In a document search method in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance, the process of registering a document includes: A registration word extraction step of extracting a sequence of characters or more as words, and determining whether the word extracted in the registration word extraction step is a high frequency word that appears frequently in a set of registered documents If the determination result in the high-frequency word determination step is a high-frequency word, one word connected to the high-frequency word is extracted,
A search index creation / registration step of creating and registering search index data for the word string consisting of the high-frequency word and the extracted one word; and a high-level input / designation of a word group to be newly registered as a high-frequency word. Referring to a set of registered documents for the word input in the frequent word input designating step and the frequent word input designating step, extracting a word string consisting of the word and one word connected to the word, A document search method, comprising: a search index expansion step of creating and registering search index data for a column and newly registering the high-frequency word as a high-frequency word. 8. A document registration program for performing a document registration process in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance. A registration word extraction procedure for extracting the above character sequence as a word, a high frequency word determination procedure for determining whether the extracted word is a word defined as a high frequency word, If the determination result in the word determination procedure is a high-frequency word, one word connected to the high-frequency word is extracted, and a search index is assigned to a word string including the high-frequency word and the extracted one word. A computer-readable recording medium recording a document registration program having a search index creation registration procedure for creating and registering data. 9. A document search program for performing a document search process in a document search system for searching for a document including a character string specified as a search term from a set of documents registered in advance. A search word extraction procedure for extracting a sequence of one or more characters from a character string as a word, a high-frequency word determination procedure for determining whether or not the extracted word is a word defined as a high-frequency word If the determination result in the high-frequency word determination procedure is a high-frequency word, one word connected to the high-frequency word is extracted, and a word string including the high-frequency word and the extracted one word is extracted. And a computer-readable recording medium storing a document search program having a search execution procedure for performing a search by referring to search index data.