JP3220865B2 - Full text search method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a full-text search method for searching a document database by specifying a character string for the entire text of a document, and more particularly to a full-text search process using an auxiliary file for search. The present invention relates to a search method and apparatus for equivalently increasing the speed.

[0002]

2. Description of the Related Art In a conventional document retrieval system, a method is employed in which a word (referred to as a keyword) representing the contents of a document to be registered is indexed. However, in this method, an indexer called an indexer had to read the document one by one, understand the contents, and assign appropriate keywords. In order to avoid the troublesome work at the time of registration, there has been proposed a method of registering all words appearing in the text as keywords in an index file as disclosed in JP-A-63-198124. However, with the above method, it is difficult to determine the smallest unit word that has meaning when creating an index file, and the analysis of a sentence fails due to inadequate word dictionary or grammatical rules. Is not extracted as a keyword.

In order to solve this problem, a document is directly registered in a computer as a character-coded text at the time of retrieval,
At the time of retrieval, a full-text search has been proposed that reads the contents of all documents in a text database and searches for documents containing a given keyword (hereinafter referred to as a search term to distinguish it from control keywords in conventional systems). ing. This full-text search method is described in “Information Processing Society of Japan Vol. 89, No. 66, Informatics Fundamentals 14-7 Text Database Management System SI
As described at the beginning of the second section of "GMA and its applications (1989. 7.27)", a major feature is that the entire text file is scanned one character at a time from the beginning. By doing so, even if there is no index file in which a document identifier or the like corresponding to the keyword is described, it is possible to search using the text body of the text database as a clue. That is, it is possible to perform a character string search on the entire text data using a given search term, and output only a document in which the search term is described as a search result.
However, this full-text search method has a problem in that it takes a long processing time to scan the entire text file one character at a time from the beginning, and cannot be applied to a large-scale database. As seen in the second section of the document, even with a general-purpose large computer, only a search processing speed of about 2 MB / s can be realized. Even at this speed, if the database is several megabytes, the search time will be within the practical range. However, a practical-scale database such as an office requires a capacity of several hundred megabytes, and in this case, a sufficient search response cannot be obtained.

[0004]

The problem to be solved by the present invention is that even when a practical-scale text database is targeted, the full text of a document in which search results can be obtained in a search time sufficient for practical use is obtained. An object of the present invention is to provide a high-speed full-text search method and apparatus to be searched.

[0005]

In order to solve the above-mentioned problems, a full-text search method including the following processing steps is used, and an apparatus for implementing the method is configured. (1) Step of storing the text itself (2) Decomposing the stored text into partial character strings at the word level, examining the mutual inclusion of the character strings between the decomposed partial character strings, and including them in other partial character strings Step of creating a condensed text composed of a set of partial character strings excluding character strings to be used (3) Step of generating a character component table in which characters used in the text are collected without duplication (4) Given search term At the character level to extract only documents that contain all the characters that make up the search term. (5) Referring to the condensed text corresponding to the document extracted from the character component table, (6) If the given search condition expression specifies the positional relationship in the text between a plurality of search terms, the condensed text is extracted using the condensed text. Referring to text data corresponding to a document, it includes a given search term, step text search for extracting only yet satisfy the search conditions, such as granted positional relationship between search terms

[0006]

As described above, the character component table search and the condensed text search are provided by providing a hierarchical pre-search means for narrowing down the text component table search and the condensed text search and finally performing the text search. Documents that do not satisfy the conditional expression can be truncated before referring to the text body to reduce the amount of searching for the text body to be searched. That is, it is possible to reduce the entire search processing time by reducing the text search processing time, which accounts for a large proportion of the search processing time. For example, given a conditional expression that specifies the positional relationship in the body of two search terms, "Search for documents in which the" image "and" processing "are in the same sentence (sentence) in the body" In the conventional method, the search processing speed is 2 MB /
Assuming s, it takes 250 seconds, or about 4 minutes, to search through the full 500 MB full text. However, in the case of hierarchical pre-search, in a typical case, the character component table accounts for 10% of the entire database,
If you can narrow down to 30%, the capacity of the condensed text is 30% of the text
In the case of, the capacity of the character component table is so small that it can be ignored from the viewpoint of the whole database. Therefore, the capacity of the condensed text to be searched is 15 MB, and the search amount of the text data is 1% of the total database size, ie, 5 MB. Even at a search speed of 2 MB / s, the search process can be completed in 10 seconds. As described above, in the “hierarchical pre-search method”, two-stage pre-searches of “character component table” and “condensed text” are performed in advance, and sieved at “character level” and “word level” respectively. By narrowing down the number of documents to be subjected to the time-consuming text search to a minimum in advance, the search document capacity can be reduced, and therefore, a very high-speed full-text search can be equivalently realized.
If the conditional expression is an AND, OR, or NOT condition in a single search term or a plurality of search terms, the result of the condensed text search can be used as the final search result. Because the words in the condensed text are
This is because there is no need to search the text again because it is always present in the text. As described above, the search at the "word level" can completely omit a text search that requires a long processing time, so that the entire search processing time can be further reduced. According to the full-text search method including the above processing steps, the amount of direct text search can be reduced in advance, so that high-speed full-text search can be performed.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to FIG. This apparatus comprises a display 100, a keyboard 101, a central control unit CPU 102, a character component table 105, a condensed text 104, a file 110 for storing text 103, a floppy disk driver 106, and a main memory 200.

In the main memory 200, a text registration program 201, a condensed text creation registration program 202, a character component table creation registration program 203, a character component table search program 204, a condensed text search program 205, a text search program 206, a hierarchical type A search control program 207 is stored, and a data area 208 is secured. These programs are executed by the CPU 102. At the time of document registration, the CPU 102 reads document data from a floppy disk inserted into the floppy disk driver 106 by a command input from the keyboard 101, executes the text registration program 201 and sets the read document data as the text 103. It is stored in the file 110. Next, the CPU 102 executes the condensed text creation / registration program 202, divides the text 103 into partial character strings at the word level, checks the mutual inclusion of the character strings among the divided partial character strings, A condensed text composed of a set of partial character strings excluding the character strings included in the column is created, and the
Store to 0. Finally, the CPU 102 executes the character component table creation registration program 203 to create a character component table in which characters used in the text 103 are collected without duplication.
This is stored in the file 110 as the character component table 105.

At the time of a search, a search condition expression input from the keyboard 101 is sent to the CPU 102. CPU
In step 102, first, the hierarchical search control program 207 is executed, and based on the control, the character component table search program 2 is executed.
04, a condensed text search program 205, and a text search program 206 are sequentially executed. That is, in the character component table search, the search terms in the input search condition expression are decomposed at the character level, and only documents containing all the characters constituting the search terms are extracted. Then, referring to the condensed text corresponding to the document extracted in the character component table, a document including the given search term is extracted. If only a single search term or a logical relationship between multiple search terms is specified in the given search condition expression, but not a positional relationship in the text, Terminates the search here and outputs the result of the condensed text search as the search result. In other cases, that is, when a given search condition expression specifies a positional relationship in the text between multiple search terms, refer to the text data corresponding to the document extracted by the condensed text search. Then, only those that include the given search terms and satisfy the search conditions such as the positional relationship given between the search terms are extracted and output as the search results. The above is the outline of the full-text search device according to the first embodiment of the present invention.

Hereinafter, registration and search methods of a character component table search, a condensed text search, and a hierarchical presearch method for performing text search and refinement, which are features of the present invention, will be briefly described. First, the "condensed text" and the "character component table" are created automatically when a document is registered. This processing content,
As shown in FIG. In this figure, when the document to be registered is entered,
First, it is stored as "text" as it is. Next, a “condensed text” is created from the “text”. "Condensed text"
Is created by dividing a character string for each character type (Kanji, Hiragana, Katakana, English characters, etc.) from the "body" and eliminating repeated words that appear repeatedly. In the case where the text is document 1 of "search technique for fuzzy search ...", "search" is truncated as a duplicate word, and "fuzzy", "search technique" and "for" It will remain as "condensed text". In addition, a "character component table" is created from the "text". Here, characters appearing in the “body” are represented by 1-bit information. In the example of the document 1, "1" is set because there are "A" and "I", and "0" is set because there is no "U". Similarly, “1” is set for “detect” and “cord”, respectively. Similarly, in the corresponding character portion of the character component table, "1" is set when the character is present in "body", and "0" is set when it does not exist. In this way, a "condensed text" and a "character component table" are automatically created when a document is registered, and preparation for hierarchical presearch is made.

At the time of retrieval, these auxiliary files are referred to in the reverse order of registration, as shown in FIG. First,
In the character component table search, the character component table is referred to, and the character component table corresponding to all the characters in the search term and a character component set to “1” is selected. Second
In the condensed text search, the condensed text of the document selected in the character component table is referred to, and a document including the search term given in the conditional expression is selected. Finally, in the text search, only those terms whose appearance in the text of the search term matches the conditional expression are selected. In the example of this figure, search [4C] comprehension, that is, an example in which a search is performed using a conditional expression of “search for a character in which“ search ”and“ comprehension ”appear within four characters in proximity to each other”. As a result, a document in which “search” and “understanding” of document 4 are separated by 4 characters in the text is extracted.

Hereinafter, a method for creating a character type division / duplicate word elimination type condensed text and a character code dependent type character component table used in this embodiment and a method for controlling a hierarchical presearch using these will be described in detail. First, a description will be given of a method of creating a character type division / duplicate word elimination type condensed text used in the present embodiment. As shown in FIG. 4, first, a character string is divided from the body text according to the character type. The character type at this time is
Kanji, Hiragana, Katakana, English letters, numbers, symbols, etc. The character string of the body is divided for each character string composed of a series of these single character types. Next, for each of the divided character strings, when the other partial character strings in the same document completely include the partial character strings, the character strings are excluded from the target of the condensed text as duplicate character strings. For example, since the partial character string “search” is completely included in another partial character string “intelligent search technology” in the same document, this “search” is not registered in the condensed text. However, in the condensed text search, even if the character string “search” is not registered in the condensed text, “intelligent search technology”
Will be hit as a substring. As described above, the overlapping registration of the partial character strings is eliminated, and a separator is inserted between the character strings for each document in the obtained partial character string as shown in FIG. In this drawing, symbols ',' are used as separators. In FIGS. 2 and 3, this separator is represented by the symbol "|", but this separator does not need to be particularly represented as a character, and a special code not assigned to a character can be used.

Next, a method of creating a character code dependent type character component table used in this embodiment will be described. As shown in FIG. 6, the character code dependent type character component table is a character component table that determines a bit position at which 1 is set as bit information indicating presence by a character code. In the figure, the shift JIS code is described as an example. In the figure, (XXXX) H is a character code displayed in hexadecimal. For example, to indicate that the character string “search” exists in the text of document 1, (8C9F) H, (8DF
5) Set 1 to H-th. The bit position corresponding to this character in the bit list is called the entry number of the character component table. For example, the entry number of “check” is (8C9
F) 35999 if H or decimal notation is displayed.

The control and search operation of the hierarchical pre-search using the character component table and the condensed text will be described.
First, the search terms in the search condition expression are decomposed into character units, and a character component table search is performed. Here, a document having a bit list in which the positions of the entry numbers in the bit list corresponding to the character codes constituting the given search term are all 1 is obtained. For example, if the character string “search” is given as a search term,
(8C9F) of the bit list corresponding to 'Detect' and 'Search'
H, (8DF5) Documents 1, 2, 3, 4,. . . Is the search result of the character component table search. In other words, “test” is indicated as shown in FIG.
C9F) A bit list 701 of H entry numbers,
An AND operation is performed for each bit with the bit list 702 of the entry number of (8DF5) H indicating “search”, and a bit AND operation result 703 is obtained. In the bit list of the bit AND operation result 703, a document number corresponding to a bit position of 1 indicates a hit document as a search result of the character component table search. That is,
A document including all of the search and the search will be extracted. If the search term is composed of only one character such as "lake", the result of the character component table search can be output here and the search can be terminated.

Next, a search is performed on the condensed text of the document extracted by the character component table search. Here, as shown in FIG. 5, the content of the condensed text registered for each document is scanned, and a document containing a given search term as a word is extracted.
That is, only documents in which the two characters “search” and “search” appear consecutively as “search” are extracted. In other words, 'test' and 'chord'
Even if is included, like "detection" and "search",
Anything that appears as another word is truncated here. For this purpose, the condensed text for each document narrowed down by the character component table search is searched while scanning one character at a time in the same manner as the text data. At this time, only the condensed text corresponding to the document number obtained as a result of the character component table search is scanned. For example, if the result of the character component table search is document numbers 1, 2, 3, 4,. . . Then, in the condensed text search, the document numbers 1, 2, 3,
4,. . . Scan the condensed text of. Then, a document in which the search term actually exists in the condensed text is output as a search result of the condensed text search.

As described above, in the "hierarchical pre-search method", two-stage pre-searches of "character component table" and "condensed text" are performed in advance, and the "character level" and "word level" are sieved, respectively. By narrowing down the number of documents to be subjected to the most time-consuming text search in advance to the minimum, the search document volume can be reduced, so that a very high-speed full text search can be equivalently realized. . That is, in the character component table search, since the character component table indicates the presence of a character by one-bit information, the data capacity to be searched can be extremely reduced, and as a result, the search time can be reduced to a short time. Becomes Further, by taking the logical product of the bit list for each character constituting the keyword, documents that are not related to the keyword can be largely discarded, and subsequent documents can be significantly narrowed down. Also, in the condensed text search, the search processing time can be shortened by the smaller amount of data than by directly scanning the text.

Next, a second embodiment of the present invention will be described.
In this embodiment, even when multiple search terms are specified,
An object of the present invention is to provide a full-text search method capable of efficiently performing a hierarchical pre-search. For example, when a conditional expression "" search "AND" understand "" is given, a character component table is searched as a first step. Here, a document containing all the characters is searched for each given search term, and then a document that satisfies the given conditions between the search terms is output. "" Search "AND
In the case of the conditional expression “understand”, a document that includes the two characters “search” and that includes the two characters “understand” is searched. In other words, "('test' AND 'search') AND ('rea' AND 'solution')" Therefore, "'test' AND 'search' AND 'reason' AND 'solution'" Search for the containing document. Next, the condensed text of the document narrowed down as a result of the character component table search is searched. Here, only documents in which the specified keyword appears as a word are extracted. That is,
Search for documents containing both "search" and "understanding" at the same time.

As in the case of this example, when the relationship between the search terms is only a logical condition such as "AND" or "OR" and no other condition for specifying the positional relationship between the keywords is specified. Here, the search is terminated, and the result of the condensed text search is output as the final search result. If the position condition is specified, the text of the document extracted by the condensed text search is searched to extract a document that satisfies the specified condition, and this is output as the final search condition. The above is the description of the search operation in this embodiment. In this way, by taking the logical product between the search terms in the character component table search and the condensed text search, even if multiple search terms are specified, hierarchical pre-search can be performed efficiently, and high-speed full text search can be performed. A search can be realized.

As a third embodiment, the search control of the hierarchical presearch will be described in more detail. FIG. 8 illustrates a procedure of controlling the hierarchical pre-search at this time with reference to a PAD diagram. Here, a search expression "search for a document containing either" computer "or" intelligent interface "", that is, "" computer "OR" intelligent interface "" will be described as an example. First, in step 8000, a character component table search is performed. Here, a document containing all the characters is searched for each given search term, and then a document that satisfies the compound condition given between the search terms is output. In this example, as shown in FIG. 9, for each of the three characters constituting the “computer”, a bit AND operation is performed between the corresponding entry numbers in the character component table, and then the “intelligent interface” is similarly constructed. The bit AND operation between the corresponding entry numbers in the character component table is performed for each of the nine characters to be performed, and the bit AND operation result for the "computer" created previously and the OR operation of the bit string are performed. That is, "('total' AND 'operation' AND 'machine') OR ('knowledge' AND 'like' AND 'i' AND 'n'AND'ta'AND'F' AND 'e'AND'-'AND'Su')". As a result, a document including all three characters constituting the “computer” or a document including all nine characters constituting the “intelligent interface” is extracted. If the number of the results of the character component table search described above is 0, a search result of 0 is output as shown in FIG. 8, and the search is terminated. Also,
Even when the search term is composed of only one character such as “lake”, the result of the character component table search is output here and the search is terminated.

If the search term is composed of a plurality of characters and the result of the character component table search is not zero, a condensed text search is performed next. The content registered in the condensed text is a character string divided for each character type.
For example, if the character type is different in the middle, such as "intelligent interface", the condensed text is decomposed into partial character strings, and a separator is inserted at the division point, such as "intelligent interface". Therefore, in the case of a search term composed of different character types, such as "intelligent interface", there is no corresponding character string even if the condensed text is searched as it is. Therefore, a search term is checked before entering the condensed text search, and a search term composed of different character types is divided for each character type. A search term that has been subjected to the process of splitting by character type in this way is distinguished from the original search term, and is called a split search term. In the condensed text search, for example, a search is performed using a split search term such as "computer", "intelligent", or "interface". However, with respect to the divided search term, a search is performed between terms having the same division source under an AND condition. For example, in the case of the conditional expression "" computer "OR" intelligent interface ", the condensed text search returns""computer" OR ("intelligent" AND "interface") ", that is,""intelligent" and "interface" Is found in the same document, or search for a document in which "computer" exists ".

If the result of the condensed text search is 0, a search result of 0 is output and the search is terminated.
Also, the text search is performed only when the neighborhood condition or the context condition is specified, or when there is a search term to be divided such as “intelligent interface”, that is, when the search term and the divided search term are different. If not,
Here, the hierarchical pre-search is completed, and the result of the condensed text is output as a search result. Here, the context condition is a conditional expression such as, for example, ““ computer ”[S]“ intelligent interface ””, which is a sentence (sentence) in which “computer” and “intelligent interface” are the same. Find what is inside. " Alternatively, the neighborhood condition is described, for example, as ““ computer ”[10C]“ intelligent interface ””, which means that “computer” and “intelligent interface” are close to each other within 10 characters. Find the document that appears. " That is, both the context condition and the neighborhood condition are search conditions that specify the positional relationship of search terms that appear in the document.

When a search condition specifying the positional relationship of the search terms appearing in the text is given, or when a search term whose character type changes in the middle of the condensed text is separated by a separator, Then, the text data corresponding to the result of the condensed text search is referred to, only those in which the search term appears in the text according to the given conditions are output as the search results, and the search ends. In this way, even when the search terms are composed of different character types, or when there is a condition specification regarding the appearance position in the text between the search terms, efficient hierarchical pre-search is performed and high-speed full text A search can be realized.

Next, a fourth embodiment of the present invention will be described. In the present embodiment, the capacity of the character component table in the first embodiment is reduced to make it compact. The character code dependent character component table used in the first embodiment is easy to process, but has a problem that the character component table becomes large because the bit list per document of the character component table is long.
In addition, there is a problem that there are many useless portions because the entry number is assigned even though there is no corresponding character code. For example, in the case of shift JIS, between (0000) H and (8140) H and from (A000) H to (E04
0) During H, that is, from 0th to 33087th and 4
There is no corresponding character code for the entry numbers from 0960 to 57408. Nevertheless, in order to determine the entry number by the character code, it is necessary to have all of this part as a table entry. The character code is once converted to eliminate useless portions in the bit list, and a character component table is created so that bit positions can be used without gaps from the 0th position. The details of the embodiment using the character code conversion type character component table will be described below.

The following formula is given as an example of a conversion formula into a character code for creating a character code conversion type character component table. FIG. 10 shows a corresponding PAD diagram. if SJIS <(A000) H then SCODE = SJIS− (8040) Helse SCODE = SJIS− (C040) H SCODE = SCODE− (SCODE / 25
6) × 64... (4-1) (However, since a small value portion of the normal character code is often used as a control code, in this formula, (8140)
Instead of H, (8040) H is given some margin. Further, the decimal part of the calculation result of (SCODE / 256) is rounded down, and the rounded-down result is multiplied by 64. ) In the formula, SJIS indicates the original shift JIS code, and SJIS
CODE indicates the character code after conversion. Since the KEIS code and other code systems are also compatible with the shift JIS code, conversion to SCODE can be performed using the same formula. The expression (4-1) can be represented by a character code table as shown in FIG.
It means a conversion like 1. That is, (000
0) From H to (FFFF) H (8140) H to (9FFC) H and (E040)
H to (FEFC) H and the character code distributed and assigned to (0000)
The character code is converted so as to be arranged without a gap from H. By performing code conversion using this equation (4-1), the length of the bit list can be extremely reduced as shown in FIG. 12, and the overall capacity of the character component table can be reduced.

The control of the hierarchical presearch is the same as in the first embodiment. That is, using the control procedure of FIG. 8 as it is, first, a character component table search is performed using the characters in the search term, and second, a condensed text search is performed using the search term. If no context condition is specified, the search result is output here and the search ends. Thirdly, if a context condition is specified, a text search is performed and the result is output. However, in the case of a character component table search, all the input search terms are (4-
The character code conversion is performed based on the expression (1). The fourth embodiment using the character code conversion type character component table has been described above. According to the present embodiment, by converting the character code and creating a character component table in which bit positions are arranged without gaps from the 0th position, entries in the character component table to which no character is allocated can be eliminated. The capacity of the character component table can be made very small.

Next, a fifth embodiment of the present invention will be described. In the present embodiment, the capacity of the character component table in the fourth embodiment is further reduced by using a hashing method. In order to further reduce the capacity of the character component table of the fourth embodiment, in this embodiment, a plurality of characters are assigned to one entry number in the bit list. That is, a method is used in which a character in a search term is associated with a bit position in a bit list using a hash function. For example, the following equation can be used as the hash function. h (SCODE) = mod (SCODE, N) (5-1) In the equation, SCODE is shifted according to the equation (4-1).
This is the character code converted from. mod is a function that outputs a remainder obtained by dividing the first argument by the second argument. N is an arbitrary integer value. If, for example, 512 is used as N, “A” becomes the entry number 480, and “M” becomes the entry number 193.

FIG. 13 shows an example of the character component table created as described above. In this case, N was set to 512,
It can be seen that only 512 bits are required to register one document. At the time of search, an entry number is obtained for each character of a given search term using the hash function of equation (5-1) in the same manner as at the time of registration, and the bit position of the corresponding character component table is referenced. For example, in the case of a character string “ambiguity”, as shown in FIG.
A document in which all bits at positions 80, 482, and 193 are 1 is set as a search result of the character component table search. Next, the condensed text of the document obtained by the character component table search is searched.

The control procedure of the condensed text search and the text search will be described below with reference to FIG. In the first embodiment, if the search term consists of one character after the character component table search, the result of the character component table search is output as the search result, and the hierarchical presearch is terminated. However,
In the character component table search of the character component table used in the present embodiment, since there is a possibility that search noise may occur, it is necessary to continue the hierarchical pre-search up to the condensed text search. For example, hiragana 'wa' (shift JIS code (82C
D) H) is the entry number 13 in the expression (5-1), but the kanji character “ship” (shift JIS code (8ACD)
H) has the same entry number 13. This means that when "ship" is given as a search term, all documents containing "ha" are also searched as a result of the character component table search. Therefore, the condensed text is further scanned to extract a document that actually contains the kanji “ship”, and this is output as a search result. The fifth embodiment has been described above. In the present embodiment, by assigning a plurality of characters to one entry of the character component table by using the hash function, the effect of significantly reducing the capacity of the character component table can be obtained.

Next, a sixth embodiment will be described. In the case of simple hashing as in the fifth embodiment, there is a possibility that a character that easily appears in a document, such as hiragana, and a character that rarely appears, such as JIS second-level kanji, have the same entry number. Come out. For example, “Hiragana” and “Kanji” have the same entry number 13, and if “Ship” is given as a search term, all documents containing “H” will be hit as a result of the character component table search. become. Since Hiragana 'ha' is a character that is very frequently used in Japanese documents, almost all documents are hit in the character component table search. As described above, when the narrowing rate in the character component table search is reduced, the amount of documents to be scanned for the condensed text also increases, so that the entire search processing time increases.

In order to prevent such a reduction in the narrowing rate, it is necessary to determine the hash function in consideration of the frequency of use of characters. The character component table used in this embodiment is called a character type hashing type character component table. In order to create a character type hashing type character component table, for example, as shown in FIG. 15, an entry area of the character component table is allocated to each character type, and a hash function is formed in the area by means of a character code. In order to realize such a hash function, a character type may be determined based on a character code and then returned using a mod function, or may be realized based on a correspondence table between character codes and entry numbers. An example of this hash function is shown in a PAD diagram in FIG. In this embodiment, hiragana,
The number of katakana and alphabetic entries is 20 each, the number of symbol entries is 10, the number of numeric entries is 10, JIS
The number of entries of the first level is 370, and the number of entries of the JIS second level is 61. First, the character type of the input search term is determined based on the character code, and the entry portion assigned to the character component table for each character type is looped back using the mod function.

That is, if the SCODE is in the range of (01DF) H to (0231) H, it is a hiragana character string, so that the SCODE is modulo 20 and this is used as the entry number. If SCODE is in the range of (0240) H to (0296) H, it is a katakana character string. Therefore, modulate the SCODE by 20 and add a value obtained by adding 20 which is the head of the katakana hashing area. Let it be an entry number. If SCODE is in the range of (01A0) H to (01DA) H, it is an alphabetic character string. Therefore, modulate the SCODE by 20 and add a value obtained by adding 40 which is the head of the hashing area of the alphabetic character to this. Let it be an entry number. SCODE changes from (018F) H to (0
198) If it is in the range of H, since it is a numeric character string, its SCODE is modulo 10 and the value obtained by adding 70 which is the head of the numeric hashing area is set as the entry number. SCODE changes from (065F) H to (123
2) If it is in the range of H, it is a JIS first-level kanji character string, so its SCODE is mod 370, and
A value obtained by adding 80, which is the head of the hashing area of the JIS first-level kanji character string, to this is set as the entry number. SC
If the ODE is in the range from (125F) H to (1FDE) H, it is a JIS second-level kanji character string.
MOD is taken as 61, and a value obtained by adding 450, which is the head of the hashing area of the JIS second-level kanji character string, to the entry is used as the entry number. In cases other than the above SCODE, it is regarded as a character string of a symbol or other character type, and the SCODE is modulated by 10, and a value obtained by adding 60, which is the head of the hashing area of the symbol, to the entry number.

The control procedure of the hierarchical presearch using the character type hashing type character component table is the same as that of the fifth embodiment. That is, first, a character component table search is performed using the characters in the search term, and second, a condensed text search is performed using the search term. If no context condition or the like is specified, the search is terminated here. If not, a third text search is performed and the result is output. As described above, according to the present embodiment, by using the character type hashing type character component table in which the entry number of the character component table is made to correspond to each character type in consideration of the use frequency, the character component table search can be performed. Since the noise can be reduced, the amount of scanning of the document in the condensed text is reduced, and a high-speed full text search can be performed accordingly.

Next, as a seventh embodiment, a hierarchical pre-search using a frequency information hashing type character component table which can further improve the narrowing rate in the character component table search and reduce the scan amount of the condensed text will be described. The control method will be described. To create a frequency information hashing type character component table, the use frequency of characters in a document registered in a database is checked, and a hash function is determined based on the frequency information. For a character with a high frequency, another character is prevented from entering the same entry as much as possible, and a hash function is adjusted so that a character with a low frequency includes a plurality of characters in the same entry. By doing so, an averagely stable narrowing-down rate can be obtained by the character component table search. Specifically, as shown in FIG. 17, based on the SCODE obtained by the equation (4-1), the number of documents using the corresponding character in the database once is checked and sorted in the order of frequency. Next, from the most frequent one, N
Take only t. Then, leaving only the entry having the highest frequency in the frequency number distribution within Nt, the other entries are sequentially assigned entry numbers of Nt or more. In order to assign an entry number equal to or greater than Nt, an entry having a higher frequency than Nt is assigned so that the (Nt + 1) th entry is the Nt entry and the (Nt + 2) th entry is the (Nt-1) th entry. Go. In the assignment process, no other entry is assigned above the entry having the highest frequency. The allocated entries are in the form of a table as shown in FIG.
The hash function is configured by referring to the stored table. That is, it can be seen that the character “check” having the SCODE of (095F) H is the entry number 231.

The control procedure of the hierarchical presearch is the same as that of the fifth embodiment. That is, using the control procedure of FIG. 14 as it is, first, a character component table search is performed using the characters in the search term, and second, a condensed text search is performed using the search term. If no context condition or the like is specified, the search is terminated here. If not, a third text search is performed and the result is output. As described above, according to the present embodiment, a character component table is created based on the frequency distribution of characters actually used in the database, so that a high narrowing-down rate can always be obtained stably in the character component table search. Therefore, a short search processing time can be stably obtained without depending on the search term.

As described above, five embodiments have been described with respect to different embodiments of the character component table. An embodiment with a different condensed text will now be described. Although the process of creating the condensed text used in the first embodiment is simple, as shown in FIG. 4, even a character string such as "for" which is not originally used in the search remains in the condensed text. become. This leads to a reduction in the compression rate of the condensed text. That is, the amount of condensed text to be scanned at the time of retrieval increases, so that the retrieval processing time increases. The main factor that reduces the compression ratio of the condensed text is that a string of adjuncts such as "for" that has no meaning in itself is registered in the condensed text. .

Therefore, as an eighth embodiment, a hierarchical pre-search using a condensed text from which a series of attached words unnecessary for this search are removed will be described. This condensed text is divided into character types
This is called a deduplication / adjunct removal type condensed text. As shown in FIG. 19, the method of creating the condensed body text is to separate the character string from the text character string of the body body into partial character strings, remove duplicate words from the partial character strings, and then remove attached words. Character type division and deduplication are the same as those in the first embodiment. Adjunct removal is performed on de-duplicated hiragana character strings. The analysis for removing the attached word is performed based on the basic word dictionary and the connection rules between words as shown in FIG. The basic word dictionary contains
As shown in FIG. 21, a verb composed of only hiragana, a demonstrative pronoun, an adjective, an adjective verb, an adverb, a conjunction, a particle, an auxiliary verb, and a conjugation ending of these parts of speech are registered together with part of speech information. In the example of this figure, the verbs <a> and <
<Naru>, <Hitaru>, etc. are registered together with their endings. In the connection rules, each word registered in the basic word dictionary is registered with which other words can be connected. For example, FIG.
As shown in the table, <verb-adjunct form> has <noun-koto>
Are connected, and it is registered that <noun-> can be connected to <noun-to>. Using such a basic word dictionary and the connection rules, it is determined whether or not the partial character string of Hiragana is composed of an accessory word, and whether or not the character string is registered in the condensed text is determined. For example, the substring "for" is <particle-><noun-for><
Since it can be analyzed as a connected character string such as a particle->, it is determined and removed as a character string composed only of attached words. On the other hand, the character string “ambiguity” cannot be analyzed as an adjunct word, so that it is registered in the condensed text without excluding it.

As described above, the compression rate of the condensed text can be increased by analyzing the attached words to eliminate the hiragana character string and deleting unnecessary information that is not used in the search. Basic word dictionaries and connection rules used for analysis are fundamentally different from conventional keyword dictionaries in which the number of registered words increases with the times, and they are universal and need not be updated once created There is an advantage.
In order to exclude only those that can be analyzed as adjuncts, even if a new word composed of hiragana that does not exist in the dictionary appears, it will always remain in the condensed text.

Next, the control of the hierarchical pre-search method using the condensed text of the character type division / duplication elimination / adjunct word removal type will be described. In the condensed text of the character type division / duplication elimination / adjunct removal type, there are cases where the hiragana character string is analyzed as an adjunct and not registered in the condensed text. For this reason, when an attempt is made to search for a specific hiragana character string, the search may be missed in the condensed text search. For example, the character string “vertigo”
It can be analyzed that the verb inflected form ending "me" is connected to the auxiliary verb "mai". A specific example is "no admission". However, even if "vertigo" is used as a noun, it is deleted from the condensed text as a result of the attached word removal processing. Therefore, in such a case, if the condensed text is searched for with “vertigo”, there is a possibility that the search may be omitted. Therefore, it is necessary to check whether the search term is an original word in the condensed text or a word that may have been removed in the process of creating the condensed text, and then search. To check whether the search term is a word to be registered in the condensed text, the algorithm for removing attached words used when creating the condensed text is applied as it is. In this example, when a search term “vertigo” is given, this can be determined to be a series of attached words.

The procedure of the above search control will be described with reference to FIG. First, a character component table search is performed. If the number of results is 0, 0 is output as a search result, and the search process ends. As described in the first embodiment, in the method not using the hash function, the search result of the character component table can be output as the final search result only when the search term is one character. That is, when using the character component table described in the first and fourth embodiments, it is checked whether or not the search term is one character, and if it is one character, the result of the character component table search is output as the search result. , End the process. When the character component table based on the hash function described in the fifth, sixth, and seventh embodiments is used, the following condensed text search is always performed without checking whether or not this search term is one character. Thereafter, similarly to the first embodiment, a divided search term is generated.

Next, attached word analysis is performed for each of the divided search terms. If any of the split search terms is determined to be an adjunct, the split search term may have been deleted from the condensed text, so the condensed text search is not performed and the search is performed based on the result of the character component table search. Search the text directly. On the other hand, as a result of the attached word analysis, if it is determined that all of the divided search terms are not attached words, a condensed text search is performed as in the first embodiment. If no neighborhood condition or context condition is specified, or if the divided search term is the same as the original search term, the result of this condensed text search is output as the final search result, and the search is terminated. If the neighborhood condition or the context condition is specified, or if the divided search term and the original search term are different, then a text search is executed, and the result is used as the final search result output. As described above, according to the present embodiment, the condensed text is analyzed by analyzing the hiragana character string, and using the condensed text of the character type division / duplication elimination / adjunct word removal type in which unnecessary attached word sequences are removed from the condensed text. The compression ratio can be improved and the search processing time can be reduced.

Next, as a ninth embodiment, a description will be given of a hierarchical presearch using a character-type division / duplication elimination / Hiragana character string removal type condensed text in which all Hiragana character strings are excluded. The condensed text described in the eighth embodiment certainly increases the compression ratio, but may erroneously analyze the attached words. For example, besides the example of the character string “vertigo” used in the eighth embodiment, there are rare cases where it is essentially impossible to correctly determine which of the adjuncts is just by adjunct analysis.
For example, the substring “Toshino” in the document “Operating in this application ...” is used to mean “to this to” or “to , It is difficult to determine whether or not leverage means a mechanical lever. When used in the latter sense,
When the search term “leverage” is specified, “leverage” is not determined to be an adjunct, so that the condensed text is searched. On the other hand, in the creation of the condensed text, "Shiteko" is analyzed as an adjunct word and is deleted from the condensed text. In order to correct the incompleteness of the attached word analysis, the ninth embodiment implements a hierarchical pre-search by a simple determination method of whether or not it is a hiragana character string. FIG. 24 shows a method of creating the condensed text. In this method, after character type division, Hiragana is removed to eliminate duplicate registration.

The control procedure of the hierarchical pre-search using the character type division / duplication elimination / Hiragana character string removal type condensed text will be described with reference to FIG. First, a character component table search is performed as in the eighth embodiment. Thereafter, a split search term is generated. Next, it is checked whether each of the divided search terms is a hiragana character string. If at least one of the divided search terms has a hiragana character string, the text is directly searched based on the result of the character component table search without performing the condensed text search. On the other hand, if there is no hiragana character string in the divided search term, a condensed text search is performed in the same way as in the first embodiment, and the neighborhood and context conditions are specified, or the divided search term is different from the original search term Has
Continue the search process until the text search. As described above, according to the present embodiment, an accurate full-text search with no search omission can be realized even for a hiragana character string by using a condensed text in which all the hiragana character strings are excluded.

Next, a tenth embodiment of the present invention will be described. In the ninth embodiment, when a search term of Hiragana is given, it is necessary to directly refer to the text. Therefore, more search time is required. Therefore, a tenth embodiment will be described as a method for performing a full-text search at a high speed even when a hiragana search term is given. In the present embodiment, in addition to the condensed text used in the ninth embodiment, a condensed text in which the removed hiragana character strings are registered in the ninth embodiment is separately created. As shown in FIG. 26, after character type division and elimination of duplicate registration, it is determined whether or not the remaining partial character string is a Hiragana character string, a character string other than the Hiragana character string is registered as a condensed text A, and the Hiragana character string is condensed text. Register as B. In this way, when a search term of only Hiragana is given, the condensed text B can be searched, so that the search time can be reduced. FIG. 27 shows the procedure of actual hierarchical presearch search control.
Shown in First, a character component table search is performed as in the eighth embodiment. If the search result is 0, the search ends here. Thereafter, a split search term is generated. Next, the divided search terms are classified into terms consisting of hiragana character strings and terms consisting of other character strings. Thereafter, if there is a divided search term consisting of a character string other than Hiragana, the condensed A is searched. Next, if there is a Hiragana split search term, the condensed B is searched. Thereafter, similarly to the first embodiment, when there is a designation of a neighborhood or a context condition, or when the divided search term is different from the original search term, the search processing is continued until the text search. In this way, by storing separately the condensed text of only Hiragana and the condensed text of other than Hiragana, the condensed text can be used effectively regardless of the search term of any character type, and high-speed full-text search is always performed. realizable.

Next, an eleventh embodiment will be described.
The present embodiment is based on a method of using an independent condensed text for each character type in order to increase the compression rate of the condensed text. The condensed text used in this embodiment is referred to as a character type division / duplication elimination / character type registration type condensed text. Figure 2 shows how to create a condensed text body with this character type division / duplication elimination / character type registration.
As shown in FIG. 8, after character type division and duplicate registration elimination, the character type of the remaining partial character string is determined, and Hiragana condensed text H, Katakana condensed text I, Kanji condensed text J, English condensed text K, and numeric condensed text It is classified and registered as a text L, a symbol or other character type condensed text M. By doing so, for example, when a search is performed using a kanji search term, only the condensed text J of the kanji character type needs to be searched, so that the search time can be further reduced. A specific hierarchical pre-search control procedure will be described with reference to FIG. First,
A character component table search is performed as in the eighth embodiment. If the number of search results is 0, the search ends here. Thereafter, a split search term is generated. Next, the divided search terms are classified for each character type. Thereafter, a condensed text to be searched is selected according to the character type of the decomposed search term, such as condensed H when there is a hiragana split search term, condensed I when there is a katakana split search term, and so on. Thereafter, similarly to the first embodiment, when there is a designation of a neighborhood or a context condition, or when the divided search term is different from the original search term, the search processing is continued until the text search. In this way, by separating the condensed text file for each character type and reducing the volume of each condensed text, full-text search with single character type search terms such as kanji only, katakana only, or hiragana only can be performed at high speed. The effect that it can be performed is obtained.

Next, a twelfth embodiment will be described with reference to FIGS. This embodiment is described in Japanese Patent Application No.
The present invention is realized by using the document search device proposed in 193015. The main configuration of this device is a keyboard 3001, a search expression analysis program 3002, a bit search processor 3007a, a string search engine 3
006, Complex Condition Determination Microprocessor 3045
a, search result storage memory 3046, display 302
0, a semiconductor memory device 3010a, a RAM disk device 3010b, a collective magnetic disk 3010c, and a search execution control program 3008. A character component table is stored in the semiconductor memory device 3010a.
Reference numeral 010b denotes a condensed text, a collective magnetic disk device 301
0c stores the text. However, the character component table and the condensed text are stored on the collective magnetic disk 3010c, and the semiconductor memory device 3010a and the RAM disk device 301, respectively, at the start of operation of this device.
0b.

The procedure of the hierarchical presearch control is the same as that described in the embodiment. The difference from the previous embodiments is that the character component table is stored in a semiconductor memory, the condensed text is stored on a RAM disk, and the text is stored on a collective magnetic disk. That is, a string search engine dedicated to search is used. The procedure of the search process will be described below.

The search condition expression input from the keyboard 3001 is a search machine control microprocessor MPU03.
050 is analyzed by the search formula analysis program 3002. In other words, the search formula analysis program 3002 separates the keyword portions constituting the search condition formula into the decoding condition description portion which describes the inclusion condition and the placement condition. The inclusion condition is described as a logical condition, and the arrangement condition is described as a neighborhood condition or a context condition. After separation and extraction,
The keyword part is also passed to the synonym expansion program 3003 on the MPU03050, and the decoding condition description part is passed to the decoding condition analysis program 3041. In the synonym expansion program 3003, the synonym of the input keyword is obtained with reference to the built-in synonym dictionary. Then, the keyword group that has undergone synonym expansion is passed to the variant notation expansion program 3004. In the case of the example in this figure, “computer”, “computer”,
“COMPUTER” or the like is generated. In the different notation development program 3004, a different notation development process is performed on the keyword group input here. In the case of the example of this figure, “computer” is changed from “computer” to “C”.
"Computer" or the like is generated from "OMPUTER". The keyword group thus expanded into the synonym and the different notation is then processed by the microprocessor M for automaton generation.
Automaton generation program 3 on PU13005a
005. Automaton generation program 30
At step 05, an automaton for collectively collating the keywords sent from the different notation development program 3004 is generated, and the search engine 300 is used as the identification code information of the keyword to be collated with the state transition table.
Set to 6. The search engine 3006 is a high-speed multiple character matching circuit based on the finite automaton method. Also,
The keyword group expanded in different notations by the different notation expansion program 3004 is passed to the bit search program 3007 on the bit search microprocessor MPU33007a together with the corresponding keywords.

On the other hand, neighborhood conditions, context conditions, AND, O
Logical conditions such as R are obtained from the search expression analysis program 3002,
The complex condition analysis program 3041, the neighborhood condition analysis program 3042, the context condition analysis program 3043, and the logical condition analysis program 3044 are sent to the complex condition determination program 3045. Necessary search information is bit search program 3007, string search engine 3
006, after being sent to the complex condition determination program 3045, the search control execution program 3008 first activates the bit search program 3007. The bit search program 3007 reads a character component table stored in the semiconductor memory device 3010a and performs a character component table search. The result of the character component table search is stored in the search result storage memory 3046.

After the character component table search is completed, the search execution control program 3008 executes the search result storage memory 3046
If the search result is 0, 0 is output as the search result and the search process is interrupted. If the search result is not 0, the string search engine 3006 is activated, and at the same time, the condensed text of the document hit by the result of the character component table search stored in the search result storage memory 3046 is read from the RAM disk device 2910b. The text is sent to the string search engine 3006, and the condensed text search is executed. As a result, the condition determination as to whether or not the number of cases is 0 is performed by the search execution control program 3008. In the string search engine 3006, the RAM disk device 3
The condensed text read from 010b is searched by the divided search term. The comparison result is a compound condition determination program 30
45. Complex condition determination program 3045
Then, the logical condition given between the search terms is determined, and the document number of the document that satisfies the condition is determined.
The data is sequentially stored in 46.

After the condensed text search is completed, the search execution control program 3008 refers to the search result storage memory 3046 again, and if the number of results is 0, outputs 0 as a search result and terminates the search. . When the search result document number is read from the search result storage memory only when the search condition is not zero and the neighborhood, the context condition is set, or the divided search term is different from the search term,
The corresponding text is stored in the collective magnetic disk drive 3010.
c, and sends it to the string search engine 3006, this time executing a text search. If the neighborhood and context conditions are not set and the divided search terms are equal to the search terms, the number of search results stored in the search result storage memory is output, and the search is terminated.

In the string search engine 3006,
The text read from the collective magnetic disk device 3010c is scanned to perform text search. The results are sequentially sent to the complex condition determination program 3045. The compound condition determination program 3045 determines not only the logical conditions given between the search terms, but also the neighborhood and the context conditions, and sequentially stores the document numbers of the documents meeting the conditions in the search result storage memory 3046. When the text search is executed, after the text search is completed, the search execution control program 3008 refers to the search result storage memory 3046, outputs the number of search results, and ends the search. In this way, large-volume text data is stored on the magnetic disk,
By storing in semiconductor memory or RAM disk,
High-speed full-text search can be realized even for a large-scale database.

Next, a description will be given of a thirteenth embodiment in which the condensed text is stored on a magnetic disk. When the condensed text is stored on the magnetic disk, processing can be performed at a higher speed than by performing a normal hierarchical presearch using the same configuration by optimizing the control procedure of the hierarchical presearch. Hereinafter, the procedure of this control will be described. A magnetic disk usually has a magnetic head that moves mechanically.
For this reason, there is a feature that it is faster to collectively read out information (referred to as sequential access) than to read out information on the disk step by step (referred to as skip access). Now, assuming that the read speed of skip access is Vskip MB / s and the read speed of sequential access is Vseq MB / s, the number of documents in all databases is Na, the number of results in the character component table search is Nc, and Assuming that the capacity is uniform, when Nc> (Vskip / Vseq) · Na (12-1), it is better to search all the condensed texts by sequential access based on the result of the character component table search. Processing time is shorter than skip access. Therefore, as shown in FIG. 32, after the character component table search, the number of results is determined in the hierarchical pre-search control program.
When the number of hits satisfying the expression (12-1) is reached,
Search the condensed text for the entire database, ignoring the results of the character component table search. Using the above method, a large amount of RAM is required to store the condensed text on the magnetic disk.
It is not necessary to use a disk, and a relatively high-speed full-text search can be realized by a low-cost document search device.

Next, a description will be given of a fourteenth embodiment in which the condensed text is stored on a magnetic disk. When the neighborhood and context conditions are specified, if the result of the character component table search is very small, it is better to search the text directly based on the result of the character component table search without performing the condensed text search. Be shorter. Now, the search speed of the condensed text is Vsr MB /
s, the search speed of the text is VtxMB / s, the number of results in the character component table is Nc, and the number of results in the condensed text is Ns
r, where Qsr is the data capacity per case of the condensed text and Qtx is the data capacity per case of the text, NcQsr / Vsr + NsrQtx / Vtx> NcQtx / Vtx When the expression Performing a text search directly without a text search shortens the search time. Nsr is not known until the condensed text is actually searched, but a constant is set in advance to determine whether to perform the condensed text search. For example, assuming that the number of documents in the entire database is Na, Nsr = αNa (0 <α <1)... (13-2) Expression (13-2) is transformed to obtain Nc <αNa (Qtx / Vtx). / (Qtx / Vtx-Qsr / Vsr) In the case of Expression (13-3), the text search is directly performed. A value is set in advance before the search using α as a threshold value, and after the character component table search, it is determined whether or not to perform a condensed text search according to the equation (13-3). By performing this control, a high-speed full-text search can be realized under the specification of the neighborhood and context conditions. In the foregoing, the thirteenth and fourteenth embodiments for realizing a full-text search with a low-cost system configuration of the twelfth embodiment have been described.

In addition, the hierarchical pre-search can also be realized by a control method that directly executes the text search from the character component table search without using the condensed text at all and omitting the step of the condensed text search. According to this method, it takes a little time to search because the amount of text to be scanned increases, but it does not require the use of an expensive RAM disk, and does not require the capacity of a magnetic disk for storing condensed text. Document retrieval device can be realized. Also, without using the character component table, the entire text search on the RAM disk or magnetic disk is directly searched, and the text search is performed only when there is a search condition specification of the positional relationship between search terms such as neighborhood and context conditions. Hierarchical pre-search can also be realized by a control method. According to this method, the search time is slightly increased because the amount of search for the condensed text increases, but a semiconductor memory for storing the character component table is not required, so that a low-priced document search device can be realized.

As shown in FIG. 33, the character component table in the bit list format used in the embodiments described above is a format in which characters appearing in a document are arranged, that is, one character is represented as one bit. Instead, a character component table stored as it is as the character code itself can be used.
Alternatively, at this time, the capacity of the character component table is reduced by associating a plurality of characters with one character entry by using the hash function described in the fifth, sixth, and seventh embodiments. You can also. A character component table search using a character component table that stores character codes in this way is realized by reading data from the file character by character and determining whether or not the corresponding character exists, as in the case of a condensed text or text search. it can. In this way, by using a character component table that collects only the characters used in the text, the data structure can be simplified, and the condensed text,
Since the same scan type search as the text search can be used, the effect that the search processing method can be simplified can be obtained.

Further, even when the character component table is also stored on the magnetic disk, the hierarchical presearch can be realized. When the character component table is stored in the magnetic disk, the bit operation process is performed while sequentially reading the bit list of the characters used in the search term from the magnetic disk in the character component table search. Alternatively, if the above character code is used directly as a character component table,
While sequentially reading the character component table, a document including all the corresponding characters is selected. According to the method of storing the character component table on the magnetic disk, it is not necessary to use a semiconductor memory, so that a lower-priced document search device can be realized.

[0057]

According to the present invention, a useless text search can be performed by using a text component table and a condensed text to hierarchically filter out documents that are not related to a search term input at a text level and a word level. Since it can be omitted, it becomes equivalently a means for realizing a high-speed full-text search, and it becomes possible to perform a full-text search with a practical response speed even in a large-scale document database.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment of the present invention.

FIG. 2 is a diagram showing a registration process for hierarchical presearch, which is a feature of the present invention.

FIG. 3 is a diagram illustrating a search process of a hierarchical presearch, which is a feature of the present invention.

FIG. 4 is a diagram showing an example of creating a condensed text.

FIG. 5 is a diagram showing a storage form of a condensed text.

FIG. 6 is a diagram showing an outline of a character component table.

FIG. 7 is a diagram showing an outline of a character component table search.

FIG. 8 is a diagram illustrating a processing procedure of a hierarchical presearch.

FIG. 9 is a diagram showing a character component table search process according to the third embodiment.

FIG. 10 is a PAD diagram showing a code conversion process of a character component table used in the fourth embodiment.

FIG. 11 is a diagram illustrating an outline of code conversion of a character component table used in the fourth embodiment.

FIG. 12 is a diagram showing an outline of a character component table used in a fourth embodiment.

FIG. 13 is a diagram showing an outline of a character component table used in the fifth embodiment.

FIG. 14 is a diagram showing a processing procedure of a hierarchical presearch used in the fifth embodiment.

FIG. 15 is a diagram showing an outline of a character component table used in the sixth embodiment.

FIG. 16 is a diagram showing a processing procedure of a hierarchical presearch used in the sixth embodiment.

FIG. 17 is a diagram showing an outline of a method of creating a character component table used in the seventh embodiment.

FIG. 18 is a diagram showing an outline of a character code-entry number correspondence table used in a hash function for a character component table used in the seventh embodiment.

FIG. 19 is a diagram showing a method of creating a condensed text used in the eighth embodiment.

FIG. 20 is a diagram illustrating a method of processing a hiragana character string for a condensed text used in the eighth embodiment.

FIG. 21 is a diagram showing a basic word dictionary for attached word analysis used in the eighth embodiment.

FIG. 22 is a diagram showing connection rules for attached word analysis used in the eighth embodiment.

FIG. 23 is a diagram showing a processing procedure of a hierarchical presearch used in the eighth embodiment.

FIG. 24 is a diagram showing a method of creating a condensed text used in the ninth embodiment.

FIG. 25 is a diagram showing a processing procedure of hierarchical presearch used in the ninth embodiment.

FIG. 26 is a diagram illustrating a method of creating a condensed text used in the tenth embodiment.

FIG. 27 is a diagram illustrating a processing procedure of hierarchical presearch used in the tenth embodiment.

FIG. 28 is a diagram illustrating a method of creating a condensed text used in the eleventh embodiment.

FIG. 29 is a diagram illustrating a processing procedure of hierarchical presearch used in the eleventh embodiment.

FIG. 30 is a diagram showing a part of the configuration of a twelfth embodiment.

FIG. 31 is a diagram showing the remaining part of the configuration of the twelfth embodiment.

FIG. 32 is a diagram illustrating a processing procedure of hierarchical presearch used in the twelfth embodiment.

FIG. 33 is a diagram showing an outline of a character component table stored as characters.

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hisamitsu Kawaguchi 1-280 Higashi Koigakubo, Kokubunji-shi, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. (56) References JP-A-57-137965 (JP, A) JP-A-63-244259 (JP, A) JP-A-2-253474 (JP, A) JP-A Sho 62- 211728 (JP, A) JP-A-59-112339 (JP, A) Patent 2986865 (JP, B2) Kanji Kato et al., "Development of text search machine for full-text search", IEICE technical report (DE89-38) ) Vol. 89 No. 335, 1989 (Heisei 01-12-14), pp. 17-24 Tadakazu Kikuchi et al., "High-speed full-text search by coding including attribute / character position of constituent characters", IEICE Technical Report (DE90-24) Vol. 90 No. 362, 1990 (Heisei 02-12-14) pp. 1-7 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06F 17/30 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. A full-text search method for searching for a document including a keyword specified by a searcher by referring to the text content of a document database in which the document information is stored as character code data. When registering, the body character string of the registered document is divided for each character type such as hiragana, kanji, and alphanumeric characters, and after removing all the hiragana character strings, between each partial character string divided for each character type. A step of creating a condensed text consisting of a set of substrings excluding character strings included in other character strings by mutually examining the inclusion relation of the character strings, and a character in which characters appearing in the original text are registered without duplication A step of creating a component table, and a step of registering, in addition to the text of the document to be registered, a plurality of condensed texts and a character component table corresponding to the character type in the document database. In the search, first, a character component table search step of extracting a document including all types of characters constituting the keyword specified by the searcher by referring to the character component table; It is determined whether or not a hiragana character string is present in the partial character strings constituting the keyword. If there is no hiragana character string, the specified partial character string is referred to by referring to the condensed text of the document extracted by the character component table search. By the step of condensed text search that extracts only documents corresponding to the
Select a document containing the specified keyword, and then refer to the text of the document narrowed down in this way to extract only those that satisfy the search conditions such as the positional relationship given between the keywords. Refers to the original text of the document extracted by the character component table search, and extracts only the text that includes the specified partial character string and satisfies the search conditions such as the positional relationship given between keywords. A full-text search method, wherein an equivalently high-speed full-text search is performed by a search step.