JPH0877188A - Full text search method - Google Patents

Abstract

PURPOSE: To reduce the quantity of an index, and to retrieve a character as the same character or as a different character as well. CONSTITUTION: The index is generated by considering each pair of the capital letter and the lower case letter of an English letter to be the same character. Character information to distinguish the pair is generated as a part of a leaf in addition to the position information of a character string in a document. In a step S1, a retrieval character string is inputted. In the step S2, the retrieval character string is converted into the character string being used in the index. In the step S3, retrieval is executed by comparing the converted character string and the index. In the step S4, it is discriminated whether the leaf 2 is reached or not, and in the case that the leaf is not reached, in the step S10, 'no corresponding character string' is returned. In the step S7, the character information in the reached leaf is fetched. In the step S8, the character information is compared with the retrieval character string, and if they match with each other, in the step S9, place information is returned.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a full text search method for searching a character string in a document.

[0002]

2. Description of the Related Art With the spread of CD-ROM and the like, a large amount of documents have been digitized. Typical electronic documents include electronic libraries, electronic publications, online manuals installed in computer devices, and the like. Along with this, a technology for efficiently and properly searching these documents and extracting desired information has been required. The full text search method is a search method in which the entire body of a document is directly searched for matching. In order to search the entire document, a direct collation method is used from the beginning, and a database consisting of a leaf that indicates the positional information of the characters in the document and an index for searching the leaf is created in advance and pre-processed. There are two major methods of searching this database.

In full-text search, how to handle uppercase and lowercase letters of English characters, full-width and half-width, full-width and half-width of numbers, Arabic and Chinese numerals of numbers, katakana and hiragana, full-width and half-width of katakana becomes a problem. . When searching for the same character, the matching rate (the ratio of the number of found items that match the desired result) is also a problem. In a system that is used by both general users and experts, it may be better for general users to search as the same character without distinguishing between uppercase and lowercase letters, and conversely distinguishing as an expert. Thank you for searching.
For example, in the case of a literature search in a library, general users will carelessly search whether some letters are in uppercase letters or lowercase letters. You will want to reduce the number of searches and increase the precision.

[0004]

However, the conventional full-text search method has the following problems. In the conventional full-text search method, the upper and lower case letters are regarded as completely different characters or the same characters, and the index is fixedly created, and there is nothing that can be selected at the time of search, which is inconvenient. In addition, when considering different characters, there is a problem that more indexes are required, storage capacity is increased, and search time is longer than when considered as identical characters.

[0005]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention uses a leaf representing position information of a character string in a document and an index for searching the leaf to identify characters in the document. In the full-text search method for searching a column, the following processing is executed. That is, uppercase and lowercase letters, full-width and half-width of letters, full-width and half-width of numbers,
Regarding the pairing of Arabic numerals and Chinese numerals, Katakana and Hiragana, and full-width and half-width Katakana, all or part of the pairs are regarded as the same character and the index is created, and the index creation process A leaf creating process for creating character information that distinguishes all pairs or some of the pairs as a part of a leaf is executed.

Then, when a search character string is designated, a search character string conversion process for converting the search character string is performed in the same manner as the index creation process in the case where the pair is regarded as the same character, and the search character string conversion process. Index search processing for searching the index based on the character string converted by the processing, and when the leaf is reached by the index search processing, the position of the leaf in the search mode without distinguishing the pair When the character string in the document is searched based on the information, and the mode in which the pair is distinguished and the search is performed, the character information of the leaf is compared with the search character string, and the document is based on the position information of the matching leaf. The leaf search process for searching the inside character string is executed.

[0007]

According to the present invention, since the full-text search method is configured as described above, the upper and lower case letters of alphabets, full-width and half-widths of numbers, full-width and half-widths of numbers, Arabic numerals and kanji of numbers are created by the indexing process. , Katakana and Hiragana, and full-width and half-width Katakana pairs, all of these pairs or some of them are regarded as the same character and an index is created. By the leaf creation process, in addition to the position information of the character string in the document, character information that distinguishes all or some of these pairs of uppercase and lowercase letters of the character string of the character string is created as part of the leaf. To do. Upper case and lower case are distinguished by the character information of this leaf. When the search character string is specified by the search character string conversion process, the search character string is converted in the same manner as the index creation process when the pair is regarded as the same character. The index search process searches the index based on the character string converted by the search character string conversion process. In this index search process, uppercase and lowercase letters of English letters are treated as the same character. When the leaf is reached by the index search process, the character string in the document is stored based on the position information of the characters in the document stored in the leaf in the mode in which the leaf search process searches the pairs without distinguishing them. In the mode of searching for a pair and distinguishing the pair, the character information of the leaf is compared with the search character string, and the character string in the document is searched based on the position information stored in the matching leaf. . Therefore, the above problem can be solved.

[0008]

2 (a) to 2 (d) are diagrams showing the structure of a database for carrying out the full-text search method according to an embodiment of the present invention. 2A is a diagram showing the structure of the entire database, FIG. 2B is a diagram showing the structure of the leaf in FIG. 2A, and FIG. 2C is a diagram showing the character information of FIG. 2B. The figure showing the structure and the figure (d) are full-width and
Half-width bit 2b-1 and upper / lower case bit 2b-2
It is a figure which shows the content of. As shown in FIG. 2A, this database for searching a search character string is composed of an index 1 and a leaf 2. The index 1 is a record that stores information for searching the leaf 2, and the leaf 2 is a record that stores the position of a character string in a document. The index 1 is realized by a tree structure, a hash structure, or the like. Figure 2 (b)
As shown in FIG. 2, the leaf 2 in FIG.
It is composed of a and character information 2b. Location information 2a
Stores information indicating the position of the character string in the document. The character information 2b stores character information corresponding to the character string in the search process.

As shown in FIG. 2 (c), the character information 2b in FIG. 2 (b) includes the full-width / half-width bit 2b-1 and capital letters.
It is composed of lowercase bits 2b-2. The full-width / half-width bit 2b-1 is information for identifying whether the character in the document is full-width or half-width, and the uppercase / lowercase bit 2b-2 is an uppercase letter when the search character is fullwidth. This is information for identifying whether it is lowercase or lowercase. The uppercase / lowercase bits 2b-2 are also used for Katakana / Hiragana bits and Arabic / Kanji numeral bits at the same time. As shown in FIG. 2D, full-width / half-width bit 2b
-1 is set to on for full-width characters and off for half-width characters in the case of letters, numbers, or katakana. Upper and lower case bits 2b-2 are on for uppercase letters in case of English letters,
It is set to off for lowercase letters, on for Arabic numerals, off for lowercase letters, off for Katakana, off for Hiragana. First Embodiment FIG. 1 is a flowchart of a full text search method showing a first embodiment of the present invention.

The full-text search method will be described below with reference to these figures. In the full-text search method of the first embodiment, index creation processing, leaf creation processing, search character string conversion processing, index search processing, and leaf search processing are executed. [Index creation process] In the index creation process,
The index 1 (for example, a tree structure) is created as follows to search the leaf 2 that indicates the position of the character string in the document. The index 1 is composed of a character string that serves as a key and the position information of the leaf 2 or the position information of the lower level index. In Index 1, upper-case and lower-case alphabetic characters, full-width and half-width alphabets, full-width and half-width numbers, Arabic numerals, kanji Katakana and hiragana-katakana full-width and half-width pairs are treated as the same without distinction. Is given to the key string. [Leaf Creation Processing] In the leaf creation processing, first, the position information 2a of the leaf 2 is created, and then the character strings in the document are converted into information in accordance with the rules shown in FIG. The character information 2b is stored in the leaf 2. [Search Character String Conversion Processing] In step S1 in FIG. 1, a search character string is input. In step S2, the search character string is converted into the character string used in the index 1, and the process proceeds to the index search processing. This is because full-width and half-width are unified in the index 1. [Index search process] In the index search process,
In step S3, the converted character string is compared with the index 1 for searching, and the process proceeds to the leaf searching process. [Leaf Search Process] In the leaf search process, step S4
In, it is determined whether or not the leaf 2 is reached. If the leaf 2 is reached, the process proceeds to step S5. If the leaf 2 is not reached, the process proceeds to step S10. In step S5, the information on the reached leaf 2 is taken out, and the process proceeds to step S6. In step S6, it is determined whether or not the search mode previously designated by the user considers the same characters. If not, the process proceeds to step S7, and if the same characters are considered, the process proceeds to step S9. In step S7, the character information 2b is extracted from the information of the leaf 2 extracted in step S5, and the process proceeds to step S8. In step S8, the extracted character information 2b is compared with the search character string. If they match, the process proceeds to step S9, and if they do not match, step S10.
Go to. In step S9, the position information 2 of the leaf 2
Returns the character string in the document from a. In step S10, "no corresponding character string" is returned and the full-text search ends.

As described above, in the first embodiment, in the index 1, uppercase letters and lowercase letters of alphabetic characters are treated as the same character, so that there is an advantage that the storage capacity can be saved. When searching for a character string, it is convenient because you can search in either a mode that treats uppercase and lowercase letters as the same character or a mode that treats it as a different character, and since the search string is searched only once,
It has the advantage of high execution speed. Second Embodiment FIG. 3 is a flowchart of a full-text search method showing a second embodiment of the present invention. In the second embodiment,
Rather than distinguishing uppercase and lowercase letters uniformly, uppercase and lowercase letters, full-width and half-width of letters, full-width and half-width of numbers, Arabic numbers and kanji of numbers, katakana and hiragana,
It is possible to specify whether to distinguish between full-width and half-width pairs of katakana. For the sake of simplicity, it will be explained as a mode in which upper and lower case letters are treated as different characters. The database of FIG. 2 is also used in the second embodiment.

The full-text search method of the second embodiment will be described below with reference to specific search character strings with reference to FIGS. 2 and 3. Similar to the first embodiment, the index creation process and the leaf creation process are executed, and then the search character string conversion process, the index search process, and the leaf search process are sequentially executed. [Search character string conversion processing] In the search character string conversion processing,
In step S20, the search character string is "A b C1.
It is assumed that the user has input "2 Sanai". In step S21, the search character mask information "1111111111111111" is created. The search character mask information is "11" when the character type distinguishes between uppercase and lowercase, etc. In case of "0
0 "is created for each character type. In the second embodiment,
It is supposed to distinguish all character types. Whether to distinguish is specified by the user for each character type. In step S22, character information corresponding to the search character string "A b C1 2 3 Ai" is created according to the rule shown in FIG. 1 (d). Half-width capital letter “A”, half-width lower case letter “b”, full-width capital letter “C”, half-width Arabic numeral “1”, full-width Arabic numeral “2”, full-width kanji numeral “three”, full-width hiragana “a”, full-width katakana “a” Character information corresponding to is "01001101111
It will be 01011 ".

In step S23, the characters are made common and the character "AB" commonly used in the index 1 is used.
The character is converted to "C123ai" and the process proceeds to the index conversion process. [Index search process] In the index search process,
In step S24, the character string "ABC123ai"
And index 1 are compared to perform a search, and the process proceeds to leaf search processing. [Leaf Search Process] In the leaf search process, step S2
In step 5, whether or not the leaf 2 is reached is checked. If the leaf 2 is reached, the process proceeds to step S26, and if the leaf 2 is not reached, the process proceeds to step S31. In step 26, the information of the leaf 2 reached is extracted. In step 27, the character information 2b is extracted from the head of the information of the leaf 2 which has been reached by the number of search characters.

In step S28, the extracted character information 2b and the character information "0100110111101011" of the search character string are respectively AN with the search character mask information "1111111111111111".
After performing the D operation and masking, these are compared to determine whether the distinguishing character types match. If they match, the process proceeds to step S29, and if they do not match, step S3.
Proceed to 1. In step S29, the location information 2a of the leaf 2 reached is returned. In step S30, it is determined whether another leaf 2 has been reached. If another leaf 2 has been reached, the process returns to step S26, and the same processing is repeated for the number of reached leaves. If all the arrived leaves 2 that have not reached another leaf are not matched in the comparison in step S28, the process proceeds to step S31, and all the arrived leaves 2 that have not reached another leaf have been reached step S2.
If there is a match in the comparison of 8, the full text search is ended. In step S31, "no corresponding character string" is returned, and the full text search ends. As described above, in the second embodiment, since it is possible to specify that no distinction is made for each character type, there is an advantage that the user can perform a full-text search more conveniently.

[0015]

As described above in detail, according to the present invention, by the indexing process, uppercase letters and lowercase letters, full-width and half-width of letters, full-width and half-width of numbers, Arabic numerals and kanji of numbers, and katakana are used. For each pair of hiragana and katakana full-width and half-width, consider all these pairs or some pairs as the same character to create an index, and in addition to the position information of the character string in the document by the leaf creation processing, Character information that distinguishes all or some of these pairs from each pair is created as a part of the leaf, and search character string conversion processing, index search processing, and leaf search processing are executed. Therefore, in the index, the uppercase and lowercase letters of alphabetic characters are regarded as the same, so that the amount of indexes can be reduced. Furthermore, since uppercase and lowercase letters are distinguished according to the leaf character information, uppercase and lowercase letters can be treated as the same or different.

[Brief description of drawings]

FIG. 1 is a flowchart of a full-text search method according to a first embodiment of the present invention.

FIG. 2 is a database structure diagram for implementing a full-text search method according to an embodiment of the present invention.

FIG. 3 is a flowchart of a full-text search method showing a second embodiment of the present invention.

[Explanation of symbols]

 1 Index 2 Leaf 2a Location information 2b Character information 2b-1 Full-width / half-width bit 2b-2 Uppercase / lowercase bit

Claims (1)

[Claims] 1. A full-text search method for searching a character string in the document using a leaf representing position information of the character string in the document and an index for searching the leaf, wherein an uppercase and lowercase alphabetic character, Regarding full-width and half-width of English characters, full-width and half-width of numbers, Arabic numerals and kanji of numbers, katakana and hiragana, and full-width and half-width of katakana, all or some of these pairs are regarded as the same character. An index creating process for creating the index; a leaf creating process for creating the position information and creating character information that distinguishes all or some of these pairs for each pair as part of a leaf; When a character string is specified, the above-mentioned detection is performed in the same manner as the index creation processing in the case where the pair is regarded as the same character. A search character string conversion process for converting a search character string, an index search process for searching the index based on the character string converted by the search character string conversion process, and a case where the leaf is reached by the index search process. In the mode of searching without distinguishing the pair, the character string in the document is searched based on the position information of the leaf, and in the mode of searching by distinguishing the pair, the character information of the leaf and the A full-text search method comprising: performing a leaf search process of comparing a search character string and searching for a character string in the document based on position information of a matching leaf.