JPH0756943A - Whole sentence data base system - Google Patents

Abstract

PURPOSE:To provide a whole sentence data base system which has an index table group small in size, executes whole sentence retrieval at sufficient retrieval speed and executes forward coincidence/backward coincidence/complete coincidence retrieval. CONSTITUTION:An input device 2 inputting document data, a construction processor 3 inserting a position mark into inputted document data and executing an intrinsic pseudo word whose character string length is more than '2' based on the document data, a storage device 4 storing document data and the intrinsic pseudo word as a real data file and an index file, an input device 6 inputting a retrieval character string and a retrieval processor 7 extracting a character address corresponding to the retrieval character string from the index file and outputting the character string in document data corresponding to the extracted character address to a display 9 and provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a full-text database system suitable for electronic publishing.

[0002]

2. Description of the Related Art In electronic publishing that requires a database search, there is a full-text search as a general method for searching desired text data from a plurality of document data stored in a CD-ROM or the like. In full-text search, the document data is stored as a sequential file. The search process for the document data (full-text database) is performed by the search operator inputting a character string included in the document data to be extracted.

Specifically, for example, the input search character string is compared with all the character strings included in the document data,
The document data including the matching character string is output. Note that it is not necessary to compare all the character strings included in the document data with the search character string depending on the number of characters in the search character string and the pattern matching method applied. Since various pattern matching methods are known, the description thereof will be omitted.

[0004]

By the way, from the past,
Card type databases and relational type databases are widely used as means for storing and utilizing various data in computers. In these databases, one card (record) has multiple fields,
Each data is stored in each field according to its attribute. For example, in a card (record) corresponding to document data for each prefecture, fields corresponding to the target prefecture name and survey year are set, and actual data is stored in these fields.

In such a database, for example, if document data whose target prefecture name is "Kyoto" is searched, document data in which actual data "Tokyo" is stored in the target prefecture name field is not extracted. It is possible to extract report data for only the desired “Kyoto”. That is, prefix match, suffix match and exact match search can be performed.

By the way, in the above-mentioned card-type database and relational-type database, since it is necessary to store each data by regularly allocating each data according to its attribute, document data is created from the beginning in consideration of database formation. There is a need to. However, in the field of electronic publishing, C
The document data stored in the D-ROM is often created without considering a database, and it is difficult to construct a card database or a relational database.

Therefore, in the field of electronic publishing, a full-text database that stores document data as a sequential file is used. However, when searching for a character string "Kyoto" in this full-text database, extraction including "Tokyo" is also performed, and a desired search result cannot be obtained. Further, in the field of electronic publishing, document data having a size of tens to hundreds of MBs is often handled, and there is a problem that a sufficient response time cannot be obtained by a search using a sequential file.

The present invention has been made in view of such a background, has an index table group having a small size, performs full-text search at a sufficient search speed, and also performs forward match, backward match and complete match. The purpose is to provide a full-text database system that can perform a search.

[0009]

A full-text database system according to the present invention is a full-text database system for extracting a character string matching a search character string from document data by using an index table group having a hierarchical structure,
Mark inserting means for inserting position marks immediately before and after a specific character string in the document data, address giving means for giving consecutive addresses to each character in the document data, and each character in the document data And a subsequent character are used to create a pseudo word of a total of k characters (k is 2 or more), and each address given to the first character of each pseudo word is
Address table creating means for storing in the address table in the order of the character code of the corresponding pseudo word, and storing the pseudo word having a unique character code as a unique pseudo word in the lowermost layer table in the index table group in the order of the character code. If the number of unique pseudowords stored in the lowest layer constructing means for associating each address in the address table with a unique pseudoword and the uppermost layer table in the index table group is larger than a preset number, the Hierarchizing means for extracting a plurality of unique pseudowords so as to divide the upper layer table substantially evenly, and storing the plurality of unique pseudowords in the upper layer table in the character code order. I am trying.

[0010]

According to the above structure, the mark inserting means inserts the position mark immediately before and after the specific character string in the document data, and the address assigning means inserts an address consecutive to each character in the document data. Give. Then, the address table creating means creates a pseudo word of a total of k characters (k is 2 or more) composed of each character and the following characters in the document data, and attached to the first character of each pseudo word. Each address is stored in the address table in the order of the character code of the corresponding pseudo word. Here, the lowest layer constructing means stores pseudo-words having a unique character code as a unique pseudo-word in the lowest-order table in the index table group in the order of the character code, and each unique pseudo-word has each address in the address table. Correspond to. When the number of unique pseudo-words stored in the uppermost layer table in the index table group is larger than a preset number, the hierarchizing unit divides the uppermost layer table into substantially equal parts. Unique pseudo-words are extracted, and the plurality of unique pseudo-words are stored in a table in an upper layer of the uppermost layer table in the order of character codes. Since the index table group has such a hierarchical structure, a sufficient search speed can be obtained at the time of search. Further, since the pseudo word stored in each table in the index table group is a unique pseudo word having a unique character code, the size of the index table group is small. Furthermore, since position marks are inserted immediately before and after the specific character string in the document data, if the search character string has a position mark as the first character, a prefix match search is performed and the position mark is used as the last character. If it has a suffix match search, and if it has position marks as the first and last characters, a perfect match search is performed.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. (1) Configuration of Construction System 1 FIG. 1 is a diagram showing a schematic configuration of a full-text database system according to an embodiment of the present invention, and FIG. 1A is a block showing a schematic configuration of a construction system 1 for constructing a full-text database. It is a figure. It is assumed that the construction system 1 is used by a database provider (or editor). In electronic publishing, the provider is the system 1
The full-text database constructed by is stored in a CD-ROM or the like.

In FIG. 1A, reference numeral 2 is an input device such as a keyboard, which supplies document data input by an operator to a construction processing device 3 (described later). The construction processing device 3 performs predetermined processing on the document data supplied from the input device 2 and supplies it to a storage device 4 (described later), creates an index file corresponding to the document data, and creates a full-text database. To construct. The contents of this full-text database construction processing will be described later in detail.

The storage device 4 comprises, for example, a hard disk having a capacity of several hundred MB, stores the document data supplied from the construction processing device 3 as an actual data file, and creates an index created by the construction processing device 3. Remember the file. Here, an example of the document data in the actual data file is shown in FIG. A position mark '@' for identifying the keywords "Tokyo" and "Kyoto" is added to the document data shown in this figure.

FIG. 12 shows an example of various tables included in the index file. FIG. 12 is a diagram showing a configuration example of an index table (hereinafter referred to as a Japanese sentence table) for the Japanese sentence range JA of the document data shown in FIG. 4, in which J2-1 is a Japanese sentence lowermost layer table,
J2-2 is the uppermost layer table of the Japanese sentence which is the uppermost layer table of the Japanese sentence lowermost layer J2-1, and J1 is the one-character table, which is the upper layer table of the lowermost Japanese sentence layer table J2-1. CAT is a character address table having a plurality of records R1.

The record J is the lowest layer table J2-1 in Japanese.
Each record R2 has a plurality of 2 and stores a character string of two characters (hereinafter, referred to as a pseudo word) existing in the Japanese sentence range JA (see FIG. 4) of the document data. These pseudo-words are unique (no spelling is the same) and are sorted in the order of the character code CC (hereinafter, the unique pseudo-word is referred to as a unique pseudo-word). Here, assuming that the character code of the first character of the pseudo word is C1 and the character code of the last character of the pseudo word is C2, the character code CC of the pseudo word is calculated, for example, by the following calculation formula (1). CC = C1 × m + C2 (1) Since there are about 7,000 character types that appear in the document data, CC is a pseudo word by multiplying C1 by the number “m (for example, 8,000)” that is greater than this character type. It is a unique character code.

Further, each record R2 corresponds to the record R
2 has a "size" indicating the number of unique pseudowords stored in the document data stored in the character address table CAT.
It is associated with a predetermined record R1 therein. Here, the record R1 with which the record R2 is associated has a “character address” indicating the position of the first character of the unique pseudo word included in the record R2.

In the Japanese sentence lowest layer table J2-1, a plurality of records R1 are associated with a plurality of records R2 having a plurality of sizes. Specifically, the record R2 having a plurality of sizes is associated with the record R1 having the smallest character address among the plurality of character addresses corresponding to the unique pseudo word included in the record R2. Thus, the record R1 having a non-minimum character address is sorted in ascending order.

The Japanese sentence top layer table J2-2 has a plurality of records R3, and each record R3 stores the unique pseudo word in the Japanese sentence bottom layer table J2-1. The number of records R3 is extremely smaller than that of the record R2. Further, each record R3 is sorted in the order of the character code of the unique pseudo word, and is associated with the record R2 having the same unique pseudo word as the unique pseudo word of the record R3. At this time, the bottom layer table J in Japanese
2-1, the distance between the records R2 associated with the records R3, that is, the number of the records R2 existing between the records R2 is set to a preset number (for example, 99). The unique pseudo word stored in the record R3 is set.

The one-character table J1 has a plurality of records R4, and each record R4 stores all types of characters appearing in the Japanese sentence range JA of the document data. These characters are unique, and each record R4 is sorted in the order of the character code. Further, each record R4 is associated with a predetermined record R2 in the Japanese sentence lowest layer table J2-1, and the characters of the associated record R4 and record R2 and the first character of the unique pseudo word match.
Here, when there is a plurality of records R2 to which one record R4 should be associated, the record R4 is associated with the record R2 at the top. On this occasion,
The size of the record R4 is the number of unique pseudo words to be associated.

Next, FIGS. 7 and 9 show an example of an index table (hereinafter referred to as a Western table) corresponding to the European range EA of the document data (see FIG. 4). The European table is a word address table WA as shown in FIG.
T, word list WL, virtual address table VAT, and European basic table EBT as shown in FIG. As shown in FIG. 7, the word address table WAT
Has a plurality of records R5, and each record R5 stores a word unit address (hereinafter, referred to as a word address) corresponding to all "words" existing in the European language range EA.

Further, the word list WL is a record R6.
, Each record R6 has one unique word (hereinafter referred to as a unique word) existing in the European language range EA, and is sorted in the character code order of each unique word. Further, a unique "unique code" is assigned to each record R6, and an "in-word address" indicating the position from the beginning in the unique word is given to each character forming each unique word.

Further, each record R6 has a "size" representing the frequency of occurrence of the unique word of itself in the document data, and is associated with a predetermined record R5 in the word address table WAT. A plurality of records R5 are associated with the record R6 having a plurality of sizes. Specifically, record R6 having a plurality of sizes
Is associated with a record R5 having the smallest word address among the plurality of word addresses corresponding to the unique word of the record R6, and following this record R5, a record R5 having a non-minimum character address.
Are sorted in ascending order.

Also, the European basic table EBT shown in FIG.
Has a plurality of records R7, and each record R7 stores one unique pseudo word for each two characters generated based on each unique word in the word list WL (see FIG. 7). Each record R7 is sorted in the order of the character code of the unique pseudo word. Each record R7 has a “size” indicating the appearance frequency of the stored unique pseudo word in the word list WL, and is associated with a predetermined record R8 in the virtual address table VAT.

The virtual address table VAT is a record R
Each record R8 stores a “virtual address” which is a set of a unique code and an in-word address. This virtual address corresponds to the unique pseudo word in the European language basic table EBT. If the record R7 has a plurality of sizes, the record R
A plurality of records R8 are associated with 7. The specific content of this association is the Japanese sentence bottom layer table J shown in FIG.
2-1 record R2 and character address table CAT
Since it is similar to the association with the record R1 of, the description thereof will be omitted.

(2) Configuration of Search System 5 On the other hand, FIG. 1B is a diagram showing the configuration of the search system 5 that performs various search processes on the full-text database constructed by the construction system 1. This search system 5
Are assumed to be used by users of the database. In electronic publishing, the user uses the system 5 to perform various search processes on a full-text database stored in a CD-ROM or the like.

In FIG. 1B, 6 is an input device such as a keyboard, which retrieves instruction data corresponding to an instruction input by a user (user) 7 (described later).
Supply to. The search processing device 7 is a general personal computer and has a CPU, a ROM, a RAM, and various I / O interfaces (not shown). This search processing device 7 executes a search program stored in RAM,
A search process according to the instruction data supplied from the input device 6 is performed. The details of this search processing will be described later.

Reference numeral 8 denotes a CD-ROM drive, which is controlled by the search processing device 7 and reads the information stored in the inserted CD-ROM. Reference numeral 9 is a display for displaying a search menu, search results, etc. according to the display data supplied from the search processing device 7, and 10 is a printer, which displays the search results according to the output data supplied from the search processing device 7. Output.

(3) Full-text Database Construction Processing Next, the full-text database construction processing executed by the construction processing device 3 (see FIG. 1A) by executing the program stored in the RAM will be described below. Here, a case will be described in which document data (see FIG. 3) in which Japanese and European are mixed is made into a database.

First, prior to the construction of the full-text database,
In the construction system 1 shown in FIG. 1A, the input device 2
Document data is input from. This document data is supplied to the storage device 4 via the construction processing device 3 and stored therein. Each process described below is performed on the document data stored in the storage device 4. When the input processing of the document data is completed and the predetermined instruction data is input from the input device 2, the construction processing device 3 executes the program shown in the flowchart of FIG.

First, at step SA1, a position mark is added to the document data (see FIG. 3). The position mark is a specific character inserted before and after an arbitrary keyword, and a symbol that does not exist in the document data, such as '@', is used. FIG. 4 is a diagram showing the document data after the above-mentioned mark addition processing is performed. In this diagram, the position mark '@' is placed before and after the character strings (keywords) "Tokyo" and "Kyoto". Has been inserted.

[0031] If you do not perform the above-mentioned mark-added processing, and full-text search the place name of "Kyoto", "Kyoto" is, of course, it will be extracted to "east of Kyoto". The mark addition process is a process performed in order to avoid such meaningless extraction. In the full-text search, "@
By inputting the character string "Kyoto @", it is possible to obtain the result that only "@ Kyoto @" is extracted and "@ Tokyo @" is not extracted.

Next, in step SA2 (see FIG. 2),
An address is given to the document data. As shown in FIG. 5, there are a character address and a word address in the address, the character address is given to the entire document data, and the word address is given to the European range EA where alphabets and numbers are continuous. For example, in the Japanese range JA of FIG. 5, the character address of the first character "many" is "1", the character address of the subsequent character "corner" is "2", and the character address of the first character "w" is in the European range EA. The address is "31
7 ”. In the European language range EA, the word address of the first word "world" is "1", and the word address of the subsequent word "wide" is "2".

In each process to be described later, since the processing in units of words is performed on the European range EA, it is not necessary to store the character address in the European range EA. However, in order to grasp the positional relationship with the Japanese sentence range JA, the first and last character addresses of the European sentence range EA are stored in the storage device 4.
(See FIG. 1A). The document data between these character addresses is regarded as the European range EA in each processing described later, and is processed in word units.

Next, in step SA3 (see FIG. 2),
The word list WL of the European range EA is created. First, all the words appearing in the European range EA of the document data of FIG. 5 are extracted, and the European contrast table CT is created as shown in FIG. Next, the records R11 in the European language contrast table CT are sorted in the order of the character code of each word and the order of the word address. Then, a plurality of records R11 having the same word (for example, refer to the word “world” in FIG. 6) are adjacent to each other.

Next, a unique unique word is extracted from each sorted record R11 and stored in each record R6 of the word list WL (see FIG. 7), and a plurality of records R5 having word addresses are created to store words. Create an address table WAT. The pointer provided in each record R6 of the word list WL points to the record R5 in the word address table WAT.

At this time, the unique word of the record R6 and the word address of the record R5, which are associated with each other by the pointer, are the same record R in the European contrast table CT.
The items stored in 11 are the same as each other. Further, when there are a plurality of records R11 having the same word in the European-language contrast table CT, the record R in the word list WL having the unique word matching the word R11.
In 6, the number of appearances of the word in the European language contrast table CT is stored as a size. For example, the word "hel
Since two "p" appear in the Western contrast table CT (see FIG. 6), the unique word "hel" of the word list WL
The size of the record R6 having p ″ is “2”.

Further, each record R of the word list WL
6 includes unique unique codes “A”, “B”, “C”,
Are assigned, and the characters forming the unique word of each record R6 are assigned an in-word address. For example, the unique code given to the unique word "can" is "A", and the in-word address for the character "c" forming the unique word is "1". The word list WL thus created is stored in the storage device 4 (see FIG. 1A).
Memorized in.

Next, in step SA4 (see FIG. 2),
A pseudo word is extracted from the document data or word list WL. In step SA5 following this, the Japanese sentence lowest layer table J2-1 and the European language basic table EBT are created using the extracted pseudowords. Since these extraction processing and creation processing differ depending on the format of the document data, they will be described below separately for the European range EA and the Japanese range JA.

A: Processing on European Scope EA In the European EA, first, the word list WL (FIG. 7) is displayed.
From the reference), a pseudo word having a character string length of “2” is extracted. This extraction processing is performed from the beginning to the end of each unique word. For example, from the unique word "can" to "
Pseudowords "ca" and "an" are extracted. The plurality of pseudowords thus extracted are stored in each record R9 of the pseudoword table PWT having the structure shown in FIG.

In the pseudo word table PWT, the record R9 in which the pseudo word is stored is the unique code given to the unique word from which the pseudo word of the record R9 is extracted and the in-word address of the first character of the pseudo word. It has a "virtual address" composed of For example,
The record R9 having the pseudo word "an" has a unique code "A" added to the unique word "can" of the extraction source.
And a virtual address "A-2" composed of the in-word address "2" of the first character "a" of the pseudo word "an".

Then, as in the case of creating the word list WL, each record R9 in the pseudo word table PWT is sorted in the order of the character code of each pseudo word, and a unique unique pseudo word is extracted. Here, the extracted unique pseudo word is
It is stored in the European language basic table EBT shown in FIG. In addition, a virtual address table VAT composed only of virtual addresses in the pseudo word table PWT is created. Figure 9
As shown in, each record R of the European basic table EBT
The pointer provided in 7 is the virtual address table VA.
It points to the corresponding record R8 in T.

Further, as in the case of creating the word list WL,
In the pseudo word table PWT, when there are a plurality of identical pseudo words, the number of occurrences of the pseudo words is stored as "size" in the record R7 having the same unique pseudo word as the pseudo words. In this way, the European basic table EBT shown in FIG. 9 is created.

B: Process for Japanese sentence range JA In the Japanese sentence range JA, first, a pseudo word having a character string length of “2” is extracted from the document data (see FIG. 5). This extraction processing is performed from the beginning to the end of the Japanese sentence range JA. For example, from the document data of FIG.
Pseudowords “polygon” and “Kakuno” are extracted. The extracted pseudowords are stored in each record R10 of the Japanese sentence pseudoword table JPT as shown in FIG.

In the Japanese pseudo word table JPT, the record R10 in which the pseudo word is stored has the "character address" of the first character of the pseudo word. For example, the record R10 having the pseudo word "polygon" has the character address "1" of the first character "many" of the pseudo word. Then, as in the case of creating the word list WL and the pseudo word table PWT, each record R10 is sorted in the order of the character code of the pseudo word, and a unique unique pseudo word is extracted.

Here, the extracted unique pseudo word is as shown in FIG.
It is stored in the Japanese sentence lowest layer table J2-1 shown in FIG. Further, a character address table CAT composed of only the character addresses in the Japanese pseudo word table JPT is created.
As shown in FIG. 11, the pointer provided in each record R2 of the Japanese sentence lowest layer table J2-1 points to the corresponding record R1 in the character address table CAT.

Also, the Japanese pseudo word table JPT (see FIG.
(See 0), if there are a plurality of identical pseudo-words, the number of occurrences of the pseudo-words in the document data is stored as “size” in the record R2 having a unique pseudo-word that matches the pseudo-words. To be done. In this way, the Japanese sentence lowest layer table J2-1 shown in FIG. 11 is created. As described above, when the European basic table EBT and the Japanese lowermost table J2-1 are created, the process proceeds to step SA6.
Go to.

At step SA6, it is judged whether or not the number of records in the uppermost Japanese sentence table (initially, the lowermost Japanese sentence table J2-1) is equal to or larger than a predetermined number n (for example, n = 500). If this determination is "Yes", the process proceeds to step SA7, and if "No", the process proceeds to step SA8.

The Japanese text table is hierarchized as necessary according to the judgment in step SA6. Here, the reason for hierarchizing will be explained. As described above, the number of character types used in Japanese sentences is about 7,000, and the types of unique pseudo-words extracted from document data, that is, the number of records R2 in the Japanese sentence lowest table J2-1 are extremely large. Since the search process described later is performed by repeating the process of extracting the unique pseudo word corresponding to the search character string, if the number of records R2 in the table to be searched (for example, the Japanese sentence lowest layer table J2-1) is large. , The search process cannot be completed within the predetermined time.

Here, when the number of records R2 in the Japanese sentence lowest layer table J2-1 is large, as shown in FIG.
If the upper layer table (Japanese sentence top layer table J2-2) is created and the unique pseudo word corresponding to the search character string cannot be extracted in the upper layer table, the lower layer table (Japanese sentence bottom layer table J2-1) A unique pseudo word is extracted within a predetermined range. Then, the number of unique pseudo words to be compared is reduced, and the search process can be performed with a predetermined response time.

It is to be noted that the above-mentioned hierarchical structure is not performed for the European basic table EBT because each peculiar pseudo word in the table EBT is a combination of alphabets and numbers having few character types, and therefore the number of records is Japanese. It is extremely smaller than the number of records in the lowermost table J2-1 (there are many pseudowords with the same spelling), and the European basic table E
This is because a sufficient response time can be obtained with BT alone.

Step SA7 is a process when the determination in step SA6 is "Yes", and here,
An upper layer Japanese text table is created for the already created Japanese text table. For example, the Japanese lowest table J shown in FIG.
For 2-1 as shown in FIG. 11, a Japanese sentence top layer table J2-2 is created. This Japanese top layer table J2
In each record R3 of -2, a unique pseudo word extracted from the Japanese sentence lowest table J2-1 such as "@ Higashi" or "Ordinary" is stored. Further, each record R3 has a "pointer" and is associated with the record R2 having the same unique pseudo word in the Japanese sentence lowest layer table J2-1.

The unique pseudo-word included in each record R3 corresponds to each record R2 corresponding to the adjacent record R3.
Distance between records (record R2 included in two records R2
Is extracted so as to be 99, for example. This distance is set according to the number of records in the Japanese sentence lowest layer table J2-1. Then, the process returns to step SA6.
In this way, the above-described layering process is performed until the number of records in the uppermost Japanese sentence table becomes less than the predetermined number n. In the example shown in FIG. 12, the number of records in the Japanese sentence uppermost layer table J2-2 is less than the predetermined number n (for example, n = 500), so the hierarchy of the Japanese sentence table is two.

Step SA8 is a process when the number of data in the uppermost Japanese sentence table is less than the predetermined number n and the determination in Step SA6 is "No", which is a one-character table for the lowermost Japanese sentence table J2-1. J1 is created. Since Chinese characters are used in Japanese sentences, a search may be performed using a single character search character string such as "mountain" or "river". The one-character table J1 is created in order to realize such one-character search at a predetermined response speed.

The process of creating the one-character table J1 will be described below. First, the Japanese language bottom layer table J2- shown in FIG.
The first character of each unique pseudo word is extracted from 1 in the order of records. Since each record R2 has already been sorted in the order of the character code, the same character is continuously extracted. Next, a unique character is extracted from the extracted character group, and the number (size) of the same characters included in the extraction source character group is set to 1
It is stored in each record R4 of the character table J1.

Further, each record R4 has a pointer,
It is associated with the extraction source record R2 in the Japanese lowest table J2-1. In this way, the one-character table J1 is created. Then, for example, the word address table WAT and the word list WL in FIG. 7, the virtual address table VAT and the European basic table EBT in FIG. 9, the character address table CAT in FIG. 12, and the Japanese lowest layer table J2-
1, Japanese uppermost layer table J2-2, 1 character table J1
And the first and last character addresses of the European range EA are stored in the storage device 4 as an index file. Document data as shown in FIG. 5 is stored in the storage device 4 as an actual data file, and a full-text database is constructed.

(4) Full-Text Search Process Next, with reference to the drawings, the full-text search process performed by the search processing device 7 (see FIG. 1B) on the full-text database constructed through the above-described process explain. Figure 1
3, FIG. 14 is a flowchart of the full-text search program stored in advance in the RAM of the search processing device 7. First, in the search system 5, the CD-ROM in which the full-text database is stored is inserted into the CD-ROM drive 8,
When the predetermined instruction data is supplied from the input device 6, the search processing device 7 executes step SB1.

At step SB1, predetermined display data is supplied to the display 9 to display the search menu shown in FIG. 15, for example. The user operates the input device 6 according to the displayed search menu to set a search mode, a rank mode, a designated distance, and the like, which will be described later. In the search mode, in addition to the normal search mode in which one search string is searched, there is a context-aware mode in which a plurality of search strings are searched in consideration of the contextual relationship, and the user operates the input device 6 to operate. A predetermined character is input to the input field 11 to select either mode.

When the branch consciousness mode is selected, the user needs to specify whether or not to consider the context between a plurality of search character strings (ranking mode). Also, in the consciousness mode of division, it is necessary to specify the upper limit of the distance (difference in leading character address) between a plurality of search character strings. Therefore, the user inputs a character designating the rank mode in the input field 12 and a numerical value (designated distance) representing the upper limit of the distance in the input field 13.

Next, in step SB2, the user operates the input device 6 to input the character string input field 1 shown in FIG.
4 or enter a search character string in the character string input field 15. Then, when predetermined instruction data is supplied from the input device 6, the search processing device 7 reads various data input in the input fields 11 to 15 on the search menu and stores these data in the RAM. . Then, the process proceeds to step SB3.

In step SB3, a character address is given to each character of the search character string, the search character string is divided into two character units, and a plurality of search pseudo words are extracted. For example, the search pseudo-words "management" and "crisis" are extracted from the search character string "management crisis" shown in FIG. If the search character string has a length of 2 characters or less, the extraction process is not performed.

Next, in step SB4, the index file stored in the storage device 4 is searched for a record having a unique pseudo word that matches each search pseudo word. This search process is performed by the character code SC of each pseudo word for search.
And the character code VC of each unique pseudo word in the index file are compared. Here, the table used for the search process is the Western basic table EBT for the search in the European range EA, and the top-level Japanese table (for example, the top-level Japanese table J2-2) or one character for the search in the Japanese range JA. It becomes table J1. Then, in step SB5, it is determined whether or not the search processing has been completed for all the pseudo words for search. If this determination is “No”, go to Step SB6, and conversely “Yes”
In the case of, the processing proceeds to step SB9.

Step SB6 is a process when the search process for the pseudo word for search is not completed, and the Japanese sentence table to be searched is the lowermost table (for example, the lowermost Japanese sentence table J2-1). Determine if there is. If this determination is “No”, go to Step SB7,
Conversely, in the case of "Yes", the process proceeds to step SB18 (see FIG. 14).

Step SB7 is a process when the Japanese text table to be searched has a lower table. Here, in the Japanese sentence table to be searched (for example, the Japanese sentence uppermost layer table J2-2), there is a unique pseudo word having a character code VC that is smaller than the character code SC of the pseudo word for retrieval and is closest to the character code SC. A record (hereinafter referred to as an approximate record) is extracted.

Then, the Japanese text table to be searched is set to 1
The lower layer table (for example, the Japanese lowermost table J2-
In 1), the same search processing as in the case of the upper layer table is performed on the specific range of this table. Here, the specific range includes a record in the lower table associated with the approximate record in the upper table, and 99 records following this record. When this search process ends, the process returns to step SB5. In step SB9, the determination made in step SB5 is “Yes”.
In this case, various addresses included in each searched record are extracted.

If the retrieved record exists in the Japanese sentence table (for example, the Japanese sentence uppermost layer table J2-2),
The record in the Japanese sentence table of the lowest layer (for example, the Japanese sentence lowest layer table J2-1) associated with the record is extracted, and the character address associated with the record is extracted. When the size of the record in the Japanese sentence table of the lowest layer extracted here is plural, from the character address and the following character address group, in order,
Extract as many character addresses as the size.

When the search character string is one character, the record in the Japanese sentence table in the lowest layer associated with the extracted record in the one-character table J1 is extracted. At this time, if the size of the retrieved record in the 1-character table J1 is plural, the size is sequentially set from the record of the Japanese sentence lowest layer table J2-1 and the record group subsequent to this record associated with the record. Records are extracted as many as. Character addresses corresponding to the respective records in the Japanese sentence lowest layer table J2-1 thus extracted are extracted and sorted in ascending order.

Next, at step SB10, a set of character addresses having a difference corresponding to the distance between the search pseudo words is extracted from the character address group extracted corresponding to each search pseudo word, The top address of the extracted set is extracted. For example, as shown in FIG. 17, if the search character string is “management crisis”, the distance between the search pseudo-words “management” and “crisis” is 2.

Therefore, a set in which the difference between the character address extracted corresponding to the search pseudo word "management" and the character address extracted corresponding to the search pseudo word "crisis" is 2 is extracted. . At this time, since the character address groups corresponding to the respective search pseudo-words are sorted, the character addresses are extracted from each address group in ascending order, and by comparing these, the character address with the difference of 2 is obtained. The set is easily extracted. Then, the leading character address of the extracted character address set (the character address corresponding to “Sutra” in the case of “management crisis”) is extracted.

If the search character string is alphabetic and one or more records R7 are extracted from the European basic table EBT, first, the virtual address table V associated with the record R7 is first extracted.
The record R8 in the AT is extracted. Then, in the extracted record R8, with respect to the record R8 having the same virtual address unique code, a set of virtual addresses having a difference in intra-word address of, for example, 2 is extracted, and a leading character address of the extracted set is extracted. To do.

Here, for example, the search character string is "wor
If it is "ld", the interval between the search pseudo-words "wo" and "rl" is 2, and the interval between the search pseudo-words "rl" and "ld" is 1. Therefore, the search pseudo-word "wo" From the virtual address group extracted corresponding to "and the virtual address group extracted corresponding to the search pseudo-word" rl ", the unique code is" A "and the difference in the intra-word address is A set of 2 virtual addresses is extracted, and the virtual address thus extracted and the search pseudo word "ld" are extracted.
Of the virtual address whose unique code is “A” and which has a unique code difference of “1”.

Then, of the extracted virtual addresses of the set, the word in the word list WL is specified from the unique code in the leading virtual address. A word address is associated with the specified word, and a character address “317” consecutive from the Japanese range JA is also associated with the first word of the European range EA. The character address is obtained. Of course, when the search character string is one character, the continuity determination described above is not performed.

Next, in step SB11, it is judged whether or not the search is for a consciousness of a pulse. If this determination is "Yes", the process proceeds to step SB12, and if "No", the process proceeds to step SB1.
The process proceeds to 3. Step SB12 is a process in the case of a consciousness search of a branch. In the case of the consciousness search of a branch, there are a plurality of search character strings (here, two are used to simplify the description. Each search character string is hereinafter referred to as a first search character string and a second search character string). And here, the first and second
It is determined whether or not the search processing for the search character string has been completed. If this determination is “No”, step SB3
The process returns to, and the above-described search process is performed on the unprocessed search character string. Conversely, if “Yes”, the process proceeds to step SB13.

In step SB13, the first character address group extracted corresponding to the first search character string and the first character address group extracted corresponding to the second search character string are compared, and the difference between the two is compared. A set of character addresses (hereinafter referred to as an in-range address set) in which is less than or equal to the specified distance is extracted. At this time, in the document data specified by the two character addresses,
When a delimiter such as a carriage return is present, it is excluded from the in-range address set even if the difference between the two is less than the specified distance.

Next, at step SB14, it is judged if the number of in-range address sets is 1 or more. If this determination is "Yes," go to Step SB15, "No."
If so, the process proceeds to step SB18. Step SB
At 15, it is determined whether or not there is a rank designation. If this determination is “Yes”, go to Step SB16 and “No”.
If so, the process proceeds to step SB17.

Step SB16 is a process in the case where the order is designated, and a set in which the order of the character addresses in the in-range address set matches the specified order (hereinafter referred to as the order matching address set) is extracted. To do. If the set extracted here is not 0, the process proceeds to step SB17. On the contrary, if the set does not exist, the process proceeds to step SB1.
Go to 8.

In step SB17, first, display data corresponding to the number of leading character addresses included in the leading character address group corresponding to each search character string is supplied to the display 9. As a result, the number of extracted data is displayed in the output field 16 of the search menu displayed on the display 9. When the user who sees this operates the input device 6 and supplies predetermined instruction data to the search processing device 7, the device 7 displays the display data corresponding to the document data having the character address in the first character address group. Supply to the display 9.

Thus, the search result is displayed on the display 9, as shown in FIG. At this time, the character string that matches the search character string in the document data is highlighted, for example, and is distinguished from other character strings. If the mode is the consciousness mode, the underline is added to the character string of the first character address which is out of the specified range or whose order does not match. Here, the user operates the input device 6 to display other search results and the like on the display 9.

Further, the step SB18 is a step SB.
In step 6, step SB14, or step SB16, the process is performed when it is determined that the character string that matches the search target character string cannot be retrieved, and the predetermined display data is supplied to the display 9 to display "the specified condition A message such as "The search string did not exist in the document" is displayed.

As described above, according to the embodiment of the present invention, the size of the Japanese sentence table itself is increased in order to hierarchically construct the Japanese sentence table having the unique pseudo word having the character string length of 2. It is possible to perform full-text search with excellent search efficiency. Further, since the one-character table is provided, the search process for the search character string having the character string length of 1 can be performed quickly. Furthermore, since the word list WL and the European language basic table EBT are created, the size corresponding to each unique pseudo word can be set to an appropriate size,
Search efficiency can be improved.

Further, in the European language range EA, since it is possible to search in pseudo word units smaller than word units, it is possible to search for words with changed endings at once. For example, if a search character string "econ" is input through the input device 6, "economic", "ec"
The word "onomy" can be extracted.

In the above-described embodiment, the CD
-The example in which the full-text database is stored in the ROM is shown, but if the storage medium has a sufficient storage capacity, the CD-
It does not have to be a ROM. Further, the search processing device 7 may be a workstation or the like, and need not be a personal computer. Further, the unit for dividing the table in the layer one step lower does not have to be 99, and it is desirable to set it according to the number of unique pseudo words. Alternatively, a stop record that ends the search operation with that record may be inserted.

Further, in the above-described embodiment, the number of characters of the pseudo word has been described as two characters, but the number of characters is not limited to two characters, and for example, a plurality of characters such as three characters, four characters ,. I wish I had it. Of course, the number of characters of the pseudo word is set according to the contents of the database, the characteristics of the search process, and the like. For example, in electronic publishing, when the document data is a general Japanese sentence, it is set to about 2 characters.

Further, in the above-mentioned one embodiment, since the example applied to the electronic publication is shown, the construction system 1 is used by the provider of the full-text database, and the search system 5 is used by the user of the full-text database. In addition, although each is configured as an individual system, it goes without saying that both are configured as an integrated system and can be applied to a general full-text database used in fields other than electronic publishing.

[0084]

As described above, according to the present invention,
The mark inserting means inserts a position mark immediately before and after a specific character string in the document data, and the address giving means gives a continuous address to each character in the document data. Then, the address table creating means creates a pseudo word of a total of k characters (k is 2 or more) composed of each character and the following characters in the document data, and attached to the first character of each pseudo word. Each address is stored in the address table in the order of the character code of the corresponding pseudo word. Here, the lowest layer constructing means stores pseudo-words having a unique character code as a unique pseudo-word in the lowest-order table in the index table group in the order of the character code, and each unique pseudo-word has each address in the address table. Correspond to. When the number of unique pseudo-words stored in the uppermost layer table in the index table group is larger than a preset number, the hierarchizing unit divides the uppermost layer table into substantially equal parts. Unique pseudo-words are extracted, and the plurality of unique pseudo-words are stored in a table in an upper layer of the uppermost layer table in the order of character codes. Since the index table group has such a hierarchical structure,
At the time of search, there is an effect that a sufficient search speed can be obtained. Further, since the pseudo word stored in each table in the index table group is a unique pseudo word having a unique character code, there is an effect that the size of the index table group can be made small. further,
A position mark is inserted immediately before and after the specific character string in the document data. Therefore, if the search character string has a position mark as the first character, it is a prefix match search, and if it has a position mark as the last character. Has an effect that it can perform a backward match search and an exact match search when it has position marks as the first and last characters.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a full-text database system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a flow of full-text database construction processing according to the embodiment.

FIG. 3 is a diagram showing an example of document data used in the same embodiment.

FIG. 4 is a diagram showing an example of document data on which mark addition processing has been performed.

FIG. 5 is a diagram showing an example of document data to which various addresses are added.

FIG. 6 is a diagram showing a schematic configuration of a European language contrast table CT.

FIG. 7 is a diagram showing a schematic configuration of a word address table WAT and a word list WL.

FIG. 8 is a diagram showing a schematic configuration of a pseudo word table PWT.

FIG. 9 is a diagram showing a schematic configuration of a virtual address table VAT and a European language basic table EBT.

FIG. 10 is a diagram showing a schematic configuration of a Japanese sentence pseudo word table JPT.

FIG. 11 is a diagram showing a schematic configuration of a character address table CAT and a Japanese sentence lowest layer table J2-1.

FIG. 12 is a diagram showing a schematic configuration of a Japanese sentence uppermost layer table J2-2, a one-character table J1 and the like.

FIG. 13 is a flowchart showing the flow of a search process in the full-text database system according to an example of the present invention.

FIG. 14 is a flowchart showing a flow of search processing in the system.

FIG. 15 is a diagram showing an example of a search menu.

FIG. 16 is a diagram showing an example of a search result.

FIG. 17 is a diagram showing an example of a search character string.

[Explanation of symbols]

3 construction processing device (mark insertion means, address assignment means, address table creation means, bottom layer construction means,
Hierarchical Means 6 Input Device (Input Means) 7 Search Device (Division Means, Search Means) CAT Character Address Table (Address Table) J2-1 Japanese Lowermost Layer Table (Lowest Layer Table) J2-2 Japanese Uppermost Layer Table ( Upper table)

Front Page Continuation (72) Inventor Masato Nara 1-5-1 Taito, Taito-ku, Tokyo Toppan Printing Co., Ltd. Within the corporation

Claims (3)

[Claims] 1. A full-text database system for extracting a character string matching a search character string from document data by using an index table group having a hierarchical structure, which is immediately before and after a specific character string in the document data. A total of k characters (mark insertion means for inserting a position mark into the document data, address assigning means for assigning consecutive addresses to each character in the document data, and each character in the document data and subsequent characters ( (k is 2 or more), an address table creating means for storing each address given to the first character of each pseudo word in the address table in the order of the character code of the corresponding pseudo word, and a unique character code The pseudo words that are stored as unique pseudo words are stored in the lowest table in the index table group in the order of the character code, and If the number of unique pseudo words stored in the lowermost layer construction means that associates each address in the address table with a pseudo word and the uppermost layer table in the index table group is larger than a preset number, the Hierarchizing means for extracting a plurality of unique pseudowords so as to divide the upper layer table substantially evenly, and storing the plurality of unique pseudowords in the upper layer table in the character code order. Full-text database system. 2. A full-text database for extracting a character string matching a search character string from document data, comprising: each character in the document data to which consecutive addresses are given, and a character following each character. And an address table for storing each address given to the first character string of each pseudo word of a total of k characters (k is 2 or more) in the order of the character code of the corresponding pseudo word, and a table group having a hierarchical structure. , A lowermost layer table in which pseudowords having unique character codes are stored as unique pseudowords in the order of character codes, and each unique pseudoword is associated with each address in the address table, which is constructed in an upper layer of this lowermost layer table. Is composed of a plurality of upper layer tables, and each upper layer table is stored in a position that divides the table of the lower layer one level substantially evenly. Input a search character string that has a hierarchical structure that stores the characters in the order of character codes, and is configured so that the number of unique pseudowords stored in the uppermost table is less than or equal to a preset number, and a search mark that has a position mark. Inputting means for dividing the search character string supplied from the inputting means into k character units to generate a plurality of search pseudowords; and a unique code having the same character code as the search pseudowords. Performing an extraction process of extracting a pseudo word from the index table group, and specifying a set of unique pseudo words that continuously exist in the document data from the difference between the addresses corresponding to the extracted unique pseudo words, Search means for outputting a character string in the document data corresponding to an address group corresponding to the set, wherein the search means includes the search pseudo word and the character code. Is not present in the table to be searched, the smallest pseudo word of the character code closest to the search pseudo word from the unique pseudo word group of the character code smaller than the search pseudo word. And the maximum pseudo word stored immediately after this minimum pseudo word, and
The table in the lower layer is the search target, and in the table, for the range sandwiched between the unique pseudo word of the character code that matches the minimum pseudo word and the unique pseudo word of the character code that matches the maximum pseudo word, A full-text database system characterized by performing extraction processing. 3. A full-text database system for extracting a search character string from document data using an index table group having a hierarchical structure, wherein position marks are inserted immediately before and after a specific character string in the document data. A total of k characters (k is 2 or more) composed of a mark inserting means, an address assigning means for assigning consecutive addresses to each character in the document data, and each character in the document data and a subsequent character. An address table creating means for creating pseudo-words of each of the pseudo-words and storing each address given to the first character of each pseudo-word in the address table in the order of the character code of the corresponding pseudo-word, and a pseudo-word having a unique character code The pseudo-words are stored in the lowest-order table in the index table group in the order of the character code, and are If the number of unique pseudowords stored in the lowermost layer constructing means for associating each address in the address table and the uppermost layer table in the index table group is larger than a preset number, the uppermost layer table is made substantially equal. A plurality of unique pseudo-words are extracted so as to be divided into, and a search character string having the position mark and the layering means for storing the plurality of unique pseudo-words in the character code order is input to the upper table of the uppermost table. Input means, dividing means for dividing the search character string supplied from the input means into k-character units to generate a plurality of search pseudo-words, and a unique pseudo-code having the same character code as the search pseudo-words Extraction processing is performed to extract words from the index table group, and in the document data from the difference of each address corresponding to each extracted unique pseudo word. And a search unit that specifies a set of unique pseudowords that continuously exist in the document data and outputs a character string in the document data according to an address group corresponding to the set. When a unique pseudo word whose character code matches the pseudo word does not exist in the table to be searched, a character closest to the search pseudo word from the unique pseudo word group having a character code smaller than the search pseudo word. The minimum pseudo word of the code and the maximum pseudo word stored immediately after this minimum pseudo word are extracted and 1
The table in the lower layer is the search target, and in the table, for the range sandwiched between the unique pseudo word of the character code that matches the minimum pseudo word and the unique pseudo word of the character code that matches the maximum pseudo word, A full-text database system characterized by performing extraction processing.