JP2002157252A - Device and method for retrieving document and computer readable recording medium recording program for computer to execute the same method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve efficiency in retrieval and to simplify a program by unnecessitating the reference of an original document by performing processing while referring to an index even in the case of executing retrieval based on a regular expression in full sentence retrieval. SOLUTION: In the document retrieving device for performing the full sentence retrieval, this device is provided with a retrieve character string/regular expression input part 101 for inputting a character string/regular expression, a regular expression processing part 102 for performing preparing processing of finite automaton corresponding to the character string of the inputted regular expression, an index data storage part 104 for storing index data and a processing part 103 for transiting the state of the finite automaton while referring to the index data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used in a personal computer or an electronic filing system, and has a function of full-text search from digitized document data. The present invention relates to a document search method and a computer-readable recording medium storing a program for causing a computer to execute the document search method.

[0002]

2. Description of the Related Art With the spread of personal computers in recent years, the number of documents created on a computer or handled through communication such as the Internet has become enormous. The document thus mass-produced is filed with a title or keyword assigned to each document so that subsequent searches (such as identification and retrieval of the document) can be easily performed. Document retrieval, such as performing retrieval and reading with reference to a key, is generally performed.

As an example of a method of performing a high-speed document search for a text, there is a method of using a search in which the appearance positions of all characters appearing in a text are recorded. In this document search method, numbers are assigned to texts, and an index is created which lists the text numbers and the appearance positions in the text of all characters appearing in the text as character keys. For example, if the first part of the document numbered 314 is "Teletext", an index consisting of the value of the appearance position corresponding to each character key is created as shown in FIG. Is done.

Conventionally, by creating such an index, it is possible to quickly search for a document in which position a specific character string appears. The conventional search procedure is shown in the flowchart of FIG. First, the length of the given character string is placed in the register L, the counter N is set to 1 (step S21), and the set into which the result is put is emptied. For example, when "text" is given, the counter N is set to 4 and N is set to 1 to empty the set S for storing the result.

[0005] Subsequently, the index is referred to for the first character of the given character string, the document is searched for the number of the character, and the character is added to the set (step S22). In this example, since “te” is the first character, (314, 1) and (314, 3) are the contents of the set.

Subsequently, the counter N is incremented by one (step S23), and it is determined whether or not the counter N is larger than the register L (step S24). Here, when it is determined that the counter N is larger than the register L (judgment, Yes), the set S at that time is a result.
Contains the document in which the character string exists and the end position (step S26), and terminates this processing.

On the other hand, in step S24, the counter N
Is less than or equal to the register L (judgment, N
o), fetch the N-th character of the given character string, obtain the number of the N-th character in which document by referring to the index, and compare it with the element of S for each;
If the document number is the same and the appearance position of the character is smaller by 1 in, replacement with that element is performed (step S25), and discarding of the unreplaced S element is repeatedly executed.

For example, when N is set to 2, "K" is obtained, and (314, 4) is obtained by referring to the index. This is because (314, 3) has the same document number and the appearance position of the character is smaller by one, so (314, 3)
Is replaced by (314, 4), and (314, 1) not replaced is discarded. By repeatedly executing the above, when N becomes 5, S becomes (31)
4, 6) remain, and it can be seen that “text” is located at a position ending with the sixth character of the document 314.

[0009]

In the conventional method as described above, it is possible to know at a high speed which document in which document a specific character string exists by referring only to the index. However, some regular expressions (regular e
The only way to know where in a document a character string that matches (xpression) is located is to examine all the original text exquisitely, and whether the search condition is a simple character string. Since the processing method differs depending on whether the processing is performed using a regular expression or a regular expression, there is a problem that a program is complicated. Also, in the case of regular expressions, all documents are exhaustively examined,
When compared with a simple character string, there was a problem that the search time was long.

The present invention has been made in view of the above. In a full-text search, even when a search using a regular expression is performed, processing is performed by referring to an index, thereby making it unnecessary to refer to an original document. It is an object of the present invention to improve the efficiency of the search and to realize the simplification of the program.

[0011]

According to another aspect of the present invention, there is provided a document search apparatus for performing a full-text search, wherein a search character for inputting a character string and a regular expression is provided. Column / regular expression input means, regular expression processing means for creating and processing a finite automaton corresponding to the character string of the regular expression input from the search character string / regular expression input means, and index data storage for storing index data Means, and processing means for performing a state transition of the finite state automaton with reference to the index data of the index data storage means.

According to the present invention, a character string and a regular expression to be subjected to full-text search are input to the search character string / regular expression input means. Since this character string is regarded as a kind of regular expression, both the character string and the regular expression are sent to the regular expression processing means,
Then a finite automaton is created. The processing means performs a state transition while referring to the index data stored in the index data storage means in accordance with the finite automaton to search for a character string, thereby enabling a search without referring to the original document.

Further, in the document search apparatus according to the second aspect, the finite state automaton is provided with two slots in a node representing a state, and the processing means determines whether a document number and a character appear in different slots. The state transition of the finite state automaton is executed while transferring a set of position sets.

According to the present invention, two slots are provided for placing a set of a document number and a character appearance position in a node representing the state of a finite state automaton, and the processing means uses different slots for the document number and the character. Executes a state transition of a finite state automaton by transferring a set of sets of occurrence positions of, and when the transition from any node becomes impossible, the set in the slot in the accepted state contains a character string that matches the regular expression The document is searched by regarding it as the last position set of the number and the character string.

According to a third aspect of the present invention, in the document search method for performing a full-text search, a finite automaton is created from the input regular expression, and an index in which the positions of characters are recorded is referred to. It detects a character string that matches the regular expression.

According to the present invention, when an inquiry about a regular expression is given, a finite automaton is created from the regular expression, and a search is performed by referring to an index in which the positions of characters are recorded in accordance with the finite automaton. It is possible to detect a character string that matches the regular expression only by referring to the index data without referring to the original text.

According to a fourth aspect of the present invention, the state transition of the finite state automaton is performed using a set of a set of a document number and a character appearance position.

According to the present invention, by performing the state transition of the finite state automaton using a set of a set of a document number and a character appearance position for each state, a document search using the finite state automaton is realized.

In the document search method according to the fifth aspect, two slots are provided in a node representing a state, and a set of a set of a document number and a character appearance position in a different slot when a state transition is performed. Is transferred.

According to the present invention, the state transition of the finite state automaton is executed by transferring the set of the document number and the appearance position of the character in a different slot, and the transition is not accepted from any node. The retrieval of the retrieval target document is executed by regarding the set in the slot in the state as the number of the document including the character string matching the regular expression and the last position set of the character string.

According to a sixth aspect of the present invention, there is provided a computer-readable recording medium on which a program for causing a computer to execute the document search method according to any one of the third to fifth aspects is recorded. is there.

According to the present invention, a program for causing a computer to execute the document search method according to any one of claims 3 to 5 is recorded, whereby the document search method according to any one of claims 3 to 5 is recorded. Can be realized by a computer.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a document search apparatus and a document search method according to the present invention and a computer-readable recording medium storing a program for causing a computer to execute the document search method will be described with reference to the accompanying drawings. Refer to and describe in detail. The present invention is not limited to this embodiment.

According to the present invention, in a full-text search system, a regular expression which is an input sequence defined recursively (a method of using a specific character (metacharacter) to express a character string)
Is given, the matching character string can be obtained at high speed only by referring to the index data without referring to the original text. In addition, by realizing a simple character string as a kind of regular expression, it realizes a simple program that can unify processing of searching for something that matches the regular expression and operation of checking for the existence of a simple character string It is. Hereinafter, a specific example of the device configuration and method will be described.

FIG. 1 is a block diagram showing a functional configuration of a full-text search system according to an embodiment of the present invention. In the figure, reference numeral 101 denotes a search character string / regular expression input unit for inputting a character string of a regular expression, reference numeral 102 denotes a regular expression processing unit for creating and processing a finite automaton, and reference numeral 103 denotes a state transition of the automaton while referring to index information. And 104, an index data storage unit for storing index data.

Here, the finite automaton (finite
(automatons). here,
Consider a case in which a finite state automaton is used to search for a given character string in a language.
The character string is stored in the initial state of the model (initial st
ate), the next state to be shifted is determined by the combination of each character of the input character string and the internal state, and the state transition (state) of the finite state automaton is determined.
(transition) is performed. If the finite state automaton is in a predetermined final state at the time when the character string has been inserted, the character string is accepted (accepted), and it is determined whether the character string belongs to the language. Determine.

In the full-text search system configured as described above, first, the search character string / regular expression input unit 101 receives a character string and a regular expression to be subjected to full-text search. Strings can be considered as a type of regular expression,
Both the character string and the regular expression are sent to the regular expression processing unit 102. The regular expression processing unit 102 creates a finite automaton from the input regular expression and sends it to the processing unit 103. The processing unit 103 receives the finite state automaton created by the regular expression processing unit 102, and executes a state transition while referring to the index data stored in the index data storage unit 104.

FIG. 2 is a block diagram showing the configuration of the document search apparatus according to the embodiment of the present invention, which embodies and implements the above-described full-text search system. In the figure,
This document search device is provided on a system bus 201 by a CPU 2
02, ROM 203, RAM 204, keyboard 205, disk device 206, display 207
And are connected.

Keyboard 205 and display 207
Corresponds to the search character string / regular expression input unit 101 in FIG. 1 and is a portion where the search character string / regular expression is input. The display 207 displays a search result. The CPU 202 is a part for executing the document search program of the present invention. The CPU 202 creates a finite automaton from the input regular expression, performs a state transition of the finite automaton while referring to the index data, and displays the result on the display 207. Control processing such as outputting is performed. A program to be executed by the CPU 202 is stored in the ROM 203 and loaded into the RAM 204 as necessary. In some cases, the program may be stored in the disk device 206 instead of storing the program in the ROM 203.

The RAM 204 stores the program when the CPU 202 executes the program. Also, the RAM 20
Numeral 4 stores a finite automaton corresponding to the regular expression, and further holds an intermediate result and a final result of the state transition. The disk device 206 stores index data. Depending on the size of the index data and the application of the search device, a configuration in which the index data is stored in the RAM 204 or the ROM 203 is also conceivable.

By the way, a method of creating a finite automaton for detecting a character string matching the regular expression from the regular expression is already known. Therefore, in the present invention, a finite automaton is created from a regular expression. A finite state automaton generally has a node having a direction connecting a node representing a state and a node, and the side has a set of characters causing a transition. Also, the initial state (initial state)
te: initial state and accept state (accept stat)
e) there is a special state, starting from the initial state, transitioning to a certain state, and causing a transition of an edge that departs from that state, if there is a given character in the set of characters, through that edge A transition is made to a state in which the character string can be reached, and if the state reaches the acceptance state, it is detected that there is a matching character string.

In the present invention, two slots (slots) for placing a set of a document number and a character appearance position in a node representing the state of a finite state automaton are prepared (hereinafter, these two slots are referred to as slots). 1 and slot 2). FIG. 3 is a flowchart illustrating a procedure of the document search method according to the embodiment of the present invention. As described above, the finite state automaton has a starting state and an accepting state. First, an empty set is set to slot 1 and slot 2 of the node in the accepting state, and a variable T
, 2 is substituted as an initial value, and the number 1 of the currently valid slot is put into S (step S11). Then, an empty set is placed in the slot T in a state other than the reception state (step S12).

Subsequently, the index of each character which causes the transition of the side appearing from the initial state is referred to, and the set of the document number and the appearance position where the character appears is added to the set of the slot S of the destination node of the side. (Step S1
3). Subsequently, in a state other than the reception state,
It is determined whether there is a non-empty set (step S14). Here, if it is determined that there is a slot S that has a non-empty set, the following operation is performed for all states other than the accepting state in which the slot S has a non-empty set (step S15).

In step S15, for a character which causes a transition of a side protruding from the node, an index is obtained to obtain a document number and an appearance position. Is added to the set of slots T of the destination node of the edge. Furthermore, S
The values of T and T are exchanged, empty sets are set in the slots T in all the states except the reception state (step S16), the process returns to step S14, and the subsequent operations are repeatedly executed. on the other hand,
If it is determined in step S14 that none of the slots S has a non-empty set, the set of slots 1 and 2 in the receiving state indicates the end of the matched character string (step S17).

That is, the finite state automaton has a starting state and an accepting state. Therefore, first, an empty set is set to slot 1 and slot 2 of the node in the accepting state, and 2 is substituted as an initial value into a variable T for inserting a transition destination slot number.
The number 1 of the currently valid slot is put in S. Next, a set of the document numbers and the appearance positions of the characters appearing by indexing the characters which cause the transition of all the sides coming out of the start state is added to the set of slots 1 in the state where the side is the destination.
Then, the following operation is repeatedly executed.

An empty set is placed in the slot pointed to by T for all nodes (however, the set of slots of the node in the accepted state is left as it is). Next, paying attention to all nodes (except the accepting state) having a non-empty set in the slot pointed to by S, referring to an index for a set of characters causing a transition to the side for all sides coming out of the node, Get the document number and the position of the character. From among them, only those having the same document number as the set placed in the slot of the node and having the character occurrence position larger by 1 are left, and are added to the set of slots indicated by T of the destination node on that side. Finally, the values of S and T are exchanged. The above operation is repeatedly executed, and when the transition becomes impossible from any node, the set in the accepting slot is the number of the document containing the character string that matches the regular expression and the last position set of the character string Becomes

Next, the above-mentioned operation will be described with a specific example. For example, from the keyboard 205, FIG.
Let us take a case where a regular expression as shown in FIG. The meaning of this regular expression is that the first part is "Japan", followed by a sequence of zero or more characters other than ".", "," And followed by "association". ing. For example, "Japan Mannenkai" and "Japan Broadcasting Corporation" match this regular expression. On the other hand, "many associations in Japan" does not match this regular expression.

In correspondence with this regular expression, a finite automaton composed of nodes and edges as shown in FIG. 5 is created. In the figure, rectangular blocks indicate states N1 to N1.
N5, the arrow drawn on the side 300, and the one written on the arrow represent a set of characters causing the transition. The leftmost block (node N1) represents the starting state 301, and the rightmost block (node N5) represents the receiving state 302. The lower two rectangles at nodes N1 to N5 are slots 303 in which a set of a document number and a set of character appearance positions is placed.

When such a finite automaton is created, first, an empty set is placed in both slots S1 and S2 of the node N5 in the accepting state 302. Assuming that slot 1 is the upper slot and slot 2 is the lower slot, an empty set is placed in a slot below the nodes N1 to N4 excluding the receiving state 302. Then, by referring to the index data, a set of a document number and an appearance position where “day” appears is obtained, and this is placed in the slot 1 of the second node N2 from the left.

Also, 1 is entered into the variable S representing the number of the currently valid slot, and 2 is entered into the transition destination slot number.
Thereafter, as a repetition, all the nodes N1 to N1 except the reception state 302 in which the non-empty set is contained in the slot S are set.
With respect to N4, the number and appearance position of the document in which the character of the character set causing the transition of the side out of the node appears are obtained by referring to the index data, and the next to the appearance position of the character in the slot S is obtained. Is added to the set of slots T, the values of S and T are exchanged, and empty sets are placed in the slots T of the nodes N1 to N4 other than the reception state 302. By performing such processing, the end position of the character string that matches the regular expression shown in FIG.

By the way, the document search method described so far can be programmed, recorded on a computer-readable recording medium, and executed on a computer. Also, a part of the document search method can be implemented on a network and realized through a communication line.

That is, the document search method described in this embodiment is realized by executing a prepared program on a computer (CPU) such as a personal computer or a workstation. In other words, this program is operated by a keyboard, etc., to operate a memory, hard disk, floppy (registered trademark)
The program is recorded on a computer-readable recording medium such as a disc, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by a computer (CPU).

Further, it is also possible to transmit and receive processing data in the document search from the communication device to an external device as needed. Further, this program can be distributed to each device such as a personal computer via the recording medium via a network such as the Internet.

[0044]

As described above, according to the document search apparatus of the present invention (claim 1), a character string and a regular expression to be subjected to full-text search are input to the search character string / regular expression input means. You. Since this character string is regarded as a kind of regular expression, both the character string and the regular expression are sent to the regular expression processing means, where a finite automaton is created. The processing means changes the state while referring to the index data stored in the index data storage means in accordance with the finite automaton to search for a character string. Therefore, the processing means refers only to the index data without referring to the original text. Can be used to find a matching string at high speed, and consider a simple string as a kind of regular expression to search for something that matches the regular expression and to check for the existence of a simple string A simple programming that can be processed uniformly is realized.

According to the document search apparatus of the present invention (claim 2), two slots are provided for placing a set of a document number and a character appearance position in a node representing a state of a finite state automaton. The processing means executes the state transition of the finite state automaton by transferring the set of the document number and the appearance position of the character in a different slot, and when the transition becomes impossible from any node, the processing means is in the accepting slot. Since the set is regarded as a document number including a character string that matches the regular expression and the last position set of the character string, a document search using a finite automaton is realized.

Further, according to the document search method of the present invention (claim 3), when an inquiry about a regular expression is given, a finite automaton is created from the regular expression, and the position of a character is determined according to the finite automaton. Searching by referring to the recorded index enables searching for matching character strings at high speed just by referring to the index data without referring to the original text. Since it is regarded as a kind, a simple program is realized in which the operation of searching for a match with the regular expression and the operation of checking for the existence of a simple character string can be processed in a unified manner.

According to the document search method of the present invention (claim 4), in claim 3, a finite state automaton is created from a regular expression, and the state transition of the finite state automaton is represented by a document number and a state number of each state. By using a set of sets of character appearance positions, a search method based on a state transition from a start state to an accepted state in a finite state automaton is realized.

According to the document search method of the present invention (claim 5), in claim 4, the state transition of the finite state automaton is changed by transferring a set of a document number and a character appearance position in a different slot. Execute, and when the transition from any node becomes impossible, the set in the accepted slot is regarded as the number of the document containing the character string that matches the regular expression and the last position set of the character string, so the search In the search for the target document, it is possible to detect the presence of a character string when it reaches the acceptance state (final state).

According to the computer-readable recording medium of the present invention (claim 6), claims 3-5
A program for causing a computer to execute the document search method according to any one of claims 1 to 3, wherein
The document search method described in any one of Items 1 to 5 can be realized by a computer.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a functional configuration of a full-text search system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a document search device according to the embodiment of the present invention.

FIG. 3 is a flowchart illustrating a procedure of a document search method according to the embodiment of the present invention.

FIG. 4 is a diagram showing an example of a character string of a regular expression according to the embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of creating a finite state automaton corresponding to a regular expression according to the embodiment of the present invention;

FIG. 6 is an explanatory diagram showing an example of a conventional text search.

FIG. 7 is a flowchart illustrating an example of a conventional document search.

[Explanation of symbols]

 101 search character string / regular expression input unit 102 regular expression processing unit 103 processing unit 104 index data storage unit 202 CPU 203 ROM 205 keyboard 207 display 300 side 301 start state 302 reception state 303 slot

Claims (6)

[Claims] 1. A document search apparatus for performing full-text search, comprising: a search character string / regular expression input unit for inputting a character string and a regular expression; and a regular expression character string input from the search character string / regular expression input unit. A regular expression processing means for creating and processing a finite automaton corresponding to: an index data storage means for storing index data; and a processing means for performing a state transition of the finite automaton while referring to the index data of the index data storage means. A document search device, comprising: 2. The finite state automaton is provided with two slots in a node representing a state, and the processing means transfers the state of the finite state automaton while transferring a set of a document number and a character appearance position set in different slots. 2. The document search device according to claim 1, wherein the document search is executed. 3. A document search method for performing a full-text search, comprising creating a finite automaton from an input regular expression, referring to an index in which character positions are recorded, and detecting a character string matching the regular expression. Characteristic document search method. 4. The document search method according to claim 3, wherein the state transition of the finite state automaton is executed using a set of a set of a document number and a character appearance position. 5. The node according to claim 4, wherein two slots are provided in a node representing a state, and when a state transition is performed, a set of a set of a document number and a character appearance position is shifted in a different slot. Document search method. 6. A computer-readable recording medium on which a program for causing a computer to execute the document search method according to claim 3 is recorded.