JP2006302082A - Character string retrieval system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and quickly perform retrieval from a compressed block string by substituting partial character strings constituting a digitized text with compressed blocks in a retrieval device which determines whether the text includes a prescribed character string described using regular expressions or not and obtains its position. <P>SOLUTION: A DFA (deterministic finite automaton) is utilized for retrieval. A compressed block (code) is acquired from the compressed block string (code string) (S21). In the case that a partial character string corresponding to the compressed block is new, the compressed block is decompressed into the partial character string and is inputted to the DFA, and the history of state transition (state history) of the DFA at this time is stored (S261 to S264). Compressed blocks corresponding to the same partial character string are not decompressed, and the state history is referred to update the state of the DFA (S251 to S254). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for retrieving a predetermined character string from a document.

  In recent years, the digitization of documents is progressing in various fields. As a large amount of digitized documents are used, the following problems have arisen.

The first problem is that it becomes difficult to find a desired document from a large amount of existing documents. Therefore, a method for efficiently searching for an electronic document is required.
As a technique for solving this problem, a search method using DFA (Deterministic Finite Automaton) is known (for example, Non-Patent Document 1).
In addition, simply searching for a fixed character string results in poor search efficiency. Therefore, the search condition is generalized by selectively specifying a part or all of the search character string or by allowing the same character string to be repeatedly specified, and similar character strings are simultaneously searched. As described above, a regular expression or the like is known as a method for expressing a search pattern in which a search character string is generalized.
It is known that an NFA (Nondeterministic Finite Automaton) that can be searched based on a regular expression can be constructed (for example, Non-Patent Document 1, Non-Patent Document 2, etc.).
Further, it is known that NFA can be converted into an equivalent DFA (for example, Non-Patent Document 1, Non-Patent Document 2, etc.).

The second problem is that the capacity of a storage device for storing a large amount of documents and that the network bandwidth is consumed when documents are exchanged via the network. In order to save the capacity of a storage device and reduce the amount of data transferred through a network, a method for efficiently compressing an electronic document is required.
As a technology for solving this problem, there are various methods such as LZ (Lempel-Ziv) 77, LZ78, LZSS (Lempel-Ziv-Storer-Syzmanski), LZW (Lempel-Ziv-Welch), Huffman coding, etc. A reversible compression method is known.

The third problem is that it is difficult to efficiently search a compressed document because the above-described search technique and compression technique have been independently developed.
As a technique for solving this problem, a method of simply searching after decompressing (restoring) a compressed text (compressed document) once is general. As a search method, for example, character string matching is performed using a state transition machine (finite automaton).
On the other hand, a method for retrieving a compressed document without decompressing it is also known (for example, Patent Document 1, Non-Patent Document 1, etc.).

  The search method described in Patent Document 1 relates to a method for searching compressed text at a high speed using a fixed search character string. In this method, a finite automaton is created by inputting a compression dictionary and a fixed search character string, and a search is performed without decompressing the compressed text using the finite automaton. .

The search method described in Non-Patent Document 1 relates to a method for searching a compressed text compressed in the LZ78 format or LZW format at high speed according to a search condition including a regular expression. In this method, it is possible to search at high speed by searching with a deterministic finite automaton without decompressing the compressed text.
JP-A-10-260980 Gonzalo Navarro, “Regular Expression Searching on Compressed Text”, Journal of Discrete Algorithms Volume 1, Issue 5-6 (October 2003), pages 423-443. E. J. et al. Hopcroft, D.M. J. et al. Ullman, “Formal Languages And Ther Relation to Automata”, Addison Wesley (1969) (Japanese Language Theory and Automata, Science, 1971)

The method of performing a search after decompressing a compressed document requires a time for decompressing the compressed text and a time for character string matching, and has a problem that the retrieval time becomes long.
The search method described in Patent Document 1 has a problem that a fixed search character string and a compression dictionary are input, and a regular expression cannot be handled as a search condition. Also, in the currently widely used lexicographic compression methods such as LZ77, LZSS, LZ78, and LZW, a compression dictionary is generated while decompressing the compressed text, so that a compression dictionary exists at the start of the search. There is a problem that the method of Patent Document 1 based on the above cannot be applied.
The search method described in Non-Patent Document 1 is based on the premise that the text is composed of a single byte code, and has a problem that it is not considered to search for text including multibyte code characters. In addition, since the method uses the characteristics of the compression dictionary in the LZ78 or LZW format, there is a problem that it is not possible to search for text compressed by another compression method that does not have such a feature in the compression dictionary.

  The present invention has been made, for example, in order to solve the above-described problems, and an object thereof is to efficiently search a compressed document.

The character string search device according to the present invention is:
An automaton that holds a state, inputs a character, calculates a transition destination state based on the held state and the input character, and updates the held state to the calculated transition destination state. Whether or not a search character string corresponding to a predetermined search pattern is included in the character string by determining whether or not the stored state is a predetermined state when characters constituting the character string are input By running an automaton configured to determine whether or not
In a character string search device for acquiring a code string obtained by replacing a partial character string included in the character string with a predetermined code corresponding to the partial character string, and searching the search character string from the character string,
An automaton execution unit for executing the automaton,
A history storage unit that stores the state held by the automaton as a state history;
A code acquisition unit for acquiring a code constituting the code string;
A condition determination unit that determines whether the first condition and the second condition are satisfied based on the state held by the automaton, the state history stored in the history storage unit, and the code acquired by the code acquisition unit When,
When the condition determination unit determines that the first condition is satisfied, the transition destination state is calculated based on the state history stored in the history storage unit, and the state held by the automaton is changed to the calculated transition destination state. A transition destination calculation unit to be updated;
When the condition determining unit determines that the second condition is satisfied, the character that restores the partial character string corresponding to the code acquired by the code acquiring unit and inputs the characters constituting the partial character string to the automaton A column restoration unit;
It is characterized by having.

  According to the present invention, for example, when searching for a character string included in a compressed text, a search condition is specified by a regular expression or the like, and this is converted into a finite automaton to perform a search. By storing the history of the search performed on the partial character string replaced with the block and omitting the state transition of the automaton using the stored history, there is an effect that the search can be performed at high speed.

  First, a search method using DFA (an example of an automaton) will be described.

  The DFA character string matching method is based on a model of a state transition machine (automaton). The state transition machine has a state and a state transition function inside. The state transition function is a function that determines the next state for the current state and the input character. In the character string matching method using DFA, the input text is read one character at a time, and a transition is made to the next state obtained by applying the state transition function to the set of the current state and the input character. According to this method, it is possible to perform collation by scanning the text once without going back, and high-speed character string collation is possible. When collating with a plurality of conditions, a finite automaton (Moore machine) with an output in which DFA is extended and outputs are defined for each state is also used in order to distinguish conditions that have been successfully collated.

FIG. 39 is a conceptual diagram showing an example of state transition in the operation of the DFA.
In FIG. 39, states 990 to 993 indicate DFA states. The DFA holds one of the states 990 to 993, and the state held by the input transitions (updates). At the start of the search, the initial state 990 is held.
Arrows in the figure indicate state transitions. When a character attached to an arrow is input, the state transitions to the state at the end of the arrow.
For example, when the current state is state 990 and the character “a” is input, the state transitions to state 991. If the character “b” or “c” is entered in the current state 990, the state 990 remains unchanged.

Here, for the sake of simplicity, it is assumed that there are only three types of characters “a”, “b”, and “c” to be input to the DFA, but there may be more types of characters to be input to the actual DFA. Of course.
The “character” here is not limited to a narrowly defined character such as an alphabet or a kanji, but may be anything as long as the computer can handle it as a character. On the computer, characters are represented by bit strings. For example, when an ASCII (American Standard Code for Information Interchange) code is used, the character “a” is represented by an 8-bit bit string “01000001” (41h). Alternatively, when a shift JIS (Japan Industry Standard) code is used, the character “A” is represented by a 16-bit bit string “1000001010100000” (82A0h). Thus, depending on the character code used, the bit length of the bit string representing it may be different. Therefore, anything that can be expressed as a bit string on a computer can be treated as “character” and can be input to the DFA.
However, since the DFA cannot be operated unless the transition destination corresponding to the input is determined in advance, the types of characters that may be input to the DFA must be limited.

In FIG. 39, a state 993 is a special state, which is called an acceptance state. A DFA configured for the purpose of retrieval is in an accepting state when the retrieval is successful.
The DFA shown in FIG. 39 is for searching the search character string “abc”.

  The operation of the DFA shown in FIG. 39 will be described by taking as an example the case where the search character string “abc” is searched from the character string “bababcc”.

Characters constituting the character string “bababcc” are input to the DFA one by one from the beginning.
FIG. 40 is a diagram illustrating how the DFA state transitions when characters are input to the DFA illustrated in FIG. 39.
At the start of the search, the DFA state is the initial state 990.
First, when the character “b” is input, the state of the DFA remains the state 990.
Next, when the character “a” is input, the DFA state transitions to the state 991.
Next, when the character “b” is input, the DFA state transitions to state 992.
Next, when the character “a” is input, the DFA state transitions to the state 991.
Next, when the character “b” is input, the DFA state transitions to state 992.
Next, when the character “c” is input, the DFA state changes to the state 993.
Since the state 993 is an accepting state, it can be seen that the character string “bababcc” includes the search character string “abc” at this point. Further, when the sixth character “c” is input, the DFA state becomes the accepting state 993, so that the search character string “abc” appears at the position ending with the sixth character of the character string “bababcc”. I understand.
Next, when the character “c” is input, the DFA state transitions to the state 990.
Since there are no more characters to be input to the DFA, the DFA ends the operation.

  As a result of the above operation, it can be seen that the search character string “abc” appears once in the character string “bababcc”, and the appearance position is a position ending with the sixth character.

  Next, the flow of processing in which the automaton execution unit executes this DFA will be described.

The automaton execution unit stores the transition destination list of FIG. 41 corresponding to the DFA of FIG.
This table shows the state of the transition destination to which the next transition is made when the character in the top column is input when the DFA is in the state of the leftmost column.
FIG. 42 is a flowchart illustrating an example of a process flow of the automaton execution unit.
At the start of the search, the automaton execution unit initializes the DFA state to the initial state (S991). For example, the initial state number 0 is stored in the memory that stores the DFA state.
Next, for example, the character string restoration unit inputs a character string character by character to the automaton (S992). That is, the character input to the automaton execution unit is notified, and the automaton execution unit acquires this.
The automaton execution unit refers to the stored transition destination list, and calculates the transition destination state based on the current DFA state and the input characters (S993).
Next, the automaton execution unit updates the state of the DFA (S994). For example, the state number of the transition destination state is stored in the memory that stores the state of the DFA.
The automaton execution unit determines whether or not the DFA state is an acceptance state (S995). If the DFA state is the accepting state, the search is successful, and the search success process is performed (S996). For example, a message indicating that the search is successful and the position where the search character string appears are displayed on the CRT display device 901.
The above processing is repeated until there are no more characters to be input to the DFA (S997).

FIG. 43 is an example of a slightly more complicated DFA.
The DFA shown in FIG. 43 is configured to be able to search for character strings “ababb”, “abca”, “aba”, and “bb”.

  Next, regular expressions will be described.

  A regular expression is a notation method for expressing a class of a language called a regular language.

A regular language is a set of character strings according to a certain rule among character strings obtained by arbitrarily concatenating characters constituting the regular language.
A regular expression is a character string made up of characters and metacharacters constituting a regular language, and expresses a rule for identifying whether a certain character string belongs to the regular language.
There are various known regular expression notations. Here, an example will be described.

In order to simplify the explanation, it is assumed that there are only three characters “a”, “b”, and “c” that constitute a regular language.
Further, there are five types of meta characters, “(” “)”, “|”, “*”, and “?”. Here, “(” “)” means grouping. “|” Means selection. “*” Means zero or more repetitions. “?” Means 0 or 1 occurrence.
For example, the regular expression “(ab | c)” represents a regular language whose elements are character strings “ab” and “c”.
For example, the regular expression “ab *” expresses a regular language whose elements are character strings “a”, “ab”, “abb”, “abbb”,.
For example, the regular expression “c? B” expresses a regular language whose elements are the character strings “b” and “cb”.

  Regular expressions that are actually known are more complicated, but are not described here (see Non-Patent Document 2).

As described above, when a regular expression is used, a set of character strings can be expressed easily. Therefore, the regular expression is used for a search pattern describing a search condition.
For example, if it is desired to search for the character strings “ababb” and “abca” and “aba” and “bb”, they are described as regular expressions as “ababb | abca | aba | bb”. Alternatively, “(aba)? (Bb)? | Abca” may be described, or “abc? A | (aba)? Bb” may be described.

  When searching for a search character string corresponding to a search pattern described using a regular expression, there is a method of searching for an NFA corresponding to the regular expression and executing the NFA.

NFA, like DFA, is a kind of automaton, but there are two or more transition destination states that transition when a single character is input for a certain state, and the state transitions even when no character is input (this Or “ε transition”), the transition destination state cannot be uniquely determined.
Therefore, in order to execute NFA, when there are two or more transition destinations, one of them is selected and executed for the time being. If execution fails, return to the branch point, select another option, and execute again. In this way, NFA can be executed by backtracking.

  However, if NFA is executed by backtracking in this way, it will take a long time to search because it will be reversed if it fails in the middle.

  Therefore, by converting NFA to DFA and executing the converted DFA, it is possible to search in a short time without going back.

  A method for obtaining NFA from a regular expression and a method for converting NFA to DFA are already known and will not be described here (for example, see Non-Patent Document 1).

Next, the document compression technique will be described.
Currently common document compression techniques include a self-reference type and a dictionary reference type. There are two types of dictionary reference types: a dictionary is prepared separately and a dictionary is embedded in a compressed document.

A self-referencing compression technique will be described. Examples of this compression method include the LZ77 method and the LZSS method.
The basic idea in the self-referencing compression technique is to shorten the code length of the entire character string by replacing one with a reference to the other when the same partial character string is at a different position in the original character string. That's it.

  FIG. 44 is a diagram illustrating an example of encoding in the LZSS scheme.

  Here, a case where the character string “cbababbcbc” is compressed will be described as an example. Here, it is assumed that the ASCII code is used. Therefore, one character is represented by an 8-bit bit string.

  The partial character string included in the character string is converted into a code according to the following rules.

Rule 1: A partial character string composed of 1: 1 characters is converted into a 9-bit code composed of a flag 981 (1 bit) and a bit string 982 (8 bits) representing the character.
Rule 2: If the partial character string matches another partial character string that appears before that, the flag 981 (1 bit), the appearance position 983 (for example, 8 bits) of the other partial character string, the partial character The code is converted into a 14-bit code having a column length of 984 (for example, 5 bits).

The flag 981 is used to distinguish whether the data is converted according to the rule 1 or the rule 2, and is used when the original character string is restored later.
Bit string 982 is an ASCII code representing the character. Here, the same code as the original character is used, but if the original character can be restored, it may be replaced with a different code.
The appearance position 983 of another partial character string is the position of the beginning of another partial character string that appears before that, expressed as the distance from the start position of the current partial character string (how many characters ago) It is. In this example, since the appearance position 983 is expressed by an 8-bit bit string, rule 2 cannot be applied if there is another partial character string before 256 characters.
The length 984 of the partial character string is the number of characters of the partial character string to be encoded by applying rule 2. In this example, since the length 984 of the partial character string is expressed by a 5-bit bit string, rule 2 cannot be applied to a partial character string of 32 characters or more.

When the character string “cbababbcbc” is compressed, the first “c”, “b”, and “a” have not appeared before that, so the rule 1 is applied character by character and converted into codes 601 to 603.
Since the three-character partial character string “bab” starting from the fourth character matches the other partial character string “bab” starting from the second character (two characters before the current position), rule 2 Applying and converting to the code | symbol 604 (In this way, another partial character string may partly overlap with self).
Since the two-character partial character string “cb” starting from the seventh character matches the other partial character string “cb” starting from the first character (six characters before the current position), rule 2 Apply to convert to 605.
Since the one-character partial character string “c” starting from the ninth character matches the one-character partial character string “c” starting from the seventh character, conversion may be performed by applying rule 2. However, if rule 2 is applied, the converted code becomes 14 bits, whereas if converted according to rule 1, it becomes only 9 bits, so that the compression efficiency becomes high (the bit length after compression becomes short). ) Apply rule 1 to convert to code 606.
Through the above conversion, the character string “cbavacbc” having a total bit length of 72 bits is replaced with a code string having a total bit length of 64 bits.

  A procedure for restoring the original character string from the code string replaced in this way will be described.

FIG. 45 is a flowchart showing an example of the flow of control when restoring the original character string from the code string in the conventional example.
First, 1 bit (flag 981) is acquired from the code string (S981), and it is determined whether the following code is converted by rule 1 or 2 (S982).
If the flag 981 is “1”, since it is rule 1, the subsequent 8 bits (symbol 982) are acquired (S983) and output as characters (S984).
The output character is stored in the output history (S985).
For example, reference numeral 601 in FIG. 44 is converted into the character “c” and output.
If the flag 981 is “0”, since it is rule 2, the following 13 bits (appearance position 983 and length 984) are acquired (S986).
For example, in the case of reference numeral 604 in FIG.
Next, the partial character string is restored with reference to the output history and output (S987).
The output character is stored as an output history (S988).
For example, in the case of reference numeral 604 in FIG. 44, two characters before the output history are read out. Since three characters “cba” are stored as the output history at this point, “b” is two characters before. Therefore, “b” is output and immediately stored as an output history. The output history is “cbab”.
This is repeated for the number of characters indicated by the length (S989).
Since the second character is “a” in the second character, the second character “a” is output. The output history is “cbaba”. In the third character, since the character before the second character is “b”, the third character “b” is output. Since three characters have been output, the processing for the reference numeral 604 ends, and the processing proceeds to the next processing. Therefore, three characters “bab” are output corresponding to the reference numeral 604.
This is repeated until the code string ends (S990).

FIG. 46 is a diagram showing an example of encoding in the LZ77 system, which is also a kind of self-reference compression technology.
The LZ77 system differs from the LZSS system in encoding rules, but the other parts are almost the same.
In the LZ77 method, the original character string is replaced with a code string according to the following two rules.

Rule 1: If the partial character string excluding the last character in the partial character string matches another partial character string that appears before that, the appearance position 983 of another partial character string (for example, 8 bits) and a partial character string length 984 (for example, 5 bits) and a total of 13-bit codes and a bit string (for example, 8 bits) indicating the last character are converted into two codes.
Rule 2: A partial character string composed of 1 character is converted into two codes: a code indicating an appearance position 0 and a length 0 (for example, 13 bits) and a bit string indicating the character (for example, 8 bits). Here, the appearance position 0 is an example indicating that there is no reference to another character string.

  According to this rule, a code indicating a pointer to another partial character string and a code indicating a bit string representing a character always appear alternately. The flag to indicate is not necessary.

  A description of the encoding and restoration operations is omitted.

A dictionary reference compression technique will be described.
The basic idea in the dictionary reference compression technique is that if the partial character string of the original character string matches a word registered in the dictionary (replacement dictionary), the partial character string is registered in the dictionary. By replacing it with a code, the bit length of the entire character string is to be shortened.
For example, assume that there is a dictionary as shown in FIG. When the character string “cbababbcbc” is compressed, a code string “12233” is obtained. If the bit length per code is 12 bits, the total bit length is 60 bits.
For example, when compressing a document describing a natural language, a high compression ratio can be obtained by registering words of the language or frequently occurring phrases in a dictionary.
The dictionary may be prepared separately from the code string or may be embedded in the code string.

An embedded dictionary reference type compression technique will be described. This compression method includes the LZ78 method and the LZW method.
The embedded dictionary reference type is a form of dictionary reference type. In the embedded dictionary reference type, a dictionary is not prepared separately, but dictionary information is embedded in a code string.
FIG. 48 is a diagram illustrating an example of encoding in the LZ78 system.

  Here, a case where the character string “cbababbcbc” is compressed will be described as an example. Here, it is assumed that the ASCII code is used. Therefore, one character is represented by an 8-bit bit string.

  The partial character string included in the character string is converted into a code string according to the following rules.

Rule 1: If a partial character string is registered in the dictionary, it is converted into a code 971 indicating the dictionary reference number (for example, 10 bits), and the next character is converted into a code of a bit string (8 bits) indicating the character. To 972.
Rule 2: When a partial character string consisting of 1 character is not registered in the dictionary, it is converted into two: a code 971 indicating the reference number 0 and a code 972 indicating a bit string indicating the character. Here, the reference number 0 is an example of a number indicating that it is not registered in the dictionary, and may be another number.

  According to this rule, a code 971 indicating a reference number and a code 972 indicating a bit string expressing a character always appear alternately, so a flag indicating which meaning of the code is necessary. Absent. However, a flag bit for distinguishing between rule 1 and rule 2 may be provided, and in the case of rule 2, it may be converted only to a code indicating the character.

  The partial character string + 1 character converted by this rule is not registered in the replacement dictionary. This is because if it is registered, a partial character string longer by one character can be converted into a code having the same bit length. Therefore, this partial character string + 1 character is newly registered in the replacement dictionary 650.

  Before starting compression, nothing is registered in the replacement dictionary 650. However, a partial character string decided in advance may be registered. For example, if all the partial character strings consisting of one character are registered in the replacement dictionary 650, the rule 2 becomes unnecessary.

Regarding the character string “abababcbc”, the first “a” is not registered in the replacement dictionary 650. Therefore, the rule 2 is applied, and the reference number “0” and the character “a” are output as reference numerals 621 and 622. Then, the partial character string “c” is registered in the replacement dictionary 650. The reference number is “1”.
Since the next “b” is not registered in the replacement dictionary 650, the reference number “0” and the character “b” are output, and “b” is registered in the replacement dictionary 650 (reference number 2).
The next “a” is registered in the replacement dictionary 650 (reference number 1), but “ab” is not registered, so the reference number “1” and the letter “b” are output, and “ab” is replaced. It is registered in the dictionary 650 (reference number 3).
The next “ab” is registered in the replacement dictionary 650 (reference number 3), but “abc” is not registered, so the reference number “3” and the character “c” are output, and “abc” is replaced. It is registered in the dictionary 650 (reference number 4).
The next “b” is registered in the replacement dictionary 650 (reference number 2), but “bc” is not registered, so the reference number “2” and the character “c” are output. “Bc” is registered in the replacement dictionary 650 (reference number 5).

  A procedure for restoring the original character string from the code string replaced in this way will be described.

FIG. 49 is a flowchart showing an example of the flow of control when restoring the original character string from the code string in the conventional example.
First, the replacement dictionary 650 is initialized (S971). For example, empty.
Next, 10 bits (symbol 971 indicating the dictionary reference number) are obtained from the code string (S972). If the reference number is other than 0 (S973), the replacement dictionary 650 is referred to, and the forward character string corresponding to the reference number is obtained. Obtain and output (S974).
8 bits (code 972 of a bit string representing a character) are acquired from the code string (S975), and the character indicated by it is output (S976).
The characters output in S974 and S976 are combined and newly registered in the replacement dictionary 650 (S977).
This is repeated until the code string is exhausted (S978).

For example, a case where the code string 600 of FIG. 48 is restored will be described.
First, the replacement dictionary 650 is emptied.
Reference numeral 621 indicates a reference number “0”, and reference numeral 622 indicates a character “a”. Therefore, the partial character string “a” is restored from these two codes and is output. Then, “a” is registered in the replacement dictionary 650 (reference number 1).
Reference numeral 623 indicates the reference number “0”, and reference numeral 624 indicates the character “b”. Therefore, the partial character string “b” is restored from these two codes and output. Then, “b” is registered in the replacement dictionary 650 (reference number 2).
Reference numeral 625 indicates the reference number “1”, and reference numeral 626 indicates the character “b”. Therefore, the partial character string “ab” is restored from these two codes and is output. Then, “ab” is registered in the replacement dictionary 650 (reference number 3).
Reference numeral 627 indicates the reference number “3”, and reference numeral 628 indicates the character “b”. Therefore, the partial character string “abc” is restored from these two codes and is output. Then, “abc” is registered in the replacement dictionary 650 (reference number 4).
Reference numeral 629 indicates the reference number “2”, and reference numeral 630 indicates the character “c”. Therefore, the partial character string “bc” is restored from these two codes and is output. Then, “bc” is registered in the replacement dictionary 650 (reference number 5).

  Instead of storing the partial character string corresponding to the reference number, the replacement dictionary 650 may store a pointer to the corresponding portion in the restored character string. Alternatively, a reference number corresponding to the front character string and the remaining one character may be stored.

  FIG. 50 is a diagram illustrating an example of encoding in the LZW method, which is also a kind of embedded dictionary reference compression technology.

  Here, a case where the character string “cbababbcbc” is compressed will be described as an example. Here, it is assumed that the ASCII code is used. Therefore, one character is represented by an 8-bit bit string.

  The partial character string included in the character string is converted into a code string according to the following rules.

  Rule 1: The partial character string is converted into a code 971 indicating the reference number of the dictionary.

  In the dictionary, all partial character strings consisting of one character that may appear are registered first. Therefore, there is no case where the partial character string is not registered in the dictionary. Therefore, unlike the LZ78 format, there is no conversion rule (rule 2) when it is not registered in the dictionary.

  In this example, since there are only three types of characters “a”, “b”, and “c” that may appear, the dictionary includes three partial character strings “a”, “b”, and “c”. Registered first.

  The converted partial character string + the next one character (unconverted) is newly registered in the dictionary.

  According to this rule, since there is only one rule, it is not necessary to distinguish the encoding rule. Further, since there are partial character strings registered in the dictionary from the beginning, the compression rate is better than that of the LZ78 method.

  A detailed description of the encoding and restoration will be omitted.

Embodiment 1 FIG.
The first embodiment will be described with reference to FIGS.

FIG. 1 is a diagram showing an example of the appearance of a compressed text search device 100 (an example of a character string search device) in this embodiment.
In FIG. 1, a compressed text search device 100 includes a system unit 910, a CRT (Cathode Ray Tube) display device 901, a keyboard (K / B) 902, a mouse 903, a compact disc device (CDD) 905, a printer device 906, a scanner device. 907, which are connected by a cable.
Further, the compressed text search apparatus 100 is connected to a FAX machine 932 and a telephone 931 via a cable, and is connected to the Internet 940 via a local area network (LAN) 942 and a gateway 941.

FIG. 2 is a diagram showing an example of a hardware configuration of the compressed text search device in this embodiment.
In FIG. 2, the compressed text search apparatus 100 includes a CPU (Central Processing Unit) 911 that executes a program. The CPU 911 includes a ROM (Read Only Memory) 913, a RAM (Random Access Memory) 914, a communication board 915, a CRT display device 901, a K / B 902, a mouse 903, a FDD (Flexible Disk Drive) 904, and a bus 912. A device 920, a CDD 905, a printer device 906, and a scanner device 907 are connected.
The RAM 914 is an example of a volatile memory. The ROM 913, the FDD 904, the CDD 905, and the magnetic disk device 920 are examples of nonvolatile memories. These are examples of a storage device or a storage unit.
The communication board 915 is connected to a FAX machine 932, a telephone 931, a LAN 942, and the like.
For example, the communication board 915, the K / B 902, the scanner device 907, the FDD 904, and the like are examples of the input unit.
Further, for example, the communication board 915, the CRT display device 901, and the like are examples of the output unit.

Here, the communication board 915 is not limited to the LAN 942 and may be directly connected to the Internet 940 or a WAN (Wide Area Network) such as ISDN. When directly connected to a WAN such as the Internet 940 or ISDN, the compressed text search apparatus 100 is connected to a WAN such as the Internet 940 or ISDN, and the gateway 941 is unnecessary.
The magnetic disk device 920 stores an operating system (OS) 921, a window system 922, a program group 923, and a file group 924. The program group 923 is executed by the CPU 911, the OS 921, and the window system 922.

The program group 923 stores programs that execute functions described as “˜units” in the description of the embodiments described below. The program is read and executed by the CPU 911.
In the file group 924, what is described as “determination result of”, “calculation result of”, and “processing result of” in the description of the embodiment described below is stored as “˜file”. Yes.
In addition, the arrow portion of the flowchart described in the description of the embodiment described below mainly indicates input / output of data, and for the input / output of the data, the data is the RAM 914 or the magnetic disk device 920, FD (Flexible Disk). , Optical disc, CD (compact disc), MD (mini disc), DVD (Digital Versatile Disk) and other recording media. Alternatively, it is transmitted through a signal line or other transmission medium.

  In addition, what is described as “unit” in the description of the embodiment described below may be realized by firmware stored in the ROM 913. Alternatively, it may be implemented by software alone, hardware alone, a combination of software and hardware, or a combination of firmware.

  A program for implementing the embodiment described below also includes a RAM 914 or a magnetic disk device 920, an FD (Flexible Disk), an optical disk, a CD (Compact Disk), an MD (Mini Disk), a DVD (Digital Versatile Disk), and the like. You may memorize | store using the recording apparatus by other recording media.

FIG. 3 is a block diagram showing an example of a block configuration of the compressed text search apparatus 100 according to this embodiment.
This compressed text search device determines whether or not there is a character string that meets the search condition in the input compressed text, and if it exists, the search device outputs the end position of the character string as a hit position It is. If it does not exist, nothing is output.
In FIG. 3, the compressed text search apparatus 100 includes a search condition input unit 102, a compressed text storage unit 103, a collation result output unit 104, a state transition table generation unit 105, a state transition table storage unit 106, and a collation unit 107. . The collation unit 107 includes a compressed block acquisition unit 108, a character acquisition unit 109, a state transition machine 110, a state storage unit 111, a state transition storage unit 112, a compression dictionary storage unit 113, a condition determination unit 114, a current position counter 115, a transition destination A calculation unit 116 and a search success determination unit 117 are included.

  The search condition input unit 102 inputs a search condition (an example of a search pattern). Search conditions are expressed using regular expressions. However, a fixed search character string may be used.

The state transition table generation unit 105 has a function of generating a state transition table corresponding to a DFA (an example of an automaton) that accepts a character string that matches a search condition when an input of the search condition is received.
That is, based on the search condition input to the search condition input unit 102, an NFA corresponding to the search condition is obtained. Further, the NFA is converted into DFA, and a corresponding state transition table (an example of a transition destination list) is generated.
The state transition table storage unit 106 stores the state transition table generated by the state transition table generation unit 105.

  The compressed text storage unit 103 stores compressed text (an example of a code string). The compressed text is obtained by encoding an electronic document (an example of a character string) using a compression technique and shortening the entire bit length. Note that the compressed text does not necessarily need to be an encoded document, and may be encoded data stored in a computer.

When the collation unit 107 receives input of the compressed text, the collation unit 107 refers to the state transition table to determine whether or not there is a character string that meets the search condition in the compressed text. It has a function to output.
That is, when the compressed text stored in the compressed text storage unit 103 is input, it is determined whether or not a partial character string that matches the search condition is included in the uncompressed document corresponding to the compressed text. In this case, the appearance position (hit position) is output.

  The hit result output by the collation unit 107 is displayed on the CRT display device 901 by the collation result output unit 104, for example.

The compressed block acquisition unit 108 (an example of a code acquisition unit) has a function of acquiring one compressed block (an example of a code) one by one from the input compressed text.
That is, the compressed blocks constituting the compressed text stored in the compressed text storage unit 103 are acquired in order from the top.

The compression dictionary storage unit 113 (an example of a dictionary storage unit) has a function of acquiring and storing a compression dictionary (an example of a replacement dictionary) from compressed text.
That is, in the embedded dictionary reference type compression method in which the compressed dictionary information is embedded in the compressed text, the compressed dictionary information embedded in the compressed text is extracted and stored. Alternatively, in the case of a dictionary reference compression method in which a compression dictionary is prepared separately from the compressed text, a separately prepared compression dictionary is acquired and stored.

The character acquisition unit 109 (an example of a character string restoration unit) acquires characters one by one from the compressed block acquired by the compression block acquisition unit 108 while referring to the compression dictionary stored in the compression dictionary storage unit 113. It has a function.
That is, the partial character string corresponding to the compressed block is obtained based on the compression dictionary stored in the compression dictionary storage unit 113. Further, the characters constituting the partial character string are acquired in order from the top and input to the state transition machine 110.

The state storage unit 111 has a function of storing the current state of the state transition machine 110. The state transition machine 110 has a function of acquiring the next state by referring to the state transition table based on the states of the character acquisition unit 109 and the state storage unit 111 and updating the state of the state storage unit 111. The state transition storage unit 112 has a function of storing a history of state transitions corresponding to the character strings in the compression dictionary storage unit 113.
That is, the state storage unit 111 and the state transition machine 110 are examples of an automaton execution unit, and execute a DFA corresponding to the state transition table stored in the state transition table storage unit 106. The state stored in the DFA is stored in the state storage unit 111. The state transition machine 110 refers to the state transition table stored in the state transition table storage unit 106 based on the state of the DFA stored in the state storage unit 111 and the character input in the character acquisition unit 109, and changes to the transition destination state. To get. The state storage unit 111 stores (updates) the transition destination state acquired by the state transition machine 110 by overwriting the old DFA state as the DFA state.

  The state transition storage unit 112 (an example of a history storage unit) stores a DFA state history (state history) stored in the state storage unit 111.

  The condition determining unit 114 determines conditions such as whether to restore the compressed block to the original partial character string.

  The current position counter 115 is a counter indicating the current position of the search in order to obtain the hit position when the original character string includes a search character string that matches the search condition.

  The transition destination calculation unit 116 calculates the DFA transition destination state based on the state history stored in the state transition storage unit 112.

  The search success determining unit 117 determines whether or not a search character string that matches the search pattern is included in the original character string, and if it is included, the appearance position is output.

  Each functional unit of the compressed text search apparatus 100 of FIG. 3 may be configured by a plurality of arithmetic devices such as CPUs and a storage device such as a memory, or operate on a single arithmetic device and one or more storage devices. It may be realized as software.

FIG. 4 is a diagram showing an example of the compressed text stored in the compressed text storage unit 103 in this embodiment.
The compressed text compressed by lexicographic compression (dictionary reference compression method) is composed of a compressed block string 300 (an example of a code string) and a compression dictionary 303 (an example of a replacement dictionary).
One compression block 302 (an example of a code) includes the reference number of one entry in the compression dictionary.
The compression dictionary storage unit 113 extracts compression dictionary information from the compressed text and stores it. The compression dictionary is a table showing correspondence between a character string 305 (an example of a partial character string) and a reference number 304 (an example of a code) of the character string.
When decompressing (restoring) the compressed block sequence 300 “121414” illustrated in FIG. 4, the compressed block 302 is acquired one by one from the compressed block sequence 300, and the compressed block 302 is converted into the character string 305 of the compression dictionary based on the reference information. (Hereinafter referred to as a compressed block reference character string).
For example, since the first compressed block 302 refers to the first entry of the compression dictionary, the first compressed block can be replaced with the character string “abcde” of the first entry. Similarly, it is possible to obtain the decompressed text “abcdecbabcdebecdabcded” from the compressed block sequence 300 by repeating for all the compressed blocks. In lexicographic compression, by replacing a character string appearing in text in this way with a compressed block having a bit length shorter than that character string, a higher compression rate can be obtained as the same character string repeatedly appears.

Here, the compression dictionary is described in the form of a table. However, the compression dictionary implementation method is not limited as long as the compression block and the compression dictionary entry can be associated one-to-one. That is, a tree structure or a hash may be used.
Hereinafter, the reference number of the compressed block is expressed as a numerical value surrounded by <>.

FIG. 5 is a diagram showing the structure of the state history stored in the state transition storage unit 112 in this embodiment.
The state transition storage unit 112 has information of a reference number 401 (an example of a code), a state transition history 402, and an acceptance position 403.
The reference number 401 is associated with the compression dictionary reference number 304 on a one-to-one basis.
The state transition history 402 stores the state transition history of the state transition machine by the character string of the corresponding compression dictionary, the state immediately before reading the character string of the compression dictionary at the beginning, The state immediately after reading everything. For example, in the case of the first entry, starting from the state [1], the state transitions to [2]-[3]-[4]-[5] each time a character string “abcde” is read, Immediately after reading the last “e”, it means that state [6] has been reached.
The receiving position 403 represents the character number of the character string in the compression dictionary and the state transition machine has reached the receiving state. For example, assume that state [4] in FIG. 4 is an accepted state. At this time, the acceptance position 403 of entry 1 means that the acceptance state [4] has been reached immediately after reading the third character “c” in the character string of the compression dictionary.
Here, there is only one receiving position for each of the first and third entries, but there may be two or more receiving positions in one state transition history.

  In this embodiment, a state transition machine capable of uniquely determining a state transition is used for matching between a regular expression and a text. A representative example of such a state transition machine is DFA.

  FIG. 6 is a diagram showing an example of the state transition table 200 stored in the state transition table storage unit 106 in this embodiment. The leftmost column of the state transition table 200 represents the current state. The first line represents the next input character. The other elements represent the next state (transition destination state). For example, when the current state is [1] and the next input character is “a”, the state [2] of the row of the state [1] and the column of the character “a” (second row and second column) is The next state.

The state transition table 200 of the state transition machine that accepts the regular expression included in the search condition is generated by the state transition table generation unit 105 until the compressed text is input and collation is started.
For example, the search condition input unit 102 inputs a regular expression “(ab | dec) [ce] e *” as a search condition (where “[ce]” is a simplified notation of “(c | e)”. ). This regular expression means “abc”, “bee”, “abce”, “abee”, “decc”, “dece”, “decce”, “decee”, and so on. The state transition table 200 in FIG. 6 is generated by the state transition table generation unit 105 based on this regular expression.

  FIG. 7 is a flowchart showing an example of the control flow of search processing of the compressed text search apparatus 100 in this embodiment.

In the initialization process (S10), based on the search condition input by the search condition input unit 102, the state transition table generation unit 105 generates a state transition table corresponding to the DFA that accepts the search condition, and the state transition table storage unit 106 stores.
Information of the compression dictionary is extracted from the compressed text stored in the compressed text storage unit 103 and stored in the compression dictionary storage unit 113.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state transition history.
The current position counter 115 initializes the current position to be stored to 0 in order to count the length of text (or original text) before compression.

  The compressed block acquisition unit 108 acquires one compressed block in order from the beginning of the compressed block sequence (S11).

The condition determining unit 114 is the first DFA state stored in the state storage unit 111 and the state transition history corresponding to the compressed block stored in the state transition storage unit 112 (hereinafter referred to as the state transition history of the compressed block). It is determined whether the state matches (S12).
That is, among the state histories stored in the state transition storage unit 112, the state transition history 402 corresponding to the compressed block acquired by the compressed block acquiring unit 108 is referred to, and the initial state (the partial character string corresponding to the compressed block is selected). The state of the DFA before being input to the DFA) is acquired. The acquired initial state is compared with the current DFA state to determine whether or not they match.

  When the condition determination unit 114 determines that they match (the first condition is satisfied) (S12), the transition destination calculation unit 116 refers to the matching state transition history and determines the last state (in the compressed block). (DFA state after the corresponding partial character string is input to the DFA) is acquired (transition destination state). The transition destination calculation unit 116 updates the DFA state stored in the state storage unit 111 to the acquired transition destination state (S13).

When the transition destination calculation unit 116 updates the state of the DFA, the state that should have passed in the middle is skipped. If there is an acceptance state, the DFA does not enter the acceptance state and proceeds to the next.
Therefore, the state transition storage unit 112 stores whether or not there is an accepting state in a state that should be passed, and if there is, the position (accepting position 403).
Therefore, the search success determination unit 117 refers to the matched state transition history among the state histories stored in the state transition storage unit 112, and determines whether there is an accepted state (S14).
When there is an acceptance state, the search success determination unit 117 matches the state transition stored in the state history stored in the state transition storage unit 112 to the current original text length (current position) stored in the current position counter 115. A receiving position 403 corresponding to the history 402 is added to obtain and output a hit position (S15). In that case, the collation result output unit 104 displays on the CRT display device 901.
In some cases, there may be a plurality of receiving positions. In that case, the appearance position (hit position) is output for all the reception positions.
Conversely, if there is no acceptance state, nothing is output.

  When the condition determining unit 114 determines that they do not match (the second condition is satisfied) (S12), the partial character string is restored from the compressed block, input to the DFA, and processing for obtaining the state transition history is performed ( S16).

  Finally, the current position counter 115 adds the length (number of characters) of the partial character string corresponding to the compressed block to the stored current position, and updates and stores it (S17).

  The above processing is repeated until the compressed block is exhausted (S18).

  FIG. 8 is a flowchart showing an example of details of the process in S16 of FIG.

  The state transition storage unit 112 prepares a temporary storage area H for the state transition history, and stores the DFA state (initial state) stored in the state storage unit 111 as an initial state (S161). Alternatively, the storage area H may be emptied and the DFA state may be added to the top of the storage area H.

  Next, the character acquisition unit 109 refers to the reference character string of the compressed block (the partial character string corresponding to the reference number acquired from the compression dictionary stored in the compression dictionary storage unit 113), and acquires one character at a time from the beginning. Then, input to the DFA (S162).

  The state transition machine 110 acquires the next state by referring to the state transition table stored in the state transition table storage unit 106 based on the input character and state. That is, referring to the state transition table stored in the state transition table storage unit 106 based on the current state of the DFA stored in the state storage unit 111 and the characters input in the character acquisition unit 109, the transition destination state is acquired. Then, it is set (updated) in the state storage unit 111. The state storage unit 111 newly stores the transition destination state acquired by the state transition machine 110 as a DFA state (S163).

  The state transition storage unit 112 adds the new state of the DFA stored in the state storage unit 111 to the end of the storage area H (S164).

  The search success determination unit 117 determines whether the new state of the DFA stored in the state storage unit 111 is an acceptance state (S165).

When the search success determination unit 117 determines that the search is in the accepting state, the state transition storage unit 112 stores the number of characters input to the DFA by the character acquisition unit 109 as a reception position in the storage area H (S166).
Further, the reception position is added to the current position stored by the current position counter 115 and output as a hit position (S167). The collation result output unit 104 displays the output hit position on the CRT display device 901.

  This is repeated until the characters constituting the reference character string are exhausted (S168).

  Finally, the state transition storage unit 112 reflects the state transition history and the reception position stored in the storage area H in the state transition history of the compressed block. That is, it is copied and stored in the position corresponding to the reference number of the compressed block (S169).

  Instead of preparing the temporary storage area H, the state transition storage unit 112 may store the state transition history and the reception position directly at the position corresponding to the reference number of the compressed block. This is preferable because the step of copying the storage contents of the storage area H can be omitted.

  Moreover, it is good also as memorize | storing only the first state and the last state, without memorize | storing the middle of a state transition history. This is preferable because it saves the storage area.

  Further, the state transition storage unit 112 may store a plurality of state histories corresponding to the initial state as the state history corresponding to the reference number of the compressed block. If it does so, possibility that the first state memorize | stored as a state history and the present state will become high and a state transition can be skipped, and it is preferable.

Alternatively, only one state history may be stored as the state history corresponding to the reference number of the compressed block. This is preferable because it saves a storage area for storing the state history.
In that case, when the state history (the first state is different) corresponding to the reference number of the compressed block is already stored, the state history may not be stored. Alternatively, the state history may be overwritten only when the initial state is a specific state (for example, the initial state). This is preferable because it is highly possible that the first state and the current state coincide with each other when the appearance frequency is high, and the state transition can be skipped.

  How the compressed text search apparatus 100 actually operates according to this procedure will be described using the specific examples shown in FIGS.

By the initialization process (S10), each storage unit initializes the stored contents.
The state transition table storage unit 106 stores the state transition table 200 shown in FIG. The state transition table 200 is generated by the state transition table generation unit 105 based on the search condition (including the regular expression) input by the search condition input unit 102. In the DFA corresponding to the state transition table 200, the initial state is a state corresponding to the state number 1, and the accepting state is a state corresponding to the state number 4. This is also stored in the state transition table storage unit 106. The initial state of DFA is always one, but there may be a plurality of acceptance states.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state history.
The compression dictionary storage unit 113 stores a compression dictionary.
The current position counter 115 stores 0 as the current position.

  The compressed block acquisition unit 108 acquires a compressed block from the compressed block sequence 300 stored in the compressed text storage unit 103 (S11). The first compressed block 302 is “1”.

The condition determination unit 114 compares the current DFA state stored in the state storage unit 111 with the first state of the state transition history corresponding to the compressed block in the state history stored in the state transition storage unit 112 ( S12).
However, since the state history is empty, the corresponding first state is not stored.
Therefore, the condition determining unit 114 determines that they do not match (to S16).

  The state transition storage unit 112 stores the current state (= 1) of the DFA stored in the state storage unit 111 in the storage area H (S161).

  The character acquisition unit 109 refers to the reference number 1 of the compression dictionary stored in the compression dictionary storage unit 113, acquires the first character “a” from the partial character string “abcde”, and inputs it to the DFA (S162).

  Since the current state of the DFA is state number 1 and the input character is “a”, the state transition machine 110 refers to the state transition table 200 stored in the state transition table storage unit 106 and sets the state number 2 as the transition destination state. get. The state storage unit 111 stores state number 2 as the new DFA state (S163).

  The state transition storage unit 112 adds the state number 2 to the end of the storage area H (S164), and becomes “12”.

  The search success determining unit 117 determines that the state (state number 2) stored in the state storage unit 111 is not the accepting state (state number 4) (S165). Therefore, the processing of S166 and S167 is not performed.

Since the characters constituting the reference character string still remain (S168), the character acquisition unit 109 acquires the next character “b” and inputs it to the DFA (S162).
Since the state transition machine 110 has the state number 2 and the input character “b”, the state transition unit 200 acquires the transition destination state number 3 from the state transition table 200 and stores the state storage unit 111 (S163).
The state transition storage unit 112 adds the state number 3 to the end of the storage area H (S164), and becomes “123”.
The search success determining unit 117 determines that the current state (state number 3) is not the accepting state (state number 4) (S165).

Since the characters are not yet exhausted (S168), the character acquisition unit 109 acquires the next character “c” and inputs it to the DFA (S162).
Since the state number is 3 and the input character is “c”, the state of the new DFA becomes the state number 4 and is added to the storage area H (S163, S164).
The search success determination unit 117 determines that the current state (4) is the acceptance state (4) (S165).
The state transition storage unit 112 stores the reception position in the storage area H (S166). Up to this point, since the number of characters input to the DFA by the character acquisition unit 109 is 3, the reception position is 3.
Further, the search success determining unit 117 adds the acceptance position (= 3) to the current position (= 0) stored by the current position counter 115, and calculates and outputs the hit position (= 3) (S167).

Since the characters are not yet exhausted (S168), the next character “d” is input to the DFA (S162).
Since the state number is 4 and the input character is “d”, the state of the new DFA becomes 5 and is stored (S163, S164).
Since the current state (5) is not the acceptance state (4), the process proceeds to the next (S165).

  In the same manner (S168), the next character “e” is input (S162), and a new state 6 is entered (S163, S164). Since it is not an acceptance state, it progresses to the next (S165).

Since the characters constituting the reference character string have been exhausted (S168), the state transition storage unit 112 corresponds to the state transition history and the reception position stored in the storage area H with reference number 1 of the compressed block in the state history. Copy to the position (S169).
Up to this point, since “123456” is stored as the state transition history and “3” is stored as the reception position in the storage area H, this is stored in the position corresponding to the reference number 1.

  The length (number of characters, = 5) of the partial character string corresponding to the compressed block is added to the current position (= 0) stored by the current position counter 115, and the current position is updated to 5 (S17).

  Since the compressed block is not yet exhausted (S18), the compressed block acquisition unit 108 acquires the next compressed block “2”.

  Since the state transition storage unit 112 does not store the state transition history corresponding to the reference number 2, the condition determination unit 114 determines that they do not match (S12).

The state transition history is obtained in the same manner as before (S16).
Since the partial character string corresponding to the reference number 2 of the compression dictionary is “cb”, the state of the DFA is the first state by inputting the state number 6, the character “c”, the state number 3, and the character “b”. To state number 1.
Since there is no acceptance state (4), the search success judgment unit 117 outputs nothing.
Therefore, the state transition storage unit 112 stores “631” as the state transition history at a position corresponding to the reference number 2 of the state history. Since there is no receiving position, it is not memorized.

  The current position (= 5) stored by the current position counter 115 is 7 by adding 2 characters (S17).

  Since the compressed block is not yet exhausted (S18), the compressed block acquisition unit 108 acquires the next compressed block “1”.

  The state transition storage unit 112 stores a state transition history corresponding to the reference number 1. The current state of the DFA stored in the state storage unit 111 is 1. Among the state histories stored by the state transition storage unit 112, the first state of the state transition history corresponding to the reference number 1 also matches with 1. Therefore, the condition determination unit 114 determines that they match (S12).

  Therefore, the transition destination calculation unit 116 acquires the state number 6 of the transition destination state from the last state of the matched state transition history, and stores it in the state storage unit 111 (S13).

  The search success determination unit 117 determines whether or not the state transition storage unit 112 stores the reception position corresponding to the matched state transition history (S14). In this case, since the acceptance position “3” is stored, the acceptance position (= 3) is added to the current position (= 7), and the hit position (= 10) is calculated and output (S15).

  The current position (= 7) stored by the current position counter 115 is 12 after adding the number of characters 5 (S17).

  Thereafter, similarly, the compressed block acquisition unit 108 continues the process until there is no compressed block to be acquired (S18), and the search process ends when the process is completed up to the end of the compressed block.

In this example, the condition determination unit 114 performs state transition corresponding to the reference number of the compressed block in the current state of the DFA stored in the state storage unit 111 and the state transition history stored in the state transition storage unit 112. It is determined whether the initial state of the history matches.
However, each time the state of the DFA is updated (transitioned), the state transition history stored in the state transition storage unit 112 is compared with the corresponding state, and if they match, the remaining state transition may be skipped.

Further, instead of comparing each time one state is updated, comparison may be made after updating a predetermined number of times.
For example, after obtaining the next state for the i-th character (i is a natural number) in the reference character string, the next state is compared with the i-th state in the state transition history, and if they match, the process in S16 is terminated, and the process in S13 It may be configured to move. At this time, the first to i-th state transition history stored in the temporary storage area H is reflected in the state transition history of the compressed block.

  Note that the state transition history of the compressed block may be constantly updated in the process of step S16, or may not always be updated. In other words, the state transition history may not be updated from the first acquired state transition history, or the history may be updated only when the state at the head of the state transition history is a specific state.

As described above, according to this embodiment, when the current state of the DFA stored in the state storage unit 111 does not match the first state of the acquired state transition history of the compressed block, the state transition Is processed by the number of steps corresponding to the length of the reference character string of the compressed block. On the other hand, when the current state matches the first state of the state transition history, the state transition can be processed in one step. In a state transition machine in which the state transition destination is uniquely determined by the current state and characters, the current state is often the initial state. For this reason, the higher the compression ratio, that is, the text in which a long character string repeatedly appears, the number of steps required for state transition can be reduced.
The collation of whether a character string that matches a regular expression exists in the text itself uses a state transition machine that has been conventionally used to uniquely determine a state transition that accepts a regular expression.
As described above, the compressed text search apparatus according to this embodiment can search the compressed text at high speed according to the search condition including the regular expression.

The search device described here has the following features.
This is a search device that searches a text compressed by a lexicographic compression method using a regular expression without decompressing the text.
The search uses a state transition machine that can uniquely determine the state transition.
It consists of a state transition machine, a state transition table generation unit, a compressed block acquisition unit, a character acquisition unit, a state storage unit, a state transition storage unit, and a compression dictionary storage unit.
When searching, for each character string in the dictionary referenced by the compressed block, the state transition history of the state transition machine is stored in the state transition storage unit, and the current state is the head of the state transition history referenced by the compressed block. Transition to a state at the end of the history in a single state transition.

  In this way, in a character string search device that performs a search using state transition of an automaton, a search is performed by directly acquiring a compressed block from the compressed text without decompressing the compressed text and restoring the original character string. By doing so, the time required for decompressing the compressed text can be reduced, and the search can be speeded up.

  If the acquired compressed block has not been searched in the past, it is restored to the original partial character string, the state transition of the automaton is performed, and the search is performed. However, if there is a search in the past and the state of the automaton at that time is the same, there is no need to perform the same state transition again. Therefore, by storing the state transitions when the search is performed in the past and completing the state transition of the automaton only once, there is an effect that the search becomes faster.

  In addition, the state after the transition is not calculated in order to complete the state transition of the automaton in one time, but the state transition when the search is actually performed is stored, so there is no unnecessary calculation. . As a result, there is an effect that the search becomes faster.

  By using a deterministic finite automaton as an automaton used for the search, the transition destination state can be uniquely calculated, so that there is no need for backtracking and the speed of the search is increased.

  Since a search pattern expressed by a regular expression can be used for specifying a search character string, the degree of freedom of search is increased, and an efficient search can be achieved.

  In general, a DFA configured to be able to search for a regular expression is more complicated and has more states than a DFA configured to be able to search for a fixed character string. For such a DFA, it is wasteful to calculate the state after the transition in advance so that the state transition of the compressed block is completed only once. However, as in this embodiment, if a system that stores state transitions when actually searched is stored as a history, there is no wasteful calculation, so that the search is fast.

Embodiment 2. FIG.
The second embodiment will be described with reference to FIGS. 3, 6, 9 to 10, and 46.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, a case where a text compressed by the LZ77 method is searched will be described.

  FIG. 9 is a diagram illustrating an example of storage contents stored in the compressed text storage unit 103 and the state transition storage unit 112 (an example of a history storage unit) in this embodiment.

  The compressed text storage unit 103 stores a compressed block sequence 300 (an example of a code sequence). The compressed block string 300 shown in FIG. 9 is obtained by replacing the original character string 500 “abcdecbabcdebecdabcded” by applying the rule of FIG. 46.

The state transition storage unit 112 (an example of a history storage unit) stores a state of a DFA (an example of an automaton) and characters input to the DFA as a state history.
The state transition storage unit 112 stores characters and states in a one-to-one correspondence as state history.
For example, the character 411 and the state 461 are stored in association with each other. Hereinafter, the character 412 and the state 462, and the character 413 and the state 463 are also stored in association with each other. In the following description, the state history is expressed by enclosing the state and characters with “()”. For example, the state history shown in FIG. 9 is expressed as “(1, a) (2, b) (3, c) (4, d).

The state history stored in the state transition storage unit 112 is initially empty. It increases little by little as search progresses.
For example, the character 411 corresponds to the first character “a” of the original character string 500 and is stored when this character is input to the DFA. The state 461 is the state of the DFA stored in the state storage unit 111 before the character 411 is input to the DFA.

  FIG. 10 is a flowchart showing an example of a control flow of search processing of the compressed text search apparatus 100 in this embodiment.

In the initialization process (S10), based on the search condition (an example of a search pattern) input by the search condition input unit 102, the state corresponding to the state transition machine (DFA) in which the state transition table generation unit 105 accepts the search condition A transition table is generated and stored in the state transition table storage unit 106.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state history.
The current position counter 115 stores 0 as the current position.

  The compressed block acquisition unit 108 (an example of a code acquisition unit) acquires one compressed block (an example of a code) in order from the beginning of the compressed block sequence (an example of a code sequence) (S21).

  Since the compressed block sequence 300 is replaced by applying the rule of FIG. 46, the odd-numbered compressed block is denoted by reference numeral 985 or 986, and the even-numbered compressed block is denoted by reference numeral 982. In S21, since an odd-numbered compressed block is acquired, if it is not a dummy pointer (reference numeral 986), information on pointers to other character strings is included.

  The condition determining unit 114 checks whether the compressed block acquired by the compressed block acquiring unit 108 is 0. If the compressed block is not 0, the condition determining unit 114 determines that the pointer information to another character string is included (S22).

  If the condition determination unit 114 determines that it includes the compressed block, it decodes the compressed block, and the appearance position 983 (distance from the current position) of the other partial character string (the distance from the current position) and the length of the partial character string 984 is obtained. Further, from the state history stored in the state transition storage unit 112, the state and characters corresponding to the appearance position (of the state history stored previously by the number of characters of the appearance position) are acquired (S23), and the acquired state and state storage are acquired. The current state of the DFA stored in the unit 111 is compared (S24).

In S24, when the condition determination unit 114 determines that they match (the first condition is satisfied), the state transition storage unit 112 adds the state and characters acquired by the condition determination unit 114 to the end of the state history (S251). ).
The transition destination calculation unit 116 acquires the next state and character from the state history (S252).
The search success determination unit 117 determines whether or not the state acquired by the transition destination calculation unit 116 is an acceptance state. If the state is an acceptance state, the collation result output unit 104 sets the current position counter 115 as a hit position. Is stored (S253).
The current position stored in the current position counter 115 is incremented by 1, and the remaining length of the partial character string is decremented by 1 (S254).
If there is any remaining partial character string, the process is repeated from S251 (S255).
When the processing of S251 to S255 is completed, the transition destination calculation unit 116 updates the state of the DFA stored in the state storage unit 111 to the state acquired in S252 (S256).

In S24, when the condition determination unit 114 determines that they do not match (the second condition is satisfied), the state transition storage unit 112 displays the state stored in the state storage unit 111 and the character acquired by the condition determination unit 114 in the state. Add to the end of the history (S261).
The character acquisition unit 109 (an example of a character string restoration unit) acquires the character acquired by the condition determination unit 114 and inputs it to the DFA. The state transition machine 110 refers to the state transition table stored in the state transition table storage unit 106 based on the characters input by the character acquisition unit 109 and the current state of the DFA stored in the state storage unit 111, and makes transitions Get the destination state. The state transition machine 110 stores the acquired transition destination state in the state storage unit 111 and updates the current state (S262).
The search success determination unit 117 determines whether or not the current state of the updated DFA is an acceptance state, and if it is an acceptance state, the collation result output unit 104 sets the current position counter 115 as a hit position. The current position to be stored is output (S263).
The current position stored in the current position counter 115 is incremented by 1, and the remaining length of the partial character string is decremented by 1 (S264).
If there is any remaining partial character string, the process is repeated from S23 (S265).

  In the iterative process, the condition determination unit 114 determines again whether the states match (S24). This is because even if the state before the partial character string is input is different from the state history, some characters may be matched after being input to the DFA.

  As a result, even if the initial states do not match, if the states match in the middle, the subsequent state transition can be completed once, and the search can be performed at high speed.

However, in order to prevent a decrease in processing speed due to an increase in condition judgment, the state comparison may be performed only at the beginning.
Or it is good also as a structure which does not compare every time but compares once in several times.

In S22, when the condition determination unit 114 determines that the pointer information to another character string is not included, or when the processing from S23 onward is completed, the compressed block acquisition unit 108 starts from the compressed block sequence 300. The compressed block is acquired (S27).
Since this is an even-numbered compressed block, it is a code 982 of a bit string representing a character.
Therefore, the condition determination unit 114 unconditionally determines that this does not include information on pointers to other character strings.

  The character acquisition unit 109 acquires a character corresponding to the compressed block acquired by the compressed block acquisition unit 108, and the state transition storage unit 112 acquires the current state of the DFA stored in the state storage unit 111 and the character acquisition unit 109. The added character is added to the end of the state history (S281).

  The character acquisition unit 109 inputs the acquired character to the DFA, the state transition machine 110 calculates the transition destination state, and updates the current state of the DFA stored in the state storage unit 111 (S282).

  The search success determination unit 117 determines whether or not the current state of the updated DFA is an acceptance state, and if it is an acceptance state, the collation result output unit 104 sets the current position counter 115 as a hit position. The current position to be stored is output (S283).

  The current position stored in the current position counter 115 is incremented by one (S284).

  If the compressed block still remains in the compressed block string 300, the process is repeated from S21 (S29).

  The operation described above will be described in detail using the specific examples shown in FIGS.

By the initialization process (S10), each storage unit initializes the stored contents.
The state transition table storage unit 106 stores the state transition table 200 shown in FIG. The state transition table 200 is generated by the state transition table generation unit 105 based on the search condition (including the regular expression) input by the search condition input unit 102. In the DFA corresponding to the state transition table 200, the initial state is a state corresponding to the state number 1, and the accepting state is a state corresponding to the state number 4. This is also stored in the state transition table storage unit 106. The initial state of DFA is always one, but there may be a plurality of acceptance states.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state history.
The current position counter 115 stores “1” as the current position.

The compressed block acquisition unit 108 acquires a compressed block from the compressed block sequence 300 stored in the compressed text storage unit 103 (S21). The first compressed block 311 is “0, 0”.
The condition determining unit 114 determines that the acquired compressed block does not include pointer information to other partial character strings (S21), and proceeds to S27.

The compressed block acquisition unit 108 acquires the next compressed block 312 “a” (S27).
The state transition storage unit 112 adds the current state “1” and the character “a” of the DFA stored in the state storage unit 111 to the end of the state history (state 461 and character 411) (S27).
The character acquisition unit 109 inputs the character “a” to the DFA, and the state of the DFA becomes “2” (S282). Since it is not in the accepting state, no output is made (S283), and the current position becomes “2” (S284). The process proceeds to the next compressed block (S29).

Since the acquired compressed block (S21) is “0, 0” (S22), the process proceeds to S27.
Since the acquired compressed block (S27) is “b”, the state “2” and the character “b” are added to the state history (state 462 and character 412) (S281), and the character “b” is input to the DFA. The DFA status becomes “3” (S282). It is determined whether it is in an accepting state (S283), and the current position becomes “3” (S284). Proceed to the next (S29).

Since the acquired compressed block (S21) is “0, 0” (S22), the process proceeds to S27.
Since the acquired compressed block (S27) is “c”, the state “3” and the character “c” are added to the state history (state 463 and character 413) (S281), and the character “c” is input to the DFA. The DFA status is “4” (S282). Since it is in the accepting state, the hit position (= 3) is output (S283), and the current position becomes “4” (S284). Proceed to the next (S29).

Since the acquired compressed block (S21) is “0, 0” (S22), the process proceeds to S27.
Since the acquired compressed block (S27) is “d”, the state “4” and the character “d” are added to the state history (S281), and when the character “d” is input to the DFA, the DFA state is “5”. (S282). It is determined whether it is in an accepting state (S283), and the current position becomes “5” (S284). Proceed to the next (S29).

Since the acquired compressed block (S21) is “0, 0” (S22), the process proceeds to S27.
Since the acquired compressed block (S27) is “e”, the state “5” and the character “e” are added to the state history (S281), and when the character “e” is input to the DFA, the DFA state is “6”. (S282). It is determined whether it is in the accepting state (S283), and the current position becomes “6” (S284). Proceed to the next (S29).

  At this time, the state history becomes “(1, a) (2, b) (3, c) (4, d) (5, e)”.

Since the acquired compressed block “3, 1” (S21) is a pointer (S22), the appearance position “3” and the length “1” are acquired, and the state 463 “3” three characters before (appearance position = 3). "And the character 413" c "are acquired (S23).
The acquired state “3” is compared with the current state (= 6) of the DFA stored in the state storage unit 111 (S24).
Since they do not match, the current state “6” and the acquired character “c” are added to the state history (S261). The character “c” is input to the DFA, and the DFA status becomes “3” (S262). It is determined whether it is in an accepting state (S263), and the current position becomes “7” (S264). Since the input for the length of characters has been completed (S265), the process proceeds to S27.

  Since the next acquired compressed block (S27) is “b”, the state “3” and the character “b” are added to the state history (S281), and when the character “b” is input to the DFA, the DFA state is “1”. (S282). It is determined whether it is in an accepting state (S283), and the current position becomes “8” (S284). Proceed to the next (S29).

  At this time, the state history becomes “(1, a) (2, b) (3, c) (4, d) (5, e) (6, c) (3, b)”.

Since the next acquired compressed block “7, 5” (S21) is a pointer (S22), the appearance position “7” and the length “5” are acquired, and the state 461 “1” 7 characters before (appearance position = 7). "And the character 411" a "are acquired (S23).
The acquired state “1” is compared with the current state (= 1) of the DFA stored in the state storage unit 111 (S24).

  Since they match, the acquired state “1” and character “a” are added to the state history (state 468 and character 418) (S251). The state 462 “2” and the character 412 “b” seven characters before are acquired from the state history (S252), and it is determined whether the acquired state “2” is an accepting state (S253). The current position is “9” (S254), and the repetition is the remaining four characters (S255).

  The acquired state “2” and character “b” are added to the state history (state 469 and character 419) (S251). The state 463 “3” and the character 413 “c” seven characters before are acquired from the state history (S252), and it is determined whether the acquired state “3” is an accepting state (S253). The current position is “10” (S254), and the repetition is the remaining three characters (S255).

  The acquired state “3” and character “c” are added to the state history (state 470 and character 420) (S251). The state “4” and the character “d” seven characters before are acquired from the state history (S252), and since the acquired state “4” is the accepting state, the hit position “10” is output (S253). The current position is “11” (S254), and the repetition is the remaining two characters (S255).

  The acquired state “4” and the character “d” are added to the state history (S251). The state “5” and the character “e” seven characters before are acquired from the state history (S252), and it is determined whether the acquired state “5” is an accepting state (S253). The current position is “12” (S254), and the repetition is the remaining one character (S255).

  The acquired state “5” and character “e” are added to the state history (S251). The state “6” and the character “c” seven characters before are acquired from the state history (S252), and it is determined whether the acquired state “6” is an accepting state (S253). The current position becomes “13” (S254), and the repetition ends (S255). The current state of the DFA is “6” (S256).

  Since the acquired compressed block (S27) is “b”, the state “6” and the character “b” are added to the state history (S281), and when the character “b” is input to the DFA, the DFA state is “1”. (S282). It is determined whether it is in an accepting state (S283), and the current position becomes “14” (S284). Proceed to the next (S29).

Since the acquired compressed block “8, 2” (S21) is a pointer (S22), the appearance position “8” and the length “2” are acquired, and the state “6” eight characters before (appearance position = 8) is acquired. And the character “c” is acquired (S23).
The acquired state “3” is compared with the current state (= 1) of the DFA stored in the state storage unit 111 (S24).
Since they do not match, the current state “1” and the acquired character “c” are added to the state history (S261). The character “c” is input to the DFA, and the DFA status becomes “1” (S262). It is determined whether it is in the accepting state (S263), and the current position becomes “15” (S264). The repetition is the remaining one character (S265).

The state “3” and the character “b” eight characters before are acquired (S23).
The acquired state “3” is compared with the current state (= 1) of the DFA stored in the state storage unit 111 (S24).
Since they do not match, the current state “1” and the acquired character “b” are added to the state history (S261). The character “b” is input to the DFA, and the DFA state becomes “1” (S262). It is determined whether it is in the accepting state (S263), and the current position becomes “16” (S264). The repetition ends (S265).

  Since the acquired compressed block (S27) is “d”, the state “1” and the character “d” are added to the state history (S281), and when the character “d” is input to the DFA, the DFA state is “5”. (S282). It is determined whether it is in the accepting state (S283), and the current position becomes “17” (S284). Proceed to the next (S29).

Since the acquired compressed block “9, 5” (S21) is a pointer (S22), the appearance position “9” and the length “5” are acquired, and the state “1” 9 characters before (appearance position = 9) is acquired. And the character “a” is acquired (S23).
The acquired state “1” is compared with the current state (= 5) of the DFA stored in the state storage unit 111 (S24).
Since they do not match, the current state “5” and the acquired character “a” are added to the state history (S261). The character “a” is input to the DFA, and the DFA status becomes “2” (S262). It is determined whether it is in the accepting state (S263), and the current position becomes “15” (S264). The repetition is the remaining 4 characters (S265).

The state “2” and the character “b” 9 characters before are acquired (S23).
The acquired state “2” is compared with the current state (= 2) of the DFA stored in the state storage unit 111 (S24).
Since they match, the acquired state “2” and character “b” are added to the state history (S251). The state “3” and the character “c” nine characters before are acquired from the state history (S252), and it is determined whether the acquired state “3” is an accepting state (S253). The current position is “16” (S254), and the repetition is the remaining three characters (S255).

  Hereinafter, since the same processing is repeated, description thereof is omitted.

  Note that the state transition storage unit 112 need not store all state histories. For example, if the bit string representing the appearance position 983 has a bit length of 8 bits, only a maximum of 255 characters before can be referred to. Therefore, even if the state history of 256 characters or more is stored, it is not used, and may be deleted in order from the oldest one.

  As described above, when searching for text compressed in the LZ77 format, the function of the sliding window generally used to restore the compressed text in the LZ77 format is expanded, and the state of the DFA at that time is stored together. Thus, it is possible to use the state transition when the search is performed in the past, and it is not necessary to repeat the same state transition again. Thereby, there is an effect that the search can be performed at high speed.

Embodiment 3 FIG.
A thirtieth embodiment will be described with reference to FIGS. 3 and 11 to 13.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another case of searching for text compressed by the LZ77 method will be described.

  FIG. 11 is a diagram showing the structure of compressed text in the LZ77 format. In the LZ77 format, instead of having a fixed compression dictionary as shown in FIG. 4, a part of its own text called a fixed-length sliding window is used as a dictionary. The length of the sliding window depends on the mounting method. The compressed text in the LZ77 format is composed of only the compressed block sequence 300. Each compression block 802 has information on the position of the matched character string in the sliding window 803, the matched character string length, and the first mismatched character. For example, in the example of FIG. 11, the compressed block 804 has a matching character string position = 1, a matching character string length = 5, and the first mismatch character = “b”. This is because the compression block 804 adds a character string “b” to the character string from the first character to the fifth character of the sliding window 803 (the length of the sliding window is 8 in this example). abcdeb "means equal. Here, the numerical value added below the sliding window 803 is added for convenience in order to clarify the character position in the sliding window. Hereinafter, a character string starting from the position of the matching character string in the sliding window and having the length of the matching character string length is referred to as a reference character string of the compressed block. The position of the character string that matches the compressed block is also referred to as a reference position, and the length of the matching character string is also referred to as a reference character string length.

  So far, various methods have been proposed for speeding up the reference of the slide window of the LZ77 format. In this embodiment, if the above-mentioned function of the slide window is provided, the implementation method is as follows. It doesn't matter.

  FIG. 12 illustrates information stored in the compression dictionary storage unit 113 (an example of a dictionary storage unit) and a state transition storage unit 112 (an example of a history storage unit) in this embodiment. The compression dictionary storage unit 113 stores the sliding window 210. The state transition storage unit 112 stores a state transition history 220 that stores a state transition of sliding window length + 1 and a reception position 230. The acceptance position 230 stores the position of the acceptance state in the state transition history 220. When the reference position of the compressed block is n, the nth state from the first state of the state transition history is referred to as a state transition history reference position.

  FIG. 13 is a flowchart of search processing in the compressed text search apparatus 100 of this embodiment. As an initial state, it is assumed that the state transition table generation unit 105 generates a state transition table from the search conditions input by the search condition input unit 102 and stores the state transition table storage unit 106. It is assumed that the initial state (= 1) is set in the state storage unit 111. It is assumed that the state transition history and the reception position of the compression dictionary storage unit 113 and the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

  First, in step S701, the compressed block acquisition unit 108 acquires compressed blocks one by one in order from the beginning of the compressed block sequence. In step S702, it is determined whether the current state of the DFA stored in the state storage unit 111 matches the state of the reference position in the state transition history. If the states match (YES), the next state transition is acquired from the top of the state transition history (reference position + reference character string length) and the first mismatch character in step S703, and the current state is set. set. In step S704, it is determined whether there is an acceptance state between the reference position of the state transition history and the (reference position + reference character string length) -th position. If there is an acceptance state (YES), the hit position is calculated and output in step S705. The hit position is the current original text length + accepted position. Even when the current state is an acceptance state, the original text length + reference character string length is output as the hit position. If there is no acceptance state (NO), the process proceeds to step S706 without doing anything. In step S706, the sliding window and the state transition history are updated.

  In updating the sliding window, first, the character string in the sliding window is shifted forward by (reference character string length + 1) characters. Next, the reference character string and the first mismatch character are added after the last character of the sliding window. Similarly, the state transition history is also shifted forward by (reference character string length + 1) characters, and the state transition history for the reference character string length from the reference position is added at the end. The current state is further added to the state transition history.

  In step S707, it is determined whether or not the end of the compressed block sequence has been reached. If not (NO), the next compressed block sequence is acquired in step S701. If it has reached (YES), the search process is terminated. In step S707, the reference character string length + 1 is added to the original text length.

  If the current state does not match the state at the reference position of the state transition history in step S702 (NO), the state transition history for the character string obtained by adding the first unmatched character to the reference character string in step S708. Ask for. That is, as in the processing flow of FIG. 7, the next state is acquired by the state transition machine while acquiring one character at a time in order from the beginning of the character string obtained by adding the mismatch character to the reference character string. When the process of step S708 ends, the sliding window and the state transition history are updated in step S706. That is, the character string in the sliding window is shifted forward by (reference character string length + 1) characters, and the reference character string and the first mismatch character are added after the last character of the sliding window. Similarly, the state transition history is similarly shifted forward by (reference character string length + 1) characters, and the state transition history acquired in step S708 is set at the end.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of times corresponding to the character string length can be processed by two state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, the compressed text compressed in the LZ77 format can be searched at high speed according to the search condition including the regular expression.

The compressed text search apparatus described here has the following features.
The LZ77 format sliding window is stored in the compression dictionary storage unit.
In the state transition storage unit, a state transition history having a length of sliding window length + 1 is stored.
When a compressed block in the LZ77 format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is transitioned by one state transition up to the last character of the reference character string, Further, the state is changed by the mismatch character.

Embodiment 4 FIG.
The fourth embodiment will be described with reference to FIGS. 3, 14, 15, and 44. FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, a case where a text compressed by the LZSS method is searched will be described.

  FIG. 14 is a diagram illustrating an example of the stored contents of the compressed text storage unit 103 and the state transition storage unit 112 in this embodiment.

  The compressed text storage unit 103 stores a compressed block sequence 300 (an example of a code sequence). A compressed block string 300 shown in FIG. 14 is obtained by replacing the original character string 500 “abcdecbabcdebecdabcded” by applying the rule of FIG.

The state transition storage unit 112 (an example of a history storage unit) stores a state of a DFA (an example of an automaton) and characters input to the DFA as a state history.
In this embodiment, unlike Embodiment 2, a plurality of states corresponding to characters can be stored.
For example, the character 431 is associated with the state 481. The character 438 is associated with the state 488 and the state 448. As specific implementation methods, structures such as a table format, a list format, and a pointer format are conceivable, but other implementation methods may be used.
Further, as in the second embodiment, characters and states may be stored in a one-to-one correspondence.

FIG. 15 is a flowchart showing an example of the control flow of search processing of the compressed text search apparatus 100 in this embodiment.
FIG. 15 is almost the same as FIG. 10 described in the second embodiment, so only different parts will be described.

  The LZSS method is different from the LZ77 method in encoding method. Therefore, the control flow also differs in the corresponding parts.

That is, the compressed block acquisition unit 108 first acquires 1 bit (flag 981) from the compressed block sequence 300, and determines the length of the compressed block. When the flag 981 is “1”, it is rule 1 in FIG. 44, and therefore the subsequent 8 bits (bit string 982) are acquired. If the flag 981 is “0”, it is the rule 2 of FIG. 44, so the subsequent 13 bits (appearance position 983 and length 984) are acquired (S21).
When the flag 981 is “1”, the condition determination unit 114 determines that the pointer information to another partial character string is not included (S22), and the character acquisition unit 109 acquires the character from the compressed block, and the state transition The storage unit 112 adds the current state and the acquired character to the end of the state history (S281).
When the flag 981 is “0”, the condition determination unit 114 determines that the pointer information to the other partial character string is included (S22), and the other partial character string (of the first character) from the compressed block. The appearance position 983 (distance from the current position) and the length 984 of the partial character string are acquired. Further, the state and characters corresponding to the appearance position are acquired from the state history stored in the state transition storage unit 112, and compared with the current state of the DFA stored in the state storage unit 111 (S24).

  This embodiment is different from the second embodiment. In this embodiment, the state transition storage unit 112 is configured so that a plurality of states can be stored corresponding to characters.

In S24, when there are a plurality of acquired states, the condition determining unit 114 determines whether there is any one that matches the current state of the DFA (S24).
If there is a match, the state transition storage unit 112 adds all the acquired states and characters including the unmatched state to the end of the state history (S251).
The transition destination calculation unit 116 acquires the next state and character (S252).
The search success determination unit 117 determines whether the state corresponding to the matched state among the acquired states is the accepting state (S253).

For example, in FIG. 14, the state transition storage unit 112 stores two states of a state 488 “4” and a state 448 “2” corresponding to the character 438 “b”. In contrast, assume that the current state is “2”.
In S23, the condition determination unit 114 acquires the states “4” and “2” and the character “b”.
In S24, the condition determination unit 114 determines that the acquired states (4, 2) match because the current state (= 2) is included.
In S251, the state transition storage unit 112 adds all the acquired states “4” and “2” and the character “b” to the end of the state history.
In S252, the transition destination calculation unit 116 acquires the next state 495 “1”, state 475 “3”, and the character 445 “c”.

  Here, the next state 489 “1” corresponds to the state 488 “4”, and indicates a state in which a transition is made by inputting the character “b” when the DFA state is the state 488 “4”. Yes. Further, the next state 449 “3” corresponds to the state 448 “2”, and indicates a state in which a transition is made by inputting the character “b” when the DFA state is the state 448 “2”. . In FIG. 15, this correspondence relationship is expressed using arrows, but the state transition storage unit 112 stores this relationship using, for example, a pointer.

  In S253, the search success determining unit 117 determines whether or not the state 449 “3” corresponding to the matched state 448 is the accepting state among the two acquired states.

  Similarly, in S256, when there are a plurality of acquired states, the transition destination state calculated by the transition destination calculating unit 116 also becomes a state corresponding to the matched state, and the state storage unit 111 stores the state (S256). ).

In S24, if none of the acquired states matches the current state of the DFA, the condition determination unit 114 determines that they do not match (S24).
The state transition storage unit 112 adds the current state, the acquired state, and the acquired character to the end of the state history (S261).

  Therefore, if the states do not match, the number of states stored corresponding to one character is increased by one.

  In the self-referencing compression technique, when the same partial character string appears many times in the original character string, the partial character string closer to the current position may be referred to. This is because it is not possible to refer to the far side due to the limitation on the number of bits when encoding the appearance position. In addition, when referring to a partial character string that is too far away, it takes too much time for compression and consumes a storage area for restoration, which may not be practical.

  In the case where the same partial character string appears many times, if only one state is stored, it is unlikely that the states match. However, if a plurality of states can be stored, there is a high possibility that the states will match. Thereby, there is an effect that the search can be performed at higher speed.

  The operation described above will be described in detail with reference to specific examples shown in FIGS.

The processing is completed up to the compression block 334, and the state transition storage unit 112 stores states 481 to 484 and characters 431 to 434 as state history.
The state storage unit 111 stores “5” as the current state of the DFA.

The next compressed block 335 is acquired (S21). Since the flag 981 is “0”, it is a pointer (S22). Since the appearance position is “4” and the length is “2”, the state 481 “1” and the character 431 “a” four characters before are acquired (S23), and the acquired state “1” and the current state “5” are obtained. Compare (S24).
Since they do not match, the current state “5”, the acquired state “1”, and the character “a” are added to the end of the state history (states 485 and 445, character 435) (S261).
The character “a” is input to the DFA, and the current state transitions to “2” (S262).
It is determined whether it is in an accepting state (S263), and the current position is advanced (S264). The repetition is one character remaining (S265).

The state 482 “2” and the character 432 “b” four characters before are acquired, and the acquired state “2” is compared with the current state “2” (S24).
Since they match, the acquired state “2” and character “a” are added to the end of the state history (state 486 and character 436) (S251).
At this time, since the same state “2” is obtained regardless of whether the previous state is “1” or “5”, the state 486 is associated with both the state 485 and the state 445 (arrows in FIG. 14).

  From the state history, the state 483 “3” and the character 433 “c” four characters before are acquired (S252), it is determined whether or not it is in an accepting state (S253), and the current position is advanced (S254). The repetition ends (S255).

  The DFA state is updated to the acquired state “3” (S256), and the process proceeds to the next compressed block (S29).

  The next compressed block 336 is acquired (S21). Since the flag 981 is “1”, it is a character (S22). The character “e” is acquired, and the current state “3” and the character “e” are added to the state history (state 487 and character 437) (S281). When the character “e” is input to the DFA, the state of the DFA becomes “4” (S282). It is determined whether it is in an accepting state (S283) and the current position is advanced (S284). The process proceeds to the next compressed block (S29).

  The next compressed block 337 is acquired (S21). Since the flag 981 is “0”, it is a pointer (S22). Since the appearance position is “6” and the length is “7”, the state 482 “2” and the character 432 “b” six characters before are acquired (S23), and the acquired state “2” and the current state “4” are obtained. Compare (S24).

Since they do not match, the current state “4” and the acquired state “2” and the character “b” are added to the state history (state 488, 448, character 438) (S261).
The character “b” is input to the DFA, and the current state transitions to “1” (S262).
It is determined whether it is in an accepting state (S263), and the current position is advanced (S264). The repetition is the remaining 6 characters (S265).

The state 483 “3” and the character 433 “c” six characters before are acquired (S23), and the acquired state “3” is compared with the current state “1” (S24).
Since they do not match, the current state “1”, the acquired state “3”, and the character “c” are added to the state history (state 489, 449, character 439) (S261).
At this time, the state 489 is associated with the state 488. Further, the state 449 is associated with the state 448.
The character “c” is input to the DFA, and the current state transitions to “1” (S262).
It is determined whether it is in an accepting state (S263), and the current position is advanced (S264). The repetition is the remaining 5 characters (S265).

The state 484 “4” and the character 434 “d” six characters before are acquired (S23), and the acquired state “4” is compared with the current state “1” (S24).
Since they do not match, the current state “1”, the acquired state “4”, and the character “d” are added to the state history (states 490, 450, character 440) (S261).
At this time, the state 490 is associated with the state 489. The state 450 is associated with the state 449.
The character “d” is input to the DFA, and the current state transitions to “5” (S262).
It is determined whether it is in an accepted state (S263), and the current position is advanced (S264). The repetition is the remaining 4 characters (S265).

The state 485 “5”, state 445 “1”, and character 435 “a” six characters before are acquired (S23), and the acquired state is compared with the current state “1” (S24).
Since it matches the state 445, the acquired states “1” and “5” and the character “a” are added to the state history (states 491 and 451, character 441) (S251).
At this time, the state 491 is associated with the state 490 and the state 450. This is because the transition to the same state “1” is made from either state. Also, the state 451 is not associated with the previous state.

The state 486 “2” and the character 435 “b” six characters before are acquired (S252).
It is determined whether the acquired state is an acceptance state (S253), and the current position is advanced (S254). The repetition is the remaining three characters (S265).

  Hereinafter, since the same processing is repeated, description thereof is omitted.

  As described above, when searching for text compressed in the LZSS format, the function of the sliding window generally used to restore the compressed text in the LZSS format is expanded, and the DFA state at that time is stored together. Thus, it is possible to use the state transition when the search is performed in the past, and it is not necessary to repeat the same state transition again. Thereby, there is an effect that the search can be performed at high speed.

  Also, by storing not only the state of the DFA at that time but also the state transitions that were searched in the past that are referenced, the past state transitions can be used even when they are not directly referenced. This eliminates the need to repeat the same state transition again. Thereby, there is an effect that the search can be performed at higher speed.

  Note that the configuration in which a plurality of states can be stored for one character as described above is not limited to LZSS compressed text, and can also be used when searching for LZ77 compressed text. .

Furthermore, there are various other self-reference compression methods. For example, the above-described LZ77 or LZSS code string is further replaced by Huffman coding (static or dynamic), and the entire bit length is further shortened.
The embodiments described here can also be applied to text compressed by these compression methods.

Embodiment 5 FIG.
Embodiment 5 will be described with reference to FIG. 3, FIG. 13, and FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another case of searching for text compressed by the LZSS method will be described.

  FIG. 16 is a diagram showing the structure of compressed text in the LZSS format. The LZSS format is a compression format for the purpose of further improving the compression efficiency by deleting redundant data in the compressed block of the LZ77 format. In the LZSS format, like the LZ77 format, instead of having a fixed compression dictionary 303 as shown in FIG. 4, a part of its own text called a fixed-length sliding window is used as a dictionary. The compressed text in the LZSS format is composed of only the compressed block sequence 300. In the LZ77 format, even when there is no reference character string in the sliding window, the compressed block includes redundant information such that reference position = 0 and reference character string length = 0. In the LZSS format, redundant data is deleted by providing a 1-bit flag at the head of the compressed block. If there is no reference character string in the sliding window, the leading bit is set to 0 as in the compressed block 1002, and then a non-matching character is set. When the reference character string exists, the leading bit is set to 1 as in the compression block 1004, and then the reference position and the reference character string length are set.

  So far, various methods have been proposed for speeding up the reference of the slide window in the LZSS format, but the compressed text search apparatus of this embodiment has the above-described function of the slide window. The implementation method does not matter.

  The flow of search processing in the compressed text search apparatus 100 of this embodiment is the same as that shown in FIG. As an initial state, it is assumed that the state transition table generation unit 105 generates a state transition table from the search conditions input by the search condition input unit 102 and the state transition table storage unit 106 stores the state transition table. In addition, it is assumed that the initial state is set in the state storage unit 111. It is assumed that the state transition history and the reception position of the compression dictionary storage unit 113 and the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

  The main difference from the third embodiment is step S702 and step S706. First, in step S701, compressed blocks are acquired one by one in order from the beginning of the compressed block sequence. In step S702, the first bit of the compressed block is determined. Further, when the first bit of the compressed block is 1, it is determined whether the current state matches the state of the reference position of the state transition history. If the states match (YES), in step S703, the (reference position + reference character string length) -th state from the top of the state transition history is set to the current state. In step S704, it is determined whether the (reference position + reference character string length) -th state from the reference position of the state transition history is the accepting state. If it is in the accepting state (YES), the hit position is calculated and output in step S705, and the process proceeds to step S706. If there is no acceptance state (NO), the process proceeds to step S706 without doing anything. In step S706, the sliding window and the state transition history are updated.

  In updating the sliding window, first, the character string in the sliding window is shifted forward by the reference character string length. Next, a reference character string is added after the last character of the sliding window. Similarly, the state transition history is also shifted forward by the reference character string length, and the state transition history for the reference character string length from the reference position is added at the end.

  In step S707, it is determined whether or not the end of the compressed block sequence has been reached. If not (NO), the next compressed block sequence is acquired in step S701. If it has reached (YES), the search process is terminated. In step S707, 1 is added to the original text length when the first bit of the compressed block is 0, and the reference character string length is added when the first bit is 1.

  If the first bit of the compressed block is 0 in step S702 or the current state does not match the state of the reference position in the state transition history (NO), in step S708, the mismatched character or reference character string is detected. Find the history of state transitions. That is, as in the processing flow of FIG. 5, the next state is acquired by the state transition machine while acquiring one character at a time from the beginning of the character string. When the process of step S708 ends, the sliding window and the state transition history are updated in step 706. That is, the character string in the sliding window is shifted forward by one character or the reference character string length. Next, an unmatched character or a reference character string is added after the last character of the sliding window. Similarly, the state transition history is also shifted forward by one character or the reference character string length, and the state transition history acquired in step S708 is added at the end.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state of the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of steps proportional to the character string length can be processed in one state transition, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, the compressed text compressed in the LZSS format can be searched at high speed according to the search condition including the regular expression.

  In addition to the LZSS format shown here, text compressed in a compression format derived from the LZ77 format, such as the LZB (Lempel-Ziv-Bell) format and the LZBW (Lempel-Ziv-Bender-Wolf) format, A similar search can be performed by the form of compressed text search device.

The compressed text search apparatus described here has the following features.
The LZSS format sliding window is stored in the compression dictionary storage unit.
In the state transition storage unit, a state transition history having a length of sliding window length + 1 is stored.
When a compressed block in the LZSS format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is transitioned by one state transition to the last character of the reference character string.

Embodiment 6 FIG.
A sixth embodiment will be described with reference to FIGS. 3, 6, 17, 18, and 48. FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, a case where a text compressed by the LZ78 method is searched will be described.

  FIG. 17 shows an example of storage contents stored in the compressed text storage unit 103, the compression dictionary storage unit 113 (an example of a dictionary storage unit), and a state transition storage unit 112 (an example of a history storage unit) in this embodiment. FIG.

  The compressed text storage unit 103 stores a compressed block sequence 300. The compressed block string 300 is obtained by replacing the original character string 500 “aabbcdecabbcdebabcdec” by applying the rule of FIG.

The state transition storage unit 112 stores the state history in the form of a table with the “reference number” of the dictionary as a row and the “first state” as a column. However, instead of such a table form, it may be stored in a list form, for example.
As the state history, “transition destination state” and “accepting position” are stored. The state history is initially empty and fills up as the search proceeds.

The compression dictionary storage unit 113 stores a compression dictionary (an example of a replacement dictionary) extracted from the compressed block sequence 300. The compression dictionary associates a “reference number” of the dictionary with a forward reference number and a suffix (an example of a backward character string). Here, the partial character string corresponding to the “reference number” is obtained by adding a suffix after the partial character string corresponding to the forward reference number (an example of the forward character string). When the forward reference number is “0”, the corresponding partial character string is a character string composed of one suffix character.
For example, reference number 1 corresponds to the partial character string “a”. Reference number 2 corresponds to a partial character string “a” corresponding to reference number 1 with a suffix “b” (“ab”). Reference number 3 is “ab” + “c”, which corresponds to “abc”.
The compression dictionary is initially empty, and entries increase as the search proceeds.

  FIG. 18 is a flowchart showing an example of the control flow of search processing of the compressed text search apparatus 100 in this embodiment.

In the initialization process (S10), based on the search condition (an example of a search pattern) input by the search condition input unit 102, the state corresponding to the state transition machine (DFA) in which the state transition table generation unit 105 accepts the search condition A transition table is generated and stored in the state transition table storage unit 106.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state history.
The compression dictionary storage unit 113 empties the compression dictionary stored in the storage unit.
The current position counter 115 stores 0 as the current position.

  The compressed block acquisition unit 108 (an example of a code acquisition unit) acquires one compressed block (an example of a code) in order from the beginning of the compressed block sequence (an example of a code sequence) (S31). The state transition storage unit 112 stores the current state of the DFA stored by the state storage unit 111 as an initial state.

  According to the rule of FIG. 48, since the code 971 indicating the reference number and the code 972 of the bit string representing the character appear alternately, the odd-numbered compressed block is the code 971 indicating the reference number.

  The compressed block acquisition unit 108 passes the reference number to the condition determination unit 114 and executes a decompression routine (S32).

In the expansion routine, the condition determining unit 114 determines whether or not the received reference number is 0 (S321).
If the reference number is 0, the corresponding partial character string is empty and the expansion routine is terminated.
When the reference number is other than 0, the corresponding partial character string is registered in the compression dictionary. Therefore, the condition determination unit 114 refers to the state history stored in the state transition storage unit 112. In the state history table, the “reference number” row and the “current state” column are looked at to see whether or not the state history when the partial character string of the reference number is searched in the past is stored (S322).

If it is stored, the transition destination calculation unit 116 acquires the transition destination state from the state history (S323), and updates the current state of the DFA stored in the state storage unit 111 to the transition destination state (S324). ).
Next, the search success determining unit 117 acquires the reception position from the state history, and if there is a reception position, calculates and outputs a hit position (S325).

When the condition determination unit 114 determines that the state history is not stored (S322), the character acquisition unit 109 (an example of a character string restoration unit) acquires a forward reference number from the compression dictionary (S41).
The compressed block acquisition unit 108 passes the forward reference number to the condition determination unit 114 and recursively executes the expansion routine (S42).

  Next, the character acquisition unit 109 acquires a suffix from the compression dictionary (S43).

  The character acquisition unit 109 inputs the suffix to the DFA, the state transition machine 110 calculates the transition destination state, and updates the current state of the DFA stored in the state storage unit 111 to the transition destination state (S44).

  The search success determination unit 117 determines whether the current state is an acceptance state, and if it is an acceptance state, calculates and outputs a hit position (S45).

  Finally, the state transition storage unit 112 stores the current state and the reception position as the state history in the “reference number” row and the “first state” column (S48), and the expansion routine ends.

When the expansion routine ends, the compressed block acquisition unit 108 acquires the next compressed block from the compressed text storage unit 103 (S33).
Since this is an even-numbered compressed block, it is a code 972 of a bit string representing a character.

  The character acquisition unit 109 inputs a character (suffix) corresponding to the compressed block to the DFA, the state transition machine 110 calculates the transition destination state, and the current state of the DFA stored in the state storage unit 111 is obtained. The state is updated to the transition destination state (S34).

  The search success determination unit 117 determines whether the current state is an acceptance state, and if it is an acceptance state, calculates and outputs a hit position (S35).

  The current position is updated by adding the lengths of the partial character strings corresponding to the two compressed blocks to the current position stored by the current position counter 115 (S36).

  The compression dictionary storage unit 113 registers the reference number and suffix in the compression dictionary (S37).

  The state transition storage unit 112 acquires a reference number corresponding to the partial character string newly registered in the compression dictionary, and stores the current state as the state history in the “reference number” row and the “first state” column of the state history. And the receiving position is stored (S38).

  The above processing is repeated until there are no more compressed blocks (S18).

  The operation described above will be described using the specific examples shown in FIGS.

By the initialization process (S10), each storage unit initializes the stored contents.
The state transition table storage unit 106 stores the state transition table 200 shown in FIG. The state transition table 200 is generated by the state transition table generation unit 105 based on the search condition (including the regular expression) input by the search condition input unit 102. In the DFA corresponding to the state transition table 200, the initial state is a state corresponding to the state number 1, and the accepting state is a state corresponding to the state number 4. This is also stored in the state transition table storage unit 106. The initial state of DFA is always one, but there may be a plurality of acceptance states.
The state storage unit 111 stores an initial state (state number = 1).
The state transition storage unit 112 empties the stored state history.
The compression dictionary storage unit 113 empties the stored compression dictionary.
The current position counter 115 stores 0 as the current position.

In the main loop, the compressed block acquisition unit 108 acquires the first compressed block from the compressed block sequence 300 stored in the compressed text storage unit 103 (S31). The first compressed block means a reference number, and the reference number is “0”.
The state transition storage unit stores the current state (= 1) of the DFA stored in the state storage unit 111 as the first state.

  Next, the compressed block acquisition unit 108 passes the reference number “0” to the condition determination unit 114 and calls the expansion routine (S32). In order to distinguish from recursive calls, calls from the main loop will be called nesting level 1.

  In the expansion routine, the condition determination unit 114 determines whether the reference number is 0 (S321). In this case, since the reference number is 0, the expansion routine (nesting level 1) ends.

  Returning to the main loop, the compressed block acquisition unit 108 acquires the next compressed block (S33). Since it is an even-numbered compressed block, it is a compressed block meaning a character, and its character (suffix) is “a”.

  The character acquisition unit 109 inputs the character “a” to the DFA, the state transition machine 110 refers to the state transition table 200 stored in the state transition table storage unit 106, calculates the transition destination state (= 2), The current state of the DFA stored in the state storage unit 111 is updated to “2” (S34).

  The search success determining unit 117 determines that the current state (= 2) is not the accepting state (= 4), and does nothing (S35).

  The current position counter 115 adds 1 to the current position, and the current position becomes “1”.

  The compression dictionary storage unit 113 registers the reference number “0” and the suffix “a” in the compression dictionary. Since the compression dictionary in the compression dictionary storage unit 113 is empty, the reference number (registration number) corresponding to the registered partial character string is “1”.

  The state transition storage unit 112 stores the current state “2” (transition destination state) and the acceptance position (“0” because there is no transition) in the column corresponding to the reference number “1” and the first state “1” ( S38).

The next compressed block (reference number “1”) is acquired (S18, S31). The initial state is “2”.
The reference number “1” is passed and the expansion routine (nesting level 1) is called (S32). Since the reference number is other than 0 (S321), the status history is checked (S322).
Since the column of the reference number “1” and the first state “2” in the state history is blank (S322), the reference number “1” column of the compression dictionary is referred to, and the forward reference number “0” is obtained (S41).
The expansion routine is recursively called by passing the forward reference number “0” (S42). Since it is a call from nesting level 1, it will be called nesting level 2.
In the nest level 2 expansion routine, since the reference number is 0 (S321), the process returns without doing anything.
Returning to nesting level 1, the column of reference number “1” in the compression dictionary is referred to, and the suffix “a” is obtained (S43).
The suffix “a” is input to the DFA, and the state of the DFA becomes “2” (S44). Since it is not in the accepting state, no output is made (S45).
The transition destination state “2” acceptance position “0” is stored in the column of the state history reference number “1” and the first state “2” (S48), and the nest level 1 expansion routine ends.

Returning to the main loop, the next compressed block (suffix “b”) is acquired (S33).
The suffix “b” is input to the DFA, and the state of the DFA becomes “3” (S34). Since it is not in the accepting state, nothing is output (S35), and the current position becomes “3” (S36).
A reference number “1” and a suffix “b” are registered in the compression dictionary. The registration number is “2”.
In the state history, the transition destination state “3” reception position “0” is stored in the field of reference number “2”, first state “2”.

The next compressed block (reference number “2”) is acquired (S18, S31). The initial state is “3”.
The reference number “2” is passed and the expansion routine (nesting level 1) is called (S32). Since the reference number is other than 0 (S321), the status history is checked (S322).
Since the column of the reference number “2” and the first state “3” of the state history is blank (S322), the forward reference number “1” is obtained by referring to the column of the reference number “2” of the compression dictionary (S41).
The forward reference number “1” is passed, and the expansion routine (nesting level 2) is recursively called (S42).

In the expansion routine of the nest level 2, since the reference number is other than 0 (S321), the state history is checked (S322).
Since the column of the reference number “1” and the first state “3” in the state history is blank (S322), the reference number “1” column of the compression dictionary is referred to, and the forward reference number “0” is obtained (S41).
The forward reference number “0” is passed, and the expansion routine (nesting level 3) is recursively called (S42).
In the expansion routine of nesting level 3, since the reference number is 0 (S321), it returns without doing anything.

Returning to nesting level 2, the column of reference number “1” in the compression dictionary is referred to, and the suffix “a” is obtained (S43).
The suffix “a” is input to the DFA, and the state of the DFA becomes “2” (S44). Since it is not in the accepting state, no output is made (S45).
The transition destination state “2” acceptance position “0” is stored in the column of the state history reference number “1” and the first state “3” (S48), and the nest level 2 expansion routine ends.

Returning to the nesting level 1, the column “2” in the compression dictionary is referred to, and the suffix “b” is obtained (S43).
The suffix “b” is input to the DFA, and the DFA status becomes “3” (S44). Since it is not in the accepting state, no output is made (S45).
The transition destination state “3” acceptance position “0” is stored in the column of the state history reference number “2” and the first state “3” (S48), and the nest level 1 expansion routine ends.

Returning to the main loop, the next compressed block (suffix “c”) is acquired (S33).
The suffix “c” is input to the DFA, and the state of the DFA becomes “4” (S34). Since it is in the accepting state, the hit position “6” is output (S35), and the current position becomes “6” (S36).
Reference number “2” and suffix “c” are registered in the compression dictionary. The registration number is “3”.
In the state history, the transition destination state “4” reception position “3” is stored in the field of the reference number “3” and the first state “3”.

  Thereafter, the same processing is repeated, and processing for the next six compressed blocks (reference number “0” suffix “d” reference number “0” suffix “e” reference number “0” suffix “c”) is completed. By the way, the compression dictionary and the state history have the contents shown in FIG. The current position stored in the current position counter 115 is “9”, and the current state of the DFA stored in the state storage unit 111 is “3”.

The next compressed block (reference number “3”) is acquired (S31). The initial state is “3”.
The reference number “3” is passed and the expansion routine (nesting level 1) is called (S32). Since the reference number is other than 0 (S321), the status history is checked (S322).
Since the state history is stored in the column of the reference number “3” and the first state “3” of the state history (S322), the transition destination state “4” is acquired from the state history (S323).
The current state of the DFA stored in the state storage unit 111 is updated to “4” (S324).
The acceptance position “3” is acquired from the state history (S325), the hit position “12” is output (S326), and the expansion routine of the nesting level 1 ends.

Returning to the main loop, the next compressed block (suffix “d”) is acquired (S33).
The suffix “d” is input to the DFA, and the state of the DFA becomes “5” (S34). Since it is not in the accepting state, nothing is output (S35), and the current position becomes “13” (S36).
A reference number “3” and a suffix “d” are registered in the compression dictionary. The registration number is “7”.
In the state history, the transition destination state “5” receiving position “3” is stored in the field of the reference number “7” and the first state “3”.

  Thereafter, the process is repeated until the compressed block is exhausted (S18).

  As described above, when searching for text compressed by the LZ78 method, the compression dictionary is constructed while reading the compressed text, and the state of the DFA at that time is stored together, so that the search is performed in the past. State transitions can be used, eliminating the need to repeat the same state transitions again. Thereby, there is an effect that the search can be performed at high speed.

  Further, in the LZ78 system, the partial character string newly registered in the dictionary is obtained by adding one character to the partial character string registered so far. Therefore, if a search is performed for one reference number, the search is also performed for the forward character string included in the search. Therefore, if these are stored in the state history by recursive calls, the state history is often stored, and it is not necessary to repeat the same state transition. Thereby, there is an effect that the search can be performed at high speed.

In this embodiment, the above-described effect is obtained by using a recursive call. However, a recursive call is not necessarily required when searching for compressed text of the LZ78 method. The method described in the first embodiment or the like may be used.
The structure of the compression dictionary stored in the compression dictionary storage unit 113 may also store reference numbers and partial character strings as described in the first embodiment.
The structure of the state history stored in the state transition storage unit 112 may also be the structure described in the first embodiment.

Embodiment 7 FIG.
The seventh embodiment will be described with reference to FIGS. 3, 6, 8, 19, and 20. FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another case of searching for text compressed by the LZ78 method will be described.

  FIG. 19 is a diagram showing the structure of compressed text in the LZ78 format. The LZ78 format does not have a fixed compression dictionary 303 as shown in FIG. The compressed text in the LZ78 format is composed of only the compressed block sequence 300. Each compression block 1202 has a reference number of the compression dictionary 1203 with the longest matching character string and information about the next mismatched character. The compression dictionary 1203 includes a character string 1205 and a reference number 1204 for referring to the character string, and entries are added as needed in the process of decompressing the compressed text. When decompressing the compressed block string as shown in the example of FIG. 19, the compressed block 1206 has the reference number = 9 of the compression dictionary 1203 and the first mismatch character = “a”, so the ninth entry of the compression dictionary 1203 It is replaced with a character string “abcdea” obtained by adding the first mismatch character to the character string. Further, this character string “abcdea” is added as a new entry at the end of the compression dictionary 1203. Hereinafter, the character string in the compression dictionary referred to by the compression block is referred to as a reference character string.

  So far, various methods using a tree structure or a hash have been proposed in order to speed up the reference to the compression dictionary in the LZ78 format. In the compressed text search apparatus according to this embodiment, an implementation method thereof is proposed. Does not matter.

  The state transition storage unit according to this embodiment is the same as that shown in FIG.

  FIG. 20 is a flowchart of search processing in the compressed text search apparatus of this embodiment. As an initial state, it is assumed that the state transition table storage unit 106 stores a state transition table generated by the state transition table generation unit 105 based on a search condition already input by the search condition input unit 102. In addition, it is assumed that the initial state is set in the state storage unit 111. It is assumed that the state transition history and the reception position of the compression dictionary storage unit 113 and the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

  First, in step S1101, the compressed block acquisition unit 108 acquires one compressed block in order from the beginning of the compressed block sequence. In step S1102, it is determined whether the current state of the state storage unit 111 matches the first state of the state transition history referred to by the compressed block. If they match (YES), the next state is acquired from the last state of the state transition history and the mismatched character of the compressed block in step S1103 and set in the state storage unit 111. Next, in step S1104, it is determined whether there is an acceptance position in the state transition history. If there is an acceptance position (YES), the hit position is calculated and output in step S1105, and the process proceeds to step S1106. Here, hit position = original text length + accepting position. Even when the current state is an acceptance state, the original text length + reference character string length is output as the hit position. If there is no receiving position in step S1104 (NO), the process proceeds to step S1106. In step S1106, the compression dictionary and the state transition history are updated.

  For the compression dictionary, a reference character string with an unmatched character added is added as a new entry in the compression dictionary. For the state transition history, the state transition history referred to by the compression dictionary plus the current state is added as a new entry.

  In step S1107, it is determined whether the end of the compressed block string has been reached. If not (NO), the next compressed block is acquired in step S1101. If the end of the compressed block sequence has been reached in step S1107 (YES), the search process is terminated. Although not clearly shown in FIG. 20, here, the reference character string length is added to the original text length.

  In step S1102, if the current state stored in the state storage unit 111 and the first state of the state transition history referred to by the compressed block do not match (NO), the process proceeds to step S1108, and the reference character string does not match The state transition history is obtained for the character string to which is added. When the state transition history is obtained in step S1108, the process proceeds to step S1107. Even when the reference number of the compressed block is 0, that is, when there is no reference character string in the compression dictionary, the process proceeds from step S1102 to step S1108.

  FIG. 8 is a diagram illustrating an example of the processing flow of step S1108. Since FIG. 8 has been described in Embodiment 1, the description thereof is omitted here.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of times corresponding to the length of the character string can be processed with at most two state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, the compressed text compressed in the LZ78 format can be searched at a high speed according to the search condition including the regular expression.

The compressed text search apparatus described here has the following features.
A compression dictionary in the LZ78 format is stored in the compression dictionary storage unit.
When a compressed block in LZ78 format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is shifted by one state transition to the last character of the reference character string. . Further, the state is changed by the mismatch character.
A character string composed of the reference character string and the mismatched character is added as a new entry in the compression dictionary, and the state transition is added as a new entry in the state transition storage unit.

Embodiment 8 FIG.
An eighth embodiment will be described with reference to FIGS.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, a case will be described in which a text compressed by the LZW method is searched.

  FIG. 21 is a diagram showing an example of storage contents stored in the compressed text storage unit 103 and the compression dictionary storage unit 113 (an example of a dictionary storage unit) in this embodiment.

  The compressed text storage unit 103 stores a compressed block sequence 300. The compressed block string 300 is obtained by replacing the original character string 500 “aabbcdecabbcdebabcdec” by applying the rule of FIG. 50.

The compression dictionary storage unit 113 (an example of a dictionary storage unit) stores a compression dictionary (an example of a replacement dictionary). In the compression dictionary, all partial character strings of one character composed of characters that may appear are registered first. In this example, it is assumed that only five types of characters “a”, “b”, “c”, “d”, and “e” appear. Accordingly, in the compression dictionary, first, five partial character strings “a”, “b”, “c”, “d”, and “e” are registered in reference numerals 1 to 5.
After the reference number 6, it is registered as the search proceeds.

  The compression dictionary storage unit 113 stores a “prefix character” as a compression dictionary in addition to the “reference number”, “forward reference number”, and “suffix” described in the sixth embodiment. The prefix character indicates the first character of the substring. It is stored so that the suffix can be obtained immediately when the dictionary is updated.

  FIG. 22 is a diagram illustrating an example of storage contents stored in the state transition storage unit 112 (an example of a history storage unit) in this embodiment.

The state transition storage unit 112 stores a state history. The state history is empty at the beginning of the search, and the state history is registered as the search proceeds.
As described in the sixth embodiment, the state transition storage unit 112 stores “state transition history” and “acceptance position” in a form that can be referred to by “reference number” and “first state”, as described in the sixth embodiment. .
The state transition history stores not only the first state and the last state (transition destination state) but also all intermediate states. However, as described in the sixth embodiment, the transition destination state may be stored.
Even in that case, it is preferable to store the next state (second state) after the first state. This is because the second state is required when updating the state history, and this can be acquired without input to the DFA.

  Here, since the state transition history and the reception position are stored in a list format, when the condition determination unit 114 determines whether the state history starting from the first state that matches the current state is stored, the list Need to search inside. With such a configuration, processing takes time compared to the case of storing in a table format as described in the sixth embodiment, but when the number of DFA states is large, the state transition storage unit 112 is in a state. The storage area required for storing the history can be saved. Therefore, it is only necessary to determine which format to store in consideration of the processing capacity of the CPU, the storage capacity of the hard disk, and the like.

  FIG. 23 is a flowchart showing an example of a control flow of search processing of the compressed text search apparatus 100 in this embodiment.

  The LZW system is different from the LZ78 system in encoding rules. Therefore, the control flow also differs in the corresponding parts. Only different parts will be described below.

  In the initialization process (S10), the compression dictionary storage unit 113 registers all partial character strings of one character composed of characters that may appear. As the reference number, for example, the same number as the character code may be used.

  If no compressed block remains in S18, the search process is terminated.

In S51, the next compressed block (reference numeral 971 indicating a reference number) is acquired. This is because it is necessary to know the next character in order to update the compression dictionary stored in the compression dictionary storage unit 113.
In S52, referring to the compression dictionary stored in the compression dictionary storage unit 113, the first character (prefix character) of the next compression block is acquired. Since the prefix character is stored in the compression dictionary, the dictionary can be updated without decompressing the next compression block.

In S38, the state history is stored for the reference number newly registered in the dictionary. At this time, it is necessary to store the state after another character is input to the DFA as the transition destination state.
Therefore, the state transition storage unit 112 acquires the second state from the state transition history stored for the next reference number.
Or you may acquire a 2nd state with reference to the state transition table which the state transition table memory | storage part 106 memorize | stored.
Alternatively, the state history may be stored after another character is input to the DFA.

  Since the processing of the unfolding routine is the same as that described with reference to FIG. 18 in the sixth embodiment, the description thereof is omitted here.

  As described above, when searching for text compressed by the LZW method, the compression dictionary is constructed while reading the compressed text, and the state of the DFA at that time is memorized together. State transitions can be used, eliminating the need to repeat the same state transitions again. Thereby, there is an effect that the search can be performed at high speed.

  Furthermore, in the LZW method, the partial character string newly registered in the dictionary is obtained by adding one character to the partial character string registered so far. Therefore, if a search is performed for one reference number, the search is also performed for the forward character string included in the search. Therefore, if these are stored in the state history by recursive calls, the state history is often stored, and it is not necessary to repeat the same state transition. Thereby, there is an effect that the search can be performed at high speed.

There are various other compression methods of the embedded dictionary reference type. For example, the above-described LZ78 system or LZW system code string is further replaced by Huffman coding (static or dynamic), and the entire bit length is further shortened.
The embodiments described here can also be applied to text compressed by these compression methods.

Embodiment 9 FIG.
Embodiment 9 will be described with reference to FIGS. 20 and 24. FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another case of searching for text compressed by the LZW method will be described.

  FIG. 24 is a diagram showing the structure of compressed text in the LZW format. The LZW format is a format derived from the LZ78 format. Like the LZ78 format, the compressed text includes only the compressed block sequence 300. The LZW format is characterized in that a one-byte character entry is previously (implicitly) stored in the compression dictionary. Each compression block 1302 has only information on the reference number of the compression dictionary with the longest matching character string. The structure of the compression dictionary is the same as that described with reference to FIG. 19 in the seventh embodiment except that the entry of the one-byte character is included.

  So far, various methods using a tree structure, a hash, and the like have been proposed in order to speed up the reference of the compression dictionary in the LZW format. Does not matter.

  Since the flow of search processing in the compressed text search apparatus of this embodiment is almost the same as that shown in FIG. 20 of Embodiment 7, the flow of search processing will be described with the aid of FIG. As an initial state, it is assumed that the state transition table generation unit 105 generates a state transition table from the search conditions already input by the search condition input unit 102 and the state transition table storage unit 106 stores the state transition table. In addition, it is assumed that the initial state is set in the state storage unit 111. It is assumed that the state transition history and the reception position of the compression dictionary storage unit 113 and the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

  When the compression dictionary storage unit 113 receives a reference to the entry numbers 0 to 255, it returns a 1-byte character having the same character code as the entry number. This eliminates the need to register 1-byte characters in the compression dictionary in advance.

  The difference from FIG. 20 of the seventh embodiment is step S1102 and step S1106. First, in step S1101, the compressed block acquisition unit 108 acquires one compressed block in order from the beginning of the compressed block sequence. In step S1102, it is determined whether the current state of the state storage unit 111 matches the first state of the state transition history referred to by the compressed block. If they match (YES), the last state of the state transition history is set in the state storage unit 111 in step S1103. Next, in step S1104, it is determined whether there is an acceptance position in the state transition history. If there is an acceptance position (YES), the hit position is calculated and output in step S1105, and the process proceeds to step S1106. Here, it is assumed that hit position = current original text length + acceptance position. If there is no receiving position in step S1104 (NO), the process proceeds to step S1106.

  In step S1106, the reference character string of the compressed block added with the first character of the reference character string of the next compressed block is added as a new entry to the compression dictionary. In the state transition history, the current state is added as a new entry to the state transition history referred to by the compressed block. Further, the next state obtained from the first character of the reference character string of the compressed block is added to the entry. If there is no next compressed block, nothing is done in step S1106.

  In step S1107, it is determined whether the end of the compressed block string has been reached. If not (NO), the next compressed block is acquired in step S1101. If the end of the compressed block sequence has been reached in step S1107 (YES), the search process is terminated. Although not clearly shown in FIG. 20, the character string length of the compression dictionary is added to the text length before compression. In this way, it is possible to know how many characters of the current uncompressed text have been searched.

  If it is determined in step S1102 that the current state of the state storage unit 111 does not match the first state of the state transition history referred to by the compressed block (NO), the process proceeds to step S1108, where To obtain the state transition history. When the state transition history is obtained in step S1108, the process proceeds to step S1107.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of times corresponding to the character string length can be processed by two state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, a compressed text compressed in the LZW format can be searched at high speed according to a search condition including a regular expression.

  Similarly to this embodiment, the compressed text compressed by the compression format derived from the LZ78 format can be searched at high speed by the compressed text search device of this embodiment.

The compressed text search apparatus described here has the following features.
A compression dictionary in the LZW format is stored in the compression dictionary storage unit.
When a compressed block in LZW format is read and the current state matches the beginning state of the history of state transitions referred to by the compressed block, the state is changed by one state transition to the last character of the reference character string. The state is changed by the first character of the reference character string of the next compressed block.
A character string composed of the reference character string and the first character of the reference character string of the next compressed block is added as a new entry in the compression dictionary, and the above state transition is added as a new entry in the state transition storage unit.

Embodiment 10 FIG.
The tenth embodiment will be described with reference to FIGS. 7 and 25 to 27.

  In the embodiments so far, it has been assumed that the bit length of the character input to the DFA in the search device matches the bit length of the character constituting the partial character string to be replaced in the compression technique.

However, depending on the code used, the bit length of the bit string representing the character may be different.
For example, when the ASCII code is used, the bit length of the bit string representing the character is 8 bits (1 byte).
On the other hand, when the shift JIS code is used, the bit length of the bit string representing the character is 16 bits (2 bytes).
Furthermore, since the shift JIS code can be mixed with the ASCII code, there may be a case where a character represented by an 8-bit bit string and a character represented by a 16-bit bit string are mixed in the same character string. .

However, in the compression technique, how many bits the bit string representing each character is is not an important problem.
This is because it is only necessary that the bit length of the entire code string obtained is shorter than the bit length of the entire original character string, and it is not necessary to understand what the character string means. .
Therefore, in the compression technique, all character strings are normally handled as being composed of characters having a bit length of 8 bits.

On the other hand, in the search device, the number of characters is more important than the bit length of the bit string representing the characters.
For example, when a character string is displayed on the screen, the user does not need to be aware of how many bits the characters constituting the character string are expressed in the computer.
Therefore, the search device usually displays on the screen that the search character string that matches the search condition is “what character was found”.
In addition, in the search condition designated by a regular expression, a designation method such as “any one character” is possible. In this case, it is irrelevant how many bits the character is represented in the computer.
Therefore, the character input to the DFA in the search device is not necessarily a bit string having a bit length of 8 bits.

  FIG. 25 is an explanatory diagram for explaining a case where the bit length of the bit string expressing the character handled in the compression technique is different from the bit length of the bit string expressing the character handled in the search device.

  When the shift JIS code is used, the original character string 500 is represented by a bit string having an internal representation 550 inside the computer. In FIG. 25, for easy understanding, the bit string of the internal representation is divided into 8 bits and expressed in hexadecimal.

  The code string 600 is obtained by compressing the code string 600 using a compression technique. As described above, in the compression technique, replacement is performed with 8 bits as one character regardless of the bit length of the bit string representing the character. Therefore, the partial character string replaced with the code may be separated at a place different from the character separation of the original character string.

  When the search device searches for this, for example, if the first code is restored and input to the DFA, “A” and “I” can be input, but the last 8 bits are left and input to the DFA. I can't.

  Therefore, this embodiment solves this problem by providing an incomplete character restoration unit 121 and a byte data storage unit 122 (an example of an incomplete character storage unit).

  Since the appearance and hardware configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

FIG. 26 is a block diagram showing an example of a block configuration of the compressed text search apparatus 100 in this embodiment.
The incomplete character restoration unit 121 determines whether the partial character string restored from the compressed block (an example of a code) includes a character (unfinished character) that cannot be input to the DFA. The data is stored in the storage unit 122.
Other portions are the same as those described in Embodiment 1 with reference to FIG.

  The incomplete character becomes a character that can be input to the DFA by combining with the first character (other incomplete characters) of the next partial character string.

  FIG. 27 shows an example of storage contents stored in the compressed text storage unit 103, the compression dictionary storage unit 113 (an example of a dictionary storage unit), and a state transition storage unit 112 (an example of a history storage unit) in this embodiment. FIG.

  The compressed text storage unit 103 stores a compressed block sequence 300. The compressed block string 300 is obtained by replacing the original character string 500 with a dictionary reference compression method and compressing it.

The state transition storage unit 112 stores a state history. At the start of the search, the state history is empty. It fills in as the search progresses.
The state transition storage unit 112 stores a state transition history, an acceptance position, an incomplete character, and an incomplete character at the end as a state history.
The “state transition history” indicates that the partial character string is input to the DFA from the state (first state) before the partial character string is input to the DFA (including the case where there is an incomplete character at the end of the previous partial character string). This is a DFA state transition history up to the state (transition destination state) after input (after input before the incomplete character if there is an incomplete character at the end of the partial character string). Note that the intermediate state may not be stored, and only the first state and the transition destination state may be stored.
The “incomplete character” is an incomplete character stored in the byte data storage unit 122 before the partial character string is expanded. By combining this with the incomplete character at the beginning of the partial character string, the character can be input to the DFA.
“Ending incomplete character” indicates an incomplete character when there is an incomplete character at the end of the partial character string.

The compression dictionary storage unit 113 stores the same dictionary as that used for compression. This may be acquired from what is stored in the compressed text storage unit 103, or may be obtained by extracting information embedded in the compressed block sequence.
The partial character string replaced by the compressed block has a character (for example, reference number 1) that cannot be entered in the DFA at the end (for example, reference number 1) and a character that cannot be entered in the DFA (other characters) as shown here. Incomplete characters) (for example, reference number 2) and in both (for example, reference number 3).

The state transition table storage unit 106 stores the state transition table generated by the state transition table generation unit 105 based on the search condition (search pattern) input by the search condition input unit 102.
For example, the search condition input unit 102 inputs the regular expression “(Ai | Eo) [Uo] e *” as a search condition (where “[Uo]” is an abbreviated notation of “(U | O)”. is there). This regular expression means "Aoi""Aio""Aoio""Aioo""Eouo""Eouo""Euouo""Euouo", etc. The state transition table in FIG. 27 is generated by the state transition table generation unit 105 based on this regular expression.

FIG. 7 is a flowchart showing an example of the control flow of search processing of the compressed text search apparatus 100 in this embodiment.
The control flow in this embodiment is substantially the same as the flow described in the first embodiment with reference to FIG. Only the differences will be described here.

  In S12, the condition determination unit 114 displays the current DFA state stored in the state storage unit 111 and the state transition history corresponding to the compressed block stored in the state transition storage unit 112 (hereinafter referred to as the state transition history of the compressed block). It is also determined whether or not the first state of the character matches the incomplete character stored in the byte data storage unit 122 and the incomplete character in the state transition history. The condition determining unit 114 determines that the both match only when both match, and proceeds to the processing after S62.

  In S13, the transition destination calculation unit 116 updates the DFA state stored in the state storage unit 111 to the transition destination state acquired from the state transition history, and converts the incomplete characters stored in the byte data storage unit 122 into the state transition. Update to the last incomplete character obtained from the history.

  In S16, if the character acquired by the character acquisition unit 109 is an incomplete character, the incomplete character restoration unit 121 determines that and stores the incomplete character in the byte data storage unit 122.

  Since the processing in other parts is the same as that described in Embodiment 1 with reference to FIG. 7, the description thereof is omitted here.

Thus, if there is an incomplete character, it is temporarily stored, and it is determined whether both the DFA status and the incomplete character match. If they match, even if this is expanded and input to the DFA, the same state transition will occur, so this is not expanded and the DFA state transition can be performed once by referring to the past history. I'll do it.
Thereby, even when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, there is an effect that the search is performed at high speed.

  Here, the case where the compressed text compressed by the dictionary reference compression method is searched has been described. However, by combining with the configuration described in the second to ninth embodiments, the compressed text is compressed by another compression method. It is also possible to search for compressed text.

Embodiment 11 FIG.
An eleventh embodiment will be described with reference to FIGS.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, another method will be described in the case where the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compressed text compression method.

  FIG. 28 shows an example of storage contents stored in the compressed text storage unit 103, the compression dictionary storage unit 113 (an example of a dictionary storage unit), and a state transition storage unit 112 (an example of a history storage unit) in this embodiment. FIG.

The state transition storage unit 112 stores a state history. At the start of the search, the state history is empty. It fills in as the search progresses.
The state transition storage unit 112 stores the state transition history, the reception position, the first incomplete character, and the last incomplete character as the state history.
The “state transition history” is obtained from the state (first state) before the partial character string is input to the DFA (after there is an incomplete character at the beginning of the partial character string). A history of DFA state transitions up to the state (transition destination state) after the character string is input to the DFA (after the character string is input before the incomplete character if there is an incomplete character at the end of the partial character string) is there. Note that the intermediate state may not be stored, and only the first state and the transition destination state may be stored.
“First unfinished character” indicates an incomplete character when the partial character string has an incomplete character at the beginning. By combining the incomplete character stored in the byte data storage unit 122 with this, the character can be input to the DFA.
“Ending incomplete character” indicates an incomplete character when there is an incomplete character at the end of the partial character string.

  Other portions are the same as those described in Embodiment 10 with reference to FIG. 27, and thus description thereof is omitted here.

FIG. 7 is a flowchart showing an example of the control flow of search processing of the compressed text search apparatus 100 in this embodiment.
The control flow in this embodiment is substantially the same as the flow described in the first embodiment with reference to FIG. Only the differences will be described here.

In S11, the compressed block acquisition unit 108 (an example of a code acquisition unit) acquires a compressed block (code). When the byte data storage unit 122 stores an incomplete character, the incomplete character restoration unit 121 combines the incomplete character (other incomplete characters) at the head of the partial character string with a character that can be input to the DFA. Obtain this and input it to the DFA to perform state transition processing.
As a result, there are no incomplete characters at the stage of processing in S12, so in S12, it is not necessary to determine whether the incomplete characters match, and it is only necessary to determine the match of the state.

  In S13, the transition destination calculation unit 116 updates the DFA state stored in the state storage unit 111 to the transition destination state acquired from the state transition history, and converts the incomplete characters stored in the byte data storage unit 122 into the state transition. Update to the last incomplete character obtained from the history.

  In S16, if the character acquired by the character acquisition unit 109 is an incomplete character, the incomplete character restoration unit 121 determines that and stores the incomplete character in the byte data storage unit 122.

  Since the processing in other parts is the same as that described in Embodiment 1 with reference to FIG. 7, the description thereof is omitted here.

In this way, if there is an incomplete character, it is temporarily stored, combined with the incomplete character at the head of the next compressed block, and input to the DFA. Since it is determined whether or not the DFA status matches after the incomplete characters disappear, the presence of incomplete characters can be ignored.
Thereby, even when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, there is an effect that the search is performed at high speed.

  Here, the case where the compressed text compressed by the dictionary reference compression method is searched has been described. However, by combining with the configuration described in the second to ninth embodiments, the compressed text is compressed by another compression method. It is also possible to search for compressed text.

Embodiment 12 FIG.
The twelfth embodiment will be described with reference to FIGS.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, another method will be described in the case where the bit length of the bit string representing the character handled by the search apparatus is different from the bit length of the bit string representing the character assumed in the compressed text compression method.

  This compressed text search device determines whether or not there is a character string that matches the search condition in the compressed text that includes the input multibyte code character, and if it exists, the appearance position of the character string is present. If not, the search device does not output anything.

  In a text including multi-byte code characters, a character having a 1-byte code and a character having a code of 2 bytes or more exist in the text. Here, it is assumed that the character code is expressed as a hexadecimal value enclosed in parentheses, such as “(82A0)”. Hereinafter, the shift JIS code will be mainly described as an example. The character code (82A0) is “A” of the shift JIS code. Further, a partial code of a character with a code of 2 bytes or more and less than one character is referred to as byte data (an example of an incomplete character). For example, the byte data of “A” is (82) or (A0).

  FIG. 29 is a diagram showing the structure of the compressed text according to this embodiment. FIG. 29 corresponds to that described in Embodiment 1 with reference to FIG.

Generally, lexicographic compression is processed in 1-byte units, so when text containing multi-byte code characters is compressed, characters consisting of 2 bytes or more are divided into two or more entries in the compression dictionary and registered. It may end up.
When decompressing the compressed text <1, 2, 1, 3, 4, 1, 3> as shown in FIG. 29, one compressed block 1702 is acquired from the compressed block sequence 300 one by one, and the compressed block is based on the reference information. 1702 is replaced with the character string 1705 of the compression dictionary.
For example, since the first compressed block 1702 refers to the first entry of the compression dictionary, the first compressed block can be replaced with the character string “Aiue (82)” of the first entry. Similarly, the second compressed block can be replaced with “(A8) Ui”. Here, since the character code (82A8) represents “o” in the shift JIS code, the character string “aiueoui” is obtained from the first and second compressed blocks. Similarly, by repeating for all the compressed blocks, the text “Aiueouaiaiueouaiueaiueai” can be obtained from the compressed block string 300 ((82A0) = “a”, (82A2) = “ ").

FIG. 30 is a configuration diagram showing the structure of the state history stored in the state transition storage unit 112 in this embodiment. The state transition storage unit 112 has information of a reference number 1801, a state transition history 1802, and an acceptance position 1803. The reference number 1801 has a one-to-one correspondence with the reference number 304 of the compression dictionary. The state transition history 1802 stores the state transition history of the state transition machine by the character string of the corresponding compression dictionary, the state immediately before reading the character string of the compression dictionary at the beginning, and the character string of the compression dictionary at the end. The state immediately after reading everything. The state transitions once for one character, not one byte of data. For example, in the case of the first state transition history, the state is [2]-[3]-[4]-[[1] every time the character string “Aiue (82)” is read starting from the state [1]. 5]. Since the last “(82)” is byte data, the state cannot be changed.
As described above, in the first embodiment, the length of the state transition history is the character string length +1 of the reference character string of the compressed block, but the text including the multi-byte code character of this embodiment is searched. In the compressed text search device, the character string length of the reference character string is not necessarily +1, but the number of characters of the reference character string is +1. The receiving position 1803 represents the number of bytes of the reference character string of the compressed block and the state transition machine has reached the receiving state. This acceptance position may be counted in character units. At this time, even if there is 1-byte data less than one character, it may or may not be counted as one character.

FIG. 31 is a diagram illustrating an example of a state transition table stored in the state transition table storage unit 106.
The leftmost column of the state transition table 1901 in FIG. 31 represents the current state. The first line represents the next input character. The state transition table of the state transition machine that accepts the regular expression included in the search condition is generated before collation is started.

  FIG. 32 is a flowchart of search processing in the compressed text search apparatus of this embodiment. As an initial state, it is assumed that the state transition table generation unit 105 generates a state transition table from the search condition input by the search condition input unit 102 and the state transition table storage unit 106 stores the state transition table. Also, it is assumed that information about the compression dictionary is extracted from the compressed text stored in the compressed text storage unit 103 and stored in the compression dictionary storage unit 113. In addition, it is assumed that the initial state is set in the state storage unit 111. It is assumed that the state transition history and the reception position in the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

  First, in step S1501, the compressed block acquisition unit 108 acquires one compressed block in order from the beginning of the compressed block sequence. In step S1502, it is determined whether there is byte data in the byte data storage unit 122. If there is byte data (YES), the process proceeds to step S1503.

  When the text is composed of character codes to which characters are allotted and there is byte data in the byte data storage unit 122, there is byte data at the head of the reference character string of the compressed block. Therefore, in step S1503, the state transition machine 110 is based on one character having the byte data in the byte data storage unit as the upper byte, the first byte data in the reference character string as the lower byte, and the current state of the state storage unit 111. To get the next state. The acquired next state is set in the state storage unit 111. At the same time, the byte data storage unit 122 is emptied.

If there is no byte data in step S1502 (NO), the process proceeds to step S1504.
In step S1504, it is determined whether the current state of the state storage unit 111 matches the first state of the state transition history referenced by the compressed block. If they match (YES), the state at the end of the state transition history is set in the state storage unit 111 in step S1505.
Next, in step S1506, it is determined whether there is an acceptance position in the state transition history. If there is an accepted position (YES), the hit position is calculated and output in step S1507. Here, hit position = current original text length + acceptance position. If there is no receiving position in step S1506 (NO), the process proceeds to step S1508.
In step S1508, it is determined whether there is byte data at the end of the reference character string of the compressed block. If there is byte data in step S1508 (YES), the byte data is set in the byte data storage unit 122 in step S1509. If there is no byte data in step S1508 (NO), the process proceeds to step S1510.
In step S1510, it is determined whether the end of the compressed block sequence has been reached. If not (NO), the next compressed block is acquired in step S1501. If the end of the compressed block string has been reached in step S1510 (YES), the search process is terminated.
Although not clearly shown in FIG. 32, the character string length of the compression dictionary is added to the original text length here.

  Here, the hit position is output as the number of bytes from the beginning of the text, but it may be output as the number of characters. In that case, instead of counting the original text length, it is preferable to count the number of characters of the original text and record the reception position of the state transition history by the number of characters.

In step S1504, if the current state stored in the state storage unit 111 and the head state of the state transition history referred to by the compressed block do not match (NO), the process proceeds to step S1511, and the reference character string of the compressed block For this, the state transition history is obtained.
In step S1502, if the head of the reference character string is byte data, a state transition history is obtained for the character string starting from the next character of the byte data.
When the state transition history is obtained in step S1511, the process proceeds to step S1510.

FIG. 33 is a flowchart of the process of step S1511 in the search process flow of FIG. Here, it is assumed that a temporary storage area H for the state transition history is prepared and is empty as an initial state.
First, in step S1601, the current state is added to the top of the state transition history in the storage area H. In step S1602, the character acquisition unit 109 acquires characters one by one in order from the beginning of the reference character string of the compressed block.
In step S1603, the next state is acquired from the state transition machine 110 using the character acquired in step S1601 and the current state stored in the state storage unit 111 as inputs, and set in the state storage unit 111.
In step S1604, the state acquired in step S1603 is added to the state transition history in the storage area H.
In step S1605, it is determined whether the state of the state storage unit 111 is an accepting state. If it is in an accepted state (YES), the hit position is output in step S1606, and the process proceeds to step S1607. Here, the hit position is the original text length (bytes) + the number of bytes from the beginning of the reference character string of the compressed block. At this time, the number of bytes from the beginning of the reference character string is set at the reception position of the state transition history in the storage area H at the same time. If NO in step S1605 (NO), the process proceeds to step S1607.
In step S1607, it is determined whether the end of the character string has been reached. If not (NO), the process proceeds to step S1608.
In step S1608, it is determined whether or not the next character is byte data. If the next character is byte data (YES), the processing ends. If it is not byte data (NO), the next character is acquired in step S1602.
If it is determined in step S1607 that the character string has been processed up to the end (YES), the process is terminated.
When the process ends, the history of the state transition history in the storage area H and the reception position are reflected in the state transition history referred to by the compressed block.

In the process flow of FIG. 33 described above, the state transition history is reflected at the end of the process, but it is not always necessary to update it.
In other words, the state transition history may not be updated from the first acquired state transition history, or the history may be updated only when the state at the head of the state transition history is a specific state.

  If the beginning of the character string is byte data at the start of the processing in FIG. 33, the processing may be terminated as it is.

  In this embodiment, in step S1504, it is determined whether the current state stored in the state storage unit 111 matches the first state of the state transition history corresponding to the compressed block in the state transition storage unit. After obtaining the next state in step S1603 of FIG. 33 (step S1511 of FIG. 32), the state is compared with the state of the state transition history, and if they match, the process of step S1511 is terminated and the process proceeds to step S1505. It may be configured.

  An example of processing of the compressed text search apparatus according to this embodiment will be described with reference to FIGS. As an initial state, the search condition input unit 102 inputs a search condition including a regular expression, and based on the input search condition, the state transition table generation unit 105 generates a DFA state transition table 1901 that accepts the regular expression. The state transition table storage unit 106 stores this information. The acceptance state is only state [4]. Further, it is assumed that the state storage unit 111 stores an initial state [1]. It is assumed that the state transition history in the state transition storage unit 112 is empty. The byte data storage unit 122 is also empty. Now, a compressed block string 300 and a compression dictionary 1703 are obtained from the compressed text stored in the compressed text storage unit 103. The compression dictionary is stored in the compression dictionary storage unit 113. It is assumed that all characters used here are 2-byte characters.

First, in step S1501, the first compressed block is acquired from the compressed block sequence 300.
In step S1502, since there is no byte data in the byte data storage unit 122, the process advances to step S1504.
In step S1504, the state transition history referred to by the compressed block is empty for the current state [1] stored in the state storage unit 111, so the states do not match. Therefore, the process proceeds to step S1511.

In step S1601 of FIG. 33, the current state is set at the head of the first state transition history in the state transition storage unit.
In step S1602, the first character “A” of the character string “AIUE (82)” in the compression dictionary referred to by the compression block is acquired.
In step S1603, the state transition machine 110 acquires the next state [2] based on the current state [1] and the character “A”. The acquired state [2] is set in the state storage unit 111, and the process proceeds to the next step S1604.
In step S1604, the current state is added to the first state transition history. Here, the first state transition history stored in the state transition storage unit 112 is “1-2”.
In step S1605, it is determined whether the current state is an accepting state. Since the present state is not an accepting state, the process proceeds to step S1607.
In step S1607, it is determined whether or not the end of the character string in the compression dictionary has been reached. Since it has not been reached, the process proceeds to step S1608, and the next character is not byte data. In step S1602, the next character “I” is acquired.

  The same processing is performed for the second character “I”, and when step S1608 is reached, the current state is [3] and the state transition history is “1-2-3”.

Next, in step 201, the next character “U” is acquired. When the character “U” is acquired in the state [3], the next state [4] is obtained from the state transition table 1901. Therefore, immediately after execution of step S1604, the current state [4] and the state transition history “1-2-3-4” are obtained.
In step S1605, it is determined whether the current state is an acceptance state. Since the current state is an acceptance state, the process proceeds to step S1606. Since the current 2 byte characters have been acquired up to the third character, 2 bytes × 3 = 6 are output as hit positions. At the same time, 6 is added to the reception position of the first state transition history.

  Similarly, the process is repeated to determine whether the character is byte data in step S1608 of the fourth character “e”. In the case of the shift JIS code, it is possible to determine whether the first byte of the character is a 1-byte character or a 2-byte character. Since the next character (82) is byte data, the processing ends. At this point, the processing from the beginning of the text to the 9th byte is completed, the current state is [5], and the first state transition history is “1-2-3-4-5”.

Next, the process proceeds to step S1508 in FIG. In step S1508, since the last character is byte data (82), (82) is set in the byte data storage unit 122 in step S1509. Then, the process proceeds to step S1510, and since the end of the compressed block sequence has not been reached, the next compressed block is acquired in step S1501. At this point, the original text length is 9 bytes.
In step S1502, since byte data (82) is stored in the byte data storage unit 122, the process advances to step S1503.
In step S1503, the character “O” (= (82A8)) in which the byte data (82) of the byte data storage unit 122 is the upper byte, the first byte data (A8) of the reference character string is the lower byte, and the current state From [5], the next state [6] is acquired by the state transition machine 110 and set in the state storage unit 111. Then, the byte data storage unit 122 is emptied.
The same processing is performed after step S1504. When the processing reaches step S1510, the processing is completed up to the 14th byte of the text. The current state is [1], and the first state transition history is “1-2-3- 4-5 "and the second state transition history is" 6-3-1 ".

In step S1501, the third compressed block is acquired. The reference destination of the third compressed block is <1>.
In step S1502, since there is no byte data in the byte data storage unit 122, the process proceeds to step S1504.
In step S1504, the current state [1] is compared with the first state [1] of the first state transition history, and since they match, the process proceeds to step S1505.
In step S1505, the state storage unit 111 stores the state [5] at the end of the state transition history as the current state.
In step S1506, since the first state transition history has an accepted state, the hit position is calculated and output in step S1507. Since the text processed immediately before obtaining the third compressed block is 14 bytes and the accepting position of the state transition history is 6, 14 + 6 = 20 is the hit position.
In step S1508, since there is no byte data at the end of the reference character string, the process advances to step S1510.
In step S1510, since it is not the end of the compressed block, the process advances to step S1501.
Thereafter, the same process is performed, and the search process is terminated when the process is completed up to the end of the compressed block string.

As described above, according to this embodiment, when the current state of the state storage unit does not match the first state of the state transition history corresponding to the acquired compressed block, the state transition is processed. On the other hand, the number of steps proportional to the character string length of the compression dictionary referred to by the compression block is required. On the other hand, when the current state matches the first state of the state transition history, it can be processed with at most two state transitions. In a state transition machine in which the state transition destination is uniquely determined by the current state and characters, the current state is often the initial state. For this reason, the higher the compression rate, that is, the text in which a long character string repeatedly appears, the number of steps required for state transition can be reduced.
The verification itself of whether or not a character string that conforms to a regular expression exists in the text uses a state transition machine that is conventionally used and that uniquely determines a state transition that accepts a regular expression.
As described above, in the compressed text search apparatus according to this embodiment, a compressed text including multibyte code characters can be searched at high speed according to a search condition including a regular expression.

  In this embodiment, shift JIS text has been described as an example, but other multi-byte code character texts can be similarly searched. For example, Japanese EUC (Extended UNIX Code) (UNIX is a registered trademark) can determine which of the 1st to 3rd byte characters in the first byte, as in Shift JIS. Watch out for bytes. JIS has information for determining whether it is a 1-byte character or a 2-byte character. The information may be stored in the byte data storage unit 122 together with the byte data.

The compressed text search apparatus described here has the following features.
This is a search device that searches text containing multi-byte code characters compressed by a lexicographic compression method using regular expressions without decompression.
The search uses a state transition machine that can uniquely determine the state transition.
At the time of retrieval, the state transition history of the state transition machine is stored for each character string in the dictionary referenced by the compressed block, and the current state matches the first state of the state transition history referenced by the compressed block. In such a case, the state transition is performed at a stretch to the state at the end of the history.
A storage unit for storing byte data less than one character is provided. When byte data less than one character is included at the end of a character string in the dictionary, the byte data is stored, and the next compressed block is stored. When the byte data is acquired from the beginning, the state transition is processed.

Embodiment 13 FIG.
The thirteenth embodiment will be described with reference to FIGS.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, the text compressed by the LZ77 method is searched. The case where it does is demonstrated.

FIG. 34 illustrates information stored in the compression dictionary storage unit 113 and the state transition storage unit 112 according to this embodiment. The compression dictionary storage unit 113 stores the sliding window 2103. The state transition storage unit 112 stores a state transition history 2104 corresponding to the sliding window and a reception position 2105. Further, the state transition history 2104 stores the state transition for the sliding window length + 1 character. The reception position 2105 stores the position of the reception state in the state transition history 2104.
In this embodiment, the sliding window 2103 of the compression dictionary storage unit 113 also serves as a byte data storage unit.

  FIG. 35 is a flowchart of search processing in the compressed text search apparatus of this embodiment. As an initial state, it is assumed that the state transition table generation unit 105 generates a state transition table from the search conditions input by the search condition input unit 102 and the state transition table storage unit 106 stores the state transition table. In addition, it is assumed that the initial state is set in the state storage unit 111. It is assumed that the state transition history and the reception position of the compression dictionary storage unit 113 and the state transition storage unit 112 are empty. Also, a counter for counting the original text length is initialized to zero.

First, in step S2001, compressed blocks are acquired one by one in order from the beginning of the compressed block sequence.
In step S2002, it is determined whether the head of the reference character string of the compressed block is byte data. In the case of byte data, if byte data exists at the end of the compression dictionary storage unit, one character with the byte data at the end as the upper byte, the byte data at the beginning of the reference character string as the lower byte, and the current state The next state is acquired by the state transition machine 110 based on the current state and set at the end of the current state and the state transition history 2104. If there is no byte data at the end of the compression dictionary storage unit, the process may proceed to step S2008.
In step S2004, it is determined whether the current state matches the first state of the state transition history referenced by the compressed block. If the beginning of the reference character string is byte data, it is compared with the state of the next character. If the leading states match (YES), in step 2005, the character state at the position of the reference window (reference character string position + reference character string length) is set to the current state. In step S2006, it is determined whether there is an acceptance state between the position of the reference character string and the position of (reference character string position + reference character string length). If there is an acceptance state (YES), the hit position is calculated and output in step S2007, and the process proceeds to step S2008. If there is no acceptance state (NO), the process proceeds to step S2008 without doing anything.
In step S2008, the sliding window and the state transition history are updated. That is, the character string in the sliding window is shifted forward by (reference character string length + 1) bytes, and the character string corresponding to the reference character string length from the position of the reference character string and the first mismatch character are added at the end. Similarly, the state transition history is also shifted forward by (reference character string length + 1) bytes, and the state transition history from the first character of the reference character string to the character after the reference character string is added at the end. . In the state transition history, when both the end of the reference character string and the non-matching character are byte data and become one character in total, the state transition machine 110 acquires the next state, the state storage unit 111, Must be set at the end of the state transition history.
In step S2009, it is determined whether the end of the compressed block sequence has been reached. If not reached (NO), the next compressed block sequence is acquired in step S2001. If it has reached (YES), the search process is terminated.

  If the current state does not match the state of the first character of the reference character string of the compressed block of the state transition history in step S2004 (NO), the character string obtained by adding the first unmatched character to the reference character string in step S2010 For the state transition history. That is, similarly to the processing flow of FIG. 33, the next state is acquired by the state transition machine while acquiring one character at a time from the beginning of the character string obtained by adding the first mismatched character to the reference character string. When the process of step S2010 is completed, the sliding window and the state transition history are updated in step S2008.

  As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires as many times as the number of characters in the character string can be processed with up to three state transitions, and processing steps can be reduced.

  In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, a compressed text including multibyte code characters compressed in the LZ77 format can be searched at high speed according to a search condition including a regular expression.

The compressed text search apparatus described here has the following features.
The LZ77 format sliding window is stored in the compression dictionary storage unit.
In the state transition storage unit, a state transition history having a length of sliding window length + 1 is stored.
The end of the sliding window is used as a byte data storage unit.
When a compressed block in the LZ77 format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is transitioned by one state transition up to the last character of the reference character string, Further, when the end of the reference character string and the mismatched character are combined into one character, the state is changed by that character.

Embodiment 14 FIG.
The fourteenth embodiment will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, the text compressed by the LZSS method is searched. The case where it does is demonstrated.

  The flow of search processing in the compressed text search apparatus of this embodiment is the same as that described in Embodiment 13 with reference to FIG. First, consider the case where the first bit of the compressed block is 1, that is, the case where there is a reference character string in the compression dictionary. The main difference from the thirteenth embodiment is that there is no mismatched character in the compressed block. That is, the search can be performed in the same manner as in the seventh embodiment except that the mismatched characters are not taken into consideration in the processes of step S2008, step S2009, and step S2010.

  Next, consider a case where the first bit of the compressed block is 0. Here, the mismatched character in the compressed block is always 1 byte. At this time, there are the following three types of processing after step S2002. First, the mismatched character is a 1-byte character. In this case, the next state is acquired from the current state and the mismatched character, set in the state storage unit 111 in step S2005, and the process of step S2008 is executed. If the unmatched character is byte data and there is byte data at the end of the sliding window, the processes of steps S2003 and S2008 are executed. If the unmatched character is byte data and there is no byte data at the end of the sliding window, it is only necessary to execute the process of step S2008.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of steps proportional to the character string length can be processed with at most two state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, a compressed text including multibyte code characters compressed in the LZSS format can be searched at high speed according to a search condition including a regular expression.

  In addition to the LZSS format shown here, text compressed in a compression format derived from the LZ77 format, such as the LZB format and the LZBW format, can be similarly searched by the compressed text search device of this embodiment.

The compressed text search apparatus described here has the following features.
The LZSS format sliding window is stored in the compression dictionary storage unit.
In the state transition storage unit, a state transition history having a length of sliding window length + 1 is stored.
The end of the sliding window is used as a byte data storage unit.
When a compressed block in the LZSS format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is transitioned by one state transition to the last character of the reference character string.

Embodiment 15 FIG.
The fifteenth embodiment will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, the text compressed by the LZ78 method is searched. The case where it does is demonstrated.

  The state transition storage unit according to this embodiment is the same as that shown in FIG.

Since the flow of search processing in the compressed text search apparatus of this embodiment is almost the same as that described in Embodiment 12, the flow of search processing will be described with the aid of FIG. Only differences from the twelfth embodiment will be described.
When searching for text compressed in the LZ78 format, it is necessary to add an entry to the compression dictionary and the state transition history immediately before step S1508 or immediately before step S1510. First, the reference character string of the compressed block plus the unmatched character is added as a new entry in the compression dictionary.
If the non-matching character is byte data and the end of the reference character string is not byte data, or if the non-matching character and the reference character string both end with byte data and are less than one character, the compressed block The state transition history to be referred to is added as a new entry in the state transition storage unit 112 as it is.
Then, the byte data is added to the byte data storage unit 122. If both the mismatched character and the end of the reference character string are combined with byte data to be 1 character, or if the mismatched character is a 1-byte character, the next state is acquired from that character and the current state, and the acquired The state is set at the state storage unit and at the end of the state transition history. Further, if the state is an acceptance state, the hit position is output and added to the acceptance position.
In step S1504, if the current state does not match the head state of the state transition history referred to by the compressed block, the state transition history acquired in step S1511 is added to the new entry in the state transition storage unit.

  In step S1511, a state transition history is obtained for a character obtained by adding a mismatch character to the reference character string.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires as many times as the number of characters in the character string can be processed with up to three state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, a compressed text including multibyte code characters compressed in the LZ78 format can be searched at high speed according to a search condition including a regular expression.

The compressed text search apparatus described here has the following features.
A compression dictionary in the LZ78 format is stored in the compression dictionary storage unit.
When a compressed block in LZ78 format is read and the current state matches the first state of the history of state transitions referenced by the compressed block, the state is shifted by one state transition to the last character of the reference character string. . Further, when the end of the reference character string and the mismatched character are combined into one character, the state is changed by that character.
A character string composed of the reference character string and the mismatched character is added as a new entry in the compression dictionary, and the state transition is added as a new entry in the state transition storage unit.

Embodiment 16 FIG.
The sixteenth embodiment will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the tenth embodiment, description thereof is omitted here.

  In this embodiment, when the bit length of the bit string representing the character handled by the search device is different from the bit length of the bit string representing the character assumed in the compression method of the compressed text, the text compressed by the LZW method is searched. The case where it does is demonstrated.

  Since the flow of search processing in the compressed text search apparatus of this embodiment is almost the same as that of Embodiment 15, the flow of search processing will be described with the aid of FIG. The difference from the search processing of the fifteenth embodiment is that the compressed block does not include mismatched characters. In the compressed text search apparatus of this embodiment, the first character of the next compressed block is used instead of the mismatched character of the fifteenth embodiment.

As described above, according to this embodiment, after acquiring one compressed block string, the current state is compared with the state at the reference position of the state transition history, and if the state matches, the original reference is made. State transition processing that requires the number of times corresponding to the length of the character string can be processed with up to three state transitions, and processing steps can be reduced.
In addition, the regular expression / text matching process itself uses a state transition machine that accepts the regular expression. Thereby, a compressed text including multibyte code characters compressed in the LZW format can be searched at high speed according to a search condition including a regular expression.

  Similarly, a compressed text including multi-byte code characters compressed by a compression format derived from the LZ78 format can be searched at high speed by the compressed text search device of this embodiment.

The compressed text search apparatus described here has the following features.
A compression dictionary in the LZW format is stored in the compression dictionary storage unit.
When a compressed block in LZW format is read and the current state matches the first state of the state transition history referenced by the compressed block, the state is changed by one state transition to the last character of the reference character string. . Furthermore, when the end of the reference character string and the first character of the reference character string of the next compressed block are combined into one character, the state is changed by that character.
A character string composed of the reference character string and the first character of the reference character string of the next compressed block is added as a new entry in the compression dictionary, and the above state transition is added as a new entry in the state transition storage unit.

Embodiment 17. FIG.
Embodiment 17 will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another example of the contents of the state history stored in the state transition storage unit 112 (an example of a history storage unit) will be described.

  FIG. 36 is a configuration diagram showing the structure of the state history stored in the state transition storage unit 112 of the compressed text search apparatus according to this embodiment. In the compressed text search apparatus of this embodiment, the state transition storage unit 112 is configured to store a plurality of state transition histories for one entry in the compression dictionary.

One entry of the state transition storage unit of the compressed text search apparatus according to this embodiment stores an entry reference number 2201, information on a state transition history 2202, and an acceptance position 2203. Reference number 1 is an identifier that corresponds one-to-one with the reference number of the compression dictionary.
The state transition history is the state transition history of the state transition machine based on the character string of the corresponding compression dictionary. The state is the state immediately before reading the character string of the compression dictionary at the beginning and the character string of the compression dictionary at the end. The state immediately after reading everything.
The accepting position 2203 represents where in the state transition history the state transition machine has reached the accepting state.
In the compressed text search apparatus of this embodiment, a set of zero or more state transition histories 2202 and reception positions 2203 is stored in an entry of one state transition storage unit. A set of the state transition history 2202 and the receiving position 2203 is called a record.

  The compressed text search apparatus according to this embodiment, for example, adds a new entry of the state transition storage unit to which the compressed block refers to the state transition history and the reception position obtained in step S207 of FIG. 7 described in the first embodiment. Add as a record.

When only one state history is stored for one compressed block, the current state stored in the state storage unit 111 and the head of the state transition history referenced by the compressed block in the state transition storage unit 112 The number of state transitions with respect to the reference character string of the compressed block can be reduced only when the state matches.
By configuring the compressed text search device of this embodiment as described above, the current state of the state storage unit and the first state of the state transition history referenced by the compressed block of the state transition storage unit are Since it is possible to store a plurality of state transition histories when determining whether or not they match, the probability that the states match can be increased. Therefore, the probability that the number of state transitions can be further reduced is increased, and the search process can be further speeded up.

  The compressed text search apparatus described here stores a plurality of state transition histories for one entry in the compression dictionary.

Embodiment 18 FIG.
An eighteenth embodiment will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another example of the contents of the state history stored in the state transition storage unit 112 (an example of a history storage unit) will be described.

  FIG. 37 is a block diagram showing the structure of the state history stored in the state transition storage unit 112 of the compressed text search apparatus according to this embodiment. In the compressed text search device of this embodiment, the state transition storage unit 112 is configured to store a plurality of state transition histories for the sliding window.

  The state transition storage unit of the compressed text search apparatus according to this embodiment includes a plurality of sets of records each composed of a set of a state transition history 2304 and a reception position 2305 for storing states corresponding to (sliding window length + 1).

  A flow of search processing of the compressed text search apparatus of this embodiment will be described.

When only one state history is stored for one compressed block, the current state stored in the state storage unit 111 and the head of the state transition history referenced by the compressed block in the state transition storage unit 112 The number of state transitions with respect to the reference character string of the compressed block can be reduced only when the state matches.
By configuring the compressed text search device of this embodiment as described above, the current state of the state storage unit and the first state of the state transition history referenced by the compressed block of the state transition storage unit are Since it is possible to store a plurality of state transition histories when determining whether or not they match, the probability that the states match can be increased. Therefore, the probability that the number of state transitions can be further reduced is increased, and the search process can be further speeded up.

  The compressed text search device described here stores a plurality of state transition histories for one character string in a dictionary with respect to a sliding window in a compressed format derived from the LZ77 format and the LZSS format. And

Embodiment 19. FIG.
The nineteenth embodiment will be described with reference to FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In this embodiment, another example of the contents of the state history stored in the state transition storage unit 112 (an example of a history storage unit) will be described.

  FIG. 38 is a block diagram showing the structure of the state history stored in the state transition storage unit 112 of the compressed text search apparatus according to this embodiment. In the compressed text search apparatus of this embodiment, the state transition storage unit 112 is configured to store only the head and tail states of the state transition history.

  The state transition storage unit of the compressed text search apparatus of this embodiment stores a reference number 2401, a head state 2402 of the state transition history, a state 2403 at the end of the state transition history, and an acceptance position 2404. The compressed text search apparatus of this embodiment sets, for example, only the head and tail states of the state transition history obtained in step S207 of FIG. 7 described in the first embodiment and the reception position in the state transition storage unit. .

When the state transition storage unit 112 stores all state transition history, a storage area is required according to the number of entries in the compression dictionary and the length of the character string in the compression dictionary. For this reason, if the number of entries in the compression dictionary or the length of the character string in the compression dictionary increases, the storage capacity of the memory or the like may be reduced.
In the compressed text retrieval apparatus of this embodiment, only the head state and tail state of the state transition and the receiving position need be known.
By configuring the compressed text search device of this embodiment as described above, the size of the storage area required by the state transition storage unit does not depend on the length of the character string in the compression dictionary, and is compressed. It can be suppressed by a constant multiple of the number of dictionary entries.

  The compressed text search apparatus described here stores only the head state and the tail state of the history when storing the state transition history for the character string in the dictionary.

Embodiment 20. FIG.
Since the appearance, hardware configuration, and block configuration of the compressed text search apparatus 100 (an example of a character string search apparatus) in this embodiment are the same as those described in the first embodiment, description thereof is omitted here.

  In the compressed text search device of this embodiment, the state transition storage unit 112 sets the state transition history only when the length of the reference character string of the compressed block is equal to or longer than a predetermined length. Is.

  For example, taking FIG. 4 as an example, if the state transition history is stored only when the length of the reference character string is 5 or more, only the first state transition is stored.

  The compressed text search apparatus of this embodiment, when the current state stored in the state storage unit 111 matches the first state of the state transition history referenced by the compressed block of the state transition storage unit 112, The state transition for the reference character string of the compressed block can be processed in one step. At this time, the number of processing steps that can be reduced increases as the length of the character string referred to by the compressed block increases. That is, when the reference character string of the compressed block is short, the effect of reducing the number of processing steps is small.

  Since the compressed text search apparatus of this embodiment is configured as described above, it is not necessary to store a state transition history corresponding to an entry having a short character string in the compression dictionary. The storage area can be reduced.

  The compressed text search device described here stores the state transition history only for character strings longer than a predetermined length when storing the state transition history for the character strings in the dictionary. It is characterized by that.

  The configuration of the state history stored in the state transition storage unit 112 described in the seventeenth to twentieth embodiments may be combined.

FIG. 3 is a diagram illustrating an example of an appearance of a compressed text search device 100 (an example of a character string search device) in the first embodiment. 2 is a diagram illustrating an example of a hardware configuration of a compressed text search device according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an example of a block configuration of the compressed text search apparatus 100 according to the first embodiment. FIG. 4 is a diagram showing an example of compressed text stored in a compressed text storage unit 103 in the first embodiment. In Embodiment 1, the figure which shows the structure of the state history which the state transition memory | storage part 112 memorize | stores. FIG. 6 is a diagram showing an example of a state transition table 200 stored in the state transition table storage unit 106 in the first embodiment. FIG. 3 is a flowchart showing an example of a control flow of search processing of the compressed text search apparatus 100 according to the first embodiment. The flowchart figure which shows an example of the detail of a process in S16 of FIG. FIG. 10 is a diagram illustrating an example of storage contents stored in a compressed text storage unit 103 and a state transition storage unit 112 in the second embodiment. The flowchart figure which shows an example of the flow of search processing of the compressed text search device 100 in Embodiment 2. The figure which shows the structure of the compression text by LZ77 format. The figure which shows the information which the compression dictionary memory | storage part 113 (an example of a dictionary memory | storage part) and the state transition memory | storage part 112 (an example of a history memory | storage part) memorize | store in Embodiment 3. FIG. 10 is a flowchart of search processing in the compressed text search apparatus 100 according to the third embodiment. In Embodiment 4, it is a figure which shows an example of the memory content of the compressed text memory | storage part 103 and the state transition memory | storage part 112. FIG. The flowchart figure which shows an example of the flow of control of the search process of the compressed text search device 100 in Embodiment 4. The figure which shows the structure of the compression text by a LZSS format. In Embodiment 6, it is a figure which shows an example of the memory content which the compressed text memory | storage part 103, the compression dictionary memory | storage part 113 (an example of a dictionary memory | storage part), and the state transition memory | storage part 112 (an example of a history memory | storage part) memorize | store. FIG. 20 is a flowchart showing an example of a control flow of search processing of the compressed text search device 100 according to Embodiment 6. The figure which shows the structure of the compression text by LZ78 format. 18 is a flowchart of search processing in the compressed text search device according to the seventh embodiment. In Embodiment 8, it is a figure which shows an example of the memory content which the compressed text memory | storage part 103 and the compression dictionary memory | storage part 113 (an example of a dictionary memory | storage part) memorize | store. In Embodiment 8, it is a figure which shows an example of the memory content which the state transition memory | storage part 112 (an example of a log | history memory | storage part) memorize | stores. FIG. 19 is a flowchart showing an example of the flow of search processing performed by the compressed text search apparatus 100 according to the eighth embodiment. The figure which shows the structure of the compression text by a LZW format. Explanatory drawing for demonstrating the case where the bit length of the bit string expressing the character handled in compression technology differs from the bit length of the bit string expressing the character handled in the search device. FIG. 25 is a block diagram showing an example of a block configuration of the compressed text search device 100 according to the tenth embodiment. In Embodiment 10, the figure which shows an example of the memory content which the compressed text memory | storage part 103, the compression dictionary memory | storage part 113 (an example of a dictionary memory | storage part), and the state transition memory | storage part 112 (an example of a history memory | storage part) memorize | store. In Embodiment 11, it is a figure which shows an example of the memory content which the compressed text memory | storage part 103, the compression dictionary memory | storage part 113 (an example of a dictionary memory | storage part), and the state transition memory | storage part 112 (an example of a history memory | storage part) memorize | store. FIG. 19 shows a structure of compressed text according to the twelfth embodiment. In Embodiment 12, it is a block diagram which shows the structure of the state history which the state transition memory | storage part 112 memorize | stores. The figure which shows an example of the state transition table which the state transition table memory | storage part 106 memorize | stores. 19 is a flowchart of search processing in the compressed text search device according to the twelfth embodiment. The flowchart of the process of step S1511 in the flow of the search process of FIG. The figure which shows the information which the compression dictionary memory | storage part 113 and the state transition memory | storage part 112 memorize | store according to Embodiment 13. FIG. 18 is a flowchart of search processing in the compressed text search device according to the thirteenth embodiment. The block diagram which shows the structure of the state history which the state transition memory | storage part 112 of the compressed text search device by Embodiment 17 memorize | stores. The block diagram which shows the structure of the state history which the state transition memory | storage part 112 memorize | stores in the compressed text search device by Embodiment 18. FIG. The block diagram which shows the structure of the state history which the state transition memory | storage part 112 memorize | stores in the compressed text search device by Embodiment 19. FIG. The conceptual diagram which shows an example of the state transition in operation | movement of DFA. Explanatory drawing about the transition of the state of DFA. The figure which shows an example of the state transition table which an automaton execution part memorize | stores in order to perform DFA. The flowchart figure which shows an example of the flow of a process of an automaton execution part. The conceptual diagram which shows an example of DFA. The figure which shows an example of the encoding in a LZSS system. The flowchart figure which shows an example of the flow of control in the case of decompress | restoring the original character string from a code string in a prior art example. The figure which shows an example of the encoding in a LZ77 system. The figure which shows an example of the encoding in a dictionary reference type compression system. The figure which shows an example of the encoding in a LZ78 system. The flowchart figure which shows an example of the flow of control in the case of decompress | restoring the original character string from a code string in a prior art example. The figure which shows an example of the encoding in a LZW system.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Compressed text search device, 102 Search condition input part, 103 Compressed text memory | storage part, 104 Collation result output part, 105 State transition table production | generation part, 106 State transition table memory | storage part, 107 Collation part, 108 Compressed block acquisition part, 109 characters Acquisition unit, 110 State transition machine, 111 State storage unit, 112 State transition storage unit, 113 Compression dictionary storage unit, 114 Condition determination unit, 115 Current position counter, 116 Transition destination calculation unit, 117 Search success determination unit, 121 Incomplete character Restoration unit, 122 byte data storage unit, 200 state transition table, 300 compressed block string, 500 original character string, 600 code string, 650 replacement dictionary, 901 CRT display device, 902 K / B, 903 mouse, 904 FDD, 905 CDD, 906 printer device, 907 scanner device, 10 system unit, 911 CPU, 912 bus, 913 ROM, 914 RAM, 915 communication board, 920 magnetic disk unit, 921 OS, 922 window system, 923 program group, 924 file group, 931 telephone, 932 FAX machine, 940 Internet 941 Gateway, 942 LAN.

Claims (15)

An automaton that holds a state, inputs a character, calculates a transition destination state based on the held state and the input character, and updates the held state to the calculated transition destination state. Whether or not a search character string corresponding to a predetermined search pattern is included in the character string by determining whether or not the stored state is a predetermined state when characters constituting the character string are input By running an automaton configured to determine whether or not
In a character string search device for acquiring a code string obtained by replacing a partial character string included in the character string with a predetermined code corresponding to the partial character string, and searching the search character string from the character string,
An automaton execution unit for executing the automaton,
A history storage unit that stores the state held by the automaton as a state history;
A code acquisition unit for acquiring a code constituting the code string;
A condition determination unit that determines whether the first condition and the second condition are satisfied based on the state held by the automaton, the state history stored in the history storage unit, and the code acquired by the code acquisition unit When,
When the condition determination unit determines that the first condition is satisfied, the transition destination state is calculated based on the state history stored in the history storage unit, and the state held by the automaton is changed to the calculated transition destination state. A transition destination calculation unit to be updated;
When the condition determining unit determines that the second condition is satisfied, the character that restores the partial character string corresponding to the code acquired by the code acquiring unit and inputs the characters constituting the partial character string to the automaton A column restoration unit;
A character string search device characterized by comprising: The automaton execution part
The character string search device according to claim 1, wherein an automaton capable of uniquely calculating the transition destination state is executed. The automaton execution part
The character string search device according to claim 1, wherein the automaton is configured to search for a search character string corresponding to a search pattern expressing at least one of concatenation, selection, and repetition of characters. When the partial character string included in the character string matches another partial character string included in the character string, the character string search device is configured to specify the partial character string as a pointer to the other partial character string. A character string search device for acquiring a code string replaced with a code including
The character string restoration part
When it is determined that the code acquired by the code acquisition unit includes information on a pointer to the other partial character string, the other partial character string is restored as a partial character string corresponding to the code,
When it is determined that the code acquired by the code acquisition unit does not include pointer information to the other partial character string, the character corresponding to the code is restored as a character string corresponding to the code. The character string search device according to claim 1. The character string search device further includes:
A dictionary storage unit for storing a correspondence relationship between a partial character string and a code corresponding to the partial character string as a replacement dictionary;
The character string restoration part
2. The character string search device according to claim 1, wherein a partial character string corresponding to the code is obtained based on a replacement dictionary stored in the dictionary storage unit. The code acquisition unit
From the code string including the information of the replacement dictionary, obtain a code constituting the code string,
The dictionary storage unit
The character string search device according to claim 5, wherein information of the replacement dictionary is acquired from the code acquired by the code acquisition unit and stored. The condition determination unit
About the code acquired by the code acquisition unit, the state held by the automaton before inputting the characters constituting the partial character string corresponding to the code to the automaton, from the state history stored by the history storage unit If the acquired state and the state held by the automaton are compared and determined to match, it is determined that the first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determination unit determines that the first condition is satisfied, the characters constituting the partial character string corresponding to the code acquired by the code acquisition unit based on the state history stored by the history storage unit are described above. Obtaining the state updated by the automaton after input to the automaton and making it the transition destination state, updating the state held by the automaton to the transition destination state,
The history storage unit
When the condition determination unit determines that the second condition is satisfied, the state held by the automaton is stored as a state history,
When the condition determining unit determines that the first condition is satisfied, the automaton holds the character constituting the partial character string corresponding to the code acquired by the code acquiring unit when the character is input to the automaton. The character string search device according to claim 1, wherein the state obtained from the state history is stored as a state history. The condition determination unit
About the code acquired by the code acquisition unit, the state stored in the automaton before inputting the characters constituting the partial character string corresponding to the code to the automaton, from the state history stored in the history storage unit If the acquired state and the state held by the automaton are compared and determined to match, it is determined that the first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determining unit determines that the first condition is satisfied, a state in which the automaton is updated after inputting characters constituting the partial character string corresponding to the code acquired by the code acquiring unit to the automaton, Obtained from the state history stored in the history storage unit as a transition destination state, update the state stored by the automaton to the transition destination state,
The history storage unit
When the condition determination unit determines that the second condition is satisfied, the code acquired by the code acquisition unit and the state held by the automaton are associated and stored as a state history. The character string search device according to 1. The character string search device further includes:
The correspondence relationship between the partial character string and the code corresponding to the partial character string is stored as a replacement dictionary, and for the partial character string composed of the forward character string and the backward character string, the code corresponding to the forward character string and the above A dictionary storage unit for further storing a correspondence relationship between a backward character string and a code corresponding to the partial character string as a replacement dictionary;
In the case where the condition determining unit determines that the second condition is satisfied,
When the dictionary storage unit stores the partial character string corresponding to the code, obtain the partial character string, input the characters constituting the partial character string to the automaton,
When the dictionary storage unit stores a code and a back character string corresponding to the front character string, the code and a back character string corresponding to the front character string are obtained, and the back character string is obtained. Is input to the above automaton,
The code acquisition unit further includes:
The character string search device according to claim 8, wherein a code corresponding to the forward character string obtained by the character string restoration unit is acquired as the code. The character string search device obtains a code string obtained by replacing a partial character string composed of characters having a bit length different from the bit length of the character input to the automaton with a code corresponding to the partial character string. A search device,
An incomplete character storage unit for storing incomplete characters;
The character string restoration part
In the restored partial character string, it is determined whether or not there is a character that cannot be input to the automaton due to the mismatch of the bit length, and when it is determined, the uncompleted character is stored in the incomplete character storage unit. The character string search device according to claim 1, wherein: The condition determination unit
Regarding the code acquired by the code acquisition unit, the state of the automaton held before the characters constituting the partial character string corresponding to the code are input to the automaton and the incompleteness stored by the incomplete character storage unit are stored. The character is acquired from the state history stored in the history storage unit, the acquired state is compared with the state held by the automaton, and the acquired incomplete character and the incomplete character stored in the incomplete character storage unit are obtained. If both are determined to match, it is determined that the first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determination unit determines that the first condition is satisfied, the characters constituting the partial character string corresponding to the code acquired by the code acquisition unit based on the state history stored by the history storage unit are described above. Obtaining the state updated by the automaton after input to the automaton and making it the transition destination state, updating the state held by the automaton to the transition destination state,
The history storage unit
When the condition determining unit determines that the second condition is satisfied, the state held by the automaton and the incomplete characters stored by the incomplete character storage unit are stored as a state history,
When the condition determining unit determines that the first condition is satisfied, the automaton holds the character constituting the partial character string corresponding to the code acquired by the code acquiring unit when the character is input to the automaton. The character string search device according to claim 10, wherein an incomplete character stored in the state and the incomplete character storage unit is acquired from the state history and stored as a state history. The condition determination unit
Regarding the code acquired by the code acquisition unit, the state of the automaton held before the characters constituting the partial character string corresponding to the code are input to the automaton and the incompleteness stored by the incomplete character storage unit are stored. The character is acquired from the state history stored in the history storage unit, the acquired state is compared with the state held by the automaton, and the acquired incomplete character and the incomplete character stored in the incomplete character storage unit are obtained. If both are determined to match, it is determined that the first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determining unit determines that the first condition is satisfied, a state in which the automaton is updated after inputting characters constituting the partial character string corresponding to the code acquired by the code acquiring unit to the automaton, Obtained from the state history stored in the history storage unit as a transition destination state, update the state stored by the automaton to the transition destination state,
The history storage unit
When the condition determining unit determines that the second condition is satisfied, the code acquired by the code acquiring unit is associated with the state held by the automaton and the incomplete character stored by the incomplete character storage unit The character string search device according to claim 10, wherein the character string search device is stored as a history. The character string search device further includes:
When the incomplete character storage unit stores incomplete characters, a portion of the partial character string corresponding to the code acquired by the code acquisition unit that becomes a character that can be input to the automaton by combining with the incomplete character The character string search device according to claim 10, further comprising: an uncompleted character restoration unit that restores another unfinished character and inputs a character obtained by combining the unfinished character and the other unfinished character to the automaton. . The condition determination unit
When the incomplete character restoration unit does not input a character to the automaton, the automaton holds the code acquired by the code acquisition unit before inputting the characters constituting the partial character string corresponding to the code to the automaton. When the first condition is satisfied when the acquired state is acquired from the state history stored in the history storage unit and the acquired state and the state held by the automaton are compared and determined to match. Judgment
When the incomplete character restoration unit inputs a character to the automaton, for the code acquired by the code acquisition unit, other characters restored by the incomplete character restoration unit among the characters constituting the partial character string corresponding to the code The state that the automaton holds before inputting the part excluding incomplete characters to the automaton is acquired from the state history stored in the history storage unit, and the acquired state and the state held by the automaton are obtained. If it is determined that they match, it is determined that the first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determination unit determines that the first condition is satisfied, the characters constituting the partial character string corresponding to the code acquired by the code acquisition unit based on the state history stored by the history storage unit are described above. Obtaining the state updated by the automaton after input to the automaton and making it the transition destination state, updating the state held by the automaton to the transition destination state,
The history storage unit
When the condition determining unit determines that the second condition is satisfied, the state held by the automaton and the incomplete characters stored by the incomplete character storage unit are stored as a state history,
When the condition determining unit determines that the first condition is satisfied, the automaton holds the character constituting the partial character string corresponding to the code acquired by the code acquiring unit when the character is input to the automaton. The character string search device according to claim 13, wherein the incomplete character stored in the incomplete character storage unit is acquired from the state history and stored as a state history. The condition determination unit
When the incomplete character restoration unit does not input a character to the automaton, the automaton holds the code acquired by the code acquisition unit before inputting the characters constituting the partial character string corresponding to the code to the automaton. When the first condition is satisfied when the acquired state is acquired from the state history stored in the history storage unit and the acquired state and the state held by the automaton are compared and determined to match. Judgment
When the incomplete character restoration unit inputs a character to the automaton, for the code acquired by the code acquisition unit, other characters restored by the incomplete character restoration unit among the characters constituting the partial character string corresponding to the code The state held by the automaton before inputting the part excluding incomplete characters into the automaton is acquired from the state history stored in the history storage unit, and the acquired state is compared with the state held by the automaton. And the above first condition is satisfied,
When it is determined that the first condition is not satisfied, it is determined that the second condition is satisfied,
The transition destination calculation unit
When the condition determining unit determines that the first condition is satisfied, a state in which the automaton is updated after inputting characters constituting the partial character string corresponding to the code acquired by the code acquiring unit to the automaton, Obtained from the state history stored in the history storage unit as a transition destination state, update the state stored by the automaton to the transition destination state,
The history storage unit
When the condition determining unit determines that the second condition is satisfied, the code acquired by the code acquiring unit is associated with the state held by the automaton and the incomplete character stored by the incomplete character storage unit The character string search device according to claim 13, wherein the character string search device is stored as a history.

Images