JPH0746362B2 - String matching method - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a relatively long character string called text.
A relatively short character string called a pattern given separately,
The present invention relates to a character string collating method for determining whether or not a substring exists and, if it exists, extracting all existing patterns.

(Prior Art) In the field of data processing systems, a collection of character string data such as sentences is searched for only those that include a specific sub-character string called a pattern, or all data included in character string data is searched. Often it is necessary to extract the pattern.

Usually, one character is expressed by a fixed-length code of n bits, so the character string data is a series of codes in n-bit units.

Generally, character string data is stored in an external storage device of an electronic computer such as a magnetic disk, and is serially transferred character by character to a central processing unit at the time of retrieval. Therefore, in order to reduce the processing time, it is necessary to perform the search at the same time as the data transfer.

FIG. 3 is an explanatory view of a conventional character string collating mechanism, 1 is a storage device in which character string data is stored, 2 is a character string collating device for collating character strings, 3 is a character string data transfer path, 4 Is a signal line for outputting a search result.

The character string data is serially input character by character from the storage device 1 to the character string collating device 2 via the data transfer path 3.

The character string collation device 2 collates a partial character string that is a pattern stored in advance with the input data, and outputs a coincidence signal to the signal line 4 when a coincidence between the two is detected.

As a method for collating character strings in the character string collating device 2, a method using a finite automan has been generally known.

(LAHollaar “Hardwarae ystem for Text Informatio
n Retrieval "ACM SIGIR 6th Conference 1983) FIG. 4 is an explanatory diagram showing the state transition of a finite automaton.

In Fig.4, 5 is the state of the automaton, 6 is the direction of the state transition, and is called "DOG" in the character string data.
You can check the pattern of characters.

This operation will be described below.

The initial state of the automaton is state (0), and when the input character is "D", it transits to state (1).

In FIG. 2, "#" represents another character, and if the input character in the state (0) is other than "D", it remains in the state (0).

The same applies to the state (1). If the input character is "O", the state is changed to the state (2). If the input character is "D", the state is changed to the state (1). Otherwise, the state is changed to the state (0).

If the input character is "G" in the state (2), the state transits to the state (3), the key (DOG) is detected, and the coincidence signal is output from the signal line 4 in FIG.

FIG. 5 is an explanatory diagram of a circuit configuration of a conventional deterministic finite automaton for character string data represented by 8-bit JIS code.

In FIG. 5, 3 string data transfer channel, 4 denotes a signal line for outputting a detection result, 7 16-bit address shea register, random access memory 64KB respectively 8 and 20 (256 × 2 ⁸ B) (Abbreviated as RAM), 9 is an address decoder, 1
0 and 21 are 8-bit memory registers, 12, 14, 24,
Reference numeral 25 is an 8-bit width data line, 13 is a 16-bit width address line, 22 is a detection pattern discrimination circuit, and 23 is an address decoder output line.

FIG. 6 is a block diagram of a conventional state transition table. In the case of the patterns “ABC” and “BCD”, the random access of FIG.
16 shows an example of the contents of a conventional state transition table stored in the memory 8, 16 is 8-bit data in hexadecimal notation and represents the next state number.

17 is the upper 8 bits of the memory address, and 18 is the lower 8 bits of the memory address. Logically, the upper address 17 of the memory corresponds to the state number, the lower address 18 of the memory corresponds to the character code, and 19 is the lower address 18 of the memory.
It is a character corresponding to the code expressed by the code.

FIG. 7 is a configuration diagram of a conventional pattern detection determination table. Similarly, when the patterns are “ABC” and “BCD”, the conventional pattern detection determination table stored in the random access memory 20 of FIG. 5 is shown. 26 represents 8-bit data and represents a detection pattern code. 27 is the upper 8 bits of the memory address, 28 is the lower 8 bits of the memory address,
The configuration is the same as the state transition table of FIG.

The input character of the text is set in the lower 8 bits of the address register 7 through the data transfer path 3. All zeros are set in the upper 8 bits of the address register 7 as an initial value and input to the address decoder 9 via the address line 3.

The output of the address decoder 9 is distributed in two directions by the output line 23, and the 8-bit data 16 stored at the address is read from the random access memory 8 and set in the memory register 10 via the data line 14. At the same time, the 8-bit data 26 stored at the same address is read from the random access memory 20 and set in the memory register 21 via the data line 24.

The contents of memory register 10 will contain the contents of the next input character of the text when the lower bit of address register 7 is set.
It is set in the upper bits of the address register 7 via the data line 12.

The content of the memory register 21 is discriminated by the data line 25.
If the value is added to 22, and the value is not all zero, the detection signal and the contents of the memory register 21 are output to the signal line 4.

By repeating the above operation every time one character is input from the data transfer path 3, the collation process of the text and the pattern is executed.

In the conventional method described above, in each state, in order to determine the next state by one table pulling operation for each input character, that is, to make a deterministic finite automaton, all character codes are supported. Needed an entry to.

In the example above, there are 2 ⁸ = 256 entries for each state.

If the character is represented by a 16-bit code, 2 ¹⁶ = 6
There are 5536 entries. However, as can be seen from the above-mentioned explanation example, among the 256 entries, significant transition information other than the initial state is entered in only a few entries. Transition information to the initial state is entered in the other entries.

As described above, the conventional method has a drawback that the RAM for storing the state transition table and the pattern detection judgment table has a capacity larger than that required for storing significant information.

(Object of the Invention) It is an object of the present invention to provide a character string collating method using a finite automaton, which solves the problem that the memory for storing the state transition table and the pattern detection judgment table becomes unnecessarily large.

(Structure of the Invention) (Differences between Features of the Invention and Conventional Techniques) The present invention relates to an associative memory (address built-in memory; Con) as a memory for storing a state transition table and a detection pattern determination table.
It is also called tent Address Memory (abbreviated as CAM)].

In the state transition table, an entry whose next state number is other than the initial one, or an entry whose detection pattern determination table has a content other than all zeros is called a significant entry, and only the combination of the state number and character code corresponding to this entry is called. The data is stored in the CAM, and the corresponding next state number and pattern detection code are stored in the corresponding addresses in the RAM associated with the CAM.

Every time one character is input from the text, the CAM is searched for a match as a set of the current state number and the input character.

If there is a matching entry, the next state number and pattern detection code can be obtained from the RAM associated with the entry.

If they do not match, the next state number is found to be the initial value, and a separate circuit creates the initial state number.

In the conventional method, since only the random access memory is used, it is necessary to store not only significant entries, but the present invention is different in that only significant entries need to be stored.

(Embodiment) FIG. 1 is a block diagram of an embodiment showing a configuration of an implementation circuit of a deterministic finite automaton using the present invention, which is a comparison method for character string data represented by an 8-bit code. FIG.

In the figure, 30 is a 1K word (1 word is 2B + 1 bit) CAM, 3
1 is an encoder that creates the address of the matching entry of CAM, 32 is an initial state generation circuit that creates an initial value as the next state number when there is no matching entry in CAM, 33 is CA
CAM access register for setting data for matching search for M, 34 for 8-bit data line selection circuit, 35 for
17-bit data line, 36 is a match entry signal line corresponding to an entry in which a match entry of CAM is displayed, 37 is an address line of a match entry of CAM, 38 is a mismatch entry signal indicating that there is no match entry in CAM Line, 40 is 1K word (1 word is 2B) RAM, 41 is address decoder, 42 is memory register, 43 is address decoder output line, 44 is 16-bit data line, 45, 46, 47 are 8-bit data Line, 301 is an entry validity indication bit storage area that stores 1-bit data indicating a valid entry of CAM, 302 is an 8-bit status number storage area, 303 is an 8-bit character code storage area, and 401 and 402 are 8-bit In the data storage area of, 401 is the next state number storage area, and 402 is the pattern detection code storage area.

FIG. 2 is a configuration diagram of the state transition table and the pattern detection judgment table corresponding to FIG. 1, and the given patterns are “ABC” and “BCD”.
2 shows an example of the contents of CAM and RAM in the case of.

In the figure, 311,312,313,411,412 are 301,302,3 respectively.
These are the contents stored in 03,401,402.

311 is 1-bit data, and when it is "1", it indicates that the entry is valid.

312 is 8-bit data in hexadecimal notation and represents the current state number.

313 is hexadecimal representation 8-bit data and represents a character code.

411 is 8-bit data in hexadecimal notation and represents the next state number.

Reference numeral 412 is 8-bit data expressed in 16 trial numbers and represents a pattern detection code.

323 is a character corresponding to the code expressed by the code of 313.

Next, the correspondence between FIG. 2 and FIGS. 6 and 7 of the conventional method will be described.

The current state number 312 corresponds to 17 in FIG. 6 and 27 in FIG.

The character code 313 corresponds to 18 in FIG. 6 and 28 in FIG. 7, but in the case of FIG. 2, only the character code in the pattern is used.

The next state number 411 corresponds to 16 in FIG. 6, but in the case of FIG. 2, the initial state number (00) is included only when the pattern detection code 412 is other than (00). .

The pattern detection code 412 corresponds to 26 in FIG.

Next, the operation of FIG. 1 will be described.

Input characters from the data transfer path 3 to CAM access register 33
Is set to the lower 8 bits of.

(00) is set as an initial value in the middle 8 bits of the CAM access register 33.
The upper 1 bit of is always set to "1".

The 17 bits of the CAM access register are searched for a match with the CAM via the data line 35.

If there is a matching entry, the address of the word entry is stored in RAM via the data line 37 by the address encoder.
Address decoder 41.

The contents 411 and 412 of RAM40 are decoded by the address decoder and the contents corresponding to the matching entry of CAM are transferred to the data line.
It is set in the memory register 42 via 44.

In the case of the present invention, it is clear from the contents of the CAM that it does not match multiple entries in the CAM.

If there is no matching entry, it is determined that the signal lines 36 are all zero, and a mismatch signal is output from the mismatch entry signal line 38.

When the disagreement signal is applied to the initial state generation circuit 32, (00) is created as the next state number and transferred to the selection circuit 47 via the data line 46.

If there is a matching entry, the upper 8 bits of the memory register 42
The bit is transferred to the selection circuit 34 via the data line 45 as the next state number.

The lower 8 bits of the memory register 42 are added as a pattern search code to the discrimination circuit 22 via the data line 25. If the value is other than (00), the detection signal and the data line 25 are transferred to the signal line 4. The output contents are output.

The selection circuit 34 selects the data line 45 when there is a matching entry, and selects the data line 46 when there is no matching entry and outputs it to the data line 47.

The data sent via the data line 47 is CA as the next status number.
It is set to the middle 8 bits of the M access register 33.

By repeating the above operation every time one character is input from the data transfer path 3, the collation processing of the text pattern is executed.

As is clear from the above description, only the significant entries of the state transition table and the detected pattern judgment table need to be stored in the CAM, so the memory capacity can be reduced.

The transition information to the initial state other than the significant entries is created by determining that there is no entry that matches the CAM.

Although (00) is used as the initial state number in the above description, any number can be used.

It is also clear that the character code is not limited to 8 bits.

If there is a matching entry in CAM,
A CAM having a function of outputting the contents of the entry instead of the address of the entry is also known.

When such a CAM is used, the present invention can be implemented by storing the information stored in the RAM in the above-mentioned embodiment in another storage area of the same entry of the CAM.

(Effects of the Invention) As described above, the present invention has the following advantages because only the significant entries of the state transition table and the pattern detection judgment table need be stored in the memory.

The upper limit of the total length of patterns that can be stored in a fixed capacity memory becomes large.

The time required to store the state transition table and the pattern detection judgment table of patterns with a fixed capacity in the memory is shortened.

The above advantages are significantly different from the conventional method especially when the character code length is long.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment showing the configuration of an expression circuit of a deterministic finite automaton using the present invention, and FIG.
Configuration diagram of state transition table and pattern detection judgment table corresponding to the figure,
FIG. 3 is an explanatory diagram of a conventional character string collating mechanism, FIG. 4 is an explanatory diagram showing state transition of a finite state automaton, and FIG.
Fig. 6 is an explanatory diagram of a circuit configuration for realizing a conventional deterministic finite automaton for character string data expressed by JIS code of bits, Fig. 6 is a block diagram of a conventional state transition table, and Fig. 7 is a conventional pattern detection judgment. It is a block diagram of a table. 1 ... storage device, 2 ... character string collating device, 3 ... data transfer path, 4 ... signal line, 5 ... state of automaton, 6 ... direction of state transition, 7 ... address register, 8, 20,40 …… Random access memory, 9,41 …… Address decoder, 10,21,42 …… Memory register, 12,14,24,25,35,44,45,46,47 …… Data line, 13,37 …… Address line, 16,26 …… Data, 17,27 …… Memory upper address, 18,28 …… Memory lower address, 19,29,323 …… Code-corresponding characters, 22 …… Judgment circuit , 23,43 …… Address decoder output line, 30 …… Associative memory (CAM), 31 …… Address encoder, 32 …… Initial state generation circuit, 33 …… CAM access register, 34 …… Selection circuit, 36 …… CAM match entry signal line, 37 ... CAM match entry address line, 38 ... CAM non-match entry signal line, 301 ... Entry exists Effectiveness indication bit storage area, 302 ... Status number storage area, 303 ... Character code storage area, 311 ... Entry validity indication bit, 312 ... Status number, 313 ... Character code, 401 ... Next status Number storage area, 402 ... Pattern detection code storage area, 411 ... Next state number, 412 ... Pattern detection code.

Claims (1)

[Claims] 1. A state number and a pattern are configured to determine whether or not a relatively short character string called a pattern exists as a substring in a relatively long character string called text. For the combination with each character, the state transition table and the state number in which the following state numbers are entered, and especially in the combination of the final character of the pattern, a pattern detection judgment table in which the pattern detection code is entered is created in advance. Every time one character of the text is input, the state transition according to the above state transition table is performed, and in the text search method based on the deterministic finite automaton for determining the existence of the pattern by referring to the pattern detection determination table, the state transition The associative memory stores the combination of the state numbers in the table and the characters forming the pattern, and the next state number, pattern detection code, and the associative memory corresponding to the combination. It is stored in the corresponding address of the associated Radam access memory, and each time one character of the text is input, the current state number and the input one character are searched for a match in the associative memory, If there is a matching entry, the next state number is obtained from the associated random access, and if there is no matching entry, the next state number is reset to the initial state, and the state entry is changed to the matching entry. A character string collating method, wherein the presence of a pattern is determined by the pattern detection code obtained from the associated random access memory.