JPS60105040A - Sentence retrieving system - Google Patents

Abstract

PURPOSE:To attain the sentence retrieval by means of a state shift table in a small size even in case a character code is long in bit length by using partial patterns obtained by dividing the character code into units equal to 2 pcs of power multiplier. CONSTITUTION:The input characters are set to low-order 8 bits of an address register 7 every 8 bits and alternately in order of high-order and low-order places from a data register 20 by a switch circuit 21. The upper 8 bits having initial values all set at 0 are supplied to an address decoder 9, and the 8-bit data is read out of an address 0 of a random access memory 8 and stored to a memory register 10. A discrimination circuit 22 neglects the contents 10 of the register 10 with a truth value action in case the high-order 8 bits of the register 7 are equal to the special value FF. Then the high-order 8 bits of the register 7 are forcibly reset ''0'' to prevent the coincidence between code patterns of different character strings.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of the Invention) The present invention relates to a text search method for determining whether a predetermined character string exists in a character string.

(Prior Art) In the field of data processing systems, it is necessary to search for only those that contain a specific substring as a key from a collection of character string data such as sentences. It is often necessary to extract the key. Usually 1
Since one sentence is expressed by a fixed length code of n bits, the character string data is a series of codes of n bits.

Character string data is generally stored in an external storage device of a computer, such as a magnetic disk, and is serially transferred one sentence at a time to a central processing unit during a search. Therefore, in order to reduce processing time, it is necessary to perform a search simultaneously with data transfer.

FIG. 1 is an explanatory diagram of such a text search mechanism. 1st
In the figure, 1 is a storage device in which character string data is stored;
Reference numeral 2 designates a character string search device that searches for character strings, 3 represents a character string data transfer path, and 4 represents a signal line that outputs search results. Character string data is serially input character by character from the storage device 1 to the character string search device 2 via the data transfer path 3. The character string search device 2 compares the input data with a pre-stored partial character string serving as a key, and outputs a match signal to the signal line 4 when a match is detected between the two. As a method for collating character strings in the character string search device 2, a method using a finite automaton is generally known. (L, A, Ho1laar' Hardwar
e Systems for Text Information
ation Retrieval llACM 5IG
IR6th Conference 1983) 2nd
The figure is an explanatory diagram showing the state transition of a finite automaton. In Figure 2, 5 represents the state of the automaton, 6 represents the direction of state transition, and nDOG is selected from the character string data.
You can search for the three-letter key ll. This operation will be explained below. The initial state of the automaton is state (0), and if the input character is +lD'', it becomes state (1).
Transition to. In Fig. 2, 1≠1 represents other characters, and if the input character in state (0) is other than "D11", it will continue to remain in state (0).The same applies to state (1), and if the input character is n □ If n, state (2)
If 'D' is 1, the state returns to state (1); otherwise, the state returns to state (0).

If the input character is +1GII in state Q), state (3
), which means that the key '1DOGI' has been detected, and a match signal is output from the signal line 4 in FIG.

FIG. 3 is an explanatory diagram of a circuit configuration for realizing a conventional finite automaton for character string data expressed in 8-bit J/S code. In Figure 3, 3 is a character string data transfer path, 4 is a signal line for outputting search results, and 7 is 1
6-bit address register, 8 is 64KB (256
X2813) random access memory, 9 is an address decoder, 10 is an 8-bit memory register, 1
1 is a discrimination circuit, 12, 14 and 15 are 8-bit wide data lines, and 13 is a 16-bit wide address line.

FIG. 4 shows the contents of the state transition table stored in the random access memory 8 of FIG.
16 is 8-bit data, 17 is the upper 8 bits of the memory address, and 18 is the lower 8 bits of the memory address. Note that logically, the upper address 17 of the memory corresponds to the state number, the lower address 18 of the memory corresponds to the character code, and 19 is a character that can be expressed by the code of the lower address 18 of the memory.

The input character is transferred from the data transfer path 3 to the lower 8 bits of the address register 7. The upper 8 bits of the address register 7 are set to all zeros as an initial value, and the 8-bit data stored at the address from the random access memory 8 can be input to the address decoder 9 via the address line 13. 16 is read out,
The data is stored in the memory register 10 via the data line 14.

In the discrimination circuit 11, the data line 15 is connected to the memory register IO.
Refer to the contents of ``7'' is a high value (in hexadecimal notation)
FF'), a match signal is output to the signal line 4, and if it is other than Highbury-, the contents of the memory register 10 are set to the upper 8 bits of the address register 7 via the data line 12. The above operation is repeated every time one character is input from the data transfer path 3, thereby executing the search process.

If we try to apply the conventional method explained above to a Japanese character string in which each character is represented by 16 bits, the number of code types will be 216, so there will be state transitions representing 256 states as in Figure 3. The size of the random access memory 8 used to store the table is IMB (256 x 2
16B) Required. Moreover, the data that can be stored here1
6 is generally mostly zero, which has the drawback of requiring a huge amount of memory to be used with extremely low utilization efficiency.

(Objective of the Invention) The present invention does not use the character code itself as an entry in the state transition table, but instead uses the character code '211. I
It is characterized by the use of partial turns divided into J multipliers, and its purpose is to make it possible to search using a small-sized state transition table even when the bit length of the character code is long. The case where a character code is divided into two will be described in detail below.

(Structure and operation of the invention) Fig. 5 is a block diagram of an embodiment showing the structure of a circuit for realizing a finite automaton using the method of the present invention. In this figure, 20 is a 16-bit data register, 21 is a switching circuit, and 23 and 24 are 8-bit wide data lines.

FIG. 6 shows the structure of the state transition table stored in the random access memory 8 of FIG. 5, and the key partial string is 1 sea level n (hexadecimal) JIS Courtesy table wast v3324. 4834v).

The input characters are stored in the transfer path 3 and the data register 20. The switching circuit 21 sets the contents of the data register 20 8 bits at a time to the lower 8 bits of the address register 7 via the data line 23 alternately in the order of 2nd place and lower order.

The upper 8 bits of the address register 7 are set to all zeros as an initial value, and are input to the address decoder 9 via the address line 13, and the 8-bit data stored at the address from the random access memory 8 is sent to the address decoder 9 via the address line 13. 16 is read out and stored in the memory register 10 via the data line 14.

FIG. 7 is a diagram showing truth values describing the operation of the discrimination circuit 22, in which 25 is an input from the data line 24, 26 is an input from the data line 15, 27 is an output to the data line 12, and 28
is a hexadecimal representation of the output to the signal line 4.

The discrimination circuit 22 operates according to the truth diagram shown in FIG.
That is, the upper 8 bits of address register 7 are v, FE
In the case of v, I2, that is, the contents of the memory register 10, is ignored and the upper 8 bits of the address register 7 are forcibly reset to voov.

Figure 8 is a concrete example showing that this compulsory cent is required, where 29 is a key substring, 30 is a hexadecimal representation of substring 29, and 31 is the character to be searched. Column data 32 is a hexadecimal representation of character string data 31.

As shown in Figure 8, if a character represented by a 16-bit code is divided into 8-bit parts and searched, the code patterns of two different character strings will match with a difference of exactly 8 bits. occurs.

In order to prevent this phenomenon, in the present invention, a special value (vFEv) ffi is embedded in the state transition coefficient, and the determination circuit 22 is forced to reset it. Note that when I□ of the discrimination circuit 22 is other than vFEv, the same operation as the conventional method is performed.

FIG. 9 is a flowchart for creating the state transition table shown in FIG. 6 and FIG. 10.

In Figure 9, 33 is 256 work areas, 34
In the figure, i is a variable representing the upper 8 bits of the address in the state transition table, and 3, denoted by 35, is a variable representing the lower 8 bits of the address.

FIG. 10 shows the contents of the state transition table and work area created according to the flowchart 1 shown in FIG. 9. The key character string is V Commerce Department V (hexadecimal code is 3E26. 4C33,3E4A).

As is clear from the flowchart of FIG. 9, creating the state transition table is simple and easy. Also, the size of the state transition table in Figure 10 is 1536 bytes) (6X2
8B), which is significantly smaller than the size of the state transition table according to the conventional method, which is 196,608 knots (3×2 B).

In the above description, the code is divided into 1/2, but it is clear that the code is not limited to this and may be divided into 1/2.

(Effects) As explained above, the present invention does not directly use the n-bit code representing one character as an entry in the state transition table, but instead uses a partial code pattern that divides the n-bit code pattern into half. is used as an entry, and is equipped with a forced cent mechanism that prevents a substring that is not originally included from being detected due to a shift of n72 bits in the character string data, so it is completely different from the conventional method. The advantage is that the same functionality can be achieved with a state transition table that is significantly smaller in size than the conventional one.

[Brief explanation of the drawing]

Figure 1 is an explanatory diagram of the text retrieval mechanism, Figure 2 is an explanatory diagram showing the state transition of a finite automaton, Figure 3 is a diagram of the circuit configuration for realizing a conventional finite automaton, and Figure 4 is a diagram of a conventional state transition table. 5 is a block diagram of an embodiment showing the configuration of a finite automaton implementation circuit using the present invention, and FIG. 6 is a configuration diagram of a state transition table stored in the random access memory of FIG. 5. , FIG. 7 is a diagram showing the truth value of the operation of the discrimination circuit according to the present invention, FIG. 8 is a specific example showing the necessity of a forced reset mechanism in the present invention, and FIG. 9 is a diagram showing the truth value of the operation of the discriminating circuit according to the present invention. FIG. 10 is a flowchart for creating the state transition table. FIG. 10 is a diagram showing the state transition table created according to flowchart 1 of FIG. 9 and the contents of the work area. 1...Storage device, 2...Character string search device, 3...Data transfer path,
4...Signal line, 5...Automaton state, 6...River...Direction of state transition,
7...Address register, 8...
・Random access memory, 9・・・・・・・・・・
・Address decoder, 10... Memory register, 11 ゛----゛ Discrimination circuit, 12.14.
15.23.24... Data line, 13.
・・・・・・・・・Address ill, 16・・・・・・・
...F-data, 17--... Upper address of memory, 18 Lower address of memory, 19...
・・Character corresponding to the code, 20 ・・・・・Data register, 21 ・・・Switching circuit, 22 ・
...Discrimination circuit, 25, 26--Input terminal, 27, 28 ...-Output terminal, 29...
・ Partial character string, 30.32 ... Hexadecimal representation of the character string, 31 ... Character string data to be searched,
33......Work area. Patent Applicant Nippon Telegraph and Telephone Public Corporation Figure 1 Figure 2 Figure 3
1 (3B) (33X24 K48) (34X64) I32
Figure 10 (3E) (26X4CX33X3EX4A) ~ '30 Figure 9

Claims (1)

[Claims] When n is a positive even number, is it composed of characters expressed by n-bit codes? In a search method that uses a finite automaton that uses a two-dimensional state transition table with codes and state numbers as entries, the state transition A partial pattern obtained by dividing an n-bit code pattern into several parts is used as a table entry, and the state transition destination is determined by indexing the state transition table as many times as the code pattern is divided. A text retrieval method characterized by having a gluing gutter mechanism for preventing output of erroneous search results due to misalignment of bits in a code.