JPH04315259A - Character string collation processing system - Google Patents

Abstract

PURPOSE:To make character string retrieval speedy and flexible by storing document data to be retrieved while dividing them for the unit of a line and executing the retrieval by an in-line character string collation processing and an inter-line character string collation processing when retrieving character strings by parallel processors. CONSTITUTION:A parallel character string collation part 72 is equipped with plural processor elements 721 and executes parallel processings. The character string same as a retrieving word inputted from the outside is detected out of the document data to be detected in a data base. The document data are stored in memory areas 74, which can be directly accessed by the processor elements 721, while being divided for the unit of the line. In the in-line character string collation, the data to be retrieved are read into the processor elements 721 and the character string retrieval is executed by the collation for the unit of a character in the retrieving word. When there is possibility for the retrieving word to be existent over an (m) line and an (m+1) line, a prescribed character component at the head of the (m+1) line is linked to the end of the (m) line, one line is formed and the collation is executed.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to a character string matching processing method that makes it possible to quickly search for a desired character string from a document database that stores natural language document data as character code strings. It is.

0002

2. Description of the Related Art Conventionally, single instruction multiple data processing (SIMD), which has a plurality of processor elements (PEs),
ngle Instruction Multiple
Character string matching that detects a character string that is the same as or similar to a search word, which is a desired character string input from the outside into a parallel processing device of type Data), from searched document data, which is a searched sentence in a database. In the processing of
"Character string search LSI capable of ambiguous search": Nikkei Electronics, June 1, 1987, No. 422).

[0003] This principle is illustrated in a character string collation processing device as shown in FIG.

In FIG. 8, 1 is a search word input section, 2 is a character string matching section, and 3 is a document data storage section (database).
, 4 is a verification result storage section, 5 is a data transfer bus, and 6 is a verification processing control section. Note that (20) 1 ... (20) N
indicates the individual word matching section.

[0005] A detailed configuration of the character string matching section 2 is shown in FIG.

In FIG. 9, 20 is an individual word matching unit;
22 is a search word storage section, 22 is a search data holding section, 23 is a character string match detection section, 24 is a search word input section, 25 is a search data input section, and 26 is a match information output section.

To operate this device, if a search word (here, the word character string is used as an example) is input from the search word input section 1 shown in FIG. is temporarily stored in each register (211, 212, . . . ) of the search word storage unit 21 in the character string matching unit 2, and waits for the data to be searched to be transferred.

The searched data is sequentially read out from the document data storage section 3 and sent to each register (221, 222) of the searched data holding section 22 via the searched data input section 25.
,...).

[0009] In the searched data holding unit 22, the searched data is inputted one character at a time, and each time, the comparators (231, 232, . . .) of the character string match detection unit 23 compare the data of the search word with the previous one. It is determined whether or not the character string matches the input search data, and when a match is found, the matching information output unit 2
Matching information is output from 6, and matching position information is recorded in the matching result storage section 4.

FIGS. 10 and 11 show how the status of the processing contents of the individual word matching unit 20 changes during character string matching.

That is, for example, after the search word "character string" is set in the search word storage section 21, the searched data "scanning the string at high speed..." is stored in the searched data holding section 2.
2 shows the process of processing within the individual word matching unit 20 until a "character string" present in the text is detected.

FIG. 10 is a state diagram at the time when a "character string" of the searched data that is the same as the search word starts being input into the searched data holding section 22 of the individual word matching section 20.

FIG. 11 shows a state in which three characters have been input to the searched data holding section 22 from the state shown in FIG. 10, and a character string match with the search word has been detected.

[0014]

[Problems to be Solved by the Invention] In the conventional method shown in FIG. becomes a bottleneck, making it impossible to obtain fast search speeds.

[0015] Furthermore, since the character string matching unit 2 is fixed in hardware, there are problems such as the inability to flexibly deal with changes in search conditions such as when the search word length becomes longer. .

An object of the present invention is to solve the conventional problems and to provide a character string matching method that can perform character string searches at high speed by efficiently operating a general-purpose parallel processing device in character string search processing. Our goal is to provide the following.

[0017]

[Means for Solving the Problems] In order to achieve the above object, the present invention provides a search word that is a desired character string that is externally input to a single-instruction multiple data processing type parallel processing device having a plurality of processor elements. In the process of character string matching to detect a character string that is the same as or similar to the searched document data that is the searched sentence in the database, the document data is stored in a memory area that can be directly accessed by the processor elements. The in-line character string matching process is divided into line units, which are stored, and is compared and determined to match the search word in response to a search request, and the line-to-line string matching process is used to match character strings that span between lines. The method is characterized by comprising a character string matching step.

[0018] Furthermore, a character string that is the same as or similar to a search word that is a desired character string that is externally input to a single-instruction multiple data processing type parallel processing device that has a plurality of processor elements is searched for in a database. In the process of character string matching detected from search target document data, which is a sentence, each processor element of the parallel processing device is arranged two-dimensionally, and the processor elements are connected vertically, horizontally, and the top end is connected to the bottom end. , has a parallel processing configuration in which the left end and right end are connected at different levels, and the line-to-line character string matching process for matching character strings that span between lines matches the characters in the line of interest to allow the length of the search word. A string merging process that determines the amount of merging between a column and the next line's string and generates a new line string, and moves the new line string to match the string that spans between lines. a character string movement processing step to make the character string the same as the character string existing in the line, and dividing the document data into a memory area that can be directly accessed by the processor element into units of lines corresponding to the number of processor elements. An in-line character string matching process that stores and matches search words in response to a search request, and a character string position that calculates the character string position in the original document data from the detected character string position. An embodiment characterized by comprising a correction processing step is effective.

[0019]

[Operation] The present invention uses a device configuration different from conventional character string matching devices, that is, a general-purpose parallel processing device that can freely change the processing content by replacing the program in the device, and is capable of achieving high speed and flexibility. This enables high-speed string matching using a general-purpose parallel processing device, and allows high-speed searches for strings that span between lines in the data area that stores the data to be searched. .

Embodiments will be described below based on the drawings.

[0021]

[Embodiment 1] FIG. 1 is a diagram illustrating an embodiment of the present invention, in which 1 is a search word input section, 3 is a document data storage section (database), 4 is a matching result storage section, and 5 is a data transfer bus. , 6 is a collation processing control unit, 7 is a parallel processing unit, 71 is a collation processing program storage unit, 72 is a character string parallel collation unit, 73 is a large-capacity data bus, 74 is a search information storage unit (verification data memory), 721 , 722, ......, 72R are processor elements.

Note that 741 indicates a search word storage area, 742 a searched data storage area, 743 a matching result primary storage area, and 744 a matching result secondary storage area.

2 and 3 are detailed diagrams of a part of the parallel processing unit 7 (verification processing system), and FIG. 2 shows a two-dimensional diagram of R processor elements (721, 722, . . . , 72R). (
FIG.

FIG. 3 shows the processor element (72
1,722, . . . , 72R) are used in a one-dimensional (line) manner.

At this time, the search information storage section 74 is used by being divided into a search word storage area 741 and a searched data storage area 742, and the processor elements (721, 722, . . . 72R) is carried out via large-capacity data buses 73 prepared for the number of sets corresponding to the number R of processor elements.

To operate this, the document data to be searched is transferred from the document data storage section 3 to the searched data storage section 742 of the parallel processing section 7 in advance, and the search word from the search word input section 1 is transferred. Prepare for input.

Next, when the search word "character string" is input from the search word input section 1, the word is stored in the search word storage area 741 in the parallel processing section 7, and then a matching process is immediately executed. Ru.

FIG. 4 is a diagram illustrating the in-line character string matching process in the matching process, in which the processor elements are composed of R=256 lines, that is, (16×16) areas. The illustration is based on the assumption that

The first stage of processing in this figure shows the data arrangement of the character string parallel processing unit 7, which is the data to be searched for, ``Quickly collate character strings...'' Search for column……” search word in the sentence “
This shows the initial state of the start of matching to detect the position where the character string "exists."

[0030] The searched data is divided into units of the number of processor elements, and the lines from "High speed..." to "...characters" are recorded as one line, and after "Search column......" is the search information storage section 74 as the next line.
It is recorded and held in the searched data storage area 742 of .

The search word is recorded in the search word storage area 741 in the depth direction of each processor element 721 in preparation for verification processing.

Next, in the second stage of processing, that is, the in-line character string matching step, the data to be searched is read into the processor element 721 from the state shown in the first stage in FIG. 4, and then matched with the search word.

At this time, the data to be searched and the search word are compared in units of characters (in units of 16 bits), starting from the first character of the search word.

The matching results are stored in the matching result primary storage area 743 as shown in the second stage of FIG.
) is stored in the form.

This figure shows a case where the search word length n is composed of a "character string" consisting of three characters.

That is, "0" indicating the existence of a matching character appears shifted by one processor element.

To detect the same character string position as the search word from this state, shift the values in each row of the matching result primary storage area 743 corresponding to the α-th character of the search word to the left by (α-1). Then, fill in "0" from the right end.

As a result of this process, as shown in the third step of FIG. 4, "0" is placed in the processor element at the position corresponding to the beginning of the matched character string in each line corresponding to the search word in the secondary matching result storage area 744. are lined up in the depth direction.

Next, by performing an AND between the lines in the verification result secondary storage area 744 of the third stage in FIG.
It can be seen from the position where "0" exists that the same character string as the search word exists.

[0040] Through this process, it is determined that the same character string as the search word exists within the line (within 256 characters), and by checking the last (n-1) characters, there is a possibility that the search word exists between the lines. can be known.

FIG. 5 is a diagram for explaining the line-to-line character string matching process, and the searched data [I] in FIG. end,
The character string matching process is shown when the (m+1)th line starts with "String..." and the search word consists of n characters (here, n=3 "character string").

For example, if "0" exists in the (n-1) processor elements from the end of the matching result secondary storage area 744 in the m-th line in-line character string matching process, the work in FIG. Searched data on the line [
II], the data to be searched on the (m+1)th line is written for X characters from the beginning (X is a value larger than n, in this case 16 is chosen as an example). Overwrite the searched data (this searched data will be referred to as m+).

Next, by moving this m+th line of searched data to the left and inputting the overflowing left end data from the right end, the characters that were separated as shown in 1PE movement [III] in FIG. The column becomes a continuous string.

[0044] In order to realize this process with this apparatus, the process shown in FIG. 6 will be performed.

That is, assuming that the search word is 16 characters or less, first, the m-th line of searched data is copied to the work line.

Next, while calling the data to be searched on the (m+1)th line, data for 16 characters from the beginning is copied to the work line.

Next, the processor element is (16×1
6) The work line data is considered as the area of 1 stage,
In other words, 16 pieces of data are simultaneously shifted to the previous stage by one processor element (1PE movement [III
).

Next, by performing in-line character string matching processing on this line, it is possible to detect whether or not a character string that matches the search word exists, and if so, the location of the character string.

Finally, by performing positional correction for X characters on this search result, character strings spanning between lines can be detected.

[0050]

Embodiment 2 In general general-purpose parallel processing devices, it is often impossible to partially extract or partially rewrite information on a line in a single process.

In such a case, that is, when the processor elements operate completely in parallel, the line-to-line string matching process detects matching strings according to the processing flow of the line-to-line string matching process shown in FIG. do.

That is, under the same conditions as Example 1,
If “0” exists within the last (n-1) characters of the matching result secondary storage area 744, for example, if the search word is within 16 characters, the processor element
In the process 6), first, the m-th line of searched data is shifted forward by 1 PE by one stage, that is, 16 pieces of data, and then the final stage processor element P
E (16 pieces of data) is masked and cleared to obtain data A.

Next, after the data to be searched on the (m+1)th line is shifted backward by one stage, data B is obtained by masking and clearing the other data, leaving the processor element PE at the last stage.

Next, by performing an in-line character string matching process after ORing data A and data B, it is possible to detect whether or not a character string that matches the search word exists. By correcting the position of the character string, character strings spanning between lines can be detected.

[0055]

[Embodiment 3] Line-to-line character string matching processing, that is, the presence or absence of a search word spanning between lines is determined by "0" within the last (n-1) characters of the m-th line matching result secondary storage area 744. It can be known from the existence of the character string on the m+1th line.

That is, in the m-th line of the searched data, there are α characters from the beginning of the search word within (n-1) characters from the end.
If individual characters are detected as a match, (m+1
) The primary storage area 743 for the matching results in the line 2-th line character string matching is (n-1) in order from the top line,
(n-2) pieces, ......, one by one from the left end, shift to the right and (n-α
), if "0" is arranged vertically, the same character string as the search word will start from the α-th character from the end of the m-th line, and the search word can be detected.

[0057] In Examples 1 to 3, the case where the processor element is composed of 16 bits has been described, but when the processor element is composed of 1 bit, the search word or searched data is composed of When character data to be searched is recorded in the search information storage section 74, if the bit arrangement of the data is stored in the depth direction of the processor element, character string search processing can be realized by the same processing as in the first to third embodiments.

[0058]

As explained above, the present invention has the following features in this embodiment.

(a) High-speed character string matching is possible using a general-purpose parallel processing device.

(b) Character strings extending between lines of the data area storing the data to be searched can be searched at high speed.

Therefore, the present invention has advantages such as (1) short search time and (2) no omissions in the search.

[Brief explanation of drawings]

FIG. 1 is a configuration diagram of a character string matching processing method according to the present invention.

FIG. 2 shows the first configuration of a string parallel matching unit.

FIG. 3 is a second configuration of a string parallel matching unit.

FIG. 4 is a state diagram for explaining processing steps for parallel character string matching.

FIG. 5 is a state diagram for explaining a line-to-line character string matching process.

FIG. 6 is a process flow diagram for explaining a line-to-line character string matching process.

FIG. 7 is a processing flow diagram of line-to-line character string matching in a fully parallel processing type without mask processing.

FIG. 8 is a configuration diagram of a conventional method of character string matching.

FIG. 9 shows the configuration of a conventional character string matching section.

FIG. 10 shows the first state change of the conventional character string matching section.

FIG. 11 is the second state change of the conventional character string matching section.

[Explanation of symbols]

1 Search word input section 2 Character string matching section 3 Document data storage section 4 Matching result storage section 5 Data transfer bus 6 Matching processing control section 7 Parallel processing section 20 Individual word matching section 21 Search word storage section 22 Searched data holding section 23 Character string match detection section 24 Search word input section 25 Searched data input section 26 Match information output section 71 Matching processing program storage section 72 Character string parallel matching section 73 Large capacity data bus 74 Search information storage section (matching data memory) 721,
722,...,72R Processor element 741 Search word storage area 742 Searched data storage area 743 Matching result primary storage area 744 Matching result secondary storage area

Claims (2)

[Claims] Claim 1: A character string that is the same as or similar to a search word that is a desired character string input externally to a single-instruction multiple data processing type parallel processing device having a plurality of processor elements is searched in a database. In the process of character string matching detected from searched document data, which is a sentence,
Document data is divided and stored in line units corresponding to the number of processor elements in a memory area that can be directly accessed by the processor element, and characters in the line are used to match and determine a match with a search word in response to a search request. A character string matching processing method comprising a column matching process and an interline character string matching process for matching character strings spanning between lines. [Claim 2] A character string that is the same as or similar to a search word that is a desired character string that is externally input to a single-instruction multiple data processing type parallel processing device that has a plurality of processor elements is searched in a database. In the process of character string matching detected from searched document data, which is a sentence,
Each processor element of the parallel processing device is arranged two-dimensionally, and the processor elements are arranged above and below each other.
It has a parallel processing configuration in which the left and right ends are connected to the bottom end, and the left and right ends are connected at different levels. A string merging process that determines and merges the amount of merging between the string of the line of interest and the string of the next line so as to allow, and generates a new line string, and the process of matching strings that span between lines. A character string movement processing step that moves a new line character string to the same character string as the character string existing in the line, and stores document data in a memory area that can be directly accessed by the processor element in units of the number of processor elements. The character string matching process is performed by dividing and storing each line into line units, and comparing and determining a match with the search word in response to a search request. A character string matching processing method comprising a character string position correction processing step of calculating a column position.