JPH10177582A - Method and device for retrieving longest match - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and device for retrieving longest match which can realize longest match retrieval of a word dictionary, etc., in natural language processing at a high speed and can efficiently realize longest match retrieval at a high speed. SOLUTION: Record groups are sorted in ascending order by key items (S1) and the record number indicating the record that matches the key item of an arbitrary record at the longest length before the arbitrary record is set as a pointer at every record (S2). The arranging position of a retrieval key word in a record group composed of records is retrieved (S3) and the length of the key in the record at the retrieved position is collated with the length of the retrieval key (S4) and, when the lengths coincide with each other, the record is decided as a longest match solution (S5). When the lengths do not coincide with each other, the the record of the next candidate is retrieved by using the pointer set in the record (S6).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a longest match search method and apparatus, and more particularly, to a search for a word dictionary frequently used in morphological analysis of natural language processing, and the like. The present invention relates to a longest match search method and apparatus that can be used to search for a record having a key that matches the longest match for a record or to search a record in a dictionary from the beginning of a search keyword.

[0002]

2. Description of the Related Art Conventionally, a longest match search is realized by a sequential search method, a trie method, a binary search method, or the like. The sequential search method is a method of sequentially searching data of a table including a key item from the head.

In the trie method, when the length of a character string is k, operations such as search and insertion can be performed with a calculation amount of O (k).
The binary search method is a method in which one group of items is divided into two based on a certain criterion, and a search target is selected. The search method is effective when data is sorted. , O (log n).

[0004]

However, performing the longest match search of a word dictionary or the like frequently used in the above-mentioned conventional natural language processing at high speed has the following problems. In the above conventional sequential search method, when the size of the table is n, and when the target data is at the end,
The search is performed n times. The average search order is O
((N + 1) / 2), and when the data amount is large,
There is a problem that the processing amount becomes extremely large.

Although the above-mentioned conventional try method is effective for searching a word dictionary, management of links between nodes and the like is complicated, and waste of a storage area is large. In addition, management different from a table frequently used in a database or the like is required, which is not suitable for management in units of records.

Further, the above-mentioned conventional binary search method is effective for searching for a keyword that matches the beginning of the keyword, and can perform dictionary maintenance easily and at high speed.
There is a problem that the longest match search cannot be realized only by this method. The present invention has been made in view of the above points, and solves the above-described conventional problems, and can realize a longest match search of a word dictionary or the like in natural language processing at high speed. It is an object of the present invention to provide a longest match search method and apparatus capable of realizing a longest match search.

A further object of the present invention is to reduce the storage capacity and the number of dictionary searches only by managing pointers, even when it is necessary to search for candidates by backtracking, such as when searching a word dictionary. Is to provide a longest match search method and apparatus that can perform the above.

[0008]

FIG. 1 is a diagram for explaining the principle of the present invention. According to the present invention, in a longest match search method for searching a record having a key that has the longest match with an arbitrary search keyword from a record group having variable-length key items, the record groups are sorted in ascending order by key items (step 1). ), A record number indicating the record that matches the key item of the record longest before any record is set as a pointer for each record (step 2), and the position of the search keyword in the record group consisting of records is determined. A search is performed (step 3), and the key in the record at the searched position is collated with the length of the search keyword (step 4). If there is a match, the record is regarded as the longest matching solution (step 5), and no match is found. in case of,
The next candidate record is searched by the pointer set in the record (step 6).

In addition, the present invention uses a binary search when searching for the position of a search keyword in a record group. According to the present invention, a pointer set in a record is a relative number from the record.

FIG. 2 is a diagram showing the principle of the present invention. The present invention is a longest match search device that searches a record group having a variable length key item for a record having a key that matches the longest with an arbitrary search keyword, and sorts each record in ascending order by the key item, For each record, a record group 10 set as a pointer with a record number indicating a record that matches the key of the record longest before the record, and a position of a row of key items in the record group 10 are searched, Record position searching means 20 for searching for the longest match candidate record, longest match matching means 30 for matching the search keyword with the length of the key item in the record of the record group, and the next candidate record using the record pointer And a next candidate search unit 40 for searching for.

The above-mentioned record position search means 20
Uses a binary search. The pointer of the record group is a relative number from the record of the record group. In the above record position search means, binary
By using the search, the position of the input keyword sequence and the record of the longest match candidate are searched, and the longest match matching means compares the key value of the searched record with the search keyword for the key length of the record. Check for a match. In general, in binary search, there is no guarantee that a record in which the key of the record matches the search keyword at the longest is obtained. However, according to the present invention, if the first character matches the first character of the key item of the record, the longest matching record can always be reached by tracing the pointer.

Therefore, the next candidate search means searches for the next candidate record based on the pointer, and the longest match matching means performs the longest match check, thereby making it possible to search for the record that matches the keyword the longest. . This can be realized by searching for the order of at most the length of the character string by the next candidate search means using a pointer.

In the search of a word dictionary in the morphological analysis of natural language processing, not only the longest matching candidate but also the longest matching word such as "Aoki", "Ao", "A" is extracted. Is required. In the present invention, the pointer set in the record group is linked to the next candidate, so that the next candidate can be obtained without searching.

[0014]

FIG. 3 shows the configuration of a longest match search apparatus according to the present invention. The longest match search device shown in FIG. 1 includes a storage unit 10 having a stack for storing a record group, a record position search unit 20, a longest match check unit 30, a next candidate search unit 40, a keyword input unit 50, and an output unit 60. Is done.

As shown in FIG. 4, the storage unit 10 stores records including a record number, a key item, a key length, and a next pointer in a stack, and each record is sorted in ascending order by the key item. . As shown in FIG. 5, when the original dictionary data is input (step 10), the next pointer of each record stored in the storage unit 10 is the ascending order of the original dictionary data by the key item (alphabetical order / key length). ) Sorting (step 11), and for each record, set as a pointer a record number that points to a record that is located before the record and that matches the key item of the record at the longest (step 12), and generates generated dictionary data, that is, Then, a sorted record group in which the next pointer is set is generated (step 13).

The record position search unit 20 searches for the position of a key item of a record group in the storage unit 10 corresponding to the input keyword, and searches for a record that is the longest match candidate. The longest match matching unit 30 checks the key of the record having the searched key item against the length of the input keyword, and if there is a match, transfers the record to the output unit 60 as the longest match. If not, the control is transferred to the next candidate search unit 40.

The next candidate search unit 40 includes the longest match matching unit 30
Then, the next candidate record is searched by the next pointer of the record that did not match in the step (a), and the information of the record searched again is transferred to the longest match matching unit 30. The keyword input unit 50 receives an input of a search keyword.

The output unit 60 outputs the longest matching solution obtained by the longest match in the longest match checking unit 30 as a search result. Next, the outline operation of the present invention will be described. FIG. 6 is a flowchart showing an outline operation of the longest match search of the present invention.

Step 101) First, a search keyword is input from the keyword input section 50. Step 102) The record position search unit 20 performs a binary search for the input search keyword,
The position of the search keyword is searched for, and the longest match candidate record is extracted from the record group in the storage unit 10.

Step 103) Longest match checking section 30
Compares the search keyword with the length of the key item of the record of the longest match candidate acquired by the record position search unit 20. For example, if the search keyword is n characters and the key item of the acquired record is m characters (where n ≧ m), the comparison is performed for m characters.

Step 104) If the result of comparison is a match, the process proceeds to step 105, and if not, the process proceeds to step 106. Step 105) The key item of the record is output from the output unit 60 as the longest matching solution.

Step 106) The next candidate search unit 40
It is determined whether or not the acquired record and the first character of the search keyword match. If they match, the process proceeds to step 107, and if they do not match, the output unit 60 outputs no solution. Step 107) The next candidate search unit 40 acquires the next pointer by referring to the pointer of the acquired record, transfers the information to the longest match matching unit 30, and proceeds to step 103.

[0023]

Embodiments of the present invention will be described below with reference to the drawings. [First Embodiment] First, a method of setting a next pointer of a group of records stored in the storage unit 10 will be described.

A record group stored in the storage unit 10 is shown in FIG.
This will be described with reference to FIG. In the figure, the records are sorted in ascending order of the key items, and the length of the key value and the next pointer are set. For example, the record number 6 of the storage unit 10
The stack of "Aokigaoka" Since the match from the beginning with the record number 6 previous record number 5 of the key, "Aoki" of key value length of 3 characters as "Aoki Gaoka", as the next pointer, refers to the record number 5 The pointer “5” is stored, and “3” is set as the pointer in the stack of record number 5 having the key value of “Aoki” because “Ao” of record number 3 matches the longest.

Next, the operation for generating the generated dictionary data stored in the storage unit 10 from the original dictionary data will be described.
FIG. 8 is a diagram for explaining a method of setting a next pointer of a record group according to the first embodiment of this invention.

When the original dictionary data including the record number, the original dictionary data key item and the key length shown in FIG. 1 is input, the key items of the original dictionary data are sorted in the order of the Japanese syllabary and in the key length. In this example, “A” having the shortest key length, which is the beginning of the Japanese syllabary, is set at the beginning of the generated dictionary data, and the next pointer is “0”. Next, in the order of the Japanese syllabary, the record of “Ioi” is located, and since the first character matches “A” of the record before the record, the next pointer is “1”. By repeating such processing, generated dictionary data is configured.

Such a procedure will be specifically described with reference to a flowchart shown in FIG. First, the first record to be processed (record number 1) is extracted (step 20).
1). The retrieved record has “A” as a key item, and refers to the stack top of the storage unit 10 (steps 202 and 203). Here, since the stack is empty (step 204), "0" is set to the next pointer (step 209). The key of the record and the record number “1” are stored in the stack of the storage unit 10 (step 210), and the next pointer “0” is stored in the storage unit 10 like the record “1” shown in FIG. Step 21
1). The stack of the storage unit 10 in this state is shown in FIG.
0 (a).

Next, when the next record (key item value “Ioi”) is obtained (step 201), the stack of the storage unit 10 is not empty in the current state (step 204, No). of the stack and the "a""aIoi"
Is compared with one key length of the stack (step 20).
5). As a result, they match (step 206, Ye
s) The record number “1” having “A” is set in the stack of the matching storage unit 10 as the next pointer of the record “2” (Step 207), and the record of “Ioi” is stored in the storage unit 10. Stack on the stack (Step 21
0). As a result, the record “2” is output to the storage unit 10 (Step 211). The stack of the storage unit 10 in this state is in the state of FIG.

Next, a record of "Ao" is obtained (Step 201), and "Ioi" and "Ao" at the top of the stack in the storage unit 10 are compared with each other for a key length of 4 characters of the stack (Step 205). , In this case a mismatch,
The record of “Ioi” is removed from the stack of the storage unit 10 (Step 208), and the process proceeds to Step 203 to refer to the key of “A”. Then, with the "A", "A O"
Are compared by one character (step 205), and because they match, the record number of the key item value “a” is set as the next pointer in the record of “ao” (step 20).
7) The record of "blue" is stacked on the stack of the storage unit 10 (step 210). Further, record “3” of the generated dictionary data is output (step 211). The stack of the storage unit 10 in this state is as shown in FIG.

As described above, the next pointer is set as in the generated dictionary data shown in FIG. Next, the processing of the longest match search in this embodiment will be specifically described. FIG.
9 is a flowchart of a longest match search process according to the first embodiment of this invention. The flowchart in FIG. 11 shows a case in which a record that has the longest match with “Aokigaokashita” is searched from the generated dictionary data in the storage unit 10 shown in FIG.

First, the record position search unit 20 searches for the position of the sequence for the keyword "Aokigaokashita" input from the input unit 50. Since the records in the storage unit 10 are sorted, a high-speed binary search algorithm can be applied as a search algorithm.
"Aokigaokashita" is positioned between "Aokigaoka" and "Aoba" (step 301).

According to the sort condition, no record matching "Aokigaokashita" with the key length of the dictionary data exists after "Aoba". Therefore, “Aokigaoka” is the longest match candidate record. Next, the longest match matching unit 30 outputs “Aokigaoka” and “Aokigaokashita”
Is compared with the key length of "Aokigaoka" in the storage unit 10 (step 302), a match is obtained (step 303). From this, records having "Aokigaoka" which is the longest match with "Aokigaokashita" were searched.

The next candidate search unit 40 searches for the next pointer pointing to "Aoki" (step 3).
04). The next candidate that matches the input keyword “Aokigaokashita” in four, three, and two characters is sequentially searched for, such that the pointer next to “Aoki” points to “Ao”. By reading the record pointed to by the pointer, it can be obtained without searching.

Next, a specific example of the longest match search will be described based on a specific example shown in FIG. The example of FIG. 13 is an example in which the keyword “Ando” is input from the keyword input unit 50. The record position search unit 20 is configured as shown in FIG.
As in the case of No. 2, “Ando” is positioned between “Awaji” and “Iai”. Since there is no match after “Ia”, “Awaji” is acquired as a candidate.

When "Awaji" and "Ando" are compared by the key length of "Awaji" in the longest match collation unit 30, they do not match, and the record of "Awaji" is not the longest match solution. Since the first character is “A” with both, the next candidate search unit 40 acquires a record (record number 1) pointing to the next pointer of “Awaji”. Longest match collation unit 30
By comparing the keyword “Ando” with the next candidate “A” with one character, a match can be obtained. Therefore, a record having “A” as the record that longest matches “Ando” is searched. The next pointer to the record of “A” is “0”, indicating that there is no longer matching record.

Further, a specific example of the longest match search will be described based on a specific example shown in FIG. FIG. 14 shows an example in which the keyword "ear" is input. First, the record position search unit 20 positions “ear” between “awaji” and “ear”. Since there is no match after “Ia”, “Awaji” is acquired as a candidate. When the longest match matching unit 30 compares “Awaji” and “Oia” with three key lengths of “Awaji”, a mismatch occurs.
The record of "Awaji" is not the longest match. Since the first character is different, even if the next candidate is obtained with the pointer next to "awaji", the longest matching solution does not exist. As a result, it is found that there is no record that longest matches the input keyword "ear".

Further, a specific example of the longest match search will be described based on a specific example shown in FIG. FIG. 15 is an example in which the keyword “Akasaka” has been input. The record position search unit 20 searches for a record that completely matches “Akasaka”. In the record where the duplicate key is permitted, the last of the same key is found, and “Akasaka” of the record (record number “12”) is set as a candidate. The longest match collating unit 30 obtains a record (record number “12”) of “Akasaka” that is a perfect match. The next candidate search in the next candidate search unit 40 is as follows.
12 and 13 described above, and a description thereof will be omitted.

[Second Embodiment] Next, as a second embodiment, an example in which a relative record number is used as a pointer will be described. FIG. 16 is an example of a record group in the storage unit according to the second embodiment of the present invention, in which a relative record number is set in a column used as a pointer in the first embodiment.

By subtracting the relative record number from the record number of the record, the pointed record can be obtained. For example, since the relative record number of the record with the record number “12” is “3”, the pointed record is the record number “9”.

With this setting, it is possible to perform the longest match search similar to the first embodiment described above.
It should be noted that the present invention is not limited to the above embodiments,
Various modifications and applications are possible within the scope of the claims.

[0041]

As described above, according to the longest match search method and apparatus of the present invention, the longest match search such as a word dictionary in natural language processing can be realized at high speed, and the binary search method and the next candidate are connected. By setting the pointer to be used when the dictionary is created, high-speed and efficient longest-match search can be performed.

Further, since the next candidate is managed by setting the pointer, even when it is necessary to search for the candidate by backtracking, such as a search for a word dictionary, the storage capacity is reduced by only managing the pointer. Also, the number of dictionary searches can be reduced. It is also possible to set a relative record number as a pointer.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a longest match search device of the present invention.

FIG. 4 is a diagram illustrating a configuration of a record group stored in a storage unit according to the present invention.

FIG. 5 is a flowchart of setting a pointer of a record in a storage unit according to the present invention.

FIG. 6 is a flowchart of an outline operation of a longest match search according to the present invention.

FIG. 7 is a diagram illustrating a method of creating a search target record according to the first embodiment of this invention.

FIG. 8 is a diagram for explaining a method of setting a next pointer of a record group according to the first embodiment of this invention.

FIG. 9 is a flowchart of an operation of setting a next pointer according to the first embodiment of this invention.

FIG. 10 is a diagram illustrating a transition of a stack of a storage unit according to the first embodiment of this invention.

FIG. 11 is a flowchart of a longest match search process according to the first embodiment of this invention.

FIG. 12 illustrates a first example of a longest match search according to the first embodiment of this invention.
Is a specific example.

FIG. 13 shows a second example of the longest match search according to the first embodiment of the present invention.
Is a specific example.

FIG. 14 shows a third example of the longest match search according to the first embodiment of the present invention.
Is a specific example.

FIG. 15 is a fourth example of the longest match search according to the first embodiment of this invention;
Is a specific example.

FIG. 16 is an example of a record group in a storage unit according to the second embodiment of this invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Search target record group, storage part 20 Record position search means, record position search part 30 Longest match matching means, longest match check part 40 Next candidate search means, Next candidate search part 50 Keyword input part 60 Output part

Claims (6)

[Claims] 1. A longest match search method for searching a record having a key that has the longest match with an arbitrary search keyword from a record group having variable length key items, wherein the record group is sorted in ascending order by key items; A record number indicating the record that matches the key item of the record at the longest before any record is set as a pointer for each record, and a search position of the search keyword in a record group including the records is searched. The key in the record at the specified position is compared with the length of the search keyword, and if they match, the record is determined to be the longest matching solution. If not, the next candidate is determined by the pointer set in the record. A longest match search method, which searches for records of the same type. 2. The longest match search method according to claim 1, wherein a binary search is used when searching for the position of the search keyword in the record group. 3. The longest match search method according to claim 1, wherein the pointer set in the record is a relative number from the record. 4. A longest match search apparatus for searching a record group having a variable length key item for a record having a key that matches the longest with an arbitrary search keyword, wherein each record is sorted in ascending order by the key item. For each record, a record group set as a pointer to a record number that points to a record that matches the key of the record longest before the record, and a search is made for a position of a row of key items in the record group, A record position search unit that searches for a record of a longest match candidate; a longest match check unit that matches the search keyword with a length of a key item in a record of the record group; and a longest match in the longest match check unit. When a record cannot be obtained, there is provided a next candidate search means for searching for a next candidate record using the pointer of the record. Longest match search apparatus according to claim Rukoto. 5. The longest match search device according to claim 4, wherein said record position search means uses a binary search. 6. The longest match search device according to claim 4, wherein each pointer of the record group is a relative number from a record of the record group.