JP5293301B2 - SEARCH DEVICE, SEARCH METHOD, AND STORAGE MEDIUM - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately determine the similarity of keywords as the result of collation by achieving ambiguous retrieval without increasing the number of states of an AC machine. <P>SOLUTION: A retrieval device 100 successively compares an AC machine 140c with the respective characters of text data when the text data are acquired by a collation processing part 150b, and when normal transition is performed, registers scores corresponding to the characters in a score array 140d, and when Feira transition is performed, puts the scores of the score array 140d ahead based on a difference in distances (dist) between the destination of transition and the origin of transition. Then, when the transition of the state to a collation state is performed, the collation processing part 150b totals the scores included in the score array 140d, and determines scores of the collated keyword. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a search device that performs pattern matching using an AC machine.

  In recent years, AC machines (Aho-Corasick machines) are used as a core engine of a search system. This AC machine performs pattern matching when text data (character string) is given, and calculates the appearance position of the character string to be searched (referred to as a keyword or pattern) from the text data at high speed. (For example, refer to Patent Document 1 and Non-Patent Documents 1 to 3).

  Since the pattern matching by the AC machine is completed only by scanning the text data D once, the calculation time is constant at O (| D |) no matter how much the number of keywords to be searched increases. The AC machine is particularly suitable for real-time processing, on-demand processing, stream processing, and the like because it does not require preprocessing of text data and can be calculated as it is in one scan.

JP-A-2-071388

A.V.Aho and M.J Corasick, "Efficient String Matching: An Aid to Bibliographic Search", Comm.ACM, Vol.18, No.6, pages 333-340,1975 S.Arikawa and T. Shinohara, "A Run-Time Efficient Realization of Aho-Corasick Pattern Matting Machines", New Generation Comput., Vol.2, No.2, pages 171-186,1984 Arikawa and Shinohara "String Pattern Matching Algorithm", Computer Software, Vol.4, No.2, pages2-23, 1987.

  For example, when a fuzzy search that does not distinguish between similar characters such as uppercase and lowercase letters is realized using an AC machine, how many uppercase letters and lowercase letters are in the hit keywords (similarity) Is required to be determined.

  However, if the above-described AC machine is applied as it is, it cannot be determined through which path on the AC machine the hit keyword has been hit when the pattern matching is completed. Therefore, it is impossible to determine how much of the hit characters are uppercase letters and how much are lowercase letters.

  If an AC machine in which all combinations of candidates for each character are developed is created, it is possible to determine how much of the hit characters are uppercase letters and how much are lowercase letters. However, this method is not realistic because the number of states included in the AC machine increases exponentially.

  That is, it is an important issue to realize a fuzzy search without increasing the number of states of the AC machine and to accurately determine the similarity of keywords as a matching result.

  The disclosed technology has been made in view of the above, and can realize a fuzzy search without increasing the number of states of an AC machine, and can accurately determine the similarity of keywords as a matching result. An object is to provide a search method and a storage medium.

  This search device uses a plurality of OR condition character string searches to set a permissible character that is not a correct character but is considered to be a condition match for a correct character included in the character string of the condition, and performs an ambiguous process. A search device for executing a search by an AC automaton, wherein correct characters and allowable characters, points corresponding to each character are set for each character digit of the plurality of conditional character strings, and a specific conditional character string is specified. Set up a failer transition that transitions to a character of a specific character digit in another condition character string when it matches up to the number of character digits but a mismatch occurs after that or when all of the specific condition character strings match. The generating means for generating the AC automaton trie structure, and for each character digit of the trie structure, the score associated with the correct character or the allowable character string that matches the target character string to be searched is The target score string and the target character string are extracted from the storage device, and the characters included in the target character string are collated with a correct character or an allowable character of a specific character digit in the trie structure. When they match, the score corresponding to the correct character or the allowable character is recorded in the score table in association with the matched character digit, and for the next character of the target character string, the next character digit of the trie structure is recorded. When collating the correct character or the allowed character in order to repeat the collation sequentially and when collating the next character of the target character string, when moving to a specific character string of the other conditional character string based on the failer transition, The difference between the number of digits of the transition source and the destination of the transition of the failer is identified, and the score of each character digit recorded in the score table is changed from the currently associated character digit to the difference of the identified digit number When the character matching is made in all the character digits of any of the condition character strings, the sum of the points recorded in the score table is calculated and the total score is calculated. It is a requirement to provide output means for outputting the similarity between the condition character string and the character string included in the target character string.

  According to this search device, it is possible to realize an ambiguous search without increasing the number of states of the AC machine, and to accurately determine the degree of similarity of the keywords that are the matching results.

FIG. 1 is a diagram illustrating an example of a data structure of a conventional AC machine. FIG. 2 is a diagram illustrating an example of a data structure of a conventional state structure. FIG. 3 is a flowchart showing a processing procedure for constructing a conventional AC machine. FIG. 4 is a flowchart showing a processing procedure of a conventional pattern set 構築 tri-T construction process. FIG. 5 is a flowchart showing a processing procedure of a conventional pattern registration process. FIG. 6 is a flowchart (1) showing the processing procedure of the conventional failer transition addition processing. FIG. 7 is a flowchart (2) showing the processing procedure of the conventional failer transition addition processing. FIG. 8 is a diagram (1) for explaining the conventional tri-T construction process for the pattern set Π. FIG. 9 is a diagram (2) for explaining the conventional pattern set ト ラ イ tri-T construction process. FIG. 10 is a diagram (3) for explaining the conventional pattern set 処理 tri-T construction process. FIG. 11 is a diagram (1) for explaining the conventional failer transition addition process. FIG. 12 is a diagram (2) for explaining the conventional failer transition addition process. FIG. 13 is a diagram (3) for explaining the conventional failer transition addition process. FIG. 14 is a diagram (4) for explaining the conventional failer transition addition process. FIG. 15 is a diagram (5) for explaining the conventional failer transition addition process. FIG. 16 is a diagram (6) for explaining the conventional failer transition addition process. FIG. 17 is a diagram (7) for explaining the conventional failer transition addition process. FIG. 18 is a diagram (8) for explaining the conventional failer transition addition process. FIG. 19 is a diagram (9) for explaining the conventional failer transition addition process. FIG. 20 is a flowchart showing a processing procedure of conventional pattern matching processing. FIG. 21 is a diagram (1) for explaining the conventional pattern matching process. FIG. 22 is a diagram (2) for explaining the conventional pattern matching process. FIG. 23 is a diagram (3) for explaining the conventional pattern matching process. FIG. 24 is a diagram (4) for explaining the conventional pattern matching process. FIG. 25 is a diagram (5) for explaining the conventional pattern matching process. FIG. 26 is a diagram (6) for explaining the conventional pattern matching process. FIG. 27 is a diagram illustrating an example of a conventional AC machine that accepts an ambiguous pattern. FIG. 28 is a diagram (1) for explaining the problems of the prior art. FIG. 29 is a diagram (2) for explaining the problems of the prior art. FIG. 30 is a diagram for explaining the outline of the search device according to the present embodiment. FIG. 31 is a diagram illustrating the configuration of the search device according to the present embodiment. FIG. 32 is a diagram illustrating an example of the data structure of the cost management table according to the present embodiment. FIG. 33 is a diagram illustrating an example of the data structure of the state structure according to the present embodiment. FIG. 34 is a diagram illustrating an example of the data structure of the score array according to the present embodiment. FIG. 35 is a diagram (1) for explaining the tri-T construction process according to the present embodiment. FIG. 36 is a diagram (2) for explaining the tri-T construction process according to the present embodiment. FIG. 37 is a diagram (3) for explaining the tri-T construction process according to the present embodiment. FIG. 38 is a diagram (1) for explaining the failer transition addition processing according to the present embodiment. FIG. 39 is a diagram (2) for explaining the failer transition addition processing according to the present embodiment. FIG. 40 is a diagram (3) for explaining the failer transition addition processing according to the present embodiment. FIG. 41 is a diagram (4) for explaining the failer transition addition processing according to the present embodiment. FIG. 42 is a diagram (5) for explaining the failer transition addition processing according to the present embodiment. FIG. 43 is a diagram (6) for explaining the failer transition addition processing according to the present embodiment. FIG. 44 is a diagram (7) for explaining the failer transition addition processing according to the present embodiment. FIG. 45 is a diagram (8) for explaining the failer transition addition processing according to the present embodiment. FIG. 46 is a diagram (9) for explaining the failer transition addition processing according to the present embodiment. FIG. 47 is a diagram (1) for explaining the pattern matching processing according to the present embodiment. FIG. 48 is a diagram (2) for explaining the pattern matching processing according to the present embodiment. FIG. 49 is a diagram (3) for explaining the pattern matching processing according to the present embodiment. FIG. 50 is a diagram (4) for explaining the pattern matching processing according to the present embodiment. FIG. 51 is a diagram (5) for explaining the pattern matching processing according to the present embodiment. FIG. 52 is a diagram (6) for explaining the pattern matching processing according to the present embodiment. FIG. 53 is a diagram illustrating an example of a collation result output by the collation result output unit. FIG. 54 is a flowchart of a process procedure in which the search device 100 according to the present embodiment constructs an AC machine. FIG. 55 is a flowchart of a process procedure of a process for constructing a trie T of a pattern set according to the present embodiment. FIG. 56 is a flowchart of the process procedure of the pattern registration process according to the present embodiment. FIG. 57 is a flowchart (1) illustrating the processing procedure of the failer transition addition processing according to the present embodiment. FIG. 58 is a flowchart (2) illustrating the processing procedure of the failer transition addition processing according to the present embodiment. FIG. 59 is a flowchart of the verification process according to the present embodiment. FIG. 60 is a diagram illustrating a hardware configuration of a computer corresponding to the search device illustrated in the embodiment.

  Hereinafter, embodiments of the present invention (name of the invention) will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  First, a conventional AC machine (AC automaton) will be described before describing this embodiment. FIG. 1 is a diagram illustrating an example of a data structure of a conventional AC machine. The AC machine shown in FIG. 1 is an AC machine that calculates the position of the keywords “AC, BA, BB, BAA, BACD” when text data is input.

  As shown in FIG. 1, the AC machine has states 0 to 8, and each state is classified into an initial state, a normal state, and a collation state. The initial state is a state in which text data is first collated, and the normal state is a state in which text data is collated second and later. The collation state is a state that transitions when the text data hits a specific keyword. In the example shown in FIG. 1, the initial state is state 1, the normal state is states 1 and 3, and the collation state is states 2 and 4-8.

  The AC machine sequentially checks the characters of the text data from the initial state 0, and repeats the normal transition and the failure transition, so that the text included in the text data (for example, the text included in the set 図 in FIG. 1) can be obtained. The appearance position is calculated.

  Here, the normal transition indicates a transition when there is a transition destination by a character to be compared with text data in a state to be collated. For example, in state 1, when the character to be compared with text data is C, a normal transition is made. When the character of the text data is C, a normal transition is made from state 1 to state 2.

  On the other hand, if the conditions for the normal transition are not met, a failer transition occurs. For example, in the state 1, when the character to be compared with the text data is other than C, a failer transition is made. When a failer transition occurs in state 1, state 0 is entered. In the AC machine, there is one failer transition in each state. It is assumed that the failer transitions omitted in FIG. 1 are all directed to the initial state 0.

  Here, the data structure of the state (state structure) of the AC machine shown in FIG. 1 will be described. FIG. 2 is a diagram illustrating an example of a data structure of a conventional state structure. As shown in FIG. 2, this state structure has a state ID for identifying each state, a pattern list, a pointer to a normal transition destination, and a pointer to a failer transition destination.

  Here, the pattern list stores a corresponding keyword when the state structure is in a collation state. For example, the pattern list included in the state structure of state 2 (state ID 2) is AC.

  Next, a processing procedure for constructing an AC machine by a conventional device (hereinafter, a conventional device; not shown) will be described. FIG. 3 is a flowchart showing a processing procedure for constructing a conventional AC machine. As shown in FIG. 3, the conventional apparatus acquires a pattern set Π (a set of keywords to be searched) (step S10), and executes a process of constructing a trie T for the pattern set （(step S11). Then, the conventional apparatus executes a failer transition addition process (step S12), and outputs an AC machine α (Π) (step S13).

  Next, the trie tree construction process for the pattern set 示 し shown in step S11 of FIG. 3 will be described. FIG. 4 is a flowchart showing a processing procedure of a conventional process for constructing a trie T of a pattern set IV. As shown in FIG. 4, the conventional apparatus creates an initial state (id = 0), and sets a trie T (Π) to a trie that includes only the initial state (step S20).

  The conventional apparatus sets all the normal transition destination pointers in the initial state to the initial state (id = 0) (step S21), and determines whether a pattern exists in the pattern set （(step S22). If the pattern does not exist (step S23, No), a trie T (Π) is output (step S24).

  On the other hand, if a pattern exists in the pattern set （(step S23, Yes), one pattern is extracted from the pattern set 、, the extracted pattern is set as the pattern p (step S25), and pattern registration processing is executed. (Step S26).

  Next, the pattern registration process shown in step S26 of FIG. 4 will be described. FIG. 5 is a flowchart showing a processing procedure of a conventional pattern registration process. As shown in FIG. 5, the conventional apparatus sets the state s to the initial state of the trie T (step S30), and determines whether or not the next character exists in the pattern p (step S31).

  When the next character does not exist in the pattern p (step S32, No), the conventional apparatus substitutes the pattern p for the pattern list Plist in the state s (step S33), and outputs a trie T (step S34).

  On the other hand, when the next character exists in the pattern p (step S32, Yes), the conventional apparatus sets the next character as a, sets the ascii code of a to code (a) (step S35), and the state It is determined whether or not the pointer g [code (a)] to the normal transition destination of s is Null (step S36).

  When the pointer g [code (a)] to the normal transition destination is not Null (step S37, No), the process proceeds to step S39. On the other hand, when the pointer g [code (a)] to the normal transition destination is null (step S37, Yes), the conventional apparatus creates a new state n and points to the normal transition destination g [ code (a)] is set to n (step S38). Then, the conventional device substitutes g [code (a)] for the state s (step S39), and proceeds to step S31.

  Next, the failer transition addition process shown in step S12 of FIG. 3 will be described. 6 and 7 are flowcharts showing a processing procedure of conventional failer transition addition processing. As shown in FIG. 6, the conventional apparatus determines a state that is a normal transition destination from the initial state, and registers the determined state in a queue (step S40).

  Then, the conventional apparatus registers the initial state in the failer transition destination of the state (state structure) registered in the queue (step S41), and determines whether or not the state exists in the queue (step S42). If there is no state in the queue (step S43, No), the trie T is output as the AC machine α (step S44).

  On the other hand, if a state exists in the queue (step S43, Yes), the head state of the queue is extracted, the extracted state is set to state s (step S45), and a pointer to the normal transition destination of state s is set. It is determined whether or not all are null (step S46). When all the pointers to the normal transition destination of the state s are Null (step S47, Yes), the process proceeds to step S42.

  On the other hand, when all the pointers to the normal transition destination in the state s are not null (step S47, No), as shown in FIG. 7, the conventional apparatus g [code (a)] ≠ Null in the state s. Are extracted and stored in the set X (step S48).

  The conventional apparatus determines whether or not a character exists in the set X (step S49). If no character exists in the set X (step S50, No), the process proceeds to step S42. On the other hand, if there is a character in the set X (step S50, Yes), one character a is extracted from the set X, and the character corresponding to the extracted character a is deleted from the set X (step S51).

  The conventional apparatus adds the normal transition destination of the state s to the tail of the queue (step S52), repeats the failer transition, and sets the first state where the normal transition destination for the character a is not null to the state f (step S53). ). The conventional apparatus determines the pointer fnext = g [code (a)] from the state of the failer transition destination to the normal transition destination of the character a (step S54).

  The conventional apparatus determines the state next to be transitioned from the state s by the letter a (step S55), sets the next transition destination of the state next to fnext = g [code (a)] (step S56), and the state next state failer. When the pattern list exists in the transition destination state, the pattern list is added to the pattern list in the state next (step S57), and the process proceeds to step S49.

  Next, the pattern set ト ラ イ tri-T construction process shown in step S11 of FIG. 3 will be described using a specific example. FIGS. 8 to 10 are diagrams for explaining a conventional T-construction process for a pattern set Π. Here, as an example, the pattern set Π is Π = {AC, BA, BB, BAA, BACD}.

  First, the conventional apparatus creates an initial state (id = 0), and sets all normal transition destinations in the state structure in the initial state to the initial state (FIG. 8, step S60). Then, the conventional apparatus extracts the pattern “AC” from the pattern set Π. The conventional apparatus selects the character A and sets the normal transition destination of the initial state (id = 0) by the character A to the normal state (id = 1). Further, the conventional apparatus selects the character C, sets the normal transition destination of the normal state (id = 1) by the character C to the collation state (id = 2), and sets the collation state pattern list to “AC”. (FIG. 8, step S61). Since the character C is the last character of the pattern “AC”, the state 2 is a collation state.

  The conventional apparatus returns to the initial state (id = 0), and extracts the pattern “BA” from the pattern set Π. The conventional apparatus selects the character B and sets the normal transition destination of the initial state (id = 0) by the character B to the normal state (id = 3). Further, the conventional apparatus selects the character A, sets the normal transition destination of the normal state (id = 3) by the character A to the collation state (id = 4), and sets the collation state pattern list to “BA”. (FIG. 8, step S62). Since the character A is the last character of the pattern “BA”, the state 4 is a collation state.

  The conventional apparatus returns to the initial state (id = 0), extracts the pattern “BB” from the pattern set Π, and selects the character B. Here, since the normal transition destination of the initial state (id = 0) by the letter B is the normal state (id = 3) and has already been created, the current state is transitioned to the normal state (id = 3). Further, the conventional apparatus selects the character B, sets the normal transition destination of the normal state (id = 3) by the character B to the collation state (id = 5), and sets the pattern list to “BB” (FIG. 9). Step S63). Since the character B is the last character of the pattern “BB”, the state 5 is a collation state.

  The conventional apparatus returns to the initial state (id = 0), extracts the pattern “BAA” from the pattern set Π, and selects the character B. Here, since the normal transition destination of the initial state (id = 0) by the letter B is the normal state (id = 3) and has already been created, the current state is transitioned to the normal state (id = 3). . Further, the conventional apparatus selects the letter A. Here, since the normal transition destination of the normal state (id = 3) by the character A is the collation state (id = 4) and has already been created, the current state is set to the collation state (id = 4). . Further, the conventional apparatus selects the letter A. The normal transition destination of the collation state (id = 4) by the character A is set to the collation state (id = 6), and the pattern list is set to “BAA” (FIG. 9, step S64). Since the character A is the last character of the pattern “BAA”, the state 6 is a collation state.

  The conventional apparatus returns to the initial state (id = 0), extracts the pattern “BACD” from the pattern set Π, and selects the character B. Here, since the normal transition destination of the initial state (id = 0) by the letter B is the normal state (id = 3) and has already been registered, the current state transitions to the normal state (id = 3). . Further, the conventional apparatus selects the letter A. Here, since the normal transition destination of the normal state (id = 3) by the character A is the collation state (id = 4) and has already been created, the current state is set to the collation state (id = 4). .

  Further, the conventional apparatus selects the character C and sets the normal transition destination of the collation state (id = 4) by the character C to the normal state (id = 7). Further, the conventional apparatus selects the character D, sets the normal transition destination of the normal state (id = 7) by the character D to the collation state (id = 8), and sets the pattern list to “BACD” (FIG. 10). Step S65). Since the character D is the last character of the pattern “BACD”, the state 8 is a collation state. At the stage where step S65 is completed, registration of all patterns included in the pattern set Π ends, and the construction process of the trie T ends.

  Next, the failer transition addition process shown in step S12 of FIG. 3 will be described using a specific example. FIGS. 11-19 is a figure for demonstrating the conventional failer transition addition process. The conventional device determines the state that is the normal transition destination from the initial state (id = 0), registers the determined state in the queue (Queue), and sets the initial state 0 to the failer transition destination of the state registered in the queue. sign up. Here, since the normal transition destination in the initial state is the normal states 1 and 3, 1 and 3 are registered in the queue. Further, the failer transition destination in the normal states 1 and 3 is set to the initial state 0 (see FIG. 11).

  Next, the conventional apparatus takes out state 1 at the head of the queue, and sets the taken-out state 1 to state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the character C and stores the character C in the set X.

  The conventional apparatus extracts the character C from the set X, and adds the state 2 that is the normal transition destination of the state s to the tail of the queue. The conventional apparatus shifts from the state 1 to the initial state 0 that has undergone a failer transition, and determines the normal transition destination for the character C, thereby determining the initial state 0 when determining the failer transition destination of the state next. The conventional apparatus determines the state next to be transitioned by the character C from the state s (normal state 1), and sets the failer transition destination of the determined state (collation state 2) to the initial state 0 (see FIG. 12).

  The conventional apparatus extracts the state 3 at the head of the queue, and sets the extracted state 3 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the characters A and B and stores the characters A and B in the set X.

  The conventional apparatus extracts the character A from the set X, and adds the state 4 that is the normal transition destination of the character A in the state s to the end of the queue. The conventional apparatus shifts from the state s to the initial state (id = 0) in which the transition is made to the state and determines the normal transition destination for the character A, so that the state transitions to the state 1 when the failure transition destination of the state next is determined. The conventional apparatus determines the state next to be transitioned by the character A from the state s (normal state 3), and sets the failer transition destination of the determined state (collation state 4) to state 1 (see state 4 in FIG. 13).

  The conventional apparatus extracts the character B from the set X and adds the state 5 that is the normal transition destination of the character B in the state s (normal state 3) to the tail of the queue. The conventional apparatus shifts from the state s to the initial state 0 in which a failer transition is performed, and determines the normal transition destination for the character B. The conventional apparatus determines the state next to be transitioned from the state s by the letter B, and sets the failer transition destination of the determined state (collation state 5) to state 3 (see state 5 in FIG. 13).

  The conventional apparatus extracts the state 2 at the head of the queue, and sets the extracted state 2 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 14).

  The conventional apparatus extracts the state 4 at the head of the queue, and sets the extracted state 4 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the characters A and C and stores the characters A and C in the set X.

  The conventional apparatus extracts the character A from the set X, and adds the state 6 that is the normal transition destination of the character A in the state s (collation state 4) to the tail of the queue. The conventional apparatus shifts from the state s to the state 1 in which a failer transition is made. In state 1, the normal transition destination for character A is Null, so a failer transition is made again, and an initial state 0 is entered.

  Then, in the initial state 0, when the normal transition destination for the character A is determined, and the failer transition destination of the state next is determined, the state 1 is obtained. The conventional apparatus determines the state next to be transitioned by the letter A from the state s (normal state 4), and sets the failer transition destination of the determined state (collation state 6) to state 1 (see state 6 in FIG. 15).

  The conventional apparatus extracts the character C from the set X and adds the state 7 that is the normal transition destination of the character C in the state s (collation state 4) to the tail of the queue. The conventional apparatus shifts from the state s to the state 1 in which the transition is made to the state 1, and determines the normal transition destination for the character C, thereby determining the state transition to the state next. The conventional apparatus determines the state next to be transitioned from the state s by the character C, and sets the determined transition state (normal state 7) to the state 2 as the failer transition destination.

  In addition, since the failure transition destination of state 7 is collation state 2 in the conventional apparatus, state 7 is set to collation state 7 by adding the pattern list of state 2 to the pattern list of state 7 (FIG. 15). (See state 7).

  The conventional apparatus extracts the state 5 at the head of the queue, and sets the extracted state 5 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 16).

  The conventional apparatus takes out the state 6 at the head of the queue, and sets the taken-out state 6 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 17).

  The conventional apparatus extracts the state 7 at the head of the queue, and sets the extracted state 7 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the character D and stores the character D in the set X.

  The conventional apparatus extracts the character D from the set X, and adds the state 8 that is the normal transition destination of the character D in the state s (collation state 7) to the end of the queue. The conventional apparatus shifts from the state s to the state 2 that has undergone a failer transition. In state 2, the normal transition destination for character D is Null, so a failer transition occurs again, and the state transitions to initial state 0.

  Then, by determining the normal transition destination for the character D in the initial state 0, the initial state 0 is obtained when the failer transition destination of the state next is determined. The conventional apparatus determines the state next to be transitioned by the character D from the state s (collation state 7), and sets the failer transition destination of the determined state (collation state 8) to the initial state 0 (see state 8 in FIG. 18). .

  The conventional apparatus extracts the state 8 at the head of the queue, and sets the extracted state 8 to the state s. The conventional apparatus extracts all the characters a satisfying g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step. Then, when there is no more state in the queue, the AC machine for the pattern set IV is completed (see FIG. 19).

  Next, a text data pattern matching process using a conventional AC machine will be described. FIG. 20 is a flowchart showing a processing procedure of conventional pattern matching processing. As shown in FIG. 20, the conventional apparatus sets the state s to the initial state of the AC machine α (step S70), sets the offset d = 0, and sets the set R to an empty set (step S71).

  The conventional device determines whether the (d + 1) -th character exists in the text data D (step S72). If the (d + 1) -th character does not exist (step S73, No), R is output (step S74).

  On the other hand, when the (d + 1) -th character exists in the text data D (step S73, Yes), the conventional device sets the (d + 1) -th character to the character a (step S75), d = d + 1 is set (step S76).

  The conventional apparatus determines whether or not the normal transition destination of the character a exists in the state s (step S77). When a normal transition destination exists (step S78, Yes), g [code (a)] of the state s is substituted for the state s (step S79), and when the state s is the collation state, the pattern of the state s The list is registered in the set R in association with the offset d (step S80), and the process proceeds to step S72.

  If there is no normal transition destination (No in step S78), the state transition destination of the state s is set to the state s (step S81), and if the state s is the collation state, the pattern list of the state s is offset d. And registered in the set R (step S82), and the process proceeds to step S77.

  Next, the pattern matching process shown in FIG. 20 will be described using a specific example. FIGS. 21 to 26 are diagrams for explaining a conventional pattern matching process. Here, as an example, an AC machine in which the pattern set Π is Π = {AC, BA, BB, BAA, BACD} is used, and the text data D is described as “CBAAC”.

  The conventional apparatus sets the state s to the initial state 0 of the AC machine α, and sets the set R to an empty set (see FIG. 21). The conventional device reads the character C from the text data D and sets the offset to d = 1. Since there is no normal transition destination by the letter C in the state s, the conventional apparatus causes the state s to undergo a failer transition and sets the state s to the initial state 0 of the failer transition destination (see FIG. 22).

  The conventional apparatus reads the character B from the text data D and sets the offset to d = 2. In the state s (initial state 0), the normal transition destination by the letter B is the state 3, so the conventional apparatus sets the state s to the state 3 (see FIG. 23).

  The conventional device reads the character A from the text data D and sets the offset to d = 3. In the state s (normal state 3), the normal transition destination by the letter A is the state 4, so the conventional apparatus sets the state s to the state 4. Since the state s becomes the collation state, the conventional apparatus registers the pattern list “BA” of the state s and the offset “3” in the set R (see FIG. 24).

  The conventional apparatus reads the character A from the text data D and sets the offset to d = 4. In the state s (collation state 4), since the normal transition destination by the character A is the state 6, the conventional apparatus sets the state s to the state 6. Since the state s becomes the collation state, the conventional apparatus registers the pattern list “BAA” of the state s and the offset “4” in the set R (see FIG. 25).

  The conventional device reads the character C from the text data D and sets the offset to d = 5. In the state s (collation state 6), there is no normal transition destination by the character C, so a failer transition is made and the state s is set to the state 1. In the state s, the normal transition destination by the character C is the state 2, so the conventional apparatus sets the state s to the state 2. Since the state s becomes the collation state, the conventional apparatus registers the pattern list “AC” of the state s and the offset “5” in the set R (see FIG. 26).

  The conventional apparatus outputs the set R shown in FIG. 26 as a collation result. Referring to the collation result, it can be seen that the keyword “BA” exists at the offset 3 of the text data, the keyword “BAA” exists at the offset 4, and the keyword “AC” exists at the offset 5.

  Next, a case will be described in which an ambiguous search that does not distinguish similar characters such as uppercase and lowercase letters (for example, A and a) is realized using an AC machine. When an ambiguous search is realized using an AC machine, it may be expressed as an automaton that makes ambiguous characters transition to the same state.

  FIG. 27 is a diagram illustrating an example of a conventional AC machine that accepts an ambiguous pattern. In the example shown in FIG. 27, the pattern set Π = {(A | a) (C | c), (B | b) (A | a), (B | b) (B | b), (B | b) (A | a) (A | a), (B | b) (A | a) (C | c) (D | d)}. Here, (x | y) indicates that x or y may be used. For example, the keywords corresponding to the keywords (A | a) (C | c) included in the pattern set Π include AC, aC, Ac, and ac.

  Here, an example of the state transition of the AC machine shown in FIG. 27 will be described. For example, when the current state is the initial state 0 and the character of the text data D is A or a, the current state is transitioned from the initial state 0 to the normal state 1. As described above, when the fuzzy search is realized by the AC machine, there are two or more normal transition destinations (two in the example shown in FIG. 27). Other state transition methods are the same as those of the conventional AC machine shown in FIG.

  By the way, when the pattern matching process is executed using the AC machine shown in FIG. 27, it is determined how much of the hit keywords (keywords registered in the set R) are uppercase letters and how much are lowercase letters. (Determining the degree of similarity).

  However, if the AC machine shown in FIG. 27 is applied as it is, it cannot be determined through which path on the AC machine the hit keyword has been hit when the pattern matching is completed. In FIG. 27, two states, “A” or “a”, from the initial state 0 to the state 1 before reaching the collation state 2, and “C” or “c” from the state 1 to the collation state 2 are reached. There are two ways.

  Accordingly, there are a total of four routes that reach the matching state 2 (the state in which the keyword (A | a) (C | c) is hit), but since any route reaches the matching state 2, each route is distinguished. It is impossible to determine how much of the hit keywords are uppercase letters and how much are lowercase letters.

  As a means to solve this problem, if you create an AC machine that expands all combinations of candidates for each character, it is possible to determine how much of the hit characters are uppercase letters and how much are lowercase letters. It becomes. For example, an AC machine shown in FIG. 29 may be created by assigning 1 point to upper case letters and 0 points to lower case letters, and assigning points by expanding all characters as shown in FIG. 28 and 29 are diagrams for explaining the problems of the prior art.

  For example, the keyword AC shown in FIG. 28 has two points because it includes two capital letters, the keywords Ac and aC have one point because they include one capital letter, and the keyword ac has zero points because it does not include capital letters. If four routes are prepared for the keyword (A | a) (C | c), it is possible to refer to the points when the keyword (A | a) (C | c) is hit, It becomes possible to determine the degree of capital letters and the degree of small letters.

  However, if all the character strings developed as shown in FIG. 28 are expressed by an AC machine, the number of states of the AC machine becomes enormous as shown in FIG. 29, which is not realistic. The number of states of the AC machine shown in FIG. 29 increases exponentially according to the keyword size.

  Next, an outline of the search device according to the present embodiment will be described. FIG. 30 is a diagram for explaining the outline of the search device according to the present embodiment. As shown in FIG. 30, the search apparatus according to the present embodiment creates an AC automaton that makes ambiguous characters transition to the same state as in FIG. 27, and assigns points to each side of the AC machine. The score is calculated sequentially while changing the state of the automaton.

  Specifically, when transitioning from a certain state to a certain state, if a normal transition is made by capital letters, 1 point is added, and if a normal transition is made by lowercase letters, 0 points are added to calculate the score. I do. For example, in FIG. 30, 1 point is added when the normal transition is made from the initial state 0 to the state 1 by the letter A, and 0 points is added when the normal transition is made from the initial state 0 to the state 1 by the letter a.

  Furthermore, the search device according to the present embodiment corrects the score based on the difference between the level of the transition source state and the level of the transition destination state when a failer transition is made from a certain state to a certain state. Thus, the score at the end of pattern matching is made accurate. Here, the state level indicates the distance from the initial state to the corresponding state.

  When the level of each state is specifically shown using FIG. 30, since the distance from the initial state 0 is 1, states 1 and 3 are level 1. The states 6 and 7 are level 3 because the distance from the initial state 0 is 3. The state 8 is level 4 because the distance from the initial state 0 is 4.

  As described above, the search device according to the present embodiment creates an AC automaton that makes ambiguous characters transition to the same state, assigns points to each side of the AC machine, and sequentially changes the state of the automaton. The points are calculated. Therefore, according to the search device according to the present embodiment, it is possible to determine how much of a hit keyword is uppercase and what is lowercase without expanding all candidate combinations in each character.

Next, terms used in this embodiment will be defined. Σ indicates an alphabet (a set of characters), and an ambiguous character set is a subset of Σ. The ambiguous character set {a, b,...} ⊆Σ is described as (a | b | ...). Arbitrary pε (Σ∪A) ^* is called a pattern. The cost table C is defined as a function that assigns an integer to each character of Σ. The text data D input to the search device is DεΣ ^* . In addition, a set of ambiguous character set and A = Σ ^2.

  As an example, the search device according to the present embodiment accepts text data D, a pattern set Π, and a cost management table for managing points, and for text data D, all the matching positions and matching costs of each pattern p∈Π are obtained. It shall be output as a search result.

  In this embodiment, only patterns satisfying the first to third conditions are handled as the pattern set Π. As a first condition, a character used in an ambiguous character set cannot be used alone. For example, if an ambiguous character set (a | A) is used in a pattern pεΠ, the characters aεΣ and AεΣ cannot be used alone in any pattern of Π.

  As a second condition, the same character cannot be used in a different ambiguous character set used in the pattern in the basket. For example, (a | A) and (a | b) cannot be used simultaneously in a cage. As a third condition, two or more scores cannot be assigned to one ambiguous character set. That is, the score assignment in each ambiguous character set must be common in the heels.

  Next, the configuration of the search device 100 according to the present embodiment will be described. FIG. 31 is a diagram illustrating the configuration of the search device 100 according to the present embodiment. As illustrated in FIG. 31, the search device 100 includes an input unit 110, an output unit 120, an input / output control unit 130, a storage unit 140, and a control unit 150.

  Among these, the input unit 110 is an input unit for inputting a pattern set, a score array, a cost management table, text data, and the like, and corresponds to a keyboard, a mouse, a microphone, and the like. The output unit 120 is an output unit that outputs a collation result by an AC machine, and corresponds to a monitor (or display or touch panel). The input / output control unit 130 is a processing unit that controls input / output of data by the input unit 110, the output unit 120, the storage unit 140, and the control unit 150.

  The storage unit 140 is a storage unit that stores data and programs necessary for various processes performed by the control unit 150. The storage unit 140 includes a pattern set 140a, a cost management table 140b, an AC machine 140c, and a score array 140d.

  Among these, the pattern set 140a is a set of keywords (ambiguous patterns) searched from text data. In this embodiment, as an example, the pattern set 140a is expressed as Π = {(A | a) (C | c), (B | b) (A | a), (B | b) (B | b), (B | B) (A | a) (A | a), (B | b) (A | a) (C | c) (D | d)}.

  The cost management table 140b is a table that stores character types and costs (points) in association with each other. FIG. 32 is a diagram illustrating an example of a data structure of the cost management table 140b according to the present embodiment.

  The AC machine 140c is an AC machine generated based on the pattern set 140a. For example, pattern set Π = {(A | a) (C | c), (B | b) (A | a), (B | b) (B | b), (B | b) (A | a) The AC machine corresponding to (A | a), (B | b) (A | a) (C | c) (D | d)} is the AC machine shown in FIG.

  The AC machine according to the present embodiment is also composed of a plurality of state structures as in the conventional case. FIG. 33 is a diagram illustrating an example of the data structure of the state structure according to the present embodiment. As shown in FIG. 33, the state structure according to this embodiment includes a state ID “id” for identifying each state, a pattern list “plist”, and pointers g [1] to g [256 to normal transition destinations. ], A pointer “fail” to the failer transition destination, and a distance “dist” from the initial state (root node).

  Returning to the description of FIG. 31, the score array 140d is a table that manages the distance from the initial state to the current state, which changes as the state changes on the AC machine, and the score. FIG. 34 is a diagram illustrating an example of a data structure of the score array 140d according to the present embodiment. As shown in FIG. 34, the score array 140d stores a distance from the initial state (a distance from the initial state to the current state) and a score in association with each other.

  The control unit 150 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. As illustrated in FIG. 31, the control unit 150 includes an AC machine construction processing unit 150a, a matching processing unit 150b, and a matching result output unit 150c.

  The AC machine construction processing unit 150a is a processing unit that creates the AC machine 140c based on the pattern set 140a. When the AC machine construction processing unit 150a creates the AC machine 140d, the AC machine construction processing unit 150a executes the failer transition addition processing after first performing the trial construction.

  First, the tri-T construction executed by the AC machine construction processing unit 150a will be specifically described. Here, the pattern set is expressed as Π = {(A | a) (C | c), (B | b) (A | a), (B | b) (B | b), (B | b) (A | A) (A | a), (B | b) (A | a) (C | c) (D | d)} will be described. 35 to 37 are diagrams for explaining the tri-T construction process according to the present embodiment.

  First, the AC machine construction processing unit 150a creates the initial state 0, and sets all the normal transition destinations in the state structure in the initial state to the initial state (FIG. 35, step S90). Then, the AC machine construction processing unit 150a extracts the pattern “(A | a) (C | c)” from the pattern set Π. The AC machine construction processing unit 150a selects the character (A | a), sets the normal transition destination of the initial state 0 by the characters A and a to the normal state 1, and sets dist to “1”.

  The AC machine construction processing unit 150a selects the character (C | c), sets the normal transition destination of the normal state 1 with the characters C and c to the collation state 2, and sets the collation state pattern list to “(A | a)”. (C | c) ”. Since the character (C | c) is the last character of the pattern “(A | a) (C | c)”, the state 2 is a collation state. Further, the AC machine construction processing unit 150a sets the dist in the collation state 2 to “2” (FIG. 35, step S91).

  The AC machine construction processing unit 150a returns to the initial state 0, and extracts the pattern “(B | b) (A | a)” from the pattern set Π. The AC machine construction processing unit 150a selects the character (B | b), sets the normal transition destination of the initial state 0 by the characters B and b to the normal state 3, and sets dist to “1”.

  The AC machine construction processing unit 150a selects the character (A | a), sets the normal transition destination of the normal state 3 by the characters A and a to the collation state 4, and sets the collation state pattern list to “(B | b)”. (A | a) ". Since the character (A | a) is the last character of the pattern “(B | b) (A | a)”, the state 4 is a collation state. Further, the AC machine construction processing unit 150a sets dist in the collation state 4 to “2” (FIG. 35, step S92).

  The AC machine construction processing unit 150a returns to the initial state 0, extracts the pattern “(B | b) (B | b)” from the pattern set Π, and selects the character (B | b). Here, the normal transition destination of the initial state 0 by the characters B and b is the normal state 3, which has already been created, so the current state is transitioned to the normal state 3. The AC machine construction processing unit 150a selects the character (B | b), sets the normal transition destination of the normal state 3 by the characters B and b to the collation state 5, and sets the pattern list to “(B | b) ( B | b) ". Since the character (B | b) is the last character of the pattern “(B | b) (B | b)”, the state 5 is a collation state. Further, the AC machine construction processing unit 150a sets the dist in the collation state 2 to “2” (FIG. 36, step S93).

  The AC machine construction processing unit 150a returns to the initial state 0, extracts the pattern “(B | b) (A | a) (A | a)” from the pattern set Π, and selects the character (B | b). Here, the normal transition destination of the initial state 0 by the characters B and b is the normal state 3 and has already been created, so the current state is transitioned to the normal state 3. Further, the AC machine construction processing unit 150a selects a character (A | a). Here, since the normal transition destination of the normal state 3 by the characters A and a is the collation state 4 and has already been created, the current state is set to the collation state 4. Further, the AC machine construction processing unit 150a selects a character (A | a). The normal transition destination of collation state 4 with characters A and a is set to collation state 6, and the pattern list is set to “(B | b) (A | a) (A | a)”. Since the character (A | a) is the last character of the pattern “(B | b) (A | a) (A | a)”, the state 6 becomes a collation state. Further, the AC machine construction processing unit 150a sets the dist in the collation state 6 to “3” (FIG. 36, step S94).

  The AC machine construction processing unit 150a returns to the initial state 0, extracts the pattern “(B | b) (A | a) (C | c) (D | d)” from the pattern set Π, and performs the character (B | b). Select. Here, the normal transition destination of the initial state 0 by the characters B and b is the normal state 3 and has already been registered, so the current state is transitioned to the normal state 3. Further, the AC machine construction processing unit 150a selects a character (A | a). Here, since the normal transition destination of the normal state 3 by the characters A and a is the collation state 4 and has already been created, the current state is set to the collation state 4.

  Further, the AC machine construction processing unit 150a selects the character (C | c), sets the normal transition destination of the collation state 4 by the characters C and c to the normal state 7, and sets dist to “3”. Further, the AC machine construction processing unit 150a selects the character (D | d), sets the normal transition destination of the normal state 7 by the characters D and d to the collation state 8, and sets the pattern list to “(B | b) ( A | a) (C | c) (D | d) ". Since the character (D | d) is the last character of the pattern “(B | b) (A | a) (C | c) (D | d)”, the state 8 is a collation state.

  Further, the AC machine construction processing unit 150a sets the dist in the collation state 8 to “4” (FIG. 37, step S95). At the stage where step S95 is completed, registration of all patterns (ambiguous patterns) included in the pattern set Π is completed, and the construction process of the trie T is completed.

  Next, the failer transition addition process executed by the AC machine construction processing unit 150a will be described. Note that the AC machine construction processing unit 150a performs a failer transition addition process on the trie created by the above-described trie T construction process.

  38 to 46 are diagrams for explaining the failer transition addition processing according to the present embodiment. The AC machine construction processing unit 150a determines the state that is the normal transition destination from the initial state 0, registers the determined state in the queue, and sets the initial state 0 to the failer transition destination of the state registered in the queue. sign up. Here, since the normal transition destination in the initial state is the normal states 1 and 3, 1 and 3 are registered in the queue. Also, the failer transition destination in the normal states 1 and 3 is set to the initial state 0 (see FIG. 38).

  Next, the AC machine construction processing unit 150a extracts the first state 1 of the queue, and sets the extracted state 1 to the state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the AC machine construction processing unit 150a extracts the character (C | c) and stores the character (C | c) in the set X.

  The AC machine construction processing unit 150a extracts the character (C | c) from the set X, and adds the state 2 that is the normal transition destination of the state s to the tail of the queue. When the AC machine construction processing unit 150a transitions from the state 1 to the initial state 0 in which the failure transition is performed and determines the normal transition destination for the character (C | c), and determines the failure transition destination in the state next, the initial state 0 It becomes. The AC machine construction processing unit 150a determines the state next to be transitioned from the state s (normal state 1) by the character C or c, and sets the failer transition destination of the determined state (collation state 2) to the initial state 0 (FIG. 39).

  The AC machine construction processing unit 150a extracts the top state 3 of the queue and sets the extracted state 3 to state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the characters (A | a) and (B | b) and stores the characters (A | a) and (B | b) in the set X.

  The AC machine construction processing unit 150a extracts the character (A | a) from the set X, and adds the state 4 that is the normal transition destination of the character (A | a) in the state s to the tail of the queue. When the AC machine construction processing unit 150a transitions from the state s to the initial state 0 in which the failure transition is performed and determines the normal transition destination for the character (A | a), and determines the failure transition destination of the state next, the state 1 Become. The AC machine construction processing unit 150a determines the state next to be transitioned by the character A or a from the state s (normal state 3), and sets the failer transition destination of the determined state (collation state 4) to the state 1 (FIG. 40). State 4).

  The AC machine construction processing unit 150a extracts the character (B | b) from the set X, and adds the state 5 that is the normal transition destination of the character (B | b) in the state s (normal state 3) to the tail of the queue. . When the AC machine construction processing unit 150a transitions from the state s to the initial state 0 in which the failure transition is performed and determines the normal transition destination for the character B, the state transitions to the state 3 when the failure transition destination of the state next is determined. The AC machine construction processing unit 150a determines the state next to be transitioned from the state s by the letter B or b, and sets the failer transition destination of the determined state (collation state 5) to the state 3 (see state 5 in FIG. 40). .

  The AC machine construction processing unit 150a takes out state 2 at the head of the queue and sets the taken out state 2 to state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 41).

  The AC machine construction processing unit 150a takes out the state 4 at the head of the queue, and sets the taken-out state 4 to the state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the AC machine construction processing unit 150a extracts characters (A | a) and (C | c) and stores the characters (A | a) and (C | c) in the set X.

  The AC machine construction processing unit 150a extracts the character (A | a) from the set X, and adds the state 6 that is the normal transition destination of the character (A | a) in the state s (collation state 4) to the tail of the queue. . The AC machine construction processing unit 150a shifts from the state s to the state 1 in which a failer transition is made. In state 1, the normal transition destination for character A is Null, so a failer transition is made again, and an initial state 0 is entered.

  Then, in the initial state 0, by determining the normal transition destination for the character (A | a) and determining the failer transition destination of the state next, the state 1 is obtained. The AC machine construction processing unit 150a determines the state next to be transitioned by the character (A | a) from the state s (normal state 4), and sets the failer transition destination of the determined state (collation state 6) to the state 1 ( (See state 6 in FIG. 42).

  The AC machine construction processing unit 150a extracts the character (C | c) from the set X, and adds the state 7 that is the normal transition destination of the character (C | c) in the state s (collation state 4) to the tail of the queue. . The AC machine construction processing unit 150a transitions from the state s to the state 1 in which the transition is made to the state 1 and determines the normal transition destination for the character (C | c). . The AC machine construction processing unit 150a determines the state next to be transitioned from the state s by the character (C | c), and sets the failer transition destination of the determined state (normal state 7) to the state 2.

  Further, the AC machine construction processing unit 150a sets the state 7 to the matching state 7 by adding the pattern list of the state 2 to the pattern list of the state 7 because the failure transition destination of the state 7 is the matching state 2. (See state 7 in FIG. 42).

  The AC machine construction processing unit 150a takes out the top state 5 of the queue and sets the taken out state 5 to the state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 43).

  The AC machine construction processing unit 150a takes out the head state 6 of the queue and sets the taken state 6 to state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step (see FIG. 44).

  The AC machine construction processing unit 150a takes out the head state 7 of the queue and sets the taken state 7 to state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. In this case, the conventional apparatus extracts the character (D | d) and stores the character (D | d) in the set X.

  The AC machine construction processing unit 150a extracts the character (D | d) from the set X, and adds the state 8 that is the normal transition destination of the character (D | d) in the state s (collation state 7) to the tail of the queue. . The AC machine construction processing unit 150a shifts from the state s to the state 2 in which a failer transition is made. In state 2, since the normal transition destination for the character (D | d) is Null, a failer transition occurs again, and the initial state 0 is reached.

  Then, by determining the normal transition destination for the character D in the initial state 0, the initial state 0 is obtained when the failer transition destination of the state next is determined. The AC machine construction processing unit 150a determines the state next to be transitioned from the state s (collation state 7) by the character (D | d), and sets the failer transition destination of the determined state (collation state 8) to the initial state 0. (See state 8 in FIG. 45).

  The AC machine construction processing unit 150a extracts the head state 8 of the queue and sets the extracted state 8 to state s. The AC machine construction processing unit 150a extracts all the characters a that satisfy g [code (a)] ≠ Null in the state s and stores them in the set X. Since there is no normal transition destination in the state s, the process proceeds to the next step. Then, when the state no longer exists in the queue, the AC machine of the pattern set IV is completed (see FIG. 46).

  As described above, the AC machine construction processing unit 150a performs the tri-T construction processing illustrated in FIGS. 35 to 37 and the failer transition addition processing illustrated in FIGS. 38 to 46, thereby performing the AC machine 140c from the pattern set 140a. Create

  Returning to the description of FIG. 31, the collation processing unit 150b is a processing unit that collates text data using the AC machine 140c and outputs a collation result. In addition, each time a normal transition is performed, the collation processing unit 150b registers the score according to the character in the score array 140d every time a normal transition is performed, and totalizes the registered scores. Calculate keyword score. Further, when a failure transition occurs in the middle of the matching process, the matching processing unit 150b generates a score array 140d based on the difference between the dist of the state of the failer transition source and the dist of the state of the failer transition destination. Prepend

  Hereinafter, the processing of the matching processing unit 150b will be specifically described. 47 to 52 are diagrams for explaining the pattern matching processing according to the present embodiment. Here, as an example, the pattern matching process will be described assuming that text data to be collated is “CbaAC”.

  The collation processing unit 150b reads the character C of the first character (offset 1) of the text data and sets the state s to the initial state 0. When the collation processing unit 150b determines the transition destination by the character C in the state s, the normal transition destination by the character C in the state s is the initial state 0, so the state s is set to the initial state 0 again (see FIG. 47). .

  The matching processing unit 150b reads the second character (offset 2) character b of the text data and determines the transition destination by the character b in the state s (initial state 0). Since the normal transition destination of the character b of the state s is the state 3, the state s is set to the state 3. Further, the collation processing unit 150b determines the score in comparison with the cost management table 140b and the character type when the normal transition is made. Then, the matching processing unit 150b registers the score in the score array 140d according to the determined score and the normal transition destination state dist.

  Since the cost of the letter b is “0” and the dist in the state 3 of the normal transition destination is “1”, the collation processing unit 150b calculates the score corresponding to “distance 1 from the initial state” in the score array 140d. Set to 0 (see FIG. 48).

  The matching processing unit 150b reads the third character (offset 3) character a of the text data and determines the transition destination of the character a in the state s (state 3). Since the normal transition destination of the character a in the state s is the state 4, the state s is set to the state 4. Further, since the cost of the letter a is “0” and the dist of the normal transition destination state 4 is “2”, the matching processing unit 150 b corresponds to “distance 2 from the initial state” of the score array 140 d. Set the score to 0.

  Further, since the state 4 is a matching state of the pattern (B | b) (A | a), the matching processing unit 150b determines that the pattern (B | b) (A | a) has been hit. Then, the matching processing unit 150b associates and registers the pattern (B | b) (A | a), the current offset 3, and the total score “0” of the current score array in the set R (see FIG. 49).

  The verification processing unit 150b reads the fourth character (offset 4) of the character A in the text data and determines the transition destination of the character A in the state s (state 4). Since the normal transition destination by the letter A of the state s is the state 6, the state s is set to the state 6. Further, since the cost of the character A is “1” and the dist of the normal transition destination state 6 is “3”, the matching processing unit 150b corresponds to “distance 3 from the initial state” of the score array 140d. Set the score to 1.

  Further, since the state 6 is a collation state of the pattern (B | b) (A | a) (A | a), the collation processing unit 150b performs the pattern (B | b) (A | a) (A | a ). Then, the matching processing unit 150b associates the pattern (B | b) (A | a) (A | a), the current offset 4, and the total value “1” of the scores of the current score array in association with the set R. (See FIG. 50).

  The matching processing unit 150b reads the fifth character (offset 5) character C of the text data and determines the transition destination of the character C in the state s (state 6). Since there is no normal transition destination by the letter C in the state s, a failure transition occurs, and the state 1 that is the failure transition destination is set to the state s.

  Here, the dist of the state 6 as the failer transition source is “3” and the dist of the state 1 as the failer transition destination is “1”, so the difference between the dists is “2”. Therefore, the collation processing unit 150b sets the score of the score array 140d by 2 and sets the score corresponding to “distances 2 and 3 from the initial state” where the score exists to Null (see FIG. 51). ).

  And the collation process part 150b determines the transition destination by the character C of the state s (state 1). Since the normal transition destination by the character C of the state s is the state 2, the state s is set to the state 2. Further, since the cost of the character C is “1” and the dist of the normal transition destination state 2 is “2”, the matching processing unit 150 b corresponds to “distance 2 from the initial state” of the score array 140 d. Set the score to 1.

  Further, since the state 2 is a matching state of the pattern (A | a) (C | c), the matching processing unit 150b determines that the pattern (A | a) (C | c) is hit. Then, the matching processing unit 150b registers the pattern (A | a) (C | c), the current offset 5, and the total score 2 of the current score array in association with each other in the set R (see FIG. 52). ). After determining the transition destination for the last character of the text data, the pattern matching process is terminated.

  In this way, the collation processing unit 150b reduces the score array 140d based on the difference between the failer transition destination and the dist after the transition, so that only the number of points from the initial state 0 to the destination state after the failer transition is obtained. Since it is left in the score array 140d, the score of each verification state can be accurately calculated.

  For example, in FIG. 51, the dist difference between the failer transition destination (state 1) and after the transition (state 6) is 2, so the score array 140d is packed by 2 before, and the failer transition from the initial state 0 Only the score up to the previous state 1 (dist1) is left in the score array 140d (the fact that the transition to state 1 by the letter A is left in the score array 140d). Therefore, in the case of FIG. 51, it is possible to accurately calculate that the normal transition is made from the state 1 to the collation state 2 and the collated pattern (A | a) (C | c) is 2.

  Returning to the description of FIG. 31, the collation result output unit 150 c is a processing unit that outputs the collation result of the collation processing unit 150 b to the output unit 120. For example, when the text data pattern matching process is executed using the AC machine shown in FIG. 52 or the like, the matching result shown in FIG. 53 is output. FIG. 53 is a diagram illustrating an example of a collation result output by the collation result output unit 150c. By referring to the collation result of FIG. 53, it is possible to determine the type of pattern included in the text data, the position of the pattern in the text data, and how much uppercase letters are present.

  Next, a processing procedure of the search device 100 according to the present embodiment will be described. FIG. 54 is a flowchart of a process procedure in which the search device 100 according to the present embodiment constructs an AC machine. As illustrated in FIG. 54, the search device 100 acquires a pattern set Π (step S101), and the AC machine construction processing unit 150a executes a process for building a pattern set ト ラ イ tri-T (step S102). Then, the AC machine construction processing unit 150a executes a failer transition addition process (step S103) and outputs an AC machine (step S104).

  Next, the process of constructing the pattern set ト ラ イ trie T shown in step S102 of FIG. 54 will be described. When the AC machine construction processing unit 150a executes the process for constructing the trie T of the pattern set IV, the trie T is generated from the pattern set 140a (see, for example, FIG. 37).

  FIG. 55 is a flowchart of a process procedure of a process for constructing a trie T of a pattern set according to the present embodiment. As shown in FIG. 55, the AC machine construction processing unit 150a creates an initial state (id = 0), and sets the trie T (Π) to a trie composed only of the initial state (step S201).

  The AC machine construction processing unit 150a sets dist in the initial state to 0 (step S202), sets all normal transition destinations in the initial state to the initial state (step S203), and determines whether or not a pattern exists in the pattern set Π. Is determined (step S204).

  If there is no pattern in the pattern set Π (No in step S205), the AC machine construction processing unit 150a outputs a trie T (Π) (step S206). On the other hand, when a pattern exists in the pattern set （(step S205, Yes), the AC machine construction processing unit 150a extracts one pattern from the pattern set 、 and sets the extracted pattern as the pattern p (step S207). ), Pattern registration processing is executed (step S208), and the process proceeds to step S204.

  Next, the pattern registration process shown in step S208 of FIG. 55 will be described. FIG. 56 is a flowchart of the process procedure of the pattern registration process according to the present embodiment. As shown in FIG. 56, the AC machine construction processing unit 150a sets the state s to the initial state of the trie T (step S301), and sets the distance d to 1 (step S302).

  The AC machine construction processing unit 150a determines whether or not the next character exists in the pattern p (step S303). If the next character does not exist (step S304, No), the pattern p is added to the pattern list plist in the state s. Substitution is performed (step S305), and a trie T is output (step S306).

  On the other hand, if the next character exists in the pattern p (Yes in step S304), the AC machine construction processing unit 150a registers d in the dist of the state s and sets d = d + 1 (step S307). The AC machine construction processing unit 150a determines that the next character is a and whether the character a is an ambiguous character (step S206).

  If the next character is an ambiguous character (step S309, Yes), the AC machine construction processing unit 150a sets the ascii code to code (a) for all characters included in a (step S310), It is determined whether or not the normal transition g [code (a)] of the state s is Null (step S311).

  When the normal transition g [code (a)] of the state s is Null (step S312, Yes), the AC machine construction processing unit 150a creates a new state n and sets g [code (a)] = n It is set (step S311), and g [code (a)] is substituted for state s (state n is set to state s) (step S313). On the other hand, when the normal transition g [code (a)] of the state s is not Null (step S312, No), the process proceeds to step S314.

  By the way, when the next character is not an ambiguous character in step S309 (step S309, No), the ascii code of the character a is set to code (a) (step S315), and the normal transition g [code of the state s It is determined whether (a)] is Null (step S316).

  When the normal transition g [code (a)] of the state s is Null (step S317, Yes), the AC machine construction processing unit 150a newly creates the state n and sets g [code (a)] = n. It is set (step S318), g [code (a)] is substituted for state s (state n is set to state s) (step S319), and the process proceeds to step S303. On the other hand, when the normal transition g [code (a)] of the state s is not Null (step S317, No), the process proceeds to step S319.

  Next, the failer transition addition process shown in step S103 of FIG. 54 will be described. By executing such a failer transition addition process, a failer transition destination is added to each state of the trie T, and the AC machine is completed. FIGS. 57 and 58 are flowcharts illustrating the processing procedure of the failer transition addition processing according to the present embodiment.

  As illustrated in FIG. 57, the AC machine construction processing unit 150a determines a state that is a normal transition destination from the initial state, and registers the determined state in a queue (step S401). However, in step S401, the AC machine construction processing unit 150a does not add the same state to the queue more than once.

  Subsequently, the AC machine construction processing unit 150a registers the initial state in the failer transition destination of the state (state structure) registered in the queue (step S402), and determines whether or not the state exists in the queue (step S402). Step S403). If no state exists in the queue (step S404, No), the trie T is output as the AC machine α (step S405).

  On the other hand, if there is a state in the queue (Yes in step S404), the AC machine construction processing unit 150a extracts the head state of the queue, sets the extracted state to state s (step S406), and state s It is determined whether all the pointers to the normal transition destination are null (step S407). When all the pointers to the normal transition destination of the state s are Null (step S408, Yes), the process proceeds to step S403.

  On the other hand, when all the pointers to the normal transition destination of the state s are not Null (step S408, No), as shown in FIG. 58, the AC machine construction processing unit 150a performs g [code (a )] ≠ Null All characters a are extracted, and the extracted characters a are stored in the set X (step S409).

  The AC machine construction processing unit 150a determines whether or not a character exists in the set X (step S410). If no character exists in the set X (step S411, No), the process proceeds to step S403. On the other hand, if a character exists in the set X (step S411, Yes), one character a is extracted from the set X, and the character corresponding to the extracted character a is deleted from the set X (step S412).

  The AC machine construction processing unit 150a adds the normal transition destination of the state s to the tail of the queue (step S413), repeats the failer transition, and sets the first state where the normal transition destination for the character a is not null to the state f. (Step S414). The AC machine construction processing unit 150a determines the pointer fnext = g [code (a)] from the state of the failure transition destination to the normal transition destination of the character a (step S415).

  The AC machine construction processing unit 150a determines the state next to be transitioned from the state s by the letter a (step S416), sets the failer transition destination of the state next to fnext = g [code (a)] (step S418), If a pattern list exists in the state of the next transition destination of the state next, the pattern list is added to the pattern list of the state next (step S417), and the process proceeds to step S410. However, in step S418, the AC machine construction processing unit 150a does not execute duplicate registration of the same pattern in the pattern list.

  Next, the processing procedure of the collation process executed by the search device according to the present embodiment will be described. FIG. 59 is a flowchart of the verification process according to the present embodiment. As shown in FIG. 59, in the search device 100, the matching processing unit 150b sets the state s to the initial state of the AC machine α (step S501), the offset d = 0, the set R is an empty set, and the score array is an empty array. (Step S502).

  The collation processing unit 150b determines whether or not the (d + 1) -th character exists in the text data D (step S503). If the (d + 1) -th character does not exist in the text data D (No in step S504). , Set R (matching result) is output (step S505).

  On the other hand, when the (d + 1) -th character exists in the text data D (step S504, Yes), the collation processing unit 150b sets the (d + 1) -th character to a (step S506), and d = d + 1. (Step S507).

  The matching processing unit 150b determines whether or not there is a normal transition destination of the character a in the state s (step S508). When the normal transition destination of the character a in the state s does not exist (step S509, No), the failure transition destination in the state s is set to the state s (step S510).

  The collation processing unit 150b prepends the score array 140d by the distance difference (difference between the transition source dist and the transition destination dist) by the failer transition, and sets the back margin after the padding to Null (step) S511). In addition, when the transition processing unit 150b transitions from the failer transition destination by the letter a and the state s is the collation state, the pattern list of the state s, the offset d, and the score are associated and registered in the set R (step S512). ), The process proceeds to step S508. In step S512, the matching processing unit 150b calculates the score by summing the scores of the score array 140d.

  By the way, in step S509, when the normal transition destination of the character a exists in the state s (step S509, Yes), the collation process part 150b substitutes g [code] of the state s for the state s (state s The transition destination by the character a is newly set to the state s) (step S513). The matching processing unit 150b registers the cost of the letter a in the cost corresponding to the dist of the state s (step S514), and when the state s is the matching state, the pattern list of the state s, the offset d, and the score are associated with each other. Then, it registers in the set R (step S515), and proceeds to step S503.

  As described above, when the collation processing unit 150b acquires the text data, the search device 100 according to the present embodiment sequentially compares the AC machine 140c with each character of the text data, and when the normal transition is made, When the score corresponding to the character is registered in the score array 140d and a failer transition is performed, the score array 140d is prepended by the difference between the transition destination and the transition source (dist). Then, when the state transitions to the collation state, the collation processing unit 150b determines the score of the collated keywords by summing the scores (similarities) included in the score array 140d. A fuzzy search can be realized without increasing the number, and the similarity of the keywords that are the matching results can be accurately determined.

  By the way, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  FIG. 60 is a diagram illustrating a hardware configuration of a computer corresponding to the search device 100 described in the embodiment. As shown in FIG. 60, this computer (search device) 10 performs data communication with other devices via an input device 11, a monitor 12, a RAM (Random Access Memory) 13, a ROM (Read Only Memory) 14, and a network. A communication control device 15, a medium reading device 16, a CPU (Central Processing Unit) 17, and an HDD (Hard Disk Drive) 18 are connected by a bus 19.

  The HDD 18 stores an AC machine construction program 18b and a collation program 18c that exhibit functions similar to the functions of the search device 100 described above. When the CPU 17 reads and executes the AC machine construction program 18b and the collation program 18c, the AC machine construction process 17a and the collation process 17b are activated.

  Here, the AC machine construction process 17a corresponds to the AC machine construction processing unit 150a shown in FIG. The collation process 17b corresponds to the collation processing unit 150b and the collation result output unit 150c illustrated in FIG.

  The HDD 18 stores various data 18a corresponding to the data stored in the storage unit 140 shown in FIG. The CPU 17 reads various data 18a stored in the HDD 18 into the RAM 13, and uses the various data 13a stored in the RAM 13 to construct an AC machine and perform pattern matching on text data.

10 Computer 11 Input Device 12 Monitor 13 RAM
13a, 18a Various data 14 ROM
15 Communication Control Device 16 Medium Reading Device 17 CPU
17a AC machine construction process 17b Verification process 18 HDD
18b AC machine construction program 18c collation program 19 bus 100 search device 110 input unit 120 output unit 130 input / output control unit 140 storage unit 140a pattern set 140b cost management table 140c AC machine 140d score array

Claims (3)

In a character string search of a plurality of OR conditions, an ambiguous search is performed in which a correct character included in the character string of the OR condition is collated by setting an allowable character that is not the correct character but is regarded as a condition match. A search device executed by an automaton,
For each character digit of multiple condition character strings, correct characters and allowable characters, and the number of points corresponding to each character are set, and when a specific character digit number of a specific condition character string is matched but a mismatch occurs after that Or a generation means for generating a trie structure of an AC automaton in which a failer transition is set to transition to a character of a specific character digit of another condition character string when all of the specific condition character strings match,
For each character digit of the trie structure, a score table for managing the score associated with the correct character or the allowed character string that matches the target character string to be searched;
The target character string is extracted from the storage device, and the characters included in the target character string are collated with a correct character or an allowable character of a specific character digit in the trie structure. The score corresponding to the correct character or the allowable character is recorded in the score table in association with the character digit, and the next character of the target character string is collated with the correct character or the allowable character of the next character digit of the trie structure. Collating means for sequentially repeating the operation,
When the next character of the target character string is collated, when moving to a specific character string of the other conditional character string based on the failer transition, the number of digits of the character digits of the transition source and the destination of the failer transition A change means for identifying a difference and changing the score of each character digit recorded in the score table to a character digit score obtained by subtracting the difference of the identified digit number from the currently associated character digit;
When all the character digits of any of the condition character strings are matched, the sum of the points recorded in the score table is calculated, and the condition character string and the character string included in the target character string are calculated based on the total score. Output means for outputting the similarity of
A search device comprising: In a character string search for a plurality of OR conditions, an ambiguous search is performed in which AC characters are included in the character string of the conditions and collated by setting an allowable character that is not the correct answer character but is regarded as a condition match. The search method of the search device executed in
The search device has a score table for managing the score associated with the correct character or the allowable character string that matches the target character string to be searched for each character digit of the trie structure,
The search device is
For each character digit in the plurality of condition character strings, correct characters and allowable characters, and the number of points corresponding to each character are set, and even a specific character digit number of a specific condition character string is matched, but then a mismatch occurs. Or a generation step for generating an AC automaton trie structure in which a failer transition is set to transition to a character of a specific character digit of another condition character string when all of the specific condition character strings are matched,
The target character string is extracted from the storage device, and the characters included in the target character string are collated with a correct character or an allowable character of a specific character digit in the trie structure. The score corresponding to the correct character or the allowable character is recorded in the score table in association with the character digit, and the next character of the target character string is collated with the correct character or the allowable character of the next character digit of the trie structure. A verification step that repeats sequentially,
When the next character of the target character string is collated, when moving to a specific character string of the other conditional character string based on the failer transition, the number of digits of the character digits of the transition source and the destination of the failer transition A change step of identifying a difference and changing the score of each character digit recorded in the score table to a character digit score obtained by subtracting the difference of the identified digit number from the currently associated character digit;
When all the character digits of any of the condition character strings are matched, the sum of the points recorded in the score table is calculated, and the condition character string and the character string included in the target character string are calculated based on the total score. Output step to output the similarity of
A search method characterized by inclusion. In a character string search of a plurality of OR conditions, an ambiguous search is performed on a correct character included in the character string of the condition by setting an allowable character that is not the correct character character but considered to be a condition match and performing a matching process. A computer-readable storage medium storing a program to be executed by an AC automaton,
The computer has a score table for managing the score associated with the correct character or the allowable character string that matches the target character string to be searched for each character digit of the trie structure,
In the computer,
For each character digit in the plurality of condition character strings, correct characters and allowable characters, and the number of points corresponding to each character are set, and even a specific character digit number of a specific condition character string is matched, but then a mismatch occurs. Or a generation function for generating a trie structure of an AC automaton that sets a failer transition to transition to a character of a specific character digit of another condition character string when all of the specific condition character strings match,
The target character string is extracted from the storage device, and the characters included in the target character string are collated with a correct character or an allowable character of a specific character digit in the trie structure. The score corresponding to the correct character or the allowable character is recorded in the score table in association with the character digit, and the next character of the target character string is collated with the correct character or the allowable character of the next character digit of the trie structure. A collation function that sequentially repeats
When the next character of the target character string is collated, when moving to a specific character string of the other conditional character string based on the failer transition, the number of digits of the character digits of the transition source and the destination of the failer transition A change procedure for identifying a difference and changing the score of each character digit recorded in the score table to a character digit score obtained by subtracting the difference of the identified digit number from the currently associated character digit;
When all the character digits of any of the condition character strings are matched, the sum of the points recorded in the score table is calculated, and the condition character string and the character string included in the target character string are calculated based on the total score. Output function to output the similarity of
A storage medium on which a program for realizing the above is recorded.

Images