JP2010231272A - Keyword string detection method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To check whether many registered keyword strings are included in an input stream with a comparatively simple configuration even if many keyword strings are to be detected. <P>SOLUTION: A keyword detection unit 20 of first hierarchy detects a keyword (KW) from an input stream STM to output an identifier KW thereof. Then, a keyword string detection unit 30 of second hierarchy detects a keyword string (KWA) composed of a plurality of keywords (KW) to output an identifier KWA thereof. Then, a W/G/B integrated determination unit 40 of third hierarchy determines whether one or more detected keyword string identifiers KWA belonging to the same group are totally any of white W, gray G, or black B, and outputs a result thereof. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a keyword string detection method and apparatus.

  While various information can be obtained and communicated with many people via the Internet, community sites such as social networking services (SNS) and blogs slander individuals, organizations, and public order and morality. If a text that goes against it is written, it becomes a social problem. When a company's bad reputation spreads to the public due to unintentional writing, it is not easy to dispel the image, and it is related to the existence of a company. Therefore, web patrol for monitoring writing on a web page is performed as a business.

  It is inefficient to perform a web patrol by inputting a keyword into a search engine and browsing a web page by clicking a link from a list of results. Web patrol can be automated by software, but it is difficult to process a large amount of information at high speed.

  If keyword search is performed using a regular expression, a large number of simple keywords can be searched with one keyword. A regular expression search can be performed using a non-deterministic automaton (NFA).

  Patent Document 1 below discloses a state machine architecture capable of operating thousands of NFAs in parallel on one chip. It is also disclosed that the NFA has a hierarchical structure.

According to this technique, since high-speed processing is possible, data can be filtered before data on the network is input to the computer.
JP 2005-537550 A

  However, when the write monitoring on the web page is performed by the state machine architecture disclosed in Patent Document 1, the following problem occurs.

  That is, in the embodiment of this document, the number (number of characters) of state transition evaluation symbols of one NFA is the keyword string detection state storage unit 32, and one symbol is 1 byte. When the number of characters in a character set is thousands and one character is 2 bytes as in Japanese, applying this state machine architecture makes one NFA large. For this reason, it is necessary to couple a large number of chips and control them as a whole, which complicates the configuration.

  On the other hand, NFA can theoretically be converted to DFA (deterministic automaton). However, for example, in the case where each of the first to third keywords includes 100 simple keywords and it is checked whether or not the first to third keywords are included in the sentence in this order, in DFA, this one state transition evaluation is performed. The state machine must be configured assuming 100 × 100 × 100 = 1 million state transitions using only the keyword string (pattern). For this reason, for example, when the number of state transition evaluation keyword strings is 100,000, the inappropriate write detection device 100,000, a state explosion occurs when trying to detect the keyword strings by DFA, which is not practical.

  Such a problem occurs not only in inappropriate writing on a Web page, but also in popular writing, surveying market needs on the Internet, sorting and organizing documents, and text mining. A similar problem occurs when each keyword constituting the keyword string is composed of codes other than characters.

  In view of the above problems, an object of the present invention is to check whether or not a large number of registered keyword strings are included in an input stream with a relatively simple configuration even if there are a large number of keyword strings to be detected. It is an object of the present invention to provide a keyword string detection method and apparatus capable of performing the above.

In a first aspect of the present invention, an input stream is supplied, and a keyword string detection device that checks which one of a plurality of registered keyword strings is included in the input stream and outputs an identifier thereof,
A keyword detecting means for checking which one of a plurality of registered keywords is included in the input stream and sequentially outputting identifiers of the matching keywords;
Corresponding to each of the plurality of keyword identifiers, conversion information storage means storing keyword column identifier / in-column position information indicating in which position of the plurality of keyword columns the keyword is included,
A keyword string detection state storage means for storing a current state indicating to which keyword each of the plurality of keyword strings has been detected;
The keyword string identifier / in-column position information corresponding to the keyword identifier detected by the keyword detecting unit is read from the conversion information storage unit, and the keyword is determined based on the read keyword column identifier / in-column position information. The corresponding current state in the column detection state storage means is read, and if this current state corresponds to the in-column position information of this information, this current state is transitioned to the next state, and the current state after the transition is output If it is in a state, it has a state transition control means for outputting a keyword string identifier.

  According to the configuration of the first aspect, the keyword detection means in the first hierarchy detects the keywords included in the input stream and sequentially outputs the identifiers, and the keyword string corresponding to the keyword identifiers in the second hierarchy. The identifier / in-column position information is read from the conversion information storage means, and the corresponding current state in the keyword string detection state storage means is read based on the read keyword string identifier / in-column position information. If the current state corresponds to the position information in the column of this information, this current state is transitioned to the next state, and if the current state after the transition is an output state, a keyword column identifier is output. Even if there are a large number, it is possible to check whether or not a large number of registered keyword strings are included in the input stream with a relatively simple configuration.

  Other objects, configurations and effects of the present invention will become apparent from the following description.

The keyword identifier is represented by KW, the keyword string identifier is represented by KWA, an index is added to each of the plurality of KWs and KWAs to distinguish them, and the identifier target is represented by (identifier). For example, the keyword string (KWA3) indicates that one sentence includes the keywords (KW5), (KWm), and (KW3) in this order, and there are zero or more arbitrary characters between the keywords. It is also good. This uses regular expressions,
(KWA3) = (KW5). * (KWm). * (KW3) (1)
It can be expressed as. “.” Is any single character including a line feed, and * means zero or more repetitions of the immediately preceding character. Note that “.” May be an arbitrary character that does not include a line break, as in a general definition, and the line break may be regarded as the end of one sentence.

  FIG. 1 is a schematic block diagram of an inappropriate writing detection apparatus 10 according to the first embodiment of the present invention.

  This device 10 automatically detects inappropriate writing on a Web page such as a bulletin board or a blog in three layers, and a keyword (KW) is detected from an input stream STM by a keyword detection unit 20 in the first layer. The identifier KW is output, and then the keyword string detection unit 30 in the second hierarchy detects the keyword string (KWA) composed of a plurality of keywords (KW), outputs the identifier KWA, and then outputs the identifier KWA. By the / G / B comprehensive determination unit 40, one or more detected keyword string identifiers KWA belonging to the same group as a whole are white (no problem) W, gray (human judgment required) G or black (inappropriate expression). B is determined and the result is output.

  For example, Black B keyword string (KWA1) = “morphine”. * Even if “I want to buy” is detected, the white W keyword string (KWA2) = “pain treatment”. * "Morphine" is also detected, and if KWA1 and KWA2 are defined as belonging to the same group, it is determined as white W as a whole.

  The keyword detection unit 20 is supplied with Web page write data stored in a memory in the computer or an auxiliary storage device as an input stream STM via the Internet.

  The keyword detection unit 20 includes a finite state automaton storage device 21 having a hardware configuration, for example, and the input stream STM is sequentially supplied to the finite state automaton storage device 21 as an input symbol, for example, in units of bytes in synchronization with the clock. If the process of reading the next state based on the symbol and the current state is repeated and it means that the current state is an output state, that is, one that matches one of the keywords (KW) registered in advance Is included in the input stream STM, the address of the memory storing this output state is output from the keyword detection unit 20 as the keyword identifier KW and supplied to the keyword string detection unit 30.

  The finite state automaton storage device 21 can detect, for example, 100,000 keywords. The keyword may be a simple keyword that does not include a regular expression or a regular expression keyword. For regular expression keywords, the NFA that detects this is expanded into DFA. Further, all the DFAs are combined into one DFA by the AC method (Aho-Corasick algorithm), and the keyword can be detected only by supplying the input stream STM once to the keyword detection unit 20.

  The keyword string detection unit 30 is a characteristic part of the embodiment of the present invention, and includes a conversion information storage unit 31, a keyword string detection state storage unit 32, and a state transition control unit 33.

  The conversion information storage unit 31 includes, for each of all the keywords that can be detected by the keyword detection unit 20, the keyword (KW) included in any position POS in a number of keyword strings (KWA) registered in advance. Is stored.

  FIG. 2 shows a KW / KWA · POS matrix table that visually represents this information. In the cell of this table, a value POS indicating the position where the keyword (KW) corresponding to the row is included is entered in the keyword column (KWA) corresponding to the column.

  It is possible to store the value of the keyword position POS in the column in the conversion information storage unit 31 using the keyword identifier KW and the keyword column identifier KWA as the row address and column address of the conversion information storage unit 31, respectively. Since the useless area corresponding to the cell increases, it is necessary to check whether or not the keyword position POS in the column is stored in each column address (keyword column identifier KWA) for each keyword identifier KW. , Processing time becomes longer.

  Therefore, the conversion information storage unit 31 stores an index table 311 as shown in FIG. 3 and keyword string identifier / in-column position information 312. The word address ADDR = i of the index table 311 stores the keyword identifier KWi, the head address SADRi of the keyword column identifier / in-column position information 312 related thereto, and the number of words WNi from this head address. . For example, in KW2, a keyword column identifier KWA related to KW2 and an in-column keyword position POS are stored in each word from the word address ADDR2 of the keyword column identifier / in-column position information 312 to 4 words.

  In the keyword string detection state storage unit 32 of FIG. 1, as shown in FIG. 4A, addresses corresponding to the values of the identifiers KWA of all registered keyword strings (values that are equal or shifted by a constant value, etc.) In addition, a current state array 321 in which the current state (current detection state) ST is stored and an initial state array 322 in which the initial state ST0 is stored are stored. In FIG. 4A, the current state of the i-th keyword string identifier KWAi is represented by STi, and the initial state is represented by ST0i. For each i value, the address of the current state STi is a value obtained by adding a certain value to that of the initial state STi0.

  Returning to FIG. 1, when the input stream STM indicates the end of the sentence, the reset signal RST supplied to the state transition control unit 33 becomes active. In response to this, the state transition control unit 33 executes the initialization process 332 shown in FIG. 4B for the keyword string detection state storage unit 32. In the following, the step identification codes in the figure are shown in parentheses.

  (S0) The initialization flag F is set.

  (S1) In FIG. 4A, the initial state ST0i is assigned to the current state STi for each of i = 1 to n. This processing may be performed by software via a CPU or by DMAC (direct memory access control).

  Here, as shown in FIG. 1, the state transition control unit 33 includes a KW queue 331, and when the finite state automaton storage device 21 is in the output state as described above, its address is the keyword identifier KW and the keyword detection unit. 20 and input to the KW queue 331 of the state transition control unit 33. A keyword identifier KW may be obtained by shifting this address by a predetermined value or by closing between addresses via a CAM memory.

  FIG. 5 is a flowchart showing the keyword string state transition process 333 by the state transition control unit 33. This keyword string state transition process 333 may be performed by software or hardware.

  (S10) One keyword identifier KW is read from the KW queue 331.

  (S11) If the KW queue 331 is empty (keyword identifier KW is NULL), the process returns to step S10; otherwise, the process proceeds to step S12.

  (S12) The word address ADDR of the index table 311 in which the keyword identifier KWi read in step S10 is stored is detected. This can be detected by, for example, a binary search method (binary search). The index table 311 may be reflected in the memory of the finite automaton device, and this search may be performed by the finite automaton device. Further, the word address ADDR may be detected by one access in the CAM memory.

  (S13) The head address SADRi and the number of words WNi are read from the detected word address ADDR, and are substituted into variables ADR and CNT, respectively. Thereafter, the processes in steps S14 to S23 are repeated WNi times.

  (S14) The keyword column identifier / in-column position information 312 is addressed by ADR, and the keyword column identifier KWA and the in-column keyword position POS are read from the keyword column identifier / in-column position information 312.

  (S15) If F = '1, that is, if the initialization process 332 in FIG. 4B is being executed, the process waits for the process to end and proceeds to step S16; Proceed to step S16.

  (S16) The current state STi is read from the keyword string identifier KWAi in the keyword string detection state storage unit 32.

  (S17) If the read current state STi corresponds to the value of the in-column keyword position POSi read in step S14, the process proceeds to step S18. Otherwise, the process proceeds to step S21.

  Here, a specific example of the expression of the current state ST and the in-column keyword position POS will be described with reference to FIG.

  The maximum number of keywords included in one keyword column (KWA) is 8, and the current state ST and the keyword position POS in the column are each represented by 1 byte. The keyword string identifier KWA corresponds to the address of this byte. For an arbitrary keyword string (KWA), in the case of (KWA3) in the above expression (1), the keyword (KW3) corresponds to the most significant bit (MSB) of 1 byte, and then j previous The keyword is made to correspond to (7-j) bits. For example, only the corresponding bit is set to ‘1’ and all other bits are set to ‘0’. That is, the initial state ST03 of the current state ST3 of KWA3 is represented as “00100000”.

  (S18) The current state ST is shifted by 1 bit to the MSB side.

  In the case of the above example, as shown in FIG. 6B, if the keyword (KW5) is detected and ST3 = POS, the current state ST is shifted by 1 bit to the MSB side, and ST3 = '01000000' To do. If the keyword (KWm) is detected when ST3 is this value, the current state ST is shifted by 1 bit to the MSB side, and ST3 = '10000000'. If the keyword (KW3) is detected when ST3 is this value, the current state ST is shifted by 1 bit to the MSB side, and ST3 = '10000000'. At this time, the carry C becomes “1”.

  (S19) If carry C is "1", the process proceeds to step S0, and if not, the process proceeds to step S21.

  (S20) In order to indicate that the keyword string (KWA) corresponding to the keyword string identifier KWA has been detected, the keyword string identifier KWA is output and supplied to the W / G / B comprehensive determination unit 40.

  (S21) The counter CNT is decremented by 1.

  (S22) If CNT <0, the process returns to step S10; otherwise, the process proceeds to step S23.

  (S23) The word address ADR is incremented by 1, and the process returns to step S14.

  In this way, it is possible to detect that the input stream STM includes a keyword string that matches any of a large number of registered keyword strings (KWA) with a relatively simple configuration.

  The W / G / B comprehensive determination unit 40 stores, for each of all the keyword strings (KWA) registered in the storage unit, the keyword string identifier KWA and whether it is white W, gray G, or black B. And the group identification ID to which the keyword string identifier KWA belongs are stored correspondingly. Based on the keyword string identifier KWA of the same group ID, whether the detected keyword string identifier KWA is W or G It is comprehensively determined whether it is B or B. For example, if a keyword column identifier KWA of B is included in a plurality of keyword column identifiers KWA in the same group but a keyword column identifier KWA of W is included, a plurality of keyword column identifiers KWA included in this group are included. Is determined to be W as a whole.

  When the writing on the Web page is a bulletin board, since one sentence is generally short, the reset signal RST is frequently activated. For this reason, when the number of keyword string identifiers KWA is as large as 100,000, for example, the execution time ratio of the initialization process 332 in FIG. 4B becomes relatively large. On the other hand, in such a case, it is considered that the ratio that the current state is the initial state is relatively large immediately before the reset, and therefore the initialization process 332 is wasted.

  Therefore, in the second embodiment of the present invention, the keyword string detection state storage unit 32 of FIG. 4A further includes an initialized bip map 323 as shown in FIG. 8, and responds to the activation of the reset signal RST. Only the initialized VIP map 323 is initialized, for example, cleared to zero.

  If the first byte address of the VIP map 323 is A0, the j-th bit of the byte address A = A0 + i of the VIP map 323 corresponds to the keyword column identifier KWA (8 * i + j) of the current state array 321. If the bit is '0', only the current state ST (8 * i + j) is initialized. That is, the process shown in FIG. 7 is executed instead of step S15 in FIG.

  (S150) As described above, the bit corresponding to the keyword string identifier KWA is read from the initialized bip map 323.

  (S151) If this value is ‘0’, the process proceeds to step S152; otherwise, the process proceeds to step S16 in FIG.

  (S152) The initial state ST0 is read from the address obtained by adding a predetermined address to the keyword string identifier KWA, and is substituted for the current state ST of the address KWA, and the process proceeds to step S16.

  Other points are the same as in the first embodiment.

  In the first and second embodiments, the case where the counter in the current state ST is configured by a shift counter has been described. However, the counter may be configured by a binary counter. If the maximum number of keywords included in one keyword column (KWA) is 7, for example, this counter can be configured with 3 bits, and two counters can be stored in one byte. When accessing in units of bytes, 2 bits are wasted, so 64 bits are accessed as one word. In this case, since 3 × 21 = 63, the remainder is 1 bit, and this is MSB.

  FIG. 9 shows the current state array 321A of the keyword string detection state storage unit 32A having such a configuration. In the current state array 321A of FIG. 9, the hatched lines written in the MSB cells indicate this remainder, and only the transition from the initial state indicates the count. The address of the 3-bit counter is specified by the address of the word including this counter and the address of the counter bit in the word. The initial state array 322A also has a configuration corresponding to the current array 321A.

  In step S18 of FIG. 5, instead of shifting the current state ST by 1 bit, the current state ST is incremented by one. The initialization of the current state ST is performed in the same manner as in the first or second embodiment in units of 64 bits.

  Other points are the same as in the first embodiment.

  According to the third embodiment, the storage capacity of the keyword string detection state storage unit 32A can be reduced, which is particularly suitable when the number of keyword string identifiers KWA is large.

  In the second embodiment, one bit of the initialized bip map 323 is associated with one keyword string identifier KWA. However, in the third embodiment, 21 states correspond to one word = 64 bits. When applying the initialized bip map 323 of the second embodiment, it is more efficient to initialize the current state in units of words.

  Therefore, in the fourth embodiment of the present invention, one bit of the initialized Bip map 323A shown in FIG. 10 is associated with each word, that is, for each 21 current states ST, to activate the reset signal RST. In response, only the initialized bip map 323A is initialized, and the same processing as in FIG. 7 is performed in units of words instead of step S15 in FIG.

  The MSB of each word of the initialized Bipmap 323A is a valid bit VB. If VB = '0', it means that the remaining 63 bits excluding the valid bit VB of this word are all '0'. If VB = “1”, it means that at least one of the 63 bits is “1”. First, the outline of necessity of initialization can be determined from the value of the effective bit VB. The keyword string detection state storage unit 32B has the same configuration as the initialized bip map 323A and the initial state array 322A and the current state array 321A of FIG.

  The k-th bit of the initialized Bip map 323A excluding the MSB corresponds to the 21 current states ST of the k-th word from the top of the current state array 321A, and the Bip map 323A and the current state array 321A. When the first word address of each is represented by WA0 and WA1, the jth bit of the word address (WA0 + i) of the VIP map 323A is the 21 current states of the word of the word address (WA1 + i * 63 + j) of the current state array 321A. Corresponds to ST.

  If the j-th bit is “0” in step S151 in FIG. 7, in step S151, the value of the word address (WA2 + i * 63 + j) of the initial state array 322A is changed to the word address (WA1 + i * 63 + j) of the current state array 321A. And 21 current states ST are initialized at the same time. Here, WA2 is the leading word address of the initial state array 322A.

  Further, in place of the keyword string identifier / in-column position information 312 shown in FIG. 3, a keyword string identifier / in-column position information 312A as shown in FIG. 11 is used. For example, the information block 312-2 in FIG. 3 corresponds to the information block 312A-2 in FIG. The information block 312A-2 includes a valid POS bitmap 313 and a POS word group 314.

  The bit map 313 is structurally identical to the bit map 323A in form. However, the bitmap 313 is not related to initialization, and the bit ‘1’ includes 21 POS (POS words) corresponding to the word of the current state array 321 </ b> A corresponding to the position in the POS word group 314. It means that

  The POS word group 314 includes only the POS word corresponding to “1” among the bits excluding the MSB of the bitmap 313, and the kth “1” for the bit group of only “1” in the valid POS bitmap 313. Corresponds to the kth word of the POS word group 314.

  More specifically, when the leading word addresses of the bitmap 313 and the POS word group 314 are represented by WA3 and WA4, respectively, the j-th bit of the word address (WA3 + i) is k-th for only “1” on the bitmap 313. POS word group 314 word address (WA4 + k-1) POS word is 21 words address (WA1 + i * 63 + j) of current state array 321A. Corresponds to the current state ST. This POS word is valid only for a POS whose value is not 0, and the valid POS is compared with the corresponding current state ST in step S17 of FIG.

  Since the number of words WN in FIG. 3 is known from the bitmap 313, the number of words WN in FIG.

  In the fourth embodiment, in particular, when the number of keyword string identifiers KWA corresponding to one head address SADR in the index table 311 is relatively large and close to each other, a large number of POSs are stored in one word of the POS word group 314. Since processing can be performed, high-speed processing is possible by performing this processing in parallel, particularly with hardware. Even when processing is performed by software, the number of memory accesses can be reduced and the amount that can be processed by a register in the CPU is increased, so that the processing speed can be increased.

  Although the preferred embodiment of the present invention has been described, the present invention includes various other modifications, and realizes other combinations of the components described in the above-described embodiments and the functions of the components. Other configurations using other configurations, and other configurations that would be conceived by those skilled in the art from these configurations or functions, are also included in the present invention. For example, the present invention includes the following modifications.

  For example, both the keyword string identifier / in-column position information 312 and the keyword string detection state storage unit 32 of Example 1 or 2 and the keyword string identifier / in-column position information 312A and the keyword string detection state storage unit 32B of Example 4 are both used. If the number of keyword strings (KWA) including one keyword (KW) is equal to or less than a set value, the keyword string identifier / in-column position information 312 and the keyword string detection state storage unit 32 of the first or second embodiment are used. If the set value is exceeded, the configuration using the keyword string identifier / in-column position information 312A and the keyword string detection state storage unit 32B of the fourth embodiment may be used.

  Further, the current state ST may be configured such that a down counter is used instead of the up counter, and a predetermined value, for example, 0 is output.

  Furthermore, a configuration may be adopted in which one of the current state ST and the position POS is a shift counter and the other is a binary counter, and the control unit determines whether or not both are substantially matched (corresponding).

  In any of the above embodiments, the case where the present invention is applied to the inappropriate writing detection device 10 has been described. However, a device that detects writing with a good reputation, a device that classifies and organizes sentences and documents, and a desired expression. It can also be applied to devices that detect market needs and perform market needs surveys, text mining devices, and the like. Furthermore, the present invention can also be applied to binary data other than text.

1 is a schematic block diagram of an inappropriate writing detection apparatus according to a first embodiment of the present invention. FIG. 2 is a diagram showing a keyword identifier KW / keyword column identifier KWA / in-column position POS matrix table that visually represents information logically stored in a conversion information storage unit of FIG. 1. It is structure explanatory drawing of the information stored in the conversion information storage part of FIG. (A) is structure explanatory drawing of the information stored in the keyword sequence detection state memory | storage part of FIG. 1, (B) is a flowchart which shows the present state initialization process by the state transition control part of FIG. It is a flowchart which shows the keyword sequence state transition process by the state transition control part of FIG. (A) is a specific example explanatory diagram of the expression of the current state ST and in-column keyword position POS, and (B) is a transition explanatory diagram of the current state ST by the in-column keyword position POS. It is a partial flowchart which shows the present state initialization process which concerns on Example 2 of this invention. It is explanatory drawing of the initialized bip map used by this present state initialization process. It is structure explanatory drawing of the information stored in the keyword sequence detection state memory | storage part which concerns on Example 3 of this invention. It is structure explanatory drawing of the information stored in the keyword sequence detection state memory | storage part which concerns on Example 4 of this invention. It is structure explanatory drawing of the information stored in the keyword sequence identifier and the position information storage part in a column based on Example 4 of this invention.

DESCRIPTION OF SYMBOLS 10 Improper write detection apparatus 20 Keyword detection part 21 Finite state automaton storage apparatus 30 Keyword sequence detection part 31 Conversion information storage part 311 Index table 312, 312A Keyword sequence identifier and in-column position information storage unit 312-2, 312A-2 Information Block 313 Valid POS bitmap 314 POS word group 32, 32A, 32B Keyword string detection state storage unit 321 321A Current state array 322 322A Initial state array 323 323A Initialized VIP map 33 State transition control unit 331 KW queue 332 Initialization process 333 Keyword string state transition process 40 W / G / B comprehensive determination unit F Initializing flag SADR, SADRi Start address WN, WNi Number of words ADDR Word address C Carry KW, K i keyword identifier KWA, Kwai keyword string identifier POS, POSi columns in keyword position ST, STi current state ST0, ST0i initial state VB valid bit

Claims (10)

In a keyword string detection device that is supplied with an input stream and checks which one of a plurality of registered keyword strings is included in the input stream and outputs the identifier thereof,
A keyword detecting means for checking which one of a plurality of registered keywords is included in the input stream and sequentially outputting identifiers of the matching keywords;
Corresponding to each of the plurality of keyword identifiers, conversion information storage means storing keyword column identifier / in-column position information indicating in which position of the plurality of keyword columns the keyword is included,
A keyword string detection state storage means for storing a current state indicating to which keyword each of the plurality of keyword strings has been detected;
The keyword string identifier / in-column position information corresponding to the keyword identifier detected by the keyword detecting unit is read from the conversion information storage unit, and the keyword is determined based on the read keyword column identifier / in-column position information. The corresponding current state in the column detection state storage means is read, and if this current state corresponds to the in-column position information of this information, this current state is transitioned to the next state, and the current state after the transition is output A state transition control means for outputting a keyword string identifier if it is in a state;
A keyword string detection device comprising: The conversion information storage means includes an index table storage unit and a keyword column identifier / in-column position information storage unit,
The index table storage unit stores the head address and word number information of the information block in the keyword column identifier / in-column position information storage unit corresponding to each of the keyword identifiers,
The information block includes a keyword string identifier / in-column position information indicating in which position of a plurality of keyword strings the keyword identified by the corresponding keyword identifier is included, and the number of words indicated by the word number information. Have
The keyword string detection apparatus according to claim 1. The keyword string detection state storage means stores a current state array in which an address corresponding to each keyword string identifier stores as a current state a counter indicating which keyword of the keyword string identified by the keyword string identifier has been detected. Having
The keyword string detection apparatus according to claim 1 or 2, wherein   4. The keyword string detection apparatus according to claim 3, wherein an address corresponding to each keyword string identifier is represented by a word address and a position of an in-word counter. The keyword string detection state storage means further includes an initial state array corresponding to the current state array,
The state transition control means assigns a part or all of the contents of the initial state array to a corresponding part or all of the current state array at a predetermined timing, and changes the contents of the counter to make a transition to the next state. When it is changed, the counter becomes a predetermined value or when the carry from the counter is set, the output state is determined.
The keyword string detection apparatus according to claim 3 or 4, wherein It further includes a bitmap for determining necessity of initialization in which each word including one or more current states corresponds to one bit,
The state transition control means initializes the initialization necessity determination bitmap in response to the activation of the supplied reset signal, and reads the contents of the word including the one or more current states. If the corresponding bit of the initialization necessity determination bitmap is initialized, the word is initialized with the corresponding word in the initial state and the bit is inverted.
The keyword string detection device according to any one of claims 1 to 5, wherein Each information block
A current state word group including one or more counter words that include a plurality of counters and can determine whether the counter is valid or invalid by a count value;
A multiple counter word presence / absence bitmap indicating that each of the multiple counter words corresponds to 1 bit, and the bit corresponding to each of the multiple counter words included in the current state word group is valid;
6. The keyword string detection device according to claim 3, wherein the keyword string detection device includes: In a keyword string detection method in which an input stream is supplied, and which of a plurality of registered keyword strings is included in the input stream and an identifier thereof is output,
Causing the keyword detection means to check which of the plurality of registered keywords is included in the input stream, and sequentially output matching keyword identifiers;
For state transition control means,
Corresponding to each of the plurality of keyword identifiers, conversion information storage means storing keyword column identifier / in-column position information indicating which position of the plurality of keyword columns the keyword is included in, The keyword string identifier corresponding to the keyword identifier detected by the keyword detection means and the position information in the string are read out,
Based on the read keyword string identifier and in-column position information, the corresponding current state in the keyword string detection state storage means for storing the current state indicating up to which keyword each of the plurality of keyword strings has been detected. If the current state corresponds to the position information in the column of this information, the current state is transitioned to the next state, and if the current state after the transition is the output state, the keyword column identifier is output. ,
A keyword string detection method characterized by the above. The keyword string detection state storage means stores a current state array in which an address corresponding to each keyword string identifier stores as a current state a counter indicating which keyword of the keyword string identified by the keyword string identifier has been detected. Have
The state transition control means causes a part or all of the contents of the initial state array corresponding to the current state array to be assigned to the corresponding part or all of the current state array at a predetermined timing, and transits to the next state. To change the contents of the counter in order to make the counter become a predetermined value or to determine the output state when the carry from the counter is set,
The keyword string detection method according to claim 8. It further includes a bitmap for determining necessity of initialization in which each word including one or more current states corresponds to one bit,
When the state transition control means initializes the initialization necessity determination bitmap in response to the activation of the supplied reset signal, and reads the contents of the word including the one or more current states If the corresponding bit of the initialization necessity determination bitmap is initialized, the word is initialized with the corresponding initial state word and the bit is inverted.
The keyword string detection method according to claim 8 or 9, characterized in that: