WO2008141519A1

WO2008141519A1 - Method and chip structure for matching multi-character string

Info

Publication number: WO2008141519A1
Application number: PCT/CN2008/000293
Authority: WO
Inventors: Tian Song
Original assignee: Beijing Zhean Technology Corporation
Priority date: 2007-05-18
Filing date: 2008-02-03
Publication date: 2008-11-27
Also published as: CN101051321A; CN100495407C

Abstract

A method for matching multi-character strings based on a cache state machine as well as a chip structure for matching multi-character strings are disclosed, wherein the chip structure is realized by a method and structure for searching a next state. The method for matching multi-character strings searches the next state in a state transformation rule database based on input characters, a current state and cached states then jumps, and caches the state by specific cache rules. In the chip structure for matching multi-character strings, a main memory which includes basic transformation rules and n step cross transformation rules and an input translation table are shared by two paths of a state register, a color register and a state cache, a color cache, so as to calculate the possible next state and acquire corresponding input characters. An auxiliary memory, which stores fail and restart transformation rules, is used to acquire the next state corresponding to the actual input and updates the state cache and the color cache. Tri-state selector implements multi-way selection of the next state based on the actually input character and the character corresponding to the possible next state, so as to update the state registers and color registers.

Description

Multi-string matching method and chip structure

The present invention relates to a method and a chip structure for information processing, and in particular to a multi-character string matching method and chip structure. Background technique

Multi-string matching technology, also known as multi-keyword matching technology, has matured and is widely used in many fields such as text processing and content filtering. The technology can find one or more of a predefined set of strings in one-dimensional content to be matched, and in the process of matching text, fully utilize the features in a set of strings, perform pre-processing, and according to the pre- The processed intermediate data structure performs content matching to achieve parallel matching of a set of predefined strings. .

In the field of network security, there is a class of content-based security applications that require the use of multi-string matching technologies such as network intrusion detection and prevention systems, spam filtering, virus scanning and filtering, malicious code scanning and filtering, and content filtering. The typical use of this type of application for multi-string matching techniques is to capture packets from the network and restore them to specific network layer data, based on pre-defined rule sets (eg, intrusion rules, virus rules, garbage). Mail rules, etc.), matching in the data. In most cases, this match utilizes multi-character string matching techniques.

Due to the rapid development of network bandwidth, in order to meet the needs of content security applications under gigabit or higher network bandwidth, the demand for high performance multi-string matching technology is urgent. In order to continuously improve the matching performance of multi-string matching technology, there are many improved software algorithms. Although the improved algorithm matching performance is improved, the improvement is still very limited, and the performance can be improved by 20%-40. %. Implementing the above existing algorithms only by software has not been able to meet the performance requirements of the actual system for the technology.

On the other hand, as the number of malicious code in network security applications increases, the number of rules contained in a predefined rule set also increases rapidly. For example, for an intrusion detection rule base, the number of existing rules exceeds 5,000; for virus rules, the number of existing rules exceeds 200,000. To this end, while pursuing improved matching performance, matching technology is also required to be able to handle the matching problem of large-scale rule bases (a rule base of more than 50,000 rules can be called a large-scale rule base). Although the traditional algorithm can support multi-string matching for a large-scale rule base, the impact of the large rule base on the matching performance is very obvious. One: It is not practical.

In the actual multi-string matching technology application, there is a kind of scheme (hereinafter referred to as scheme A) which is favored because of the following characteristics: The matching performance is independent of the size of the rule base, the matching performance and the minimum length of the rule base. Irrelevant, matching performance and rule base and text to be matched The relationship is irrelevant.

For example, to match the string set P={SEC, SSH}, scenario A preprocesses P and constructs a finite state automaton (DFA), as shown in Figure 1. (where the circle indicates the state and the line indicates the conversion rule)

With the intermediate structure of the finite state automaton, for the one-dimensional text to be matched (such as SSSIG), one character can be read at a time, and in the above structure, according to the conversion relationship, each time advances to a position, when reaching S3 or S5 When the location is located, ^ 艮 a valid match.

It is the use of the characteristics of DFA that Scheme A achieves the aforementioned advantages. At the same time, it should be noted that although it has the above advantages, it has obvious drawbacks. For a simple rule set such as P={SEC, SSH}, the intermediate structure of scheme A requires a total of 6 states and 16 conversion rules. As the number of rules in the rule set increases, the size of the intermediate structure of the scheme A will increase rapidly. It is because of this space explosion that Option A is limited in practical applications.

The conversion rules (lines with arrows) in Figure 1 are divided into four categories, as described below:

Basic conversion rules: No. 1, 2, 3, 4, 5, functionally the path to correctly receive the rule set; '

Cross-conversion rules: No. 6, a path that is converted between multiple rule paths;

Restart the conversion rule: No. 7, 8, 9, 10, the path to a state after returning to the initial state; Failure conversion rule: No. 11, 12, 13, 14, 15, 16, Return to the path of the initial state.

In the 25th issue of IEEE INFOCOM in April 2006, Jan van Lunteren's paper "High-Performance Pattern-Matching for Intrusion Detection" (herein referred to as "thesis") proposes an implementation.

The scheme of the paper 1 adopts the scheme Α, and proposes a priority conversion rule storage method, which can merge all the failure conversion rules and all the restart conversion rules in Fig. 1 into a maximum of 256 rules. In practical applications, the number of conversion rules can be greatly reduced.

The technical basis of the scheme in Paper 1 is the commonality of the failed transition rules in Figure 1 (taking the state back to the initial state) and the commonality of restarting the conversion rules ( bringing the state back to the lower state of the initial state). To do this, you can set the failure conversion rule to the lowest priority and the restart conversion rule to the next lowest priority. For example, Figure 2 and Table 1: Conversion Rules Current Status Input Character Next State Priority

1 S2 1 S3 2

2 * 1 S1 1

3 S1 2 S2 2

4 S4 Β > S5 2 5 * A ► S4 1

6 * * ► SO 0 Although there are many conversion rules in Figure 2, there are only six rules in the final description by priority, as shown in Table 1.

Paper 1 does not completely solve the problem of increasing storage space with the increase of the number of rules. Matching large-scale feature sets requires a great space cost.

The state machine contains state and conversion rules. Implementing the state machine with a chip structure means that the conversion rules in the state machine are stored in a specific memory, and these conversion rules are accessed as needed. The information contained in each conversion rule includes: pre-state, input characters, and post-state. The pre-state refers to the current state of the state machine. The conversion rule indicates the process of receiving a character to jump to a certain state after the previous state. For each (pre-state, input character) pair, the state machine has a unique conversion rule that corresponds to it.

For a state machine, there may be many conversion rules. How to locate the conversion rules that need to be found in these conversion rules is a technical problem. It is also a problem that the state machine must face in the implementation of the chip. This problem can be abstracted as: Using the known pre-state and input characters, find the corresponding post-state. - In response to this problem, Sensory's patent US 7,082,044 B2 proposes a method of storing all conversion rules in TCAM in the format of [pre-state, input character, post-state] (TCAM is tri-state content) Addressing memory), since the pre-state and input characters are known during rule lookup, they are entered into the TCAM and the corresponding conversion rules can be found using the TCAM's parallel lookup function.

This solution is very intuitive, but requires the use of a special bead memory device (TCAM), which has the characteristics of large area, high cost, high power consumption, and limited storage capacity. Therefore, the hardware state machine implemented by this structure cannot contain many conversion rules, the state machine scale needs to be small, and the size of the feature set that can be matched is very limited.

It can be seen that the above existing multi-string matching method and chip structure are obviously not practical enough and have defects, and need to be further improved. In order to solve the above problems, the relevant manufacturers have not tried their best to find a solution, but the design that has not been applied for a long time has been developed, which is obviously an issue that the relevant industry is anxious to solve. Therefore, how to create a new multi-string matching method and chip structure is one of the current important research and development topics, and it has become a goal that the industry needs to improve.

In view of the above existing multi-string matching methods and chip defects, the inventors have been engaged in the design and manufacture of such products for many years of practical experience and professional knowledge, and with the use of academics, actively research and innovation, with a view to creating A new multi-string matching method and chip structure can improve the existing multi-string matching method and chip structure, making it more practical. After continuous research and design, and after repeated trials and improvements, it finally created The invention of practical value. Summary of the invention

The main object of the present invention is to provide a multi-string matching method and chip structure, and the technical problem to be solved is to enable high matching speed and matching to a large-scale rule set, which is very suitable for practical use.

The object of the present invention and solving the technical problems thereof are achieved by the following technical solutions. The cache state machine according to the present invention includes: a status register: for registering a current state; a cache status register: for registering a cache state; a conversion rule module: for storing and accessing a state conversion rule base, and according to characters received by the interface module The current state of the status register register and the cache status of the cache status register register look for the next state, output to the status register; and assign the cache status register according to a specific cache rule.

The object of the present invention and solving the technical problems thereof are also achieved by the following technical solutions. A multi-string matching method according to the present invention, comprising the steps of: sequentially taking characters as input characters from a received input character stream; for each input character, performing the following steps: The current state and the cache state are searched for in the state transition rule base; the jump to the post state; the state cache is performed according to a specific cache rule; the post state is taken as the current state, and the cached state is used as the cache state, An input character is used as the current input character, and the steps performed for each input character are repeated until all the characters in the character stream are judged.

Preferably, in the foregoing multi-string matching method, the step of the post-find state includes: first determining whether the current dog state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if present, if present, Then, the post state is used as a search result; if not, it is determined whether the cache state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if yes, the post state is used as the search result; If it does not exist, it is judged whether the initial state receives the current input character in the basic conversion rule and the n-step cross-conversion rule. If it exists, the post-state is used as the search result; otherwise, the initial state is used as the search result.

The step of performing state buffering according to a specific cache rule is: if the initial state receives the corresponding post state of the current input character in the basic conversion rule, the post state is cached; otherwise, the initial state is cached.

Preferably, in the foregoing multi-string matching method, the step of the post-find state includes: determining a type of the current state, and if it is a converged state or a general state, searching in the state transition rule set according to the current input character and the current state. Post state; if it is a detached state, the post state is looked up in the detached state transition rule set according to the current input character, the current state, and the cache state.

The separated state transition rule set is set to receive three inputs: the current input character, when The pre-state and the cache state provide an output accordingly: post-state.

The step of buffering according to a specific cache rule is: if the current state is a convergence state, the current state is cached.

The present invention also provides a computer readable storage medium storing a plurality of instructions, when the instructions are executed by a processor, causing the processor to: receive an input character; for each input character, perform the next Steps: searching for a post state in the state transition rule base according to the current input character, current state, and cache state; jumping to the post state; performing state caching according to a specific caching rule; using the post state as a current state, The state of the cache is used as the cache state, and the next input character is used as the current input character, and the steps performed for each input character are repeated until all the characters in the character stream are judged.

The present invention also provides a system comprising: a processor; a bus coupled to the processor for transferring data between portions of the system; a communication interface coupled to the bus for receiving a stream of character data a main memory, coupled to the bus, in which is stored a number of instructions, when the instructions are executed by the processor, causing the processor to perform the following steps: sequentially extracting characters from the received character data stream as Enter characters; for each input character, perform the following steps: Find the post state in the state transition rule base according to the current input character, current state, and cache state; jump to the post state; perform state buffer according to a specific cache rule The post state is taken as the current state, the cached state is taken as the cache state, and the next input character is used as the current input character, and the steps performed for each input character are repeated until all the characters in the character stream are judged. .

The object of the present invention and solving the technical problems thereof are additionally achieved by the following technical solutions. The post-state search method according to the present invention includes: calculating a possible post-state according to the current state and the input character in conjunction with the input translation table; and searching the rule storage table according to the possible post-state to obtain a corresponding input character; Whether the actual input characters are consistent with the characters obtained by searching the rule storage table; if the results are consistent, the state is switched to the possible post state; if the results are inconsistent, the state is reset to zero.

The numbering rule of the state includes: if the current state has only one corresponding output conversion rule, the number of the state after the output conversion rule is the number of the current state plus one.

Preferably, in the foregoing post-state search method, the step of calculating a possible post-state includes: according to a certain rule set, if the current state has only one corresponding output conversion rule, the number of the current state is added a number for obtaining a possible post state; if there are a plurality of corresponding output conversion rules for the current state, taking the color of the current state and the input character as inputs, searching the input translation table to obtain the The difference between the possible post state and the current state, and the number of the current state is added to the difference to obtain the number of possible post states. The rule storage table is configured to: the input is a post state, and the corresponding output is a color of the post state and an input character corresponding to the post state.

The input translation table is configured to: the input is the color of the current state and the input character, and the corresponding output is the difference between the possible post state and the current state.

Preferably, the foregoing post-state search method further includes performing entry merging on the input translation table, where each row of the input translation table corresponds to a current state, and each column corresponds to one input character, and the entry is Merging it includes the steps of judging whether there is a resource conflict and an overlay conflict, and judging each of the two rows to be merged, the judgment of the kth column is as follows: If one of the two columns is empty, judging the corresponding of the empty column Whether the character received by the non-empty column data after the merge is equal to k, if yes, it is the overlay conflict, the two columns cannot be merged, and exit; if not, the following judgment is made; if both columns are empty or both If it is not empty, determine whether the corresponding values of the two columns are the same. If not, the resource conflicts. The two columns cannot be merged and exit. If yes, the next column is judged. The resource conflict refers to the value of the corresponding column in the ITT table entry. It is empty and different; the coverage conflict refers to the non-null value of a column in the ITT table entry that covers the null value, which is equivalent to the original state. The external conversion rule, the additional conversion rule conflicts with the original conversion rule, that is, the overlay conflict; until it is determined that if all the columns in the two rows to be merged do not have the resource conflict and the overlay conflict, the corresponding row is performed. Merge, where non-null values cover null values.

Preferably, the foregoing post-state search method further includes performing group associative optimization on the input translation table, and the method includes the following steps of determining whether there is a resource conflict: for the N-way group association, dividing the ITT table into a row 256/N groups, for a group, judge the number of valid values contained in two rows. If the number is greater than N, it indicates that there is a resource conflict in the group; otherwise, judge another group; until all 256/N groups are determined If there are no resource conflicts, the two rows are merged.

The present invention also provides a computer readable storage medium storing a plurality of instructions, when the instructions are executed by the processor, causing the processor to perform the following steps: calculating the input translation table according to the current state and the input characters a possible post state; searching the rule storage table according to the possible post state to obtain a corresponding input character; comparing whether the actual input character and the character obtained by searching the rule storage table are consistent; if the results are consistent, The state is converted to the possible post state described; if the results are inconsistent, the state is zeroed.

The numbering rule of the state includes: if the current state has only one corresponding output conversion rule, the number of the state after the output conversion rule is the number of the current state plus one; the calculation is possible The step of the post state includes: a certain rule set, if the current state has only one corresponding output conversion rule, add a number of the current state to obtain a number of possible post states; if the current state exists a plurality of corresponding output conversion rules, taking the color of the current state and the input character as inputs, and searching the input translation table to obtain a difference between the number of the possible post state and the current state, And adding the difference by the number of the current state to obtain the number of possible post-states. The rule storage table is configured to: the input is a post state, and the corresponding output is a color of the post state and an input character corresponding to the post state.

Preferably, each row of the input translation table corresponds to a current state, and each column corresponds to one input character, and the input translation table is merged by an entry, and the combination of the entries is performed as follows: Each column of the two rows is judged, and the judgment of the kth column is as follows: If one of the two columns is empty, it is judged whether the state corresponding to the empty column is equal to k when the character received by the non-null column data after the merge is equal. If yes, it is an override conflict, two columns cannot be merged, and exit. If not, the following judgment is made; if both columns are empty or not empty, it is judged whether the corresponding values of the two columns are the same, if not, then For resource conflicts, the two columns cannot be merged, exit, and if so, the next column is judged; until all the columns in the two rows to be merged are determined to have no resource conflicts and overlay conflicts, the corresponding rows are merged, and the corresponding rows are not empty. The value overrides the null value.

Preferably, the input translation table is optimized by group association, and the group association optimization includes the following steps of determining whether there is a resource conflict: For the N-way group association, the ITT table is divided into 256/N. Groups, for a group, determine the number of valid values contained in the two rows. If the number is greater than N, it indicates that there is a resource conflict in the group; otherwise, judge another group; until it is determined that all 256/N groups do not have resources Conflict, then merge the two lines.

The present invention also provides a system, comprising: a main processor, an organization input data stream; a coprocessor unit, connected to the main processor; the coprocessor unit performs the following operations: according to the current state and the input characters Entering a translation table to calculate a possible post state; searching the rule storage table according to the possible post state to obtain a corresponding input character; comparing whether the actual input character and the character obtained by searching the rule storage table are consistent; The results are consistent, then the state is transitioned to the possible post state; if the results are inconsistent, the state is zeroed.

The numbering rule of the state includes: if the current state has only one corresponding output conversion rule, the number of the state after the output conversion rule is the number of the current state plus one; the calculation is possible The step of the post state includes: according to a certain rule set, if the current state has only one corresponding output conversion rule, add a number of the current state to obtain a number of possible post states; if the current state exists a plurality of corresponding output conversion rules, taking the color of the current state and the input character as inputs, and searching the input translation table to obtain a difference between the number of the possible post state and the current state, And adding the difference by the number of the current state to obtain the number of possible post-states.

The rule storage table is configured to: the input is a post state, and the corresponding output is the color of the post state and the input character corresponding to the post state.

The input translation table is configured to: the input is the color of the current state and the input character, and the corresponding output is the difference between the possible post state and the current state. Preferably, each row of the input translation table corresponds to a current state, and each column corresponds to one input character, and the input translation table is merged by an entry, and the combination of the entries is performed as follows: Each of the two rows is judged by the ^^, and the judgment of the kth column is as follows: If one of the two columns is empty, it is judged whether the character corresponding to the empty column is the character received by the non-null column data after the merge. Equivalent to k, if yes, it is an overlay conflict, the two columns cannot be merged, and exit. If not, the following judgment is made; if both columns are empty or not empty, it is judged whether the corresponding values of the two columns are the same, if not, Then, for resource conflicts, the two columns cannot be merged and exited. If yes, the next column is judged; until all the columns in the two rows to be merged are determined to have no resource conflicts and overlay conflicts, the corresponding rows are merged, and the non- A null value covers a null value.

Preferably, the input translation table is optimized by group association, and the group association optimization includes the following steps of determining whether there is a resource conflict: For the N-way group association, the ITT table is divided into 256/N. Groups, for a group, determine the number of valid values contained in the two rows. If the number is greater than N, it indicates that there is a resource conflict in the group; otherwise, judge another group; until all 256 N groups are determined to have no resource conflicts , then merge the two lines. - The object of the present invention and solving the technical problems thereof are additionally achieved by the following technical solutions. A post-state lookup structure according to the present invention includes: a main memory: storing a basic conversion rule and a cross-conversion rule, the input of which is a possible post-state calculated according to the current state and the input character in conjunction with the input translation table, Outputting the color of the possible post state and the input character corresponding to the possible post state according to the stored conversion rule; the secondary memory: storing the failure conversion rule and restarting the conversion rule, and the input is the actual input character Outputting a post state corresponding to the actual input character and its color according to the stored conversion rule; inputting a translation table: the input is the color of the current state and the actual input character, and the corresponding output is possible The difference between the number of the post state and the current state; the two-state gate: according to the comparison result between the character output by the main memory and the actual input character: if equal, the current The state transitions to the calculated possible post state, while the current state of the face The color is converted to the color of the possible post state output by the main memory; otherwise, the current state and its color are converted to the output of the secondary memory. .

Preferably, the post state lookup structure further includes a comparator for performing the main memory. Preferably, the post state lookup structure further includes: a status register: configured to store the current state; a color register: A color used to store the current state.

Preferably, the post-state lookup structure further includes a gate: configured to selectively output the output value of the input translation table and the value 1 according to the value of the color register.

Preferably, the post state lookup structure further includes an adder: configured to add the number of the current state to an output value of the gate to calculate a possible post state. The object of the present invention and solving the technical problems thereof are also achieved by the following technical solutions. A multi-string matching structure according to the present invention, comprising: a status register: for storing a current state; a color register: for storing a color of a current state; a status buffer: for storing a buffer state; a color buffer: The color used to store the cache state; the main memory: stores the basic conversion rule and the _n- step cross conversion rule, and the first input is the first possible post state calculated according to the current state and the input character combined with the input translation table, corresponding to The first way output is the color of the first possible post state obtained according to the stored conversion rule and the input character corresponding to the first possible post state; the second input is > cache state and The input character is matched with the second possible post state calculated by the input translation table, and the corresponding second output is the color of the second possible post state obtained according to the stored conversion rule and the second possible post state Corresponding input characters; secondary memory: stored with a failure conversion rule and a restart conversion rule, the input of which is described The input character is output as the post-state corresponding to the actual input character obtained by the stored conversion rule and its color; in each conversion period of the current state, the device performs secondary coverage; The second input is the color of the current state and the actual input character, and the corresponding first way output is the difference between the number of the first possible post state and the current state; the second input thereof For the color of the cache state and the actual input character, the corresponding second way output is between the second possible post state and the cache state.

The road character is the same as the actual input character, the state register is overwritten with the first possible post state, and the color register is overwritten with the color of the first possible post state; if the first path character and The actual input characters are different, but the second path character is the same as the actual input character, the state register is overwritten by the second possible post state, and the color is covered by the second possible post state The color register; otherwise, the status register and the color register are respectively covered by the post state output and the color thereof.

Preferably, the multi-string matching structure further includes: a first comparator, configured to perform a comparison between a first path character output by the main memory and an actual input character; and a second comparator, A comparison between the second pass character output by the main memory and the actual input character is performed.

Preferably, the multi-string matching structure further includes: a first strobe: configured to select and output an output value of the input translation table and a value 1 according to a value of the color register; and the second strobe: The value of the color buffer is selected for the output value of the input translation table and the value 1.

Preferably, the multi-string matching structure further includes: a first adder: configured to add a number of the current state to an output value of the first gate to calculate a first possible post state a second adder: configured to compare the number of the buffer state with an output value of the second gate Add to calculate the second possible post state.

The object of the present invention and solving the technical problems thereof are further achieved by the following technical solutions. A multi-regular expression matching method according to the present invention, comprising the steps of: sequentially taking characters as input characters from a received input character stream; for each input character, performing the following steps: according to the current input The character, current state, and cache state are looked up in the state transition rule base; jump to the post state; state cache according to a specific cache rule; the post state as the current state, and the cached state as the cache state The next input character is used as the current input character, and the step performed for each input character is repeated until all the characters in the character stream are judged.

Preferably, in the foregoing multi-regular expression matching method, the step of the post-find state includes: first determining whether the current state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if present, if present, Then, the post state is used as a search result; if not, it is determined whether the cache state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if yes, the post state is used as the search result; If it does not exist, it is judged whether the initial state receives the current input character in the basic conversion rule and the n-step cross-conversion rule; if it exists, the post state is used as the search result; otherwise, the initial state is used as the search result; The step of performing state buffering according to a specific cache rule is: if the initial state receives the corresponding post state of the current input character in the basic conversion rule, the post state is cached; otherwise, the initial state is cached.

Preferably, in the foregoing multi-regular expression matching method, the step of the post-find state includes: determining a type of the current state, and if it is a converged state or a general state, according to the current input character and the current state in the state transition rule set After the lookup state; if it is a detached state, the post state is searched in the detached state transition rule set according to the current input character, the current state, and the cache state; the detached state transition rule set is set to receive three inputs: the current input character, the current The status and the cache status are respectively provided with an output: a post state; the step of caching according to a specific cache rule is: If the current state is a converged state, the current state is cached.

The present invention has significant advantages and advantageous effects over the prior art. With the above technical solution, the multi-string matching method based on the cache state machine and the chip structure based on the "post-state lookup" have at least the following advantages and beneficial effects:

It can eliminate more than 95% of all cross-conversion rules; it can reduce the number of base winter conversion rules, thereby reducing the number of required states, etc.; can achieve higher matching speed than other methods. In short, it can meet the demand for high-speed large-scale multi-string matching technology. The performance of the matching is independent of the size of the rule base. The performance of the matching is independent of the minimum length of the rule base. The performance of the matching is independent of the relationship between the rule base and the text to be matched. It can support large-scale rule sets, with the number of rules. Increase the sub-linearity of storage space, effectively reduce space requirements, and be effective Store and access conversion rules in the state machine.

The above description is only an overview of the technical solutions of the present invention, and the technical means of the present invention can be more clearly understood, and can be implemented in accordance with the contents of the specification, and the above and other objects, features and advantages of the present invention can be more clearly understood. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1: The finite state automaton constructed in the existing multi-string matching scheme A.

Figure 2: A finite state automaton constructed according to the scheme of the prior art 1, in which different priorities are set for the conversion rules.

Figure 3: State machine model.

Figure 4: Cache state machine model.

Figure 5: A finite state automaton constructed according to scenario A, where the restart conversion rules and the failed conversion rules have been removed.

Figure 6: Cache state machine constructed to implement dynamic cross-conversion loading.

Figure 7: Flow chart of the dynamic cross-conversion loading method.

Figure 8: A finite state automaton constructed according to scenario A, with a homogeneous path. Figure 9: The ideal framework for feature set {betters, pattern} optimization.

Figure 10: Conformation path merge based on cache state machine.

Figure 11: Three states in the homogeneous path merge method.

Figure 12: Conversion function for three states in the isomorphic path merge method.

Figure 13: Two observations based on the post-state lookup structure.

Figure 14: Post-state lookup framework.

Figure 15: Detailed structure of the post-state lookup structure.

Figure 16: Input translation table (ITT) structure.

Figure 17: Schematic diagram of the consolidation of ITT entries.

Figure 18: One of the ITT table optimizations: The idea of consolidation of entries.

Figure 19: One of the ITT table optimizations: Table item consolidation method.

Figure 20: ITT Table Optimization 2: 2-way set associative ITT table structure.

Figure 21: ITT table optimization 2: N-way group association ITT table optimization method.

Figure 22: Chip structure ACC-NSA structure for implementing multi-string matching technology based on cache state machine.

Figure 23: Applying the dynamic cross-conversion loading method to eliminate the effect of cross-conversion rules (ClamAV rule). ·

Figure 24: Applying a dynamic cross-conversion loading method to eliminate the effect of a cross-conversion rule (Snort rule). Figure 25: Effect diagram of applying the merged isomorphic path method to reduce the basic conversion rules. BEST MODE FOR CARRYING OUT THE INVENTION The specific embodiments, steps, structures, features and functions of the multi-string matching method and chip structure according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

In order to meet the demand for high-speed large-scale multi-string matching technology, a scheme called "cache state machine" is adopted in the scheme of the present invention, and its design idea is derived from the aforementioned deterministic finite state automaton (DFA). - Deterministic Finite Automata ). A representation of DFA is shown in Figure 3. Each DFA has a current state (in the status register) that accepts the conversion rules for that character based on the input character and the current state, and proceeds to the next state. When the next character comes, the "next state" becomes the "current state". DFA can perform state transitions based on the internal data structure shown in Figure 1 driven by input characters. The main features of DFA are: Its next state is determined only by the current state and the currently entered characters.

The traditional state machine model (DFA and NFA) is a simplified form of the Turing machine model, regardless of the deterministic finite state automaton (DFA) or the uncertainty finite state automaton (NFA), the next state is only the current state and The current input decision is shown in Figure 3. NFA can be converted to DFA equivalently.

The deterministic finite state automaton (DFA) is defined as a five-tuple, M = {JiT, ∑, s. ,F, }, including:

• A finite state set, denoted K, is a collection of all states;

• A collection of alphabets, denoted as ∑, that is, a collection of characters received by the state machine;

• The start state of the state machine, denoted as s. ; '

• Receive state set, denoted as F, receive state set is a subset of the finite state set,

• State transition function, δ Κ χ Σ→Κ \

Among them, the state transition function is a binary function, which determines the next state according to the current state of the state machine and the received characters. The CDFA - Cached Deterministic Finite Automata is proposed by the present invention, and one of its manifestations is shown in FIG. Referring to DFA, CDFA includes a cache state (in the state buffer) in addition to a current state. In the cache state machine, its next state is determined by the current state, the currently input character and the cache state. The next cache state is determined by the internal mechanism of the cache state machine. No external input is required, and the cache state machine can be used. The specific needs of the flexible customization.

The Cache State Machine (CDFA) breaks the traditional state machine's "the next state is determined only by the current state and the current input". By recording the history information, the richness of the operation of the state machine in the post-determination state is increased. The cache state machine achieves the above design goal by adding a state cache function to the state machine, as shown in FIG. From the perspective of the external interface, the cache state machine, like the traditional state machine, receives only input characters and outputs the state machine judgment result. The difference is that a state buffer ( Cache ) is added internally to enable a certain policy to cache the state.

The Cache State Machine (CDFA) can be defined as a seven-tuple, {/, ∑, , Λ ^, }, including: • A finite state set, denoted Κ, that is, a set of all states in the state set;

• The start state of the state machine, denoted as s ₀ ;

• Receive state set, denoted as F, F ^ K ;

• The number of caches contained in the state machine, denoted as N;

• State transition function, δ : Κ χ Κ ^Ν χ Σ → Κ ;

• Cache policy function, θ '· Κ χ Σ →Κ ^Ν

The cache policy function determines the state to be cached according to the current state and the current input; the state transition function δ determines the next state according to the current state, the cached state, and the input characters.

Since the newly added state buffer is controlled by the state machine according to the cache policy function, it is invisible and inoperable, which is similar to the design principle of the cache in the computer storage system. Therefore, the new state machine model is named as the cache state machine. model.

The cache policy function can remember both historical information that the state machine has experienced, and can also "remember" other state information in a certain way.

According to the above description, the structure of the cache state machine is as follows, which includes:

Status register: used to register the current status;

Cache Status Register: Used to register the cache status. The number of states that can be registered is Ν, Ν > 1 ; Conversion Rule Module: Used to store the state conversion rule base, and according to the characters received by the interface module, the current state of the status register registration and the cache. The status register registered cache status looks for the next state.

In addition, when the cache state machine is implemented, the following structure should also be matched:

Interface module: used to receive input characters;

Control module: Used to control the characters that the interface module normally receives input, control the status register to update the current state, control the cache status register to update the buffer status, and control the conversion rule module to find the next status.

As mentioned earlier, DFA-based solutions 造成 when multi-string matching occurs, which increases the explosion of storage space with rules. The space explosion comes from the exponential increase in the number of conversion rules. After research, the space explosion comes from three types of conversion rules (cross-conversion rules, restart conversion rules, and failure conversion rules). In order to solve the space explosion problem, it is necessary to effectively control the increase in the number of these three types of conversion rules.

The prioritized approach used in paper 1 will be able to restart the conversion rules and failure conversion rules. Then the number is controlled within 256. In the present invention, both types of conversion rules can be solved in this way.

The invention utilizes the principle of a buffer state machine, mainly to eliminate nearly all cross-conversion rules, thereby completely solving the space explosion problem. In addition, the present invention utilizes the principle of a buffer state machine, and can also reduce the number of basic conversion rules, thereby achieving a sub-linear increase in storage space with the number of rules. The implementation is as follows.

The principle of the cache state machine uses Method 1: "Dynamic Cross-Conversion Loading" to eliminate more than 95% or even all of the cross-conversion rules. This method is named ACC.

For example, P={slice, cross}, the DF structure of the scheme Α is shown in Figure 5 (where the restart conversion rules and the failure conversion rules have been removed). There are three cross-conversion rules.

If the text to be matched is croslice, the constructor cache state machine is shown in Figure 6:

Among them, the cross-conversion rule has been eliminated, and replaced by a cache space. The principle of any one of the cross-conversion rules is: in the current state ^ S ₃ , the received character is s, while switching to state S ₄ , another path is opened from S _G (ie, where S ₆ is located) path). For the next input character, if the basic conversion rule of the current path is not met (if the current state S ₄ , the input character is S, then the state is converted to state S ₅ ), but the basic conversion rule of the other path is met (if the current state S ₆ , if the input character is 1, then the condition of transition to state S ₇ ) is generated, then the cross-conversion rule is generated, that is, if the next input character is 1, the state jumps from state S ₄ to S ₇ .

The operations performed by the cache state machine are as follows. If the current state S at position _3, the current character is received 3, according to the principle of cross conversion rule generation, S ₆ is cached state. At the same time, the state machine enters the next state S ₄ . In S ₄ state, the received character is 1, the next state, the input characters (1) and the state of the buffer _{(S. 6)} determined by the current state (S _4), because the S in the basic conversion rule path ₄ does not accept characters 1, and S ₆ accepts the character 1, so S ₇ is determined to be the next state.

The dynamic cross-conversion loading dynamically generates the cross-conversion rules originally described by DF A using the CDFA principle, thereby greatly reducing the number of stored conversion rules.

The buffer state machine CDFA used in the ACC method is a seven-tuple {K, ∑, s _Q , F, 1, δ, θ }, and the required number of buffer states N = l (that is, a register is required for state buffering).

The state transition function δ can be divided into the following two categories:

• 6 _basic indicates the state transition function of the basic conversion rule, S _bask ι Κ χ Σ -^ Κ , the definition of S _basic in the ACC method is the same as the scheme A.

• S _nerass represents the state transition function of the n-step cross-conversion rule, S _llcross K ∑→K , and the definition of δ n cross in the ACC method is the same as that of scheme _A.

The state transition function δ is defined as

Where priority is the priority identifier, A is the highest priority, and D is the highest priority. If the high priority result is valid (not empty), the result is taken first; if the high priority result is invalid, the 4 priority result is adopted. The invalid result means that a certain state Si is in S _basie and S _ncr . There is no rule in the _ss conversion function that accepts the character c.

As shown in the flowchart of FIG. 7, the meaning of the state transition function δ is that, for the state transition of the CDFA in the ACC, it is first determined whether the current state Si has a conversion rule for receiving the current character c in the basic conversion rule and the n-step cross-conversion rule. If yes, apply the rule to jump to the next state; if there is no corresponding conversion rule, the cached state S _{k is} taken out, and the _Sk state is the current state in the basic conversion rule and the n-step cross conversion The rule searches for a conversion rule that accepts the current character c. If it exists, it jumps to the corresponding next state; if there is no corresponding conversion rule, it determines whether the initial state So receives the character c; if it receives, jumps to the corresponding state , otherwise jump to the initial state So.

Regardless of the above four priority operations, parallel operations can be used without affecting the final performance. The cache policy function is defined as

Where the meaning is "empty", indicating that there is no corresponding conversion rule.

The meaning of the cache policy function 是 is that for the buffer space of the CDFA in ACC (only one), each cycle is cached, and the cached content is the initial state So accepts the next state corresponding to the current input character c; The corresponding conversion rule is not included in the rule, and the initial state S _{G is} cached. As you can see, the cache policy function has nothing to do with the current state Si.

The ACC method is based on the above cache state machine. The method mainly consists of two steps: preprocessing and matching. The work in the preprocessing stage is to read in the feature set and construct the cache state machine; the job of the matching phase is to read in the text to be matched, perform state machine conversion, and report the match in a specific state. - The above description is for N=l, that is, there is only one storage space in the CDFA, and one state can be cached. This method can be applied to the case of N>1.

The principle of the cache state machine uses Method 2: "Homomorphic path merge" to reduce basic conversion rules and states. This method is named ACS.

The idea of the ACS method is to combine the homogeneous paths in the state machine to reduce the number of states and basic conversion rules in the state machine.

Taking P={betters, pattern} as an example, the DFA constructed by scheme A is shown in Fig. 8, among which Requires 14 basic conversion rules and 15 states. After analysis, it can be found that S ₂ -S ₅ has the same properties as S ₉ _S ₁₂ , and all receive the string "tter", which is called an isomorphic path.

According to the above analysis, the ideal frame after optimization of the feature set {betters, pattern} is shown in Fig. 9. This framework represents the merging of the state generated by the "tter" substring and the conversion rules.

The inventors believe that it is not possible to combine isomorphic paths using traditional state machine theory (DFA or NFA). For example, over the frame as shown in FIG. 9 serious errors, as in state S _6, when the input character "s", will jump to the state S _9, therefore, the string "patters" may also be successfully matched. However, in the state machine before optimization, the state S _{9 is} only jumped when the feature "betters" comes. It can be seen that the use of the DFA theory for the homomorphic path merging essentially removes the historical information represented by the different states of the state machine, thus causing an error in the matching result.

The ACS method uses a cache state machine model. The cache state machine can effectively remember the characteristics of the state transition history information, and perform isomorphic path merging to ensure the correctness of the matching. Taking the feature set {pattern, betters} as an example, the isomorphic path using the idea of the cache state machine is combined as shown in Fig. 10.

The idea of merging the isomorphic path based on the cache state machine is to dynamically store the path source state (S ₈ or S in Figure 10 is stored in the cache of the cache state machine) when the path is merged. If the received characters cause the state transition to arrive at the same isolated path configuration of the position (s ₆ state), a state will be cached taken to determine the configuration according to jump to the state where the source of the same path. for this reason, if the text input at this time is "patters", the state in which the same Si configuration at the beginning of the path is cached, the state S ₆ when taken out, because the path is not derived from S _8, even when the input character is "s", not to jump to state S _9.

The cache state machine CDFA used in the ACS method is a seven-tuple {K, ∑, so, F, 1, δ, θ}, and the required number of buffer states Ν = 1 (that is, a register is required for history state caching). Each state in the CDFA corresponds to one color, and the CDFA contains three colors. The color is used to distinguish three different states in the merge process of the isomorphic path, as shown in Figure 11.

The three state descriptions and their correspondence with colors are as follows:

Converging states: Yellow, mesh, defined as the last state before entering the isomorphic path, which represents the history information of the state machine before the isomorphic path. This state triggers its own state cache. This set of states is denoted as K _c . _v .

Diverging states: Pink, strip, defined as the last state of a homogeneous path, which determines the state that needs to be jumped after receiving a character based on historical information (ie, the state being cached). This state triggers a cache read. The state set is recorded as K _Div .

• Common states: White, blank, defined as all other states of the non-convergence state and the split state. In this type of state, the cache within the CDFA is not manipulated. This set of states is denoted as K _c . _m .

The state transition function δ of the cache state machine CDFA can be divided into the following two categories: For the convergence of the four dog states and the general state, δ is a binary function, δ : Κχ Σ -^ Κ , the definition of δ in the ACS method is the same as that of the scheme A.

* For the separation ^ I dog state, δ is a ternary function, _: χΛΓχΣ → , the conversion function δ of the separation state in the ACS method is defined as the current state, the cache state, and the current character.

The state transition function δ is defined as

S. e K _t Cov

S(S c) Com

Div

Where is the current state, c is the current input, and S _k is the currently cached state.

The transition rule for the state transition function in δ is different from the traditional transition rule. It contains three inputs and one output. The three inputs contain the aggregation status of the source before the isomorphic path merge, as shown in Figure 12. In the aggregation state and the general state, the conversion rule set (two inputs) is found according to the current input and the current state to obtain the next state. In the separated state, in addition to the current input and current state, it is also necessary to find a separate state transition rule set (different from the conversion rule set, three inputs) according to the state being cached to obtain the next state.

The cache policy function Θ is defined as

S _i e K

[ Φ , ^ Ε {Κ _αιη , Κ _οίν } where meaning "null operation" means no action is taken on the cache state.

The cache policy function Θ means that for the CDFA cache space in ACS (only one), when the current state is the aggregation state, the state is cached to the cache space. In other cases, nothing is done with the cache space.

Therefore, in the method, the type of the current state is first determined, and then the corresponding action is performed according to the judgment result. If it is the aggregation state, the next state is obtained by searching the conversion rule set according to the current input and the current state, and the current state is cached to the cache space; if it is the general state, the conversion rule set is obtained according to the current input and the current state to obtain the next state. ; If it is a detached state, the next state is obtained by looking up the separation state transition rule set according to the current input, current state, and cache state.

The merged CDFA removes 5 states and 4 basic conversion rules, and space can be further saved. The overhead required is the storage of a state storage space as a cache.

The practitioners in this field can consider that a regular expression is a string consisting of a series of special characters. For the introduction of regular expressions, refer to related materials. The traditional AC algorithm can solve the problem of multi-regular expression matching by converting regular expressions into DFA and using DFA.

CDFA, and use CDFA to receive input characters for matching. The specific matching method includes eliminating 1 step Cross-conversion rules and homogeneous path merges, etc.

The essence of state machine conversion is how to find the corresponding conversion rule Tr in the conversion rule base according to the known current state Si and the current character input c, where Tr(S _i5 c) = Sj, and jump to the state. The technical difficulty of hardware implementation is: How to effectively store the conversion rule base in the memory and how to effectively locate the conversion rule Tr. For convenience of subsequent description, Si in the conversion rule Tr is referred to as "input state" and c is referred to as "input character", which is referred to as "output state".

This case is proposed

access.

The design of the post-state lookup stems from two observations of the state machine, as shown in Figure 13.

First, there are a large number of Linear Trie structures in the state machine, especially the cache state machine generated by scenario A. The so-called "linear tree" means that each state in the state machine contains only one transformation rule pointing to the next state, and forms a linear one-dimensional structure. Due to the existence of a large number of linear trees, the status numbers can be arranged incrementally. Therefore, the number of the next state can be calculated from the current state, that is, the predicted state.

Second, for each state in the state machine, it is determined based on the particular character received, ie, the characters it accepts are deterministic, regardless of the type of conversion rules entered. If the state S ₇ receives the basic conversion rule and the cross conversion rule, the character received by the state is "i" regardless of the conversion rule. Therefore, if the post state, that is, the output state, is obtained, the characters accepted by it can be uniquely determined, and by comparing with the actually input characters, it can be verified whether the calculated post state is a real post state. .

Based on the above two observations, the structure of the post-state lookup is proposed. The structure uses a "predictive" and verification approach, as shown in Figure 14. According to the current state Si and the current input character c, a possible post-state is calculated through an Input Translation Table (ITT) or a possible post-state is directly calculated, and the post-state is used as an address to index the rule storage table to obtain the state. Converts the input character of rule Tr and compares whether the current input character c is consistent with the character (as shown by the two dashed lines in Figure 14). If the results are consistent, a state transition is performed. If the results are inconsistent, it means that no conversion rules correspond to the current input and the status is zero.

In the post-state lookup structure, the rule storage table can be stored by using an inexpensive memory such as SRAM or DDR, and the internal conversion rules of the memory are compactly distributed, and there is no "gap".

The post-state lookup is effective and comes from the use and optimization of ITT tables. According to observation 1, it can be known that since the state machine contains a large number of linear trees, the post-state of each state in the linear tree can be obtained by simple incrementing without looking up the ITT table. Only a small number of states with multiple conversion rule outputs need to enter the ITT table to get the difference between states. In addition, optimization of ITT tables can further reduce the use of storage space. ·

The detailed design of the NSA structure is divided into two parts. One is the conversion rule in the input translation table ITT. And the rules store the storage in the table; the second is the access path design of the conversion rules.

The overall structure of the NSA is shown in Figure 15. This includes the main space "TRM-1, (Transition Rule Memory - 1) stored in the conversion rule and the storage space "TRM-0" (Transition Rule Memory -0 ) that resolves the failure conversion rule and restarts the conversion rule.

For Figure 15, after the character is entered, ■ according to the current status register and color register to determine how to operate. A strobe MUX is provided for selecting and outputting the output value (ie, the difference between the states) obtained by accessing the ITT table according to the value of the color register and the value 1. If the color register value is 0, it is considered that there is no color in the current state, MUX selects output 1, and the current status number is incremented by 1 to obtain the post status number, and the corresponding state is used to access TRM-1 to obtain the corresponding value. The corresponding value includes the color of the next state and a character. If the color register value is not 0, it is considered that the previous state has color, that is, the current state and the currently input character are input into the table together to obtain the output value, and the MUX selects the output value obtained by accessing the ITT table, that is, the current state number is added. The post-state number is obtained after the difference between the states, and the corresponding value is obtained by accessing the TRM-1 with the post-state. Regardless of the value of the color register, the input character is input to TRM-0 to obtain an output value, which includes the next state and a color value.

The character value output from the TRM-1 is compared with the current input character at a comparator CMP, and the following operation is performed by a two-state gate according to the comparison result: If equal, the color of the next state output by the TRM-1 is used. The color register is overwritten, and the status register is overwritten with the calculated address of the access TRM-1 (ie, the post state), thereby realizing state transition in the case of verification. Otherwise, the state register is overwritten with the state of the TRM-0 output, and the color is overwritten with the color register, thereby realizing zeroing in the case of verification failure.

Failed conversion rules and restart conversion rules can be combined into a maximum of 256 with priority policies. For these conversion rules, since their output state is the initial state S. Or the post-state of the initial state, so the input character is used as the address for indexing. That is, the initial state So or the post state of the initial state is output according to the input character. To do this, build a failure to resolve both types of conversion rules and restart the transformation rule memory TRM-0. It uses character addressing to store the two types of conversion rules. According to the output state that the input character can jump, if there is a conversion rule for the corresponding character, the post state of the initial state is stored in the corresponding position. If there is no conversion rule for the corresponding character, The initial state is stored in the corresponding location. Since the input characters are up to 256, TRM0 contains 256 entries.

Other types of conversion rules determine the matching of state machines, and the meaning of these conversion rules is decomposed and stored in two parts.

First, the character sequence accepted by each state is stored in the main conversion rule memory TRM-1 according to the state number. This part of the space is compact.

In addition, define a new concept, the color of the state, each state can be made into any color. At the same time, the input translation table uses color as an index for access. For the current state Si in the state machine, it is set to the input state of the k conversion rules, ie for this state, there are k characters that cause it to jump to the new state. (The failure conversion rules and restart conversion rules are not considered here). '

If k is 1, that the state is in the Linear Trie, there is only one corresponding conversion rule Tr(S c)=Sj. The state is colored white (color=0), and the number of states in the state machine is made to meet the following conditions: j = i + l , that is, the state numbers in the linear tree are sequentially incremented.

If k is not 1, that is, the state contains multiple output transitions, it is a new color. As shown in FIG. 16, both the state Si and the state S _k contain two output conversion rules. To be able to predict the next state, the two states are respectively associated with a new row of the ITT table, and different colors are used to index the ITT table.

The internal structure of the ITT table is as shown in Fig. 16, wherein each color corresponds to 256 values, and each value is a state number difference value in which the state Si receives the corresponding column character and jumps to the new state. In Figure 16, state Si receives the character 0x01 and jumps to state S _k , which corresponds to the ITT table. The 0x01 column of color 1 stores the difference between state S _k and state Si: k - i. Where 0 represents a null value.

In combination with the ITT table design and color concept, for the current state of the state machine S if it is white (color=0), the possible post state is S _i+1 ; if the color is not white, access the ITT with color and current input Table, obtain the state difference, and then calculate the post state S _{i+ i} . Using the post state to access the main conversion rule memory TRM-1, the corresponding character c accepted in the subsequent state is obtained. Since the post state is calculated, although the current state information is used and the current input character information may be used, this use is not sufficient to actually determine the post state. To this end, it is necessary to compare the accessed character c, and the current character. c. If the two characters are the same, the calculated post state is the real post state and jumps to the state. If the two characters are different, jump to the state obtained by the TRM-0 access, that is, apply the failure conversion rule or restart the conversion rule.

In the above design of the NSA structure, each state containing multiple output conversion rules is assigned a new color, i.e., a portion of the ITT table is allocated as a basis for the post-calculation state. It should be noted that for most colors, there are only a few post-states, so the ITT table has a large number of nulls (0) per line. In order to effectively use the ITT table space, an optimization method for the ITT table is given here: Table item merge.

The idea of merging the entries of the ITT table is to combine multiple entries of the ITT table into one, so as to effectively utilize the space resources. Another implication of merging is to make the color of the state in the state machine.

Figure 17 shows the merge of the entries in the ITT table. The left state machine contains 4 colors, and after merging, the right state machine contains only 2 colors.

Two entries in the ITT table can be merged if and only if they do not conflict. Conflicts fall into two categories: resource conflicts and coverage conflicts.

(1) Resource conflict means that the value of the corresponding column in the ITT table entry is not empty and different; as shown in Figure 18, color 2 and color 4. (2) Coverage conflict means that after a non-null value of a column in the ITT table entry covers a null value, an additional (virtual) conversion rule is added for the original state. It is to be ensured that the added extra conversion rule does not conflict with the original conversion rule, that is, the post state obtained according to the (virtual) conversion rule does not receive the character corresponding to the existing conversion rule.

An example of a coverage conflict is as follows: As shown in Figure 18, if color 3 and color 1 are merged, the 0x63 column of color 3 will overwrite the column of color 1. That is, the 0x63 column of color 1 is originally a null value, and merges to become a non-null value of 3. This can lead to errors. If it is also the color 1, when 0x63 is input, in order to calculate the post state, the state corresponding to the color 1 is incremented by 3 to become a new state. However, it is impossible to know whether the received character corresponding to the new state is 0x63. If it is 0x63, the state jump will be performed according to the post-state lookup structure of Figure 15. That is, when color 1 receives 0x63, it also jumps to a certain state, which is wrong. This is the conflict that occurs when a non-null value overwrites a null value when merging.

After avoiding these two types of conflicts, the ITT table can be used to merge the entries. The related method is shown in Figure 19.

Figure 19 shows the judgment of whether two ITT table entries can be merged. The judging method judges each of the two rows to be merged. The judgment of the kth column is as follows: If one of the two columns is empty, it is judged whether the state corresponding to the empty column is equal to k if the character received by the non-empty column data after the combination is merged, and if so, the overlay conflict, two columns Cannot merge, exit, if not, proceed to the next judgment; if both columns are empty or not empty, judge whether the corresponding values of the two columns are the same, if not, the resource conflicts, the two columns cannot be merged, and exit, if Yes, then judge the next column. Until all the columns in the two rows to be merged are determined to have no resource conflicts and overlay conflicts, the corresponding rows are merged, with non-null values covering the null values.

When the ITT table is merged, the method of two-two judgment is taken. As shown in Fig. 18, the color 2 and the color 1 are first combined and judged, then the color 3 and the color 1 are combined and judged, and so on. Until all possible merged colors are merged.

In Figure 18, although color 4 uses only one location in the ITT table entry, it cannot be merged with color 2 due to resource conflicts. In order to further optimize the use of ITT table space, a group association optimization strategy for ITT tables is proposed to solve the above problems.

The group association optimization strategy is similar to the group association strategy cached in the computer storage system. The idea is to break the boundaries of the ITT table column by group association, and the same column data can be stored in different columns. The 2-way set associative ITT table structure is shown in Figure 20. With this structure, the color 4 in Fig. 18 can be combined with the color 2.

For the N-way group association, the ITT table is divided into 256/N groups. For two colors, the method for judging whether they can use the group association strategy for merging is shown in FIG. 21. Two ITT table entries can be optimized by group association if and only if there is no conflict in the state in the same group. The conflict here is just a resource conflict, that is, any group contains non-empty elements that exceed N. The method in Figure 21 is to determine if two ITT table entries conflict. For a group p, determine the number of valid values contained in the two lines. If it is greater than N, it means that there is a resource conflict in the group (the total number of valid values exceeds N). Therefore the two lines cannot be merged. Otherwise, another group is judged, and until it is determined that there is no resource conflict in all 256/N groups, the two lines can be merged.

Similar to the cached group association strategy, the group association policy of the ITT table needs to add a tag bit (Tag) to distinguish each content. The tag here requires two fields, one is the input tag field, and the other is the color tag field. Use these two fields to distinguish between different rows and different columns before the merge.

Post-state lookup The NSA is an efficient hardware state machine implementation. This effectiveness stems from accurate access to the memory, the absence of conflicting items to determine, and the use of inexpensive SRAM, DDR, etc. memories. Although there are certain storage gaps in the ITT table of the NSA, this gap can be effectively controlled by the combination of the table entry and the optimization of the group association strategy.

Corresponding chip structure: In order to implement a multi-string matching technology based on a cache state machine at a high speed, the present invention designs a corresponding chip structure, and the overall structure is as shown in FIG. The structure is a feature matching structure including an ACC method and an NSA structure for string matching. Corresponding to the cache state machine structure of Figure 4, the ACC-NSA structure of Figure 22 includes a conversion rules module, a status register, and a cache status register module.

The NSA structure can implement a state machine efficiently by hardware, and the ACC method is based on the principle of the cache state machine. To this end, the main problem solved by combining the ACC-NSA structure is to provide a cache using the NSA structure to implement the ACC method. state machine.

Figure 22 shows the ACC-NSA structure framework. It can be seen that the structure is based on the post-state lookup structure shown in Figure 15, adding the "state buffer" and "color buffer" related paths. These two sets of paths share a set of memory ITT tables and TRM-1 memory.

In this architecture, the TRM-1 design and the TRM-1 design are implemented in a dual-port memory that supports parallel access to the registers and cache. (If you do not consider the parallel access feature, single-port memory can also be used.) In the ACC-NSA architecture, TriMUX (three-state gate) is implemented with a priority strategy.

It contains 3 inputs, 1 output and 2 bit control signals. The control signal is the output of the "Compare" module CMP, numbered " and "2". TriMU functions as if , , V, = equal high a priority

TriMux(state, color) = (state ₉ color ^2 ^) y,2 ', = equal middle _ priority ( 5-1 )

(state, color ^ 3 ⁿ ) others low priority In this formula, ( state, color, "1" ) represents the post state value and its color calculated using the state in the register; ( state, color, "2" ) Represents the post state value and its color calculated using the state in the cache; (state, color, "3" ) represents the post state value of the TRM-0 output and its color. If "l"=equal, the meaning of high_priority is: If the input state calculated by using the state in the register is consistent with the actual input character after accessing the TRM-1, TriMUX The preferred output outputs the post-state value and its color calculated using the state in the register, which has a high priority. If "2,,=equal, middle_pi'iority means: If the input state calculated by using the state in the cache is consistent with the actual input character after accessing the TRM-1, the TriMU does not satisfy the preference. When the output uses the post state value calculated by the state in the register and its color, the output is selected from the post state value calculated by the state in the buffer and its color, and the input has a medium priority. If the conditions of the situation are not met, then the third input is selected, that is, the post-state value and its color of the TRM-0 output obtained by the application failure and restarting the conversion rule.

When the input character arrives, the register (status register and color register) and the cache (status buffer and color buffer) access the ITT table together, calculate the possible post-state value, and access TRM-1 to extract the character corresponding to the conversion rule. At the same time, according to the input character access TRM-0, the failure conversion rule and the state value corresponding to the restart conversion rule are obtained. The three-way result is entered into the TriMUX module. At the same time, according to the two outputs of TRM-1, it is judged whether the real occurrence occurs by comparing the input characters, and the TriMUX module is controlled to select the correct result to be overwritten into the register. At the same time, the result of TRM-0 is updated. A state transition is formed.

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Although the present invention has been disclosed in the above preferred embodiments, it is not intended to limit the present invention. The skilled person can make some modifications or modifications to the equivalent embodiments by using the above-disclosed technical contents without departing from the technical scope of the present invention. It is still within the scope of the technical solution of the present invention to make any simple modifications, equivalent changes and modifications to the above embodiments. Taking the matching of ClamAV 5 Wanna virus rule and Snort 1785 intrusion detection rule as an example, the industrial applicability of this case is illustrated.

For the use of the principle of the cache state machine: "dynamic cross-conversion loading", the effect of eliminating cross-conversion rules is shown in Figure 23 and Figure 24.

One of the one-step cross-conversion rules can be eliminated by the method of the present invention. It can be seen that the multi-string matching method based on the cache state machine "dynamic cross-conversion loading" can reduce the space to the original 4.1% (ClamAV Rule) and 20.8% (Snort rule).

How to use the principle of the cache state machine 2: Combine the isomorphic paths, and reduce the effect of the basic conversion rules as shown in Figure 25.

The dotted line is the data of the traditional method using DFA, and the solid line is the data of the method using the buffer state machine CDFA. It can be seen that the CDFA-based method of the present invention can reduce the number of basic conversion rules by up to 21.4%. (Snort rules).

The corresponding chip structure performance is shown in Table 2:

Table 2

Based on the Cache State Machine (CDFA) method, the chip structure ACC-NSA structure can achieve a maximum matching speed of 11.7 Gbps (under 0.18 micron process). It has a faster speed than other methods. Industrial applicability

The multi-string matching method based on the cache state machine and the chip structure based on the "post-state lookup" have at least the following advantages and beneficial effects:

It can eliminate 95°/. All of the above cross-conversion rules; can reduce the number of basic conversion rules, thereby reducing the number of states required, etc.; can achieve higher matching speed than other methods. In short, it can meet the demand for high-speed large-scale multi-string matching technology. The performance of the matching is independent of the size of the rule base. The performance of the matching is independent of the minimum length of the rule base. The performance of the matching is independent of the relationship between the rule base and the text to be matched. It can support large-scale rule sets, with the number of rules. Increasing the storage space sub-linear increase, can effectively reduce the space requirements, can effectively store and access the conversion rules in the state machine.

Claims

Rights request

A cache state machine, characterized in that it comprises:

Status register: used to register the current status;

Cache Status Register: Used to register the cache status;

Conversion rule module: for storing and accessing the state conversion rule base, and searching for the next state according to the character received by the interface module, the current state of the state register registration, and the cache state of the cache state register registration, and outputting to the status register; The cache rule assigns a value to the cache status register. -

2. A multi-string matching method, characterized in that it comprises the following steps:

The characters are sequentially taken out from the received input character stream as input characters; for each input character, the following steps are performed:

Find the post state in the state transition rule base according to the current input character, current state, and cache state;

Jump to the post state;

Stateful caching according to specific caching rules;

The post state is taken as the current state, the cached state is taken as the cache state, and the next input character is used as the current input character, and the steps performed for each input character are repeated until all the characters in the character stream are judged.

The multi-string matching method according to claim 2, wherein the step of the post-find state comprises: first determining whether the current state receives the current input character in the basic conversion rule and the n-step cross-conversion rule. Post-state, if present, the post-state is used as the search result; if not, it is determined whether the cache state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if so, the The post state is used as the search result; if it does not exist, it is judged whether the initial state receives the current input character in the basic conversion rule and the n-step cross-conversion rule; if it exists, the post state is used as the search result; otherwise, the initial state is As a result of the search.

The multi-string matching method according to claim 3, wherein the step of performing state buffering according to a specific cache rule is: if the initial state receives the current input character after the corresponding corresponding in the basic conversion rule State, then cache the post state; otherwise, cache the initial state.

The multi-string matching method according to claim 2, wherein the step of the post-find state comprises: determining a type of the current state, and if it is a converged state or a general state, according to the current input character and the current The state is in the state transition rule set to find the post state; if it is the detached state, the detached state transition rule is based on the current input character, the current state, and the cache state. Then focus on the post-state.

6. The multi-string matching method according to claim 5, wherein the separated state transition rule set is configured to receive three inputs: a current input character, a current state, and a cache state, and respectively provide an output: After the state.

The multi-string matching method according to claim 5, wherein the step of buffering according to a specific cache rule is: if the current state is a converged state, the current state is cached.

8. A computer readable storage medium storing a plurality of instructions, wherein when said instructions are executed by a processor, said processor causes said steps to be:

Receive input characters; For each input character, perform the following steps:

Find the post state in the state transition rule base according to the current input character, current state, and cache state; - jump to the post state;

Stateful caching according to specific caching rules;

9. A system, comprising:

Processor

a bus coupled to the processor for transferring data between portions of the system; a communication interface coupled to the bus for receiving a stream of character data;

A main memory, coupled to the bus, in which is stored a number of instructions that, when executed by the processor, cause the processor to:

The characters are sequentially taken out as input characters from the received character stream; for each input character, the following steps are performed:

Jump to the post state;

Stateful caching according to specific caching rules;

A post-state search method, comprising: calculating a possible post-state according to a current state and an input character in conjunction with an input translation table; and searching a rule storage table according to the possible post-state to obtain a corresponding input character Comparing the actual input characters with whether the characters obtained by searching the rule storage table are consistent; if the results are consistent, converting the state to the Possible post state; if the results are inconsistent, the state is zeroed.

The post-state search method according to claim 10, wherein the numbering rule of the state comprises: if the current state has only one corresponding output conversion rule, the strip outputs a post-state to which the conversion rule points The number is the number of the current state plus one.

The post-state finding method according to claim 11, wherein the calculating the possible post-state comprises: according to a certain rule set, if the current state has only one corresponding output conversion rule, The number of the current state is incremented by one to obtain a number of possible post-states; if there is a corresponding output-converting rule for the current state, the color of the current state and the input character are taken as inputs, and the input translation is searched for The table obtains a difference between the number of the possible post state and the current state, and adds the difference by the number of the current state to obtain a number of possible post states.

The post-state finding method according to claim 10, wherein the rule storage table is configured to: have an input of a post state, and the corresponding output is a color of the post state and the post state Corresponding input characters.

The post-state finding method according to claim 10 or 12, wherein: the input translation table is configured to: the input is a color of the current state and the input character, and the corresponding output is a possible post state The difference in number between the current states.

The post-state search method according to claim 14, further comprising: performing entry merging on the input translation table, each row of the input translation table corresponding to a current state, and each column corresponding to one input character, The combination of the items is determined for each of the two rows to be merged, and the judgment of the kth column is as follows: If one of the two columns is empty, it is determined that the state corresponding to the empty column is non-empty after the merge. Whether the character received by the column data is equal to k, if it is, it is the overlay conflict, the two columns cannot be merged, and the exit, if not, the following judgment is made; if both columns are empty or not empty, judge two Whether the column corresponds to the same value, if not, it is a resource conflict, the two columns cannot be merged, and the exit, if yes, the next column is judged; until all the columns in the two rows to be merged are determined to have no resource conflict and overwrite conflict, then The corresponding rows are merged, and the non-null values cover the null values.

16. The post-state finding method according to claim 14 or 15, wherein: - further comprising performing group associative optimization on said input translation table, comprising the following steps of determining whether there is a resource conflict: The group is associated, and the ITT table is divided into 256/N groups. For a group, the number of valid values contained in the two rows is judged. If the number is greater than N, it indicates that there is a resource conflict in the group; otherwise, another one is judged. Group; The two lines are merged until it is determined that there are no resource conflicts in all 256/N groups.

17. A computer readable storage medium storing a plurality of instructions, wherein when the instructions are executed by a processor, causing the processor to perform the following steps: calculating a translation table according to a current state and input characters. a possible post state; check according to the possible post state Finding a rule storage table to obtain a corresponding input character; comparing whether the actual input character and the character obtained by searching the rule storage table are consistent; if the result is consistent, converting the state to the possible post state; If the results are inconsistent, the status is zeroed. .

The computer readable storage medium according to claim 17, wherein the numbering rule of the state comprises: if the current state has only one corresponding output conversion rule, the output switching rule points to The number of the subsequent state is the number of the current state plus one; the step of calculating the possible post state includes: according to a certain rule set, if the current state has only one corresponding output conversion rule, the current state is used The number is incremented by one to obtain a number of possible post-states; if there are multiple corresponding output conversion rules in the current state, the color of the current state and the input character are taken as inputs, and the input translation table is searched for A difference between the number of the possible post state and the current state is obtained, and the difference is added by the number of the current state to obtain a number of possible post states.

The computer readable storage medium according to claim 17, wherein the rule storage table is: the input is a post state, and the corresponding output is the color of the post state and the rear The input character corresponding to the status.

The computer readable storage medium according to claim 17, wherein the input translation table is configured to: input the color of the current state and the input character, and the corresponding output is a possible post state The difference between the numbers and the current state.

The computer readable storage medium according to claim 20, wherein each row of the input translation table corresponds to a current state, each column corresponds to one input character, and the input translation table is a past entry. In combination, the combination of the items is performed as follows: For each of the two rows to be merged, the judgment of the kth column is as follows: If one of the two columns is empty, the corresponding empty column is determined. Whether the character received by the non-null column data after the merge is equal to k, if yes, it is the overlay conflict, the two columns cannot be merged, and the exit, if not, the following judgment is made; if both columns are empty or both If it is not empty, judge whether the corresponding values of the two columns are the same. If not, it is a resource conflict. The two columns cannot be merged and exit. If yes, the next column is judged. Until all the columns in the two rows to be merged are determined to have no resources. Conflicts and overlay conflicts, the corresponding rows are merged, and the non-null values cover the null values.

The computer readable storage medium according to claim 20 or 21, wherein the input translation table is optimized by group association, and the group association optimization includes determining whether there is a resource conflict as follows. Step: For the N-way group association, divide the ITT table into 256/N groups. For one group, judge the number of valid values contained in the two lines. If the number is greater than N, it indicates that the group has a resource conflict. Otherwise, judge another group; until it is determined that there are no resource conflicts in all 256/N groups, then the two lines are merged.

23. A system, comprising:

The main processor, organizes the input data stream; a coprocessor unit, connected to the main processor;

Performing the following operations in the coprocessor unit: calculating a possible post state according to the current state and the input character in conjunction with the input translation table; searching the rule storage table according to the possible post state to obtain a corresponding input character; The actual input characters are consistent with the characters obtained by looking up the rule storage table; if the results are consistent, the state is converted to the post-energy state; if the results are inconsistent, the state is zeroed.

The system according to claim 23, wherein the numbering rule of the state comprises: if the current state has only one corresponding output conversion rule, the number of the post state indicated by the output conversion rule Adding a number to the current state; the step of calculating a possible post state includes: a certain rule set, if the current state has only one corresponding output conversion rule, adding one by the current state number Obtaining a number of possible post-states; if there are multiple corresponding output conversion rules in the current state, taking the color of the current state and the input character as inputs, searching the input translation table to obtain the possible The difference between the number of the post state and the current state, and the number of the current state is added to the difference to obtain the number of possible post states.

The system according to claim 23, wherein the rule storage table is configured to: the input is a post state, and the corresponding output is a color of the post state and a corresponding state of the post state Enter the characters. .

The system of claim 23, wherein the input translation table is configured to: input the color of the current state and the input character, and the corresponding output is a possible post state and the current The difference between the numbers in the status.

The system according to claim 26, wherein each row of the input translation table corresponds to a current state, each column corresponds to an input character, and the input translation table is merged by an entry, The merging of the table entries is as follows: For each of the two rows to be merged, the judgment of the kth column is as follows: If one of the two columns is empty, the state corresponding to the empty column is determined. Whether the characters received by the non-null column data after the merge are equal to k, if yes, the overlay conflict, the two columns cannot be merged, and the exit, if not, the following judgment is made; if both columns are empty or not Empty, determine whether the corresponding values of the two columns are the same. If not, the resource conflicts. The two columns cannot be merged and exit. If yes, the next column is judged. Until all the columns in the two rows to be merged are determined to have no resource conflicts. If the conflict is overwritten, the corresponding rows are merged, and the non-null values cover the null values.

The system according to claim 26 or 27, wherein the input translation table is optimized by group association, and the group association optimization includes the following steps of determining whether there is a resource conflict: The road group is connected, and the ITT table is divided into 256/N groups. For one group, the number of valid values included in the two rows is judged. If the number is greater than N, it indicates that the group has a resource conflict; otherwise, it is judged that a group; until all 256 N groups are determined to be incapable If the source conflicts, the two lines are merged.

29. A post state lookup structure, characterized in that it comprises:

Main memory: The basic conversion rule and the cross conversion rule are stored, and the input is a possible post state calculated according to the current state and the input character in conjunction with the input translation table, and the possible post state is output according to the stored conversion rule. a color and an input character corresponding to the possible post state;

Secondary memory: stores a failure conversion rule and a restart conversion rule, and the input is an actual input character, and outputs a post state corresponding to the actual input character and its color according to the stored conversion rule;

Input translation table: the input is the color of the current state and the actual input character, and the corresponding output is the difference between the possible post state and the current state;

A two-state gate: performing the following operations according to a comparison result between the character outputted by the main memory and the actual input character: if equal, converting the current state to the calculated possible post state, Converting the color of the current state to the color of the possible post state output by the main memory; otherwise, converting the current state and its color to the output of the secondary memory.

30. The post state lookup structure of claim 29, further comprising a comparator for performing a comparison between the character output by said main memory and the actual input character.

31. The post state lookup structure of claim 29, further comprising: a status register: configured to store the current state;

Color register: The color used to store the current state. ·

32. The post state lookup structure according to claim 29, further comprising: a gate: configured to selectively output the output value of the input translation table and the value 1 according to the value of the color register.

33. The post state lookup structure according to claim 32, further comprising: an adder: configured to add the number of the current state to an output value of the gate to calculate a possible post status.

34. A multi-string matching structure, characterized in that it comprises:

^! Large state register: used to store the current state;

Color register: The color used to store the current state;

Status buffer: used to store the cache status;

Color buffer: The color used to store the cache state;

Main memory: The basic conversion rule and the n-step cross conversion rule are stored, and the first input is the first possible post state calculated according to the current state and the input character and the input translation table, and the corresponding first output is the basis. The color of the first possible post state obtained by the stored conversion rule and the input character corresponding to the first possible post state; the second input is slow The stored state and the input character cooperate with the second possible post state calculated by the input translation table, and the corresponding second output is the color of the second possible post state obtained according to the stored conversion rule and the first The input characters corresponding to the two possible states;

Secondary memory: storing a failure conversion rule and a restart conversion rule, the input of which is the actual input character, and the output is the post state corresponding to the actual input character obtained according to the stored conversion rule and its color; The switching period of each current state is respectively covered by the state buffer and the color buffer by the post state and the color output by the secondary memory;

Input translation table: the first input is the color of the current ^1 dog state and the actual input character, and the corresponding first way output is the number between the first possible state and the current state The difference between the second input is the color of the buffer state and the actual input character, and the corresponding second output is the difference between the number of the second possible post state and the cache state; a three-state gate: performing the following operations according to a comparison result between the first path character and the second path character outputted by the main memory and the actual input character: if the first path character and the actual input If the characters are the same, the status register is overwritten with the first possible post state, while the color register is overwritten with the color of the first possible post state; if the first path character is different from the actual input character And the second way character is the same as the actual input character, the state register is overwritten by the second possible post state, and the color is covered by the second possible post state Said color register; otherwise, the state and the color memory with the secondary output of said status register and respectively covering the color register.

The multi-string matching structure according to claim 34, further comprising: a first comparator, configured to perform a comparison between the first path character output by the main memory and the actual input character; ·

And a second comparator, configured to perform a comparison between the second path character output by the main memory and the actual input character.

36. The multi-string matching structure according to claim 34, further comprising: a first strobe: configured to select and output an output value of the input translation table and a value 1 according to a value of the color register;

Second strobe: used to select and output the output value of the input translation table and the value 1 according to the value of the color buffer.

37. The multi-string matching structure according to claim 36, further comprising: a first adder: configured to add the number of the current state to an output value of the first gate to calculate The first possible post state;

The second adder is configured to add the number of the buffer state to the output value of the second gate to calculate a second possible post state.

38. A method of matching multiple regular expressions, comprising the steps of: sequentially taking characters as input characters from a received input character stream; and performing, for each input character, the following steps:

Jump to the post state;

Stateful caching according to specific caching rules;

The multi-regular expression matching method according to claim 38, wherein the step of the post-find state comprises: first determining whether the current state receives the current input character in the basic conversion rule and the n-step cross-conversion rule a post-existence state, if present, the post-state as a search result; if not, determining whether the cache state receives the current input character in the basic conversion rule and the n-step cross-conversion rule, and if so, The post state is used as the search result; if it does not exist, it is judged whether the initial state receives the current input character in the basic conversion rule and the n-step cross-conversion rule; if it exists, the post state is used as the search result; otherwise, the initial state is As a result of the search;

The multi-regular expression matching method according to claim 38, wherein the step of searching for the state includes: determining a type of the current state, and if it is a convergence state or a general state, according to the current input character and The current state is in the state transition rule set to find the post state; if it is the split state, the post state is searched in the separated state transition rule set according to the current input character, the current state, and the cache state;

The separated state transition rule set is set to receive three inputs: a current input character, a current state, and a cache state, and an output is provided correspondingly: a post state;