Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L63/00—Network architectures or network communication protocols for network security
  • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
  • H04L63/0227—Filtering policies
  • H04L63/0245—Filtering by information in the payload
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L43/00—Arrangements for monitoring or testing data switching networks
  • H04L43/02—Capturing of monitoring data
  • H04L43/026—Capturing of monitoring data using flow identification
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L63/00—Network architectures or network communication protocols for network security
  • H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
  • H04L63/0227—Filtering policies
  • H04L63/0263—Rule management
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L63/00—Network architectures or network communication protocols for network security
  • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
  • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
  • H04L63/1416—Event detection, e.g. attack signature detection
• • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
  • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
  • Y02D30/00—Reducing energy consumption in communication networks
  • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Definitions

• the field of invention relates generally to network equipment and, more specifically but not exclusively relates to techniques for performing packet content inspection across packet boundaries.
• Enterprise networks are threatened with a variety of security issues on a daily basis. These threats can be either internal or external. Internal threats range from break in attempts by disgruntled employees, virus or worms let loose by employees or just employee network access behavior deemed unacceptable per enterprise security policies. External threats range from dedicated attacks on the enterprise network in an attempt to steal intellectual property, denial of service attacks, unauthorized intrusions, viruses and worms etc. In all of these cases, a packet firewall is a network administrator's primary form of defense.
• Packet firewalls sit inline with network traffic, intercept incoming packets, and verify each packet against a set of firewall rules to accept, reject and optionally log the packet.
• network administrators sometimes also use packet filters to enforce traffic management policies. Such policies are useful in limiting or controlling offensive behavior. Due to the fact that packet firewalls sit inline with and inspect all network traffic, it is important that the firewall should be able to provide sufficient network throughput to keep up with network traffic demands.
• firewall rules are typically limited to 2500 Cisco ACL (access control list) rules These rules are fairly specific, and are designed to allow or reject specific activities or hosts. The ACL rules are limited to this number for performance and manageability reasons.
• the number of rules directly affects router performance, hence these rules are maintained at a low number. Moreover, large number of rules also makes rules management more error- prone and difficult to verify or modify. In a large organization, the rules are distributed across firewalls in various sites, and adapted as necessary, increasing chances of an error. For these reasons, the number of rules must be maintained at a manageable level. [0004] Under a typical advanced firewall implementation, two levels of filtering are employed. At the first level, filtering is performed based on applicable ACL rules. In this instance, a highest-priority rule corresponding to the ACL database is identified based on the packet header information.
• the rule may be identified based on a five-tuple input corresponding to values for the source and destination addresses, source and destination ports, and protocol using well-known classification algorithms. Since many attacks (particularly denial of service attacks) will originate from a known source address using a particular port, packets corresponding to these attacks can be readily identified, and appropriate rules (e.g., drop packet) may be employed to effect a desired firewall policy.
• This first level of filtering can be implemented at line-rate speeds using modem networking equipment. Under some implementations, dedicated components or separate computers are employed for performing these filtering operations.
• the second level of filtering relates to packet inspection. In this case, the actual packet pay load is searched for a particular string or set of strings.
• firewall applications may need to search for certain strings indicative of a virus or Internet worm that is present in the packet.
• other non-security applications may likewise need to peek into the packet payload, such as for load balancing or billing purposes.
• These operations known as “content inspection” or “(deep) packet inspection,” involve inspecting the packet payload for candidate patterns and taking actions based on the presence or absence of these patterns.
• packet/content inspection is offloaded to a separate application or sub-system that does not support line-rate speeds. For example, these operations may be performed by a separate computer host or embedded general-purpose processor coupled to or provided by a network device.
• Network processors also referred to as network processor units (NPUs)
• NPUs network processor units
• Figure 1 is a schematic diagram illustrating a technique for processing multiple functions via multiple compute engines using a context pipeline
• Figure 2 is a schematic diagram illustrating a software block architecture for implementing aspects of the embodiments of the invention described herein;
• Figure 3 is a flowchart illustrating operations and logic performed by one embodiment to determine whether content inspection is to be performed on a packet
• Figure 4a is shows various parts of a packet that are employed for flow classification
• Figure 4b shows a set of classification rules and associated flows
• Figure 5 is illustrates one embodiment of a hash table lookup scheme employing primary and secondary hash tables
• Figure 6 is a flowchart illustrating operations and logic performed during pre-processing operations to generate pattern data structures
• Figure 7 is a schematic diagram illustrating data flows and functional blocks employed by a network line card including an NPU having multiple compute engines used to execute instruction threads to perform packet processing operations including content inspection;
• Figure 8a is a flowchart illustrating operations and logic performed during content inspection of a first packet in a flow;
• Figure 8b is a flowchart illustrating operations and logic performed during content inspection of packets other than a first packet in a flow;
• Figure 9 is a flowchart illustrating content inspection operations that employ offset and depth search information;
• Figure 10 shows a goto function, and failure function, and an output function corresponding to an exemplary AC state machine data structure;
• Figure 11 shows one embodiment of a pattern data structure to be stored in memory
• Figure 12 shows one embodiment of an AC machine node memory data structure
• Figure 13 shows one embodiment of an AC machine index memory data structure
• Figure 14 shows one embodiment of a DAWG data structure to be stored in memory
• Figure 15 shows one embodiment of a suffix tree memory data structure
• Figure 16 shows one embodiment of a SHIFT table memory data structure
• Figure 17 shows one embodiment of a HASH table memory data structure
• Figure 18 shows one embodiment of a PREFIX table memory data structure
• Figure 19 shows one embodiment of a root node memory data structure
• Figure 20 shows one embodiment of a child node memory data structure
• Figure 21 shows one embodiment of a condition-action rule memory data structure.
• mechanisms that facilitate content inspection across packet boundaries while maintaining line-rate speeds and providing scalable solutions. These mechanisms employ state-based content inspection algorithms that allow content inspection to stop and restart at packet boundaries, while maintaining information about the results of partial string searches. At the same time, the mechanisms can be implemented with minimal buffering, while supporting line-rate speeds using NPUs. In order to better appreciate and understand the implementation of the mechanisms on modern network processors, a description of typical pipelined-based NPU processing operations is first described. Following this, details of the various embodiments are discussed.
• Modern network processors such as Intel's® IXP2xxx family of network processors, employ multiple multi- threaded processing elements (e.g., microengines (MEs)) to facilitate line-rate packet processing operations.
• MEs microengines
• the network processor In order to process a packet, the network processor (and/or network equipment employing the network processor) needs to extract data from the packet header indicating the destination of the packet, class of service, etc., store the payload data in memory, perform packet classification and queuing operations, determine the next hop for the packet, select an appropriate network port via which to forward the packet, etc. These operations are generally referred to as "packet processing" operations.
• packet processing Some of the operations on packets are well-defined, with minimal interface to other functions or strict order implementation.
• Examples include update-of-packet-state information, such as the current address of packet data in a DRAM buffer for sequential segments of a packet, updating linked-list pointers while enqueuing/dequeuing for transmit, and policing or marking packets of a connection flow.
• the operations can be performed within the predefined cycle-stage budget.
• difficulties may arise in keeping operations on successive packets in strict order and at the same time achieving cycle budget across many stages.
• a block of code performing this type of functionality is called a context pipe stage.
• different functions are performed on different microengines (MEs) as time progresses, and the packet context is passed between the functions or MEs, as shown in Figure 1.
• MEs microengines
• z MEs lOOo-j are used for packet processing operations, with each ME running n threads.
• Each ME constitutes a context pipe stage corresponding to a respective function executed by that ME.
• Cascading two or more context pipe stages constitutes a context pipeline.
• the name context pipeline is derived from the observation that it is the context that moves through the pipeline.
• each thread in an ME is assigned a packet, and each thread performs the same function but on different packets. As packets arrive, they are assigned to the ME threads in strict order. For example, there are eight threads typically assigned in an Intel IXP2800® ME context pipe stage.
• a more advanced context pipelining technique employs interleaved phased piping. This technique interleaves multiple packets on the same thread, spaced eight packets apart. An example would be MEO.1 completing pipe-stage 0 work on packet 1, while starting pipe-stage 0 work on packet 9. Similarly, ME0.2 would be working on packet 2 and 10. In effect, 16 packets would be processed in a pipe stage at one time. Pipe-stage 0 must still advance every 8-packet arrival rates. The advantage of interleaving is that memory latency is covered by a complete 8- packet arrival rate.
• novel content inspection schemes and mechanisms are implemented using NPUs that employ multiple multi-threaded processing elements.
• the schemes are implemented via a context pipelining technique under which content inspection of respective packets assigned to a common flow are performed using different instruction threads executing on one or more compute engines (e.g., microengines).
• content inspection state information is maintained in data structures that are accessible to the instruction threads and compute engines. For example, a first thread executing on a first compute engine is used to perform a string search on a first of multiple packets that collectively contain a string that matches a candidate pattern, yet is not contained entirely within a single packet (i.e., respective portions of the string are contained in respective packets).
• the first thread maintains state information pertaining to the string search, with corresponding information identifying a partial match when the end of the first packet is encountered.
• a second thread executing on either the same or a different compute engine is employed to perform a string search on a second packet that contains the remaining portion of the candidate pattern at its beginning.
• the second thread employs the state information generated by the first thread to continue the search at the point the first thread left off.
• appropriate action may be taken based on corresponding predefined rules or actions (a.k.a. "action targets").
• action targets a.k.a. "action targets"
• FIG. 2 shows a high-level view of various components, data structures and process blocks of a software architecture 200 for implementing aspects of packet-processing operations described herein.
• the components include a Deep Packet Inspector (DPI) module 202, a flow manager module 204, a rules engine module 206, a logger module 208, and a target handler module 210.
• the data structures include a flow table 212 and rules tables 214.
• the process blocks include an input verification block 216, a flow validation block 218, a rule match block 220, a log exception block 222, an exception generation block 224, a policy performance block 226, and a response generation block 228.
• the Deep Packet Inspection module 202 is used to perform various string search operations based on applicable search patterns, as described below in further detail.
• the flow manager 204 provides an API (Application Program Interface) to access and manage flow table 212.
• the rules engine 206 provides an API to the rules table 214, implementing a level of translation. For example, a user specified rule will undergo multiple levels of translations before it gets to a form usable by the fast path (e.g., compute engine) processing code.
• DPI module 202 should be considered as a collection of various protocol-specific functions that are implemented via corresponding threads run on NPU compute engines.
• the logger module 208 manages the logging function. This module may either handle logging directly or use services from another component in a different processor (e.g., a general-purpose processor in an NPU or a processor in a separate computer host) to perform the logging function. Typically, the logging function in a firewall might range from simple text-based logging to dumping data into relational databases for extensive report generation. Complexity of this module will vary depending on the functionality provided.
• a packet As a packet enters the system, it is first verified for protocol conformance, checksum etc., by input verification block 216. Once this sanity check is performed, the flow validation block 218 verifies whether the packet belongs to a known (e.g., existing) flow or requires a new flow. If the packet does not belong to any known flow, a new entry is added to flow table 212, and the packet metadata (e.g., header data) is passed to rules matching block 220. This block will iterate through a set of match structures, evaluating conditions for every match. The target actions are encoded in the packet metadata as they are matched. Thus a packet can have more than one target action associated with it. For example, it might be required to log the packet as well as forward it.
• a packet can have more than one target action associated with it. For example, it might be required to log the packet as well as forward it.
• the packet is passed to the target handler block 210 for further processing.
• This block performs the actions associated with the packet metadata.
• the following operations are performed during processing of each input packet.
• the process begins at a block 300, wherein flow classification is performed. In general, flow classification may be performed by one of several schemes that are commonly employed for this purpose.
• flows will be classified based on an ACL rule database; however, other classification mechanisms may also be employed.
• the purpose of flow classification is to assign the packet to a corresponding flow.
• the rules for classifying a message e.g., packet
• filters or rules in firewall terminology
• the packet or flow classification problem is to determine the lowest cost matching filter or rule for each incoming message at the network element.
• the relevant information is contained in K distinct fields in each message (packet).
• the relevant fields for an IPv4 packet could comprise the Destination Address (32 bits), the Source Address (32 bits), the Protocol Field (8 bits), the Destination Port (16 bits), the Source Port (16 bits), and, optionally, the TCP flags (8 bits).
• Figure 4a shows an exemplary packet structure used illustrating a 5-tuple flow classification scheme. In general, this scheme can be extended for any .K-tuple classification.
• Each packet will include various fields in its header, with the number and specific fields being dependent on its transport mechanisms (e.g., TCP/IP packets will have different fields than UDP packets).
• the packet payload data also commonly referred to packet data or payload follows the packet header data.
• An optional packet footer may be included at the end of the packet.
• FLDs 1-5 Each packet field contains a corresponding field value (Vl - VN).
• FLDs 1-5 represent header information, such as source and destination addresses, source and destination ports, protocol information, error checking information, and the like.
• Payload field 400 defines that portion of the packet containing payload data.
• Information in the footer, as depicted by FLD N, may also be employed for flow classification in some cases.
• a packet When a packet is received, it is parsed to extract selected field values from its header based on the classification scheme. For the illustrated example, these will be values V1-V5 from FLDs 1-5. The extracted field values are then used to search a rule database to determine the set of classification rules that the packet matches, and hence the associated set of resulting flows.
• the filter (ACL) database consists of a finite set of filters, filti, fif ⁇ ... with each filter associated with a corresponding classification rule and flow.
• Each filter is a combination of K values, one for each field (or partial field) applicable to the classification scheme.
• Each field in a filter is allowed three kinds of matches: exact match, prefix match, or range match.
• exact match the header field of the packet should exactly match the filter field.
• the filter field In a prefix match, the filter field should be a prefix of the header field.
• the header values should like in the range specified by the filter.
• Each filter/?/? / has an associated directive ⁇ i.e., action), which specifies how to handle a packet matching the filter.
• classification rule matching may be performed using one of many known techniques. For example, under one embodiment an aggregated bit vector scheme is employed, in which each field value is associated with a corresponding bit vector identifying matching rules in the ACL table. (In practice, the bit vectors are stored using an aggregated or hierarchical scheme to reduce the storage size of each bit vector, since many if not most of the bit values will be O's, indicating no match.) The bit vectors are then logically ANDed, with the result identifying the matching rules. The highest-priority matching rule may then be easily identified.
• the flow table entries are maintained in a hash table that uses a double hashing scheme.
• the hash table is organized as two tables, a primary table maintained in SRAM (synchronous random access memory) and a secondary table maintained in the DRAM (dynamic random access memory).
• the primary table is divided into a set of slots, while each slot entry corresponds to a flow entry in the secondary table.
• the primary table is indexed by two indices: the first value identifies the slot word (a slot is an 8 bit value - 7 bits of signature and 1 flag bit), while the second value (the signature) is stored in the slot and is used to identify the slot.
• the flow entry in the secondary table is indexed by the slot word index and the slot number with a matching signature value.
• the flow table is maintained as two sets of hash table; if a search for a flow entry in the first table fails, a search of that entry in the second table is performed.
• content inspection may or may not be required. For example, packets that are sent from a "trusted" source address or sent to a particular destination port may be deemed by the rule database as trustworthy and assigned to a corresponding trusted flow having an associated policy (i.e., action) of "forward.” In contrast, packets sent for an untrusted source address might be assigned to an untrusted flow that has an associated policy of "drop.” In other cases, the flow may have an associated policy of "inspect content,” which requires content inspection to be performed. As depicted by a decision block 308, if content inspection is required, content inspection operations are initiated with the retrieved state information in a block 312. For packets for which content inspection is not required, continuation of packet processing is performed, as depicted by a continuation block 310.
• a decision block 308 if content inspection is required, content inspection operations are initiated with the retrieved state information in a block 312. For packets for which content inspection is not required, continuation of packet processing is performed, as depicted by a continuation block 310.
• the implementation of the search mechanism involves two phases: a build or pre-process phase and a run-time phase.
• a build phase various data structures and tables are built and saved to be later stored in memory during associated initialization operations.
• the run-time phase which is generally ongoing, searching is performed on received packets using the data structures and tables.
• FIG. 6 Details of one embodiment of the build phase are shown in Figure 6.
• the process begins in a block 600, wherein the search criteria are defined.
• This can be stated in a very general way as the problem of searching for the occurrence of a pattern (string) within another string (packet data in this instance).
• This problem is identical to the classic string- matching problem, which is stated in formal terms as: Given a text string T of 'n' characters and a pattern string P of 'm' characters, find the positions of all occurrences of P in T.
• the above problem can be mapped to the problem of finding the presence of a virus or worm signature, protocol command or the name of a known executables in packet data. It will usually be sufficient to just search for the presence of P in T, rather than to find all occurrences (which does not really mitigate the problem, given that finding the first occurrence is usually as "hard” as finding multiple occurrences, in theoretical terms).
• the signature string to search for may not be known beforehand; the best that can be done is to narrow the field down to a set of patterns that need to be search for, in the context of a particular flow.
• a more general problem is to search for occurrences of any, or all, of a set of patterns ⁇ P ⁇ within a given text T.
• the exact string, or strings, to be matched will not be known beforehand. Signatures can also occur in morphed forms, where the morphing is usually a small change to the original pattern. In such cases, there is a need to handle approximate matching of strings also. Sometimes there may also be a need to match regular expressions instead of known strings. Techniques for all of these are currently being employed in data mining and search applications. In general, these algorithms are somewhat more complicated than the simple string searching one, and have higher memory and cycle requirements.
• pattern sets ⁇ P ⁇ may apply to different policies. For example, certain flows may only need to be searched for a particular pattern set, while other flows may include searches for broader or entirely different pattern sets. Accordingly, an appropriate pattern set is specified for each policy. In general, and depending on the search algorithm that is employed, the pattern sets may include exact matches and/or approximate matches (e.g., matches using wildcard characters).
• DS pattern data structures
• offset and or depth information may also be associated with the search criteria. Oftentimes, more information is known about a particular pattern than just the string of bytes it is comprised of. Such information may help narrow down the region in the packet (or associated bit stream) where the pattern could occur. As used herein, this information includes the Offset and Depth.
• the Offset value indicates the number of bytes that can be skipped from the beginning of a packet (or bit stream) before starting the search.
• the Depth value indicates the maximum number of bytes within the packet (or bit stream) that need to be searched to have a chance of finding the pattern.
• the Offset and Depth values are also referred to as "Tagged" values.
• Pattern data structures are incorporated (e.g., adding) appropriate tagged value information in the applicable Pattern data structures, as depicted in a block 606 and explained in further detail below.
• the pattern data structures are saved to a file or the like in a block 608.
• the pattern data structures will be loaded into memory and/or code stores, and corresponding pattern data lookup data will be loaded into memory, as depicted by respective blocks 610 and 612. [0066] To illustrate how and where this data might be stored, attention is directed to the execution environment illustrated in Figure 7.
• the execution environment pertains to a network line card 700 including an NPU 702 coupled to an SRAM store (SRAM) 704 via an SRAM interface (I/F) 705, and coupled to a DRAM store (DRAM) 706 via a DRAM interface 707.
• Selected modules are also depicted for NPU 702, including a deep packet inspection module 202, a flow manager module 204, a queue manager module 708, a receive engine 710, and a buffer manager 712.
• the operations associated with each of these modules are facilitates by corresponding instruction threads executing on MEs 714.
• the instruction threads are initially stored in an instruction store 715 comprising a non- volatile storage device, such as flash memory or a mass storage device or the like.
• SRAM 704. various data structures and tables are stored in SRAM 704. These include a flow table 212, a policy data structure table 716, and a pattern data structure table 718.
• packet metadata 720 is typically stored in SRAM as well. As described above with reference to Figure 5, respective portions of a flow table may be split between SRAM 704 and DRAM 706; for simplicity, all of the flow table 212 data is depicted in SRAM 704 in Figure 7.
• information that is frequently accessed for packet processing e.g., metadata 720
• DRAM DRAM
• the memory space available in the DRAM store is much larger than that provided by the SRAM store.
• each ME 714 includes a local memory 722, local registers 724, separate SRAM and DRAM read and write buffers 726 (depicted as a single block for convenience), a code store 728, and a compute core (e.g., Arithmetic Logic Unit (ALU)) 730.
• ALU Arithmetic Logic Unit
• information may be passed to and from an ME via the SRAM and DRAM write and read buffers, respectively.
• a next neighbor buffer (not shown) is provided that enables data to be efficiently passed between ME's that are configured in a chain or cluster.
• each pattern data structure will provide information for effectuating a corresponding state machine or table-based machine.
• the various pattern data structures will typically be stored in pattern data structure table 718, as illustrated in Figure 7. However, there may be instances in which selected pattern data structures are stored in selected code stores that are configured to store both instruction code and data.
• FIG. 8a An overview of operations performed during the run-time content inspection phase are illustrated in Figures 8a and 8b, wherein Figure 8a depicts operations performed during processing of a first packet in a flow, and Figure 8b depicts operations performed during processing of any subsequent packet in the flow (as depicted by the I th packet).
• Each flow has an associated queue that is stored in DRAM 706, as depicted by a flow queue 736.
• the queues are generally managed by queue manager 708 and/or flow manager 204.
• associated information is added to metadata 720, such that the first (head) and last (tail) packet in the queue can be readily identified for subsequent operations, such as dequeuing.
• Packet streams are received from various network nodes in an asynchronous manner, based on flow policies and other criteria, as well as less predictable network operations.
• packets from different flows may be received in an intermixed manner, as illustrated in Figure 7.
• each of input packets 732 is labeled with F#-#, wherein the F# identifies the flow, and the -# identifies the sequential packet for a given flow.
• packets do not contain information specifically identifying the flow to which they are designed, but rather such information is determined during flow classification.
• the packet sequence data is provided in applicable packet headers, such as TCP headers (e.g., TCP packet sequence #).
• flow queue 736 contains the first five packets in a Flow #1. As before, packets are added to associated flow queues after they have been classified during enqueuing operations performed in block 800.
• each flow table entry includes a flow ID, a buffer pointer, a policy pointer, a pattern data structure pointer, state information, and statistics information. (A flow table entry may also include other information that is not shown.)
• the flow ID is used for flow table entry lookup, and in one embodiment comprises a hash index in accordance with the flow table index scheme shown in Figure 5.
• the buffer pointer points to applicable metadata 720, which in turn includes pointers to the head and tail packets in the flow queue associated with the flow table entry.
• the policy pointer points to a policy data structure in policy data structure table 716. As discussed above, each flow is associated with a policy. During flow classification, the policy is identified, and a corresponding policy pointer is provided to flow manager 204.
• each policy that requires content inspection is also associated with a pattern data structure.
• the pattern data structure pointer may be extracted from the policy data structure.
• the state information is used to store the state of a corresponding algorithm state used by deep packet inspection module 202, and is described in further detail below.
• the statistics information is used for various purposes, including identifying the current sequence number of the last packet in the flow for which content inspection was completed. In this manner, the next packet to be inspected for a given flow may be identified based on its relative location to the head packet in an associated flow queue.
• a first thread will be launched on an ME 714 to perform a pattern search of the packet to identify any matches in view of the applicable pattern set ⁇ P ⁇ .
• data corresponding to the new flow table entry for this packet is read by the thread, including the state information, which will indicate the state machine is in the initial state.
• the location of the packet in the associated flow queue is identified using the buffer pointer and metadata, and the packet is copied to local memory 722 of the ME.
• the applicable pattern data structure (as identified by the pattern data structure pointer) is retrieved from pattern data structure table 718 and copied into local memory 722.
• the existing state 738 of the corresponding search algorithm is initialized to the initial state, which indicates no partial matches currently exist.
• the pattern search operations begin at the start of the packet (e.g., start of the packet payload data) and is performed in an iterative manner until the end of the packet is reached.
• the applicable algorithm machine employs a search increment that may comprise one or more characters, depending on the particular search algorithm.
• the existing state 738 of the machine is updated, as applicable.
• a block 812 a determination is made to whether a full match has been identified by the machine. If so, the associated action for the search condition is performed, as depicted by a continuation block 814, and the thread is terminated.
• the associated action may be performed by the same or another (more likely) thread on the same or another ME.
• the associated action may be performed by the NPU' s general-purpose processor (not shown in Figure for clarity), hi the case that the matching string identifies the existence of a malicious worm or the like, the associated action will typically be to drop all remaining packets in the flow, and to possibly update the ACL database action for the source address.
• Dropping the remaining packets may be accomplished by updating the policy pointer in the associated flow table entry such that the policy for the flow is effectively switched from "inspect content" to "drop.”
• the state and statistics information in the associated flow table entry is updated in a block 818.
• execution of the first thread is terminated.
• Figure 8b upon receipt of the subsequent packets classified to the same flow, similar operations are performed, as indicated by like-numbered blocks in Figure 8a and 8b.
• an /th thread is launched on the same or another ME 714, wherein i represents the packet sequence number (normalized from the actual sequence number in the header).
• the flow table entry that is read will be an existing entry that includes an existing state that was updated by the previous packet.
• the flow table entry information including the statistics, is used to locate the packet in the flow queue, whereupon it is copied to the ME's local memory 722 in block 804.
• the applicable pattern data structure is retrieved by the ME in a manner similar to block 806A of Figure 8a.
• the state of the machine will be initialized to the existing state updated by that previous packet. If a partial match was present at the end of the previous packet, the existing state will be somewhere in the machine other than the initial state (e.g., 0). If no partial match was present, the existing state will be the initial state.
• the pattern search operations of blocks 808, 810, 812, 814, and 816 are preformed in a similar manner as applied to the first packet, with the only difference being the initial state of the machine. Accordingly, if a partial match existed at the end of the previous packet, the pattern search is began at the point the state machine left off, looking for the second portion (the remaining matching portion) one or more strings in the pattern set ⁇ P ⁇ (depending on the existing state). As before, if a full match is identified, the associated action is performed in continuation block 814.
• the search criteria may specify offset and depth values for respective pattern strings.
• the pattern search processing is augmented to account for this additional search information.
• Figure 9 we begin at the start of a packet, with the prior operations of Figure 8a or 8b (as applicable) having already been performed.
• a block 900 a portion of the packet is skipped to reach the offset, at which point the search will begin.
• this operation may be implemented on an individual pattern search basis, such that some patterns are offset at different points than others when a multi- pattern search is employed. If the packet is a first packet in a flow, the number of bytes corresponding to the offset will be skipped from the beginning of the packet.
• the offset for one or more search patterns will be greater than the entire length of the first packet. Accordingly, in this case, the entire packet is skipped for those search patterns, and the length of the first packet will be deducted from the offset value, such that the portion of the offset to be skipped for the following packet will consider how much of the bit stream will have been already skipped. For example, consider a case in which the offset for a particular search pattern is greater than the maximum size of the packet pay load for a give protocol. For this particular search pattern, the entire first packet will be skipped, and the search will commence at an offset in the second packet that equals the difference between the original offset and the size of the first packet.
• this multi-packet offset scheme may be facilitated by updating appropriate statistics information in the associated flow table entry, as discussed below with reference to block 920.
• the pattern search involves an iteration over the packet payload data, with the location within the data being advanced by the applicable search increment being employed by the search algorithm. The starting point for each search pattern in pattern set ⁇ P ⁇ will be the effective offset for that search pattern.
• the machine state is updated in a block 904. As indicated by a decision block 906, if a search string match is indicated by the machine, an associated action is performed in a continuation block 908.
• the logic proceeds to a decision block 910 to determine whether the depth for the applicable search string has been reached. If so, searching for that string is complete, as depicted by an end block 912. If not, the process loops back to start loop block 902 to begin the next iteration.
• a decision block 916 Upon reaching the end of the packet, two actions may result, depending on the implementation, as depicted by a decision block 916. For embodiments that only search the first packet in a flow, the search process for the flow is complete, with no match being found. As a result, the policy pointer in the flow table entry is updated in a block 918 to point to a policy that does not perform a content search, such that a search is not performed on the remaining packets in the flow. If a multi- packet search (with regard to the offset aspect) is supported, the statistics information is updated in the flow table entry to account for the size of the skipped packet in the offset to be employed for the following packet, as depicted in block 920.
• the effective depth employed for a given search pattern can be adjusted to support multi-packet searches. For example, consider a situation in which the offset is less than the size of a first packet, while the depth is greater than the size of the first packet. This situation may be handled by reducing the depth value that is applied to the second packet by the size of the first packet. This operation may also be performed in block 920. It is further noted that under this scenario, the state information would also be updated in the flow table entry to reflect the state of the state machine at the end of the first packet in the same manner discussed above. [0090] In one embodiment the mechanism for effecting offset and depth operations is provided via data contained in the applicable pattern data structure.
• a block 922 is included in Figure 9 to illustrate that the pattern data structure is adjusted to account for the offset and/or depth under the multi-packet implementation. This operation will be performed by another thread (launched to process a next packet in the flow) using the statistics information that was updated in block 920.
• the preprocessing mainly involves computing the machine transition function.
• a run of the algorithm involves feeding the search text into the FSM as input. An occurrence of P is detected whenever an accepting state is reached.
• any type of FSM algorithm may be implemented for the pattern searches employed by the embodiments described herein.
• Aho-Corasick (AC) algorithm A. V. Aho, M. Corasick, "Efficient string matching: an aid to bibliographic search", Communications of the ACM, vol. 18 (1975) 333 — 340
• K ⁇ y ⁇ , yi ..., yO be a finite set of strings called keywords and let JC be an arbitrary string which we shall call the text string.
• Our problem is to locate and identify all substrings of JC which are keywords in K. Substrings may overlap with one another.
• a pattern matching machine for K is a program which takes as input the text string JC and produces as output the locations in x at which keywords of K appear as substrings.
• the pattern matching machine consists of a set of states. Each state is represented by a number. The machine processes the text string x by successively reading the symbols in JC, making state transitions and occasionally emitting output.
• FIG. 10 shows the behavior of the pattern matching machine for the set of keywords ⁇ he, she, his, hers ⁇ , wherein the states are 0, 1 . . . . ,9.
• the goto function g maps a pair consisting of a state and an input symbol into a state or the message fail.
• All the pattern matching machines have the property that g (0, ⁇ ) ⁇ fail for all input symbols ⁇ . This property of the goto function on state 0 ensures that one input symbol will be processed by the machine in every machine cycle.
• the failure function /maps a state into a state. The failure function is consulted whenever the goto function reports fail.
• Certain states are designated as output states that indicate that a set of keywords has been found. The output function formalizes this concept by associating a set of keywords (possibly empty) with every state.
• the current state of the machine is the start state and the first symbol of the text string is the current input symbol.
• the machine then processes the text string by making one operating cycle on each symbol of the text string.
• an FSM-based algorithm known as the DAWG- Match algorithm (M. Crochemore, A. Czumaj, L. Gasieniec, T. Lecroq, W. Plandowski, and W. Rytter, "Fast practical multi-pattern matching", Inf. Process. Lett., 71((3-4)):107-113, 1999) is employed.
• This algorithm combines the ideas of Aho-Corasick with Boyer-Moore to get a better average running time. It requires computation of both an AC state machine as well as a DAWG (Directed Acyclic Word Graph) for the reverse patterns of the given pattern set.
• a shift table is also computed, which stores the maximal shift at a particular state that guarantees no occurrence of a pattern in the skipped area (the state corresponds to the longest prefix of a pattern matched to a suffix of the text scanned so far); this function corresponds to the horrico-Giancarlo skip function.
• the DAWG is used to first scan from right to left until a mismatch, upon which the AC machine is used to scan from left to right to recognize prefixes, and decide on the shift value. If the set of character matched using the DAWG overlap with the prefix matched by AC in the previous phase, a match is reported.
• Prev and Next values will be NULL if not defined. Default values of offset and distance will be 0, unless explicitly defined. Default values of depth, within, exact and relative will be -1, unless explicitly defined.
• pointer values prev and next are used.
• prev_pat_num and next_pat_num are employed as pattern index values are used more ubiquitously (actual pattern address can be computed form these values, as pattern structure is of constant size).
• addresses are aligned to word (4 bytes) boundaries.
• the actual pattern string is a sequence of bytes starting at the byte address indicated in the first field of the data structure.
• pattern strings are stored contiguously and immediately after the end of the last pattern data structure. This enables pre-allocation of memory, computation of relative addresses during pre-processing and requires only one pointer (address of first pattern structure) to be maintained.
• the AC machine employs a deterministic finite state automaton (Q, ⁇ , f, so, T), where Q is the set of states, ⁇ is the transition function, / is the failure function, so is the start state and T is the set of accepting states.
• the transition function determines which state to move to from a particular state on a given input.
• the failure function indicates exception transitions, for ⁇ state, input) pairs that are not defined by the transition function.
• the AC machine is structured as a trie of nodes, and can be specified just by the pointer to the start state.
• the follow data structure is employed, while a corresponding memory data structure is shown in Figure 12.
• struct transition ⁇ char input ; /* Byte value that triggers state transition */ struct AC_node * destination; /* Pointer to destination AC-machine state (node) */ ⁇ ; struct AC_node ⁇ int node_id; /* Unique integer as state id */ int shift; /* Maximum safe shift value (for DAWG- Match): shift is the minimal distance that can be skipped with a guarantee that no pattern can occur in the skipped area */ int offset ; /* Prefix indicated by this state must end on or after first offset bytes of text*/ int depth; /* Prefix indicated by this state must end within depth bytes of text */ struct AC_node *fail ; /* Point
• the default value of depth is — 1, unless explicitly defined or calculated. Pointers are used for destination, fail and output only during pre-processing (computation of the machine).
• the microcode implementation uses pattern_numbers and node_id for states (actual memory addresses are computed from these).
• an AC machine index structure is employed, as shown in Figure 13.
• the index and each of the AC machine nodes are designed to be independently relocatable in memory, it is advantageous to store the state information in contiguous fashion ordered by state ID, and to store the structure information immediately after the index. This allows different addresses (relative) to be computed during pre-processing, reduces the number of pointers to be maintained to one (base address of index structure), and enables pre-allocation of memory.
• a directed acyclic word graph, or a suffix automaton recognizes all suffixes of patterns in a given set (the DAWG-match algorithm takes reversed patterns as input).
• a DAWG does not have a failure function, and states don't need to have output patterns associated with them.
• struct DAWG_node ⁇ int node_id ' /* Unique integer as state id */ int num_transitions ; /* Number of transitions from this state */ struct suf_transition *next ; /* List of transitions from this state, ordered by ASCII value of input byte */ struct DAWG_node *suf f ix_pointer; /* Pointer to parent in a virtual tree that is used for computation of the DAWG */ ⁇ ;
• the Default values of flag and numjransitions are 0, unless explicitly specified or calculated.
• the flag is used only for construction of the DAWG, and has no role to play in actual string matching. Pointers are used for destination only during pre-processing (computation of the machine).
• the microcode implementation uses node_id for states (actual memory addresses are computed from these).
• the DAWG machine index structure is identical to the AC machine index structure.
• the DAWG indices and structures are stored contiguously in a manner similar to that discussed above for the AC machine.
• a suffix tree like a DAWG, recognizes all suffixes of patterns in a given set (the AC-Suffix-Tree algorithm takes reversed patterns as input). This structure is essentially a DAWG unraveled into a tree, thus having a unique path from source to any other state.
• struct tree_transition ⁇ char input ; /* Byte value that triggers state transition */ struct suff ix_node *destination; /* Pointer to destination suffix tree state (node) */ ⁇ ; struct suffix_node ⁇ int node_id; /* Unique integer as state id */ int num_transitions ; /* Number of transitions from this state */ struct tree_transition *next; /* List of transitions from this state, ordered by ASCII value of input byte */ struct AC_node *ACnode; /* AC-machine state where scan will end, starting from the source and the reverse of the pattern recognized by this suffix tree state as input */ ⁇ ;
• the Default value of numjransitions is 0, unless explicitly specified or calculated.
• the ACnode is what differentiates a DAWG and a Suffix Tree. In the latter, each state is reached by a unique path of bytes (substring of a pattern) from the start state. Therefore, each state will have a unique AC machine node representing that traversed string. For example, let state s in the suffix tree be reachable from the start state by a string C 1 C 2 C 3 .... ⁇ . Let this string be provided as input to the AC machine. If the AC machine scan stops at a state d, the value of s- > ACnode will be set to d.
• Pointers are used for destination and ACnode only during pre-processing (computation of the machine).
• the microcode implementation uses nodejd for states (actual memory addresses are computed from these).
• the suffix tree machine employs an index structure that is identical to the AC machine index structure, while the suffix tree indices and structures are stored contiguously in a manner similar to that discussed above for the AC machine.
• the pattern list is scanned for case-independent patterns.
• case-independent patterns In order to keep the state machines deterministic and to be able to recognize such patterns, whatever combinations of case they occur in (within the packet payload), all combinations of byte strings that the particular pattern could occur as are generated in one embodiment. All these new combinations are added to the pattern list, but the prev pointer is used to point to the original pattern, so that memory space is not wasted in storing these exponential numbers of new patterns.
• Offset and Depth computation If patterns have associated offset and depth that are not equal to the default start and end of packet, we need to compute values for each state. If we did not compute these values for each state, we would be making unnecessary state transitions; only when an accepting state was found would we know whether or not the output pattern was valid, and in the process, we might miss some valid patterns; and Incorporating case independent patterns: By computing all possible combinations of strings for case independent patterns, we ensure that a case independent pattern will always be recognized. In one embodiment, the output at any accepting state as the original pattern is marked, rather than having to store unnecessary combinations of patterns.
• the DAWG and suffix tree algorithms are implemented via execution of microcode threads on selected ME's.
• the operations are summarized as follows. First, an attempt is made to try to match the longest prefix possible of any pattern in the pattern set ⁇ P ⁇ to a suffix of T.
• the prefix value can be represented by an equivalent state of the AC machine that recognizes patterns in the set ⁇ P ⁇ .
• the prefix of a pattern matched in the previous pattern is the empty string, so the prev_state input will be the start state of the AC machine. If, for any packet, this value is not equal to the start state, the AC machine is used to scan T from its first character in a forward direction, starting from prev_state, till a suitable shift value is obtained and the DAWG can start its scan.
• This mechanism for detecting patterns spanning multiple packets will work only if packets of the flow arrive in sequence.
• the prev_state value makes sense only if the current packet has a sequence number that is one higher than the packet that arrived previously in the flow. If packets don't arrive in order (this applies to TCP and other connection-oriented protocols), such pattern detection cannot be performed just by storing one prev_state value in the flow entry. This problem may be addressed for some situations by storing multiple instances of such state values, along with associated packet sequence numbers.
• the offset and depth for a state are calculated based on distance from start state and distance until the end of the pattern, respectively.
• the offset is calculated as the minimum of the current computed value and the previously set value;
• the depth is calculated as the maximum of the current computed value and the previously set value (an unspecified depth will be infinity, a value higher than all specified values).
• Offset and depth need not be changed during fail state computation; if we come across a state with an invalid offset or depth during packet scan, a jump is automatically made to the fail state.
• search schemes based on the Wu-Manber algorithms [WM92] S. Wu, U. Manber, "Fast Text Searching Allowing Errors", Communications of the ACM 35(10): 83-91(1992); and [WM94] S. Wu and U. Manber. "A fast algorithm for multi-pattern searching”. Technical Report TR-94-17, University of Arizona, 1994) are employed.
• Wu and Manber take a very different approach from the state-machine based algorithms described above. These algorithms also differ from other approaches by matching blocks of characters at a time, rather than single characters. If the total length of all patterns is M, the recommended size of the block (B) is log ⁇ ⁇ ⁇ 2M.
• the algorithm pre-computes three lookup tables called SHIFT, HASH and PREFIX.
• SHIFT table is indexed by an integer that is a hashed mapping of a block of characters; this table is equivalent to the bad character table in the well-known Boyer-Moore algorithm (Boyer and J. Moore. "A fast string searching algorithm". Communications of the ACM, 20:762- 772, 1977.)- Entries for substrings common to multiple patterns contain the minimum possible shift value, to be more conservative. For larger values of B, tables could be compressed by mapping several strings to the same location, setting the minimum of the shift values in each case.
• the integer used to index to SHIFT is also used to index to HASH, which stores pointers to strings that map to that integer.
• the PREFIX table contains hash values for prefixes of each pattern, the prefix size being implementation-dependent.
• a set of B characters at the right end of the text substring being scanned
• an integer which is used to lookup the SHIFT value. If this value is greater than 0, the text is shifted. Otherwise, the hash of the prefix of this substring (of size m s , where m s is the length of the shortest pattern) is computed, and matched to the prefix of each string in the corresponding HASH entry (using the PREFIX table). If a prefix matches, actual text match takes place. This algorithm depends on the pattern sizes being mostly uniform, since only the shortest pattern length is considered during matching.
• the pattern data structure as well as its corresponding memory data structure are identical to that used for the DAWG-Match algorithms. Standardizing the pattern data structure makes it easy to extend the content inspection framework and implement to different search algorithms.
• the shift table is structured as a 2- dimensional array of integers S[n][4]. As shown in the exemplary shift table memory data structure of Figure 16 and below, each table entry S[k] comprises a pointer to an integer array, which has the following structure:
• 'k' is the value (usually a long integer) used to index to the shift table. If the size of the alphabet is A and the size of the substring (chunk of string) that we will use as hash value is B, the maximum size of the table is A B .
• 'k' is a (unique) hash value obtained from a block of B characters (the simplest hash function is to consider the block to be a base-A number of B digits).
• the primary shift refers to the distance by which the scan position must be shifted to the right. This value is 0 only if the scanned substring matches a substring at a suitable position in one of the patterns.
• the offset is the least distance from the beginning of the packet content where a valid pattern ending with the current substring under consideration (given that all patterns are truncated to the least pattern size) may occur.
• the depth is the maximum distance from the beginning of the packet content within which a valid pattern ending with the current substring under consideration (given that all patterns are truncated to the least pattern size) must end.
• the offset, depth and secondary jshift values make sense, and are used, only if the primary shift value is 0, i.e., when a substring in the text (packet content) matches the end of one of the patterns under consideration.
• the HASH table is structured as a single dimensional array of pointers H[n].
• each table entry H[k] comprises a pointer to a pattern data structure.
• the index into the table is identical to that for the Shift table.
• the patterns (truncated to the minimum pattern size) are sorted by the hash values of the chunk of B characters they end with.
• the value of H[k] is the first pattern in this sorted list that has a hash value greater than or equal to 'k'.
• the patterns data structures have a fixed size, they can be accessed in a random manner from memory. Therefore, the address of a pattern data structure can be calculated using the sequence number of the pattern in the array. Accordingly, the values stored in the HASH table in memory comprise pattern index numbers (ranging from 0 to number _of_patterns - 1), rather than actual addresses, in order to make the patterns relocatable.
• the PREFIX table is structured as a single dimensional array of integers (usually long integers). The size of the table is equal to the number of patterns in the pattern set, and the pattern (or pattern index number) is used as index to any entry of the Prefix table.
• the value Prefix[k] is a hash of the first B' characters in pattern number 'k' ('k' ranging from 0 to number _pf_patterns -1).
• the hash is typically computed in an identical way to that computed for the Shift table (by considering the prefix to be a base- A number consisting of B' digits).
• each table entry is the unmodified prefix value, since the alphabet to be employed comprises the entire ASCII set.
• Both the SHIFT and HASH tables contain a large amount of redundant information. The size of these tables is exponential, while the set of patterns is likely to be few in number.
• This B+ tree is structured as a tree of nodes, where each leaf has constant distance from the root. Though the number of levels could be arbitrary, we have chosen the depth to be two, which will suffice for most practical purposes.
• Each child node is an array of the following structure: struct hash_record ⁇ long key_value ; /* Hash value (index to SHIFT and HASH tables) */ int primary_shift ,- /* Primary shift (as in the SHIFT table) */ int secondary_shi f t ; /* Secondary shift (as in the SHIFT table) */ int of fset ; /* Offset (as in the SHIFT table) */ int depth ; /* Depth (as in the SHIFT table) */ struct Pattern *patlist_start ; /* Pointer to start of pattern list (HASH table entry)*/ struct Pattern *patlist_end; /* Pointer to end of pattern list (next HASH table
• the root node is special and is a constant-size array, each element being structured as follows: struct bplus_tree_node ⁇ struct hash_record *list ; /* Pointer to child node */ int num_elements ; /* Number of elements in list */ ⁇ ; struct bplus_tr ⁇ _nod ⁇ root [A] ;
• FIG. 19 An exemplary memory structure of a root node is shown in Figure 19.
• the root has 'A' elements of type struct bplus_tree_node, where A is the alphabet size (256 in the case of the ASCII set). [00130]
• the index value is a hash of a chunk of B characters C 1 C 2 -X B
• each element of the root can be accessed randomly by computing the hash value of the most significant byte of that chunk C 1 .
• the remainder of that chunk (c 2 ..C B ) is mapped to the key_value of an elements of a child node.
• Each child node need not contain entries for all possible hash values for strings of size B- 1; it only contains entries for those elements that don't have associated default primary shift value (which will be equal to Minimum_pattern_size-B+1).
• patlist_end and pend parameters do not refer to the last pattern in the list of patterns that are obtained from a particular HASH table entry; but rather refer to the pattern pointed to by the next HASH entry, or the pattern following this list of patterns in the global sorted pattern list.
• Figure 20 shows one embodiment of a child node memory data structure. Although the root nodes and each of the child nodes are designed to be independently relocatable in memory, the embodiment of Figure 20 is implemented such that the child nodes immediately follow the root structure. Also, child nodes are written into memory in contiguous fashion, ordered by the index value of the parent element. Storing this data contiguously makes memory pre-allocation easy, and only one pointer (the address of the first word of the root) need be specified explicitly. [00133] The Wu-Manber algorithm assumes that all patterns are of equal length; its correct operation relies on this assumption. Since that is not a practical assumption, we need to truncate all strings to the size of the shortest pattern. During an algorithm run, the complete pattern is matched when a candidate position is discovered. The patterns that are passed to all pre-processing functions are truncated.
• the largest prefix of any pattern that is matched at the end (suffix) of a previous packet is provided as input to the algorithm when it is executed for the next packet.
• This information is provided in two parts: prefix_address, the address of the first byte of the pattern string whose prefix was matched, and prefixjtength, the size of the prefix (or the number of bytes of the pattern that were matched successfully).
• prefix_address the address of the first byte of the pattern string whose prefix was matched
• prefixjtength the size of the prefix (or the number of bytes of the pattern that were matched successfully).
• the prefix of a pattern matched in the previous pattern is the empty string, so the prefixjength parameter value will be 0, and the prefix_address uninitialized.
• the offset and depth values are set in the same way as the shift values are, by selecting the safest possible value. If a chunk of bytes S is used to index into the SHIFT table, the corresponding offset is set to be the minimum offset of any pattern that ends with this chunk (the pattern is truncated first to the shortest pattern length). Similarly the depth is set to be the maximum value of (depth- length+minimum_pattern_size), as this indicates where the truncated pattern must end within the text. [00140] During a packet scan, when a primary shift value of 0 is observed, we first check the offset and depth (looked up from the table) and match them to the current text position.
• Normal index using the characters of the chunk as they are; and Lower case index, computed by converting all upper case characters in the
• the second index is necessary as the HASH entry for a case independent pattern is computed from lower case characters only.
• the entire cycle of checking prefixes, followed by matching complete patterns is done twice, one for the normal index value and another for the lower case index value.
• both normal and lower case versions of the pattern prefixes are computed. If the pattern under consideration is case independent, the lower case prefix is matched, otherwise the normal one is matched.
• the algorithm gets down to matching patterns byte- by-byte with the packet content, case-independent matching becomes straightforward.
• the content inspection framework is designed to be a module that could be plugged into a previously-implemented stateful firewall framework.
• the content inspection framework can be plugged in as a single microblock into any packet-processing pipeline.
• the firewall uses CA (Condition- Action) pairs for rule processing.
• CA Consumer- Action
• certain CA rules could pertain to content inspection, like checking for a particular offending pattern with packet content (condition) and dropping the packet if the pattern is detected (action).
• Algorithm code This indicates the type of search algorithm to use.
• SRAM address This indicates the address (or pointer to index of addresses) of data structures used for pattern searching. For the first 4 algorithms, this address points to the first word of the master index; this contains information about the addresses of the various data structures that are used for inspection.
• this information is stored in the following manner: the first word is the expected sequence number of the first byte in the next packet, the second word is the AC machine state from the end of the previous scan (for algorithm codes 1 and 2) or the prefix length (algorithm codes 3 and 4) and the third word is the partially matched pattern address (used only for algorithm codes 3 and 4).
• Initialization and Code Load [00148] Returning to Figure 7, during initialization of a network line card or network device, coded instructions (e.g., microcode) to facilitate various packet- processing and content inspection functions and operations described herein are loaded into the control stores 728 of the selected ME's.
• the instructions may be loaded from an instruction store 715 comprising a non- volatile storage device, such as a flash memory device.
• a non- volatile storage device such as a flash memory device.
• Other examples of non-volatile stores include read-only memories (ROMs), programmable ROMs (PROMs), and electronically erasable PROMs (EEPROMs).
• ROMs read-only memories
• PROMs programmable ROMs
• EEPROMs electronically erasable PROMs
• instructions may be loaded from an external source.
• the instructions are stored on a disk drive hosted by another line card (not shown) or otherwise provided by the network element in which line card 700 is installed.
• the instructions are downloaded from a remote server or the like via a network as a carrier wave.
• each of the functional blocks illustrated in the figures and discussed herein may be implemented using programmable logic in hardware, programmable logic in firmware/software via execution of corresponding threads on one or more compute engines, or a combination of the two.
• the operations depicted for each of deep packet inspection module 202, flow manager 204, queue manager 708, and receive engine 710 are facilitated by executing corresponding instruction threads on one or more compute engines in an NPU in a manner integrated into a context pipeline used for packet processing.
• all or a portion of the various operations provided by the foregoing functional blocks may be implemented by appropriately-programmed logic embedded on an NPU or the like.
• a machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer).
• a machine-readable medium can include such as a read only memory (ROM); a random access memory (RAM); a magnetic disk storage media; an optical storage media; and a flash memory device, etc.
• a machine-readable medium can include propagated signals such as electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.).
• Input Array of patterns, Chunk size used for hashing into table (B), Alphabet size (A)
• SHIFT_table [ Sval ] [1] least value of (min_size-j-B) that is strictly greater than SHIFT_table [Sval] [0] ; /* Compute offset as least offset among all patterns containing this substring */
• Input Array of patterns, Chunk size used for hashing into table (B), Alphabet size (A)
• Input Array of patterns, Size of prefix (B'), Alphabet size (A)
• Output Pointer to PREFIX table long int *build_PREFIX_table( struct Pattern *P, int num_pat, int B', int A)
• PREFIX_table [ i ] HASH_value (S); ⁇ return PREFIX_table;
• Input Array of patterns sorted by hash value (hash computed in the same way as for the SHIhT and HASH tables), Size of prefix (B'), Alphabet size (A), SHIFT table, HASH table
• the field key_v ⁇ lue is actually the index of the h ⁇ shjrecord structure list, if all records (inclusive of the ones with default shift values) were added). Therefore, the above algorithm ensures that list (for each tree child node) is sorted in ascending order of key_v ⁇ lue.
• Wu-Manber algorithm Input Packet content (string of bytes) T[O...n- 1], SHIFT table, HASH table, PREFIX table, Length of shortest pattern, Number of patterns, Chunk size (B), Prefix size (B'), Flag (policy) indicating whether or not we need to check for patterns spanning packet boundaries. Length of a pattern prefix matched at the end of the previous packet, Address of pattern string whose prefix was matched at the end of the previous packet.
• prefix_val HASH_value (prefix)
• prefix_lower_case_val HASH_value (convert_to_lower_case (prefix) ) ;
• Input Packet content (string of bytes) T[O...n-1], B+ tree root pointer, PREFIX table, Length of shortest pattern, Number of patterns, Chunk size (B), Prefix size (B'), Flag (policy) indicating whether or not we need to check for patterns spanning packet boundaries. Length of a pattern prefix matched at the end of the previous packet, Address of pattern string whose prefix was matched at the end of the previous packet
• Output Identity of pattern detected (or - 1 if none detected), Position of start of pattern as offset from the start of the string T.
• the algorithm is identical to the earlier described Wu-Manber algorithm, except for the data structures used for shift values and pointer lookup.
• the primary and secondary shifts, as well as the offset and depth values are now looked up from a B+ tree child entry (from a hashjrecord structure) instead of a SHlFl table.
• Pointers to the candidate patterns that might occur at a given position are also obtained from the same B+ tree child node entry instead of a HASH table. But unlike a SHIFT or a HASH table, these values cannot be looked up directly (in constant time). A sequential scan would be too inefficient. Therefore, we use binary search to locate the appropriate record; the lookup steps are described below:
• C 1 Use most significant byte (C 1 ) to locate the appropriate child node pointer and number of child nodes (root [C 1 ] . list and root [C 1 ] .num_elements) .
• C 2 ..c B Use the rest of the chunk c 2 ..c B , mapped to an integer, to locate the appropriate hash_record structure. This integer will be the key_value of any matching structure. Since all structures are sorted in ascending order based on key_value, a simple binary search will return the required data) . For an alphabet of size 256, a maximum of 8 lookups will be required to locate a record.

Applications Claiming Priority (2)

Publications (2)

Family

ID=37619733

Family Applications (1)

Country Status (7)

Cited By (1)

Families Citing this family (143)

Family Cites Families (16)

• 2005
  • 2005-06-30 US US11/174,373 patent/US7784094B2/en active Active
• 2006
  • 2006-06-30 EP EP06850502A patent/EP1905213B1/de not_active Not-in-force
  • 2006-06-30 AT AT06850502T patent/ATE504148T1/de not_active IP Right Cessation
  • 2006-06-30 JP JP2008519707A patent/JP4598127B2/ja not_active Expired - Fee Related
  • 2006-06-30 DE DE602006021028T patent/DE602006021028D1/de active Active
  • 2006-06-30 CN CN200680024231XA patent/CN101258721B/zh not_active Expired - Fee Related
  • 2006-06-30 WO PCT/US2006/026069 patent/WO2007120165A2/en active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events