WO2005111805A1 - Procédé de détection de la signature du trafic d'un réseau - Google Patents

Procédé de détection de la signature du trafic d'un réseau Download PDF

Info

Publication number
WO2005111805A1
WO2005111805A1 PCT/NZ2004/000093 NZ2004000093W WO2005111805A1 WO 2005111805 A1 WO2005111805 A1 WO 2005111805A1 NZ 2004000093 W NZ2004000093 W NZ 2004000093W WO 2005111805 A1 WO2005111805 A1 WO 2005111805A1
Authority
WO
WIPO (PCT)
Prior art keywords
network traffic
parts
deviation
pattern
identified
Prior art date
Application number
PCT/NZ2004/000093
Other languages
English (en)
Inventor
Juergen Brendel
Dennis Vshivkov
Original Assignee
Esphion Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Esphion Limited filed Critical Esphion Limited
Priority to PCT/NZ2004/000093 priority Critical patent/WO2005111805A1/fr
Publication of WO2005111805A1 publication Critical patent/WO2005111805A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416—Event detection, e.g. attack signature detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2463/00—Additional details relating to network architectures or network communication protocols for network security covered by H04L63/00
    • H04L2463/141—Denial of service attacks against endpoints in a network

Definitions

  • the present invention relates to methods and apparatus for the detection of signatures present in computer network traffic.
  • Firewalls, filters and the like in combination with passwords have been traditionally used to protect against unauthorised access to confidential or private information.
  • DoS denial of service
  • a DoS attack may be directed at mission critical web sites, network installations, network devices, and servers for various reasons.
  • a first kind of DoS attack is aimed at particular weaknesses in a server's or router's operating system.
  • a specific packet or command can crash or disable the device.
  • the manufacturer of the device will produce a patch immediately after the problem becomes known.
  • defences against these attacks are usually readily available.
  • IDS Intrusion Detection Systems
  • IDS Intrusion Detection Systems
  • a flood-style DoS attack often in the form of a Distributed Denial of Service (DDoS) attack, is an attack against the resources, for example network bandwidth, attempting to deplete this resource, rather than an attempt to gain access into a particular system. Most commonly, such an attack consists of flooding the victim with massive amount of network traffic, often simply junk packets with fake source addresses. Flood- style attacks are easily executed and are therefore popular amongst even unskilled attackers. Defenses are not readily available, since an attack victim usually does not have control over the amount of traffic an attacker can produce.
  • DDoS Distributed Denial of Service
  • a flood-style DoS attack may be performed by using remote hosts to generate unusually high volumes of network traffic and direct the data packets to a corporate site.
  • the remote hosts generate such a high amount of information that the bandwidth of the communication channels and processing capabilities within the network hosting the corporate site become overloaded with invalid information. The effect of this is that no legitimate traffic can pass through the network. This leaves the network essentially inoperable, which may cause lost productivity and sales. Large floods have the potential to seriously disrupt the operations of entire portions of the network.
  • New Zealand which connects to the rest of the world via relatively small network connections. A flood against any target within New Zealand may take the entire country off-line.
  • Examples of high-profile DDoS attacks took place early 2000, taking down some of the world's largest web-sites, such as CNN and Yahoo. Today, DDoS attacks are some of the most common attacks and take place around 4000 times per week, around the world. Recent, high-profile examples include the DDoS attacks against the web-sites of SCO and Microsoft.
  • firewalls are typically unable to detect and deflect flood attacks. This is due to the data packets being transmitted to the network not having the traditional characteristics of other forms of attack such as viruses, Trojan horses and unauthorised access.
  • a denial of service attack may also be generated from within the network, which cannot typically be detected using a firewall or a device monitoring solely incoming and outgoing communications.
  • Network resource exhaustion which may be caused by non-malicious activities, for example an accidental network mis-configuration, or a sudden flash crowd to a site, may also result in similar effects as a flood style attack. Thus, handling these conditions is similarly of interest to the network operator.
  • a worm may result in a flood-like increase of traffic.
  • the network is merely a medium for a worm or virus to spread. But eventually, even for the network operator this has become an important issue, especially considering that the rapid spread of recent worms has consumed massive amounts of network bandwidth, and therefore also causes flood-attack style symptoms.
  • the Blaster/Sapphire worm It spread by sending out individual UDP packets. These packets would at first appear as a UDP DDoS flood, but instead contained the spreading attempt of the worm. It produced massive amounts of traffic, effectively choking many networks.
  • Another example is the Welchia Worm, which spread in August 2003. It sent a specific scanning packet out, in order to discover further vulnerable clients to infect. This scanning traffic introduced significant disruptions at various ISPs, causing undue bandwidth utilization and cost, as well as a disruption to various maintenance and billing systems of the network provider.
  • An ACK-storm happens when two TCP connection end-points (such as two mail-servers, or a web-client and server) become out-of-synch.
  • TCP connection end-points such as two mail-servers, or a web-client and server.
  • One of the most common reasons for this is the failure a transparent proxy in a network. If the proxy fails, a mismatch between the end-points results, causing the end-points begin to send so-called ACK-packets in order to correct the perceived mistake of the other side. They never will be able to resolve the situation successfully, though, and therefore continue to send packets to each other at a rapid rate. Therefore, the hoped-for savings in bandwidth charges by a proxy instantly evaporate in a flood of unnecessary traffic.
  • IPSs intrusion prevention systems
  • Filtering devices such as routers and IPSs, normally need to be configured with a set of signatures for which they can search in a packet.
  • a certain attack may be filtered by looking for specific bit-patterns in the packets.
  • these signatures need to be developed by human analysts in a pain-staking . manual process. Sitting in their centralized analysis centres, which are operated by companies such as Tipping Point, Symantec and others, those analysts examine traffic samples collected by various means from around the world.
  • the network operator needs to log into a router, manually collect the packet samples, and transfer the sample-files for analysis to a second machine. This then needs to be repeated for as many routers as possible. Finally, the number patterns of the packets usually get printed out in large stacks of paper, and the analyst then manually scans through page after page, hoping he/she will find some similarity in those patterns.
  • the Applicant has proposed methods and apparatus for network traffic management that involves detecting a flood attack in its international patent publication number WO 03/055148.
  • the method involves passively observing the network traffic for particular characteristics indicative of a flood attack and can often identify the type of attack that is occurring. Therefore, even if the routers should fail, any analysis or findings can still be transferred to the operators.
  • volume when used with reference to volume of information communicated has been used with reference to the depleting effect network communications have on network communication resources.
  • volume is intended to include, for example, a measure of the number of packets communicated, regardless of their size. This is in addition to other measurements that may be bandwidth related, such as the number of bytes communicated.
  • a computer- implemented method for identifying anomalous network traffic comprising: a) observing network traffic and determining a value related to the deviation from at least one of the normal and expected traffic characteristics for at least one network traffic characteristic; b) when step a) indicates the presence of a significant deviation, analysing a structured portion of a sample of the network traffic to identify at least one of instances of commonality present in the contents of the network traffic and instances of a pattern present in the contents of the network traffic, which occur with a frequency that would approximately provide the deviation from the normal or expected traffic characteristics determined in step a) if the network traffic having the commonality or pattern was the cause of the deviation; and c) providing an output indicative of at least one commonality or pattern identified in step b).
  • step a) comprises classifying observed network traffic into classes and watching for a significant deviation in at least one class of traffic. More preferably, in one embodiment of the invention, step a) comprises calculating a ratio of a measure of volume of data in at least one class against at least one other class, whereby a deviation is deemed significant based on the variation of the calculated ratio from a normal or expected ratio value and wherein the step of determining a value comprises calculating a comparative measure of the volume of data belonging to the class that caused the variation of the calculated ratio per unit volume of network traffic to the expected volume of data belonging to that class.
  • the network traffic is in the form of packets and the structured part comprises packet headers.
  • Step b) may comprise identifying one of a commonalities and patterns in the contents of the network traffic at like locations relative to the start of the packet headers.
  • the output may then comprise information indicative of the header in which one of a commonality and pattern has been identified, the location within the header in which said one of a commonality and pattern is located and a value representative of the commonality or pattern.
  • the step of identifying at least one commonality present in the contents of the network traffic which occurs with a frequency that would approximately provide the deviation from the normal or expected traffic characteristics determined in step a) comprises the sub-steps: i) parsing like structured portions of the sample into parts; ii) identifying at least one commonality in the contents of said parts across a plurality of structured portions and counting the number of parts having any identified commonality in the sample; and iii) comparing the counted number of parts with the value related to the deviation from the normal or expected traffic characteristics and identifying any commonality between parts that involve parts that would approximately provide said deviation if they were the cause of the deviation.
  • the step may comprise the further sub-step: iv) determining combinations of the parts identified in sub-step iii) across different structured portions of the sample; and wherein step c) comprises outputting the identified combinations.
  • the method may further comprise the step of selecting an identified combination from a plurality of identified combinations by evaluating the identified combinations against at least one of: A) a count of the number of parts that make up the combination; B) a difference between the frequency of occurrence in the sample of the parts that make up the combination; and C) a difference between the frequency of occurrence of the parts that make up the combination and the value related to the deviation from at least one of the normal and expected traffic characteristics in at least one network traffic characteristic.
  • the step of identifying at least one pattern present in the contents of the network traffic which occurs with a frequency that would approximately provide the deviation from the normal or expected traffic characteristics determined in step a) comprises the sub-steps: i) parsing structured portions of the sample into parts; ii) identifying at least one pattern in the contents of corresponding parts of the structured portions and counting the number of parts making up any identified pattern in the sample; and iii) comparing the counted number of parts with the value related to the deviation from the normal or expected traffic characteristics and identifying any pattern that involve parts that would approximately provide said deviation if they were the cause of the deviation.
  • the method may further comprise the step of selecting at least one pattern for output dependent on at least one of: A) the proportion of the structured data that includes the pattern; B) a difference between the frequency of occurrence in the sample of the parts that make up the pattern; and C) a difference between the sum of the frequency of occurrence of the parts that make up the pattern and the value related to the deviation from at least one of the normal and expected traffic characteristics in at least one network traffic characteristic.
  • step b) if more than one commonality or pattern is identified in step b), then a single commonality or pattern is selected for output in step c) according predetermined criteria and wherein the method comprises the further steps of: d) repeating step a) after a filter according to the selected commonality or pattern has been implemented; and e) selecting another commonality or pattern from those identified in step b) for output in step c) if the deviation is observed to be still present in the network traffic.
  • the method may further comprise repeating steps d) and e) until the deviation is no longer observed in the network traffic or all commonalities or patterns identified in step b) are exhausted.
  • steps b) and c) involving identifying and outputting only instances of commonality present in the contents of the network traffic.
  • step c) comprises formatting outputted information into a human readable form.
  • step c) comprises formatting outputted information into filter instructions able to be implemented by a device capable of filtering network traffic.
  • the sample of network traffic analysed in step b) is limited to a portion of the network traffic that has the characteristic with the significant deviation.
  • a computer-implemented method for signature detection in network traffic comprising receiving information defining a proportion of network traffic that the data having a signature is to occupy, parsing structured data in a sample of the network traffic into parts and identifying instances of commonality present in the contents of the network traffic across parts sharing a common location in the structured data, which occur with a frequency that would approximately provide the predefined proportion of network traffic, the method further comprising outputting information describing identified instances of commonality.
  • the method further comprises determining combinations of the identified parts at different locations in the structured data, wherein the step of outputting information comprises outputting at least the determined combinations.
  • the step of outputting information comprises only outputting determined combinations having a predefined number of member parts or more.
  • the method further comprises determining combinations of the identified parts at different locations in the structured data and then selecting a combination dependent on at least one of: A) a count of the number of parts that make up the combination; B) a difference in the frequency of occurrence in the sample between each of the parts that make up the combination; and C) a difference between the frequency of occurrence of the parts that make up the combination and the information defining a proportion of network traffic that the data having a signature is to occupy; wherein the step of outputting information comprises outputting only the selected combination.
  • the structured data comprises packet headers.
  • the common location in the structured data may be determined relative to the packet headers so as to accommodate for variable header lengths.
  • the step of parsing structured data comprises parsing the structured data into mutually exclusive bytes.
  • a computer- implemented method for signature detection in network traffic comprising receiving information defining a proportion of network traffic that the data having a signature is to occupy, parsing structured data in a sample of the network traffic into parts and determining at least one pattern present in parts that occur with approximately the same frequency and share a common location in the structured data, and from the determined patterns identifying those that are associated with data that forms a proportion of the entire sample approximate to the predefined proportion of network traffic, the method further comprising outputting information describing any identified patterns.
  • a computer program or a computer readable medium for identifying a signature in network traffic the computer program or a computer readable medium containing instructions to cause a computer to perform the steps of: a) receive information related to the characteristics of a sample of network traffic communicated over a link; b) parse at least a part of the sample that is structured into parts and identify the contents of each part; c) determine the frequency with which the same contents are present at the same location in the structured data, compare this with a predefined frequency and identify the contents that occur with approximately the predefined frequency; d) output information defining the identified contents and their location.
  • the computer program or computer readable medium further contains instructions to cause a computer to determine combinations of contents at different locations in the structured data, evaluate the combinations against the predefined frequency according to predefined criteria and output a best matching combination against the predefined criteria.
  • the predefined criteria comprises the number of parts in the combination.
  • the predefined criteria may also comprise at least one of the difference between the frequency of occurrence of each member part of a combination and the predefined frequency and the variation in the frequency of occurrence of the member parts in the combination.
  • Figure 1 shows diagramatically a communication network incorporating the present invention.
  • Figure 2 shows a block diagram of an apparatus for implementing the present invention.
  • Figure 3 shows a flow chart of the processes performed in accordance with the present invention.
  • Figure 4 shows a flow chart expanding on step 13 in Figure 3.
  • Figure 5 shows a portion of an example histogram resulting from the analysis of network traffic in accordance with the present invention.
  • the present invention relates to methods and apparatus for the detection of signatures present in computer network traffic.
  • the invention may thus have application to network communication security and management systems.
  • FIG. 1 shows a block diagram broadly showing a simplified communication network 1 including an apparatus 100 for implementing the present invention.
  • the communication network in this example, includes an enterprise network 3, which is connected to the internet 2 through a link 4, an intrusion protection system (IPS) 5 and a switch 6.
  • the enterprise network 3 includes a www server 7. The normal route of communications between the enterprise network 3 and the internet 2 is to traverse the link 4, IPS 5 and switch 6.
  • IPS intrusion protection system
  • the apparatus 100 has two main functions, represented in Figure 1 by an observation module 00A that observes network traffic characteristics and an analysis module 100B.
  • the observation module passively observes the network traffic, for example via a fiber-tap or spanning port (not shown), and receives observation data through communication channel 100C.
  • the observation module 100A may observe the traffic via an active device such as a router, switch or any other suitable device. Accordingly, the observation module may receive data from the IPS 5 shown in Figure 1.
  • the analysis module 100B optionally includes a communication channel 100D to the IPS 5 or another filtering device that allows the apparatus 100 to provide filtering or redirection instructions to the IPS 5.
  • the analysis module may optionally further include a communication channel 100E to a network administrator and an optional communication channel 100F to receive data present on the link 4 for filtering (see herein below), in this example via the IPS 5. If the communication channel 100C can provide the functionality of communication channel 100F, communication channel 100F may be omitted.
  • observation module 100A and analysis module 100B are shown to be in the same device in Figure 1, this is not essential. Furthermore, some of the functions of the analysis module 100B described herein below may be performed by the observation module 100A and vice-versa.
  • Figure 2 shows a block diagram of the main physical components of an apparatus 100.
  • a processor 101 is provided to perform the functions of the observation module 100A and analysis module 100B.
  • the processor 101 may be a microprocessor, digital signal processor, microcontroller or other device or a combination of devices suitable for performing the processing functions of the present invention.
  • a user interface 102 or other communication interface may be provided to allow reconfiguration of the apparatus 100 as required.
  • a memory 103 readable by the processor 101 contains the instructions governing the operation of the processor 101 and data relating to existing activities of the apparatus 100, as well as any historical data that may be used to identify traffic anomalies.
  • the memory 103 may provide both permanent and temporary storage functions as required. Some part of the memory 103 is used to collect network traffic, typically packets, for analysis purposes. A couple of megabytes of memory typically allows a few thousand packets to be analysed, although such a large buffer will not be necessary in most situations.
  • the memory 103 may also include a separate database for historical data relating to network communications. Those skilled in the relevant arts will appreciate that the memory 103 collectively represents multiple forms of computer memory that may be logically and physically distinct.
  • a data interface 104 which may be one or more a network cards, is provided to allow the observation and sampling of data that is communicated to or within a network.
  • the data interface 104 or another interface, may also send information to a filtering device, which may be the IPS 5. Accordingly, the data interface 104 manages the communication of the apparatus through communication channels 100C, 100D and 100F (see Figure 1).
  • the apparatus 100 after identifying a signature of anomalous traffic may directly instruct a router or other device capable of filtering data to filter out traffic having that signature, and/or may output, through communication channel 100E, the results of its analysis to an operator via a text message, email, SNMP trap or otherwise.
  • the latter option may be preferred to provide some administrative auditing of the signature analysis performed by the apparatus 100.
  • the identified signature and the proposed filter instructions for a filtering device may be displayed to an operator, with the filtering instructions not applied until the operator has approved them, for example using a suitable GUI.
  • the apparatus 100 may include instructions to convert the detected bit and/or byte patterns into a different format if required, for example into a human readable form and output it through communication channel 100E using a suitable communication interface 105 for that channel (which may be the same as user interface 102), or into filtering instructions for a specific device.
  • the apparatus 100 is preferably not deployed in line, allowing it to passively observe network traffic and providing some immunity to any attack that may be occurring on the network.
  • an in line implementation is also possible.
  • the apparatus is not deployed in line, but the data interface 104, through communication channel 100F, has the capacity to receive all data communicated over the link 4, in which case the apparatus 100 may receive the data communicated over link 4 from the IPS 5 (or other device) under the instruction of the apparatus 100.
  • the apparatus 100 could then include an appropriate network device to filter or redirect traffic * based on the signature that it has determined is indicative of unwanted traffic and then return the traffic to the link 4.
  • the apparatus 100 therefore monitors characteristics of the network traffic for activity that is representative of an attack (malicious, inadvertant or otherwise) on the network communication resources.
  • an attack malicious, inadvertant or otherwise
  • There are various known methods for identifying that an attack may be occurring including performing a ratio analysis of classes of traffic as described in the Applicant's international publication number WO 03/055148, the contents of which is incorporated herein in its entirety. Most methods involve classifying packets according to their type, source, destination, and/or length and comparing the resource depleting effects of classes of traffic in comparison to normal conditions, such normal conditions being set in advance and/or being dependent on collected historical data. These methods can thus determine the type of attack that is occurring and provide a measure of the deviation from normal communication conditions caused by the attack. For example, the apparatus may detect a disproportionately high number of SYN packets, say 50% more than normal, which may be indicative of a SYN attack. If more than one anomaly is detected, the apparatus 100 can work on them simultaneously and
  • a ratio outside of certain bounds indicates an attack.
  • the detected disproportionately high number of SYN packets may be detected as an abnormal ratio of SYN:FIN packets.
  • the expected ratio and/or the historical ratio of SYN:FIN and the actual FIN value are used to calculate an expected number of SYN packets. The difference between the actual number of SYN packets and expected number of SYN packets then indicates the number of anomalous SYN packets that are expected.
  • FIG. 3 shows a flow diagram of the steps performed in accordance with the present invention.
  • Steps 10 and 11 involve observing the network traffic and classifying the traffic into classes in order to allow detection of an anomaly in the observed traffic in step 12.
  • an output 01 in the form of an alert may optionally be generated.
  • the alert may be sent to a network administrator, for example through the communication interface 105 and communication channel 100E.
  • a second output 02 is generated, identifying some measure of the deviation of the volume and/or resource depleting effects of the data in that class from those that are present in non- anomalous conditions.
  • the output 02 may also identify the class of traffic in which the anomaly was detected. .
  • the class information in output 02 assists the signature detection step 13 by defining a subset of the network traffic that needs to be analysed for a signature, allowing an initial filtering of traffic before signature analysis is performed.
  • this is not essential and the sample of network traffic analysed for a signature may simply be a sample of all traffic passing through a point in the network.
  • Step 13 involves detecting signatures in the anomalous traffic.
  • Figure 4 shows the three sub-steps 131-133 involved in step 13.
  • Step 131 involves obtaining a sample of the network traffic, using the class information in the output 02 if available and if the applicable network devices are capable of implementing at least a simple filter.
  • the sample may be taken by the IPS 5 with the assistance of some low-level packet processing code and sent to the apparatus 100 via communication channel 10OF. If the apparatus 100 is not to filter the sampled traffic itself, the necessary information may be obtained through communication channel 100C and communication channel 100F omitted from apparatus 100. Alternatively, as described herein above, communication channel 100F may also be omitted if data for filtering and forwarding can be received through communication channel 100C.
  • Sampling of traffic will typically be a separate step from the observation step 10 and only performed when an anomaly has been detected.
  • the sampling of traffic may be performed by retaining the packets in memory after the observation step 10 has been completed only if an anomaly is detected.
  • the sampled data may also be forwarded to another application, stored in a database or otherwise used as required.
  • the sample may, for example, comprise either 1000 SYN packets or a five second sample, whichever ends first.
  • the apparatus 100 after identifying the class of traffic in which the anomalous traffic is present, may instruct the IPS 5 to sample the smallest possible subset of traffic that it can that still contains the class of traffic in which the anomalous traffic is present. This process is described in more detail in the Applicant's international publication number WO 03/055148.
  • a statistical analysis on the sample is then performed to identify a signature of the attack traffic.
  • the statistical analysis is performed on a structured part of the network traffic, for example the packet headers.
  • other parts of the network traffic may be structured, allowing a signature analysis on those parts. Therefore, an analysis of the body of the packet may be fruitful.
  • each packet header is parsed into parts comprising a window of n bits and the instances of each value in each part are recorded, forming a histogram of packet part values.
  • An example partial histogram is shown in Figure 5.
  • n 8 so that the instances of each value of each byte are recorded.
  • Each window of n bits may be separate or may overlap.
  • step 132 involves searching for the byte patterns that occur about 33% of the time. Referring to the histogram shown Figure 5, seventeen possible permutations of byte combinations with this level of occurrence are (positionxontents):
  • the analysis module 100B could output all seventeen possibilities to a network administrator, allowing them to select one or more signatures to base a filter on. Alternatively, the analysis module 100B could automatically select a best or group of best options. Typically, the longest match is to be preferred, which narrows the selection to combinations 14-17. Of these, the three elements in combination 17 all appear with almost the same frequency and very close to the expected frequency of the anomaly. Therefore, combination 17 is selected as the primary identified signature and an output 03 is generated identifying the byte pattern.
  • the analysis module 00B could simply output the single part combinations, which are options 1 - 5, placing the task of identification of the other 12 multi-part combinations on the network administrator. However, this is likely to increase the instances of human error missing combinations, particularly if the number of single item permutations are large. Therefore, it is preferred that the identification of multi-part combinations is performed automatically, for example by the analysis module 100B. Also, in some embodiments, any combinations with less than a predefined number of member parts may be automatically excluded from consideration.
  • the analysis module 100B could include a threshold below which a filter is not recommended or implemented.
  • the analysis module 100B may select none of the permutations for notification or automatic filtering. There may, of course, be separate thresholds for notification and automatic filter implementation or a threshold for automatic implementation only. If every member of the set of longest matching signatures falls below a threshold, the analysis module 100B may look to the next longest matching set to see if there is a closer match. In addition, or alternatively, if a signature that is a member of the set of second longest matching signatures may be preferred if it is more than, for example 15% closer to the expected volume of anomalous traffic than the closest signature in the set of longest matching signatures.
  • tests could be used to select a "best" combination or group of combinations from a list of possible permutations, depending on the requirements and preferences of the network administrator. For example, there could be a series of tests that each eliminates a proportion of the candidates and that performed in order of importance. Alternatively, a weighted score evaluation method may be used.
  • searching for a pattern involves analysing the entries in the contents column in the same position for a pattern. There is no readily identifiable pattern in Figure 5, but a possible pattern may be a series of steadily increasing values. If each of the entries, or at least the majority of the entries (to accommodate for some that match significant volumes of traffic communicated for another reason), in the pattern occurs with approximately the same frequency, they can be identified as a pattern. The frequencies can then be added and the sum compared with the expected volume of anomalous traffic.
  • the analysis module 100B checks whether any pattern continues over to the next position, in which case three permutations are identified, the pattern in the first position, the pattern in the second position and the pattern across the first and second positions. Longer patterns are preferred for filtering purposes.
  • Step 14 involves the optional additional step of translating the output 03 into output 04, which are specific filter instructions.
  • the filter instructions may be human or machine readable, the advantage of the latter being that a device capable of filtering may receive the filter instructions and automatically commence filtering of packets having the identified signature.
  • Step 14 may be achieved purely by mechanical mapping.
  • the method may include a check on the effectiveness of the filter. Thus, if after a filter has been implemented that removes network traffic having the identified signature, the observation module 100A still observes the same anomaly, the analysis module 100B may select the next best signature for notification and/or automatic implementation.
  • other aspects of network traffic may be detected, for example looking for patterns/trends such as increasing or decreasing values representative of port scans, and the like. Those skilled in the relevant arts will appreciate that the particular patterns searched for will often depend on the types of attack that may occur in the current operating environment for the network.
  • n could be variable.
  • the present invention may analyse header field values instead.
  • the header field values for packets may be as small as 3 bits or 4 bytes or longer, with smaller and larger header filed values being possible depending on the communication protocol used.
  • the protocol used could be fixed and specified in advance, or alternatively, the method may include an additional step of determining the communication protocol after the network traffic has been observed and then retrieving from memory the values for n and the positions of each packet part that are most appropriate for that protocol.
  • each member of the signature comprises a triplet of members, namely the offset and value as described herein above and a third value, referred to herein as the "level".
  • the level identifies the depth of the byte (or other bit group) within the protocol stack.
  • an offset that points into an IP header would have a level of 0 or 1
  • an offset that points into a TCP or UDP header would be at level 2
  • an offset pointing into a header that is encapsulated within UDP would be at level 3.
  • the offsets are then represented relative to the start of that particular level header, which overcomes the problem of headers being located. at different positions in a packet when options (which may be present in some packets and not others) are used.
  • an example signature may be (level: offset: value): ⁇ 0:4:0 - 0:5:0 - 0:8:12 - 0:20:255 - 1:0:0 - 1 :1:80 ⁇ .
  • the first four members of the signature are bytes at the various offsets in the IP header and the fifth member is in the next header, which may be either be TCP or UDP.
  • the third value in each triplet provides the value that is present across multiple packets (or if a pattern is identified a number allocated to that pattern) and the second value in the triplet specifies the offset where that value is located relative to the start of the header.
  • headers that are used for analysis may vary depending on the communication protocol used and dependent and/or on configuration options.
  • an Ethernet header may optionally also be allocated a level.
  • Example - SYN-flood A more detailed example of a SYN attack is now provided.
  • an attacker on terminal 8 starts a SYN-flood DDoS attack against the www server 7. Therefore, a large quantity of TCP-SYN packets flows through the link 4, IPS 5 and SWITCH 6 to the www server 7.
  • SYN-flood generator programs There are many different SYN-flood generator programs in existence. It is therefore not possible to make a definitive statement about what characteristics the generated packets will have, besides the fact that they are TCP-SYN packets, with the specified source and destination addresses and ports.
  • this example assumes that the flood has the following characteristics: -
  • the particular SYN-generator that is used in the attack sets a fixed value of hexadecimal 0x01020304 for the SEQ-number in the TCP header field. This is in contrast to normal network communications, in which the SEQ-number is more or less randomly generated at the start of a TCP session, and is stored in a four-byte field in the TCP header.
  • the SYN-generator uses a fixed IP-ID of zero.
  • the IP-ID is normally a sequentially increasing two-byte identifier that is contained in each IP header sent out by a host's network stack.
  • the source IP-address is completely randomized, as is the source port in the TCP header.
  • the destination IP-address is that of the www server 7, and the destination port is 80.
  • the TCP flags are according to that of a normal TCP-SYN packet.
  • the ACK-field in the TCP header is set to zero (exactly as it should be in a connection establishing SYN-packet).
  • the observation module 100A notices that the number of SYN packets has increased. Normally, let's say, the link 4 carries on average 1000 SYN packets per second and that as a result of the DDoS flood, 1500 SYN packets per second are being carried by the link 4. In other words: 33% of all SYN packets appear to be anomalous. This deviation from normal conditions is deemed significant enough to warrant further investigation. What represents anomalous communications could be determined by comparison with historical data, accumulated over time. The threshold for a significant deviation may be set in advance and may be specific to the particular characteristic of the data observed (in this case the number of SYN packets). A sample is taken of the SYN packets on the link 4 for a certain while.
  • This packet sample along with the number of packets (900) and the percentage of packets we expect to be bad (33%) is passed to the analysis module 100B, using any suitable protocol, for example CORBA.
  • the analysis module 100B examines the packet sample, forming a histogram of packet part contents as described herein above and then identifies the longest signature that has all elements occurring with a frequency at about the same level and at about 33%. It identifies the following pattern as the most likely anomaly-signature: ⁇ 4:0 - 5:0 - 24:1 - 25:2 - 26:3 -27:4 ⁇
  • the first two patterns, at offsets 4 and 5, identify the IP-ID, which consists of two bytes that are set to zero.
  • the remaining four patterns at offsets 24 - 27, identify the four bytes used for the TCP sequence number, representing the value 0x01020304. It should be noted that there are no patterns for the destination port or address, even though the same values on those fields are shared by all packets of the attack. The reason for this is that most of the 'good' packets, which are using www, also share the same values. Therefore, including them in the signature does not add any distinguishing information.
  • the method may include adding to the signature the two or three packet parts that occur with the highest frequency and/or have a frequency over a certain threshold, the packets of which entirely contain the packets defined by the identified signature. In essence, the signature is then over-specified. In this case, the destination IP address, the destination port, and other offset/value pairs, which might be shared with 'good' packets, can appear in the signature as well.
  • the result may not only include the specific IP-ID and SEQ-number, but also: - The IP-version and header length, combined in one byte with the value 0x45. - Possibly the IP-TOS (type-of-service) field, which normally is left zero. - The IP-length, which for a SYN packet is often 40 bytes. - The fragmentation flags and offset, which are often zero. - The protocol, which is 6 for TCP. - The destination IP-address for the www server 7.
  • TCP header there exists: - The ACK-number of zero, which is normal for SYN-packets. - TCP flags are set to 2, in order to indicate a SYN-packet. - The window size may be 32767, which is a common value for certain network stacks, and which may also be used by the SYN-generator used in the attack. - The urgent-pointer is set to zero, which is also common.
  • the complete signature therefore may be (assuming the IP-address of www server 7 is 192.1.2.123): ⁇ 0:45 - 1 :0 - 2:0 - 3:40 - 4:0 - 5:0 - 6:0 - 7:0 - 9:6 - 16:192 - 17:1 - 18:2 - 19:123 - 22:0 - 23:80 - 24:1 - 25:2 - 26:3 - 27:4 - 28:0 - 29:0 - 30:0 - 31:0 - 32:0 - 33:2 - 34:127 - 35:255 - 38:0 - 39:0 ⁇ While this is the complete signature, it is clearly not at all human-readable or otherwise very useful. Therefore, a translation may occur.
  • IP-Version 4 IP-Header-Length: 5 IP-TOS: 0 IP-Length: 40 IP-ID: 0 IP-Flags/Fragments: 0 IP-Protocol: TCP (6) IP-DstAddr: 192.1.2.123 TCP-Seq.
  • the output text may be displayed in a GUI, in an e-mail, a report or any other means.
  • the offset/content pair-sets of the analyser module 100B output may be run through device specific output modules, so that rules in the specific rule-definition language of a given device may be created. In this case then, it is furthermore possible to provide additional communication means (such as communication channel 100D) to apply such filtering rules automatically to a device (such as IPS 5).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne un procédé visant à identifier un trafic de réseau anormal. Le procédé comporte les étapes consistant à: observer le trafic de réseau (étapes 10, 11) en vue de détecter un écart par rapport aux caractéristiques de trafic normal ou prévu (étape 12); lorsqu'un écart important est détecté, analyser (étape 13) une partie structurée d'un échantillon du trafic de réseau afin d'identifier un trait commun ou un motif, présent dans le contenu du trafic de réseau et qui se produit selon une fréquence correspondant approximativement à l'écart par rapport aux caractéristiques de trafic normal ou prévu; traduire (étape 14) une signature identifiée dans une forme lisible par machine ou par l'être humain.
PCT/NZ2004/000093 2004-05-18 2004-05-18 Procédé de détection de la signature du trafic d'un réseau WO2005111805A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/NZ2004/000093 WO2005111805A1 (fr) 2004-05-18 2004-05-18 Procédé de détection de la signature du trafic d'un réseau

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/NZ2004/000093 WO2005111805A1 (fr) 2004-05-18 2004-05-18 Procédé de détection de la signature du trafic d'un réseau

Publications (1)

Publication Number Publication Date
WO2005111805A1 true WO2005111805A1 (fr) 2005-11-24

Family

ID=35394320

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NZ2004/000093 WO2005111805A1 (fr) 2004-05-18 2004-05-18 Procédé de détection de la signature du trafic d'un réseau

Country Status (1)

Country Link
WO (1) WO2005111805A1 (fr)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2629457A1 (fr) * 2012-02-15 2013-08-21 JDS Uniphase Corporation Procédé et système de surveillance de réseau utilisant des paquets de signature
WO2015105684A1 (fr) * 2014-01-07 2015-07-16 Cpacket Networks, Inc. Appareil, système et procédé permettant de meilleures surveillance et interception de données de réseau
WO2016069119A1 (fr) * 2014-10-31 2016-05-06 Cyber Crucible Inc. Système et procédé destinés à la détection d'intrusion de réseau de canaux cachés en fonction du trafic de réseau hors ligne
US9397895B2 (en) 2011-12-13 2016-07-19 Viavi Solutions Inc. Method and system for collecting topology information
US9407645B2 (en) 2014-08-29 2016-08-02 Accenture Global Services Limited Security threat information analysis
US9503467B2 (en) 2014-05-22 2016-11-22 Accenture Global Services Limited Network anomaly detection
US9716721B2 (en) 2014-08-29 2017-07-25 Accenture Global Services Limited Unstructured security threat information analysis
US9886582B2 (en) 2015-08-31 2018-02-06 Accenture Global Sevices Limited Contextualization of threat data
US9979743B2 (en) 2015-08-13 2018-05-22 Accenture Global Services Limited Computer asset vulnerabilities
CN109634802A (zh) * 2018-11-12 2019-04-16 平安科技（深圳）有限公司 进程监控方法及终端设备
WO2020037118A1 (fr) * 2018-08-17 2020-02-20 Hughes Network Systems, Llc Atténuation d'attaques sur des réseaux de satellites
US11934937B2 (en) 2017-07-10 2024-03-19 Accenture Global Solutions Limited System and method for detecting the occurrence of an event and determining a response to the event

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030061513A1 (en) * 2001-09-27 2003-03-27 Guy Tsafnat Method and apparatus for detecting denial-of-service attacks using kernel execution profiles
US20030097439A1 (en) * 2000-10-23 2003-05-22 Strayer William Timothy Systems and methods for identifying anomalies in network data streams

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030097439A1 (en) * 2000-10-23 2003-05-22 Strayer William Timothy Systems and methods for identifying anomalies in network data streams
US20030061513A1 (en) * 2001-09-27 2003-03-27 Guy Tsafnat Method and apparatus for detecting denial-of-service attacks using kernel execution profiles

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9942101B2 (en) 2011-12-13 2018-04-10 Viavi Solutions Inc. Method and system for collecting topology information
US9397895B2 (en) 2011-12-13 2016-07-19 Viavi Solutions Inc. Method and system for collecting topology information
US9141506B2 (en) 2012-02-15 2015-09-22 Jds Uniphase Corporation Method and system for network monitoring using signature packets
US10129115B2 (en) 2012-02-15 2018-11-13 Viavi Solutions Inc. Method and system for network monitoring using signature packets
EP2629457A1 (fr) * 2012-02-15 2013-08-21 JDS Uniphase Corporation Procédé et système de surveillance de réseau utilisant des paquets de signature
WO2015105684A1 (fr) * 2014-01-07 2015-07-16 Cpacket Networks, Inc. Appareil, système et procédé permettant de meilleures surveillance et interception de données de réseau
US9503467B2 (en) 2014-05-22 2016-11-22 Accenture Global Services Limited Network anomaly detection
US10009366B2 (en) 2014-05-22 2018-06-26 Accenture Global Services Limited Network anomaly detection
US9729568B2 (en) 2014-05-22 2017-08-08 Accenture Global Services Limited Network anomaly detection
US9407645B2 (en) 2014-08-29 2016-08-02 Accenture Global Services Limited Security threat information analysis
US10880320B2 (en) 2014-08-29 2020-12-29 Accenture Global Services Limited Unstructured security threat information analysis
US9762617B2 (en) 2014-08-29 2017-09-12 Accenture Global Services Limited Security threat information analysis
US10063573B2 (en) 2014-08-29 2018-08-28 Accenture Global Services Limited Unstructured security threat information analysis
US9716721B2 (en) 2014-08-29 2017-07-25 Accenture Global Services Limited Unstructured security threat information analysis
WO2016069119A1 (fr) * 2014-10-31 2016-05-06 Cyber Crucible Inc. Système et procédé destinés à la détection d'intrusion de réseau de canaux cachés en fonction du trafic de réseau hors ligne
US9979743B2 (en) 2015-08-13 2018-05-22 Accenture Global Services Limited Computer asset vulnerabilities
US10313389B2 (en) 2015-08-13 2019-06-04 Accenture Global Services Limited Computer asset vulnerabilities
US9886582B2 (en) 2015-08-31 2018-02-06 Accenture Global Sevices Limited Contextualization of threat data
US11934937B2 (en) 2017-07-10 2024-03-19 Accenture Global Solutions Limited System and method for detecting the occurrence of an event and determining a response to the event
WO2020037118A1 (fr) * 2018-08-17 2020-02-20 Hughes Network Systems, Llc Atténuation d'attaques sur des réseaux de satellites
US10951581B2 (en) 2018-08-17 2021-03-16 Hughes Network Systems, Llc Mitigation of attacks on satellite networks
CN109634802A (zh) * 2018-11-12 2019-04-16 平安科技（深圳）有限公司 进程监控方法及终端设备
CN109634802B (zh) * 2018-11-12 2023-04-14 平安科技（深圳）有限公司 进程监控方法及终端设备

Similar Documents

Publication Publication Date Title
US7716742B1 (en) Systems and methods for determining characteristics of a network and analyzing vulnerabilities
US7317693B1 (en) Systems and methods for determining the network topology of a network
JP6053091B2 (ja) トラヒック特徴情報抽出方法、トラヒック特徴情報抽出装置及びトラヒック特徴情報抽出プログラム
US7197762B2 (en) Method, computer readable medium, and node for a three-layered intrusion prevention system for detecting network exploits
CN101589595B (zh) 用于潜在被污染端系统的牵制机制
US8042182B2 (en) Method and system for network intrusion detection, related network and computer program product
US20030084326A1 (en) Method, node and computer readable medium for identifying data in a network exploit
US20060161816A1 (en) System and method for managing events
US20030084319A1 (en) Node, method and computer readable medium for inserting an intrusion prevention system into a network stack
US20030097557A1 (en) Method, node and computer readable medium for performing multiple signature matching in an intrusion prevention system
US20050108415A1 (en) System and method for traffic analysis
US11457025B2 (en) Method and system for detecting and preventing data exfiltration attacks
Maselli et al. Design and implementation of an anomaly detection system: An empirical approach
CN103795709A (zh) 一种网络安全检测方法和系统
WO2005103899A1 (fr) Detection d'attaques de reseaux publics au moyen de signatures et d'analyse de contenu rapide
Li et al. HiFIND: A high-speed flow-level intrusion detection approach with DoS resiliency
Debar et al. Intrusion detection: Introduction to intrusion detection and security information management
CN114124516B (zh) 态势感知预测方法、装置及系统
US20040250158A1 (en) System and method for protecting an IP transmission network against the denial of service attacks
WO2005111805A1 (fr) Procédé de détection de la signature du trafic d'un réseau
Kaushik et al. Network forensic system for ICMP attacks
WO2006008307A1 (fr) Procede, systeme et programme informatique pour detecter un balayage non autorise sur un reseau
Khosravifar et al. An experience improving intrusion detection systems false alarm ratio by using honeypot
Vrat et al. Anomaly detection in IPv4 and IPv6 networks using machine learning
Stanciu Technologies, methodologies and challenges in network intrusion detection and prevention systems.

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPEN Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase