US20070234424A1 - Design and evaluation of a fast and robust worm detection algorithm - Google Patents
Design and evaluation of a fast and robust worm detection algorithm Download PDFInfo
- Publication number
- US20070234424A1 US20070234424A1 US11/395,053 US39505306A US2007234424A1 US 20070234424 A1 US20070234424 A1 US 20070234424A1 US 39505306 A US39505306 A US 39505306A US 2007234424 A1 US2007234424 A1 US 2007234424A1
- Authority
- US
- United States
- Prior art keywords
- change
- worm
- arrival rates
- detecting
- traffic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
- G06F21/561—Virus type analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/145—Countermeasures against malicious traffic the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
Definitions
- the invention pertains to identification of Internet worm propagation.
- Malicious computer worms are a danger to any computer that is accessible via a computer network, such as the Internet.
- a computer worm is a self-replicating program, similar to a computer virus. However, unlike a virus, which attaches itself to and infects an executable file on a computer, a worm is self-contained and does not need to be part of another program to propagate itself.
- Worms are often designed to exploit the file transmission capabilities of many computers.
- a worm uses a network to send copies of itself to other systems and it does so without any necessary human intervention, such as forwarding by email, which is a common method of spreading a virus.
- Scan-based worms use a form of scanning (transmission of packets) from an infected host to a potential new host as a propagation technique. Based on the potential host's response to this scan (i.e., does the potential host respond positively, or does the response indicate that the potential host will not accept additional packets from the infected host), the infected host determines whether to spread the worm to the potential host. It is also possible that a worm can be carried in a single packet. In this situation, the infected host transmits the packet to another address without the need for a response from the potential new host.
- Typical approaches to preventing a worm outbreak involve worm detection, dissection and signature development.
- Signature development occurs once the worm has been identified, and a common pattern is found which can be used to identify the worm. This signature must then be propagated throughout the network, either to a firewall running security software or to each individually connected computer running a certain security program. Once the security program receives the signature, the database of signatures the security program recognizes as malicious is updated, and the computer running the security program is protected against the identified worm. But this approach does not address the case of previously unidentified worms for which no signature has been identified.
- Another non-signature based approach involves detecting a worm by identifying the exponential growth trend of scanning rates on a particular network connection.
- this process requires studying the growth trend over a given interval of time.
- Different worms have different propagation times. For example, a worm may inhabit a host computer for an hour before propagating to a new host. If the wrong interval of time is chosen to study the growth trend, then relevant information relating to the growth trend is missed and a worm cannot be effectively detected.
- This method should be accurate and robust (i.e., it must quickly and accurately identify different propagation characteristics of different worms), and work quickly enough so that a worm can be detected at the inception of the worm spread, before its propagation hits its exponential growth rate.
- a new worm detection technique utilizes a process to detect the outbreak of a new worm without knowing the signature of the worm.
- Changes in the traffic pattern of unsolicited packets are detected, and any changes in traffic patterns are analyzed to determine if they are consistent with changes in traffic associated with worm propagation. More specifically, traffic arrival patterns are monitored, primarily for unsolicited traffic, i.e., traffic coming into a computer network connection that was not first requested. Next, changes in the traffic patterns are analyzed. During this analysis, certain patterns of growth rates relating to the unsolicited traffic that are indicative of the presence of worm propagation are searched for, such as an exponential growth rate of unsolicited traffic from numerous senders. When such a pattern is detected, it is assumed that a worm is present so that measures can be implemented to halt its progress.
- FIG. 1 is a diagram illustrating a basic computer network.
- FIG. 2 is a flow chart illustrating a method in accordance with one particular embodiment of the invention.
- FIG. 3 is a printout of a worm detection algorithm according to one embodiment of the present invention.
- FIG. 4 is a graph illustrating the effectiveness of the algorithm of FIG. 3 in detecting the outbreak of the Slammer Worm.
- a non-signature based method for detecting Internet worms is presented. By monitoring and analyzing traffic patterns at a network connection, a worm can be detected.
- FIG. 1 illustrates a computer network 100 .
- clients 102 , 104 and 106 connect to the server 115 through router 110 .
- server 115 Once connected to server 115 , clients 102 , 104 and 106 have access to Internet 120 .
- client 130 Also connected to Internet 120 through server 125 is client 130 .
- Router 110 is equipped with a firewall running security software intended to monitor network traffic, specifically the packets sent and received through the router, and identify and stop any malicious traffic.
- Clients 102 , 104 , 106 and 130 are also running a desktop security program for scanning individual packets sent to the client.
- Conventional security software identifies malicious packets based on the signature of the packet, or a unique identifier for each packet.
- new worms are being designed which can infect millions of hosts well before a signature can be found.
- client 130 is infected with a previously unidentified worm such that client 130 becomes a scanner, meaning the worm sends out a scan, which is a series of packets intended to poll a potential host computer.
- Client 130 scans clients 102 , 104 and 106 . Since the worm has no known signature, neither the firewall nor the desktop security program would recognize that the packets are malicious. If clients 102 , 104 and 106 respond positively to the scan, client 130 passes the worm to clients 102 , 104 and 106 , and they become new hosts.
- FIG. 2 is a flow chart illustrating a worm detection method in accordance with the principles of the present invention.
- Step 200 the unsolicited traffic being received at a specific network location is identified and isolated. Unsolicited traffic refers to network traffic that was not requested by a receiving computer. For example, the traffic at router 10 from FIG. 1 is monitored. This unsolicited traffic is isolated from the solicited traffic, to produce a traffic trace based only on the unsolicited packets received at the router.
- step 205 any changes in traffic arrival patterns are determined. Though all changes in traffic arrival patterns may not be due to worm propagation, worm propagation usually results in traffic arrival pattern changes with certain similar characteristics. As described in further detail below, the system uses cumulative summing, or CUSUM, a common statistical analysis tool used to detect changes in data sets, to study the arrival rates to determine any changes. CUSUM will detect a trend of increasing unsolicited packet arrival rate.
- step 207 in which, if CUSUM has detected a change in the arrival rates, the process continues to step 210 . If CUSUM has failed to detect a change, flow returns to step 200 .
- Step 210 the changes are analyzed to determine if the changes are related to worm propagation. Specifically, the changes are analyzed to determine whether the changes have some exponential growth patterns in arrival rates.
- a Maximum Likelihood Estimation (MLE) is used to produce a non-stationary Poisson process and estimate its rate. Poisson processes are commonly used in statistical analysis to examine the number of times an event happens during a given time interval, where the probability for the event occurring is constant with respect to time. An alarm will trigger when the MLE yields a significant increase in propagation rate with a high level of confidence.
- Steps 205 and 210 are further explained herein below.
- step 205 first the inter-arrival times of the unsolicited packets are determined.
- T n denotes the arrival time of the n-th unsolicited packet in a t-sample (a sample taken at most once every t seconds), and
- the inter-arrival times ⁇ X n : n w ⁇ n ⁇ should have a decreasing mean that is less than ⁇ .
- This shift in the distribution of inter-arrival times may be considered a change point in statistical terms and CUSUM is designed for detecting changes from one distribution to another such as this change in inter-arrival times.
- CUSUM The theory behind CUSUM is that, if the mean of X n shifts from ⁇ to something smaller than ⁇ p ⁇ at sample n w then S n will tend to accumulate positive increments after n w and thus eventually cross the threshold h and signal a change.
- ⁇ is not known, as arrival times can vary due to network conditions; but an estimate, such as an Exponentially Weighted Moving Average (EWMA) can be used in its place.
- EWMA Exponentially Weighted Moving Average
- h requires trading off between detection delay (i.e., sensitivity) and the false detection rate. Small values of h provide quick detection when changes are present but also give more false alarms.
- the threshold h can be calculated from the expected time between false alarms, known as the Average Run Length (ARL) in quality control.
- ARL Average Run Length
- the CUSUM process used in step 205 is not used to directly trigger a worm alarm, but only as a first stage toward worm detection.
- the process proceeds to step 210 in which the detected changes are analyzed and a worm propagation model is estimated.
- T n0 denote the most recent time (prior to the current signal) when the CUSUM transitioned from a value of 0 to a positive value.
- Case 1 If a worm exists, its arrival is most likely earlier than T n0 (hereinafter Case 1 ). However, it is possible for a worm to arrive between T n0 and the CUSUM signaling time (hereinafter Case 2 ). This happens rarely and the lag between worm infection and the CUSUM signal transitioning from 0 to a positive value will most likely be small, e.g., on the order of second. Of course, it is also possible that no worm exists (hereinafter Case 3 ), which statistically is the most likely case. Let us first focus on the statistical estimation of the worm propagation model based on Case 1 . It will be shown below that this also includes Case 3 and also serves as a good approximation for Case 2 .
- Scanner arrivals in a t-sample before a worm outbreak are well-modeled as a Poisson process with rate b(t) that changes slowly with time.
- b(t) can be estimated easily using local averaging. Propagation characteristics are described by the parameters a and r that depend on the efficiency of the worm and the size of the network being monitored. Although a is not identifiable (i.e., cannot be estimated statistically) when t w is unknown, the exponential rate r is identifiable. A worm alarm is triggered when the data indicates with high confidence that r is significantly higher than a small tolerable rate r 0 .
- T n is the current arrival relative to the signaling time T n0 .
- any estimators of a and r must be based on ( T 1 , . . . , T n ), the distribution depends on the unknowns n 0 and T n0 .
- the following theorem and its corollary demonstrate that the r can be estimated from the T j , but a cannot.
- MLE maximum likelihood estimates
- the MLE ⁇ circumflex over (r) ⁇ and its estimated asymptotic variance are used repeatedly in the second stage to test whether r is significantly positive.
- r>r 0 is tested against r ⁇ r 0 , where r 0 (say 0.0001) is the maximal rate that can be ignored.
- the CUSUM statistic is reset to zero after a signal is triggered.
- a large CUSUM is required for the step 210 of FIG. 2 to operate.
- the CUSUM is not reset immediately upon crossing the threshold h, rather the reset occurs only after a substantial downward trend is seen following the trigger.
- the algorithm identifies a downtrend if the current CUSUM value is, for example, less than 80% of the maximum value recorded since the previous reset.
- the algorithm may implement the following random tail-draw technique.
- ⁇ n-1 be the most recent exponentially weighted moving average (EWMA) estimate of E(X n ). If X n lies outside of the 0.01% and 99.99% percentiles of the exponential( ⁇ n-1 ) distribution, then it is replaced with a random draw ⁇ tilde over (X) ⁇ n from the corresponding distribution for the purpose of calculating S n .
- EWMA exponentially weighted moving average
- FIG. 3 shows an exemplary worm detection algorithm in accordance with principles of the present invention.
- This algorithm corresponds to steps 205 - 210 in FIG. 2 .
- Line by line the algorithm proceeds as follows.
- Lines 1 and 2 initialize the CUSUM and an EWMA estimate of the mean inter-arrival time. Starting the EWMA based on the median of an initial sample provides robustness against outliers. Dividing the median by log(2) produces an estimate of the mean.
- Line 4 computes the current CUSUM and Line 5 the current EWMA. No further action is required if the CUSUM is zero.
- the EWMA parameter w determines the depth of the memory and the relative weight between the current and previous data.
- FIG. 4 plots the number of scanners arriving at the firewall every second observed 1,000 seconds surrounding the outbreak of Slammer.
- the first dashed vertical line 405 marks the time of arrival of the first Slammer scan and the second dashed vertical line 410 marks when the worm detector of the present invention signals a worm outbreak.
- the average number of unsolicited packets is about 2.5 per second before the first worm scan arrives at time 364 seconds.
- the alarm is raised at just 16 seconds after the initial Slammer scan and at the time the scanners rate has increased to about 6.5 per second. Scans from Slammer peak at about 600 seconds when almost all vulnerable hosts world-wide have become infected.
- the algorithm was able to give a warning in as little as 6.7% of the time it took for Slammer to infect all hosts. In the trace, only 60 hosts had been affected before Slammer would have been detected, whereas a total of 72,516 were actually infected in total when the worm was left to propagate naturally.
- FIG. 4 is shown only as an example of the functionality of the worm detection algorithm. It illustrates one embodiment of the present invention and is not intended to limit the present invention in any matter.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Virology (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- The invention pertains to identification of Internet worm propagation.
- Malicious computer worms (or Internet worms) are a danger to any computer that is accessible via a computer network, such as the Internet. A computer worm is a self-replicating program, similar to a computer virus. However, unlike a virus, which attaches itself to and infects an executable file on a computer, a worm is self-contained and does not need to be part of another program to propagate itself.
- Worms are often designed to exploit the file transmission capabilities of many computers. A worm uses a network to send copies of itself to other systems and it does so without any necessary human intervention, such as forwarding by email, which is a common method of spreading a virus. Scan-based worms use a form of scanning (transmission of packets) from an infected host to a potential new host as a propagation technique. Based on the potential host's response to this scan (i.e., does the potential host respond positively, or does the response indicate that the potential host will not accept additional packets from the infected host), the infected host determines whether to spread the worm to the potential host. It is also possible that a worm can be carried in a single packet. In this situation, the infected host transmits the packet to another address without the need for a response from the potential new host.
- Typical approaches to preventing a worm outbreak involve worm detection, dissection and signature development. Signature development occurs once the worm has been identified, and a common pattern is found which can be used to identify the worm. This signature must then be propagated throughout the network, either to a firewall running security software or to each individually connected computer running a certain security program. Once the security program receives the signature, the database of signatures the security program recognizes as malicious is updated, and the computer running the security program is protected against the identified worm. But this approach does not address the case of previously unidentified worms for which no signature has been identified.
- Previously unidentified, fast spreading worms are a reality, as amply demonstrated by worms such as the Slammer worm. The release and propagation of the Slammer worm in 2003 was a revolutionary event in the study of computer worm propagation. It not only demonstrated in an unprecedented way the scale and disruption that is possible in the real world with a relatively compact worm, it also showed the ineffectiveness of current techniques in detecting and countering these new fast spreading worms. More specifically, in the early phase of Slammer propagation, it doubled in size every 8.5 seconds. It reached a maximum scan rate of 55 million addresses per second and was able to infect more than 90 percent of vulnerable hosts within 10 minutes. In the end, even though Slammer carried no malicious payload and its main damage was in network resource (bandwidth and CPU) consumption, it served as a wake-up call to network administrators and the computer security industry.
- With these kinds of fast spreading worms, the traditional approach of signature-based detection is no longer sufficient. Worms can infect all vulnerable hosts well before a signature can be identified. Several approaches have been proposed utilizing non-signature based detection means. One such approach detects a worm by monitoring the correlation between the incoming and outgoing packets at a network connection. More specifically, this approach studies the correlation of the payloads and packet headers of the incoming and outgoing packets. However, this correlation is not always reliable. Specifically, the technique was most effective against earlier worms that used a fixed destination port, or a portion of the network address specifying the port where the packet is received on the network connection, which made correlation studies easier as a single destination port could be monitored across the network. However, recent worm attacks randomize the destination port on the network connection. This renders monitoring of destination port incoming and outgoing packets and studying the correlation between the two packet types less reliable for worm detection.
- Another non-signature based approach involves detecting a worm by identifying the exponential growth trend of scanning rates on a particular network connection. However, this process requires studying the growth trend over a given interval of time. Different worms have different propagation times. For example, a worm may inhabit a host computer for an hour before propagating to a new host. If the wrong interval of time is chosen to study the growth trend, then relevant information relating to the growth trend is missed and a worm cannot be effectively detected.
- What is needed is a fast method to detect worms lacking known signatures. This method should be accurate and robust (i.e., it must quickly and accurately identify different propagation characteristics of different worms), and work quickly enough so that a worm can be detected at the inception of the worm spread, before its propagation hits its exponential growth rate.
- In accordance with the principles of the present invention, a new worm detection technique is presented that utilizes a process to detect the outbreak of a new worm without knowing the signature of the worm. Changes in the traffic pattern of unsolicited packets are detected, and any changes in traffic patterns are analyzed to determine if they are consistent with changes in traffic associated with worm propagation. More specifically, traffic arrival patterns are monitored, primarily for unsolicited traffic, i.e., traffic coming into a computer network connection that was not first requested. Next, changes in the traffic patterns are analyzed. During this analysis, certain patterns of growth rates relating to the unsolicited traffic that are indicative of the presence of worm propagation are searched for, such as an exponential growth rate of unsolicited traffic from numerous senders. When such a pattern is detected, it is assumed that a worm is present so that measures can be implemented to halt its progress.
-
FIG. 1 is a diagram illustrating a basic computer network. -
FIG. 2 is a flow chart illustrating a method in accordance with one particular embodiment of the invention. -
FIG. 3 is a printout of a worm detection algorithm according to one embodiment of the present invention. -
FIG. 4 is a graph illustrating the effectiveness of the algorithm ofFIG. 3 in detecting the outbreak of the Slammer Worm. - In accordance with the present invention, a non-signature based method for detecting Internet worms is presented. By monitoring and analyzing traffic patterns at a network connection, a worm can be detected.
-
FIG. 1 illustrates acomputer network 100. In this network,clients server 115 through router 110. Once connected toserver 115,clients server 125 isclient 130. Router 110 is equipped with a firewall running security software intended to monitor network traffic, specifically the packets sent and received through the router, and identify and stop any malicious traffic.Clients - Conventional security software identifies malicious packets based on the signature of the packet, or a unique identifier for each packet. However, new worms are being designed which can infect millions of hosts well before a signature can be found. For example,
client 130 is infected with a previously unidentified worm such thatclient 130 becomes a scanner, meaning the worm sends out a scan, which is a series of packets intended to poll a potential host computer.Client 130scans clients clients client 130 passes the worm toclients -
FIG. 2 is a flow chart illustrating a worm detection method in accordance with the principles of the present invention. InStep 200, the unsolicited traffic being received at a specific network location is identified and isolated. Unsolicited traffic refers to network traffic that was not requested by a receiving computer. For example, the traffic atrouter 10 fromFIG. 1 is monitored. This unsolicited traffic is isolated from the solicited traffic, to produce a traffic trace based only on the unsolicited packets received at the router. - The system now proceeds to step 205 where any changes in traffic arrival patterns are determined. Though all changes in traffic arrival patterns may not be due to worm propagation, worm propagation usually results in traffic arrival pattern changes with certain similar characteristics. As described in further detail below, the system uses cumulative summing, or CUSUM, a common statistical analysis tool used to detect changes in data sets, to study the arrival rates to determine any changes. CUSUM will detect a trend of increasing unsolicited packet arrival rate.
- The process continues to
decision step 207, in which, if CUSUM has detected a change in the arrival rates, the process continues to step 210. If CUSUM has failed to detect a change, flow returns to step 200. - If a change is detected in
Step 205, the system proceeds to Step 210 where the changes are analyzed to determine if the changes are related to worm propagation. Specifically, the changes are analyzed to determine whether the changes have some exponential growth patterns in arrival rates. A Maximum Likelihood Estimation (MLE) is used to produce a non-stationary Poisson process and estimate its rate. Poisson processes are commonly used in statistical analysis to examine the number of times an event happens during a given time interval, where the probability for the event occurring is constant with respect to time. An alarm will trigger when the MLE yields a significant increase in propagation rate with a high level of confidence. -
Steps step 205, first the inter-arrival times of the unsolicited packets are determined. Tn denotes the arrival time of the n-th unsolicited packet in a t-sample (a sample taken at most once every t seconds), and Xn=Tn−Tn-1, is the inter-arrival time where T0=0. It is assumed that the inter-arrival times {Xn: 1≦n≦nw} before the worm starts are independently and identically distributed with mean μ, where Tnw represents the time of the first worm scan. After a worm arrives, the inter-arrival times {Xn: nw≦n≦∞} should have a decreasing mean that is less than μ. This shift in the distribution of inter-arrival times may be considered a change point in statistical terms and CUSUM is designed for detecting changes from one distribution to another such as this change in inter-arrival times. - The CUSUM scheme can be applied as follows. Set S0=0 and define
S n=max(0,S n-1 +μ−X n −pμ), n=1, 2, . . .
where p is dependant on the expected drop in mean inter-arrival times due to a worm. Typically, pμ is set to about half the size of the drop in mean inter-arrival time that is crucial to detect a change in arrival rates quickly. A change of inter-arrival time is signaled whenever Sn exceeds a certain threshold h. The theory behind CUSUM is that, if the mean of Xn shifts from μ to something smaller than μ−pμ at sample nw then Sn will tend to accumulate positive increments after nw and thus eventually cross the threshold h and signal a change. In practice, μ is not known, as arrival times can vary due to network conditions; but an estimate, such as an Exponentially Weighted Moving Average (EWMA) can be used in its place. The EWMA is based on the median of an initial sample of inter-arrival times. - Choosing the threshold parameter, h, requires trading off between detection delay (i.e., sensitivity) and the false detection rate. Small values of h provide quick detection when changes are present but also give more false alarms. The threshold h can be calculated from the expected time between false alarms, known as the Average Run Length (ARL) in quality control.
- As seen in the flowchart, the CUSUM process used in
step 205 is not used to directly trigger a worm alarm, but only as a first stage toward worm detection. As previously noted, if the CUSUM value Sn exceeds threshold h, the process proceeds to step 210 in which the detected changes are analyzed and a worm propagation model is estimated. However, if a new worm outbreak is in progress, it is probable that some time has elapsed between the outbreak and the CUSUM signal. Whenstep 205 detects an unusual increase in unsolicited network traffic, there are three relevant cases that this increase might indicate. Let Tn0 denote the most recent time (prior to the current signal) when the CUSUM transitioned from a value of 0 to a positive value. If a worm exists, its arrival is most likely earlier than Tn0 (hereinafter Case 1). However, it is possible for a worm to arrive between Tn0 and the CUSUM signaling time (hereinafter Case 2). This happens rarely and the lag between worm infection and the CUSUM signal transitioning from 0 to a positive value will most likely be small, e.g., on the order of second. Of course, it is also possible that no worm exists (hereinafter Case 3), which statistically is the most likely case. Let us first focus on the statistical estimation of the worm propagation model based onCase 1. It will be shown below that this also includesCase 3 and also serves as a good approximation forCase 2. - Scanner arrivals in a t-sample before a worm outbreak are well-modeled as a Poisson process with rate b(t) that changes slowly with time. Scanners that arise from a fresh worm outbreak can be modeled as a non-stationary Poisson process with rate:
λ(t)=ae r(t−tw) I(t≧t w)
where tw is the time when the first worm scan arrives; a is the expected number of worm scanner arrivals in the first second; r is the exponential propagation rate; and I(x) is an indicatorfunction having value 1 when x is true and 0 otherwise. It is assumed that any background scanners (non-malicious scanners) and the ones caused by a new worm are independent. The superposition of background and worm scanners is thus modeled as a non-stationary Poisson process with rate:
λ(t)=b(t)+ae r(t−tw) I(t≧t w). - Because the background traffic is approximately stationary, its rate b(t) can be estimated easily using local averaging. Propagation characteristics are described by the parameters a and r that depend on the efficiency of the worm and the size of the network being monitored. Although a is not identifiable (i.e., cannot be estimated statistically) when tw is unknown, the exponential rate r is identifiable. A worm alarm is triggered when the data indicates with high confidence that r is significantly higher than a small tolerable rate r0.
- For simplicity, assume that the worm starts at 0 (i.e., tw=0), unsolicited scanners arrive at times T1, T2, . . . according to a Poisson process with rate λ(t)=b+aert, t≧0, and the corresponding CUSUM sequence S1, S2, . . . remains below the threshold h until some arrival Tn0 (n0≧1) when the CUSUM exceeds h and therefore causes flow to proceed to step 210 in which the change is to be further analyzed.
- With respect to step 210, let us define
T j=Tn0+j−Tn0 for j=1, 2, . . . , n, whereT n is the current arrival relative to the signaling time Tn0. Note that we can only observeT 1, . . . ,T n and not the complete stream of arrivals T1, . . . T0, Tn0+1, . . . , Tn0+n because the worm outbreak time tw=0 is not generally known. Thus, any estimators of a and r must be based on (T 1, . . . ,T n), the distribution depends on the unknowns n0 and Tn0. The following theorem and its corollary demonstrate that the r can be estimated from theT j, but a cannot. -
Theorem 1. Let T1, T2, . . . denote consecutive arrival times from a Poisson process with positive rate λ(t)=b+aert beginning at t=0. DefineT j=Tn0+j−Tn0 for j=1, 2, . . . and for some n0≧1. Then, given Tn0=t0, the relative timesT 1,T 2, . . . are arrivals from a Poisson process with rateλ (t)=b+a ert, t≧0, wherea =aeˆ(rt0). -
Corollary 1. Under the conditions ofTheorem 1 and assuming that a>0, the parametersa , b and r are identified by the data (T T n) for n≧3 but the parameter a is not identified unless to is known. - The exception a=0 corresponds to no worm and in this case the propagation rate r has no meaning. Fortunately, for the purpose of worm detection, r is the most interesting parameter and it can be estimated by maximum likelihood inference as discussed next.
- Let
Λ (t)=∫0 rλ (s)ds. Then the normalized arrival timesΛ (T 1),Λ (T 2), . . . follow a stationary Poisson process withrate 1. Let ln(r,a )=log p(T 1, . . . ,T n|Tn0=t0) be the log-likelihood function for theT j's conditional on Tn0. By the density transformation formula
the maximum likelihood estimates (MLE) are defined as
({circumflex over (r)},â )=arg max l n(r,a ).
Let θ=(r,a )T and {circumflex over (θ)}=({circumflex over (r)},{circumflex over (a )})T Denote ln(θ)=ln(r,a ). Then the MLE {circumflex over (θ)} has positive properties as summarized inTheorem 2 below. -
Theorem 2. Under the conditions ofTheorem 1, if θ is bounded, then as n→∞,
{circumflex over (θ)}→θ,
in probability and
√{square root over (n)}({circumflex over (θ)}−θ)→N(0,I(θ)−1),
in distribution where I(θ) is the information matrix,
and can be estimated consistently by - The MLE {circumflex over (r)} and its estimated asymptotic variance are used repeatedly in the second stage to test whether r is significantly positive. In particular, r>r0 is tested against r≦r0, where r0 (say 0.0001) is the maximal rate that can be ignored. Let se({circumflex over (r)}) be the asymptotic standard error of {circumflex over (r)}, that is,
se({circumflex over (r)})=√{square root over ([Î −1]11 /n)}.
Since Zn≡({circumflex over (r)}−r0)/se({circumflex over (r)}) is asymptotically normally distributed with mean 0 andvariance 1 under the null hypothesis r=r0, the second stage declares a worm outbreak when Zn>qc, where qc is a threshold such as the 99.99 percentile of the standard Normal distribution. For example qc=3.8 is the 99.99% quantile of the Normal distribution. - In most CUSUM monitoring applications, the CUSUM statistic is reset to zero after a signal is triggered. In the present algorithm, however, a large CUSUM is required for the
step 210 ofFIG. 2 to operate. Hence, the CUSUM is not reset immediately upon crossing the threshold h, rather the reset occurs only after a substantial downward trend is seen following the trigger. The algorithm identifies a downtrend if the current CUSUM value is, for example, less than 80% of the maximum value recorded since the previous reset. - Although scanner arrivals, for the most part, resemble a locally stationary Poisson process, outliers do occasionally occur in arrival traces. These are cases in which the inter-arrival time between scanners is abnormally large for one reason or another. These outliers never trigger a false alarm because the MLE does not yield a large r in
step 210. However, the outliers can easily lead to a CUSUM signal and thus needlessly trigger the MLE computations. - To reduce the impact of outliers in creating such false alarms, the algorithm may implement the following random tail-draw technique. Let μn-1 be the most recent exponentially weighted moving average (EWMA) estimate of E(Xn). If Xn lies outside of the 0.01% and 99.99% percentiles of the exponential(μn-1) distribution, then it is replaced with a random draw {tilde over (X)}n from the corresponding distribution for the purpose of calculating Sn.
-
FIG. 3 shows an exemplary worm detection algorithm in accordance with principles of the present invention. This algorithm corresponds to steps 205-210 inFIG. 2 . Line by line, the algorithm proceeds as follows.Lines Line 4 computes the current CUSUM andLine 5 the current EWMA. No further action is required if the CUSUM is zero. The EWMA parameter w determines the depth of the memory and the relative weight between the current and previous data. Although there is no general rule for the optimal choice of w, in our experiments, performance of the algorithm is similar for various values of w between 10−4 to 10−7. Whenever the CUSUM becomes positive,lines line 10 updates j and Smax andline 11 resets the CUSUM to zero if a downtrend is recognized with respect to the local maximum.Line 12 triggers estimation of the propagation rate inlines Lines 15 through 17 test whether the data suggest a significantly large propagation rate with high confidence. If so, the alarm is raised until such time as the CUSUM is reset to zero again. - A trace of the Slammer Worm outbreak was used to test the algorithm.
FIG. 4 plots the number of scanners arriving at the firewall every second observed 1,000 seconds surrounding the outbreak of Slammer. The first dashedvertical line 405 marks the time of arrival of the first Slammer scan and the second dashedvertical line 410 marks when the worm detector of the present invention signals a worm outbreak. The average number of unsolicited packets is about 2.5 per second before the first worm scan arrives at time 364 seconds. The alarm is raised at just 16 seconds after the initial Slammer scan and at the time the scanners rate has increased to about 6.5 per second. Scans from Slammer peak at about 600 seconds when almost all vulnerable hosts world-wide have become infected. The algorithm was able to give a warning in as little as 6.7% of the time it took for Slammer to infect all hosts. In the trace, only 60 hosts had been affected before Slammer would have been detected, whereas a total of 72,516 were actually infected in total when the worm was left to propagate naturally. -
FIG. 4 is shown only as an example of the functionality of the worm detection algorithm. It illustrates one embodiment of the present invention and is not intended to limit the present invention in any matter. - It should be clear to persons familiar with the related arts that the process, procedures and/or steps of the invention described herein can be performed by a programmed computing device running software designed to cause the computing device to perform the processes, procedures and/or steps described herein. These processes, procedures and/or steps also could be performed by other forms of circuitry including, but not limited to, application-specific integrated circuits, logic circuits, and state machines.
- Having thus described a particular embodiment of the invention, various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements as are made obvious by this disclosure are intended to be part of this description though not expressly stated herein, and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description is by way of example only, and not limiting. The invention is limited only as defined in the following claims and equivalents thereto.
Claims (17)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/395,053 US20070234424A1 (en) | 2006-03-31 | 2006-03-31 | Design and evaluation of a fast and robust worm detection algorithm |
US14/096,145 US9069962B2 (en) | 2006-03-31 | 2013-12-04 | Evaluation of a fast and robust worm detection algorithm |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/395,053 US20070234424A1 (en) | 2006-03-31 | 2006-03-31 | Design and evaluation of a fast and robust worm detection algorithm |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/096,145 Continuation US9069962B2 (en) | 2006-03-31 | 2013-12-04 | Evaluation of a fast and robust worm detection algorithm |
Publications (1)
Publication Number | Publication Date |
---|---|
US20070234424A1 true US20070234424A1 (en) | 2007-10-04 |
Family
ID=38561112
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/395,053 Abandoned US20070234424A1 (en) | 2006-03-31 | 2006-03-31 | Design and evaluation of a fast and robust worm detection algorithm |
US14/096,145 Expired - Fee Related US9069962B2 (en) | 2006-03-31 | 2013-12-04 | Evaluation of a fast and robust worm detection algorithm |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/096,145 Expired - Fee Related US9069962B2 (en) | 2006-03-31 | 2013-12-04 | Evaluation of a fast and robust worm detection algorithm |
Country Status (1)
Country | Link |
---|---|
US (2) | US20070234424A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080209557A1 (en) * | 2007-02-28 | 2008-08-28 | Microsoft Corporation | Spyware detection mechanism |
US20110209001A1 (en) * | 2007-12-03 | 2011-08-25 | Microsoft Corporation | Time modulated generative probabilistic models for automated causal discovery |
US20140189872A1 (en) * | 2012-04-09 | 2014-07-03 | Tencent Technology (Shenzhen) Company Limited | Method and device for monitoring virus trend abnormality |
US20150249676A1 (en) * | 2014-02-28 | 2015-09-03 | Fujitsu Limited | Monitoring method and monitoring apparatus |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105428548A (en) * | 2015-11-24 | 2016-03-23 | 昆山国显光电有限公司 | Oled display structure and manufacturing method |
US9548991B1 (en) | 2015-12-29 | 2017-01-17 | International Business Machines Corporation | Preventing application-level denial-of-service in a multi-tenant system using parametric-sensitive transaction weighting |
US10015139B2 (en) * | 2016-02-03 | 2018-07-03 | Oracle International Corporation | Inter-arrival time intrusion-detection technique to provide enhanced cybersecurity |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040107321A1 (en) * | 2000-12-14 | 2004-06-03 | Altman Erik R. | Symmetric multi-processing system |
US20050066053A1 (en) * | 2001-03-20 | 2005-03-24 | Worldcom, Inc. | System, method and apparatus that isolate virtual private network (VPN) and best effort traffic to resist denial of service attacks |
US20060173996A1 (en) * | 1997-10-28 | 2006-08-03 | Philip Bates | Multi-user computer system |
US20060173992A1 (en) * | 2002-11-04 | 2006-08-03 | Daniel Weber | Event detection/anomaly correlation heuristics |
US20060174341A1 (en) * | 2002-03-08 | 2006-08-03 | Ciphertrust, Inc., A Georgia Corporation | Systems and methods for message threat management |
US20060210942A1 (en) * | 2003-09-08 | 2006-09-21 | Dentaurum J.P. Winkelstroeter Kg | Orthodontic fastening element |
US20080209521A1 (en) * | 2004-07-07 | 2008-08-28 | Robert Anderson Malaney | Location-Enabled Security Services in Wireless Network |
US7627899B1 (en) * | 2005-04-22 | 2009-12-01 | Sun Microsystems, Inc. | Method and apparatus for improving user experience for legitimate traffic of a service impacted by denial of service attack |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7917647B2 (en) * | 2000-06-16 | 2011-03-29 | Mcafee, Inc. | Method and apparatus for rate limiting |
US7017186B2 (en) * | 2002-07-30 | 2006-03-21 | Steelcloud, Inc. | Intrusion detection system using self-organizing clusters |
US20040078592A1 (en) * | 2002-10-16 | 2004-04-22 | At & T Corp. | System and method for deploying honeypot systems in a network |
US8176527B1 (en) * | 2002-12-02 | 2012-05-08 | Hewlett-Packard Development Company, L. P. | Correlation engine with support for time-based rules |
CN1898922A (en) * | 2004-10-21 | 2007-01-17 | 日本电信电话株式会社 | Protect device, protect method, protect program, and network attack protect system |
-
2006
- 2006-03-31 US US11/395,053 patent/US20070234424A1/en not_active Abandoned
-
2013
- 2013-12-04 US US14/096,145 patent/US9069962B2/en not_active Expired - Fee Related
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060173996A1 (en) * | 1997-10-28 | 2006-08-03 | Philip Bates | Multi-user computer system |
US20040107321A1 (en) * | 2000-12-14 | 2004-06-03 | Altman Erik R. | Symmetric multi-processing system |
US20050066053A1 (en) * | 2001-03-20 | 2005-03-24 | Worldcom, Inc. | System, method and apparatus that isolate virtual private network (VPN) and best effort traffic to resist denial of service attacks |
US20060174341A1 (en) * | 2002-03-08 | 2006-08-03 | Ciphertrust, Inc., A Georgia Corporation | Systems and methods for message threat management |
US20060173992A1 (en) * | 2002-11-04 | 2006-08-03 | Daniel Weber | Event detection/anomaly correlation heuristics |
US7363656B2 (en) * | 2002-11-04 | 2008-04-22 | Mazu Networks, Inc. | Event detection/anomaly correlation heuristics |
US20060210942A1 (en) * | 2003-09-08 | 2006-09-21 | Dentaurum J.P. Winkelstroeter Kg | Orthodontic fastening element |
US20080209521A1 (en) * | 2004-07-07 | 2008-08-28 | Robert Anderson Malaney | Location-Enabled Security Services in Wireless Network |
US7627899B1 (en) * | 2005-04-22 | 2009-12-01 | Sun Microsystems, Inc. | Method and apparatus for improving user experience for legitimate traffic of a service impacted by denial of service attack |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080209557A1 (en) * | 2007-02-28 | 2008-08-28 | Microsoft Corporation | Spyware detection mechanism |
US9021590B2 (en) * | 2007-02-28 | 2015-04-28 | Microsoft Technology Licensing, Llc | Spyware detection mechanism |
US20110209001A1 (en) * | 2007-12-03 | 2011-08-25 | Microsoft Corporation | Time modulated generative probabilistic models for automated causal discovery |
US20140189872A1 (en) * | 2012-04-09 | 2014-07-03 | Tencent Technology (Shenzhen) Company Limited | Method and device for monitoring virus trend abnormality |
US9817973B2 (en) * | 2012-04-09 | 2017-11-14 | Tencent Technology (Shenzhen) Company Limited | Method and device for monitoring virus trend abnormality |
US20150249676A1 (en) * | 2014-02-28 | 2015-09-03 | Fujitsu Limited | Monitoring method and monitoring apparatus |
US9516050B2 (en) * | 2014-02-28 | 2016-12-06 | Fujitsu Limited | Monitoring propagation in a network |
Also Published As
Publication number | Publication date |
---|---|
US20140181978A1 (en) | 2014-06-26 |
US9069962B2 (en) | 2015-06-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9069962B2 (en) | Evaluation of a fast and robust worm detection algorithm | |
KR101377014B1 (en) | System and Method of Malware Diagnosis Mechanism Based on Immune Database | |
US7712134B1 (en) | Method and apparatus for worm detection and containment in the internet core | |
CN106330944B (en) | Malicious system vulnerability scanner identification method and device | |
CN109587179A (en) | A kind of SSH agreement behavior pattern recognition and alarm method based on bypass network full flow | |
Fu et al. | On recognizing virtual honeypots and countermeasures | |
JP3957712B2 (en) | Communication monitoring system | |
JP6869100B2 (en) | Information processing device, fraudulent activity classification method and fraudulent activity classification program | |
US20100150008A1 (en) | Apparatus and method for displaying state of network | |
CN114021135A (en) | LDoS attack detection and defense method based on R-SAX | |
KR100950079B1 (en) | Network abnormal state detection device using HMMHidden Markov Model and Method thereof | |
JP2002007234A (en) | Detection device, countermeasure system, detecting method, and countermeasure method for illegal message, and computer-readable recording medium | |
CN110061998B (en) | Attack defense method and device | |
Lee et al. | Multi-stage intrusion detection system using hidden markov model algorithm | |
Kumar et al. | Intrusion detection system-false positive alert reduction technique | |
CN109120600A (en) | A kind of LDoS rapid detection method based on flow frequency disribution feature | |
US7908657B1 (en) | Detecting variants of known threats | |
Choi et al. | PCAV: Internet attack visualization on parallel coordinates | |
JP2010250607A (en) | System, method and program for analysis of unauthorized access | |
JP2007074339A (en) | Spread unauthorized access detection method and system | |
KR100803029B1 (en) | Method for cooperatively defending of ddos attack using statistical detection | |
CN111654499A (en) | Method and device for identifying attack breach based on protocol stack | |
Bu et al. | Design and evaluation of a fast and robust worm detection algorithm | |
KR20180101868A (en) | Apparatus and method for detecting of suspected malignant information | |
US11184369B2 (en) | Malicious relay and jump-system detection using behavioral indicators of actors |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: LUCENT TECHNOLOGIES INC., NEW JERSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BU, TIAN;CHEN, AIYOU;WIEL, SCOTT ALAN VANDER;AND OTHERS;REEL/FRAME:017926/0167;SIGNING DATES FROM 20060626 TO 20060706 |
|
AS | Assignment |
Owner name: CREDIT SUISSE AG, NEW YORK Free format text: SECURITY INTEREST;ASSIGNOR:ALCATEL-LUCENT USA INC.;REEL/FRAME:030510/0627 Effective date: 20130130 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |
|
AS | Assignment |
Owner name: ALCATEL-LUCENT USA INC., NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG;REEL/FRAME:033949/0016 Effective date: 20140819 |
|
AS | Assignment |
Owner name: ALCATEL-LUCENT USA INC., NEW JERSEY Free format text: MERGER AND CHANGE OF NAME;ASSIGNORS:LUCENT TECHNOLOGIES INC.;ALCATEL USA MARKETING, INC.;ALCATEL USA SOURCING, INC.;AND OTHERS;REEL/FRAME:035881/0156 Effective date: 20081101 |
|
AS | Assignment |
Owner name: ALCATEL LUCENT, FRANCE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ALCATEL-LUCENT USA INC.;REEL/FRAME:035938/0300 Effective date: 20150625 |