KR20080044145A - Anomaly detection system and method of web application attacks using web log correlation - Google Patents

Abstract

A system and a method for detecting web application attacks by using a web log correlation analysis are provided to detect unknown attacks according as determining whether an access to a specific web page is normal or not, and to reduce false positive rate. A system for detecting web application attacks includes a web log preprocessor(110), a web log database(120), a web log correlation unit(130), and a correlation analysis information database(133). The web log preprocessor converts log information stored at a web log into formatted web log, constructs the formatted web log in a formatted log and stores the formatted web log at the web log database. The formatted web log stored at the web log database is transferred to a web log correlation unit. The web log correlation unit, including a connection state analysis unit(131) and a similarity measurement unit(132), calls the formatted web log, analyzes the formatted web log, and detects an intrusion with respect to a client request by using the analysis result. The connection state analysis unit manages site topology information, namely detects an abnormal access to a site by maintaining information on a state of connection between web pages. The connection state analysis unit determines an abnormal access or not by using a status code of the formatted web log in the formatted log table. Then, the connection state analysis unit stores a query string of a user to the correlation analysis information database in case of determining that an access to the web site is abnormal. The similarity measurement unit checks whether there exists string similar to query string of an attacker at the correlation analysis information database by calculating similarity of the query string of the attacker on the basis of the Ratcliff pattern matching algorithm.

Description

Anomaly Detection System and Method of Web Application Attacks using Web Log Correlation}

1 is a block diagram showing an abnormal behavior detection system of a web application attack using web log correlation analysis;

2 shows an example of a similarity comparison mechanism performed by a similarity measurer in a web log correlator.

3 is an explanatory diagram of a user accessing a web page, (a) shows sequential access to a specific page, (b) shows direct access to a specific page,

4 is a flowchart illustrating an operation process of an abnormal behavior detection system of a web application attack using web log correlation analysis according to the present invention.

<Description of Signs for Major Components of Drawings>

100. Anomaly Detection System of Web Application Attack

110. Web log preprocessor 120. Standardized web log DB (database)

130. Web Log Correlator 131. Connection Status Analyzer

132. Similarity Meter 133. Correlation Analysis DB

The present invention relates to a system and method for detecting abnormal behavior of a web-based application attack using correlation analysis of web logs, and more particularly, by constructing site topology information to determine whether or not access to a specific web page is normal. The present invention relates to a system and method for detecting file access that does not belong to a site topology, and detecting abnormal requests of a user by using similarity of strings in a web log.

Today, the Web is recognized as an important infrastructure that has a major impact on the major activities and competitiveness of businesses, schools, and institutions. With the increase of web usage, the incidence of web applications is increasing rapidly, and the attack techniques of web applications are getting more complicated and new.

In order to prevent such a breach, various web application security measures are applied. Web application intrusion detection system is one of such security measures, and prevents accidents by detecting web security threats in advance.

In the web application intrusion detection system approach, the misuse detection technique that detects a specific signature pattern is mainly used like the traditional network-based intrusion detection technique.

Misuse detection technology is not suitable for use in web application attack threats for two reasons: First, misuse detection has technical limitations. Only known attacks and fixed patterns can be detected, and techniques exist to circumvent misuse detection. And encrypted attacks cannot be detected. Second, there are limitations in management operation. Most of the attack signatures available on common domains are inappropriate for web servers and web applications that you want to protect. Intrusion detection systems that do not consider the environment of web servers and web applications generate many intrusion determination errors.

In order to overcome this limitation of misuse detection technology, misuse detection technology needs to be fused with abnormal behavior detection technology.

The present invention is to solve the problem of the misuse detection technology as described above, the object of the present invention is to apply a web log correlation analysis on the web log that remains the attacker's trial and error information even if a new unknown attack occurs It provides a system and method for detecting abnormal behavior of web-based application attacks.

Therefore, in order to achieve the above object of the present invention, the abnormal behavior detection system of web application attack using the web log correlation analysis of the present invention uses Ratcliff's pattern matching algorithm to determine whether or not access to a specific web page is normal. By comparing the similarity between the query string stored in the web log and the request query string to detect abnormal request behavior.

Meanwhile, the method for detecting abnormal behavior of a web application attack using web log correlation analysis according to the present invention includes (a) performing web log correlation analysis on a web log and a request question string for a web page access request. Comparing the similarities, and (b) determining normal access and abnormal access according to the presence or absence of a similar question string.

In the system and method for detecting abnormal behavior of web application attacks using the web log correlation analysis proposed in the present invention, the trace of trial and error that occurs because the attacker does not know the environment of the target system is used as a clue for detecting the abnormal behavior.

In addition, the present invention analyzes the logs of general web servers such as Apache and IIS to detect abnormal request behaviors, and also focuses on attacks using web application vulnerabilities rather than attacks using the web server's own vulnerabilities. .

Hereinafter, a system for detecting abnormal behavior of a web application attack using web log correlation analysis according to the present invention will be described in detail with reference to the accompanying drawings.

Figure 1 shows the overall configuration of the abnormal behavior detection system of web application attack using web log correlation analysis according to the present invention.

The abnormal behavior detection system 100 of the web application attack according to the present invention uses the fact that web attackers do not know the environmental factors of the target computer, so that the attack must be attempted several times in order to succeed. If there is a web page request with high similarity to the web page request that has been abnormally processed several times on the remaining status code, it is determined as an attack.

That is, the abnormal behavior detection system 100 of the web application attack of the present invention uses the web log correlation analysis made by comparing the similarity between web page requests.

Referring to FIG. 1, when a user 10 accesses an arbitrary site, the web application 30 of the site may log a log generated after processing the request of the user 10 on the web server 40. 50).

Then, the abnormal behavior detection system 100 of the web application attack according to the present invention detects the abnormal request behavior of the user using log information stored in the web log 50.

Specifically, the abnormal behavior detection system 100 of a web application attack includes a web log preprocessor 110, a formal web log DB (database) 120, a web log correlator 130, and a correlation analysis information DB ( 133, the web log correlator 130 includes a connection state analyzer 131 and a similarity measurer 132.

The web log preprocessor 110 converts the log information stored in the web log 50 into a formatted web log. Then, the structured web log is organized into a structured log table as shown in the following [Table] and stored in the structured web log DB (120). After that, the structured web log stored in the structured web log DB 120 is transmitted to the web log correlator 130 for analysis.

[Table] Formatted Log Table

The web log correlator 130 is composed of a connection state analyzer 131 and a similarity measurer 132. The web log correlator 130 detects an invasion of a client request with the analyzed result by calling and analyzing a standardized web log.

The connection state analyzer 131 manages site topology information. That is, it detects abnormal access of the site by maintaining information about the connection relations between web pages. Also, in the standardized log table of [Table], the presence or absence of abnormal access is determined using the status code of the standardized web log.

That is, when the user transmits a question string to the page in the web site, the connection state analyzer 131 determines that the access code is normal when the status code is 2xx, and the abnormal access when the status code is 4xx and 5xx instead of 2xx.

Then, the request information (question string) of the user when it is determined that the abnormal access to the web site is stored in the correlation analysis information database 133.

The similarity measurer 132 attacks the target site as the attacker changes the question string little by little within the overall attack flow, so that Ratcliff's pattern matching algorithm (J., W.Ratcliff and D.Metzener's " Pattern Matching: The Gestalt Approach ", Dr. Dobb's Journal, July 1988.) to determine the similarity of question strings to check if a string similar to the attacker's question string exists in the correlation information database (133).

That is, the similarity check is performed between the question string requested by the attacker and the question string stored in the correlation analysis information database 133 to detect abnormal site access.

If a similar string is found, the attacker's access to the web site is regarded as an abnormal request behavior, and the request information (question string), the log information, and all logs of the IP for the attacker's request are correlated. The information is stored in the database 133.

Alternatively, even if a similar string does not exist in the correlation information database 133, if the status code of the requesting web log is not 2xx, it is regarded as an abnormal request, and log information about the request and all logs of the corresponding IP are related to the correlation information database. Save to (133).

The stored web log correlation analysis information is then used as an element to determine whether the user request is normal or abnormal.

FIG. 2 illustrates an example of a similarity comparison mechanism performed by the similarity measurer 132 in the web log correlator 130, in which an attacker accesses a web site using a sequential approach. And the sequential approach is demonstrated in FIG.

First, while an attacker is attempting to attack a web site, the web log contains traces of information gathering activities by the attacker, trial and error caused by environmental factors of various websites, and successes by attacking vulnerabilities. Will remain.

The important point here is that because the attacker does not know the environment of the web server and web application, several attack attempts are repeated in order for the attack to succeed and break into the web site.

In other words, if an attacker recently attacks a web site using an automated tool, an attack using an automated attack tool designed for a specific environment requires several trial and error. And even without the use of automated attack tools, this trial and error occurs.

This trial and error before the attacker succeeds, but even after a new unknown attacker succeeds, traces of the attacker's trial and error remain in the web log, and the information left in this web log can detect the attacker's intrusion. This is important information.

Referring to FIG. 2, an attacker performs a normal web page request until the vulnerable web page is accessed, such as requests 1, 2, and 3. When an attacker accesses a vulnerable web page, it sends an abnormal string to the target web page, as in request # 4.

Since the attacker does not know the target web server and web application environment, the attack on the vulnerable web page such as request 4 will fail. At this time, the attacker will attack again by modifying the question string, and the attack attempt will be repeated until the attack is successful as in 4-2 request.

Since the 4-2 and 4-3 requests are successful attacks, the connection state analyzer 131 determines that the access is normal. However, there is a high similarity between the question string requested by the attacker in previous requests 4 and 4-1 and the question string requested by the attacker in requests 4-2 and 4-3. By checking similarity, abnormal access detection is possible.

Because the attacker attacks the target site by changing the question string little by little within the whole attack flow, even if the HTTP status code appears to be normal access, it can be judged as an abnormal string by checking the similarity of the question string.

In the present invention, Ratcliff's pattern matching algorithm is used for similarity measurement, and similarity checking algorithms in the field of bioinformatics may be used.

For reference, referring to a user's web page access process, FIG. 3 is an explanatory diagram of a user's web page access method. The web page access method is divided into a sequential access to a specific page and a direct access to a specific page.

3 (a) shows a sequential access process of sequentially accessing K pages along a link from a main screen to access K pages (specific pages). That is, sequential access to a specific page is sequentially accessed through pages between the main screen of the site and the specific page when the user requests a specific page.

3 (b) shows direct access to a K page (specific page). In other words, when a user requests a specific page, the direct access to the specific page is to access the specific page instead of sequentially accessing the pages between the main screen and the specific page of the site.

Next, an intrusion detection method for detecting an abnormal request behavior by the operation of the abnormal behavior detection system 100 of the web application attack described above will be described with reference to the accompanying drawings.

4 is a flowchart illustrating an operation process of an abnormal behavior detection system 100 of a web application attack using web log correlation analysis according to the present invention.

The web log preprocessor 110 converts the log information stored in the web log 50 into a standardized web log and stores the converted log information (S400).

Then, if there is a request for an attacker's specific website, the web log correlator 130 compares the formatted question log with the request question string stored in the correlation information DB 133 and checks the status code. (S401).

In detail, the similarity measurer 132 checks whether a similar question string exists through web log correlation analysis (S402). As a result of the check, if a similar question string exists (YES), it is determined as an abnormal request behavior, and log information about the request and all logs of the corresponding IP are stored in the correlation analysis information database 133 (S405).

On the other hand, if a similar question string does not exist (NO), the connection state analyzer 131 checks whether the status code of the request question string is 2xx (S403). If the status code is 2xx (YES), it is determined to be a normal request behavior (S404), and if the status code is not 2xx (NO), it is determined to be an abnormal request behavior and correlates log information about the request and all logs of the corresponding IP. The data is stored in the database 133 (S405).

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

As described above, the system and method for detecting abnormal behavior of a web application attack using the web log correlation analysis according to the present invention uses an unknown web log by using the web log in which the attacker's trial and error information remains even if a new unknown attack occurs. Attacks can be detected and the false positive rate can be lower than the comparison information generated by learning of the existing intrusion detection system because the comparison information is generated from the time when the attacker starts the web application attack. have.

Claims (11)

A web log preprocessor for converting log information stored in the web log into a standardized web log; A structured web log database for storing the structured web log; And Web log correlator for determining abnormal behavior for the user's request through web log correlation analysis that checks the status code of the web log and the similarity of the user's request question string Abnormal behavior detection system of a web application attack using web log correlation analysis comprising a. The web log correlator of claim 1, A connection state analyzer for determining abnormal behavior for a user's request by using a status code of the web log; And Web log correlation using ratcliff's pattern matching algorithm is performed to determine similarity between the request question string of the standardized web log and the question string stored in the correlation analysis information database to determine abnormal behavior for the user's request. Similarity Meter Abnormal behavior detection system of a web application attack using web log correlation analysis comprising a. The web log according to claim 2, wherein the system further comprises a correlation analysis information database for storing the user's request information when the determination result of the web log correlator and the abnormal behavior are determined. An Anomaly Detection System of Web Application Attack Using Correlation Analysis. The abnormality detection of a web application attack using web log correlation analysis according to claim 2, wherein the query string used for the similarity check in the similarity measurer is a variable length string that does not have a specific pattern. system. The method of claim 2, The connection state analyzer determines that the normal web log is a normal access when the status code is 2xx, and an abnormal access when the status code is 4xx and 5xx instead of 2xx. Anomaly Detection System. The method of claim 5, If it is determined that the abnormal access, the request information of the user is stored in the correlation analysis information database, characterized in that the abnormal behavior detection system for web application attack using the web log correlation analysis. (a) comparing the similarity with the request question string stored in the correlation analysis information DB by performing web log correlation analysis on the request question string of the web log formatted for the access request to a specific web page; And (b) determining normal access and abnormal access according to the presence or absence of a similar character string as a result of the comparison; Method for detecting abnormal behavior of web application attacks using web log correlation analysis, characterized in that configured to include. The web using the web log correlation analysis according to claim 7, wherein in step (b), if a similar character string is present, it is determined as abnormal access and the request information of the user is stored in a correlation analysis information database. How to detect anomalous behavior in application attacks. The web application attack according to claim 8, wherein in step (b), if the similar string does not exist, the status code of the request question string of the standard web log is checked to determine normal access if 2xx. How to detect abnormal behaviors. 10. The method of claim 9, wherein in step (b), if the similar string does not exist and the status code is not 2xx, it is determined that the access is abnormal and the request information of the user is stored in the correlation analysis information database. A method for detecting abnormal behavior of web application attack using web log correlation analysis. The method of claim 7, wherein the method comprises converting and storing the information stored in the web log into a formal web log after the access request for a specific web page, before the step (a). A method for detecting abnormal behavior of web application attack using web log correlation analysis.

Images