KR20190062115A - ICAP protocol extension method for providing network forensic service of encrypted traffic, network forensic device supporting it and web proxy - Google Patents

Abstract

Disclosed are an Internet contents adaptation protocol (ICAP) extension method for providing a network forensic service of encrypted traffic, a network forensic device supporting the same, and a web proxy. According to an embodiment of the present invention, a network forensic device comprises: an ICAP server for receiving an ICAP message including decrypted hypertext transfer protocol (HTTP) traffic from an ICAP client of the web proxy through an ICAP and extracting the HTTP traffic from the received ICAP message; and an HTTP traffic storage device for storing the HTTP traffic extracted in the ICAP server.

Description

Technical Field [0001] The present invention relates to an ICAP protocol extension method for providing a network forensic service of encryption traffic, a network forensic device and a web proxy for supporting the ICAP protocol extension method and a network forensic service supporting the Internet protocol,

The present invention relates to network traffic management techniques.

Encryption is the process of processing information so that it can not be seen by others for the sake of information security. This is used to ensure that information is not leaked (meaning to encrypt) when information is exchanged through the network, increasing the importance of information, and changing other people irrelevant to it to read or not to view it. On the other hand, as opposed to encryption, the task of finding out the original information with encrypted data is called decryption or decryption.

According to an exemplary embodiment, an ICAP protocol extension method for providing a network forensic service of encrypted traffic, a network forensic device supporting the same, and a web proxy are proposed.

The network forensic device according to an embodiment includes an ICAP server for receiving an ICAP message including HTTP traffic decrypted from an ICAP client of a web proxy through an ICAP protocol and extracting HTTP traffic from the received ICAP message, Lt; RTI ID = 0.0 > HTTP < / RTI >

The ICAP server can receive an ICAP RESPMOD request message including an HTTP response message decoded from the ICAP client of the web proxy through the ICAP protocol, extract the HTTP response message from the received ICAP RESPMOD request message, and store the extracted HTTP response message in the HTTP traffic storage device . The ICAP server may receive an ICAP REQMOD request message including an HTTP request message decrypted from a web proxy that is an ICAP client through the ICAP protocol, extract an HTTP request message from the received ICAP REQMOD request message, and store the extracted HTTP request message in the HTTP traffic storage device .

The ICAP server includes an ICAP thread for extracting HTTP traffic transmitted and received between the web client and the web server from the ICAP RESPMOD request message received from the web proxy and storing the extracted HTTP traffic in the HTTP traffic storage unit for each data object, An aging thread for managing aging time information for each hash table entry to prevent duplication of the aging thread, and a swap thread for swapping a data object and a hash table stored up to that time based on a predetermined time window section.

The ICAP thread receives the ICAP RESPMOD request message from the ICAP client of the web proxy, extracts the data object from the received ICAP RESPMOD request message, calculates the fingerprint of the extracted data object, and extracts the data object identifier (URL information) A hash key is generated, and a hash table of the memory can be searched using the generated hash key as an index. The ICAP thread can generate a hash key by reflecting an arbitrary hash function (H) on the extracted data object identifier (URL information).

The swap thread has a 1: 1 association with the data object storage file to directly index each data object stored in the data object storage file, and a data object storage file in which all the data objects received in the swap time window section are stored Save the local hash table You can save the file. The data object storage file and the local hash table storage file are stored in a timeline directory in a tree form according to a predetermined unit time period based on the storage time and stored in the corresponding time zone by inputting a search condition including a time condition through a timeline directory configuration It is possible to provide a search function of a specific data object.

The web proxy according to another embodiment includes an HTTPS decryption unit for extracting HTTP traffic by decrypting HTTPS traffic received from a web client or a web server, and an HTTPS decryption unit for receiving HTTP traffic from the HTTPS decryption unit and delivering the HTTP traffic to the network forensic device Gt; ICAP < / RTI >

The HTTPS decryption unit can extract plaintext HTTP traffic from the HTTPS traffic by hooking a function of the encryption library.

The HTTPS decryption unit decrypts the HTTPS request message received from the web client, extracts the HTTP request message from the HTTPS request message, and transmits the extracted HTTP request message to the ICAP client. The HTTPS decryption unit decrypts the HTTPS response message received from the web server, And a second HTTPS decryption unit for extracting an HTTP response message from the response message and transmitting the extracted HTTP response message to the ICAP client.

The ICAP client includes an upper ICAP client that receives an HTTP request message from the HTTPS decryption unit and transmits an ICAP REQMOD request message including an HTTP request message to the network forensic device, and an upper ICAP client that receives the HTTP response message from the HTTPS decryption unit, And a downward ICAP client sending an ICAP RESPMOD request message including an HTTP response message.

The web proxy may further include an HTTP traffic processing unit for separately storing the decrypted HTTP traffic in the content cache.

According to another embodiment of the present invention, an ICAP protocol extension method for providing a network forensic service of HTTPS traffic includes receiving an encrypted HTTPS request message from a web client using a HTTPS protocol, Extracting an HTTP request message from an HTTPS request message, transmitting an ICAP REQMOD request message including an extracted HTTP request message to a network forensic device using an ICAP protocol, And storing the HTTP request message included in the REQMOD message.

An ICAP protocol extension method for providing a network forensic service of HTTPS traffic includes the steps of transmitting an ICAP response message to a web proxy using an ICAP protocol after the network forensic device stores an HTTP request message, Encrypting the HTTP request message with the HTTPS request message and transmitting the HTTPS request message to the web server, receiving an HTTPS response message from the web server in response to the HTTPS request message, decrypting the received HTTPS response message, Extracting an HTTP response message from the message and transmitting an ICAP RESPMOD request message including the HTTP response message and the HTTP request message to the network forensic device using the ICAP protocol; Response messages and HTTP request messages Chapter stage that can be further included.

An ICAP protocol extension method for providing a network forensic service of HTTPS traffic includes the steps of a web proxy receiving an ICAP response message from a network forensic device using an ICAP protocol in response to an ICAP RESPMOD request message, Encrypting the HTTP response message included in the response message with an HTTPS response message, and transmitting the HTTP response message to the web client.

A network forensic device according to an embodiment includes a plaintext deciphered from the ICAP client of the web proxy at the location of the ICAP server based on the ICAP protocol extension for forensic analysis on encrypted traffic (for example, HTTPS traffic) Of HTTP traffic can be collected and stored. Accordingly, it is not necessary to separately extract the HTTPS encryption key to decrypt the HTTPS traffic to be analyzed for network forensic analysis, thereby solving the problem of security vulnerability. Furthermore, since the decrypted HTTP data itself is stored, it is necessary to perform a preprocessing step (for example, encryption key extraction, flow generation time extraction, flow identification information extraction, The flow extraction, decoding and visualization step by reassembly) can be omitted. Furthermore, network forensic devices can interoperate with third party web proxies and web security devices that support the ICAP protocol, thereby achieving the advantages of an open system.

1 is a block diagram illustrating a configuration of a web proxy according to an exemplary embodiment of the present invention;
FIG. 2 illustrates a network forensic device and a web proxy according to an exemplary embodiment of the present invention. FIG.
FIG. 3A is a diagram illustrating a structure of a Web proxy for explaining a message transmission operation through an ICAP protocol in a web proxy supporting an ICAP protocol to facilitate understanding of the present invention, and FIGS. Fig.
4A is a diagram illustrating a structure of a Web proxy for explaining a message transmission operation of a Web proxy when the ICAP server responds with a 3. ICAP response message for the 2. ICAP REQMOD request in the web page request step of FIG. FIGS. 4B to 4E illustrate transmission message structures at this time,
5 is a diagram illustrating a configuration of a network forensic tool for facilitating understanding of the present invention.
6 illustrates a process of collecting and processing packets to facilitate understanding of the present invention,
7 is a diagram illustrating a traffic collection, retrieval, and analysis process for network forensics to facilitate understanding of the present invention.
FIGS. 8 and 9 are diagrams showing a configuration of a Web proxy to introduce various methods for providing a network forensic function for HTTPS traffic;
10 is a diagram illustrating a configuration of a web proxy for providing a network forensic function for HTTPS traffic according to an exemplary embodiment of the present invention;
11 is a configuration diagram of a network forensic device using an HTTPS encryption key,
FIG. 12 illustrates a network system for network forensics according to an embodiment of the present invention. FIG.
13 is a diagram illustrating a protocol used between components of a network system according to an embodiment of the present invention;
FIG. 14 illustrates a process of a Web proxy for decrypting HTTPS traffic according to an embodiment of the present invention; FIG.
FIG. 15 illustrates a function for hooking an OpenSSL library when a web proxy according to an embodiment of the present invention decodes HTTPS traffic; FIG.
16 is a diagram illustrating a data transmission process between network elements for explaining an ICAP protocol extension method for providing a network forensic service of HTTPS traffic according to an embodiment of the present invention;
17 is a diagram illustrating a software function configuration of a network forensic apparatus according to an embodiment of the present invention.
FIG. 18 illustrates a structure of the hash table of FIG. 17 according to an embodiment of the present invention; FIG.
FIG. 19 is a diagram illustrating a structure of a data object storage file and a local hash table storage file of FIG. 17 according to an embodiment of the present invention; FIG.
20 is a diagram for explaining a search function according to an embodiment of the present invention;
FIG. 21 and FIG. 22 are diagrams illustrating an operation process of the ICAP server thread of FIG. 17 according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. , Which may vary depending on the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification.

Each block of the accompanying block diagrams and combinations of steps of the flowcharts may be performed by computer program instructions (execution engines), which may be stored in a general-purpose computer, special purpose computer, or other processor of a programmable data processing apparatus The instructions that are executed through the processor of the computer or other programmable data processing equipment will generate means for performing the functions described in each block or flowchart of the block diagram.

These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible for the instructions stored in the block diagram to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of the flowchart.

And computer program instructions may be loaded onto a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the data processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing the specified logical functions, and in some alternative embodiments, It should be noted that functions may occur out of order. For example, two successive blocks or steps may actually be performed substantially concurrently, and it is also possible that the blocks or steps are performed in the reverse order of the function as needed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to enable those skilled in the art to more fully understand the present invention.

FIG. 1 is a diagram illustrating a configuration of a web proxy according to an embodiment of the present invention.

1, the web proxy 10 includes a first HTTPS decryption unit 121, a second HTTPS decryption unit 122, an HTTP traffic processing unit 130, a first HTTPS encryption unit 141, (142).

The web proxy 10 according to one embodiment provides a proxy function for encrypted traffic (for example, HTTPS traffic) or decrypted traffic (for example, HTTP traffic) distributed between an internal company network and the Internet do. Proxy is used by proxy and proxy for purposes such as firewall and web cache. When a user's web client specifies a proxy in a web browser, the URL requested from the web client is not directly connected to the web server but is connected via a proxy. At this time, the proxy receiving the request from the web client connects with the corresponding web server of the URL, sends the request, receives the response instead of the web client, and returns it to the web client. In this way, since the session connection between the web client and the web server goes through the intermediate proxy, the session connection between the web client, the proxy, the proxy and the web server, that is, two session connections is established. In a similar manner, the web proxy 10 performs proxy functions between a web client and a web server, as shown in Fig. 1, and generates a self-generated secret Key in a transparent proxy mode that acts as a web server for the web client.

The web proxy 10 according to one embodiment is located in a man-in-the-middle form between a web client and a web server. The web proxy 10 relays traffic (e.g., HTTPS traffic) encrypted with an encryption protocol (e.g., SSL / TLS protocol) between the web client 12 and the web server 14. The web proxy 10 decrypts the encrypted HTTPS traffic into HTTP traffic, extracts the content, caches the content in the contents cache 150, re-encrypts the HTTP traffic, and retransmits the HTTP traffic to the first destination web server do.

The web proxy 10 according to one embodiment includes an HTTPS traffic decryption function of the HTTPS decryption units 121 and 122 performed for caching purposes, an HTTPS traffic encryption function of the HTTPS encryption units 141 and 142, Extracts the HTTP traffic changed to the plain text of the content cache 130 and stores the HTTP traffic separately in the content cache 150. The HTTP traffic processing unit 130 separately stores plain text HTTP traffic obtained by decoding the HTTPS traffic received from the web client 12 or the web server 14 by the web proxy 10 in the content cache 150.

According to one embodiment, the encrypted traffic is HTTPS traffic, the decrypted traffic is HTTP traffic, and the web proxy 10 is an HTTPS proxy for processing HTTPS traffic. In the following description, the encrypted traffic is defined as HTTPS traffic, and the decrypted traffic is defined as HTTP traffic. However, the present invention is not limited thereto.

2 is a diagram illustrating a configuration of a network forensic apparatus and a web proxy according to an embodiment of the present invention.

Referring to FIG. 2, the network forensic apparatus 20 according to an exemplary embodiment collects and stores encryption / decryption traffic distributed between an internal company network and the Internet to provide a network forensics function. The network forensic device 20 collects and stores the HTTP traffic decrypted in the web proxy 10 in cooperation with the web proxy 10 for network forensic analysis. At this time, the network forensic device 20 is interworked with the web proxy 10 through an Internet contents adaptation protocol (ICAP). The network forensic device 20 is located externally from the web proxy 10 and includes an ICAP server 200 and an HTTP traffic storage device 210. The ICAP server 200 is connected to the ICAP client 100 of the web proxy 10 via the ICAP protocol and receives HTTP traffic from the web proxy 10 and processes the HTTP traffic to store the HTTP traffic in the HTTP traffic storage device 210. At this time, the stored HTTP traffic is the plain-text HTTP traffic decrypted by the web proxy 10 from the web client or the web server.

Since the network forensic device 20 according to the embodiment relays the plaintext form HTTP traffic decrypted from the web proxy 10 as described above, the network forensic device 20 transmits the HTTPS cryptographic key in the web proxy 10 in order to decrypt the HTTPS traffic It is not necessary to extract and store it as a separate file. Therefore, the problem of security vulnerability can be solved. In addition, preprocessing steps (e.g., flow tracking and reassembly from packets) that must be preceded to extract the HTTPS traffic to decrypt for forensic analysis may be omitted.

The network forensic apparatus 20 according to an exemplary embodiment can interoperate with a third party web proxy 10 supporting a ICAP protocol that is an open communication protocol, a web security apparatus, and the like, thereby obtaining an advantage as an open system . Web security appliances include, for example, web firewalls, DLPs, and the like.

FIG. 3A is a diagram illustrating a structure of a Web proxy for explaining a message transmission operation through an ICAP protocol in a web proxy supporting an ICAP protocol to facilitate understanding of the present invention, and FIGS. Fig.

3A to 3F, the web proxy 10 operates as an HTTP proxy for processing HTTP traffic transmitted between the web client 12 and the web server 14. The Internet contents adaptation protocol (ICAP) protocol is described in IETF 3507 as a protocol similar to hypertext transfer protocol (HTTP). ICAP defines a method for adding or extending a new function to a web proxy server operating in a transparent mode. By defining the method, the scalability, re-usability, and interoperability of the open system are improved. And the like. Currently, the ICAP protocol is used as a standard protocol for interworking with an external server that performs functions such as virus scanning, content filtering, and content modification / modification such as insertion / removal of banner advertisements in a caching apparatus that operates in a transparent web proxy mode . Recently, various web proxy products support ICAP protocol by default. Accordingly, the web security solution also supports the ICAP protocol for interworking with the web proxy.

3A, the ICAP client 100 of the web proxy 10 receives a 1. HTTP request (hereinafter referred to as an HTTP request) from the web client 12, 5. HTTP response (hereinafter referred to as HTTP reply) message from the web server 14 or a 2. ICAP request modification (REQMOD) defined in the ICAP protocol, (ICAP REQMOD request) or 6. ICAP response modification (RESPMOD) request message (hereinafter, referred to as ICAP RESPMOD request) message to the ICAP server 22 . Thereafter, the ICAP server 22 transmits a 7. ICAP reply (hereinafter referred to as ICAP reply) message to the ICAP client 100 according to the processing result of the 6. ICAP RESPMOD request message. The web proxy 10 supporting the ICAP client function performs the function of processing the HTTP message contained in the ICAP reply message according to the protocol defined in IETF 3507. [

3A to 3G show a processing procedure and a transmission message structure of the web proxy 10 when the ICAP server 22 responds to the ICAP reply message for the 6. ICAP RESPMOD request message in the web page response step . For example, if the web client 12 requests a web page www.unknow-server.com, the web browser of the web client 12 sends an HTTP GET message (Fig. 3B) to the web proxy 10. The web proxy 10 transmits an HTTP GET message (Fig. 3B) to the ICAP server 22 in an ICAP REQMOD request message (Fig. 3C). ICAP server 22 transmits an ICAP reply message (FIG. 3D) to web proxy 10 when requesting an authorized web page. When the web proxy 10 receives an HTTP request message contained in the ICAP reply message (FIG. 3D) from the ICAP server 22 according to the ICAP protocol, the web proxy 10 transmits an HTTP GET message to the web server (www.origin-server.com) Forward. In this case, the HTTP request message may be modified by the ICAP server 22. The web proxy 10 receives the HTTP reply message from the web server 14 and transmits it to the ICAP server 22 in an ICAP RESPMOD request message (Fig. 3E). In this case, in the RESPMOD request message (FIG. 3E), the HTTP request message is also transmitted together with the HTTP reply message. The ICAP server 22 transmits an ICAP reply message (FIG. 3F) to the Web proxy 10 according to the content filtering result in the HTTP reply message displayed in the ICAP RESPMOD request message (FIG. 3E). The web proxy 10 transmits an HTTP reply message (Fig. 3G) listed in the ICAP reply message (Fig. 3F) to the web browser of the web client 12. In this case, the HTTP reply message (FIG. 3G) can be changed in accordance with the content filtering result of the ICAP server 22.

4A is a diagram illustrating a structure of a Web proxy for explaining a message transmission operation of a Web proxy when the ICAP server responds with a 3. ICAP response message for the 2. ICAP REQMOD request in the web page request step of FIG. FIGS. 4B to 4E are diagrams illustrating a transmission message structure at this time.

Referring to FIG. 4A, when the web client 12 requests a www.sex.com web page, the web browser of the web client 12 transmits an HTTP GET message (FIG. 4B) to the web proxy 10. The web proxy 10 transmits an HTTP GET message (Fig. 4B) to the ICAP server 22 in an ICAP REQMOD request message (Fig. 4C). ICAP server 22 transmits an ICAP response message (FIG. 4D) to web proxy 10 when requesting an unauthorized web page. When the web proxy 10 receives an HTTP response message (FIG. 4E) for the HTTP request from the ICAP server 22 according to the ICAP protocol, the web proxy 10 transmits the HTTP response message to the web browser of the web client 12.

5 to 7 are diagrams for explaining a network forensic technology for facilitating understanding of the present invention. 5 is a diagram showing the configuration of a network forensic tool, FIG. 6 is a view showing a process of collecting and processing packets, and FIG. 7 is a diagram showing a traffic collection, search and analysis process for network forensic .

Network forensic technology is a technology that can analyze and detect the cause of network security threats by collecting, storing and analyzing all data traffic such as data or password transmitted through the network. In order to analyze the traffic of the encrypted data in this process, a decryption process for the encrypted data is required.

There are two types of data traffic collection methods for network forensics.

1) Analysis method after storage: All the traffic passing through the monitoring point is collected and stored, and it requires storage capacity proportional to the network link speed to be monitored. A forensic function is provided through a post-analysis step, which is typically performed in a batch mode, on a pcap file stored at a packet level.

2) Analysis after storage method: Among the traffic passing through the surveillance point, only the traffic that is expected to need post-analysis is selected and stored. As a result, the processor performance for traffic analysis proportional to the network link speed Is required. This method is a method in which analysis is first performed for storing traffic, and it is a method for improving efficiency of storage capacity and post-analysis through an analysis process such as a packet level, a flow level, and an application level according to the performance of a processor.

As described above, since the network forensic technology must apply the appropriate storage and analysis method according to the type of network attack, various types of network forensic tools are provided to support different features and functions according to vendors. In particular, in the case of the traffic collection method, a post-storage analysis method is preferred for analyzing the cause of a DDoS (Distributed Denial of Service Attack) attack mainly occurring at a packet or a flow level. This is because it is advantageous to find an abnormal symptom of a packet level or a flow level using traffic pattern matching or statistical method for traffic of the entire network stored for a long time without loss.

On the other hand, network forensic tools that are mainly used for APT (Advanced Persistent Threat) attacks, which are performed due to the spread of malicious codes over a long period of time, are more advantageous in storage after analysis. This is because forensic analysis based on malware code analysis is inevitably carried out on content stored at the application level, which is selected and refined as much as possible according to the characteristics of the attack pattern that the APT attack itself is performed at the application level for a long time.

The network forensic tool is installed from the network point of view as shown in FIG. The network connection is directly connected to the network link to be monitored through the SPAN (Switch Port Analyzer) port of the network tap, switch or router. The collected traffic is collected and analyzed as shown in FIG. 6 and finally stored in a storage medium. In addition, for the purpose of improving traffic search performance for forensic analysis, indexing information of packet level, flow level, and application level is extracted and stored during the traffic storing process. These indexing information is used to enable faster traffic analysis and search functions.

Referring to FIG. 7, the traffic collected / analyzed / selected / stored (700) by the method described above with reference to FIG. 6 is subjected to post-analysis through time condition or filtering rule application as shown in FIG. Only the traffic to be the target is selected (710). Then, finally, a direct cause analysis of the security threat is performed (720). Of course, the traffic screening 710 and the subsequent analysis process 720 are repeated until the final cause analysis of the forensic process is completed. If a more efficient network forensic tool is used, . The cause analysis performed during the forensic analysis process 720 can be performed in various forms on a case-by-case basis. For example, when a specific traffic such as a specific host address or a traffic inflow point that is essential in the traffic selection process can not be targeted, a pattern matching or statistical method used for detecting a network attack in a general network security system is used, And analyze the cause of the final network attack through a drill-down type of iterative procedure targeting these traffic.

8 and 9 are diagrams showing the configuration of a Web proxy to introduce various methods for providing a network forensic function for HTTPS traffic.

Referring to FIGS. 8 and 9, the web proxy 10 is an HTTPS proxy in that HTTPS traffic is targeted. There are following methods 1 and 2 to provide the network forensic function for HTTPS traffic.

The method 1 is a method in which the HTTPS decryption unit 120 of the web proxy 10 decrypts the HTTPS traffic and the HTTP traffic processing unit 130 stores the decrypted HTTP traffic as shown in FIG. Method 1 extracts and stores HTTP traffic from the web proxy 10 in a man-in-the-middle form. The forensic analysis is performed in an offline manner using HTTP traffic stored in the web proxy 10.

Method 2 is a method for storing encrypted HTTPS traffic itself as shown in FIG. At this time, an HTTPS traffic storage unit 900 for storing HTTPS traffic and a web proxy 10 for storing an HTTPS encryption key are functionally separated. Method 2 requires an HTTPS cryptographic key to decrypt HTTPS traffic for forensic analysis. Accordingly, the HTTPS encryption key storage unit 920 extracts a cryptographic key used in HTTPS from the same traffic at the same time as storing the traffic. Forensic analysis is performed by detecting the extracted HTTPS encryption key and the stored HTTPS traffic and decrypting the HTTPS traffic using the HTTPS encryption key. Method 2 has a disadvantage in that it is vulnerable to security because the HTTPS encryption key must be extracted and stored separately.

10 is a diagram illustrating a configuration of a web proxy for providing a network forensic function targeting HTTPS traffic according to an embodiment of the present invention.

Referring to FIG. 10, the web proxy 10 according to an embodiment does not extract the HTTPS encryption key separately for network forensation as described above with reference to FIG. 9, and the HTTPS decryption unit 120 does not extract HTTPS traffic The HTTP traffic processing unit 130 extracts and stores an HTTP packet at an intermediate stage between the step of decrypting and the step of encrypting the HTTP traffic again in the HTTPS encryption unit 140. At this time, the network forensic device 20 functionally separated from the web proxy 10 relays the HTTP packet from the web proxy 10 in a manner of interlocking with the web proxy 10 using the ICAP protocol, 210).

11 is a configuration diagram of a network forensic apparatus using an HTTPS encryption key.

Referring to FIG. 11, the network forensic device 20 includes a web proxy 10 and an HTTPS traffic storage device 220. The network forensic device 20 includes a web proxy 10 and is characterized by storing HTTPS traffic in the HTTPS traffic storage device 220. The network forensic device 20 finds the HTTPS encryption key extracted from the web proxy 10 and the corresponding HTTPS traffic stored therein, decrypts the HTTPS traffic using the HTTPS encryption key, and uses it for network forensic analysis. In this case, since the HTTPS encryption key must be extracted and stored separately, there is a drawback that it is vulnerable to security. In addition, when the HTTPS encryption key is used, a preprocessing step (for example, extraction of an encryption key in an encryption key storage file, extraction of a flow generation time in an encrypted traffic indexing file, , Flow extraction and decoding, and visualization step by packet reassembly) should be preceded.

12 is a diagram illustrating a configuration of a network system for network forensics according to an embodiment of the present invention.

12, a network system 1 for network forensics according to an embodiment includes a web proxy 10, a web client 12, a web server 14 and a network forensic device 20, And may further include a security device 30. The web security apparatus 30 is, for example, a web firewall 32, a DLP (Data Loss Prevention) 34, and the like.

The network forensic device 20 operates as an ICAP server, and the web proxy 10 operates as an ICAP client. The network forensic device 20 collects and stores the HTTP traffic decrypted in the web proxy 10 at the location of the ICAP server. The web proxy 10 according to the embodiment extracts the HTTP data and stores it in an intermediate step between decrypting the HTTPS encrypted packet and encrypting the decrypted HTTP packet without separately extracting the HTTPS encryption key. In this case, it is not necessary to separately extract the HTTPS encryption key from the web proxy 10, thereby solving the problem of security vulnerability. Furthermore, since the decrypted HTTP data itself is stored, it is necessary to perform a preprocessing step (for example, encryption key extraction, flow generation time extraction, flow identification information extraction, The flow extraction, decoding and visualization step by reassembly) can be omitted. The network forensic apparatus 20 can interoperate with the third party web proxy 10 and the web security apparatus 30 that support the ICAP protocol and thereby obtain the advantages of the open system.

13 is a diagram illustrating a protocol used between elements of a network system according to an embodiment of the present invention.

Referring to FIG. 13, the web proxy 10 performs a function of relaying HTTPS traffic encrypted by the encryption protocol between the web client 12 and the web server 14 for the purpose of caching. Decrypts the HTTPS traffic, and relays the decrypted HTTPS traffic to the network forensic device 20.

The network forensic device 20 functions to store HTTP traffic relayed by the web proxy 10 for network forensic purposes. The stored HTTP traffic is plain text HTTP traffic obtained by decrypting the HTTPS traffic received from the web client 12 or the web server 14 by the web proxy 10. The network forensic device 20 performs an ICAP server function for ICAP protocol interworking with the web proxy 10 performing the ICAP client function. If the network forensic device 20 is functionally separated from the web proxy 10, the network forensic device 20 may be physically implemented in the same system or separately implemented in a separate system.

The HTTPS protocol is used between the web client 12 and the web proxy 10, and between the web proxy 10 and the web server 14. The ICAP protocol is used between the web proxy 10 and the network forensic device 20. However, ICAP protocol can use SSL encryption protocol for security.

FIG. 14 is a diagram illustrating a process of a Web proxy for decrypting HTTPS traffic according to an exemplary embodiment of the present invention. Referring to FIG.

14, the web proxy 10 is located between the web client 12 and the web server 14, and is located as a web server for the web client 12 and as a web client for the web server 14 Establishes two independent SSL session connections, and decrypts and re-encrypts the encrypted HTTPS traffic between them. To this end, the web proxy 10 includes a proxy server function unit 104 and a proxy client function unit 106. [

The proxy server function unit 104 must successfully authenticate the user of the Web client 12 using the SSL certificate generated by the proxy server function unit 104 in order to establish an SSL session connection with the Web client 12. The SSL protocol is a public-key cryptosystem using asymmetric-key for data encryption / decryption basically.

Between the proxy client function unit 106 and the web server 14, the web server 14 transmits the certificate of the web server 14 including the public key to the proxy client function unit 106 The proxy client function unit 106 extracts the public key of the web server and verifies the web server certificate, generates a session key to be used as an encryption key at the time of data transmission / reception, (14) and transmits it to the web server (14). The web server 14 decrypts and extracts the encrypted session key received from the proxy client function unit 106 by using its own private key and extracts the symmetric encryption key, that is, the session key, with the web client 12 And uses a shared session key for encryption / decryption of data to be transmitted / received thereafter. This series of processes is also performed in the SSL session setting (handshaking) process between the web client 12 and the proxy server function unit 104 in Fig. After the web proxy 10 establishes two independent SSL session connections between the web server 14 and the web client 12, the web proxy 14 can perform HTTPS communication between the web server 14 and the web client 12 have.

The web proxy 10 can decrypt the HTTPS traffic sent between the web client 12 and the web server 14 because the web proxy 10 can know the secret key generated by the web proxy 10 and the session key of the specific SSL session.

The web proxy 10 extracts the HTTP plain text from the HTTPS traffic in the course of performing the SSL protocol proxy function. As a method of implementing this function, a method of decrypting a corresponding encryption packet received using an encryption key extracted during the SSL protocol proxy function, a specific function of an encryption library (eg, OpenSSL, GnuTLS, etc.) (SSL_read or SSL_write function).

15 is a diagram illustrating a function for hooking an OpenSSL library when a web proxy according to an embodiment of the present invention decodes HTTPS traffic.

Referring to FIG. 15, a web proxy uses a method of hooking a specific function of an encryption library for HTTP plain text extraction. For example, the hooking procedure of the OpenSSL library is as shown in Fig.

16 is a diagram illustrating a data transmission process between network elements to explain an ICAP protocol extension method for providing a network forensic service of HTTPS traffic according to an embodiment of the present invention.

16, the web proxy 10 includes an upstream ICAP client 101, a downstream ICAP client 102, a proxy server function 104, and a proxy client function 106. [ The network forensic device 20 includes an ICAP server 200.

The proxy server function unit 104 of the web proxy 10 corresponds to the first HTTPS decryption unit 121 and the first HTTPS encryption unit 141 of Fig. 1, and the proxy client function unit 106 corresponds to the 2 HTTPS encryption unit 142 and the second HTTPS decryption unit 122, respectively. The upper ICAP client 101 and the lower ICAP client 102 are included in the ICAP client 100 of FIG.

The proxy client function unit 106 decrypts the HTTPS traffic received from the web client 12, extracts the HTTP plain text, and transmits the HTTP plain text to the upstream ICAP client 101. The proxy server function unit 104 decrypts the HTTPS traffic received from the web server 14, extracts the HTTP plain text, and transmits the HTTP plain text to the downstream ICAP client 102.

The upstream ICAP client 101 and the downstream ICAP client 102 deliver the HTTP plaintext to the ICAP server 200 of the network forensic device 20. At this time, the ICAP protocol is used for HTTP plain text transmission. The upper direction ICAP client 101 and the lower direction ICAP client 102 perform a function corresponding to the client in the ICAP protocol. The ICAP protocol uses TCP / IP as a transport protocol. The TCP flow of the ICAP protocol is initiated from the ICAP client to the ICAP server using a predefined specific port. In this case, the ICAP server must wait in the listening state in manual mode.

Hereinafter, an ICAP protocol extension method for providing a network forensic service of HTTPS traffic will be described with reference to FIG. The procedure for delivering the HTTP plain text extracted from the proxy client functional unit 106 and the proxy server functional unit 104 to the ICAP server 200 is as follows. In this procedure, it is assumed that the SSL session connection is established between the Web client 12, the proxy server function unit 104, the proxy client function unit 106, and the Web server 14.

1. The Web client 12 sends an HTTPS request message to the Web server 14.

2. The proxy server function unit 104 of the web proxy 10 extracts the HTTP request message from the HTTPS request message through the decryption process and transmits the extracted HTTP request message to the upstream ICAP client 101. [

3. The uplink ICAP client 101 sends an ICAP REQMOD request message to the ICAP server 200. The ICAP REQMOD request message contains an HTTP request message that decrypts the HTTPS request message.

4. The ICAP server 200 processes the HTTP request message stored in the ICAP REQMOD message and then transmits the ICAP reply message to the ICAP client 101 in the upstream direction.

5. The upstream ICAP client 101 delivers the HTTP request message included in the ICAP reply message to the proxy client function unit 106. [

6. The proxy client function unit 106 encrypts the HTTP request message with the HTTPS request message and transmits the encrypted HTTP request message to the web server 14.

7. The Web server 14 sends an HTTPS reply message to the Web client 12 in response to step 6.

8. The proxy client function unit 106 extracts the HTTP reply message from the HTTPS reply message through the decryption process and delivers the HTTP reply message to the downward ICAP client 102. [

9. The downward direction ICAP client 102 transmits an ICAP RESPMOD request message to the ICAP server 200. [ The ICAP RESPMOD request message contains an HTTP reply message decrypting the HTTP request message of section 3 and the HTTPS reply message of section 7.

10. The ICAP server 200 processes the HTTP request message and the HTTP reply message stored in the ICAP RESPMOD message, and then transmits the ICAP reply message to the ICAP client 102 in the downward direction.

11. The downward direction ICAP client 102 delivers the HTTP reply message included in the ICAP reply message to the proxy server function unit 104. [

12. The proxy server function unit 104 encrypts the HTTP reply message with the HTTPS reply message and transmits the HTTP reply message to the web client 12.

The network forensic device 20 according to one embodiment is detached from the web proxy 10 and placed outside, and interlocked with the ICAP protocol between them. Accordingly, the network forensic device 20 performs a function as the ICAP server 200. [ The network forensic device 20 includes a protocol processing function as an ICAP server 200, an ICAP RESPMOD request message processing function, and a function of storing an HTTP message extracted from an ICAP RESPMOD request message.

17 is a diagram illustrating a software function configuration of a network forensic apparatus according to an embodiment of the present invention.

17, a network forensic device 20 according to one embodiment includes a processor 170, a memory 172, a network interface card 174, a storage adapter 176, a disk 178, and a system bus 179 ) In memory 172 on general server hardware. The software function of the network forensic device 20 comprises one or more ICAP server threads 1720, an aging thread 1722, a swap thread 1724, and a hash table 1726.

The ICAP server thread 1720 of FIG. 17 corresponds to the ICAP server 200 of FIGS. 2 and 16, and the disk 178 corresponds to the HTTP traffic storage device 210 of FIGS. 2 and 16 .

The ICAP server thread 1720 extracts the HTTP data traffic transmitted and received between the web client and the web server from the received ICAP RESPMOD request message and stores the extracted HTTP data traffic on the disk 178 for each data object. At this time, the hash table information located in the memory 172 is also stored.

The aging thread 1722 manages the aging time information for each hash table entry in order to prevent the same data object from being duplicated on the disk within the aging time. The setting of the initial value of the aging time information is performed when a new hash table entry is created and then decremented by -1 in a timer interrupt routine that is performed regularly thereafter. When the aging time information of the hash table entry is not 0, the disk storage operation of the received data object is omitted.

The swap thread 1724 performs a function of swapping the hash table and the data object stored up to that time based on a predetermined time window period. That is, in FIG. 17, the local hash table corresponding to the current-t0 period and the data object are finally stored in the form of a file. As a result of the performance of this function, as the time progresses, t0-t1, t1-t2, t2-t3, ... The files having the local hash table and the data object are generated in the order of a series of time windows.

FIG. 18 is a diagram illustrating the structure of the hash table of FIG. 17 according to an embodiment of the present invention.

17 and 18, a hash key 186 for indexing a hash table entry may include an arbitrary hash function H ((H)) in the data object identifier (URL information) 182 extracted from the ICAP RESPMOD request message 180 184). If a specific entry in the hash table is duplicated, it is managed in a list form. The information stored for each hash table entry includes a data object identifier (the same as the URL information used for generating the hash key), a storage location of the data object (file name and offset where the corresponding data object is stored) and size, A fingerprint (MD5) of the corresponding data object, and the like.

FIG. 19 is a diagram illustrating a structure of a data object storage file and a local hash table storage file of FIG. 17 according to an embodiment of the present invention.

Referring to FIGS. 17 and 19, the data object storage file 190 and the local hash table storage file 192, which are created by the swap thread 1724 in units of a swap time interval (variable by a configuration file) The structure is as shown in Fig.

In the data object storage file 190, all the data objects received within the swap time window section are stored. A separate local hash table storage file 192 is stored in order to directly index each data object stored in the data object storage file 190. [ The data object storage file 190 and the local hash table storage file 192 have a 1: 1 association. For example, take the same file name and different file extensions. (Offset) and size (size of data object) information of the corresponding data object are stored for each hash table entry in the local hash table storage file 192. [

20 is a diagram for explaining a search function according to an embodiment of the present invention.

17 and 20, the data object storage file (corresponding to pcap in FIG. 20) stored in the swap thread 1724 and the local hash table storage file (corresponding to the index in FIG. 20) A timeline directory in the form of a tree is created for each time zone and stored in the corresponding directory. Through such a time line directory configuration, a search function of a specific data object stored in a corresponding time zone can be provided by inputting a search condition including a time condition as shown in FIG.

FIG. 21 and FIG. 22 are diagrams illustrating an operation process of the ICAP server thread of FIG. 17 according to an embodiment of the present invention.

17, 21, and 22, the ICAP server thread 1720 receives (2100) an ICAP RESPMOD request message from the ICAP client, extracts a data object from the received ICAP RESPMOD request message, The print MD5 is calculated 2102. Subsequently, a hash key is generated 2104 from the data object identifier (URL), and a hash table of the memory is retrieved using the generated hash key as an index (2106). At this time, it is confirmed whether the new entry is a new entry 2108. If the new entry is a new entry, a hash table entry is newly created 2110 in the memory. If the entry is not a new entry, (2114). In contrast, if the fingerprints match, it is confirmed 2116 whether or not the finger is aging. If the fingerprint is not being aged, the hash table entry is updated 2118 in the memory. At this time, it is terminated if it is in the middle of aging. When a hash table entry is newly created or updated, a new local hash table entry is created on the disk and the data object is stored (2120).

The embodiments of the present invention have been described above. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (16)

An ICAP server that receives an ICAP message including HTTP traffic decrypted from an ICAP client of the web proxy through an ICAP protocol and extracts HTTP traffic from the received ICAP message; And
An HTTP traffic storage device in which HTTP traffic extracted from the ICAP server is stored;
The network forensic device comprising: The ICAP server according to claim 1, wherein the ICAP server
Receives an ICAP RESPMOD request message including an HTTP response message decrypted from an ICAP client of the web proxy through an ICAP protocol, extracts an HTTP response message from the received ICAP RESPMOD request message, and stores the extracted HTTP response message in the HTTP traffic storage device Network forensic device. The ICAP server according to claim 1, wherein the ICAP server
Receiving an ICAP REQMOD request message including an HTTP request message decrypted from a web proxy which is an ICAP client through an ICAP protocol, extracting an HTTP request message from the received ICAP REQMOD request message, and storing the extracted HTTP request message in the HTTP traffic storage device Network forensic device. The ICAP server according to claim 1, wherein the ICAP server
An ICAP thread for extracting HTTP traffic transmitted and received between the web client and the web server from the ICAP RESPMOD request message received from the web proxy and storing the extracted HTTP traffic in the HTTP traffic storage unit on a data object basis;
An aging thread for managing aging time information for each hash table entry to prevent duplicate storage of the same data object on the disk within the aging time; And
A swap thread for swapping the data object and the hash table stored until then based on a predetermined time window section;
The network forensic device comprising: 5. The method of claim 4, wherein the ICAP thread
Receives an ICAP RESPMOD request message from an ICAP client of the web proxy, extracts a data object from the received ICAP RESPMOD request message, calculates a fingerprint of the extracted data object, generates a hash key from the data object identifier (URL information) And searches the hash table of the memory using the generated hash key as an index. 6. The method of claim 5, wherein the ICAP thread
And generates a hash key by reflecting an arbitrary hash function (H) on the extracted data object identifier (URL information). 5. The method of claim 4, wherein the swap thread
A data object storage file in which all the data objects received within the swap time window section are stored and a local hash table having a 1: 1 association with the data object storage file for directly indexing the respective data objects stored in the data object storage file Wherein the network forensic device stores the file. 8. The method of claim 7,
The data object storage file and the local hash table storage file are stored in a timeline directory in a tree form according to a predetermined unit time period based on the storage time and stored in the corresponding time zone by inputting a search condition including a time condition through a timeline directory configuration Wherein the network forensic device provides a search function of a specific data object. An HTTPS decryption unit for extracting HTTP traffic by decoding HTTPS traffic received from a web client or a web server; And
An ICAP client for receiving HTTP traffic from an HTTPS decryption unit and delivering the HTTP traffic to a network forensic device through an ICAP protocol;
The web proxy comprising: The method of claim 9, wherein the HTTPS decoding unit
And extracting plaintext HTTP traffic from the HTTPS traffic by hooking a function of the encryption library. The method of claim 9, wherein the HTTPS decoding unit
A first HTTPS decryption unit for decrypting an HTTPS request message received from a web client to extract an HTTP request message from an HTTPS request message and transmitting the extracted HTTP request message to an ICAP client; And
A second HTTPS decryption unit for decrypting the HTTPS response message received from the web server, extracting an HTTP response message from the HTTPS response message, and transmitting the extracted HTTP response message to the ICAP client;
The web proxy comprising: 10. The method of claim 9, wherein the ICAP client
An upstream ICAP client receiving an HTTP request message from the HTTPS decryption unit and transmitting an ICAP REQMOD request message including an HTTP request message to the network forensic device; And
A downward ICAP client that receives an HTTP response message from the HTTPS decryption unit and transmits an ICAP RESPMOD request message including an HTTP response message to the network forensic device;
The web proxy comprising: 10. The method of claim 9, wherein the web proxy
An HTTP traffic processing unit for separately storing the decrypted HTTP traffic in a content cache;
Further comprising a web proxy. The web proxy receiving an encrypted HTTPS request message from a web client using the HTTPS protocol;
Extracting an HTTP request message from an HTTPS request message by decrypting an HTTPS request message received by the web proxy;
Transmitting an ICAP REQMOD request message including an extracted HTTP request message to a network forensic device using an ICAP protocol; And
The network forensic device storing an HTTP request message included in an ICAP REQMOD message;
The method of claim 1, further comprising the steps of: 15. The ICAP protocol extension method for providing a network forensic service of HTTPS traffic according to claim 14,
Transmitting the ICAP response message to the web proxy using the ICAP protocol after the network forensic device stores the HTTP request message;
The web proxy encrypts the HTTP request message included in the ICAP response message with the HTTPS request message and transmits the HTTP request message to the web server and receives the HTTPS response message from the web server in response to the HTTPS request message;
Extracting an HTTP response message from the HTTPS response message by decrypting the HTTPS response message received by the web proxy, and transmitting an ICAP RESPMOD request message including the extracted HTTP response message and the HTTP request message to the network forensic device using the ICAP protocol ; And
The network forensic device storing an HTTP response message and an HTTP request message included in the ICAP RESPMOD request message;
The method of claim 1, further comprising the steps of: 16. The method as claimed in claim 15, wherein the ICAP protocol extension method for providing the network forensic service of the HTTPS traffic comprises:
The web proxy receiving an ICAP response message from the network forensic device using the ICAP protocol in response to the ICAP RESPMOD request message; And
Encrypting an HTTP response message included in the received ICAP response message with an HTTPS response message, and transmitting the HTTP response message to a web client;
The method of claim 1, further comprising the steps of:

Images