KR101834503B1 - Log generating method and apparatus based on packet gathering per session in big data system - Google Patents

Abstract

The present invention relates to a method and apparatus for log generation based on a session-specific packet collection in a big data system, and an apparatus according to the present invention includes a session management unit including a plurality of packet storage units, A packet distribution unit for receiving packets exchanged between the servers and distributing the packets to a plurality of packet storage units for each session; and reassembling the request packets and the response packets, which are stored for each session, in the packet storage unit into a request message and a response message, And a log generation unit for generating a log by mapping the information extracted from the response message and the extracted information. According to the present invention, there is an advantage that a log can be generated by mapping a request message and a corresponding response message to each other. In addition, since the request packet and the response packet are separated by the session and stored in a stacked manner, there is an advantage in that high-speed log collection through the packet is possible by not paying attention to processing of SYN, ACK, and the like in network communication.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for generating a log based on a session-specific packet collection in a big data system,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a log generation method and apparatus, and more particularly, to a method and apparatus for generating a session-based packet collection based log in a big data system.

Recently, the development of IT technology has facilitated the collection and sharing of various information, and the path for the leakage of important internal information of the enterprise or organization has been diversified. Meanwhile, a company or a public entity collects a packet exchanged between a client and a server in order to detect a security anomaly or a security anomaly (hereinafter referred to as an abnormal symptom), and generates a large amount of log data. Generally, Is merely about capturing and recording packets transmitted and received in a network section.

There is a need for a method of generating log data by generating a vast amount of large data in a log format by minimizing the load on the system from the log generation step and matching the response and the request.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a method and apparatus for generating a log by mapping a request message corresponding to each session based on a session-based packet collection in a big data system and information extracted from a response message.

According to an aspect of the present invention, there is provided an apparatus for generating a packet-based log for a session according to the present invention, including: a session management unit including a plurality of packet storage units generated for each session established between a client and a server; A packet distribution unit for receiving a packet and distributing it to the plurality of packet storage units for each session; and reassembling the request packet and the response packet stored for each session in the packet storage unit into a request message and a response message, And a log generation unit for generating a log by mapping information extracted from the response message and information extracted from the response message.

The session management unit may generate a packet storage unit corresponding to the established session each time a session is established between the client and the server.

The session management unit may generate the plurality of packet storage units to have a logical structure capable of scale-out expansion on an in-memory.

The plurality of packet storage units may store a packet distributed by a session in the packet distribution unit into a request packet and a response packet, and store the packet in a stack manner.

The session management unit, when it is determined that both the request packet and the response packet constituting the request message and the response message are stored in the predetermined packet storage unit among the plurality of packet storage units, And a response packet to the log generation unit.

The log generation unit reassembles the request message and the response message according to the protocol of the request packet and the response packet, and transmits the reassembled request message and the reassembled response message, The log can be extracted.

According to another aspect of the present invention, there is provided a method of generating a packet-based log for each session, the method comprising: generating a packet storage unit for each session established between a client and a server; And reassembling the request packet and the response packet, which are stored for each session, into a request message and a response message in a packet storage unit, and a step of reassembling the request packet and the response packet into a request message and a response message, And generating a log by mapping the extracted information.

According to the present invention, there is an advantage that a log can be generated by mapping a request message and a corresponding response message to each other. In addition, since the request packet and the response packet are separated by the session and stored in a stacked manner, there is an advantage in that high-speed log collection through the packet is possible by not paying attention to processing of SYN, ACK, and the like in network communication.

FIG. 1 is a block diagram illustrating a configuration of an apparatus for generating a packet-based log for a session according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the log generating apparatus of FIG. 1 in more detail.
FIG. 3 is a flowchart illustrating a method of generating a packet-based log for each session according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

FIG. 1 is a block diagram illustrating a configuration of an apparatus for generating a packet-based log for a session according to an embodiment of the present invention.

Referring to FIG. 1, the client 10a, 10b, ..., 10n and the server 20 can exchange a request message and a response message via the communication network 30. [

The communication network 30 does not need to include a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and the Internet, It does not matter what communication method it is.

The clients 10a, 10b, ..., 10n and the server 20 establish a session and exchange request packets and response packets based on protocols such as HTTP, FTP, IP, TCP, and DNS.

The session-specific packet-collection-based log generating apparatus 100 collects packets exchanged between the clients 10a, 10b, ..., 10n and the server 20 via the communication network 30 and generates a log.

2 is a block diagram showing the configuration of the log generating apparatus 100 of FIG. 1 in more detail.

2, the log generation apparatus 100 according to the present invention may include a packet distribution unit 110, a session management unit 120, and a log generation unit 130. The session management unit 120 may include a plurality of packet storage units 120a, 120b, ..., 120m.

The packet distribution unit 110 receives packets exchanged between the clients 10a, 10b, ..., 10n and the server 20 and classifies them according to the sessions, and stores the packet storage units 120a, 120b, ..., 120m).

The session management unit 120 may generate a packet storage unit for each session established in the clients 10a, 10b, ..., 10n and the server 20. [ For example, whenever a new session is established between the client 10a, 10b, ..., 10n and the server 20, the session management unit 120 stores the packet storage unit corresponding to the newly established session on an in-memory Can be created to have a logical structure capable of scale-out expansion. Of course, if there is a terminated session, the session management unit 120 can delete the packet storage unit of the session.

The session management unit 120 can generate a packet storage unit using the client side IP: port of the newly established session and the server side IP: port as unique keys. The packet distribution unit 110 distributes the received packet You can distribute packets by session based on the source IP: port and destination IP: port.

The plurality of packet storage units 120a, 120b, ..., and 120m may store packets distributed in a packet distribution unit 110 for each session into a request packet and a response packet in a stacked manner.

If it is determined that the request packet constituting the request message and the response packet constituting the response message are stored in the predetermined packet storage unit among the plurality of packet storage units 120a, 120b, ..., 120m, It is possible to provide the log generation unit 130 with a request packet and a response packet stored in a predetermined packet storage unit.

For example, when 'session A' is established between the client 10a and the server 20, the session management unit 120 may generate a packet storage unit 120m corresponding to 'session A'. The request packet of the request message transmitted from the client 10a to the server 20 via the 'session A' is transmitted to and stored in the packet storage unit 120m in the order received by the packet distribution unit 110. [ Similarly, when the server 20 transmits a response message to the client 10a through 'session A', the response packet of the response message is delivered to the packet storage unit 120m in the order received by the packet distribution unit 110, do.

Accordingly, when a new request packet is received after a series of request packets and a series of response packets are received, it can be determined that packets constituting a pair of request message and response message are all stored in the packet storage unit 120m. Of course, even if a new request packet is not received, if a new request packet is not received even after a predetermined time (for example, a predetermined time such as 10 seconds) has elapsed after the last response packet of the session is received, It can be determined that all the packets constituting the message and the response message are stored in the packet storage unit 120m.

The log generation unit 130 reassembles the request packet and the response packet stored in the plurality of packet storage units 120a, 120b, ..., and 120m for each session into a request message and a response message, And generates a log by mapping the information extracted from the information and response message.

The log generation unit 130 reassembles the request message and the response message according to the protocol of the request packet and the response packet, extracts the preset information according to the protocol from the reassembled request message and the response message, have.

Table 1 below illustrates information extracted by the log generation unit 130 for log generation according to the protocol.

protocol Extractable information HTTP ㅇ Request Header Item
Date: Provides the date and time of when the message was created Via: Shows what message this message has been through (proxy, gateway)

Request Header Item
Client-IP: provides the IP of the computer on which the client is running
Host: Provides the host name and port of the server that is the target of the request.
Referer: provides the URL of the document that contained the current request URI.

ㅇ Response Header item
Server: Name and version of the server application IP ㅇ IP header information
Source IP Address
Destination IP Address TCP or FTP TCP header information
Source port (16 bits): send port
Destination port (16 bits): Receive port DNS About Body
Domain Name
DNS Question & Answer Data

Of course, Table 1 illustrates the information that can be extracted from the request message and the response message for log generation in the log generation unit 130, and other information may be extracted and included in the log.

FIG. 3 is a flowchart illustrating a method of generating a packet-based log for each session according to an embodiment of the present invention.

1 to 3, the session management unit 120 stores the packet storage units 120a, 120b, ..., 120m for each session established between the clients 10a, 10b, ..., 10n and the server 20, (S310). In step S310, the session management unit 120 may generate a packet storage unit corresponding to a newly established session so as to have a logical structure capable of scale-out expansion on an in-memory. Of course, if there is a terminated session, the session management unit 120 can delete the packet storage unit of the session.

Next, the packet distribution unit 110 classifies the packets exchanged between the clients 10a, 10b, ..., 10n and the server 20 for each session and stores them in the corresponding packet storage units 120a, 120b, ..., 120m to be stored (S320). In step S320, the packet storage units 120a, 120b, ..., and 120m may separate the received packet into a request packet and a response packet, and store the received packet in a stacked manner.

If a new session is established between the client 10a, 10b, ..., 10n and the server 20 during the execution of the step S320, the session management unit 120 may perform step S310.

In step S310 to step S320, the session management unit 120 stores a request packet and a response message in a predetermined packet storage unit among the plurality of packet storage units 120a, 120b, ..., (S330-Y), it is possible to provide the log generation unit 130 with the request packet and the response packet stored in the predetermined packet storage unit (S340).

Next, the log generation unit 130 reassembles the request packet and the response packet provided in the packet storage units 120a, 120b, ..., 120m into a request message and a response message (S350) And the information extracted from the response message is mapped to generate the log (S360).

In step S360, the log generator 130 reassembles the request message and the response message according to the protocol of the request packet and the response packet, and extracts preset information according to the protocol from the reassembled request message and the response message You can create a log.

Embodiments of the present invention include a computer-readable medium having program instructions for performing various computer-implemented operations. This medium records a program for executing the above-described method for generating a session-based packet collection based log. The medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of such media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD and DVD, programmed instructions such as floptical disk and magneto-optical media, ROM, RAM, And a hardware device configured to store and execute the program. Or such medium may be a transmission medium, such as optical or metal lines, waveguides, etc., including a carrier wave that transmits a signal specifying a program command, data structure, or the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, Of the right.

Claims (12)

A session management unit including a plurality of packet storage units generated for each session established between the client and the server,
A packet distribution unit for receiving packets exchanged between the client and the server and distributing the packets to the plurality of packet storage units for each session;
A log generation unit for reassembling the request packet and the response packet stored in the packet storage unit for each session into a request message and a response message and mapping the information extracted in the request message and the information extracted in the response message,
Lt; / RTI >
The session management unit,
Wherein each time a session is established between a client and a server, a packet storage unit corresponding to the established session is created. delete The method of claim 1,
The session management unit,
Wherein the plurality of packet storage units are configured to have a logical structure capable of scale-out expansion on an in-memory basis. The method of claim 1,
Wherein the plurality of packet storage units comprise:
Wherein the packets are divided into a request packet and a response packet in a stacked manner. 5. The method of claim 4,
The session management unit,
If it is determined that both the request packet and the response packet constituting the request message and the response message are stored in the predetermined packet storage unit among the plurality of packet storage units, A log generation unit for generating a session-specific packet collection based on a log; The method of claim 5,
The log generation unit may include:
Reassembles the request message and the response message according to the protocol of the request packet and the response packet, and extracts the preset information according to the protocol from the reassembled request message and the reassembled response message to generate a log Based packet collection based log generation device. Generating a packet storage unit for each session established between the client and the server,
Receiving a packet exchanged between a client and a server, distributing the packet to each session, and storing the packet in a corresponding packet storage unit,
Reassembling the request packet and the response packet, which are stored for each session in the packet storage unit, into a request message and a response message; and
Generating a log by mapping information extracted from the request message and information extracted from the response message
Lt; / RTI >
And generating a packet storage unit corresponding to the established session whenever a session is established between the client and the server. delete 8. The method of claim 7,
Wherein the packet storage unit is generated to have a logical structure capable of scale-out expansion on an in-memory basis. 8. The method of claim 7,
The packet storage unit stores,
And storing the packets distributed by the session into a request packet and a response packet in a stacked manner. 11. The method of claim 10,
If it is determined that both the request packet and the response packet constituting the request message and the response message are stored in the packet storage unit, the request packet and the response packet stored in the packet storage unit are connected in the reverse order stored in the packet storage unit And reassembling the request message and the response message. 12. The method of claim 11,
Reassembling the request message and the response message according to the protocol of the request packet and the response packet, extracting predetermined information according to the protocol from the reassembled request message and the reassembled response message, Generating per-session packet collection based log generation method.

Images