CN113645256B - Aggregation method without reducing TCP session data value density - Google Patents
Aggregation method without reducing TCP session data value density Download PDFInfo
- Publication number
- CN113645256B CN113645256B CN202111192962.8A CN202111192962A CN113645256B CN 113645256 B CN113645256 B CN 113645256B CN 202111192962 A CN202111192962 A CN 202111192962A CN 113645256 B CN113645256 B CN 113645256B
- Authority
- CN
- China
- Prior art keywords
- session
- condition
- data
- suspicious
- aggregation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
Abstract
The invention relates to the technical field of network equipment safety management, in particular to a method for aggregating data value density without reducing TCP (Transmission control protocol) session. By the method, the effective session and other sessions can be distinguished, the aggregation scheme of the data is respectively established, the data value can be maximally reserved, and the problems of large data volume and low data analysis efficiency can be effectively solved.
Description
Technical Field
The invention relates to the technical field of network flow security analysis (stealing analysis), in particular to a polymerization method without reducing the value density of TCP session data.
Background
The network stealing analysis is mainly concerned with the detection of data transmission between network hosts with substantial communication behavior. In the network data, a large number of scanning behaviors, detecting behaviors and P2P transmission cause huge TCP session volume, and influence the whole network stealing analysis. The session information and other messy information which can be used for analysis need to be processed in a distinguishing way, but if the data is directly ignored, the scanning action and the detecting action before partial secret stealing action are left alone.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method for aggregating data without reducing the value density of TCP session data, which can distinguish effective sessions from other sessions, respectively establish a data aggregation scheme, maximally retain the data value, and effectively solve the problems of large data volume and low data analysis efficiency.
The invention is realized by adopting the following technical scheme:
an aggregation method without reducing the value density of TCP session data, characterized in that: the method comprises the following steps:
a. collecting network flow, and recombining and de-duplicating data frames in a TCP session;
b. traversing the recombined and de-duplicated TCP session, and respectively judging whether the three-way handshake is successful, whether the three-way handshake is an effective communication load and whether the three-way handshake is an effective service;
c. and recording the success of the three-way handshake as a condition A, the effective communication load as a condition B, the effective service as a condition C, and performing combined judgment according to the following sequence to realize the classification of the TCP session:
classifying the condition A, the condition B and the condition C as an identification session if the condition A, the condition B and the condition C are simultaneously satisfied or the condition B and the condition C are simultaneously satisfied;
if both condition a and condition B are satisfied or only condition B is satisfied, classifying it as an unknown session;
if only condition A is met, classifying the test object as a suspicious probing session;
if the situation is other, classifying the situation as a suspicious scanning session;
d. respectively carrying out data aggregation on the identification session, the unknown session, the suspicious detection session and the suspicious scanning session according to the following methods:
if the session is an identification session or an unknown session, reserving a complete quadruple without combination, wherein the quadruple comprises a client IP, a client port, a server IP and a server port;
if the suspicious detection session is detected, merging the data packets according to the three-way handshake direction, carrying out data aggregation on the client IP, the server IP and the server port in the data packets as keys, and recording the aggregation times;
and if the session is a suspicious scanning session, aggregating the client IP and the server IP as keys, and recording the aggregation times.
The step b of judging whether the communication load is an effective communication load specifically includes: in a TCP session, in the load layer, the following 2 conditions need to be satisfied after the bidirectional traffic is summed up:
b1a total of data transmissions containing no less than 60 bytes;
b2a number of data frames greater than 2.
The step b of judging whether the service is valid specifically includes: and identifying an application layer protocol through DPI, DFI, network behavior or a port which is commonly used for providing services, and determining the type of the access service and the corresponding service end and client.
In the step d, if the session is identified or unknown, the session is stored once after data aggregation, and if the session is a suspicious probing session or suspicious scanning session, caching and waiting are performed after data aggregation.
The step b of judging whether the three-way handshake is successful specifically means: a complete SYN-SYNACK-ACK step is included, which is a TCP three-way handshake process.
Compared with the prior art, the invention has the beneficial effects that:
1. during query, effective sessions and other sessions can be discriminated, data aggregation schemes are respectively established, data values are reserved to the maximum extent, data volume is effectively reduced, data analysis and mining are more targeted, and data analysis efficiency is improved.
2. Simple security analysis can be carried out on scanning and invalid sessions through classification, and some network threat results can be obtained.
3. After generalizing service access, analysis for network behavior is more meaningful, which reduces interference of additional data for validity determination of network behavior.
Detailed Description
Example 1
As a basic implementation mode of the invention, the invention comprises an aggregation method which does not reduce the data value density of a TCP session, and the aggregation method comprises the following steps:
a. and collecting network flow, and recombining and de-duplicating data frames in the TCP session to prevent interference when fields such as effective load and the like are calculated.
b. And traversing the recombined and de-duplicated TCP session, and respectively judging whether the three-way handshake is successful, whether the three-way handshake is an effective communication load and whether the three-way handshake is an effective service.
c. And recording the success of the three-way handshake as a condition A, the effective communication load as a condition B, the effective service as a condition C, and performing combined judgment according to the following sequence to realize the classification of the TCP session:
classifying the condition A, the condition B and the condition C as an identification session if the condition A, the condition B and the condition C are simultaneously satisfied or the condition B and the condition C are simultaneously satisfied;
if both condition a and condition B are satisfied or only condition B is satisfied, classifying it as an unknown session;
if only condition A is met, classifying the test object as a suspicious probing session;
if the situation is other, classifying the situation as a suspicious scanning session.
d. Respectively carrying out data aggregation on the identification session, the unknown session, the suspicious detection session and the suspicious scanning session according to the following methods:
if the session is an identification session or an unknown session, reserving a complete quadruple without combination, wherein the quadruple comprises a client IP, a client port, a server IP and a server port;
if the suspicious detection session is detected, merging the data packets according to the three-way handshake direction, carrying out data aggregation on the client IP, the server IP and the server port in the data packets as keys, and recording the aggregation times;
and if the session is a suspicious scanning session, aggregating the client IP and the server IP as keys, and recording the aggregation times.
Example 2
As a best mode for implementing the invention, the invention comprises an aggregation method which does not reduce the value density of TCP session data, and the aggregation method comprises the following steps:
a. and collecting network flow, recombining the TCP session according to the TCP SEQ, carrying out duplicate removal on data frames, and carrying out duplicate removal measurement according to a judgment mode of a duplicate packet.
b. And traversing the recombined and de-duplicated TCP session, and respectively judging whether the three-way handshake is successful, whether the three-way handshake is an effective communication load and whether the three-way handshake is an effective service.
The three-way handshake is successful: and if the step of the complete SYN-SYNACK-ACK is included and the step is successful, the server end is the target IP end of the SYN packet. The SYN-SYNACK-ACK is a TCP three-way handshake process. When a TCP connection is established, specifically, when one TCP connection is established, a total of 3 packets need to be sent by the client and the server to confirm the establishment of the connection. In socket programming, this process is triggered by the client executing a connect. In the TCP/IP protocol, the TCP protocol provides reliable connection services, and a connection is established using three-way handshaking. First handshake: setting a flag bit SYN to 1 by the Client, randomly generating a value seq = J, sending the data packet to the Server, and enabling the Client to enter a SYN _ SENT state to wait for the confirmation of the Server. SYN: synchronization Sequence Numbers (synchronization Sequence Numbers). Second handshake: after receiving the data packet, the Server knows that the Client requests to establish connection through the flag bit SYN =1, sets the flag bit SYN and ACK to 1, and sets ACK = J +1, randomly generates a value seq = K, sends the data packet to the Client to confirm the connection request, and enters a SYN _ RCVD state. Third handshake: after receiving the confirmation, the Client checks whether ACK is J +1 or not, whether ACK is 1 or not, if the ACK is correct, the flag bit ACK is 1, ACK = K +1, the data packet is sent to the Server, the Server checks whether ACK is K +1 or not, whether ACK is 1 or not, if the ACK is correct, connection establishment is successful, the Client and the Server enter an ESTABLISHED state, three-way handshake is completed, and then data transmission can be started between the Client and the Server.
Payload communication: in a TCP session, in the load layer, the following 2 conditions (normal traffic interaction) need to be satisfied after the bidirectional traffic is aggregated:
b1a total of data transmissions containing no less than 60 bytes;
b2a number of data frames greater than 2.
Effective service: application layer protocols (according to DPI, DFI) are identified through DPI, DF, network behavior or ports which are commonly used for providing services, and the type of access service and corresponding service end and client end are determined.
c. And recording the success of the three-way handshake as a condition A, the effective communication load as a condition B, the effective service as a condition C, and performing combined judgment according to the following sequence to realize the classification of the TCP session:
if the condition A, the condition B and the condition C are simultaneously met or the condition B and the condition C are simultaneously met (packet loss is collected to obtain other exceptions), a correct and effective access is performed on the network, the correct identification of the client and the server is performed, and the session is the key point of analysis and is classified as an identification session;
if the condition A and the condition B are simultaneously met or only the condition B is met (packet loss is collected to obtain other exceptions), the value of the session cannot be judged, and the value of the session is not determined for the stealing behavior, the session is classified as an unknown session;
if only the condition A is met, judging the service probing to be service probing, and if the probing is successful, classifying the service probing into suspicious probing sessions;
if the situation is other, classifying the situation as a suspicious scanning session.
d. Respectively carrying out data aggregation on the identification session, the unknown session, the suspicious detection session and the suspicious scanning session according to the following methods:
if the conversation is identified or unknown, the complete quadruplet is reserved, the combination is not carried out (the conversation is stored once), and the complete value is reserved. The quadruplet comprises a client IP, a client port, a server IP and a server port. The identification session or the unknown session contains network communication contents, and network security risks and clues contained in the contents can be obtained through analysis of the contents.
If the suspicious probing session is a suspicious probing session, merging is performed according to the data packets of the three-way handshake and the three-way handshake direction, multiple probing may exist in the network, and the probing ports opened by the client are inconsistent (random), ignoring the client port in the quadruple, performing data aggregation (cache waiting, such as 1 minute, one hour, and the like) on the client IP, the server IP, and the server port in the data packets as keys, and recording the aggregation times (equivalent to the probing success times). The suspicious probing session does not contain communication content, but as a successful handshake session, the suspicious probing session can be used as a proof that a port is opened by a server, the port can be used as a port provided by a service, and a single client port lacks data significance, so that whether a large number of probing behaviors occur can be judged by counting the aggregation times.
If the session is a suspicious scanning session, the suspicious scanning session probes a plurality of host nodes of the network or a plurality of ports of the nodes, and the session is often high-frequency, so that low-value data of the client port and the service port are ignored, the client IP and the server IP are aggregated into keys (cache waiting, such as 1 minute, one hour and the like), and the aggregation times (scanning times) are recorded. The suspicious scanning session is a session which is not successfully established, has no analysis value, and a large amount of behaviors cause a large amount of invalid four-tuple information in a network, and a client port and a server port are meaningless, and the aggregation times are many, which indicates that the scanning behavior is in progress; for example, port scanning, one IP will scan the 0-65535 ports of another IP once to determine which ports the IP has developed for network information collection, etc.
In summary, after reading the present disclosure, those skilled in the art should make various other modifications without creative efforts according to the technical solutions and concepts of the present disclosure, which are within the protection scope of the present disclosure.
Claims (3)
1. An aggregation method without reducing the value density of TCP session data, characterized in that: the method comprises the following steps:
a. collecting network flow, and recombining and de-duplicating data frames in a TCP session;
b. traversing the recombined and de-duplicated TCP session, and respectively judging whether the three-way handshake is successful, whether the three-way handshake is an effective communication load and whether the three-way handshake is an effective service;
wherein, judging whether the effective communication load is specifically: in a TCP session, in the load layer, the following 2 conditions need to be satisfied after the bidirectional traffic is summed up:
b1a total of data transmissions containing no less than 60 bytes;
b2the number of data frames greater than 2;
wherein, judging whether the service is valid specifically means: identifying an application layer protocol through DPI, DFI, network behavior or a common service providing port, and determining an access service type and a corresponding service end and a client;
c. and recording the success of the three-way handshake as a condition A, the effective communication load as a condition B, the effective service as a condition C, and performing combined judgment according to the following sequence to realize the classification of the TCP session:
classifying the condition A, the condition B and the condition C as an identification session if the condition A, the condition B and the condition C are simultaneously satisfied or the condition B and the condition C are simultaneously satisfied;
if both condition a and condition B are satisfied or only condition B is satisfied, classifying it as an unknown session;
if only condition A is met, classifying the test object as a suspicious probing session;
if the situation is other, classifying the situation as a suspicious scanning session;
d. respectively carrying out data aggregation on the identification session, the unknown session, the suspicious detection session and the suspicious scanning session according to the following methods:
if the session is an identification session or an unknown session, reserving a complete quadruple without combination, wherein the quadruple comprises a client IP, a client port, a server IP and a server port;
if the suspicious detection session is detected, merging the data packets according to the three-way handshake direction, carrying out data aggregation on the client IP, the server IP and the server port in the data packets as keys, and recording the aggregation times;
and if the session is a suspicious scanning session, aggregating the client IP and the server IP as keys, and recording the aggregation times.
2. The method of claim 1, wherein the aggregation method does not reduce the data cost density of the TCP session, and comprises: in the step d, if the session is identified or unknown, the session is stored once after data aggregation, and if the session is a suspicious probing session or suspicious scanning session, caching and waiting are performed after data aggregation.
3. The method of claim 1, wherein the aggregation method does not reduce the data cost density of the TCP session, and comprises: the step b of judging whether the three-way handshake is successful specifically means: a complete SYN-SYNACK-ACK step is included, which is a TCP three-way handshake process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111192962.8A CN113645256B (en) | 2021-10-13 | 2021-10-13 | Aggregation method without reducing TCP session data value density |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111192962.8A CN113645256B (en) | 2021-10-13 | 2021-10-13 | Aggregation method without reducing TCP session data value density |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113645256A CN113645256A (en) | 2021-11-12 |
| CN113645256B true CN113645256B (en) | 2021-12-28 |
Family
ID=78426617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111192962.8A Active CN113645256B (en) | 2021-10-13 | 2021-10-13 | Aggregation method without reducing TCP session data value density |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113645256B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114567501B (en) * | 2022-03-04 | 2023-10-31 | 科来网络技术股份有限公司 | Automatic asset identification method, system and equipment based on label scoring |
| CN115314325A (en) * | 2022-10-11 | 2022-11-08 | 科来网络技术股份有限公司 | Access relation analysis method, system, device and medium based on TCP communication |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103248605A (en) * | 2012-02-02 | 2013-08-14 | 哈尔滨安天科技股份有限公司 | TCP (transmission control protocol) flow convergence method and system based on IPV 6 (internet protocol version 6) |
| FR3023106A1 (en) * | 2014-06-30 | 2016-01-01 | Orange | TCP COMMUNICATION METHOD VIA MULTIPLE PATHS BETWEEN TWO TERMINALS |
| CN106534048A (en) * | 2015-09-11 | 2017-03-22 | 中国电信股份有限公司 | Method of preventing SDN denial of service attack, switch and system |
| CN107087006A (en) * | 2017-05-24 | 2017-08-22 | 全讯汇聚网络科技（北京）有限公司 | A kind of agreement shunt method, system and server |
| CN111641628A (en) * | 2020-05-26 | 2020-09-08 | 南京云利来软件科技有限公司 | Monitoring and early warning method for DDoS attack in subnet deception |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6768992B1 (en) * | 1999-05-17 | 2004-07-27 | Lynne G. Jolitz | Term addressable memory of an accelerator system and method |
| US7716729B2 (en) * | 2005-11-23 | 2010-05-11 | Genband Inc. | Method for responding to denial of service attacks at the session layer or above |
| US9479480B2 (en) * | 2010-01-29 | 2016-10-25 | Citrix Systems, Inc. | Systems and methods of using SSL pools for WAN acceleration |
| CN102299831B (en) * | 2011-09-27 | 2014-02-05 | 杭州华三通信技术有限公司 | Method for detecting invalidation of group members of internal server and network address translation (NAT) equipment |
| CN102573111A (en) * | 2012-01-10 | 2012-07-11 | 中兴通讯股份有限公司 | Method and device for releasing transfer control protocol resources |
| JP6428507B2 (en) * | 2015-06-29 | 2018-11-28 | 富士ゼロックス株式会社 | Information processing apparatus and information processing system |
| CN108880942A (en) * | 2018-06-29 | 2018-11-23 | 咪咕音乐有限公司 | A kind of method of specifying error, device and the storage medium of TCP session stream |
| ES2881255T3 (en) * | 2018-10-24 | 2021-11-29 | Acklio | Simple communication protocol for data transmission over limited networks |
| CN111092900B (en) * | 2019-12-24 | 2022-04-05 | 北京北信源软件股份有限公司 | Method and device for monitoring abnormal connection and scanning behavior of server |
| CN111447218B (en) * | 2020-03-25 | 2022-08-05 | 北京天地和兴科技有限公司 | TCP port scanning detection method |
| CN112751862A (en) * | 2020-12-30 | 2021-05-04 | 杭州迪普科技股份有限公司 | Port scanning attack detection method and device and electronic equipment |
-
2021
- 2021-10-13 CN CN202111192962.8A patent/CN113645256B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103248605A (en) * | 2012-02-02 | 2013-08-14 | 哈尔滨安天科技股份有限公司 | TCP (transmission control protocol) flow convergence method and system based on IPV 6 (internet protocol version 6) |
| FR3023106A1 (en) * | 2014-06-30 | 2016-01-01 | Orange | TCP COMMUNICATION METHOD VIA MULTIPLE PATHS BETWEEN TWO TERMINALS |
| CN106534048A (en) * | 2015-09-11 | 2017-03-22 | 中国电信股份有限公司 | Method of preventing SDN denial of service attack, switch and system |
| CN107087006A (en) * | 2017-05-24 | 2017-08-22 | 全讯汇聚网络科技（北京）有限公司 | A kind of agreement shunt method, system and server |
| CN111641628A (en) * | 2020-05-26 | 2020-09-08 | 南京云利来软件科技有限公司 | Monitoring and early warning method for DDoS attack in subnet deception |
Non-Patent Citations (2)
| Title |
|---|
| NS Vadlamani.A Survey on Detection and Defense of Application Layer DDoS Attacks.《digitalscholarship》.2013, * |
| 陈兴蜀等.基于会话流聚合的隐蔽性通信行为检测方法.《电子科技大学学报》.2019,(第03期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113645256A (en) | 2021-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113645256B (en) | Aggregation method without reducing TCP session data value density | |
| Handley et al. | Network Intrusion Detection: Evasion, Traffic Normalization, and {End-to-End} Protocol Semantics | |
| Zhang et al. | Detecting backdoors | |
| KR101280910B1 (en) | Two-stage intrusion detection system for high speed packet process using network processor and method thereof | |
| US7818786B2 (en) | Apparatus and method for managing session state | |
| US7472416B2 (en) | Preventing network reset denial of service attacks using embedded authentication information | |
| US7373663B2 (en) | Secret hashing for TCP SYN/FIN correspondence | |
| KR20000054538A (en) | System and method for intrusion detection in network and it's readable record medium by computer | |
| WO2019178966A1 (en) | Network attack defense method and apparatus, and computer device and storage medium | |
| Taleck | Ambiguity resolution via passive OS fingerprinting | |
| CN102655509B (en) | Network attack identification method and device | |
| Yang et al. | A real-time algorithm to detect long connection chains of interactive terminal sessions | |
| Llamas et al. | Covert channels in internet protocols: A survey | |
| US20080008171A1 (en) | System and method for detecting and interception of ip sharer | |
| EP3432533A1 (en) | Method and system for processing forged tcp data packet | |
| EP1842389A1 (en) | Method, device and programme for detecting ip spoofing in a wireless network | |
| US8490173B2 (en) | Unauthorized communication detection method | |
| CN114268429A (en) | Terminal-specific encrypted communication access device | |
| KR101263381B1 (en) | Method and apparatus for defending against denial of service attack in tcp/ip networks | |
| CN110691097A (en) | Industrial honey pot system based on hpfeeds protocol and working method thereof | |
| CN113872956A (en) | Method and system for inspecting IPSEC VPN transmission content | |
| CN112235329A (en) | Method, device and network equipment for identifying authenticity of SYN message | |
| KR101141919B1 (en) | High performance network equipment with a fuction of multi-decryption in ssl/tls sessions' traffic and data processing method of the same | |
| CN114710343A (en) | Intrusion detection method and detection equipment | |
| CN115022281A (en) | NAT (network Address translation) penetration method, client and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |