CN113315678A - Encrypted TCP (Transmission control protocol) traffic acquisition method and device - Google Patents

Encrypted TCP (Transmission control protocol) traffic acquisition method and device Download PDF

Info

Publication number
CN113315678A
CN113315678A CN202110577572.6A CN202110577572A CN113315678A CN 113315678 A CN113315678 A CN 113315678A CN 202110577572 A CN202110577572 A CN 202110577572A CN 113315678 A CN113315678 A CN 113315678A
Authority
CN
China
Prior art keywords
tcp
tcp session
data packet
session
fingerprint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110577572.6A
Other languages
Chinese (zh)
Inventor
陈平
谢东峰
樊俊锋
李志奇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Open Security Research Inc
Original Assignee
Open Security Research Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Open Security Research Inc filed Critical Open Security Research Inc
Priority to CN202110577572.6A priority Critical patent/CN113315678A/en
Publication of CN113315678A publication Critical patent/CN113315678A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Environmental & Geological Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides an encrypted TCP flow acquisition method and device. The method comprises the following steps: identifying a TCP session establishment data packet for establishing a TCP session from the obtained encrypted TCP flow according to a three-way handshake process of TCP; adding TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until TCP session ending data packets used for ending the TCP session are identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session; and carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session. By analyzing the TCP session, bidirectional TCP session flow is collected, content fields collected by the encrypted TCP flow are enriched, and subsequent flow monitoring and analysis are facilitated.

Description

Encrypted TCP (Transmission control protocol) traffic acquisition method and device
Technical Field
The invention relates to the technical field of communication, in particular to an encrypted TCP (transmission control protocol) flow acquisition method and device.
Background
A Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte stream-based transport layer communication Protocol, and is one of the most basic communication protocols in the internet, and daily network applications such as WeChat, QQ, browsing web pages, sending and receiving e-mails and the like cannot be separated from the TCP, and network monitoring and threat identification can be performed by collecting TCP flow. Driven by the requirements of network security and privacy protection, network communication encryption has become a trend, and most of network data traffic is encrypted at present. The increase in encrypted traffic undoubtedly increases the security of the network, but also poses severe challenges for network traffic collection and analysis.
Traditional content fields based on an application layer Protocol are used for capturing and analyzing full traffic, for example, webpage content based on a hypertext Transfer Protocol (HTTP) is collected and analyzed, but the method can only be used for unencrypted network traffic and is not suitable for encrypted traffic, and because the encrypted application layer Protocol content cannot be decrypted, the significance of collecting the encrypted traffic is not great. The Netflow technology can record flow information of a Transmission Control Protocol/Internet Protocol (TCP/IP) layer, one Netflow flow is defined as a unidirectional packet flow transmitted between a source IP address and a destination IP address, and all packets have a common port number of a Transmission layer source and a destination. Although the Netflow technology can collect encrypted TCP flow, the Netflow technology focuses on the flow behavior of an IP network layer, and the granularity is too coarse to meet the requirement of subsequent flow analysis.
Disclosure of Invention
The embodiment of the invention provides an encrypted TCP (transmission control protocol) traffic acquisition method and device, which are used for solving the problems that the granularity of the conventional traffic acquisition method is too coarse, the acquired information is limited, and the subsequent traffic analysis requirements cannot be met.
In a first aspect, an embodiment of the present invention provides an encrypted TCP flow collecting method, including:
identifying a TCP session establishment data packet for establishing a TCP session from the obtained encrypted TCP flow according to a three-way handshake process of TCP;
adding TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until TCP session ending data packets used for ending the TCP session are identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session;
and carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session.
In one embodiment, the feature information includes:
a source IP address, a source TCP port, a destination IP address, a destination TCP port, a session packet total number, a source packet total size, a destination packet total number, a destination packet total size, a TCP Flag seven-element array, a session start time, and a session end time.
In one embodiment, the method further comprises:
acquiring a first TCP data packet sent by a source after a TCP session is established from a data packet set of the TCP session;
determining an encryption protocol of the TCP session according to the first TCP data packet;
and determining a fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
In one embodiment, if the encryption protocol of the TCP session is the TLS protocol, determining a fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet includes:
acquiring a ClientHello message, a ServerHello message and a Certificate message from the handshake data packet;
performing fingerprint extraction on the ClientHello message by adopting a first fingerprint extraction algorithm to obtain a fingerprint of the ClientHello message;
performing fingerprint extraction on the ServerHello message by adopting a second fingerprint extraction algorithm to obtain a fingerprint of the ServerHello message;
analyzing the Certificate message according to the X.509 Certificate format and calculating the SHA-1 fingerprint of the Certificate message;
and adding the ClientHello message and the fingerprint corresponding to the ClientHello message, the ServerHello message and the fingerprint corresponding to the ServerHello message, and the Certificate message and the fingerprint corresponding to the Certificate message into a fingerprint library of the TCP session.
In one embodiment, if the encryption protocol of the TCP session is SSH protocol, determining a fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet includes:
acquiring an SSH client key exchange message and an SSH server key exchange message from the handshake data packet;
performing fingerprint extraction on the SSH client key exchange message by adopting a third fingerprint extraction algorithm to obtain a fingerprint of the SSH client key exchange message;
performing fingerprint extraction on the SSH server key exchange message by adopting a fourth fingerprint extraction algorithm to obtain a fingerprint of the SSH server key exchange message;
and adding the SSH client key exchange message and the fingerprint corresponding to the SSH client key exchange message and the SSH server key exchange message and the fingerprint corresponding to the SSH server key exchange message into a fingerprint library of the TCP session.
In one embodiment, if the encryption protocol of the TCP session is a private encryption protocol, determining a fingerprint library of the TCP session according to the encryption protocol of the TCP session and a handshake data packet includes:
adding a source first load and a target first load of a TCP session into a fingerprint library of the TCP session, wherein the source first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a source, and the target first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a destination.
In a second aspect, an embodiment of the present invention provides an encrypted TCP traffic collection apparatus, including:
the TCP session establishment identification module is used for identifying a TCP session establishment data packet used for establishing a TCP session from the obtained encrypted TCP flow according to the three-way handshake process of the TCP;
the TCP session flow acquisition module is used for adding the TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until the TCP session ending data packets used for ending the TCP session are identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session;
and the TCP session characteristic acquisition module is used for carrying out statistical analysis on the data packet set of the TCP session and acquiring the characteristic information of the TCP session.
In one embodiment, the apparatus further comprises:
the acquisition module is used for acquiring a first TCP data packet sent by a source after a TCP session is established from a data packet set of the TCP session;
the protocol identification module is used for determining an encryption protocol of the TCP session according to the first TCP data packet;
and the processing module is used for determining a fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
In a third aspect, an embodiment of the present invention provides an encrypted TCP traffic collection apparatus, including:
at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the memory-stored computer-executable instructions causes the at least one processor to perform the encrypted TCP traffic collection method of any of the first aspects.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is configured to implement the encrypted TCP traffic collection method according to any one of the first aspect.
According to the encrypted TCP flow acquisition method and device provided by the embodiment of the invention, a TCP session establishment data packet for establishing a TCP session is identified from the acquired encrypted TCP flow according to the three-way handshake process of TCP; adding TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until TCP session ending data packets used for ending the TCP session are identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session; and carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session. By analyzing the TCP session, bidirectional TCP session flow is collected, content fields collected by the encrypted TCP flow are enriched, and subsequent flow monitoring and analysis are facilitated.
Drawings
Fig. 1 is a flowchart of an embodiment of an encrypted TCP traffic collection method provided in the present invention;
fig. 2 is a schematic diagram of a TCP session establishment process according to an embodiment;
FIG. 3 is a diagram illustrating a TCP session termination process according to an embodiment;
fig. 4 is a diagram illustrating a handshake negotiation process of the TLS protocol according to an embodiment;
fig. 5 is a schematic diagram of an SSH protocol handshake negotiation process according to an embodiment;
fig. 6 is a flowchart of an encrypted TCP flow collecting method according to another embodiment of the present invention;
fig. 7 is a schematic structural diagram of an embodiment of an encrypted TCP traffic collection apparatus provided in the present invention;
fig. 8 is a schematic structural diagram of an embodiment of an encrypted TCP traffic collection device provided in the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
Although the existing Netflow technology can be used for collecting encrypted TCP flow, the Netflow is a unidirectional data packet flow defined to be transmitted between a source IP address and a destination IP address, all data packets have a common transmission layer source and destination port number, and the flow collection granularity is too coarse to meet the requirements of flow monitoring and analysis. TCP is a reliable, connection-oriented delivery service protocol, connection-oriented meaning that a session must be established before two communicating parties exchange data, the establishment of a TCP session being accomplished by a three-way handshake and the termination of a TCP session being accomplished by a four-way swipe. In order to refine the granularity of flow collection and enrich the content of flow collection, the method and the device make full use of the characteristics of TCP session establishment and termination to analyze the TCP session and realize the collection of encrypted TCP flow. This will be further illustrated by the detailed examples below.
Fig. 1 is a flowchart of an embodiment of an encrypted TCP traffic collection method according to the present invention. The method provided by the embodiment can be integrated and applied to the existing network equipment, such as a router, a switch and the like, and can also be applied to independent traffic collection equipment. As shown in fig. 1, the encrypted TCP traffic collection method provided in this embodiment may include:
s101, identifying a TCP session establishment data packet for establishing a TCP session from the obtained encrypted TCP flow according to the three-way handshake process of the TCP.
The encrypted TCP traffic in this embodiment may be obtained, for example, by mirroring the traffic with the core switch. The TCP session establishment process is shown in fig. 2, in this embodiment, after obtaining the encrypted TCP traffic, it may be identified whether the data packet is establishing a TCP session according to a three-way handshake process shown in fig. 2. In order to facilitate traffic collection with a TCP session as a granularity, in this embodiment, a data packet for establishing a TCP session, that is, a TCP session establishment data packet, needs to be identified from the obtained encrypted TCP traffic according to a three-way handshake process of the TCP.
S102, adding the TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until a TCP session ending data packet for ending the TCP session is identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain the data packet set of the TCP session.
TCP session end process after identifying the TCP session establishment packet, as shown in fig. 3, it can be identified whether the packet is ending the TCP session according to the four-wave process as shown in fig. 3. In this embodiment, a data packet for ending a TCP session, that is, a TCP session end data packet, is identified from the obtained encrypted TCP traffic according to the four-time waving process of the TCP. It will be appreciated that the data packets between the TCP session establishment data packet and the TCP session end data packet belong to the TCP session. It should be noted that if the collected data packet is not a TCP data packet, a discard process is required.
Further, considering that some TCP sessions have long connection settings, which will result in an increasing number of TCP packets in a single TCP session, it is necessary to determine whether the number of packets in the entire TCP session has reached a preset number, for example, 10000 packets, in addition to identifying the TCP session end packet, and if so, determine that the TCP session has ended.
In summary, in this embodiment, after the TCP session establishment packet is identified, the TCP packet belonging to the TCP session after the TCP session establishment packet is added to the packet set of the TCP session until the TCP session end packet for ending the TCP session is identified from the obtained encrypted TCP traffic according to the four-time waving process of the TCP, or the number of the packets in the packet set reaches the preset number, so as to obtain the packet set of the TCP session.
S103, carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session.
In this embodiment, after the data packet set of the TCP session is obtained, the data packet set of the TCP session is statistically analyzed to obtain the characteristic information of a complete TCP session. In an alternative embodiment, the characteristic information of the TCP session may include: source IP address (srcIP), source TCP port (srcPort), destination IP address (dstIP), destination TCP port (dstPort), total number of session packets (totalPackets), total number of source packets (srcPackets), total size of source packets (srcPacketSize), total number of destination packets (dstPacket), total size of destination packets (dstPacketSize), TCP Flag seven-element array (tcpFlags: number of SYN, SYN-ACK, PSH, FIN, RST, URG), session start time (sessionStart), and session end time (sessionStop).
In the encrypted TCP flow collecting method provided in this embodiment, a TCP session establishment packet for establishing a TCP session is identified from the obtained encrypted TCP flow according to a three-way handshake process of the TCP; adding TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until TCP session ending data packets used for ending the TCP session are identified from the obtained encrypted TCP flow according to the four-time waving process of the TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session; and carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session. By analyzing the TCP session, bidirectional TCP session flow is collected, content fields collected by the encrypted TCP flow are enriched, and subsequent flow monitoring and analysis are facilitated.
The Transport Layer Security (TLS) and Secure Shell (SSH) protocols are two common TCP traffic encryption protocols. The TLS protocol handshake negotiation procedure and the SSH protocol handshake negotiation procedure may refer to fig. 4 and 5, respectively. As can be seen from fig. 4 and 5, the TLS protocol and the SSH protocol have some unencrypted information in the handshake protocol process, such as TLS protocol version, TLS encryption suite, ClientHello message sent by the Client, ServerHello message returned by the Server, Certificate message sent by the Server, SSH protocol version, SSH Client key exchange message (Client SSH _ MSG _ key) and SSH Server key exchange message (Server SSH _ MSG _ key), and the like, and the identification of these unencrypted information is helpful for traffic monitoring and analysis. However, the existing Netflow technology does not identify and analyze the protocol above the transport layer, so some unencrypted contents in the encryption protocol are not collected, for example, the handshake phase in the TLS encryption communication protocol is unencrypted, but the handshake messages cannot be collected by the Netflow technology.
In order to identify unencrypted information in the handshake negotiation process in the encrypted TCP flow, the encrypted TCP flow acquisition method provided in this embodiment adds the encrypted protocol content sensing to identify the encrypted protocol on the basis of the above embodiment. As shown in fig. 6, the encrypted TCP traffic collection method provided in this embodiment may further include, on the basis of the embodiment shown in fig. 1:
s104, acquiring a first TCP data packet sent by the source after the TCP session is established from the data packet set of the TCP session.
In order to identify the encryption protocol used to encrypt TCP traffic, in this embodiment, the first TCP packet sent by the source after the TCP session is established needs to be obtained from the data packet set of the TCP session.
And S105, determining the encryption protocol of the TCP session according to the first TCP data packet.
The encryption protocol in the present embodiment may include, for example, TLS protocol, SSH protocol, and private encryption protocol. Private encryption protocols are understood to mean encryption protocols other than the TLS protocol and SSH protocol. In this embodiment, the encryption protocol used by the TCP session may be determined according to the first TCP packet sent by the source and the TLS protocol handshake negotiation process and the SSH protocol handshake negotiation process shown in fig. 4 and 5.
And S106, determining a fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
After determining the encryption protocol used by the TCP session, the corresponding handshake packet may be obtained from the packet set of the TCP session according to the type of encryption protocol. It can be understood that positions and sizes of handshake data packets of TCP sessions using different encryption protocols may be different, and specific reference is made to corresponding encryption protocol specifications, which are not described herein again. After the corresponding handshake data packet is obtained, a fingerprint library of the TCP session may be determined according to the encryption protocol of the TCP session and the handshake data packet. It will be appreciated that TCP sessions using different encryption protocols will have different fingerprint libraries formed. The process of creating the fingerprint library will be described below for TCP sessions that employ the TLS protocol, SSH protocol, and private encryption protocol, respectively.
In an optional embodiment, if the encryption protocol of the TCP session is the TLS protocol, determining the fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet may include: acquiring a ClientHello message, a ServerHello message and a Certificate message from the handshake data packet; performing fingerprint extraction on the ClientHello message by adopting a first fingerprint extraction algorithm to obtain a fingerprint of the ClientHello message; performing fingerprint extraction on the ServerHello message by adopting a second fingerprint extraction algorithm to obtain a fingerprint of the ServerHello message; analyzing the Certificate message according to the X.509 Certificate format and calculating the SHA-1 fingerprint of the Certificate message; and adding the ClientHello message and the fingerprint corresponding to the ClientHello message, the ServerHello message and the fingerprint corresponding to the ServerHello message, and the Certificate message and the fingerprint corresponding to the Certificate message into a fingerprint library of the TCP session.
For example, the TLS protocol handshake negotiation process shown in fig. 4 may be referred to obtain a handshake packet from a packet set of a TCP session, and obtain a ClientHello message, a ServerHello message, and a Certificate message from the handshake packet. The first fingerprint extraction algorithm may be a message digest algorithm, for example, an MD5 algorithm may be used to obtain a fingerprint (TLSClient fingerprint) corresponding to a ClientHello message based on the content of the ClientHello message, and specifically, reference may be made to the JA3 method disclosed by Salesforce; the second fingerprint extraction algorithm may be a message digest algorithm, for example, an MD5 algorithm may be used to obtain a fingerprint (TLSServer fingerprint) corresponding to the ServerHello message based on the content of the ServerHello message, and specifically, reference may be made to the JA3S method disclosed by Salesforce. It should be noted that the first fingerprint extraction algorithm and the second fingerprint extraction algorithm may be the same or different, and may be selected according to actual needs. For the Certificate message, it needs to parse according to the x.509 Certificate format, and calculate a corresponding SHA1 fingerprint (Certificate fingerprint). And finally, adding the ClientHello message and the fingerprint corresponding to the ClientHello message, the ServerHello message and the fingerprint corresponding to the ServerHello message, and the Certificate message and the fingerprint corresponding to the Certificate message into a fingerprint library of the TCP session. That is, the fingerprint library of the TCP session using the TLS encryption protocol includes: ClientHello message and corresponding TLSClient fingerprint, ServerHello message and corresponding TLSServer fingerprint, Certificate message and corresponding Certificate fingerprint.
In another alternative embodiment, if the encryption protocol of the TCP session is SSH protocol, determining the fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet may include: acquiring an SSH client key exchange message and an SSH server key exchange message from the handshake data packet; performing fingerprint extraction on the SSH client key exchange message by adopting a third fingerprint extraction algorithm to obtain a fingerprint of the SSH client key exchange message; performing fingerprint extraction on the SSH server key exchange message by adopting a fourth fingerprint extraction algorithm to obtain a fingerprint of the SSH server key exchange message; and adding the SSH client key exchange message and the fingerprint corresponding to the SSH client key exchange message and the SSH server key exchange message and the fingerprint corresponding to the SSH server key exchange message into a fingerprint library of the TCP session.
For example, the SSH protocol handshake negotiation process shown in fig. 5 may be referred to obtain a handshake packet from a packet set of the TCP session, and obtain an SSH Client key exchange message (Client SSH _ MSG _ KEXINIT) and an SSH Server key exchange message (Server SSH _ MSG _ KEXINIT) from the handshake packet. The third fingerprint extraction algorithm may be a message digest algorithm, for example, an MD5 algorithm may be used to obtain a corresponding fingerprint (SSHClient fingerprint) based on SSH client key exchange message content, and specifically, a hassh method disclosed by Salesforce may be referred to; the fourth fingerprint extraction algorithm may be a message digest algorithm, for example, an MD5 algorithm may be used to obtain a corresponding fingerprint (SSHServer fingerprint) based on SSH server key exchange message content, and specifically, refer to the hasshServer method disclosed by Salesforce. It should be noted that the third fingerprint extraction algorithm and the fourth fingerprint extraction algorithm may be the same or different, and may be selected according to actual needs. And finally, adding the SSH client key exchange message and the fingerprint corresponding to the SSH client key exchange message and the SSH server key exchange message and the fingerprint corresponding to the SSH server key exchange message into a fingerprint library of the TCP session. That is, the fingerprint library of the TCP session using the SSH encryption protocol includes: SSH Client key exchange messages (Client SSH _ MSG _ KEXINIT) and corresponding SSHClient fingerprints, SSH Server key exchange messages (Server SSH _ MSG _ KEXINIT) and corresponding SSHServer fingerprints.
In another optional embodiment, if the encryption protocol of the TCP session is a private encryption protocol, determining the fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet may include: adding a source first load and a target first load of a TCP session into a fingerprint library of the TCP session, wherein the source first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a source, and the target first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a destination. For example, the source first payload may be the hex value of the first 8 bytes of the first TCP packet payload issued by the source (srcPayload8) and the destination first payload may be the hex value of the first 8 bytes of the first TCP packet payload issued by the destination (dstPayload 8).
On the basis of the embodiment, the encrypted TCP flow acquisition method provided by the embodiment further increases the analysis of the handshake negotiation process of the cryptographic protocol, establishes the fingerprint library according to the important handshake messages, and can effectively help the user to monitor and identify the threat to the encrypted TCP flow through the increased characteristic acquisition.
Fig. 7 is a schematic structural diagram of an embodiment of an encrypted TCP traffic collection apparatus provided in the present invention. As shown in fig. 7, the encrypted TCP traffic collecting apparatus 20 provided in this embodiment may include: a TCP session establishment identification module 201, a TCP session flow collection module 202 and a TCP session characteristic collection module 203.
A TCP session establishment identifying module 201, configured to identify a TCP session establishment packet for establishing a TCP session from the obtained encrypted TCP traffic according to a three-way handshake process of the TCP.
The TCP session traffic collection module 202 is configured to add a TCP data packet belonging to a TCP session after the TCP session establishes the data packet to a data packet set of the TCP session until a TCP session end data packet for ending the TCP session is identified from the obtained encrypted TCP traffic according to the four-time waving process of the TCP, or the number of data packets in the data packet set reaches a preset number, so as to obtain a data packet set of the TCP session.
In an optional implementation, the TCP session traffic collection module 202 may include: a TCP session packet buffer module and a TCP session ending module. The TCP session packet caching module is responsible for sorting and caching data packets for each TCP session, namely classifying the TCP data packets into the established session, and discarding the data packets if the data packets cannot be classified into the session; the TCP session ending module is responsible for managing the TCP session in the TCP session packet caching module, checking whether a TCP session is ended according to the four-time waving process of the TCP or the number of data packets in the data packet set, and if the TCP session is ended, handing all data packets of the TCP session to the TCP session feature collecting module 203 for processing.
The TCP session characteristic collection module 203 is configured to perform statistical analysis on the data packet set of the TCP session to obtain characteristic information of the TCP session.
The encrypted TCP traffic collection apparatus provided in this embodiment may be used to implement the technical solution of the method embodiment corresponding to fig. 1, and the implementation principle and the technical effect are similar, which are not described herein again.
On the basis of the embodiment shown in fig. 7, the encrypted TCP traffic collecting apparatus provided in this embodiment may further include: the device comprises an acquisition module, a protocol identification module and a processing module. The acquisition module is used for acquiring a first TCP data packet sent by a source after a TCP session is established from a data packet set of the TCP session; the protocol identification module is used for determining an encryption protocol of the TCP session according to the first TCP data packet; and the processing module is used for determining a fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
In an optional implementation manner, the processing module may include a TLS protocol feature acquisition module, an SSH protocol feature acquisition module, and a private protocol feature acquisition module. The TLS protocol feature acquisition module is used for acquiring a ClientHello message, a ServerHello message and a Certificate message from a handshake data packet if an encryption protocol of a TCP session is a TLS protocol, performing fingerprint extraction on the ClientHello message by adopting a first fingerprint extraction algorithm to obtain a fingerprint of the ClientHello message, performing fingerprint extraction on the ServerHello message by adopting a second fingerprint extraction algorithm to obtain a fingerprint of the ServerHello message, analyzing the Certificate message according to an X.509 Certificate format and calculating a SHA-1 fingerprint of the Certificate message, and adding the ClientHello message and a fingerprint corresponding to the ClientHello message, the ServerHello message and a fingerprint corresponding to the Certificate message into a fingerprint library of the TCP session; the SSH protocol characteristic acquisition module is used for acquiring an SSH client key exchange message and an SSH server key exchange message from a handshake data packet if an encryption protocol of a TCP session is an SSH protocol, performing fingerprint extraction on the SSH client key exchange message by adopting a third fingerprint extraction algorithm to obtain a fingerprint of the SSH client key exchange message, performing fingerprint extraction on the SSH server key exchange message by adopting a fourth fingerprint extraction algorithm to obtain a fingerprint of the SSH server key exchange message, and adding the SSH client key exchange message, the fingerprint corresponding to the SSH server key exchange message and the fingerprint corresponding to the SSH server key exchange message into a fingerprint library of the TCP session; the private protocol characteristic acquisition module is used for adding a source first load and a target first load of a TCP session into a fingerprint library of the TCP session if the encryption protocol of the TCP session is a private encryption protocol, wherein the source first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a source, and the target first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a destination.
It should be noted that after the collection of the encrypted TCP flow characteristic information is completed, all data packets of the TCP session cached in the TCP session packet caching module may be cleared, so as to improve the storage efficiency.
The embodiment of the invention also provides encrypted TCP flow acquisition equipment, which can be integrated in a switch and a router or exist independently. Referring to fig. 8, the embodiment of the present invention is illustrated by using fig. 8 as an example, and the present invention is not limited thereto. Fig. 8 is a schematic structural diagram of an embodiment of an encrypted TCP traffic collection device provided in the present invention. As shown in fig. 8, the encrypted TCP traffic collecting device 30 provided in this embodiment may include: memory 301, processor 302, and bus 303. The bus 303 is used to realize connection between the elements.
The memory 301 stores a computer program, and when the computer program is executed by the processor 302, the technical solution of the encrypted TCP traffic collection method provided by any of the above method embodiments may be implemented.
Wherein, the memory 301 and the processor 302 are electrically connected directly or indirectly to realize data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines, such as bus 303. The memory 301 stores a computer program for implementing the encrypted TCP traffic collection method, which includes at least one software functional module that can be stored in the memory 301 in the form of software or firmware, and the processor 302 executes various functional applications and data processing by running the software program and the module stored in the memory 301.
The Memory 301 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 301 is used for storing programs, and the processor 302 executes the programs after receiving the execution instructions. Further, the software programs and modules within the memory 301 may also include an operating system, which may include various software components and/or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.
The processor 302 may be an integrated circuit chip having signal processing capabilities. The Processor 302 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and so on. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. It will be appreciated that the configuration of fig. 8 is merely illustrative and may include more or fewer components than shown in fig. 8 or have a different configuration than shown in fig. 8. The components shown in fig. 8 may be implemented in hardware and/or software.
Reference is made herein to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope hereof. For example, the various operational steps, as well as the components used to perform the operational steps, may be implemented in differing ways depending upon the particular application or consideration of any number of cost functions associated with operation of the system (e.g., one or more steps may be deleted, modified or incorporated into other steps).
Additionally, as will be appreciated by one skilled in the art, the principles herein may be reflected in a computer program product on a computer readable storage medium, which is pre-loaded with computer readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu Ray disks, etc.), flash memory, and/or the like. These computer program instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions which execute on the computer or other programmable data processing apparatus create means for implementing the functions specified. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including means for implementing the function specified. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. For a person skilled in the art to which the invention pertains, several simple deductions, modifications or substitutions may be made according to the idea of the invention.

Claims (10)

1. An encrypted TCP traffic collection method is characterized by comprising the following steps:
identifying a TCP session establishment data packet for establishing a TCP session from the obtained encrypted TCP flow according to a three-way handshake process of TCP;
adding the TCP data packets belonging to the TCP session after the TCP session establishes the data packets into a data packet set of the TCP session until a TCP session ending data packet for ending the TCP session is identified from the obtained encrypted TCP flow according to four hand waving processes of TCP, or the number of the data packets in the data packet set reaches a preset number so as to obtain a data packet set of the TCP session;
and carrying out statistical analysis on the data packet set of the TCP session to obtain the characteristic information of the TCP session.
2. The method of claim 1, wherein the characteristic information comprises:
a source IP address, a source TCP port, a destination IP address, a destination TCP port, a session packet total number, a source packet total size, a destination packet total number, a destination packet total size, a TCP Flag seven-element array, a session start time, and a session end time.
3. The method of claim 1, wherein the method further comprises:
acquiring a first TCP data packet sent by a source after a TCP session is established from a data packet set of the TCP session;
determining an encryption protocol of the TCP session according to the first TCP data packet;
and determining a fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
4. The method of claim 3, wherein if the encryption protocol of the TCP session is TLS protocol, the determining the fingerprint repository for the TCP session based on the encryption protocol of the TCP session and handshake packets comprises:
obtaining a ClientHello message, a ServerHello message and a Certificate message from the handshake data packet;
performing fingerprint extraction on the ClientHello message by adopting a first fingerprint extraction algorithm to obtain a fingerprint of the ClientHello message;
performing fingerprint extraction on the ServerHello message by adopting a second fingerprint extraction algorithm to obtain a fingerprint of the ServerHello message;
analyzing the Certificate message according to an X.509 Certificate format and calculating the SHA-1 fingerprint of the Certificate message;
and adding the ClientHello message and the fingerprint corresponding to the ClientHello message, the ServerHello message and the fingerprint corresponding to the ServerHello message, and the Certificate message and the fingerprint corresponding to the Certificate message into a fingerprint library of the TCP session.
5. The method of claim 3, wherein if the encryption protocol of the TCP session is SSH protocol, the determining the fingerprint library of the TCP session according to the encryption protocol of the TCP session and the handshake data packet comprises:
acquiring an SSH client key exchange message and an SSH server key exchange message from the handshake data packet;
performing fingerprint extraction on the SSH client key exchange message by adopting a third fingerprint extraction algorithm to obtain a fingerprint of the SSH client key exchange message;
performing fingerprint extraction on the SSH server key exchange message by adopting a fourth fingerprint extraction algorithm to obtain a fingerprint of the SSH server key exchange message;
and adding the SSH client key exchange message and the fingerprint corresponding to the SSH client key exchange message and the SSH server key exchange message and the fingerprint corresponding to the SSH server key exchange message into a fingerprint library of the TCP session.
6. The method of claim 3, wherein if the encryption protocol of the TCP session is a private encryption protocol, the determining the fingerprint repository for the TCP session from the encryption protocol and handshake packets of the TCP session comprises:
and adding a source first load and a target first load of the TCP session into a fingerprint library of the TCP session, wherein the source first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a source, and the target first load is a preset number of bytes at a preset position in a first TCP data packet load sent by a destination.
7. An encrypted TCP traffic collection device, comprising:
the TCP session establishment identification module is used for identifying a TCP session establishment data packet used for establishing a TCP session from the obtained encrypted TCP flow according to the three-way handshake process of the TCP;
a TCP session traffic collection module, configured to add a TCP data packet belonging to the TCP session after the TCP session establishes the data packet to a data packet set of the TCP session until a TCP session end data packet for ending the TCP session is identified from the obtained encrypted TCP traffic according to four waving processes of the TCP, or the number of data packets in the data packet set reaches a preset number, so as to obtain a data packet set of the TCP session;
and the TCP session characteristic acquisition module is used for carrying out statistical analysis on the data packet set of the TCP session to acquire the characteristic information of the TCP session.
8. The apparatus of claim 7, further comprising:
the acquisition module is used for acquiring a first TCP data packet sent by a source after a TCP session is established from a data packet set of the TCP session;
the protocol identification module is used for determining an encryption protocol of the TCP session according to the first TCP data packet;
and the processing module is used for determining the fingerprint library of the TCP session according to the encryption protocol and the handshake data packet of the TCP session.
9. An encrypted TCP traffic collection apparatus, comprising: at least one processor and memory;
the memory is used for storing programs;
the at least one processor is configured to implement the encrypted TCP traffic collection method of any of claims 1-6 by executing a program stored in the memory.
10. A computer-readable storage medium, characterized in that the medium has stored thereon a program executable by a processor to implement the encrypted TCP traffic collection method according to any one of claims 1 to 6.
CN202110577572.6A 2021-05-26 2021-05-26 Encrypted TCP (Transmission control protocol) traffic acquisition method and device Pending CN113315678A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110577572.6A CN113315678A (en) 2021-05-26 2021-05-26 Encrypted TCP (Transmission control protocol) traffic acquisition method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110577572.6A CN113315678A (en) 2021-05-26 2021-05-26 Encrypted TCP (Transmission control protocol) traffic acquisition method and device

Publications (1)

Publication Number Publication Date
CN113315678A true CN113315678A (en) 2021-08-27

Family

ID=77374838

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110577572.6A Pending CN113315678A (en) 2021-05-26 2021-05-26 Encrypted TCP (Transmission control protocol) traffic acquisition method and device

Country Status (1)

Country Link
CN (1) CN113315678A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115037537A (en) * 2022-06-06 2022-09-09 恒安嘉新（北京）科技股份公司 Abnormal traffic interception and abnormal domain name identification method, device, equipment and medium
CN115348337A (en) * 2022-07-11 2022-11-15 广州市玄武无线科技股份有限公司 TCP data packet analysis method and device based on multiple protocols

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106330611A (en) * 2016-08-31 2017-01-11 哈尔滨工业大学（威海） Anonymous protocol classification method based on statistical feature classification
CN108289125A (en) * 2018-01-26 2018-07-17 华南理工大学 TCP sessions recombination based on Stream Processing and statistical data extracting method
CN111277578A (en) * 2020-01-14 2020-06-12 西安电子科技大学 Encrypted flow analysis feature extraction method, system, storage medium and security device
CN112019574A (en) * 2020-10-22 2020-12-01 腾讯科技（深圳）有限公司 Abnormal network data detection method and device, computer equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106330611A (en) * 2016-08-31 2017-01-11 哈尔滨工业大学（威海） Anonymous protocol classification method based on statistical feature classification
CN108289125A (en) * 2018-01-26 2018-07-17 华南理工大学 TCP sessions recombination based on Stream Processing and statistical data extracting method
CN111277578A (en) * 2020-01-14 2020-06-12 西安电子科技大学 Encrypted flow analysis feature extraction method, system, storage medium and security device
CN112019574A (en) * 2020-10-22 2020-12-01 腾讯科技（深圳）有限公司 Abnormal network data detection method and device, computer equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
潘吴斌等: "网络加密流量识别研究综述及展望", 《通信学报》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115037537A (en) * 2022-06-06 2022-09-09 恒安嘉新（北京）科技股份公司 Abnormal traffic interception and abnormal domain name identification method, device, equipment and medium
CN115348337A (en) * 2022-07-11 2022-11-15 广州市玄武无线科技股份有限公司 TCP data packet analysis method and device based on multiple protocols

Similar Documents

Publication Publication Date Title
US11800260B2 (en) Network telemetry with byte distribution and cryptographic protocol data elements
Wang et al. Seeing through network-protocol obfuscation
US20240089301A1 (en) Method and system for capture of visited links from encrypted and non-encrypted network traffic
US9660833B2 (en) Application identification in records of network flows
US20150264072A1 (en) System and Method for Extracting and Preserving Metadata for Analyzing Network Communications
CN113315678A (en) Encrypted TCP (Transmission control protocol) traffic acquisition method and device
CN109450895B (en) Traffic identification method, traffic identification device, server and storage medium
JP6502902B2 (en) Attack detection device, attack detection system and attack detection method
WO2020252897A1 (en) Distributed link data authentication method, device and apparatus, and storage medium
CN113574841A (en) Information processing method and device, equipment and storage medium
Liu et al. Maldetect: A structure of encrypted malware traffic detection
US8010787B2 (en) Communication device, communication log transmitting method suitable for communication device, and communication system
CN113872956A (en) Method and system for inspecting IPSEC VPN transmission content
US20140068761A1 (en) Abuse identification of front-end based services
CN116723238A (en) API encrypted flow collection and labeling method based on man-in-the-middle agent
KR101996044B1 (en) ICAP protocol extension method for providing network forensic service of encrypted traffic, network forensic device supporting it and web proxy
KR101865690B1 (en) security monitoring system and method of network for visibility of HTTPS-based connection
WO2022104738A1 (en) Trojan detection method and apparatus, and device
CN114117429A (en) Network flow detection method and device
CN114301802A (en) Confidential evaluation detection method and device and electronic equipment
CN112929357A (en) Virtual machine data analysis method, device, equipment and storage medium
KR101919762B1 (en) An encrypted traffic management apparatus and method for decrypting encrypted traffics
Lokesh et al. A review on analysis of transport layer security in open quantum safe cryptographic algorithm
CN117395023A (en) Network equipment identification method and device for encryption gateway
Al Sadi et al. Unleashing Dynamic Pipeline Reconfiguration of P4 Switches for Efficient Network Monitoring

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210827

RJ01 Rejection of invention patent application after publication