CN107124385B - Mirror flow-based SSL/TLS protocol plaintext data acquisition method - Google Patents

Abstract

The present invention provides a method for collecting plaintext data of SSL/TLS protocol based on mirror stream. The method includes: step 1) receiving mirrored SSL/TLS data packets, extracting records in the data packets, and generating several complete records , put into the record queue; step 2) extract several complete messages from the records in the record queue and put them into the message queue; step 3) parse the messages in the message queue to obtain plaintext data. According to the encapsulation format of the SSL/TLS protocol, the method of the present invention splices data packets into a completed record, further extracts a complete single message before processing, and determines whether the server combines multiple messages into one record or encapsulates a single message into multiple records. There is no requirement, and new messages that need to be parsed can be flexibly selected and added, and the expansibility is good; the method of the invention processes the mirror data of the switch to obtain plaintext data, does not interfere with the original business of the system, and does not affect the performance of the system.

Description

A method for collecting plaintext data of SSL/TLS protocol based on image stream

technical field

The invention belongs to the technical field of network security communication, and in particular relates to a method for collecting plaintext data of SSL/TLS protocol based on mirror stream.

Background technique

The SSL protocol and its successor, the TLS protocol, are a security protocol that provides security and data integrity for network security. The SSL/TLS protocol is located between the TCP/IP protocol and the application layer protocol, and can provide security guarantees for various application layer protocols, such as FTP and TELNET protocols. Currently, the most widely used SSL/TLS protocol is to protect the security of the HTTP protocol. The SSL/TLS protocol consists of two layers: the record layer protocol and the handshake protocol. The record protocol provides basic security services for the high-level handshake protocol to ensure data integrity, including compression and decompression, encryption and decryption, calculation and verification of MAC, etc. The handshake layer protocol includes a handshake protocol, a password parameter modification protocol, an alarm protocol, and an application data protocol.

The high-level protocol protected by the SSL/TLS protocol transmits ciphertext data between the client and the server, and does not appear in plaintext, which brings difficulties to data auditing. Some technical means need to be taken to decrypt the ciphertext data in the SSL/TLS communication process into plaintext data, and then parse and audit the corresponding plaintext data. As shown in Figure 1, in the prior art, an SSL/TLS proxy server is usually introduced between the client and the server, the proxy server is connected in series between the client and the server, and two SSL/TLS connections are established with the client and the server respectively. . Since the SSL/TLS proxy server needs to obtain the data, decrypt it to obtain the plaintext data, and then encrypt the plaintext data and send it to the client. The additional encryption operation brings a burden to the system, resulting in a longer system response time and a lower throughput rate.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the above-mentioned defects existing when decrypting ciphertext data into plaintext data in the current SSL/TLS communication process, and proposes a method for collecting plaintext data of SSL/TLS protocol based on mirror stream, through which the The SSL/TLS proxy server directly uses the legally held server certificate and private key to parse the ciphertext data of the SSL/TLS protocol, thereby directly obtaining plaintext data, reducing intermediate links and improving the efficiency of plaintext data acquisition.

In order to achieve the above object, the present invention proposes a method for collecting plaintext data of SSL/TLS protocol based on mirror stream, the method includes:

Step 1) receive the mirrored SSL/TLS data packets, extract the records in the data packets, generate several complete records, and put them into the record queue;

Step 2) extract several complete messages from the records in the record queue and put them into the message queue;

Step 3) Parse the message in the message queue to obtain plaintext data.

In the above technical solution, the step 1) specifically includes:

Step 101) receive the mirrored SSL/TLS packet;

Step 102) extract the first record from the data packet, check whether the record buffer area has a buffer record, if there is no buffer record, then directly calculate the length of the first record received; if there is a buffer record in the record buffer area, Then, after splicing the data packet to the cache record, calculate the length of the first record in the current cache;

Step 103) compare the length of the first record with the length of the data packet, if the length of the first record adds the length of the record header and is equal to the length of the data packet, go to step 104); if the length of the first record adds the length of the record head is less than the length of the data packet, go to step 105); if the length of the first record plus the length of the record header is greater than the length of the data packet, go to step 106);

Step 104) put this record into the record queue;

Step 105) split the data packet, cyclically extract a single complete record and put it into the record queue, and put the last incomplete record into the record buffer area;

Step 106) put this record into the record buffer area; go to step 101);

Incomplete records placed in the record buffer wait for subsequent packets to be spliced, and then complete records are generated.

In the above technical solution, the step 2) specifically includes:

Step 201) take out a record from the record queue as the current record,

Step 202) Check the type of the current record, if the current record type is the application data protocol, the change password specification protocol or the alarm message, go to step 203); if the current record type is the handshake protocol, go to step 204);

Step 203) is currently recorded as a single complete message, put into the message queue;

Step 204) check whether there is a cached message in the message buffer area, if there is no cached message, then directly calculate the length of the first message in the current record; if there is a cached message in the cache, then splicing the current record to the cached message, calculate The length of the first message currently cached;

Step 205) compare the length of the first message of the calculation with the current record length, if the length of the first message plus the header length is equal to the current record length, go to step 206); if the length of the first message adds the message The header length is less than the current record length, go to step 207); If the length of the first message plus the message header length is greater than the current record length, go to step 208);

Step 206) is currently recorded as a single complete message, put into the message queue;

Step 207) contains multiple messages in the current record, then this record is split, and several complete messages are extracted circularly and put into the message queue; and the last incomplete message is put into the message buffer area;

Step 208) is currently recorded as an incomplete message, put into the message buffer area, and goes to step 201);

Incomplete messages put into the message buffer wait for subsequent records to be spliced, and then complete messages are generated.

In the above-mentioned technical scheme, the concrete realization process of described step 3) is:

A message is taken from the message queue. If the message type is a handshake protocol, the cipher specification and secret key information are extracted; if the message type is a change cipher protocol specification, the following records will be protected by the newly negotiated cipher specification and secret key. ; If the message type is an alarm message, analyze the severity and alarm description of the message; if the message type is an application data protocol, decrypt the transmitted data using the extracted cipher specification and secret key to obtain plaintext data.

In the above-mentioned technical solution, the handshake protocol includes: ClientHello, ServerHello, SeverCertificate, ServerHelloDone, ClientKeyExange, NewSessionTicket and Finished, and the specific steps for extracting password specification and secret key information are:

Step 301) if the handshake protocol type is ClientHello, then record the client random number, session_id and session_ticket;

Step 302) If the handshake protocol type is ServerHello, then judge whether session reuse occurs, if session reuse occurs, then extract the master key and encryption algorithm specification from the cached session information, and generate a password parameter; if session reuse does not occur, then record Protocol version, server random number, session_id, cipher suite and compression algorithm;

Step 303) if the handshake protocol type is SeverCertificate, then extract the server public key, search for a matching certificate to obtain the server private key;

Step 304) if the handshake protocol type is ServerHelloDone, then the server has completed the key exchange message at this time;

Step 305) if the handshake protocol type is ClientKeyExange, then utilize the server to decrypt the pre-master key, calculate the master key, and generate security parameters; if the session is a new session, then the session information is cached;

Step 306) if the handshake protocol type is NewSessionTicket, if there is no session reuse at this time, then fill in the session_tiket item in the cached session information, if session reuse occurs, then update the session_tieket item in the cached session information;

Step 307) If the type of the handshake protocol is Finished, verify all the handshake data in the corresponding direction, and after both parties pass the verification, the application data transmission can be started.

In the above technical solution, the step 3) further includes: after acquiring the plaintext data, parsing the plaintext data, generating and storing an audit log.

Compared with the prior art, the advantages of the present invention are:

1. According to the encapsulation format of the SSL/TLS protocol, the method of the present invention splices data packets into a completed record, further extracts a complete single message, and then processes it, whether the server synthesizes multiple messages into one record or encapsulates a single message into multiple There is no requirement in each record, and new messages that need to be parsed can be flexibly selected, and the scalability is good;

2. The method of the present invention processes the mirror data of the switch to obtain plaintext data. Compared with the traditional method of introducing a proxy server, the method of the present invention does not interfere with the original business of the system and does not affect the performance of the system.

Description of drawings

Fig. 1 is a schematic diagram of an existing SSL/TLS protocol plaintext data acquisition system;

Fig. 2 is the flow chart of the SSL/TLS protocol plaintext data collection method based on mirror stream of the present invention;

FIG. 3 is a schematic diagram of an image stream-based SSL/TLS protocol plaintext data collection system according to an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

As shown in Figure 2, a method for collecting plaintext data of SSL/TLS protocol based on image stream, the method includes:

Step 1) Receive the mirrored SSL/TLS data packet, extract the records in the data packet, generate several complete records, and put them into the record queue; specifically include:

Step 101) receive the mirrored SSL/TLS packet;

Step 102) extract the first record from the data packet, check whether the record buffer area has a buffer record, if there is no buffer record, then directly calculate the length of the first record received; if there is a buffer record in the record buffer area, Then, after splicing the data packet to the cache record, calculate the length of the first record in the current cache;

The SSL/TLS protocol record encapsulation format is shown in Table 1, in which the record header has a total of 5 bytes:

Table 1

Step 103) compare the length of the first record with the length of the data packet, if the length of the first record adds the length of the record header and is equal to the length of the data packet, go to step 104); if the length of the first record adds the length of the record head is less than the length of the data packet, go to step 105); if the length of the first record plus the length of the record header is greater than the length of the data packet, go to step 106);

Step 104) put this record into the record queue;

Records placed in the record queue are full records.

Step 105) split the data packet, cyclically extract a single complete record and put it into the record queue, and put the last incomplete record into the record buffer area;

Step 106) put this record into the record buffer area; go to step 101);

Incomplete records placed in the record buffer wait for subsequent packets to be spliced, and then complete records are generated.

Step 2) extract several complete messages from the records in the record queue and put them into the message queue;

Since the SSL/TLS protocol record is not used as the boundary of the message, after the complete record is extracted, the complete message needs to be further extracted, and the step 2) specifically includes:

Step 201) take out a record from the record queue as the current record,

Step 202) Check the type of the current record, if the current record type is the application data protocol, the change password specification protocol or the alarm message, go to step 203); if the current record type is the handshake protocol, go to step 204);

The format of the SSL/TLS record handshake message is shown in Table 2, where the message header has a total of 4 bytes:

Table 2

Step 203) is currently recorded as a single complete message, put into the message queue;

Step 204) check whether there is a cached message in the message buffer area, if there is no cached message, then directly calculate the length of the first message in the current record; if there is a cached message in the cache, then splicing the current record to the cached message, calculate The length of the first message currently cached;

Step 205) compare the length of the first message of the calculation with the current record length, if the length of the first message plus the header length is equal to the current record length, go to step 206); if the length of the first message adds the message The header length is less than the current record length, go to step 207); If the length of the first message plus the message header length is greater than the current record length, go to step 208);

Step 206) is currently recorded as a single complete message, put into the message queue;

Step 207) contains multiple messages in the current record, then this record is split, and several complete messages are extracted circularly and put into the message queue; and the last incomplete message is put into the message buffer area;

Step 208) is currently recorded as an incomplete message, put into the message buffer area, and goes to step 201);

Incomplete messages put into the message buffer wait for subsequent records to be spliced, and then complete messages are generated.

Step 3) parse the message in the message queue to obtain plaintext data;

A message is taken from the message queue. If the message type is a handshake protocol, the cipher specification and secret key information are extracted; if the message type is a change cipher protocol specification, the following records will be protected by the newly negotiated cipher specification and secret key. ; If the message type is an alarm message, analyze the severity and alarm description of the message; if the message type is an application data protocol, decrypt the transmitted data using the extracted cipher specification and secret key to obtain plaintext data.

The handshake protocol includes: ClientHello, ServerHello, SeverCertificate, ServerHelloDone, ClientKeyExange, NewSessionTicket and Finished, and the specific steps for extracting the password specification and secret key information are:

Step 301) if the handshake protocol type is ClientHello, then record the client random number, session_id and session_ticket;

Step 302) If the handshake protocol type is ServerHello, then judge whether session reuse occurs, if session reuse occurs, then extract the master key and encryption algorithm specification from the cached session information, and generate a password parameter; if session reuse does not occur, then record Protocol version, server random number, session_id, cipher suite and compression algorithm;

Step 303) if the handshake protocol type is SeverCertificate, then extract the server public key, search for a matching certificate to obtain the server private key;

Step 304) if the handshake protocol type is ServerHelloDone, then the server has completed the key exchange message at this time;

Step 305) if the handshake protocol type is ClientKeyExange, then utilize the server to decrypt the pre-master key, calculate the master key, and generate security parameters; if the session is a new session, then the session information is cached;

Step 306) if the handshake protocol type is NewSessionTicket, if there is no session reuse at this time, then fill in the session_tiket item in the cached session information, if session reuse occurs, then update the session_tieket item in the cached session information;

Step 307) If the type of the handshake protocol is Finished, verify all the handshake data in the corresponding direction, and after both parties pass the verification, the application data transmission can be started.

In this embodiment, the plaintext data is HTTP data, and the plaintext data is parsed, and fields such as URL are obtained to generate an audit log and save it for subsequent auditing by auditors.

As shown in FIG. 3 , through the method of the present invention, the client and the server establish an SSL/TLS connection, the acquisition device uses the switch mirror to obtain the interactive data between the client and the server, and the acquisition device legally holds the server certificate and private key.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified. or equivalent replacement, without departing from the spirit and scope of the technical solutions of the present invention, and should be included in the scope of the claims of the present invention.

Claims (5)

1. A mirror flow-based SSL/TLS protocol plaintext data collection method, the method comprising:

step 1) receiving a mirrored SSL/TLS data packet, extracting records in the data packet, generating a plurality of complete records, and putting the records into a record queue;

step 2) extracting a plurality of complete messages from the records in the record queue and putting the messages into a message queue;

step 3) analyzing the messages in the message queue to obtain plaintext data;

the step 1) specifically comprises the following steps:

step 101) receiving a mirrored SSL/TLS data packet;

step 102) extracting a first record from the data packet, checking whether a record cache region has a cache record, and directly calculating the length of the received first record if the record cache region does not have the cache record; if the record cache region has cache records, splicing the data packet to the cache records, and calculating the length of the first record of the current cache;

step 103) comparing the length of the first record with the length of the data packet, and if the length of the first record plus the length of the recording head is equal to the length of the data packet, turning to step 104); if the length of the first record and the length of the recording head are smaller than the length of the data packet, the step 105) is carried out; if the length of the first record plus the length of the recording head is larger than the length of the data packet, the step 106) is carried out;

step 104) placing the record in a record queue;

step 105) splitting the data packet, circularly extracting a single complete record and putting the single complete record into a record queue, and putting the last incomplete record into a record cache region;

step 106) placing the record in a record buffer area; turning to step 101);

and the incomplete records placed in the record buffer area wait for splicing of subsequent data packets, and then complete records are generated.

2. The image flow-based SSL/TLS protocol plaintext data collection method according to claim 1, wherein the step 2) specifically includes:

step 201) takes a record from the record queue as the current record,

step 202) checking the type of the current record, and if the type of the current record is an application data protocol, a password specification changing protocol or an alarm message, turning to step 203); if the current record type is a handshake protocol, go to step 204);

step 203) recording the current information as a single complete information, and putting the information into an information queue;

step 204) checking whether a message cache region has cache messages or not, and if no cache message exists, directly calculating the length of a first message in the current record; if the cache message exists in the cache, the length of the first message of the current cache is calculated after the current record is spliced to the cache message;

step 205) comparing the calculated length of the first message with the current recording length, and if the length of the first message plus the length of the message header is equal to the current recording length, turning to step 206); if the length of the first message plus the length of the message header is smaller than the current recording length, go to step 207); if the length of the first message plus the length of the message header is greater than the current record length, go to step 208);

step 206), currently recording the message as a single complete message, and placing the message into a message queue;

step 207) if the current record contains a plurality of messages, splitting the record, circularly extracting a plurality of complete messages and putting the messages into a message queue; putting the final incomplete message into a message buffer area;

step 208), recording the current message as an incomplete message, putting the message into a message buffer area, and switching to step 201);

the incomplete message put into the message buffer area waits for the subsequent records to be spliced, and then the complete message is generated.

3. The image flow-based SSL/TLS protocol plaintext data collection method according to claim 1 or 2, wherein the step 3) is implemented by:

taking out a message from the message queue, and if the message type is a handshake protocol, extracting the password specification and the key information; if the message type is the cipher protocol specification changing, the following records are protected by the newly negotiated cipher specification and the secret key; if the message type is the alarm message, analyzing the severity and the alarm description of the message; and if the message type is the application data protocol, decrypting the transmission data by using the extracted password specification and the key to obtain plaintext data.

4. The image flow-based SSL/TLS protocol plaintext data collection method according to claim 3, wherein the handshake protocol comprises: ClientHello, ServerHello, SeverCertificate, serverhellolodone, clientkeyExange, NewSessionsTicket and Finished, wherein the specific steps of extracting the password specification and the key information are as follows:

step 301) if the handshake protocol type is ClientHello, recording a client random number, a session _ id and a session _ ticket;

step 302), if the handshake protocol type is ServerHello, judging whether session reuse occurs, if the session reuse occurs, extracting a master key and an encryption algorithm specification from the cached session information, and generating a password parameter; if the session reuse does not occur, recording a protocol version, a server random number, a sesseion _ id, an encryption suite and a compression algorithm;

step 303) if the handshake protocol type is the SeverCertification, extracting a server public key, and searching a matching certificate to obtain a server private key;

step 304), if the handshake protocol type is serverhellododone, the server has completed key exchange message at this time;

step 305), if the handshake protocol type is ClientKeyExange, decrypting the pre-master key by using the server, calculating the master key and generating a security parameter; if the conversation is a new conversation, caching the conversation information;

step 306), if the handshake protocol type is NewSessionTicket, if no session reuse occurs at this time, filling the session _ token item in the cached session information, and if the session reuse occurs, updating the session _ token item in the cached session information;

step 307) if the type of the handshake protocol is Finished, verifying all handshake data in the corresponding direction, and after the two parties pass the verification, starting application data transmission.

5. The image flow-based SSL/TLS protocol plaintext data collection method according to claim 1, wherein the step 3) further comprises: and after the plaintext data is obtained, analyzing the plaintext data, generating an audit log and storing the audit log.

Images