CN108924138B

CN108924138B - Method for realizing TCP proxy complete transparency

Info

Publication number: CN108924138B
Application number: CN201810731684.0A
Authority: CN
Inventors: 刘佳; 范渊; 吴永越; 郑学新; 刘韬
Original assignee: Chengdu DBAPPSecurity Co Ltd
Current assignee: Chengdu DBAPPSecurity Co Ltd
Priority date: 2018-07-05
Filing date: 2018-07-05
Publication date: 2020-10-23
Anticipated expiration: 2038-07-05
Also published as: CN108924138A

Abstract

The invention discloses a method for realizing TCP proxy complete transparency, a TCP proxy Server intercepts and caches a TCP Syn message flowing from a Client to a Server to realize MAC address transparency; the TCP proxy Server checks and copies the header field of the message sent between the Client and the Server, so as to realize the transparency of the TOS and TTL fields of the IP header; the TCP proxy server establishes network namespace for the Client, and TCP port transparency is realized. The invention simultaneously realizes MAC address transparency, IP header TOS/TTL field transparency and TCP header port field transparency.

Description

Method for realizing TCP proxy complete transparency

Technical Field

The invention relates to the technical field of network security, in particular to a method for realizing complete transparency of a TCP (transmission control protocol) proxy.

Background

At present, security devices widely applied to enterprise networks, such as behavior management devices and Web firewalls, need to perform deep analysis processing on TCP traffic. The security device is typically deployed in the user's network as a "man-in-the-middle" role, and the granularity of the security device's control over the TCP connection is weak if it is not proxied.

For example, a non-proxy WAF may have difficulty modifying certain HTTP session messages without affecting other messages in the TCP stream. To achieve strong control granularity, TCP proxy technology is inevitably used. There are many kinds of TCP proxies including forward proxies, reverse proxies, transparent proxies, and the like. The forward proxy needs to configure the client and is not generally applicable to the use scenario of the network security equipment; the reverse proxy means that the client does not sense the proxy device, and from the network perspective, the proxy device should replace the network location of the original server, and the reverse proxy may be transparent or opaque, and if transparent, has minimal impact on the user network configuration and service. Therefore, the transparent proxy has a very wide application prospect in network security equipment.

In fact, the TCP transparent proxy is widely applied to security devices, because users generally want the security devices not to change the originally deployed network environment, not to affect the original service, and not to introduce complex management. However, most of the security devices on the market currently do not implement a true TCP transparent proxy, that is, MAC address transparency, IP header transparency, and TCP port transparency are satisfied at the same time, but only IP address transparency is implemented. First, if the MAC address is not transparent, which would not allow the intermediate device to function as a bridge in the customer network topology, the customer network would need to be modified. Some transparent proxy products complete proxy work by adopting a network bridge packet capturing mode, and try to achieve the purpose of MAC address transparency, but the first connection still cannot achieve MAC address transparency without Syn cache. The reason is that when the proxy device replies the TCP connection request of the Client, the Server does not send the message to the intermediate device. The opaqueness of the TOS and TTL fields of the IP header may cause the user's diagnostic tool to fail to work properly or the QoS policy to be biased. The TCP port is not transparent, which may cause deviation of the load balancing policy of the network device of the Server or the TCP Server itself, and may also cause a judgment error of some user policies. Therefore, in many scenarios, the proxy device will still affect the network traffic of the user.

TCP proxy software exists on the market, such as haproxy, and the traditional transparency mainly refers to the transparency of IP addresses. The method comprises the steps that after receiving a TCP connection request of a Client, the intermediate device directly pretends to be a Server to respond, establishes the connection between the Client and the intermediate person, constructs a message according to the request of the Client and sends the message to a Server, and establishes the connection with the Server. And after the bidirectional connection is established, the data content is proxied. This proxy approach does not enable MAC address transparency, TOS and TTL transparency, and TCP port transparency. Some vendors have attempted to achieve TCP proxy MAC address transparency by introducing Bridge in the proxy device. The method comprises the following steps:

the ARP message is directly forwarded through Bridge, so that a Client and a Server can learn the Mac address of the other side;

2. when the agent equipment receives the message sent by the Client, the target Mac address of the agent equipment is recorded;

3. when the agent replies to the Client, replacing the source MAC address in the message by the MAC address recorded in the step 2, and pretending to be the Mac address of the Server;

4. when the proxy equipment receives the message sent by the Server, the destination Mac address of the proxy equipment is recorded;

5. and when the proxy replies the Server, replacing the source Mac address in the message by the Mac address recorded in the 4, and pretending to be the Mac address of the Client.

This approach has a problem: the source Mac address used by the Server's datagram may not be the Mac address learned by the Client. One typical scenario is a Server-connected router of a proxy device, on which an ARP proxy is opened. In another scenario, the Server of the proxy device is connected to a load-balanced switch, and the switch balances data to two routers. The source Mac address in step 3 is still replaced with the destination Mac address of the message replied by the Server, because the Server end does not send the reply message when the Client establishes the first connection with the TCP proxy device in the conventional scheme. The MAC address transparency scheme of this patent will solve this problem. The invention discloses a method for realizing a completely transparent TCP proxy, which eliminates the adverse effect of the incompletely transparent proxy on the network service of a user.

Disclosure of Invention

The invention aims to provide a method for realizing complete transparency of a TCP (transmission control protocol) agent, which is used for realizing the Mac transparency of the TCP agent based on Tcp Syn message caching and MAC (media access control) address delayed learning; through copying the Client data, the transparency of the TOS/TTL field in the TCP agent message header is kept; the network namespace is established through the network name space technology, and port transparency of the TCP agent is realized.

The invention is realized by the following technical scheme: a method for realizing TCP proxy complete transparency specifically comprises the following steps:

step F1: the TCP proxy Server intercepts and caches a TCP Syn message flowing from the Client to the Server, so that the MAC address is transparent;

step F2: the TCP proxy Server checks and copies the header field of the message sent between the Client and the Server, so as to realize the transparency of the TOS and TTL fields of the IP header;

step F3: the TCP proxy server establishes network namespace for the Client, and TCP port transparency is realized.

Further, in order to better implement the present invention, the TCP proxy server includes a TCP proxy module, a Syn _ handler module, and a Bridge, which are connected to each other, and the TCP proxy module includes a Tcpproxy _ client, a Tcpproxy _ server, and an Fd _ binder;

the step F1 specifically includes the following steps:

step F101: the Client sends a Tcp Syn message request, tries to establish connection with the Server, and learns and records the source MAC address of the Client by the Bridge;

step F102: a Tcp Syn message sent by the Client reaches a TCP proxy server, and is intercepted and cached by a Syn _ handler module;

step F103: the Syn _ handler module sends a message to the TCP agent module, informs the TCP agent module to record intercepted Tcp Syn message quintuple information and initiates TCP connection with a Server;

step F104: the Tcppproxy _ client in the TCP proxy module sends a Tcp Syn message to the Server through the Bridge and tries to establish connection with the Server;

step F105: the Bridge in the TCP proxy Server forwards a Tcp Syn message sent by a Tcppproxy _ Client to the Server, at the moment, the source MAC address of the message sent by the TCP proxy Server is changed into the MAC address of the Client, and the Tcppproxy _ Client pretends to be the Client;

step F106: the Server replies and establishes connection with a TCP proxy module handshake through Bridge, and the Bridge performs two-layer information learning through the message replied by the Server and records the source MAC address of the Server;

step F107: the Bridge forwards the message replied by the Server to the Tcpproxy _ Client, and meanwhile, the TCP proxy module sends a confirmation to the Bridge and obtains a file descriptor Client fd connected with the Server;

step F108: the Tcpproxy _ Client of the TCP agent module submits a connected file descriptor Client Fd to the Fd _ binder;

step F109: the TCP agent module informs a Syn _ handler module to release the original Tcp Syn message sent by the Client in the step F102;

step F110: the Syn _ handler module submits an original Tcp Syn message sent by a Client to a TCP agent module;

step F111: the TCP proxy module replies an ACK confirmation message to the Client, and through the interactive handshake between Bridge and the Client, all interactive message source MAC addresses sent by the TCP proxy Server are modified into the MAC address of the Server, and at the moment, the connection between the TCP proxy Server and the Client is established, so that a file descriptor Server fd connected with the Client is obtained;

step F112: the Tcpproxy _ Server of the TCP agent module submits a connection file descriptor Server Fd to an Fd _ binder, and the Fd _ binder associates and binds the file descriptor Server Fd and the file descriptor Client Fd to form an Fd association set according to five-tuple information;

step F113: and the Tcppproxy _ server and Tcppproxy _ client forward data through the Fd association group bound by the Fd _ binder.

Further, in order to better implement the present invention, the step F2 specifically includes the following steps:

step F201: the TCP proxy Server checks the header field of the message sent by the Client to the Server;

step F202: the TOS and TTL values of the header field of the message in step F201 are copied by the Tcpproxy _ client to a request message sent by the TCP proxy Server to the Server;

step F203: the TCP proxy Server checks the header field of the message sent by the Server to the Client;

step F204: the TOS and TTL values of the header field of the message in step F203 are copied by the Tcpproxy _ server to a request message sent to the Client by the TCP proxy server.

Furthermore, in order to better implement the present invention, a plurality of clients are provided and are respectively connected with the TCP proxy server, a plurality of network namespaces are provided and correspond to the plurality of clients one by one, and the network namespaces are respectively connected with the Bridge;

the step F3 specifically includes the following steps:

step F301: any Client initiates a Tcp Syn message request which is intercepted by a TCP proxy server;

step F302: the TCP proxy server checks whether the received Tcp Syn message belongs to a certain existing Client, and if so, finds the network namespace corresponding to the Client; if not, establishing a corresponding network namespace for the Client;

step F303: f301, the Tcp Syn message sent by the Client is sent to a corresponding network space for processing;

step F304: the TCP proxy module process establishes connection and communication with the Server through the network namespace in step F303, and the IP address and the TCP source port used by the TCP proxy _ Client are completely consistent with those of the Client in step F301.

The working principle is as follows:

the TCP proxy Server intercepts and forwards the data stream and the control stream flowing to the Server from the Client, so as to realize MAC address transparency.

And 2, the TCP proxy Server checks and copies the header field of the message sent between the Client and the Server, so as to realize the transparency of the TOS/TTL field of the IP header.

And 3, the TCP proxy server establishes network namespace for the Client to realize TCP port transparency.

Compared with the prior art, the invention has the following advantages and beneficial effects:

(1) the invention realizes the transparency of the MAC address;

(2) the invention realizes the transparency of the TOS/TTL field of the IP header;

(3) the invention realizes TCP header port field transparency.

Drawings

FIG. 1 is a diagram of MAC address transparent proxy data and control flow;

FIG. 2 is a schematic diagram illustrating the TOS/TTL field transparency principle;

fig. 3 is a diagram of a TCP port transparent proxy.

Detailed Description

The present invention will be described in further detail with reference to examples, but the embodiments of the present invention are not limited thereto.

Example 1:

the invention is realized by the following technical scheme, as shown in fig. 1-3, a method for realizing complete transparency of a TCP proxy specifically comprises the following steps:

step F2: the TCP proxy Server checks and copies the header field of the message sent between the Client and the Server, and realizes the transparency of the TOS/TTL field of the IP header;

It should be noted that, with the above improvement, the Client is a proxied TCP Client, and the Server is a proxied Server. The invention provides a method for realizing TCP proxy complete transparency, which is based on the principle that response to a client is delayed by utilizing Tcp Syn message cache, a Mac address of a server is obtained first, and the problem that an MAC address is not transparent in a handshake stage is solved, so that the MAC address transparency is realized.

And copying fields transmitted between the client and the server to realize the transparency of TTL and TOS fields in the IP header.

And (3) utilizing a network name space technology, namely establishing network namespace for the Client in the TCP proxy server, and realizing port number transparency in the TCP header.

Other parts of this embodiment are the same as those of the above embodiment, and thus are not described again.

Example 2:

the embodiment is further optimized on the basis of the above embodiment, and as shown in fig. 1, the TCP proxy server includes a TCP proxy module, a Syn _ handler module, and a Bridge that are connected to each other, where the TCP proxy module includes Tcpproxy _ client, Tcpproxy _ server, and Fd _ binder;

the step F1 specifically includes the following steps:

It should be noted that, with the above modification, the steps F103, F108, F109, and F122 are descriptions of control flows, and the remaining steps are descriptions of data flows. The principle of the embodiment is that the response to the client is delayed by utilizing the Tcp Syn message cache, the Mac address of the server is obtained first, and the problem that the MAC address is not transparent in the handshaking stage is solved, so that the MAC address is transparent.

The TCP proxy server comprises a TCP proxy module, a Syn _ handler module and a Bridge which are connected with each other, wherein the Bridge refers to a switch supporting the traditional network two-layer forwarding. The TCP agent module comprises a Tcpproxy _ client, a Tcpproxy _ server and an Fd _ binder which are connected with each other. The Client side sends a Tcp Syn message request to the Server through the TCP proxy Server, tries to establish connection with the Server, and firstly passes through Bridge which is used for learning and recording the source MAC address of the Client. The Tcp Syn message sent by the Client reaches the Syn _ handler module through the Bridge, and is intercepted and cached by the Syn _ handler module. The Syn _ handler module sends a message to a Tcpproxy _ server of the TCP agent module to inform the Tcpproxy _ server to record quintuple information of an intercepted Tcp Syn message, wherein the quintuple information comprises a source IP address, a source port, a destination IP address, a destination port and a transmission layer protocol. The TCP agent module sends Tcp Syn message to the Server through Bridge, and tries to establish connection with the Server. The Tcp Syn message sent by the Tcppproxy _ Client of the TCP proxy module reaches the Server, the source MAC addresses of all interactive messages sent by the TCP proxy Server are changed into the MAC addresses of the Client, the Tcppproxy _ Client pretends to be the Client and communicates with the Server, and all interactive messages sent by the TCP proxy Server comprise the Tcp Syn message and the ACK confirmation message.

The Server replies after receiving the message, handshake with the TCP proxy module and establish connection, and the Bridge performs two-layer information learning and records the source MAC address of the Server through the message replied by the Server. The Bridge forwards a reply message of the Server to the Tcppproxy _ Client, the TCP proxy module sends a message for determining the receipt of the reply message to the Bridge and obtains a file descriptor Client Fd connected with the Server, then the Tcppproxy _ Client submits the file descriptor Client Fd to the Fd _ binder, and the TCP proxy module informs the Syn _ handler module to release the original Tcp Syn message sent by the Client. And the Syn _ handler module refers the original Tcp Syn message sent by the Client to the TCP agent module, the TCP agent module replies an ACK (acknowledgement) message to the Client, and the Client interacts a handshake message through Bridge.

In the same way, when the Tcp Syn message replied by the Tcp proxy _ Server of the TCP proxy module reaches the Client, the source MAC address of the Tcp Syn message sent by the TCP proxy module is changed into the MAC address of the Server, so the Tcp Syn message is pretended to be the Server, and the file descriptor Server fd is obtained. The Tcpproxy _ Server of the TCP agent module submits the Server Fd to the Fd _ binder, and the Fd _ binder associates and binds the Server Fd and the Client Fd to form an Fd association set through quintuple information. And the Tcppproxy _ server and Tcppproxy _ client forward data through the Fd association group bound by the Fd _ binder.

The Client carries out data communication through the proxy connection established with the TCP proxy module so as to communicate with the Server, and actually the TCP proxy module disguised Tcproxy _ Server, and the Server carries out communication through the established connection so as to communicate with the Client, and actually the TCP proxy module disguised Tcproxy _ Client.

Example 3:

the present embodiment is further optimized based on the foregoing embodiment, and as shown in fig. 2, the step F2 specifically includes the following steps:

It should be noted that, through the above improvement, the method for implementing the transparency of the TOS and TTL fields is that the Client sends a message to the Server through the TCP proxy Server, the TCP proxy Server checks the header field of the message, and the Tcpproxy _ Client copies the TOS and TTL values of the header field of the message into a request message sent to the Server by the TCP proxy Server.

In the same way, the Server sends a message to the Client through the TCP proxy Server, the TCP proxy Server checks the header field of the message, and the Tcppproxy _ Server copies the TOS and TTL values of the header field of the message to a request message sent to the Client by the TCP proxy Server.

Example 4:

the embodiment is further optimized on the basis of the above embodiment, as shown in fig. 3, a plurality of clients are respectively connected to the TCP proxy server, a plurality of network namespaces are provided and correspond to the plurality of clients one by one, and the network namespaces are respectively connected to the Bridge;

the step F3 specifically includes the following steps:

It should be noted that, with the above improvement, in practice, the number of clients is usually large, and each Client may also establish multiple connections with the Server, which results in a problem that the port on the TCP proxy Server is not enough. In order to support more than 65536 concurrent ports, it is a common practice for the existing TCP proxy server to configure multiple virtual IPs, so as to achieve the purpose that the source IP addresses in the five-tuple are different when the TCP proxy server establishes a new connection. It is often not feasible to select different virtual IP addresses for different clients because the number of virtual IPs allowed to be configured in a network is usually limited, but the number of clients is very large.

The invention provides a new mode, which is to establish a network namespace for each Client instead of using different virtual IP addresses. The virtual IP addresses used in each network namespace can be the same, and the ports can also be the same, so that the isolation of the network namespace solves the problem of limitation of the number of concurrent connections, and the source port can be completely transparent.

Assuming that the number of clients is N, the number of network namespaces also is N, and the network namespaces correspond to the clients one by one, where ns is short for the network namespaces, and the network namespaces are respectively butted with a Bridge, and the Bridge in this embodiment are the same module. When a ClientX initiates a connection request to a TCP proxy module, the connection request is intercepted by a TCP proxy server, the TCP proxy server checks whether the received message belongs to a certain existing Client, and if not, a network namespace X corresponding to the ClientX is established for the ClientX. At this time, the message sent by the ClientX to the TCP agent module is sent to the network namespace X for processing, the TCP agent module process establishes connection and communication with the Server in the network namespace X, and the Tcpproxy _ client uses the IP address and the TCP source port to be completely consistent with the ClientX, so that the TCP port transparency is realized.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications and equivalent variations of the above embodiments according to the technical spirit of the present invention are included in the scope of the present invention.

Claims

1.A method for realizing TCP proxy complete transparency is characterized in that: the method specifically comprises the following steps:

step F3: the TCP proxy server establishes network namespace for the Client to realize TCP port transparency;

the TCP proxy server comprises a TCP proxy module, a Syn _ handler module and a Bridge which are connected with each other, wherein the TCP proxy module comprises a Tcpproxy _ client, a Tcpproxy _ server and an Fd _ binder;

the step F1 specifically includes the following steps:

step F109: the TCP agent module informs a Syn _ handler module to release the original TcpSyn message sent by the Client in the step F102;

step F112: the Tcpproxy _ Server of the TCP agent module submits a connection file descriptor Serverfd to an Fd _ binder, and the Fd _ binder associates and binds the file descriptor Serverfd and the file descriptor Client Fd to form an Fd association group according to five-tuple information;

2. A method for achieving TCP proxy complete transparency according to claim 1, characterized by: the step F2 specifically includes the following steps:

3. A method for achieving TCP proxy complete transparency according to claim 1, characterized by: the number of the clients is multiple and is respectively connected with the TCP proxy server, the number of the network namespaces is multiple and is in one-to-one correspondence with the multiple clients, and the network namespaces are respectively connected with the Bridge;

the step F3 specifically includes the following steps: