WO2022151867A1 - 一种http转https双向透明代理的方法和装置 - Google Patents

一种http转https双向透明代理的方法和装置 Download PDF

Info

Publication number
WO2022151867A1
WO2022151867A1 PCT/CN2021/135682 CN2021135682W WO2022151867A1 WO 2022151867 A1 WO2022151867 A1 WO 2022151867A1 CN 2021135682 W CN2021135682 W CN 2021135682W WO 2022151867 A1 WO2022151867 A1 WO 2022151867A1
Authority
WO
WIPO (PCT)
Prior art keywords
http
https
proxy
tcp
server
Prior art date
Application number
PCT/CN2021/135682
Other languages
English (en)
French (fr)
Inventor
王赟
侯贺明
程波
Original Assignee
武汉绿色网络信息服务有限责任公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 武汉绿色网络信息服务有限责任公司 filed Critical 武汉绿色网络信息服务有限责任公司
Publication of WO2022151867A1 publication Critical patent/WO2022151867A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/09—Mapping addresses
    • H04L61/25—Mapping addresses of the same type
    • H04L61/2503—Translation of Internet protocol [IP] addresses
    • H04L61/2521—Translation architectures other than single NAT servers
    • H04L61/2528—Translation at a proxy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers

Definitions

  • the present invention relates to the technical field of network communication technologies, and in particular, to a method and device for a bidirectional transparent proxy from HTTP to HTTPS.
  • proxy can be divided into HTTP proxy, SSL proxy, FTP proxy, mail proxy, TCP proxy, etc.; from the location of the proxy server, it can be divided into forward proxy and reverse proxy; from the perspective of whether it can be perceived, it can be divided into For transparent proxy and non-transparent proxy.
  • Transparent proxy can be divided into client-side transparency and server-side transparency.
  • Client-side transparency means that the proxied user does not need to do any configuration, and its own traffic will passively accept the proxy, and the user will not perceive the whole process; server-side transparency refers to Yes, the destination server that the user accesses will not feel that the accessed traffic passes through the proxy server, and the IP address seen by the server is the user's IP address, not the IP address of the intermediate proxy server.
  • Two-way transparent proxy means that neither the end user nor the server will feel the existence of the proxy server, the end user does not need to do any proxy-related configuration, and the target IP accessed is also the real IP address of the server, not the IP of the proxy server;
  • the access IP address seen by the server is also the IP address of the real end user, not the IP address of the proxy server.
  • the IP address of the intermediate proxy server does not appear.
  • the traffic is processed by the proxy server, neither party can perceive that the traffic is being proxied. This is the meaning of bidirectional transparent proxy.
  • the traffic of the user accessing the server must first pass through the proxy server, that is to say, the proxy server must be deployed between the end user and the server, and can access the communication traffic between them. Secondly, some special processing needs to be done for the forwarding of the message. By default, the message will only flow to the node with the destination IP address. For the proxy server, since it is not the server that the user wants to access, although the message passes through the proxy server, the It will not be passed to the application layer, and the application layer has no corresponding socket to handle. Since it is acting as a proxy, it is necessary to control the routing of packets so that the packets that should be forwarded to the target server continue to be transmitted to the application layer and then processed.
  • Another key technology is to achieve two-way transparency, that is, to hide the IP address of the proxy server during the entire communication process, and only the IP address of the end user and the IP address of the server appear in the whole process.
  • the core of distinguishing various transparent proxy servers is to distinguish which technology is used for packet routing and which technology is used to achieve IP address transparency.
  • the traditional transparent proxy technology mainly has two routes: the first method is to achieve transparency by modifying the IP address of the packet; because the target IP address is modified, the routing system of the proxy router will route the packet to itself. , instead of forwarding, the message will then be processed by the TCP/TP protocol stack and reach the application layer as a proxy. Specifically, for the message arriving at the proxy server, modify the destination IP and destination port to the IP address and socket port of the proxy server; for the message leaving the proxy server, change the source IP and source port from the IP address and port of the proxy server. Modify it to the IP address and port of the client or server according to the type of connection.
  • the second method is not to do IP address translation, but to use the Tproxy feature of the Linux kernel. Tproxy is the abbreviation of transparent proxy. It can achieve two-way transparent proxy without making any changes to the IP address.
  • the transparent proxy systems implemented using Tproxy technology generally use virtual bridge devices to connect two or more physical network cards.
  • One of the network cards is an uplink network port and receives client traffic; the other network card is a downlink network port and connects to the server. end traffic.
  • the transparent proxy system needs to deliver the packets generated by the application layer agent program to the network.
  • it is necessary to configure the IP address for the bridge device, and configure the correct gateway address and routing table to achieve this. Correct packet forwarding.
  • the technical problem to be solved by the present invention is the existing transparent proxy system, which needs to deliver the message generated by the application layer proxy program to the network, and the transparent proxy system using the Linux bridge device needs to configure the IP address for the bridge device, and Correct packet forwarding can only be achieved by configuring the correct gateway address and routing table. At this time, it is often necessary to configure an IP address for the bridge device to participate in IP address-based routing. In some complex network environments, configure an IP address for the bridge device and configure the relevant routing table. is very tedious and error-prone.
  • the present invention adopts following technical scheme:
  • the present invention provides a method for converting HTTP to HTTPS bidirectional transparent proxy, including:
  • the proxy system After receiving the first HTTP request sent by the client, the proxy system parses the HTTP header field, obtains the Host field, compares the Host field with the built-in domain name list, and stores the content of the first HTTP request; wherein, the A list of domain names for storing domain names that satisfy HTTPS redirection;
  • the proxy system initiates a TCP handshake with a target port of 443 to the server, so as to establish a first TCP channel and perform a TLS negotiation process through the first TCP channel; wherein, the client used in the TCP handshake IP address of the terminal;
  • the proxy system sends a first HTTPS request to the server through the first TLS channel established by TLS negotiation; wherein, the first HTTPS request carries the encrypted content of the first HTTP request content;
  • the proxy system After the proxy system receives the first HTTPS response returned by the server, the proxy system deletes the secure attribute of the cookie field in the HTTP header field of the first HTTPS response and the Strict-Transport-Security field contained in the HSTS, and sends the content of the first HTTPS response. After decrypting it into plaintext, it is transparently transmitted to the client.
  • the method further includes:
  • the proxy system initiates a TCP handshake with a target port of 80 to the server, so as to establish a second TCP channel;
  • the proxy system sends a second HTTP request to the server through the second TCP channel, where the second HTTP request carries the content of the first HTTP request;
  • the proxy system receives the second HTTP response returned by the server, and checks whether the second HTTP response is an HTTPS redirection;
  • the proxy system adds the Host to the domain name list, and discards the second HTTP response, and simultaneously sends a TCP Reset message to the server to close the first TCP channel.
  • the method further includes:
  • the proxy system initiates a TCP handshake with a target port of 443 to the server, so as to establish a second TCP channel and carry out a TLS negotiation process through the second TCP channel; wherein, the IP address of the client used in the TCP handshake;
  • the proxy system sends a second HTTPS request to the server through the second TLS channel established by TLS negotiation; wherein, the second HTTPS request carries the encrypted content of the second HTTP request content;
  • the proxy system After the proxy system receives the second HTTPS response returned by the server, the proxy system deletes the secure attribute of the cookie field in the HTTP header field of the second HTTPS response, and the Strict-Transport-Security field included in the HSTS, and sends the content of the second HTTPS response. After decryption into plaintext, it is transparently transmitted to the client.
  • checking whether the second HTTP response is an HTTPS redirection specifically includes:
  • the method further includes:
  • the proxy system transparently transmits the received second HTTP response to the client.
  • the proxy system includes: a data packet transceiver module, a virtual network card module, a data packet routing module, and an agent program module, specifically:
  • the data packet sending and receiving module is used for sending and receiving messages at the bottom layer. It consists of a data packet receiving module and a data packet sending module.
  • the data packet receiving module is responsible for receiving data packets from the physical network card and forwarding them to the virtual network card;
  • the virtual network card module is a standard TUN virtual network card working at the transport layer, and is used to parse the received IP data packet through the protocol stack of the operating system;
  • the data packet routing module is used to route the target message to the agent program module for processing.
  • the data packet routing module includes iptables rule configuration and policy routing configuration.
  • the agent program module works at the application layer of the operating system and supports the Tproxy mechanism, specifically:
  • IP_TRANSPARENT Set the IP_TRANSPARENT parameter on the socket property to accept socket connections with any destination IP address, and also generate data packets with any IP address as the source IP, so that the socket can bind any IP address.
  • the method further includes:
  • the first rule indicates that the packet whose transmission protocol is TCP and whose destination port is 80 is marked with a tag value of 1;
  • the second rule indicates that the packet whose transmission protocol is TCP and whose source port is 443 is marked with a tag value of 1;
  • the third rule indicates that the packet whose transmission protocol is TCP and whose destination port is 443 is marked with a tag value of 2;
  • the fourth rule indicates that the packet whose transmission protocol is TCP and whose source port is 80 is marked with a tag value of 2.
  • the method further includes policy routing configuration, specifically:
  • ip route add default via 10.0.0.2 dev tun1 table 200.
  • the present invention also provides a device for converting HTTP to HTTPS bidirectional transparent proxy, which is used to implement the method for converting HTTP to HTTPS bidirectional transparent proxy described in the first aspect, and the device includes:
  • the at least one processor and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for The method for performing the HTTP to HTTPS bidirectional transparent proxy described in the first aspect is executed.
  • the present invention also provides a non-volatile computer storage medium storing computer-executable instructions, the computer-executable instructions being executed by one or more processors for completing the first The method of the HTTP to HTTPS bidirectional transparent proxy described in the aspect.
  • the invention realizes a bidirectional transparent proxy method and device for converting HTTP to HTTPS.
  • the program directly controls the receiving and sending of messages without changing the MAC address of the original message.
  • the upstream and downstream routing devices realize two-way transparency; at the IP transport layer, the source IP address and destination IP address can be combined without changing the source IP address and destination IP address to achieve two-way transparency between the client direction and the server direction; at the application layer
  • the HTTP to HTTPS proxy is implemented, the proxy system maintains an HTTP connection with the client, and maintains an HTTPS connection with the server, realizing a two-way transparent proxy at the application layer.
  • Fig. 1 is a system overall architecture diagram of a bidirectional transparent proxy from HTTP to HTTPS provided by an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the flow of a message in a proxy system according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for converting HTTP to HTTPS bidirectional transparent proxy provided by an embodiment of the present invention
  • FIG. 4 is a schematic flowchart of a method for converting HTTP to HTTPS bidirectional transparent proxy provided by an embodiment of the present invention
  • FIG. 5 is a schematic flowchart of a method for converting HTTP to HTTPS bidirectional transparent proxy provided by an embodiment of the present invention
  • FIG. 6 is a schematic flowchart of a processing method of a data packet receiving module provided by an embodiment of the present invention.
  • FIG. 7 is a schematic flowchart of a processing method of a data packet sending module provided by an embodiment of the present invention.
  • FIG. 8 is a schematic flowchart of a processing method for a configuration process provided by an embodiment of the present invention.
  • FIG. 10 is a signaling diagram of an interaction process A provided by an embodiment of the present invention.
  • FIG. 11 is a signaling diagram of an interaction process B provided by an embodiment of the present invention.
  • FIG. 13 is a schematic diagram of the content of an original HTTP response header field sent by a server to a client according to an embodiment of the present invention
  • FIG. 14 is a schematic diagram showing the content of an HTTP response header modified by a proxy system provided by an embodiment of the present invention.
  • FIG. 15 is a schematic structural diagram of an apparatus for an HTTP-to-HTTPS bidirectional transparent proxy provided by an embodiment of the present invention.
  • the transparent proxy construction method used in the present invention does not use IP address modification or network bridge equipment, but directly reads the data packets of the network card from the bottom layer, and then combines the virtual network card technology to proxy the data packets.
  • the feature of this method is that a program is used to directly control the network card sending and receiving of data packets, instead of relying on the operating system to route the packets. There are two benefits to this:
  • the second is to avoid obtaining the relevant address information of the upstream and downstream routing nodes of the current proxy system.
  • the proxy system does not need to know the IP address and MAC address of the next hop node for data packet routing in advance, because the proxy system internally retains the MAC address of the original packet. When receiving and sending packets, the MAC address of the original packet will not be modified.
  • the proxy system is transparent to the upstream and downstream routing devices, so that the proxy system can be easily embedded in complex network environments. middle.
  • the traditional proxy for HTTPS generally only performs data transfer at the TCP layer, and simply performs data forwarding between the HTTPS client and the HTTPS server. The reason for this is that HTTPS itself can prevent the middleman from modifying data.
  • the HTTPS client will report an error.
  • a typical HTTPS client is a browser.
  • the browser will alert the end user, warning the user that the traffic may be hijacked.
  • the present invention proposes an HTTP to HTTPS proxy, which bypasses the process of HTTPS encrypting client traffic, and solves the problem that under HTTPS proxy, the client browser will not trust the proxy.
  • the certificate alarm problem caused by the certificate of the system.
  • the invention realizes a bidirectional transparent proxy system for HTTP to HTTPS.
  • the program directly controls the receiving and sending of the message without changing the MAC address of the original message.
  • the routing device realizes two-way transparency; at the IP transport layer, combined with the virtual network card technology and Tproxy technology, the source IP address and destination IP address are not changed, and the two-way transparency between the client direction and the server direction is realized; at the application layer, it is realized HTTP to HTTPS proxy, the proxy system maintains an HTTP connection with the client, and maintains an HTTPS connection with the server, realizing a two-way transparent proxy at the application layer.
  • the specific technical scheme consists of two parts.
  • the first part is the underlying implementation of the transparent proxy technology, including sending and receiving packets and implementing IP address transparency; the second part is how the proxy system interacts with clients and servers.
  • the overall role is divided into three categories, client, proxy system and server.
  • the proxy system maintains two connections at the same time, one is the connection established with the client, the other is the connection established with the server, and the proxy system transfers data between the two connections.
  • the proxy system is divided into four modules as a whole, the first is the data packet transceiver module, the second is the virtual network card module, the third is the data packet routing module, and the fourth is the proxy system module.
  • the data packet sending and receiving module is mainly responsible for sending and receiving messages at the bottom layer. It consists of a data packet receiving module and a data packet sending module.
  • the data packet receiving module is responsible for receiving data packets from the network card and forwarding them to the virtual network card device;
  • the virtual network card module is a
  • the standard TUN virtual network card working at the transport layer is mainly responsible for entering the received IP data packets into the protocol stack of the operating system for analysis;
  • the data packet routing module is mainly responsible for routing the target packets to the agent program module for processing, including iptables rule configuration, policy routing configuration, etc.
  • the data packets of the IP address can be delivered to the proxy module of the application layer for processing.
  • Tproxy technology is mainly used here.
  • the proxy program module works in the application layer of the operating system. It is an application program.
  • the IP_TRANSPARENT parameter is set on the socket attribute, so that any destination IP can be accepted.
  • the socket connection of the address can also generate data packets with any IP address as the source IP, that is, the socket can be bound to any IP address.
  • the flow of data packets in the proxy system is shown in Figure 2.
  • the proxy program module works at the application layer. It maintains two socket connections with the client and the server at the same time, numbered 1, 2, 3, 4, 5, and 6.
  • the flow composed of several lines represents the connection maintained by the proxy system and the client, and the flow composed of several lines numbered 7, 8, 9, 10, 11, and 12 represents the connection maintained by the proxy system and the server.
  • the line numbered 13 indicates that non-HTTP traffic will not be delivered to the application layer for processing, but will be forwarded directly by the network card.
  • the traffic that the data packet transceiver module is responsible for processing is the traffic packets numbered 2, 5, 8, 11, and 13.
  • the data packet receiving module is responsible for processing traffic packets represented by numbers 2, 11, and 13.
  • the sending module is responsible for processing traffic packets represented by numbers 5 and 8; the data packet routing module is responsible for processing traffic packets represented by numbers 3, 4, 7, and 12.
  • the HTTP-to-HTTPS proxy in the present invention essentially implements an HTTP redirection jump hijacking.
  • Modern websites gradually use HTTPS to protect the transmitted content from being modified, but in order to be compatible with the HTTP traffic accessed by users by default, the HTTP service of port 80 is generally reserved at the same time.
  • the redirection mechanism of the HTTP protocol will be used. , send a 301/302 redirection command to the user, and direct the user to the corresponding HTTPS service.
  • the HTTP-to-HTTPS proxy function realized by the present invention essentially utilizes and hijacks the redirection mechanism.
  • the proxy system of the present invention When the client initiates an HTTP request, the proxy system of the present invention first detects the target domain name. The specific detection process is to initiate an HTTP request to the corresponding domain name, and check whether the HTTP response status code is a redirection status code such as 301/302. If it is a redirection, and the URL after redirection is the HTTPS version of the URL, it means that the domain name satisfies the HTTPS redirection conditions, then an HTTPS request is initiated to the corresponding domain name, and the returned HTTPS response content is delivered to the client through the HTTP channel end.
  • the specific detection process is to initiate an HTTP request to the corresponding domain name, and check whether the HTTP response status code is a redirection status code such as 301/302. If it is a redirection, and the URL after redirection is the HTTPS version of the URL, it means that the domain name satisfies the HTTPS redirection conditions, then an HTTPS request is initiated to the corresponding domain name, and the returned HTTPS response content is delivered
  • the proxy system maintains an HTTP connection with the client, but maintains an HTTPS connection with the server, which successfully hijacks the 302 redirection action of the client and prevents the client from upgrading to an HTTPS connection. If the result of the detection is that the target domain name does not meet the HTTPS redirection conditions, then the proxy system communicates with both the client and the server using the HTTP channel, and transfers data between the two. At this time, it works in the HTTP proxy mode.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • Embodiment 1 of the present invention provides a method for converting HTTP to HTTPS bidirectional transparent proxy, as shown in FIG. 3 , including:
  • the proxy system parses the HTTP header field, obtains the Host field, compares the Host field with the built-in domain name list, and stores the content of the first HTTP request ; wherein, the domain name list is used to store domain names that satisfy HTTPS redirection.
  • the proxy system initiates a TCP handshake with a target port of 443 to the server, so as to establish a first TCP channel and perform a TLS negotiation process through the first TCP channel; wherein, the TCP The IP address of the client used in the handshake; wherein, the TLS negotiation process includes one or more processes of negotiating an encryption suite, passing a certificate, verifying a certificate, and calculating a secret key.
  • the proxy system sends a first HTTPS request to the server through the first TLS channel established by TLS negotiation (in other embodiments of the present invention, it is also expressed as an HTTP request, and this is for the intuitive expression of technical characteristics, so described as an HTTPS request); wherein, the first HTTPS request carries the encrypted content of the first HTTP request content.
  • step 104 after the proxy system receives the first HTTPS response returned by the server, the proxy system deletes the secure attribute of the cookie field in the HTTP header field of the first HTTPS response, and the Strict-Transport-Security field contained in the HSTS, and sends After the first HTTPS response content is decrypted into plaintext, it is transparently transmitted to the client.
  • the embodiment of the present invention implements a bidirectional transparent proxy method for converting HTTP to HTTPS.
  • the program directly controls the receiving and sending of messages without changing the MAC address of the original message.
  • the upstream and downstream routing devices realize two-way transparency; at the IP transport layer, the source IP address and destination IP address can be combined without changing the source IP address and destination IP address to achieve two-way transparency between the client direction and the server direction; at the application layer
  • the HTTP to HTTPS proxy is implemented.
  • the proxy system maintains an HTTP connection with the client, and maintains an HTTPS connection with the server, realizing a two-way transparent proxy at the application layer.
  • step 102 in Embodiment 1 if it is found that the Host domain name is not in the list, as shown in FIG. 4 , the method further includes:
  • step 105 the proxy system initiates a TCP handshake with a target port of 80 to the server, so as to establish a second TCP channel.
  • step 106 the proxy system sends a second HTTP request to the server through the second TCP channel, where the second HTTP request carries the content of the first HTTP request.
  • step 107 the proxy system receives the second HTTP response returned by the server, and checks whether the second HTTP response is an HTTPS redirection.
  • checking whether the second HTTP response is an HTTPS redirection specifically includes:
  • step 108 if it is found that it is HTTPS redirection, then the proxy system adds the Host to the domain name list, and discards the second HTTP response, and simultaneously sends a TCP Reset message to the server to close the first TCP channel.
  • the method further includes:
  • the proxy system initiates a TCP handshake with a target port of 443 to the server, so as to establish a second TCP channel and perform a TLS negotiation process through the second TCP channel; wherein, the IP address of the client used in the TCP handshake , wherein the TLS negotiation process includes: one or more processes of negotiating an encryption suite, delivering a certificate, verifying a certificate, and calculating a secret key.
  • the proxy system sends a second HTTPS request to the server through the second TLS channel established by TLS negotiation; wherein, the second HTTPS request carries the encrypted content of the second HTTP request content;
  • step 111 after the proxy system receives the second HTTPS response returned by the server, the proxy system deletes the secure attribute of the cookie field in the HTTP header field of the second HTTPS response, and the Strict-Transport-Security field contained in the HSTS, and sends After the second HTTPS response content is decrypted into plaintext, it is transparently transmitted to the client.
  • the method further includes: the proxy system will receive the received The second HTTP response is transparently transmitted to the client.
  • a preferred proxy system framework structure including: a data packet transceiver module, a virtual network card module, a data packet routing module, and an agent program module, specifically:
  • the data packet sending and receiving module is used for sending and receiving messages at the bottom layer. It consists of a data packet receiving module and a data packet sending module.
  • the data packet receiving module is responsible for receiving data packets from the physical network card and forwarding them to the virtual network card;
  • the virtual network card module is a standard TUN virtual network card working at the transport layer, and is used to parse the received IP data packet through the protocol stack of the operating system;
  • the data packet routing module is used to route the target message to the agent program module for processing.
  • the data packet routing module includes iptables rule configuration and policy routing configuration. By default, when the operating system sees a data packet whose destination IP address is not its own IP, it will not be passed to the application layer, but will be discarded or forwarded. Because the packet routing module is configured with special routing rules, any destination IP address The data packets can be delivered to the proxy module of the application layer for processing. Tproxy technology is mainly used here.
  • the agent program module works at the application layer of the operating system and supports the Tproxy mechanism, specifically:
  • IP_TRANSPARENT Set the IP_TRANSPARENT parameter on the socket property to accept socket connections with any destination IP address, and also generate data packets with any IP address as the source IP, so that the socket can bind any IP address.
  • the method further includes:
  • the first rule indicates that the packet whose transmission protocol is TCP and whose destination port is 80 is marked with a tag value of 1;
  • the second rule indicates that the packet whose transmission protocol is TCP and whose source port is 443 is marked with a tag value of 1;
  • the third rule indicates that the packet whose transmission protocol is TCP and whose destination port is 443 is marked with a tag value of 2;
  • the fourth rule indicates that the packet whose transmission protocol is TCP and whose source port is 80 is marked with a tag value of 2.
  • routing table 100 for the packet with the tag value of 1 (the routing table 100 here is only for the convenience of the presentation of the following instructions, in the actual use process, the 100 can also be expressed as other custom strings , therefore, it should not be regarded as a special limitation of the protection scope of the present invention):
  • ip route add default via 10.0.0.2 dev tun1 table 200.
  • the present invention will describe the method execution processes of the main functional modules in the proxy system involved in the present invention one by one through a plurality of embodiments, and the corresponding embodiments will not continue the method in Embodiment 1 of the present invention.
  • the first and second expressions it should be pointed out that the first, second and third prefixes used in the embodiments of the present invention are only used to distinguish objects more clearly in the citation process, and do not have special The meaning is limited, and the following embodiments can be related to the related objects involved in the embodiments of the present invention through the description of the context even if the first and second calibrations are not added, and the following contents are not one by one. Repeat.
  • step S121 the packet is received from the physical network card.
  • the network card in this step includes both the uplink network card and the downlink network card.
  • the program receives the packet without relying on the analysis and processing of the packet by the operating system, but directly from the network card. Read the message. At this time, the message contains the Ethernet header, IP header, etc., which is a complete message.
  • step S122 the packet is parsed. This step is to parse the packet, obtain the source MAC address, source IP address, source port, destination MAC address, destination IP address, and destination port of the packet, and record the IP address and the destination port.
  • the correspondence between the MAC addresses is stored in the IP-MAC correspondence table.
  • step S123 check whether the message is a target message.
  • the condition is an OR relationship. In other words, as long as one of the conditions is hit, the packet belongs to the target packet.
  • step S124 the Ethernet header is stripped, and this step is to strip off the Ethernet header of the packet hit in the step S123, and only keep the data part above the IP header.
  • step S125 send to the virtual network card, and this step is to deliver the IP data packet generated in step S124 to the virtual network card device.
  • sending to the physical network card of the opposite end refers to directly delivering the packets that are not hit in step S123 to the physical network card of the opposite end.
  • the peer physical NIC means that if the packet is received from the upstream NIC, it will be forwarded from the downstream NIC; otherwise, if the packet is received from the downstream NIC, it will be forwarded from the upstream NIC.
  • step S131 a packet is received from the virtual network card.
  • This step refers to reading the packet from the virtual network card device.
  • the virtual network card at this time refers to a TUN type network card, and the received packet is an IP with an IP header. Data packets, without an Ethernet header.
  • step S132 an Ethernet header is added, and this step refers to adding an Ethernet header to the IP data packet, including source MAC and destination MAC addresses.
  • the principle of adding is to query the IP-MAC correspondence table generated by the data packet receiving module, set the source MAC as the MAC address corresponding to the source IP, and set the destination MAC as the MAC address corresponding to the destination IP.
  • step S133 it is detected whether the target port is port 443, and if the target port of the message is port 443 of TCP, the judgment condition is satisfied.
  • step S134 send to the physical network card-downlink port. This step is to send the Ethernet data packet that completes the Ethernet header to the physical network card of the downlink port.
  • step S135 send to the physical network card-upstream port, and this step is to send the Ethernet data packet that completes the Ethernet header to the physical network card of the upstream port.
  • Virtual network card and routing configuration module the configuration process involves routing forwarding configuration, virtual network card configuration, iptables rule configuration, policy routing configuration and other aspects, as shown in Figure 8.
  • S141 is the route forwarding configuration
  • the purpose is to enable the route forwarding function of the Linux system.
  • the operating system will discard the data packets whose destination IP address is not its own IP address; if the routing forwarding function is enabled, the operating system will forward the data packets whose destination IP address is not its own IP address. Instead of discarding it, it is equivalent to the operating system routing the data packets.
  • the method of enabling routing forwarding is relatively simple. You can configure it through commands or write configuration files. Different Linux system distributions may have different names and locations of network configuration files. The commands to configure through the command line are as follows:
  • S142 is the configuration of the virtual network card, and the purpose is to generate a virtual network card to receive the IP packet, so that the IP packet enters the protocol stack of the local machine for processing.
  • a typical configuration method is as follows:
  • IP address of the virtual network card here will not affect the function of the overall proxy system and can be set arbitrarily.
  • S143 is the iptables rule configuration.
  • the main purpose of this part is to label the target data packets.
  • the target data packet contains two parts, the first part is the traffic represented by lines 3 and 12 in Figure 2, and the second part is the traffic represented by lines 4 and 7 in Figure 2.
  • the traffic represented by lines 3 and 12 is labeled so that packets whose destination address is not the local IP address can be routed to the local application layer for processing instead of being forwarded;
  • the traffic represented by line 7 is tagged because this part of the traffic queries the default routing table and will be routed to the physical network card. In the present invention, in order to route it to the virtual network card, it is also marked.
  • the first type of rules is to label the traffic represented by lines 3 and 12 in Figure 2.
  • the typical configuration method is as follows. Use the iptables tool to add the following rules:
  • This rule indicates that the packet whose transmission protocol is TCP and the destination port is 80 is marked with a tag value of 1, and is delivered to the application layer to the program whose port is 80 for processing. This part of the traffic matches what is shown by line 3 in Figure 2. represented traffic.
  • This rule indicates that the packet whose transmission protocol is TCP and the source port is 443 is marked with a tag value of 1. This part of the traffic matches the traffic represented by line 12 in Figure 2.
  • the second type of rule is to label the traffic represented by lines 4 and 7 in Figure 2:
  • S144 is the policy routing configuration, which is used in conjunction with iptables rules. It mainly consists of two parts, one is to allow the packets of the virtual network card to be delivered to the application layer program for processing, that is, the traffic packets represented by lines 3 and 12 in Figure 2 will match this policy route; the second is to let the application The packets generated by the program can be delivered to the virtual NIC instead of the physical NIC. The traffic packets represented by lines 4 and 7 in Figure 2 will match this policy route.
  • the specific configuration method is as follows, let the packet with the label value 1 query the routing table 100:
  • ip route add default via 10.0.0.2 dev tun5 table 200.
  • Fig. 9 shows the processing process of the message by the agent program module.
  • the agent program module maintains the HTTP or HTTPS connection with the client and the server at the same time, and transfers data between the client and the server.
  • Client representing the client, a typical HTTP client is a browser
  • Proxy representing a proxy system
  • Server representing the server side, a typical HTTP server side such as various websites visited;
  • TCP Transmission Control Protocol, Transmission Control Protocol
  • HTTP HyperText Transfer Protocol, hypertext transfer protocol
  • TLS Transport Layer Security, secure transport layer protocol
  • HSTS HTTP Strict Transport Security, HTTP Strict Transport Security;
  • TCP handshake refers to the three-way handshake process that needs to be performed when the two parties establish a connection in the TCP protocol;
  • TLS handshake refers to the negotiation process of establishing a TLS connection between the client and the server in the TLS protocol, and there are multiple packet exchanges in this process;
  • HTTP GET refers to the resource request initiated by the client in the HTTP protocol
  • HTTP Response In the HTTP protocol, the server side replies to the client's response content
  • Host refers to an HTTP header field in the HTTP request message, and its value is the target domain name of the visit;
  • step S201 the client and the proxy system initiate a TCP handshake to establish a TCP connection, and the target port is port 80. From the client's point of view, it is making a TCP connection with the server, but because the traffic at this time has been routed to the proxy system, the client actually establishes a TCP connection with the proxy system.
  • step S202 the client initiates an HTTP request to the proxy system, the HTTP at this time is a plaintext request, and the target port is port 80. From the client's point of view, it is initiating an HTTP request to the server. Because the traffic at this time has been routed to the proxy system, the client actually sends the request to the proxy system.
  • step S203 after receiving the HTTP request sent by the client, the proxy system parses the HTTP header field, obtains the Host field, and compares the Host field with the built-in domain name list; Directed domain name. If a domain name is listed in this list, it means that this domain name has been detected by HTTPS redirection before, and this domain name will be redirected by HTTPS. If the Host domain name is found in the list, go to step S204; otherwise, go to step S210.
  • step S204 the proxy system initiates a TCP handshake to the server, and the target port is 443; at this time, the IP address seen by the server is the IP address of the client, and the existence of the proxy system will not be discovered.
  • step S205 the proxy system and the server perform a TLS negotiation process, which is based on the TCP channel established in S204; in the TLS negotiation process, there will be multiple packet exchanges. Pass the certificate, verify the certificate, calculate the secret key, etc.
  • step S206 the proxy system sends an HTTP request to the server, this request is based on the TLS channel established in step S205, this HTTP request is an encrypted request, and the specific request content is obtained from step S202;
  • step S207 the server replies to the proxy system with the content of the HTTP response.
  • the HTTP response is based on the TLS channel established in step S205, which is encrypted and not plaintext.
  • step S208 the HTTP response is processed.
  • the proxy system After the proxy system receives the HTTP response from the server, it needs to process the response content before sending it to the client.
  • the reason for dealing with HTTP responses here is to adapt HTTPS to HTTP.
  • the attributes of some fields in the HTTP header field involve the HTTPS transmission mode. There are typically two fields. One is the secure attribute of the cookie field. This attribute indicates that the cookie can only be transmitted in the HTTPS channel and cannot be transmitted in the HTTP channel; If this attribute is not deleted, the cookie will not be transmitted in the plaintext transmission channel such as HTTP, which will cause various server authentication problems; the second is HSTS.
  • the function of HSTS is to force the client to use HTTPS to create a connection with the server.
  • the set method is to include the Strict-Transport-Security field in the HTTP response header.
  • the HTTP protocol stipulates that the HSTS field set during non-encrypted transmission is invalid, but the HSTS field that appears in the HTTP plaintext transmission channel will cause the client's suspicion, so this field should be deleted.
  • step S209 the proxy system returns an HTTP response to the client, and the content of the HTTP response at this time may have been adapted through step S208, or may not have been changed; if it comes from step S213 to this step, then No modification of the HTTP response content is required.
  • step S210 the proxy system initiates a TCP handshake to the server, and the target port is port 80.
  • step S211 the proxy system sends an HTTP request to the server, the request is based on the TCP channel established in step S210, the HTTP request is a plaintext request, and the specific request content is obtained from step S202.
  • step S212 the server replies to the proxy system with HTTP response content, and the HTTP response is based on the TCP channel established in step S210 and is transmitted in plain text.
  • step S213 it is checked whether the HTTP response is an HTTPS redirection.
  • This step mainly checks the content of the HTTP response obtained in step S212. One is to check whether the status code of the HTTP response is between 300 and 399. According to the HTTP protocol specification, the response status code in this interval indicates redirection; Whether the target address is the HTTPS version of the original Host, because the redirection function itself is not only HTTPS redirection, but can perform arbitrary URL redirection, and HTTPS redirection is only one of the applications. After checking, if it is found that it is an HTTPS redirection, then jump to step S214, otherwise, jump to S209.
  • step S214 the proxy system and the server disconnect the TCP connection.
  • the proxy system finds that the content of the HTTP response is HTTPS redirection.
  • the proxy system does not forward the response to the client, but discards the HTTP response and sends a TCP Reset message to the server to close the TCP channel;
  • step S215 the Host is added to the domain name list.
  • the proxy system has completed the detection of the target domain name, and knows that the target domain name meets the conditions for HTTPS redirection, so it is added to the domain name list, and the next time the proxy system processes this domain name, it can directly proxy, and there is no need to repeat the detection;
  • HTTP proxy mode that is, the proxy system and the client use HTTP connection, and the server side also uses HTTP connection; the proxy system first detects the server and finds that the server does not meet the HTTPS redirection conditions, and then goes to HTTP proxy mode;
  • the second, interactive process B HTTP to HTTPS proxy mode, that is, the proxy system and the client use HTTP connections, and use HTTPS connections with the server; the proxy system has detected this domain name before and found that it meets the HTTPS redirection conditions. HTTPS proxy directly;
  • the third, interactive process C HTTP to HTTPS proxy mode, superimposing the initial detection process; the proxy system first detects the server, finds that the server meets the HTTPS redirection conditions, and then transfers to the HTTPS proxy;
  • the working steps are S201, S202, S203, S210, S211, S212, S213, and S209, and the message interaction process is shown in FIG. 10 .
  • the IP address used by the client is 1.1.1.1 and the port is 1024; the IP address of the server is 2.2.2.2 and the port is 80; the IP address of the proxy system itself is 3.3.3.3.
  • the connection between the client and the proxy system uses IP addresses 1.1.1.1 and 2.2.2.2, and ports 1024 and 80.
  • the proxy system pretends to be a server to establish a connection with the client.
  • the IP addresses used for the connection between the proxy system and the server are 1.1.1.1 and 2.2.2.2, and the ports are 2000 and 80.
  • the proxy system pretends to be a client to establish a connection with the server. It can be seen that in the whole communication process, the proxy system's own IP address does not appear, only the source port changes, because the source port is randomly selected, so even if the source port is changed, the proxy system is still a two-way transparent proxy system . After passing through the proxy system, the MAC address, IP address, and content of the HTTP message remain unchanged, but the source port field of the TCP protocol changes.
  • the working steps are S201, S202, S203, S204, S205, S206, S207, S208, and S209, and the message interaction process is shown in FIG. 11 .
  • the IP address used by the client is 1.1.1.1 and the port is 1024;
  • the IP address of the server is 2.2.2.2, the ports are 80 and 443;
  • the IP address of the proxy system itself is 3.3.3.3.
  • the proxy system will use the IP address of 2.2.2.2 and port 80 to establish an HTTP connection with the client, and the proxy system will use the IP address of 1.1.1.1 and a random source port and the real server. Establish an HTTPS connection.
  • the proxy system In the whole communication process, there is an HTTP connection between the client and the proxy system; there is an HTTPS connection between the proxy system and the server, and this connection is a trusted connection, because the proxy system is used as a client at this time.
  • the certificate is the server's certificate, which is a trusted certificate.
  • the proxy system's own IP address does not appear, only the source port changes, because the source port is randomly selected by the client, so even if the source port is changed, the proxy system is still a two-way transparent proxy system. After passing through the proxy system, the MAC address and IP address of the message remain unchanged, but the source port field of the TCP protocol changes, and the HTTP protocol becomes the HTTPS protocol, and the HTTP header field may change. .
  • FIG. 13 it represents the original HTTP response header field sent by the server to the client; please note that there is a Strict-Transport-Security header field, and the Set-Cookie field sets a secure attribute to the cookie.
  • the proxy system deletes the secure attribute of the cookie and deletes the Strict-Transport-Security header field.
  • the working steps are S201, S202, S203, S210, S211, S212, S213, S214, S215, S204, S205, S206, S207, S208, S209, and the message interaction process is shown in Figure 12 shown.
  • the IP address used by the client is 1.1.1.1 and the port is 1024;
  • the IP address of the server is 2.2.2.2, the ports are 80 and 443;
  • the IP address of the proxy system itself is 3.3.3.3.
  • the proxy system will use the IP address of 2.2.2.2 and port 80 to establish an HTTP connection with the client, and the proxy system will use the IP address of 1.1.1.1 and a random source port and the real server.
  • the proxy system closes the HTTP connection with the server and establishes an HTTPS connection with the server using the IP address of 1.1.1.1 and a random source port.
  • HTTPS connection is a trusted connection, because the proxy system acts as a client in the use.
  • the source port has changed. The source port is randomly selected by the client and can be changed.
  • the server port is 80; in the connection between the proxy system and the server, the server port is 443, but in the whole communication process, the proxy system's own IP address will not appear, the proxy system is a two-way Transparent proxy system.
  • FIG. 15 it is a schematic structural diagram of an apparatus for a bidirectional transparent proxy from HTTP to HTTPS according to an embodiment of the present invention.
  • the apparatus for a bidirectional transparent proxy from HTTP to HTTPS in this embodiment includes one or more processors 21 and a memory 22 .
  • one processor 21 is taken as an example in FIG. 15 .
  • the processor 21 and the memory 22 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 15 .
  • the memory 22 can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the method for HTTP to HTTPS bidirectional transparent proxy in Embodiment 1.
  • the processor 21 executes the method of HTTP to HTTPS bidirectional transparent proxy by running the non-volatile software programs and instructions stored in the memory 22 .
  • Memory 22 may include high speed random access memory, and may also include nonvolatile memory, such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, memory 22 may optionally include memory located remotely relative to processor 21, which may be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the program instructions/modules are stored in the memory 22, and when executed by the one or more processors 21, execute the HTTP to HTTPS bidirectional transparent proxy method in the above-mentioned Embodiment 1, for example, execute the above-described method.

Abstract

本发明涉及网络通信技术领域，提供了一种HTTP转HTTPS双向透明代理的方法和装置。方法包括代理系统收到客户端发送的第一HTTP请求后，解析HTTP头部字段，获得Host字段，把Host字段和内置的域名列表进行比对，并存储所述第一HTTP请求内容；其中，所述域名列表，用于存储满足HTTPS重定向的域名；若发现Host域名在列表中，代理系统向服务器发起目标端口为443的TCP握手，以便建立第一TCP通道和通过所述第一TCP通道进行TLS协商过程。本发明实现了一种不易出错的，精简的双向透明代理。

Description

一种HTTP转HTTPS双向透明代理的方法和装置 【技术领域】
本发明涉及网络通信技技术领域，特别是涉及一种HTTP转HTTPS双向透明代理的方法和装置。
【背景技术】
随着网络环境的日益复杂，各种代理技术应运而生。从代理的协议层面可以划分为HTTP代理、SSL代理、FTP代理、邮件代理、TCP代理等；从代理服务器位置上，可以划分为正向代理和反向代理；从是否能被感知的角度可以分为透明代理和非透明代理。透明代理又可以分为客户端透明和服务器端透明，客户端透明就是指被代理的用户不需要做任何配置，自己的流量就会被动的接受代理，整个过程用户无感知；服务器端透明指的是，用户访问的目的服务器不会感觉到访问的流量是经过代理服务器的，服务器看到的IP地址是用户的IP地址，而不是中间代理服务器的IP地址。双向透明代理指的是终端用户以及服务器端都不会感受到代理服务器的存在，终端用户不需要做任何代理相关的配置，访问的目标IP也是服务器的真实IP地址，而不是代理服务器的IP；服务器端看到的访问IP地址也是真实的终端用户的IP地址，而不是代理服务器的IP地址。在整个的流量传输过程中，并不会出现中间代理服务器的IP地址，虽然流量经过了代理服务器的处理，但是双方都感知不到流量被代理了，这就是双向透明代理的含义所在。
为了实现透明代理，首先需要用户访问服务器的流量要经过代理服务器，也就是说代理服务器部署的位置一定是在终端用户和服务器之间，而且能够访问到它们之间的通信流量。其次需要对报文的转发做一些特殊处理，默认情况下报文只会流向目的IP地址的节点，对于代理服务器来说，由于自身并不是用户要访问的服务器，报文虽然经过代理服务器，但是不会向应用层传递，应用层也没有对应的socket来处理。既然是做代理，那么必须要对报文的路由做一些控制，使本该转发给目标服务器的报文，继续向应用层传递，进而得到处理。另外一个关键的技术是要实现双向透明，即在整个通信过程中要隐藏代理服务器的IP地址，全程只出现终端用户的IP地址和服务器的IP地址。区分各种不同的透明代理服务器，核心就是要区分使用何种技术做的报文路由，使用何种技术实现的IP地址透明。
传统的透明代理技术，主要有两种路线：第一种方法是通过修改报文IP地址的方式来实现透明；因为修改了目标IP地址，所以代理路由器的路由系统会把此报文路由到自身，而不是转发走，报文进而就会经过TCP/TP协议栈处理，到达应用层做代理。具体而言，对于到达代理服务器的报文，修改目的IP和目的端口为代理服务器的IP地址和socket端口；对于离开代理服务器的报文，把源IP和源端口从代理服务器的IP地址和端口根据连接的不同类型修改为客 户端或者服务器的IP地址和端口。第二种方法是不做IP地址转换，而是使用Linux内核的Tproxy特性，Tproxy是透明代理的简称，它可以不对IP地址做任何变更，就可以实现双向透明代理。
目前使用Tproxy技术实现的透明代理系统一般都是使用虚拟网桥设备来连接两个或者多个物理网卡，其中一个网卡是上行网口，接客户端流量；另外一个网卡是下行网口，接服务器端流量。透明代理系统，需要把应用层代理程序产生的报文投递到网络上，使用Linux网桥设备的透明代理系统，需要给网桥设备配置IP地址，并配置正确的网关地址和路由表，才能实现正确的报文转发。此时往往需要给网桥设备配置有一个IP地址来参与到基于IP地址的路由中来，在某些网络组网环境复杂的情况下，给网桥设备配置IP地址并配置相关的路由表，是非常繁琐的事情，而且很容易出错。
鉴于此，克服该现有技术所存在的缺陷是本技术领域亟待解决的问题。
【发明内容】
本发明要解决的技术问题是现有的透明代理系统，需要把应用层代理程序产生的报文投递到网络上，使用Linux网桥设备的透明代理系统，需要给网桥设备配置IP地址，并配置正确的网关地址和路由表，才能实现正确的报文转发。此时往往需要给网桥设备配置有一个IP地址来参与到基于IP地址的路由中来，在某些网络组网环境复杂的情况下，给网桥设备配置IP地址并配置相关的路由表，是非常繁琐的事情，而且很容易出错。
本发明采用如下技术方案：
第一方面，本发明提供了一种HTTP转HTTPS双向透明代理的方法，包括：
代理系统收到客户端发送的第一HTTP请求后，解析HTTP头部字段，获得Host字段，把Host字段和内置的域名列表进行比对，并存储所述第一HTTP请求内容；其中，所述域名列表，用于存储满足HTTPS重定向的域名；
若发现Host域名在列表中，代理系统向服务器发起目标端口为443的TCP握手，以便建立第一TCP通道和通过所述第一TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址；
代理系统通过TLS协商建立的第一TLS通道，向服务器发送第一HTTPS请求；其中，所述第一HTTPS请求中携带所述第一HTTP请求内容经过加密后的内容；
代理系统接收服务器返回的第一HTTPS响应，代理系统删除第一HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第一HTTPS响应内容解密为明文后，透传给客户端。
优选的，若发现Host域名不在列表中，所述方法还包括：
代理系统向服务器发起目标端口为80的TCP握手，以便建立第二TCP通道；
代理系统通过所述第二TCP通道，向服务器发送第二HTTP请求，所述第二HTTP请求中携带所述第一HTTP请求内容；
代理系统接收服务器返回的第二HTTP响应，并检查所述第二HTTP响应是 否是HTTPS重定向；
若发现是HTTPS重定向，则代理系统把Host加入域名列表中，并丢弃掉所述第二HTTP响应，同时向服务器发送TCP Reset报文，关闭所述第一TCP通道。
优选的，所述方法还包括：
代理系统向服务器发起目标端口为443的TCP握手，以便建立第二TCP通道和通过所述第二TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址；
代理系统通过TLS协商建立的第二TLS通道，向服务器发送第二HTTPS请求；其中，所述第二HTTPS请求中携带所述第二HTTP请求内容经过加密后的内容；
代理系统接收服务器返回的第二HTTPS响应，代理系统删除第二HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第二HTTPS响应内容解密为明文后，透传给客户端。
优选的，检查所述第二HTTP响应是否是HTTPS重定向，具体包括：
检查所述第二HTTP响应的状态码是否在300～399之间，并且，检查重定向的目标地址是原来Host的HTTPS版本；其中，根据HTTP协议规范，此区间的响应状态码表示重定向。
优选的，若发现不是HTTPS重定向，所述方法还包括：
代理系统将接收到的第二HTTP响应透传给所述客户端。
优选的，代理系统包括：数据包收发模块、虚拟网卡模块、数据包路由模块、代理程序模块，具体的：
数据包收发模块，用于底层的报文收发，由数据包接收模块和数据包发送模块组成，数据包接收模块负责从物理网卡接收数据包，转发到虚拟网卡上；
虚拟网卡模块为工作在传输层的标准的TUN虚拟网卡，用于将收到的IP数据包经由操作系统的协议栈进行解析；
数据包路由模块，用于将目标报文路由到代理程序模块去处理，数据包路由模块中包括iptables规则配置和策略路由配置。
优选的，所述代理程序模块工作在操作系统的应用层，并且支持Tproxy机制，具体的：
在socket属性上设置IP_TRANSPARENT参数，使得接受任意目的IP地址的socket连接，同时也可以以任意IP地址为源IP产生数据报文，从而使socket可以绑定任意IP地址。
优选的，方法还包括：
在iptables规则中添加第一规则：
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY--tproxy-mark 0x1/0x1 --on-port 0；
所述第一规则表示，把传输协议是TCP，目的端口是80的报文，打上标签值1；
在iptables规则中添加第二规则：
iptables -t mangle -A PREROUTING -p tcp --sport 443 -j MARK--set-mark 1；
所述第二规则表示，把传输协议是TCP，源端口是443的报文，打上标签值1；
在iptables规则中添加第三规则：
iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark2；
所述第三规则表示，把传输协议是TCP，目的端口是443的报文，打上标签值2；
在iptables规则中添加第四规则：
iptables -t mangle -A OUTPUT -p tcp --sport 80 -j MARK --set-mark2；
所述第四规则表示，把传输协议是TCP，源端口是80的报文，打上标签值2。
优选的，方法还包括策略路由配置，具体的：
将带有标签值1的报文查询路由表100：
ip rule add fwmark 1 lookup 100；
建立编号为100的路由表，设置内容为从代理系统的虚拟网卡路由到代理系统的应用层：
ip route add local 0.0.0.0/0 dev lo table 100；
配置带有标签值2的报文查询路由表200：
ip rule add fwmark 0x2 table 200；
建立号为200的路由表，让报文通过虚拟网卡tun1去发送：
ip route add default via 10.0.0.2 dev tun1 table 200。
第二方面，本发明还提供了一种HTTP转HTTPS双向透明代理的装置，用于实现第一方面所述的HTTP转HTTPS双向透明代理的方法，所述装置包括：
至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述处理器执行，用于执行第一方面所述的HTTP转HTTPS双向透明代理的方法。
第三方面，本发明还提供了一种非易失性计算机存储介质，所述计算机存储介质存储有计算机可执行指令，该计算机可执行指令被一个或多个处理器执行，用于完成第一方面所述的HTTP转HTTPS双向透明代理的方法。
本发明实现了一种针对HTTP转HTTPS的双向的透明代理方法和装置，在数据链路层，由程序直接控制报文接收和发送，不改变原始报文的MAC地址，在数据链路层对上下游的路由设备实现了双向透明；在IP传输层，可以通过结合虚拟网卡技术和Tproxy技术，不改变源IP地址和目的IP地址，实现了客户端方向和服务器方向的双向透明；在应用层上，实现了HTTP转HTTPS代理，代理系统和客户端维持一个HTTP连接，同时和服务器端维持一个HTTPS连接，实现了应用层的双向透明代理。
【附图说明】
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例中所需要使用的附图作简单地介绍。显而易见地，下面所描述的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种HTTP转HTTPS双向透明代理的系统整体架构图；
图2是本发明实施例提供的一种报文在代理系统内部的流向示意图；
图3是本发明实施例提供的一种HTTP转HTTPS双向透明代理的方法流程示意图；
图4是本发明实施例提供的一种HTTP转HTTPS双向透明代理的方法流程示意图；
图5是本发明实施例提供的一种HTTP转HTTPS双向透明代理的方法流程示意图；
图6是本发明实施例提供的一种数据包接收模块的处理方法流程示意图；
图7是本发明实施例提供的一种数据包发送模块的处理方法流程示意图；
图8是本发明实施例提供的一种配置流程的处理方法流程示意图；
图9是本发明实施例提供的一种代理系统整体工作流程；
图10是本发明实施例提供的一种交互流程A的信令图；
图11是本发明实施例提供的一种交互流程B的信令图；
图12是本发明实施例提供的一种交互流程C的信令图；
图13是本发明实施例提供的一种服务器发送给客户端的原始的HTTP响应头部字段内容示意图；
图14是本发明实施例提供的一种表示被代理系统修改后的HTTP响应头部内容示意图；
图15是本发明实施例提供的一种HTTP转HTTPS双向透明代理的装置结构示意图。
【具体实施方式】
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
在本发明的描述中，术语“内”、“外”、“纵向”、“横向”、“上”、“下”、“顶”、“底”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明而不是要求本发明必须以特定的方位构造和操作，因此不应当理解为对本发明的限制。
本发明中使用的透明代理搭建方法，没有使用IP地址修改，也没有采用网桥设备，而是采用直接从底层读取网卡的数据包，然后结合虚拟网卡技术对数据包进行代理。此方法的特点是，使用程序直接控制数据报文的网卡收发，而不再依赖于操作系统对报文进行路由。这样的好处有两个：
一是避免了大量的路由配置工作，如对网桥配置IP地址，配置操作系统路由表，配置网桥默认网关地址，在操作系统中配置默认网关的ARP地址等等；
二是避免了获取当前代理系统上下游路由节点的相关地址信息，代理系统不需要提前知道数据报文路由的下一跳节点的IP地址和MAC地址，因为代理系统内部保留了原始报文的MAC地址，在接收和发送报文时，并不会修改原始报文的MAC地址，代理系统对于上下游的路由设备来说是透明的，这样一来使得代理系统可以方便的嵌入到复杂的网络环境中。
传统针对HTTPS的代理，一般只是做TCP层的数据中转，在HTTPS客户端和HTTPS服务器之间单纯的做数据转发。之所以会这样，是因为HTTPS本身是可以防止中间人进行数据修改的，在没有合法证书的情况下，如果中间人对HTTPS流量进行代理，会导致HTTPS客户端报错，比如典型的HTTPS客户端是浏览器，浏览器会对终端用户进行报警，警告用户流量可能受到劫持。为了应对HTTPS流量难以解密和代理的问题，本发明提出了一种HTTP转HTTPS代理，绕过了HTTPS对客户端流量加密的过程，解决了HTTPS代理下，客户端端浏览器会由于不信任代理系统的证书而产生的证书告警问题。
本发明实现了一种针对HTTP转HTTPS的双向的透明代理系统，在数据链路层，由程序直接控制报文接收和发送，不改变原始报文的MAC地址，在数据链路层对上下游的路由设备实现了双向透明；在IP传输层，结合虚拟网卡技术和Tproxy技术，不改变源IP地址和目的IP地址，实现了客户端方向和服务器方向的双向透明；在应用层上，实现了HTTP转HTTPS代理，代理系统和客户端维持一个HTTP连接，同时和服务器端维持一个HTTPS连接，实现了应用层的双向透明代理。
具体技术方案包含两部分，第一部分是透明代理技术的底层实现，包括报文收发和实现IP地址透明；第二部分是代理系统如何和客户端、服务器端进行报文交互。
透明代理底层实现：
整体角色分为三类，客户端、代理系统和服务器。代理系统同时维护了两个连接，一个是和客户端建立的连接，一个是和服务器建立的连接，代理系统在两个连接之间中转数据。
如图1所示，代理系统整体上分为四大模块，一是数据包收发模块，二是虚拟网卡模块，三是数据包路由模块，四是代理系统模块。
数据包收发模块主要是负责底层的报文收发，它由数据包接收模块和数据包发送模块组成，数据包接收模块负责从网卡接收数据包，并且转发到虚拟网卡设备上；虚拟网卡模块是一个工作在传输层的标准的TUN虚拟网卡，它主要负责让收到的IP数据包进入操作系统的协议栈进行解析；数据包路由模块主要负责把目标报文路由到代理程序模块去处理，具体包括iptables规则配置，策略路由配置等。默认情况下，操作系统见到目的IP地址不是自身IP的数据包，是不会传递给应用层的，而是丢弃或者转发，由于数据包路由模块配置了本发明定制的路由规则，使得任意目的IP地址的数据包都可以投递到应用层的代理程序模块去处理，这里主要使用了Tproxy技术。代理程序模块工作在操作系统的应用层，是一个应用程序，它为了能够处理任意目的IP地址的报文，也就是支持Tproxy机制，在socket属性上设置了IP_TRANSPARENT参数，这样就可以 接受任意目的IP地址的socket连接，同时也可以以任意IP地址为源IP产生数据报文，也就是使socket可以绑定任意IP地址。
数据包在代理系统内部的流向如图2所示，代理程序模块工作在应用层，它同时和客户端、服务器维持了两个socket连接，编号为1，2，3，4，5，6的几条线所组成的流向代表了代理系统和客户端维持的连接，编号为7，8，9，10，11，12的几条线所组成的流向代表了代理系统和服务器端维持的连接，编号为13的线条，表示非HTTP的流量，不会投递到应用层去处理，而是由网卡直接转发出去。数据包收发模块负责处理的流量是编号为2,5,8,11,13这些流量报文，具体来说，数据包接收模块负责处理编号2,11,13所代表的流量报文，数据包发送模块负责处理编号5，8所代表的流量报文；数据包路由模块负责处理编号为3,4,7,12所代理的流量报文。
代理程序模块的报文处理流程：
本发明中的HTTP转HTTPS代理，本质上是实现了一个HTTP重定向跳转劫持。现代网站逐渐采用HTTPS保护传输内容不被修改，但是为了兼容用户默认访问的HTTP流量，一般会同时保留80端口的HTTP服务，当用户访问80端口的HTTP服务时，会利用HTTP协议的重定向机制，向用户发送301/302重定向命令，把用户导向对应的HTTPS服务上面，用户的HTTP客户端收到重定向之后，会按照HTTP协议的规范去访问重定向之后的HTTPS服务。本发明实现的HTTP转HTTPS代理功能，本质上就是利用并劫持了这个重定向机制。
代理程序模块对客户端和服务器的报文转发和交互流程：
当客户端发起一个HTTP请求时，本发明的代理系统先对目标域名进行探测，具体探测过程是向对应域名发起HTTP请求，并查看HTTP响应状态码是否是301/302等重定向状态码，如果是重定向，而且重定向之后的网址是网址的HTTPS版本，那就说明域名满足HTTPS重定向条件，就向对应的域名发起HTTPS请求，并把返回的HTTPS的响应内容，以HTTP通道传递给客户端。通过这样操作，代理系统和客户端维持一个HTTP连接，但是和服务器维持一个HTTPS连接，成功的劫持了客户端的302重定向动作，使客户端始终不能升级到HTTPS连接。如果探测的结果是目标域名不满足HTTPS重定向条件，那么代理系统就同时和客户端、服务器都使用HTTP通道通信，在二者之间中转数据，此时就工作在HTTP代理模式下。
此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
实施例1:
本发明实施例1提供了一种HTTP转HTTPS双向透明代理的方法，如图3所示，包括：
在步骤101中，代理系统收到客户端发送的第一HTTP请求后，解析HTTP头部字段，获得Host字段，把Host字段和内置的域名列表进行比对，并存储所述第一HTTP请求内容；其中，所述域名列表，用于存储满足HTTPS重定向的域名。
在步骤102中，若发现Host域名在列表中，代理系统向服务器发起目标端 口为443的TCP握手，以便建立第一TCP通道和通过所述第一TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址；其中，所述TLS协商过程包括：协商加密套件、传递证书、校验证书和计算秘钥中的一项或者多项过程。
在步骤103中，代理系统通过TLS协商建立的第一TLS通道，向服务器发送第一HTTPS请求(在本发明其他实施例中也被表述为HTTP请求，此处是为了直观的表现技术特性，因此描述为HTTPS请求)；其中，所述第一HTTPS请求中携带所述第一HTTP请求内容经过加密后的内容。
在步骤104中，代理系统接收服务器返回的第一HTTPS响应，代理系统删除第一HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第一HTTPS响应内容解密为明文后，透传给客户端。
本发明实施例实现了一种针对HTTP转HTTPS的双向的透明代理方法，在数据链路层，由程序直接控制报文接收和发送，不改变原始报文的MAC地址，在数据链路层对上下游的路由设备实现了双向透明；在IP传输层，可以通过结合虚拟网卡技术和Tproxy技术，不改变源IP地址和目的IP地址，实现了客户端方向和服务器方向的双向透明；在应用层上，实现了HTTP转HTTPS代理，代理系统和客户端维持一个HTTP连接，同时和服务器端维持一个HTTPS连接，实现了应用层的双向透明代理。
结合本发明实施例，还存在一种扩展实现方案，作为实施例1中步骤102的并列可选情况，若发现Host域名不在列表中，如图4所示，所述方法还包括：
在步骤105中，代理系统向服务器发起目标端口为80的TCP握手，以便建立第二TCP通道。
在步骤106中，代理系统通过所述第二TCP通道，向服务器发送第二HTTP请求，所述第二HTTP请求中携带所述第一HTTP请求内容。
在步骤107中，代理系统接收服务器返回的第二HTTP响应，并检查所述第二HTTP响应是否是HTTPS重定向。
其中，检查所述第二HTTP响应是否是HTTPS重定向，具体包括：
检查所述第二HTTP响应的状态码是否在300～399之间(因为重定向功能本身不仅仅是做HTTPS重定向，而是可以进行任意URL重定向，HTTPS重定向只是其中的一种应用)，并且，检查重定向的目标地址是原来Host的HTTPS版本；其中，根据HTTP协议规范，此区间的响应状态码表示重定向。
在步骤108中，若发现是HTTPS重定向，则代理系统把Host加入域名列表中，并丢弃掉所述第二HTTP响应，同时向服务器发送TCP Reset报文，关闭所述第一TCP通道。
结合本发明实施例，还存在一种扩展实现方案，如图5所示，在执行完上述步骤108之后，优选的，所述方法还包括：
在步骤109中，代理系统向服务器发起目标端口为443的TCP握手，以便建立第二TCP通道和通过所述第二TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址，其中，所述TLS协商过程包括：协商加密套件、 传递证书、校验证书和计算秘钥中的一项或者多项过程。
在步骤110中，代理系统通过TLS协商建立的第二TLS通道，向服务器发送第二HTTPS请求；其中，所述第二HTTPS请求中携带所述第二HTTP请求内容经过加密后的内容；
在步骤111中，代理系统接收服务器返回的第二HTTPS响应，代理系统删除第二HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第二HTTPS响应内容解密为明文后，透传给客户端。
作为完整技术方案实现考虑，若在上述步骤108中判定结果是另一种情况，则相应的技术方案实现内容表现为：若发现不是HTTPS重定向，所述方法还包括：代理系统将接收到的第二HTTP响应透传给所述客户端。
在本发明实施例中，还提出了一种优选的代理系统框架结构，如图2所示，包括：数据包收发模块、虚拟网卡模块、数据包路由模块、代理程序模块，具体的：
数据包收发模块，用于底层的报文收发，由数据包接收模块和数据包发送模块组成，数据包接收模块负责从物理网卡接收数据包，转发到虚拟网卡上；
虚拟网卡模块为工作在传输层的标准的TUN虚拟网卡，用于将收到的IP数据包经由操作系统的协议栈进行解析；
数据包路由模块，用于将目标报文路由到代理程序模块去处理，数据包路由模块中包括iptables规则配置和策略路由配置。默认情况下，操作系统见到目的IP地址不是自身IP的数据包，是不会传递给应用层的，而是丢弃或者转发，由于数据包路由模块配置了特殊的路由规则，使得任意目的IP地址的数据包都可以投递到应用层的代理程序模块去处理，这里主要使用了Tproxy技术。
所述代理程序模块工作在操作系统的应用层，并且支持Tproxy机制，具体的：
在socket属性上设置IP_TRANSPARENT参数，使得接受任意目的IP地址的socket连接，同时也可以以任意IP地址为源IP产生数据报文，从而使socket可以绑定任意IP地址。
作为支撑本发明实施例1相关方法步骤实现的底层机制要素，方法还包括：
在iptables规则中添加第一规则：
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY--tproxy-mark 0x1/0x1 --on-port 0；
所述第一规则表示，把传输协议是TCP，目的端口是80的报文，打上标签值1；
在iptables规则中添加第二规则：
iptables -t mangle -A PREROUTING -p tcp --sport 443 -j MARK--set-mark 1；
所述第二规则表示，把传输协议是TCP，源端口是443的报文，打上标签值1；
在iptables规则中添加第三规则：
iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 2；
所述第三规则表示，把传输协议是TCP，目的端口是443的报文，打上标签值2；
在iptables规则中添加第四规则：
iptables -t mangle -A OUTPUT -p tcp --sport 80 -j MARK --set-mark 2；
所述第四规则表示，把传输协议是TCP，源端口是80的报文，打上标签值2。
配套上述的iptables规则，还需要对9应的策略路由配置来实现，具体的：
将带有标签值1的报文查询路由表100(此处的路由表100仅仅是为了方便下面的指令的呈现，在实际使用过程中，所述100还可以被表述为其他自定义的字符串，因此，不应将其作为本发明保护范围的特殊限定)：
ip rule add fwmark 1 lookup 100；
建立编号为100的路由表，设置内容为从代理系统的虚拟网卡路由到代理系统的应用层：
ip route add local 0.0.0.0/0 dev lo table 100；
配置带有标签值2的报文查询路由表200：
ip rule add fwmark 0x2 table 200；
建立号为200的路由表，让报文通过虚拟网卡tun1(实际还可以是其他编号的虚拟网卡，例如tun2，tun5等等，在此不做特殊限定)去发送：
ip route add default via 10.0.0.2 dev tun1 table 200。
接下来，本发明将通过多个实施例来将本发明中涉及的代理系统中的主要功能模块的方法执行过程进行逐一阐述，在相应的实施例中将不再延续本发明实施例1中的第一、第二表述，需要指出的是，在本发明实施例中所使用的第一、第二、第三前缀，仅仅是为了引述过程中更清楚的区别对象而使用，并不具备特殊的限定意义，而接下来的实施例则可以通过上下文的表述即便不增加第一、第二的标定也能够与本发明实施例中所涉及的相关对象进行关联，在此和后续内容不再一一赘述。
实施例2：
数据包接收模块的工作流程如图6所示：
在S121步骤中，从物理网卡接收报文，此步骤中的网卡同时包括上行网卡和下行网卡，此时程序接收报文并不依赖于操作系统对报文的解析和处理，而是直接从网卡读取报文，此时报文包含以太网头部，IP头部等，是一个完整的报文。
在S122步骤中，解析报文，此步骤是对报文进行解析，获取报文的源MAC地址，源IP地址，源端口，目的MAC地址，目的IP地址，目的端口，同时记录下IP地址和MAC地址的对应关系，存储到IP-MAC对应表中。
在S123步骤中，检查报文是否是目标报文，此步骤的检查条件有两个，一是检查目的端口是否是TCP的80端口，二是检查源端口是否是TCP的443端口， 这两个条件是或的关系，换句话说只要有一个条件命中了，此报文就属于目标报文。
在S124步骤中，剥除以太网头部，此步骤是把S123步骤中命中的报文的以太网头部剥离掉，只保留IP头部以上的数据部分。
在S125步骤中，发送到虚拟网卡，此步骤是把S124步骤中产生的IP数据报文，投递到虚拟网卡设备上。
在S126步骤中，发送到对端物理网卡，是指把在S123步骤中没有命中的报文，直接投递给对端的物理网卡。这里对端物理网卡，是指如果报文从上行网卡接收，那么就从下行网卡转出；反之如果报文从下行网卡接收，那么就从上行网卡转出。
实施例3：
数据包接收模块的工作流程如图7所示：
在S131步骤中，从虚拟网卡接收报文，此步骤是指从虚拟网卡设备上读取报文，此时的虚拟网卡是指TUN类型的网卡，接收到的报文是带IP头部的IP数据报文，没有以太网头部。
在S132步骤中，添加以太网头部，此步骤是指为IP数据报文添加以太网头部，包括源MAC和目的MAC地址。添加的原则就是，查询数据包接收模块生成的IP-MAC对应表，设置源MAC为源IP对应的MAC地址，设置目的MAC为目的IP对应的MAC地址。
在S133步骤中，检测目标端口是否是443端口，如果报文的目标端口是TCP的443端口，那么就满足判断条件。
在S134步骤中，发送到物理网卡-下行口，此步骤是把补全以太网头部的以太网数据报文，发送到下行口的物理网卡上。
在S135步骤中，发送到物理网卡-上行口，此步骤是把补全以太网头部的以太网数据报文，发送到上行口的物理网卡上。
实施例4：
虚拟网卡和路由配置模块，配置过程涉及到路由转发配置，虚拟网卡配置，iptables规则配置，策略路由配置等几个方面，如图8所示。
S141是路由转发配置，目的是开启Linux系统的路由转发功能。默认情况下，操作系统见到目的IP地址不是自身IP地址的数据报文，会丢弃掉；如果开启了路由转发功能，那么操作系统对目的IP地址不是自身IP的数据包是会做转发处理，而不是丢弃掉，此时相当于操作系统对数据报文做了路由处理。开启路由转发的方法比较简单，可以通过命令的方式进行配置，也可以通过写配置文件的方式进行配置，不同的Linux系统发行版本可能网络配置文件的名称和位置也不相同。通过命令行来配置的命令如下：
sysctl -w net.ipv4.ip_forward＝1；
S142是虚拟网卡配置，目的是生成一个虚拟网卡，用来接收IP分组报文，使得IP分组进入本机的协议栈进行处理。
典型的配置方法如下：
首先创建一个TUN类型的虚拟网卡设备，命名此设备为tun5:
ip tuntap add mode tun tun5；
激活此虚拟网卡：
ip link set tun5 up；
为此虚拟网卡添加IP地址：
ip addr add 10.0.0.2/24 dev tun5；
这里的虚拟网卡IP地址不会影响整体代理系统的功能，可以任意设置。
S143是iptables规则配置，此部分主要目的是给目标数据包打标签。目标数据包包含两部分，第一部分是图2中3号线条和12号线条所代表的流量，第二部分是图2中4号线条和7号线条所代表的流量。给3号线条和12号线条所代表的流量打标签，是为了使目的地址不是本机IP地址的报文可以路由到本机的应用层去处理，而不是被转发走；给4号线条和7号线条所代表的流量打标签，是因为这部分流量查询默认的路由表，是会被路由到物理网卡上的，本发明为了使它路由到虚拟网卡上，所以也打上标签。
第一类规则，是为了给图2中3号线条和12号线条所代表的流量打标签，典型的配置方法如下，使用iptables工具添加如下规则：
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY--tproxy-mark 0x1/0x1 --on-port 0；
这条规则表示，把传输协议是TCP，目的端口是80的报文，打上标签值1，向应用层投递到端口为80的程序去处理，此部分流量匹配的是图2中3号线条所代表的流量。
iptables -t mangle -A PREROUTING -p tcp --sport 443 -j MARK--set-mark 1；
这条规则表示，把传输协议是TCP，源端口是443的报文，打上标签值1，此部分流量匹配的是图2中12号线条所代表的流量。
第二类规则是为了给图2中4号线条和7号线条所代表的流量打标签：
iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 2；
iptables -t mangle -A OUTPUT -p tcp --sport 80 -j MARK --set-mark 2；
S144是策略路由配置，这部分是为了和iptables规则配合使用。主要包含两部分，一是让虚拟网卡的报文可以投递给应用层程序处理，即图2中3号线条和12号线条所代表的流量报文会匹配到这条策略路由；二是让应用程序产生的报文可以投递给虚拟网卡，而不是物理网卡，图2中4号线条和7号线条所代表的流量报文会匹配到这条策略路由。
具体配置方法如下，让带有标签值1的报文查询路由表100：
ip rule add fwmark 1 lookup 100；
建立编号为100的路由表，设置内容为路由到应用层：
ip route add local 0.0.0.0/0 dev lo table 100；
配置带有标签值2的报文查询路由表200：
ip rule add fwmark 0x2 table 200；
建立号为200的路由表，让报文通过虚拟网卡tun5去发送：
ip route add default via 10.0.0.2 dev tun5 table 200。
实施例5：
图9是代理程序模块对报文的处理过程，代理程序模块同时和客户端、服务器保持HTTP或者HTTPS连接，在客户端和服务器之间中转数据。
角色：
C：Client，表示客户端，典型的HTTP客户端是浏览器；
P：Proxy，表示代理系统；
S：Server，表示服务器端，典型的HTTP服务器端如访问的各种网站；
交互方向：
->表示单方向发送报文；
<->表示双方相互发送和接收报文；
名称解释：
TCP：Transmission Control Protocol，传输控制协议；
HTTP：HyperText Transfer Protocol,超文本传输协议；
TLS：Transport Layer Security，安全传输层协议；
HSTS：HTTP Strict Transport Security,HTTP严格安全传输；
TCP握手：是指TCP协议中双方建立连接时，需要执行的三次握手过程；
TLS握手：是指TLS协议中，客户端和服务器端建立TLS连接的协商过程，此过程有多个报文交互；
HTTP GET：是指HTTP协议中，客户端发起的资源请求；
HTTP Response:指HTTP协议中，服务器端回复客户端的响应内容；
Host:是指HTTP请求报文中的一个HTTP头部字段，它的值就是访问的目标域名；
如图9所示，代理系统和客户端、服务器的报文交互过程如下：
在S201步骤中，客户端和代理系统发起TCP握手，建立TCP连接，目标端口是80端口。在客户端看来，是在和服务器进行TCP连接，但是因为此时的流量已经被路由给代理系统，所以实际上客户端是和代理系统建立了TCP连接。
在S202步骤中，客户端向代理系统发起HTTP请求，此时的HTTP是明文请求，目标端口是80端口。在客户端看来，是在向服务器发起HTTP请求，因为此时的流量已经被路由给代理系统，所以实际上客户端是把请求发给了代理系统。
在S203步骤中，代理系统受到客户端发送的HTTP请求后，解析HTTP头部字段，获得Host字段，并把Host字段和内置的域名列表进行比对；这里的域名列表，是指所有满足HTTPS重定向的域名，如果一个域名在此列表职工，说明之前已经对此域名进行过HTTPS重定向探测，而且此域名会进行HTTPS重定向。如果发现Host域名在列表中，则进入S204步骤；否则进入S210步骤。
在S204步骤中，代理系统向服务器发起TCP握手，目标端口443；此时服务器看到的IP地址就是客户端的IP地址，不会发现代理系统的存在。
在S205步骤中，代理系统和服务器进行TLS协商过程，此协商过程是基于 S204中建立的TCP通道的；TLS协商过程，会有多个报文交互，这个过程中，双方会完成协商加密套件，传递证书，校验证书，计算秘钥等工作。
在S206步骤中，代理系统向服务器发送HTTP请求，此请求是基于S205步骤中建立的TLS通道的，此HTTP请求是一个加密的请求，具体的请求内容是从S202步骤中获取的；
在S207步骤中，服务器回复给代理系统HTTP响应内容，此HTTP响应是基于S205步骤中建立的TLS通道的，是经过加密的，不是明文的。
在S208步骤中，处理HTTP响应。代理系统收到服务器回复的HTTP响应之后，需要对响应内容进行处理，才能发送给客户端。这里之所以要处理HTTP响应，是因为要进行HTTPS到HTTP的适配。HTTP头部字段中有些字段的属性涉及到了HTTPS传输方式，典型的有两个字段，一是cookie字段的secure属性，此属性表示此cookie只能在HTTPS通道中传输，不能在HTTP通道中传输；如果不删除这个属性，会导致此cookie不会在HTTP这种明文传输通道进行传送，会引起各种服务器鉴权问题；二是HSTS，HSTS的作用是强制客户端使用HTTPS与服务器创建连接，HSTS的设置方法是在HTTP响应头中包含Strict-Transport-Security字段。HTTP协议规定，非加密传输时设置的HSTS字段无效，但是出现在HTTP明文传输通道中的HSTS字段会引起客户端的怀疑，所以要删除掉这个字段。
在S209步骤中，代理系统向客户端返回HTTP响应，此时的HTTP响应内容，可能是经过S208步骤适配过的，也可能是没有改动过的；如果是从S213步骤来到此步骤，则不需要对HTTP响应内容进行修改。
在S210步骤中，代理系统向服务器发起TCP握手，目标端口是80端口。
在S211步骤中，代理系统向服务器发送HTTP请求，此请求是基于S210步骤中建立的TCP通道的，此HTTP请求是一个明文的请求，具体的请求内容是从S202步骤中获取的。
在S212步骤中，服务器回复给代理系统HTTP响应内容，此HTTP响应是基于S210步骤中建立的TCP通道的，是明文传输的。
在S213步骤中，检查HTTP响应是否是HTTPS重定向。此步骤主要检查S212步骤中获取的HTTP响应内容，一是检查HTTP响应的状态码是否在300～399之间，根据HTTP协议规范，此区间的响应状态码表示重定向；二是检查重定向的目标地址是否是原来Host的HTTPS版本，因为重定向功能本身不仅仅是做HTTPS重定向，而是可以进行任意URL重定向，HTTPS重定向只是其中的一种应用。检查后如果发现是HTTPS重定向，那么就跳转到S214步骤，否则跳转到S209。
在S214步骤中，代理系统和服务器断开TCP连接。代理系统此时发现HTTP响应内容是HTTPS重定向，此时代理系统并不去转发此响应给客户端，而是丢弃掉此HTTP响应，同时向服务器发送TCP Reset报文，关闭TCP通道；
在S215步骤中，把Host加入域名列表中。此时，代理系统已经完成了对目标域名的探测，得知目标域名满足HTTPS重定向的条件，所以加入域名列表，下次代理系统处理此域名时，直接进行代理即可，不需要重复探测；
所有经过代理系统的流量按照不同的情况，可以分为3种交互流程：
第一种、交互流程A：HTTP代理模式，即代理系统和客户端使用HTTP连接，和服务器端也使用HTTP连接；代理系统先对服务器进行探测，发现服务器不满足HTTPS重定向条件，进而转到HTTP代理模式；
第二种、交互流程B：HTTP转HTTPS代理模式，即代理系统和客户端使用HTTP连接，和服务器之间使用HTTPS连接；代理系统之前对此域名进行过探测，发现它满足HTTPS重定向条件，直接进行HTTPS代理；
第三种、交互流程C：HTTP转HTTPS代理模式，叠加初始探测流程；代理系统先对服务器进行探测，发现服务器满足HTTPS重定向条件，进而转到HTTPS代理；
当代理系统使用交互流程A时，工作步骤是S201，S202，S203，S210，S211，S212，S213，S209，报文的交互流程如图10所示。假设客户端使用的IP地址为1.1.1.1，端口为1024；服务器端的IP地址为2.2.2.2，端口为80；代理系统本身的IP地址为3.3.3.3。客户端和代理系统之间的连接使用的IP地址是1.1.1.1和2.2.2.2，端口是1024和80，此时代理系统假装自己是服务器来和客户端建立连接。代理系统和服务器之间的连接使用的IP地址是1.1.1.1和2.2.2.2，端口是2000和80，此时代理系统假装自己是客户端来和服务器建立连接。可以看到在整个通信过程中，不会出现代理系统的自身IP地址，只有源端口发生了改变，因为源端口是随机选择的，所以即使改变了源端口，代理系统仍然是一个双向透明代理系统。经过代理系统后，报文的MAC地址，IP地址，HTTP报文内容等是保持不变的，发生改变的是TCP协议的源端口字段。
当代理系统使用交互流程B时，工作步骤是S201，S202，S203，S204，S205，S206，S207，S208，S209，报文的交互流程如图11所示。假设客户端使用的IP地址为1.1.1.1，端口为1024；服务器端的IP地址为2.2.2.2，端口为80和443；代理系统本身的IP地址为3.3.3.3。当客户端向服务器发起HTTP访问时，代理系统会使用2.2.2.2的IP地址和80端口和客户端建立起一个HTTP连接，同时代理系统使用1.1.1.1的IP地址和一个随机源端口和真实服务器建立一个HTTPS连接。在整个通信过程中，客户端和代理系统之间是一个HTTP连接；代理系统和服务器之间是一个HTTPS连接，此连接是一个可信的连接，因为这时代理系统是作为客户端在使用，证书是服务器的证书，是可信的证书。在整个通信过程中，不会出现代理系统的自身IP地址，只有源端口发生了改变，因为源端口是客户端随机选择的，所以即使改变了源端口，代理系统仍然是一个双向透明代理系统。经过代理系统后，报文的MAC地址，IP地址是保持不变的，发生改变的是TCP协议的源端口字段，还有就是HTTP协议变成了HTTPS协议，可能发生变化的是HTTP头部字段。
如图13所示，表示服务器发送给客户端的原始的HTTP响应头部字段；请注意，里面有一个Strict-Transport-Security头部字段，另外就是Set-Cookie字段对cookie设置了一个secure的属性。
如图14所示，表示被代理系统修改后的HTTP响应头部；代理系统删除了cookie的secure属性，同时删除了Strict-Transport-Security头部字段。
当代理系统使用交互流程C时，工作步骤是S201，S202，S203，S210，S211， S212，S213，S214，S215，S204，S205，S206，S207，S208，S209，报文的交互流程如图12所示。假设客户端使用的IP地址为1.1.1.1，端口为1024；服务器端的IP地址为2.2.2.2，端口为80和443；代理系统本身的IP地址为3.3.3.3。当客户端向服务器发起HTTP访问时，代理系统会使用2.2.2.2的IP地址和80端口和客户端建立起一个HTTP连接，同时代理系统使用1.1.1.1的IP地址和一个随机源端口和真实服务器建立一个HTTP连接，并对HTTP的响应是否是HTTPS重定向进行探测。一旦探测完成，发现服务器满足HTTPS重定向条件，代理系统则关闭和服务器之间的HTTP连接，同时使用1.1.1.1的IP地址和一个随机源端口和服务器建立一个HTTPS连接。在之后的通信过程中，客户端和代理系统之间是一个HTTP连接，代理系统和服务器之间是一个HTTPS连接，此HTTPS连接是一个可信的连接，因为这时代理系统是作为客户端在使用。在整体通信过程中，源端口发生了改变，源端口是客户端随机选择的，本身是可以变化的。在代理系统和客户端的连接中，服务器端口是80；在代理系统和服务器的连接中，服务器端口是443，但是在整个通信过程中，不会出现代理系统的自身IP地址，代理系统是一个双向透明代理系统。
实施例6：
如图15所示，是本发明实施例的HTTP转HTTPS双向透明代理的装置的架构示意图。本实施例的HTTP转HTTPS双向透明代理的装置包括一个或多个处理器21以及存储器22。其中，图15中以一个处理器21为例。
处理器21和存储器22可以通过总线或者其他方式连接，图15中以通过总线连接为例。
存储器22作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序和非易失性计算机可执行程序，如实施例1中的HTTP转HTTPS双向透明代理的方法。处理器21通过运行存储在存储器22中的非易失性软件程序和指令，从而执行HTTP转HTTPS双向透明代理的方法。
存储器22可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中，存储器22可选包括相对于处理器21远程设置的存储器，这些远程存储器可以通过网络连接至处理器21。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
所述程序指令/模块存储在所述存储器22中，当被所述一个或者多个处理器21执行时，执行上述实施例1中的HTTP转HTTPS双向透明代理的方法，例如，执行以上描述的图3-图9所示的各个步骤。
值得说明的是，上述装置和系统内的模块、单元之间的信息交互、执行过程等内容，由于与本发明的处理方法实施例基于同一构思，具体内容可参见本发明方法实施例中的叙述，此处不再赘述。
本领域普通技术人员可以理解实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁盘或光盘等。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种HTTP转HTTPS双向透明代理的方法，其特征在于，包括：
    代理系统收到客户端发送的第一HTTP请求后，解析HTTP头部字段，获得Host字段，把Host字段和内置的域名列表进行比对，并存储所述第一HTTP请求内容；其中，所述域名列表，用于存储满足HTTPS重定向的域名；
    若发现Host域名在列表中，代理系统向服务器发起目标端口为443的TCP握手，以便建立第一TCP通道和通过所述第一TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址；
    代理系统通过TLS协商建立的第一TLS通道，向服务器发送第一HTTPS请求；其中，所述第一HTTPS请求中携带所述第一HTTP请求内容经过加密后的内容；
    代理系统接收服务器返回的第一HTTPS响应，代理系统删除第一HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第一HTTPS响应内容解密为明文后，透传给客户端。
  2. 根据权利要求1所述的HTTP转HTTPS双向透明代理的方法，其特征在于，若发现Host域名不在列表中，所述方法还包括：
    代理系统向服务器发起目标端口为80的TCP握手，以便建立第二TCP通道；
    代理系统通过所述第二TCP通道，向服务器发送第二HTTP请求，所述第二HTTP请求中携带所述第一HTTP请求内容；
    代理系统接收服务器返回的第二HTTP响应，并检查所述第二HTTP响应是否是HTTPS重定向；
    若发现是HTTPS重定向，则代理系统把Host加入域名列表中，并丢弃掉所述第二HTTP响应，同时向服务器发送TCP Reset报文，关闭所述第一TCP通道。
  3. 根据权利要求2所述的HTTP转HTTPS双向透明代理的方法，其特征在于，所述方法还包括：
    代理系统向服务器发起目标端口为443的TCP握手，以便建立第二TCP通道和通过所述第二TCP通道进行TLS协商过程；其中，所述TCP握手中使用的客户端的IP地址；
    代理系统通过TLS协商建立的第二TLS通道，向服务器发送第二HTTPS请求；其中，所述第二HTTPS请求中携带所述第二HTTP请求内容经过加密后的内容；
    代理系统接收服务器返回的第二HTTPS响应，代理系统删除第二HTTPS响应的HTTP头部字段中cookie字段的secure属性，以及HSTS中包含的Strict-Transport-Security字段后，并将第二HTTPS响应内容解密为明文后，透传给客户端。
  4. 根据权利要求2所述的HTTP转HTTPS双向透明代理的方法，其特征在于，检查所述第二HTTP响应是否是HTTPS重定向，具体包括：
    检查所述第二HTTP响应的状态码是否在300～399之间，并且，检查重定向 的目标地址是原来Host的HTTPS版本；其中，根据HTTP协议规范，此区间的响应状态码表示重定向。
  5. 根据权利要求2所述的HTTP转HTTPS双向透明代理的方法，其特征在于，若发现不是HTTPS重定向，所述方法还包括：
    代理系统将接收到的第二HTTP响应透传给所述客户端。
  6. 根据权利要求1-5任一所述的HTTP转HTTPS双向透明代理的方法，其特征在于，代理系统包括：数据包收发模块、虚拟网卡模块、数据包路由模块、代理程序模块，具体的：
    数据包收发模块，用于底层的报文收发，由数据包接收模块和数据包发送模块组成，数据包接收模块负责从物理网卡接收数据包，转发到虚拟网卡上；
    虚拟网卡模块为工作在传输层的标准的TUN虚拟网卡，用于将收到的IP数据包经由操作系统的协议栈进行解析；
    数据包路由模块，用于将目标报文路由到代理程序模块去处理，数据包路由模块中包括iptables规则配置和策略路由配置。
  7. 根据权利要求6所述的HTTP转HTTPS双向透明代理的方法，其特征在于，所述代理程序模块工作在操作系统的应用层，并且支持Tproxy机制，具体的：
    在socket属性上设置IP_TRANSPARENT参数，使得接受任意目的IP地址的socket连接，同时也可以以任意IP地址为源IP产生数据报文，从而使socket可以绑定任意IP地址。
  8. 根据权利要求6所述的HTTP转HTTPS双向透明代理的方法，其特征在于，方法还包括：
    在iptables规则中添加第一规则：
    iptables-t mangle-A PREROUTING-p tcp--dport 80-j TPROXY--tproxy-mark 0x1/0x1--on-port 0；
    所述第一规则表示，把传输协议是TCP，目的端口是80的报文，打上标签值1；
    在iptables规则中添加第二规则：
    iptables-t mangle-A PREROUTING-p tcp--sport 443-j MARK--set-mark 1；
    所述第二规则表示，把传输协议是TCP，源端口是443的报文，打上标签值1；
    在iptables规则中添加第三规则：
    iptables-t mangle-A OUTPUT-p tcp--dport 443-j MARK--set-mark2；
    所述第三规则表示，把传输协议是TCP，目的端口是443的报文，打上标签值2；
    在iptables规则中添加第四规则：
    iptables-t mangle-A OUTPUT-p tcp--sport 80-j MARK--set-mark2；
    所述第四规则表示，把传输协议是TCP，源端口是80的报文，打上标签值2。
  9. 根据权利要求8所述的HTTP转HTTPS双向透明代理的方法，其特征在于，方法还包括策略路由配置，具体的：
    将带有标签值1的报文查询路由表100：
    ip rule add fwmark 1 lookup 100；
    建立编号为100的路由表，设置内容为从代理系统的虚拟网卡路由到代理系统的应用层：
    ip route add local 0.0.0.0/0 dev lo table 100；
    配置带有标签值2的报文查询路由表200：
    ip rule add fwmark 0x2 table 200；
    建立号为200的路由表，让报文通过虚拟网卡tun1去发送：
    ip route add default via 10.0.0.2 dev tun1 table 200。
  10. 一种HTTP转HTTPS双向透明代理的装置，其特征在于，所述装置包括：
    至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述处理器执行，用于执行权利要求1-9任一所述的HTTP转HTTPS双向透明代理的方法。
PCT/CN2021/135682 2021-01-18 2021-12-06 一种http转https双向透明代理的方法和装置 WO2022151867A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110062859.5 2021-01-18
CN202110062859.5A CN112954001B (zh) 2021-01-18 2021-01-18 一种http转https双向透明代理的方法和装置

Publications (1)

Publication Number Publication Date
WO2022151867A1 true WO2022151867A1 (zh) 2022-07-21

Family

ID=76235497

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/135682 WO2022151867A1 (zh) 2021-01-18 2021-12-06 一种http转https双向透明代理的方法和装置

Country Status (2)

Country Link
CN (1) CN112954001B (zh)
WO (1) WO2022151867A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115720222A (zh) * 2022-12-19 2023-02-28 广西大学 基于dpdk的在arm多核架构上实现http转发的方法及存储介质
CN116028980A (zh) * 2023-03-29 2023-04-28 北京中安星云软件技术有限公司 一种数据库防绕过方法、系统、设备及介质
CN117076144A (zh) * 2023-08-17 2023-11-17 合芯科技有限公司 系统并行转换方法、装置、计算机设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954001B (zh) * 2021-01-18 2022-02-15 武汉绿色网络信息服务有限责任公司 一种http转https双向透明代理的方法和装置
CN113810380B (zh) * 2021-08-23 2023-08-01 杭州安恒信息安全技术有限公司 代理层次切换方法、系统、可读存储介质及计算机设备
CN114025030A (zh) * 2021-11-08 2022-02-08 北京天融信网络安全技术有限公司 透明代理实现方法、装置、计算机设备和介质
CN114125030A (zh) * 2021-11-30 2022-03-01 北京天融信网络安全技术有限公司 连接跟踪方法、装置、电子设备和计算机可读存储介质
CN115277837B (zh) * 2022-07-22 2023-04-25 杭州迪普科技股份有限公司 基于代理的重定向方法和装置
CN115242766A (zh) * 2022-08-02 2022-10-25 亚数信息科技（上海）有限公司 一种基于二层网桥的https透明网关的方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005060202A1 (en) * 2003-12-10 2005-06-30 International Business Machines Corporation Method and system for analysing and filtering https traffic in corporate networks
CN102685165A (zh) * 2011-03-16 2012-09-19 中兴通讯股份有限公司 基于代理网关对访问请求进行控制的方法及装置
CN107613036A (zh) * 2017-09-04 2018-01-19 北京新流万联网络技术有限公司 实现https透明代理的方法和系统
CN111314499A (zh) * 2020-02-17 2020-06-19 深信服科技股份有限公司 一种域名代理方法、装置、设备及可读存储介质
CN112954001A (zh) * 2021-01-18 2021-06-11 武汉绿色网络信息服务有限责任公司 一种http转https双向透明代理的方法和装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242336B (zh) * 2008-03-13 2010-12-01 杭州华三通信技术有限公司 远程访问内网Web服务器的方法及Web代理服务器
CN105376209A (zh) * 2014-09-02 2016-03-02 松下电器产业株式会社 网络代理设备和楼宇监控系统及其方法
US9888350B2 (en) * 2015-12-22 2018-02-06 Intel IP Corporation System, method and apparatus for hybrid wireless fine-timing measurement
US20170223054A1 (en) * 2016-02-02 2017-08-03 Cisco Technology, Inc. Methods and Apparatus for Verifying Transport Layer Security Server by Proxy
WO2017161081A1 (en) * 2016-03-16 2017-09-21 Affirmed Networks, Inc. Systems and methods for intelligent transport layer security
US10264079B2 (en) * 2016-05-18 2019-04-16 Cisco Technology, Inc. Fastpath web sessions with HTTP header modification by redirecting clients
US10693893B2 (en) * 2018-01-16 2020-06-23 International Business Machines Corporation Detection of man-in-the-middle in HTTPS transactions independent of certificate trust chain

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005060202A1 (en) * 2003-12-10 2005-06-30 International Business Machines Corporation Method and system for analysing and filtering https traffic in corporate networks
CN102685165A (zh) * 2011-03-16 2012-09-19 中兴通讯股份有限公司 基于代理网关对访问请求进行控制的方法及装置
CN107613036A (zh) * 2017-09-04 2018-01-19 北京新流万联网络技术有限公司 实现https透明代理的方法和系统
CN111314499A (zh) * 2020-02-17 2020-06-19 深信服科技股份有限公司 一种域名代理方法、装置、设备及可读存储介质
CN112954001A (zh) * 2021-01-18 2021-06-11 武汉绿色网络信息服务有限责任公司 一种http转https双向透明代理的方法和装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115720222A (zh) * 2022-12-19 2023-02-28 广西大学 基于dpdk的在arm多核架构上实现http转发的方法及存储介质
CN116028980A (zh) * 2023-03-29 2023-04-28 北京中安星云软件技术有限公司 一种数据库防绕过方法、系统、设备及介质
CN116028980B (zh) * 2023-03-29 2023-08-25 北京中安星云软件技术有限公司 一种数据库防绕过方法、系统、设备及介质
CN117076144A (zh) * 2023-08-17 2023-11-17 合芯科技有限公司 系统并行转换方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN112954001A (zh) 2021-06-11
CN112954001B (zh) 2022-02-15

Similar Documents

Publication Publication Date Title
WO2022151867A1 (zh) 一种http转https双向透明代理的方法和装置
US11870809B2 (en) Systems and methods for reducing the number of open ports on a host computer
US8850553B2 (en) Service binding
US10313397B2 (en) Methods and devices for access control of data flows in software defined networking system
US8473620B2 (en) Interception of a cloud-based communication connection
JP4579934B2 (ja) レガシーノードとｈｉｐノード間のホストアイデンティティプロトコル（ｈｉｐ）接続を確立するためのアドレス指定方法及び装置
US8938553B2 (en) Cooperative proxy auto-discovery and connection interception through network address translation
US11882199B2 (en) Virtual private network (VPN) whose traffic is intelligently routed
US20070297430A1 (en) Terminal reachability
US9002923B2 (en) Transparent web proxy
US11888818B2 (en) Multi-access interface for internet protocol security
JP2007516625A (ja) パーソナルリモートファイヤウォール
US11329916B2 (en) Device information method and apparatus for directing link-layer communication
JP2018514956A (ja) データをルーティングするために証明書データを使用する装置と方法
US11575577B2 (en) User information method and apparatus for directing link-layer communication
US20030131258A1 (en) Peer-to-peer communication across firewall using internal contact point
Henderson et al. The Host Identity Protocol (HIP) Experiment Report
US10361997B2 (en) Auto discovery between proxies in an IPv6 network
US20130290699A1 (en) Methods for secure communication between network device services and devices thereof
Huawei Technologies Co., Ltd. TCP/IP
Feinstein et al. Internet− Draft IAP March 2001

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21919058

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21919058

Country of ref document: EP

Kind code of ref document: A1