US20200403914A1 - Packet Sending Method, Network Device, And Computer-Readable Storage Medium - Google Patents

Packet Sending Method, Network Device, And Computer-Readable Storage Medium Download PDF

Info

Publication number
US20200403914A1
US20200403914A1 US17/015,503 US202017015503A US2020403914A1 US 20200403914 A1 US20200403914 A1 US 20200403914A1 US 202017015503 A US202017015503 A US 202017015503A US 2020403914 A1 US2020403914 A1 US 2020403914A1
Authority
US
United States
Prior art keywords
packet
address
service node
target service
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US17/015,503
Other versions
US11546259B2 (en
Inventor
Bingyang LIU
Chenggui XIA
Guangpeng LI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Assigned to HUAWEI TECHNOLOGIES CO., LTD reassignment HUAWEI TECHNOLOGIES CO., LTD ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LI, Guangpeng, LIU, Bingyang, XIA, Chenggui
Publication of US20200403914A1 publication Critical patent/US20200403914A1/en
Application granted granted Critical
Publication of US11546259B2 publication Critical patent/US11546259B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163In-band adaptation of TCP data exchange; In-band control procedures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/74Address processing for routing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/02Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
    • H04L63/0227Filtering policies
    • H04L63/0236Filtering by address, protocol, port number or service, e.g. IP-address or URL
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/141Setup of application sessions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/146Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding

Definitions

  • This application relates to the computer field, and more specifically, to a packet sending method, a network device, and a computer-readable storage medium.
  • a client Before accessing a server (such as a data center), a client first needs to establish a connection to the server.
  • a transport layer connection is bound to a 5-tuple. Once the connection between the client and the server is successfully established, a 5-tuple used in a connection establishment process is not allowed to be changed; otherwise, the connection is disconnected.
  • the client usually initiates a connection establishment request by using an internet protocol (internet protocol, IP) address of the server (which may be, for example, an IP address of a front-end server of the server) as a destination IP address.
  • IP internet protocol
  • the server usually includes at least one service node, and a target service node configured to serve the client is one of the at least one network node (such as a topologically nearest node of the client).
  • the IP address of the server is used as the destination address for all packets sent by the client, and the packets usually cannot be routed to the target service node based only on the IP address of the server. Therefore, in the conventional technology, the server needs to record a large amount of binding relationship information (which may be used to indicate the target service node corresponding to the client), so as to correctly route the packets sent by the client to the target service node.
  • Continuous scale expansion of the server is accompanied with an increasing quantity of service nodes included in the server, and consequently the server needs to record an increasing amount of binding relationship information, causing high network overheads by recording a large amount of binding relationship information.
  • a packet sending method includes: receiving, by a target service node of a server, a first packet sent by a client, where a destination address of the first packet is an IP address of the server, the server includes at least one service node, the target service node is one of the at least one service node, and the IP address of the server is different from an IP address of the target service node; sending, by the target service node, a second packet to the client, where the second packet includes the iP address of the target service node; and receiving, by the target service node, a third packet sent by the client, where a destination address of the third packet is the IP address of the target service node.
  • the target service node in a process of establishing a connection between the client and the server, records the IP address of the node in the second packet.
  • the client changes the destination address from the IP address of the server to the IP address of the target service node to send the third packet.
  • the server may directly forward the third packet to the target service node with no need to record a correspondence between the client and the target service node, thereby reducing network overheads.
  • a destination port number of the first packet is a port number of the server
  • a destination port number of the second packet is a port number of the client
  • the second packet further includes a port number of the target service node
  • a destination port number of the third packet is the port number of the target service node.
  • the target service node when sending the second packet, may record the port number of the node in the packet.
  • the client After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • the second packet is a QUIC protocol-based packet.
  • the first packet is a QUIC initial packet
  • the second packet is a QUIC handshake packet or a QUIC retry packet.
  • a source address of the first packet is an IP address of the client
  • a source address of the second packet is the IP address of the server
  • a destination address of the second packet is the IP address of the client.
  • a connection established between the client and the target service node is a first connection
  • the second packet includes a connection identifier
  • the third packet includes the connection identifier
  • the method further includes: determining, by the target service node based on 5-tuple information and/or the connection identifier in the third packet, that the third packet belongs to the first connection.
  • the method further includes: sending, by the target service node, a fourth packet to the client, where a source address of the fourth packet is the IP address of the target service node, and a destination address of the fourth packet is the IP address of the client.
  • a packet sending method includes: sending, by a client, a first packet, where a destination address of the first packet is an IP address of a server; receiving, by the client, a second packet sent by a target service node, where the second packet includes an IP address of the target service node, the server includes at least one service node, the target service node is one of the at least one service node, and the IP address of the target service node is different from the IP address of the server; and sending, by the client, a third packet to the target service node, where a destination address of the third packet is the IP address of the target service node.
  • the target service node in a process of establishing a connection between the client and the server, records the IP address of the node in the second packet.
  • the client changes the destination address from the IP address of the server to the IP address of the target service node to send the third packet.
  • the server may directly forward the third packet to the target service node with no need to record a correspondence between the client and the target service node, thereby reducing network overheads.
  • a destination port number of the first packet is a port number of the server
  • a destination port number of the second packet is a port number of the client
  • the first packet further includes a port number of the target service node
  • a destination port number of the third packet is the port number of the target service node.
  • the target service node when sending the second packet, may record the port number of the node in the packet.
  • the client After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • the second packet is a QUIC protocol-based packet.
  • the first packet is a QUIC initial packet
  • the second packet is a QUIC handshake packet or a QUIC retry packet.
  • a source address of the first packet is an IP address of the client
  • a source address of the second packet is the IP address of the server
  • a destination address of the second packet is the IP address of the client.
  • a connection established between the client and the target service node is a first connection
  • the method further includes: determining, by the client based on 5-tuple information in the second packet, that the second packet belongs to the first connection.
  • the third packet includes a connection identifier
  • the method further includes: receiving, by the client, a fourth packet sent by the target service node, where the fourth packet includes the connection identifier; and determining, by the client based on 5-tuple information and/or the connection identifier in the fourth packet, that the fourth packet belongs to the first connection.
  • a network device includes a module configured to perform the method in any one of the first aspect or the implementations of the first aspect.
  • a network device includes a module configured to perform the method in any one of the second aspect or the implementations of the second aspect.
  • a network device includes a processor and a transceiver for performing the method in any one of the first aspect or the implementations of the first aspect.
  • a network device includes a processor and a transceiver for performing the method in any one of the second aspect or the implementations of the second aspect.
  • a computer-readable storage medium is provided.
  • the computer-readable storage medium is configured to store a computer software instruction, and the computer-readable storage medium includes a program designed for performing the method in any one of the first aspect or the implementations of the first aspect.
  • a computer-readable storage medium is provided.
  • the computer-readable storage medium is configured to store a computer software instruction, and the computer-readable storage medium includes a program designed for performing the method in the second aspect or any implementation of the second aspect.
  • a computer program product includes computer program code, and when the computer program code is run on a computer, the computer is enabled to perform the method in any one of the first aspect or the implementations of the first aspect.
  • a computer program product includes computer program code, and when the computer program code is run on a computer, the computer is enabled to perform the method in any one of the second aspect or the implementations of the second aspect.
  • FIG. 1 is a schematic diagram of a conventional mode of communication between a client and a service node
  • FIG. 2 is a schematic flowchart of a packet sending method according to an embodiment of this application
  • FIG. 3 is an example diagram of a first packet according to an embodiment of this application.
  • FIG. 4 is an example diagram of a second packet according to an embodiment of this application.
  • FIG. 5 is an example diagram of a third packet according to an embodiment of this application:
  • FIG. 6 is a schematic block diagram of a network device according to an embodiment of this application.
  • FIG. 7 is a schematic block diagram of another network device according to an embodiment of this application.
  • FIG. 8 is a schematic block diagram of another network device according to an embodiment of this application.
  • FIG. 9 is a schematic block diagram of another network device according to an embodiment of this application.
  • Technical solutions provided in the embodiments of this application may be applied to a scenario in which a client communicates with a server by establishing a transport layer connection.
  • the technical solutions may be applied to a scenario in which a client performs data communication with a data center, or may be applied to a content delivery network (content delivery network, CDN).
  • CDN content delivery network
  • the client mentioned in the embodiments of this application may be located on a terminal device, and the client may be, for example, an application (application, APP) on the terminal device.
  • the terminal device may be user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile console, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communications device, a user agent, or a user apparatus.
  • the terminal device may alternatively be a cellular phone, a cordless telephone set, a session initiation protocol (session initiation protocol, SIP) phone, a wireless local loop (wireless local loop, WLL) station, a personal digital assistant (personal digital assistant, PDA), a handheld device having a wireless communication function, a computing device or another processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a future 5G network, a terminal device in a future evolved public land mobile network (public land mobile network, PLMN), or the like.
  • SIP session initiation protocol
  • WLL wireless local loop
  • PDA personal digital assistant
  • PDA personal digital assistant
  • the service node may be an entity on the server that can provide one or more services.
  • the service node may also be referred to as a service instance on some occasions.
  • the service node may be, for example, a physical server, a virtual machine, or even a load balancer on the server that has an application layer protocol.
  • a specific form of the service node is not limited in the embodiments of this application.
  • the following uses a data center as an example to describe a conventional mode of communication between a client and a service node.
  • the client When the client needs to obtain a specific service from a server, the client sends a first packet by using an IP address of the server as a destination address.
  • the first packet may be, for example, a connection request packet.
  • the IP address of the server may be, for example, an IP address of a front-end server of the server. As shown in FIG. 1 , the server is the data center, and an IP address of the data center is 1.1.1.1.
  • the client may send the first packet to the data center by using 1.1.1.1 as the destination IP address.
  • the data center may select a service node 1 from service nodes 1 to 6 as a target service node of the client based on a load status of a service node in the data center or a topological distance between a service node and the client.
  • a 5-tuple including an IP address of the client, a port number of the client, the IP address of the server, a port number of the server, and a transport layer protocol type used in a connection establishment process is bound to the connection between the client and the server, and is not allowed to be changed (once the 5-tuple is changed, the connection is disconnected).
  • the server needs to record a binding relationship between the client and the service node 1 .
  • high network overheads are caused by recording the binding relationship between the client and the service node.
  • the method may include step 210 to step 230 .
  • Step 210 A client sends a first packet.
  • a source address of the first packet is an IP address of the client, and a destination IP address of the first packet is an IP address of a server.
  • the IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server.
  • the client may find the IP address of the server by using, for example, a domain name system (domain name system, DNS).
  • domain name system domain name system
  • the first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the client to the server.
  • QUIC is used as an example.
  • the first packet may be a QUIC initial packet (QUIC initial packet).
  • the first packet is received by a target service node of the server.
  • a manner of determining the target service node is not specifically limited in this embodiment of this application.
  • the server may select a lightly-loaded service node as the target service node based on a load status of a service node.
  • the server may select a topologically nearest node of the client as the target service node based on a topological relationship between the client and a service node.
  • FIG. 3 is an example diagram of a first packet according to an embodiment of this application.
  • QUIC is used as an example.
  • the client may record 5-tuple information such as a source address and a destination address in an IP header of the first packet.
  • Step 220 The target service node sends a second packet to the client.
  • a source address of the second packet is the IP address of the server, and a destination address is the IP address of the client.
  • the second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the client.
  • a QUIC protocol is used as an example.
  • the second packet may be a QUIC retry packet (QUIC retry packet) or a QUIC handshake packet (QUIC handshake packet).
  • the second packet may include an IP address of the target service node.
  • the IP address of the target service node is different from the IP address of the server.
  • the IP address of the target service node may uniquely identify a location of the target service node.
  • a packet between the client and the target service node may be forwarded by using an intermediate node (such as a front-end server of the server or a router), and a location of the intermediate node and a quantity of intermediate nodes are related to a specific network form.
  • an intermediate node such as a front-end server of the server or a router
  • FIG. 4 is an example diagram of the second packet according to this embodiment of this application.
  • a QUIC protocol is used as an example.
  • the target service node sets the IP address of the node in a QUIC header, and sets 5-tuple information such as a source address and a destination address in an IP header.
  • FIG. 4 is merely an example for description. This application is not limited thereto.
  • the target service node may alternatively set the IP address of the node at a load location of a QUIC packet.
  • Step 230 The client sends a third packet to the target service node by using the IP address of the target service node as a destination address.
  • the client may extract the IP address of the target service node from the second packet, and use the IP address of the target service node as the destination IP address of the subsequently sent third packet.
  • the third packet may be a QUIC protected packet (QUIC protected packet), a QUIC initial packet, or a QUIC handshake packet.
  • the QUIC protected packet may be a data packet.
  • FIG. 5 is an example diagram of the third packet according to this embodiment of this application.
  • QUIC is used as an example. After extracting the IP address of the target service node, the client sets the destination address of the third packet to the IP address of the target service node.
  • the destination address of the third packet is the IP address of the target service node, so that the server may directly forward the third packet to the target service node with no need to record a binding relationship between the server and the client by using a conventional technology, thereby reducing network overheads.
  • the third packet is any packet sent by the client after the second packet.
  • the second packet may further include a port number of the target service node.
  • the client may send the third packet to the target service node by using the port number of the target service node as a destination port number.
  • a destination port number of the first packet is a port number of the server
  • a destination port number of the second packet is a port number of the client
  • the second packet further includes the port number of the target service node
  • the destination port number of the third packet is the port number of the target service node.
  • the target service node when sending the second packet, may record the port number of the node in the packet.
  • the client After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • the source address and the destination address mentioned in this application each may be a public IP address or a private IP address.
  • the public IP address is an address obtained through translation by using an address translator.
  • the private IP address is an original address not translated by using an address translator.
  • a source port number and the destination port number mentioned in this application each may be a public network port number or a private network port number. This embodiment of this application imposes no specific limitation thereto.
  • the client after sending the first packet, the client locally records a connection related to the first packet as a first connection. Then the client determines that all packets related to the first packet are packets within the first connection.
  • the client after receiving the second packet sent by the target service node, the client identifies the second packet based on 5-tuple information in the second packet, and determines that the second packet belongs to the first connection.
  • a communication protocol on which a connection establishment manner shown in FIG. 2 is based is not specifically limited in this embodiment of this application.
  • the communication protocol may be a newly introduced communication protocol, or may be an improvement of connection establishment in a conventional protocol.
  • the following uses a QUIC protocol as an example for detailed description.
  • the QUIC protocol is a UDP-based low-latency internet transport layer protocol.
  • the QUIC protocol allows changing information in a 5-tuple, such as the IP addresses and the port numbers of the client and the server in the 5-tuple, after the connection is established and it is ensured that the connection does not break.
  • the conventional QUIC protocol does not allow changing the information in the 5-tuple in the connection establishment process. Consequently, in the connection establishment process and in an initial phase after connection establishment, a network side still needs to maintain a binding relationship between the client and the target service node, causing relatively high network overheads.
  • the connection establishment process for the QUIC protocol is improved, thereby reducing network overheads.
  • this embodiment of this application briefly describes a handshake communication process for the QUIC protocol.
  • Step 1 The client sends a first initial packet, where the first initial packet includes a connection identifier connection ID′ determined by the client.
  • the client locally records a connection related to the first initial packet as the first connection.
  • Step 2 After receiving the first initial packet, the target service node sends a retry packet or a first handshake packet to the client according to an actual situation.
  • the retry packet or the first handshake packet carries the IP address and/or the port number of the target service node.
  • the target service node may further add a connection identifier connection ID determined by the node to the retry packet or the first handshake packet.
  • a destination IP address and/or a port number of the first initial packet are/is different from the IP address and/or the port number of the target service node.
  • connection ID carried in the retry packet or the first handshake packet may be the same as or may be different from the connection ID′.
  • This embodiment of this application imposes no specific limitation thereto.
  • the target service node After sending the first handshake packet to the client, the target service node locally records a connection related to the first handshake packet as the first connection.
  • Step 3 After receiving the retry packet sent by the target service node, the client sends, by using the IP address of the target service node as a destination address and/or by using the port number of the target service node as a destination port number, a second initial packet to the target service node to re-establish a connection.
  • a connection identifier used by the second initial packet may be any connection identifier.
  • the connection identifier used by the second initial packet may be the connection ID or the connection ID′.
  • the connection identifier used by the second initial packet may be another connection identifier different from the connection ID and the connection ID′.
  • the client may determine, based on 5-tuple information in the retry packet, that the retry packet is a packet within the first connection.
  • the client After receiving the first handshake packet sent by the target service node, the client sends a protected packet or a second handshake packet to the target service node by using the IP address of the target service node as a destination address.
  • a connection identifier used by the second handshake packet may be the connection ID
  • a connection identifier used by the protected packet may be the connection ID′ or the connection ID.
  • Step 4 After receiving the second initial packet sent by the client, the target service node may send the retry packet or the handshake packet to the client according to an actual situation.
  • the target service node After receiving the protected packet or the handshake packet sent by the client, the target service node determines, based on 5-tuple information and/or the connection identifier in the protected packet or the handshake packet, that the protected packet or the handshake packet belongs to a packet within the first connection. In addition, the target service node sends a fourth packet to the client.
  • a source address of the fourth packet is the IP address of the target service node
  • a destination address is the IP address of the client
  • a connection identifier used by the fourth packet is the connection ID.
  • Step 5 After receiving the fourth packet, the client identifies the fourth packet based on 5-tuple information and/or the connection identifier in the fourth packet, and determines that the fourth packet belongs to a packet within the first connection.
  • FIG. 6 is a schematic block diagram of a network device according to an embodiment of this application.
  • the network device 600 includes a first receiving unit 610 , a first sending unit 620 , and a second receiving unit 630 .
  • the first receiving unit 610 is configured to receive a first packet sent by a client, where a source address of the first packet is an IP address of the client, and a destination IP address of the first packet is an IP address of a server.
  • the IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server.
  • the client may find the IP address of the server by using, for example, a DNS.
  • the first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the client to the server.
  • QUIC is used as an example.
  • the first packet may be a QUIC initial packet.
  • the first sending unit 620 is configured to send a second packet to the client, where the second packet includes an IP address of the network device.
  • the second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the client.
  • a QUIC protocol is used as an example.
  • the second packet may be a QUIC retry packet or a QUIC handshake packet.
  • the second receiving unit 630 is configured to receive a third packet sent by the client, where a destination address of the third packet is the IP address of the network device.
  • the client may extract the IP address of the network device from the second packet, and use the IP address of the network device as the destination IP address of the subsequently sent third packet.
  • the third packet may be a QUIC protected packet, a QUIC initial packet, or a QUIC handshake packet.
  • the QUIC protected packet may be a data packet.
  • the destination address of the third packet is the IP address of the network device, so that the server may directly forward the third packet to the network device with no need to record a binding relationship between the server and the client by using a conventional technology, thereby reducing network overheads.
  • the second packet may further include a port number of a target service node.
  • the client may send the third packet to the target service node by using the port number of the target service node as a destination port number.
  • a destination port number of the first packet is a port number of the server
  • a destination port number of the second packet is a port number of the client
  • the second packet further includes the port number of the target service node
  • the destination port number of the third packet is the port number of the target service node.
  • the network device when sending the second packet, may record the port number of the node in the packet.
  • the client After receiving the second packet, the client changes the destination port number of the third packet to the port number of the network device, so that the third packet may be directly routed to an accurate port location of the network device, thereby simplifying a processing procedure.
  • the second packet may be a QUIC protocol-based packet.
  • an embodiment of this application further provides a network device 700 .
  • the network device 700 includes a processor 710 , a memory 720 , and a transceiver 730 .
  • the memory 720 is configured to store an instruction
  • the processor 710 and the transceiver 730 are configured to execute the instruction stored in the memory 720 .
  • the network device 600 shown in FIG. 6 or the network device 700 shown in FIG. 7 may be configured to perform a related operation or procedure in the method embodiments, and operations and/or functions of the units in the network device 600 or the network device 700 are separately used to implement corresponding procedures in the method embodiments. For brevity, details are not described herein again.
  • FIG. 8 is a schematic block diagram of another network device according to an embodiment of this application.
  • the network device 800 includes a first receiving unit 810 , a first sending unit 820 , and a second receiving unit 830 .
  • the first sending unit 810 is configured to send a first packet, where a source address of the first packet is an IP address of a client, and a destination IP address of the first packet is an IP address of a server.
  • the IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server.
  • the client may find the IP address of the server by using, for example, a DNS.
  • the first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the network device to the server.
  • QUIC is used as an example.
  • the first packet may be a QUIC initial packet.
  • the first receiving unit 820 is configured to receive a second packet sent by a target service node, where the second packet includes an IP address of the target service node.
  • the second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the network device.
  • a QUIC protocol is used as an example.
  • the second packet may be a QUIC retry packet or a QUIC handshake packet.
  • the second sending unit 830 is configured to send a third packet to the target service node, where a destination address of the third packet is the IP address of the target service node.
  • the network device may extract the IP address of the target service node from the second packet, and use the IP address of the target service node as the destination IP address of the subsequently sent third packet.
  • the third packet may be a QUIC protected packet, a QUIC initial packet, or a QUIC handshake packet.
  • the QUIC protected packet may be a data packet.
  • the destination address of the third packet is the IP address of the target service node, so that the server may directly forward the third packet to the target service node with no need to record a binding relationship between the server and the network device by using a conventional technology, thereby reducing network overheads.
  • the second packet may further include a port number of the target service node.
  • the network device may send the third packet to the target service node by using the port number of the target service node as a destination port number.
  • a destination port number of the first packet is a port number of the server
  • a destination port number of the second packet is a port number of the client
  • the second packet further includes the port number of the target service node
  • the destination port number of the third packet is the port number of the target service node.
  • the target service node when sending second packet, may record the port number of the node in the packet.
  • the network device After receiving the second packet, the network device changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • the second packet may be a QUIC protocol based-packet.
  • an embodiment of this application further provides a network device 900 .
  • the network device 900 includes a processor 910 , a memory 920 , and a transceiver 930 .
  • the memory 920 is configured to store an instruction
  • the processor 910 and the transceiver 930 are configured to execute the instruction stored in the memory 920 .
  • the network device 800 shown in FIG. 8 or the network device 900 shown in FIG. 9 may be configured to perform a related operation or procedure in the method embodiments, and operations and/or functions of the units in the network device 800 or the network device 900 are separately used to implement corresponding procedures in the method embodiments. For brevity, details are not described herein again.
  • the processor mentioned in this embodiment of this application may be a central processing unit (central processing unit, CPU), or may be another general-purpose processor, a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like.
  • the general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.
  • the memory mentioned in this embodiment of this application may be a volatile memory or a nonvolatile memory, or may include a volatile memory and a nonvolatile memory.
  • the nonvolatile memory may be a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or a flash memory.
  • the volatile memory may be a random access memory (random access memory, RAM), and serves as an external cache.
  • RAMs may be used, for example, a static random access memory (static RAM, SRAM), a dynamic random access memory (dynamic RAM, DRAM), a synchronous dynamic random access memory (synchronous DRAM, SDRAM), a double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), an enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), a synchlink dynamic random access memory (synchlink DRAM, SLDRAM), and a direct rambus random access memory (direct rambus RAM, DR RAM).
  • static random access memory static random access memory
  • DRAM dynamic random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • double data rate SDRAM double data rate SDRAM
  • DDR SDRAM double data rate SDRAM
  • ESDRAM enhanced synchronous dynamic random access memory
  • synchlink dynamic random access memory synchlink dynamic random access memory
  • direct rambus RAM direct rambus RAM, DR RAM
  • the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component
  • the memory storage module
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the described apparatus embodiment is merely an example.
  • the unit division is merely logical function division and may be other division during actual implementation.
  • a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces.
  • the indirect couplings or communication connections between the apparatuses or units may be implemented in electrical, mechanical, or other forms.
  • the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual requirements to achieve the objectives of the solutions of the embodiments.
  • the functions When the functions are implemented in the form of a softvare functional unit and sold or used as an independent product, the functions may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of this application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product.
  • the computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application.
  • the foregoing storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (read-only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.
  • program code such as a USB flash drive, a removable hard disk, a read-only memory (read-only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.

Abstract

This application provides a packet sending method. The method includes: receiving, by a target service node, a first packet sent by a client, where a destination address of the first packet is an IP address of a server; sending, by the target service node, a second packet to the client, where the second packet includes an IP address of the target service node; and receiving, by the target service node, a third packet sent by the client, where a destination address of the third packet is the IP address of the target service node. The client changes a destination address from the IP address of the server to the IP address of the target service node to send the third packet, and the server directly forwards the third packet to the target service node with no need to record a correspondence between the client and the target service node.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application is a continuation of International Application No. PCT/CN2019/077243, filed on Mar. 7, 2019, which claims priority to Chinese Patent Application No. 201810193364.4, filed on Mar. 9, 2018. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties.
  • TECHNICAL FIELD
  • This application relates to the computer field, and more specifically, to a packet sending method, a network device, and a computer-readable storage medium.
  • BACKGROUND
  • Before accessing a server (such as a data center), a client first needs to establish a connection to the server. In a conventional technology, a transport layer connection is bound to a 5-tuple. Once the connection between the client and the server is successfully established, a 5-tuple used in a connection establishment process is not allowed to be changed; otherwise, the connection is disconnected.
  • In an actual communication process, the client usually initiates a connection establishment request by using an internet protocol (internet protocol, IP) address of the server (which may be, for example, an IP address of a front-end server of the server) as a destination IP address. Once the connection between the client and the server is successfully established, the IP address of the server that is used as the destination IP address of the client is bound to the connection, and is not allowed to be changed.
  • The server usually includes at least one service node, and a target service node configured to serve the client is one of the at least one network node (such as a topologically nearest node of the client). The IP address of the server is used as the destination address for all packets sent by the client, and the packets usually cannot be routed to the target service node based only on the IP address of the server. Therefore, in the conventional technology, the server needs to record a large amount of binding relationship information (which may be used to indicate the target service node corresponding to the client), so as to correctly route the packets sent by the client to the target service node. Continuous scale expansion of the server is accompanied with an increasing quantity of service nodes included in the server, and consequently the server needs to record an increasing amount of binding relationship information, causing high network overheads by recording a large amount of binding relationship information.
  • SUMMARY
  • According to a first aspect, a packet sending method is provided. The method includes: receiving, by a target service node of a server, a first packet sent by a client, where a destination address of the first packet is an IP address of the server, the server includes at least one service node, the target service node is one of the at least one service node, and the IP address of the server is different from an IP address of the target service node; sending, by the target service node, a second packet to the client, where the second packet includes the iP address of the target service node; and receiving, by the target service node, a third packet sent by the client, where a destination address of the third packet is the IP address of the target service node.
  • According to the method provided in this application, in a process of establishing a connection between the client and the server, the target service node records the IP address of the node in the second packet. After receiving the second packet, the client changes the destination address from the IP address of the server to the IP address of the target service node to send the third packet. In this way, the server may directly forward the third packet to the target service node with no need to record a correspondence between the client and the target service node, thereby reducing network overheads.
  • In a possible implementation, a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the second packet further includes a port number of the target service node, and a destination port number of the third packet is the port number of the target service node.
  • According to the method provided in this application, when sending the second packet, the target service node may record the port number of the node in the packet. After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • In a possible implementation, the second packet is a QUIC protocol-based packet.
  • In a possible implementation, the first packet is a QUIC initial packet, and the second packet is a QUIC handshake packet or a QUIC retry packet.
  • In a possible implementation, a source address of the first packet is an IP address of the client, a source address of the second packet is the IP address of the server, and a destination address of the second packet is the IP address of the client.
  • In a possible implementation, a connection established between the client and the target service node is a first connection, the second packet includes a connection identifier, the third packet includes the connection identifier, and the method further includes: determining, by the target service node based on 5-tuple information and/or the connection identifier in the third packet, that the third packet belongs to the first connection.
  • In a possible implementation, the method further includes: sending, by the target service node, a fourth packet to the client, where a source address of the fourth packet is the IP address of the target service node, and a destination address of the fourth packet is the IP address of the client.
  • According to a second aspect, a packet sending method is provided. The method includes: sending, by a client, a first packet, where a destination address of the first packet is an IP address of a server; receiving, by the client, a second packet sent by a target service node, where the second packet includes an IP address of the target service node, the server includes at least one service node, the target service node is one of the at least one service node, and the IP address of the target service node is different from the IP address of the server; and sending, by the client, a third packet to the target service node, where a destination address of the third packet is the IP address of the target service node.
  • According to the method provided in this application, in a process of establishing a connection between the client and the server, the target service node records the IP address of the node in the second packet. After receiving the second packet, the client changes the destination address from the IP address of the server to the IP address of the target service node to send the third packet. In this way, the server may directly forward the third packet to the target service node with no need to record a correspondence between the client and the target service node, thereby reducing network overheads.
  • In a possible implementation, a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the first packet further includes a port number of the target service node, and a destination port number of the third packet is the port number of the target service node.
  • According to the method provided in this application, when sending the second packet, the target service node may record the port number of the node in the packet. After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • In a possible implementation, the second packet is a QUIC protocol-based packet.
  • In a possible implementation, the first packet is a QUIC initial packet, and the second packet is a QUIC handshake packet or a QUIC retry packet.
  • In a possible implementation, a source address of the first packet is an IP address of the client, a source address of the second packet is the IP address of the server, and a destination address of the second packet is the IP address of the client.
  • In a possible implementation, a connection established between the client and the target service node is a first connection, and the method further includes: determining, by the client based on 5-tuple information in the second packet, that the second packet belongs to the first connection.
  • In a possible implementation, the third packet includes a connection identifier, and the method further includes: receiving, by the client, a fourth packet sent by the target service node, where the fourth packet includes the connection identifier; and determining, by the client based on 5-tuple information and/or the connection identifier in the fourth packet, that the fourth packet belongs to the first connection.
  • According to a third aspect, a network device is provided, and includes a module configured to perform the method in any one of the first aspect or the implementations of the first aspect.
  • According to a fourth aspect, a network device is provided, and includes a module configured to perform the method in any one of the second aspect or the implementations of the second aspect.
  • According to a fifth aspect, a network device is provided, and the network device includes a processor and a transceiver for performing the method in any one of the first aspect or the implementations of the first aspect.
  • According to a sixth aspect, a network device is provided, and the network device includes a processor and a transceiver for performing the method in any one of the second aspect or the implementations of the second aspect.
  • According to a seventh aspect, a computer-readable storage medium is provided. The computer-readable storage medium is configured to store a computer software instruction, and the computer-readable storage medium includes a program designed for performing the method in any one of the first aspect or the implementations of the first aspect.
  • According to an eighth aspect, a computer-readable storage medium is provided. The computer-readable storage medium is configured to store a computer software instruction, and the computer-readable storage medium includes a program designed for performing the method in the second aspect or any implementation of the second aspect.
  • According to a ninth aspect, a computer program product is provided. The computer program product includes computer program code, and when the computer program code is run on a computer, the computer is enabled to perform the method in any one of the first aspect or the implementations of the first aspect.
  • According to a tenth aspect, a computer program product is provided. The computer program product includes computer program code, and when the computer program code is run on a computer, the computer is enabled to perform the method in any one of the second aspect or the implementations of the second aspect.
  • BRIEF DESCRIPTION OF DRAWINGS
  • FIG. 1 is a schematic diagram of a conventional mode of communication between a client and a service node;
  • FIG. 2 is a schematic flowchart of a packet sending method according to an embodiment of this application;
  • FIG. 3 is an example diagram of a first packet according to an embodiment of this application;
  • FIG. 4 is an example diagram of a second packet according to an embodiment of this application;
  • FIG. 5 is an example diagram of a third packet according to an embodiment of this application:
  • FIG. 6 is a schematic block diagram of a network device according to an embodiment of this application;
  • FIG. 7 is a schematic block diagram of another network device according to an embodiment of this application;
  • FIG. 8 is a schematic block diagram of another network device according to an embodiment of this application; and
  • FIG. 9 is a schematic block diagram of another network device according to an embodiment of this application.
  • DESCRIPTION OF EMBODIMENTS
  • Technical solutions provided in the embodiments of this application may be applied to a scenario in which a client communicates with a server by establishing a transport layer connection. For example, the technical solutions may be applied to a scenario in which a client performs data communication with a data center, or may be applied to a content delivery network (content delivery network, CDN).
  • The client mentioned in the embodiments of this application may be located on a terminal device, and the client may be, for example, an application (application, APP) on the terminal device. The terminal device may be user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile console, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communications device, a user agent, or a user apparatus. The terminal device may alternatively be a cellular phone, a cordless telephone set, a session initiation protocol (session initiation protocol, SIP) phone, a wireless local loop (wireless local loop, WLL) station, a personal digital assistant (personal digital assistant, PDA), a handheld device having a wireless communication function, a computing device or another processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a future 5G network, a terminal device in a future evolved public land mobile network (public land mobile network, PLMN), or the like. The embodiments of this application impose no limitation thereto.
  • Currently, on the Internet, as a quantity of clients continuously increases, a quantity of clients accessing a service also continuously increases. To meet access requirements of a large quantity of clients, at least one service node usually needs to be set on a server. Generally, at least two or at least three service nodes needs to be set on the server. The service node may be an entity on the server that can provide one or more services. The service node may also be referred to as a service instance on some occasions. The service node may be, for example, a physical server, a virtual machine, or even a load balancer on the server that has an application layer protocol. A specific form of the service node is not limited in the embodiments of this application.
  • With reference to FIG. 1, the following uses a data center as an example to describe a conventional mode of communication between a client and a service node.
  • When the client needs to obtain a specific service from a server, the client sends a first packet by using an IP address of the server as a destination address. The first packet may be, for example, a connection request packet. The IP address of the server may be, for example, an IP address of a front-end server of the server. As shown in FIG. 1, the server is the data center, and an IP address of the data center is 1.1.1.1. The client may send the first packet to the data center by using 1.1.1.1 as the destination IP address. After receiving the connection establishment request, the data center may select a service node 1 from service nodes 1 to 6 as a target service node of the client based on a load status of a service node in the data center or a topological distance between a service node and the client. After a connection is successfully established, a 5-tuple including an IP address of the client, a port number of the client, the IP address of the server, a port number of the server, and a transport layer protocol type used in a connection establishment process is bound to the connection between the client and the server, and is not allowed to be changed (once the 5-tuple is changed, the connection is disconnected). To ensure that a subsequent packet sent by the client through the connection can be correctly routed to the service node 1, the server needs to record a binding relationship between the client and the service node 1. As quantities of clients and service nodes increase, high network overheads are caused by recording the binding relationship between the client and the service node.
  • As communications technologies develop, some new protocol types emerge, such as a quick user datagram protocol internet connections (quick UDP internet connection, QUIC) protocol. These new protocols allow changing the 5-tuple after a connection is established and it is ensured that the connection does not break. In the embodiments of this application, flexibility of these protocols is used to improve a communication mode in the connection establishment process, thereby reducing network overheads required for in the conventional mode of communication between the client and the server shown in FIG. 1.
  • With reference to FIG. 2, the following describes in detail a packet sending method according to an embodiment of this application. As shown in FIG. 2, the method may include step 210 to step 230.
  • Step 210: A client sends a first packet. A source address of the first packet is an IP address of the client, and a destination IP address of the first packet is an IP address of a server. The IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server. The client may find the IP address of the server by using, for example, a domain name system (domain name system, DNS).
  • The first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the client to the server. QUIC is used as an example. The first packet may be a QUIC initial packet (QUIC initial packet).
  • The first packet is received by a target service node of the server. A manner of determining the target service node is not specifically limited in this embodiment of this application. For example, the server may select a lightly-loaded service node as the target service node based on a load status of a service node. For another example, the server may select a topologically nearest node of the client as the target service node based on a topological relationship between the client and a service node.
  • FIG. 3 is an example diagram of a first packet according to an embodiment of this application. QUIC is used as an example. The client may record 5-tuple information such as a source address and a destination address in an IP header of the first packet.
  • Step 220: The target service node sends a second packet to the client. A source address of the second packet is the IP address of the server, and a destination address is the IP address of the client.
  • The second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the client. A QUIC protocol is used as an example. The second packet may be a QUIC retry packet (QUIC retry packet) or a QUIC handshake packet (QUIC handshake packet).
  • The second packet may include an IP address of the target service node. The IP address of the target service node is different from the IP address of the server. The IP address of the target service node may uniquely identify a location of the target service node.
  • It should be understood that a packet between the client and the target service node may be forwarded by using an intermediate node (such as a front-end server of the server or a router), and a location of the intermediate node and a quantity of intermediate nodes are related to a specific network form. This embodiment of this application imposes no limitation thereto.
  • FIG. 4 is an example diagram of the second packet according to this embodiment of this application. A QUIC protocol is used as an example. The target service node sets the IP address of the node in a QUIC header, and sets 5-tuple information such as a source address and a destination address in an IP header.
  • FIG. 4 is merely an example for description. This application is not limited thereto. For example, the target service node may alternatively set the IP address of the node at a load location of a QUIC packet.
  • Step 230: The client sends a third packet to the target service node by using the IP address of the target service node as a destination address.
  • After receiving the second packet, the client may extract the IP address of the target service node from the second packet, and use the IP address of the target service node as the destination IP address of the subsequently sent third packet.
  • In this embodiment of this application, the third packet may be a QUIC protected packet (QUIC protected packet), a QUIC initial packet, or a QUIC handshake packet. The QUIC protected packet may be a data packet.
  • FIG. 5 is an example diagram of the third packet according to this embodiment of this application. QUIC is used as an example. After extracting the IP address of the target service node, the client sets the destination address of the third packet to the IP address of the target service node.
  • The destination address of the third packet is the IP address of the target service node, so that the server may directly forward the third packet to the target service node with no need to record a binding relationship between the server and the client by using a conventional technology, thereby reducing network overheads.
  • It should be noted that, in this embodiment of this application, for ease of description, one or more packets sent by the client after the second packet are collectively referred to as the third packet. In other words, it can be understood that the third packet is any packet sent by the client after the second packet.
  • Optionally, in some embodiments, the second packet may further include a port number of the target service node. The client may send the third packet to the target service node by using the port number of the target service node as a destination port number. Specifically, a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the second packet further includes the port number of the target service node, and the destination port number of the third packet is the port number of the target service node.
  • According to the method provided in this embodiment of this application, when sending the second packet, the target service node may record the port number of the node in the packet. After receiving the second packet, the client changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • The source address and the destination address mentioned in this application each may be a public IP address or a private IP address. This embodiment of this application imposes no specific limitation thereto. The public IP address is an address obtained through translation by using an address translator. The private IP address is an original address not translated by using an address translator.
  • In addition, a source port number and the destination port number mentioned in this application each may be a public network port number or a private network port number. This embodiment of this application imposes no specific limitation thereto.
  • Optionally, for example, after sending the first packet, the client locally records a connection related to the first packet as a first connection. Then the client determines that all packets related to the first packet are packets within the first connection.
  • Specifically, after receiving the second packet sent by the target service node, the client identifies the second packet based on 5-tuple information in the second packet, and determines that the second packet belongs to the first connection.
  • A communication protocol on which a connection establishment manner shown in FIG. 2 is based is not specifically limited in this embodiment of this application. The communication protocol may be a newly introduced communication protocol, or may be an improvement of connection establishment in a conventional protocol. The following uses a QUIC protocol as an example for detailed description.
  • The QUIC protocol is a UDP-based low-latency internet transport layer protocol. The QUIC protocol allows changing information in a 5-tuple, such as the IP addresses and the port numbers of the client and the server in the 5-tuple, after the connection is established and it is ensured that the connection does not break. However, the conventional QUIC protocol does not allow changing the information in the 5-tuple in the connection establishment process. Consequently, in the connection establishment process and in an initial phase after connection establishment, a network side still needs to maintain a binding relationship between the client and the target service node, causing relatively high network overheads. In this embodiment of this application, the connection establishment process for the QUIC protocol is improved, thereby reducing network overheads.
  • For ease of understanding, this embodiment of this application briefly describes a handshake communication process for the QUIC protocol.
  • Step 1: The client sends a first initial packet, where the first initial packet includes a connection identifier connection ID′ determined by the client. In addition, the client locally records a connection related to the first initial packet as the first connection.
  • Step 2: After receiving the first initial packet, the target service node sends a retry packet or a first handshake packet to the client according to an actual situation.
  • It should be noted that the retry packet or the first handshake packet carries the IP address and/or the port number of the target service node. In addition, the target service node may further add a connection identifier connection ID determined by the node to the retry packet or the first handshake packet. A destination IP address and/or a port number of the first initial packet are/is different from the IP address and/or the port number of the target service node.
  • It should be understood that the connection ID carried in the retry packet or the first handshake packet may be the same as or may be different from the connection ID′. This embodiment of this application imposes no specific limitation thereto.
  • After sending the first handshake packet to the client, the target service node locally records a connection related to the first handshake packet as the first connection.
  • Step 3: After receiving the retry packet sent by the target service node, the client sends, by using the IP address of the target service node as a destination address and/or by using the port number of the target service node as a destination port number, a second initial packet to the target service node to re-establish a connection. A connection identifier used by the second initial packet may be any connection identifier. For example, the connection identifier used by the second initial packet may be the connection ID or the connection ID′. For another example, the connection identifier used by the second initial packet may be another connection identifier different from the connection ID and the connection ID′.
  • Optionally, the client may determine, based on 5-tuple information in the retry packet, that the retry packet is a packet within the first connection.
  • After receiving the first handshake packet sent by the target service node, the client sends a protected packet or a second handshake packet to the target service node by using the IP address of the target service node as a destination address. A connection identifier used by the second handshake packet may be the connection ID, and a connection identifier used by the protected packet may be the connection ID′ or the connection ID.
  • Step 4: After receiving the second initial packet sent by the client, the target service node may send the retry packet or the handshake packet to the client according to an actual situation.
  • After receiving the protected packet or the handshake packet sent by the client, the target service node determines, based on 5-tuple information and/or the connection identifier in the protected packet or the handshake packet, that the protected packet or the handshake packet belongs to a packet within the first connection. In addition, the target service node sends a fourth packet to the client. A source address of the fourth packet is the IP address of the target service node, a destination address is the IP address of the client, and a connection identifier used by the fourth packet is the connection ID.
  • Step 5: After receiving the fourth packet, the client identifies the fourth packet based on 5-tuple information and/or the connection identifier in the fourth packet, and determines that the fourth packet belongs to a packet within the first connection.
  • The foregoing describes in detail the method embodiment of this application with reference to FIG. 1 to FIG. 5. The following describes in detail apparatus embodiments of this application with reference to FIG. 6 to FIG. 9. It should be understood that descriptions of the method embodiment correspond to descriptions of the apparatus embodiments. Therefore, for parts that are not described in detail, refer to the method embodiment above.
  • FIG. 6 is a schematic block diagram of a network device according to an embodiment of this application. The network device 600 includes a first receiving unit 610, a first sending unit 620, and a second receiving unit 630.
  • The first receiving unit 610 is configured to receive a first packet sent by a client, where a source address of the first packet is an IP address of the client, and a destination IP address of the first packet is an IP address of a server. The IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server. The client may find the IP address of the server by using, for example, a DNS.
  • The first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the client to the server. QUIC is used as an example. The first packet may be a QUIC initial packet.
  • The first sending unit 620 is configured to send a second packet to the client, where the second packet includes an IP address of the network device.
  • The second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the client. A QUIC protocol is used as an example. The second packet may be a QUIC retry packet or a QUIC handshake packet.
  • The second receiving unit 630 is configured to receive a third packet sent by the client, where a destination address of the third packet is the IP address of the network device.
  • After receiving the second packet, the client may extract the IP address of the network device from the second packet, and use the IP address of the network device as the destination IP address of the subsequently sent third packet.
  • In this embodiment of this application, the third packet may be a QUIC protected packet, a QUIC initial packet, or a QUIC handshake packet. The QUIC protected packet may be a data packet.
  • The destination address of the third packet is the IP address of the network device, so that the server may directly forward the third packet to the network device with no need to record a binding relationship between the server and the client by using a conventional technology, thereby reducing network overheads.
  • Optionally, in some embodiments, the second packet may further include a port number of a target service node. The client may send the third packet to the target service node by using the port number of the target service node as a destination port number.
  • Specifically, a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the second packet further includes the port number of the target service node, and the destination port number of the third packet is the port number of the target service node.
  • According to the method provided in this embodiment of this application, when sending the second packet, the network device may record the port number of the node in the packet. After receiving the second packet, the client changes the destination port number of the third packet to the port number of the network device, so that the third packet may be directly routed to an accurate port location of the network device, thereby simplifying a processing procedure.
  • Optionally, in this embodiment of this application, the second packet may be a QUIC protocol-based packet.
  • As shown in FIG. 7, an embodiment of this application further provides a network device 700. The network device 700 includes a processor 710, a memory 720, and a transceiver 730. The memory 720 is configured to store an instruction, and the processor 710 and the transceiver 730 are configured to execute the instruction stored in the memory 720.
  • It should be understood that the network device 600 shown in FIG. 6 or the network device 700 shown in FIG. 7 may be configured to perform a related operation or procedure in the method embodiments, and operations and/or functions of the units in the network device 600 or the network device 700 are separately used to implement corresponding procedures in the method embodiments. For brevity, details are not described herein again.
  • FIG. 8 is a schematic block diagram of another network device according to an embodiment of this application. The network device 800 includes a first receiving unit 810, a first sending unit 820, and a second receiving unit 830.
  • The first sending unit 810 is configured to send a first packet, where a source address of the first packet is an IP address of a client, and a destination IP address of the first packet is an IP address of a server. The IP address of the server may be, for example, a front-end IP address of the server, or another IP address of the server that is open for external access to the server. The client may find the IP address of the server by using, for example, a DNS.
  • The first packet may be referred to as a connection request packet, and the first packet may be, for example, an initial packet sent by the network device to the server. QUIC is used as an example. The first packet may be a QUIC initial packet.
  • The first receiving unit 820 is configured to receive a second packet sent by a target service node, where the second packet includes an IP address of the target service node.
  • The second packet may be referred to as a connection reply packet, and the second packet may be, for example, an initial packet sent by the server to the network device. A QUIC protocol is used as an example. The second packet may be a QUIC retry packet or a QUIC handshake packet.
  • The second sending unit 830 is configured to send a third packet to the target service node, where a destination address of the third packet is the IP address of the target service node.
  • After receiving the second packet, the network device may extract the IP address of the target service node from the second packet, and use the IP address of the target service node as the destination IP address of the subsequently sent third packet.
  • In this embodiment of this application, the third packet may be a QUIC protected packet, a QUIC initial packet, or a QUIC handshake packet. The QUIC protected packet may be a data packet.
  • The destination address of the third packet is the IP address of the target service node, so that the server may directly forward the third packet to the target service node with no need to record a binding relationship between the server and the network device by using a conventional technology, thereby reducing network overheads.
  • Optionally, in some embodiments, the second packet may further include a port number of the target service node. The network device may send the third packet to the target service node by using the port number of the target service node as a destination port number.
  • Specifically, a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the second packet further includes the port number of the target service node, and the destination port number of the third packet is the port number of the target service node.
  • According to the method provided in this embodiment of this application, when sending second packet, the target service node may record the port number of the node in the packet. After receiving the second packet, the network device changes the destination port number of the third packet to the port number of the target service node, so that the third packet may be directly routed to an accurate port location of the target service node, thereby simplifying a processing procedure.
  • Optionally, in this embodiment of this application, the second packet may be a QUIC protocol based-packet.
  • As shown in FIG. 9, an embodiment of this application further provides a network device 900. The network device 900 includes a processor 910, a memory 920, and a transceiver 930. The memory 920 is configured to store an instruction, and the processor 910 and the transceiver 930 are configured to execute the instruction stored in the memory 920.
  • It should be understood that the network device 800 shown in FIG. 8 or the network device 900 shown in FIG. 9 may be configured to perform a related operation or procedure in the method embodiments, and operations and/or functions of the units in the network device 800 or the network device 900 are separately used to implement corresponding procedures in the method embodiments. For brevity, details are not described herein again.
  • It should be understood that the processor mentioned in this embodiment of this application may be a central processing unit (central processing unit, CPU), or may be another general-purpose processor, a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.
  • It should be further understood that the memory mentioned in this embodiment of this application may be a volatile memory or a nonvolatile memory, or may include a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (random access memory, RAM), and serves as an external cache. By way of example but not limitation, many forms of RAMs may be used, for example, a static random access memory (static RAM, SRAM), a dynamic random access memory (dynamic RAM, DRAM), a synchronous dynamic random access memory (synchronous DRAM, SDRAM), a double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), an enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), a synchlink dynamic random access memory (synchlink DRAM, SLDRAM), and a direct rambus random access memory (direct rambus RAM, DR RAM).
  • It should be noted that, when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, the memory (storage module) is integrated into the processor.
  • It should be noted that the memory described in this specification is intended to include but not limited to these memories and any memory of another proper type.
  • A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.
  • It can be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, reference may be made to a corresponding process in the foregoing method embodiments, and details are not described herein again.
  • In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the unit division is merely logical function division and may be other division during actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electrical, mechanical, or other forms.
  • The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual requirements to achieve the objectives of the solutions of the embodiments.
  • In addition, functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.
  • When the functions are implemented in the form of a softvare functional unit and sold or used as an independent product, the functions may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of this application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application. The foregoing storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (read-only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.
  • The foregoing descriptions are merely specific implementations of this application, but are not intended to limit the protection scope of this application. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in this application shall fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (20)

What is claimed is:
1. A packet sending method, comprising:
receiving, by a target service node of a server, a first packet sent by a client, wherein a destination address of the first packet is an internet protocol IP address of the server, the server comprises at least one service node, the target service node is one of the at least one service node, and the IP address of the server is different from an IP address of the target service node;
sending, by the target service node, a second packet to the client, wherein the second packet comprises the IP address of the target service node; and
receiving, by the target service node, a third packet sent by the client, wherein a destination address of the third packet is the IP address of the target service node.
2. The method according to claim 1, wherein a destination port number of the first packet is a port number of the server a destination port number of the second packet is a port number of the client, the second packet further comprises a port number of the target service node, and a destination port number of the third packet is the port number of the target service node.
3. The method according to claim 1, wherein the second packet is a quick user datagram protocol internet connections QUIC protocol-based packet.
4. The method according to claim 3, wherein the first packet is a QUIC initial packet, and the second packet is a QUIC handshake packet or a QUIC retry packet.
5. The method according to claim 1, wherein a source address of the first packet is an IP address of the client, a source address of the second packet is the IP address of the server, and a destination address of the second packet is the IP address of the client.
6. The method according to claim 1, wherein a connection established between the client and the target service node is a first connection, the second packet comprises a connection identifier, and the third packet comprises the connection identifier; and
the method further comprises:
determining, by the target service node based on 5-tuple information and/or the connection identifier in the third packet, that the third packet belongs to the first connection.
7. The method according to claim 1, wherein the method further comprises:
sending, by the target service node, a fourth packet to the client, wherein a source address of the fourth packet is the IP address of the target service node, and a destination address of the fourth packet is the IP address of the client.
8. A packet sending method, comprising:
sending, by a client, a first packet, wherein a destination address of the first packet is an internet protocol IP address of a server;
receiving, by the client, a second packet sent by a target service node of the server, wherein the second packet comprises an IP address of the target service node, the server comprises at least one service node, the target service node is one of the at least one service node, and the IP address of the target service node is different from the IP address of the server; and
sending, by the client, a third packet to the target service node, wherein a destination address of the third packet is the IP address of the target service node.
9. The method according to claim 8, wherein a destination port number of the first packet is a port number of the server, a destination port number of the second packet is a port number of the client, the second packet further comprises a port number of the target service node, and a destination port number of the third packet is the port number of the target service node.
10. The method according to claim 8, wherein the second packet is a quick user datagram protocol internet connections QUIC protocol-based packet.
11. The method according to claim 10, wherein the first packet is a QUIC initial packet, and the second packet is a QUIC handshake packet or a QUIC retry packet.
12. The method according to claim 8, wherein a source address of the first packet is an IP address of the client, a source address of the second packet is the IP address of the server, and a destination address of the second packet is the iP address of the client.
13. The method according to claim 8, wherein a connection established between the client and the target service node is a first connection, and the method further comprises:
determining, by the client based on 5-tuple information in the second packet, that the second packet belongs to the first connection.
14. The method according to claim 13, wherein the third packet comprises a connection identifier, and the method further comprises:
receiving, by the client, a fourth packet sent by the target service node, wherein the fourth packet comprises the connection identifier; and
determining, by the client based on 5-tuple information and/or the connection identifier in the fourth packet, that the fourth packet belongs to the first connection.
15. A network device, comprising:
a first sender, configured to send a first packet, wherein a destination address of the first packet is an internet protocol IP address of a server:
a first receiver, configured to receive a second packet sent by a target service node, wherein the second packet comprises an IP address of the target service node, the server comprises at least one service node, the target service node is one of the at least one service node, and the IP address of the target service node is different from the IP address of the server; and
a second sender, configured to send a third packet to the target service node, wherein a destination address of the third packet is the IP address of the target service node.
16. The network device according to claim 15, wherein the second packet further comprises a port number of the target service node, and a destination port number of the third packet is the port number of the target service node.
17. The network device according to claim 15, wherein the second packet is a quick user datagram protocol internet connections QUIC protocol-based packet.
18. The network device according to claim 15, wherein the first packet is a QUIC initial packet, and the second packet is a QUIC handshake packet or a QUIC retry packet.
19. The network device according to claim 15, wherein a source address of the first packet is an IP address of the network device, a source address of the second packet is the IP address of the server, and a destination address of the second packet is the IP address of the network device.
20. The network device according to claim 15, wherein a connection established between the client and the target service node is a first connection, and the network device further comprises:
a first determining unit, configured to determine, based on 5-tuple information in the second packet, that the second packet belongs to the first connection.
US17/015,503 2018-03-09 2020-09-09 Packet sending method, network device, and computer-readable storage medium Active US11546259B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201810193364.4 2018-03-09
CN201810193364.4A CN110247848B (en) 2018-03-09 2018-03-09 Method for sending message, network equipment and computer readable storage medium
PCT/CN2019/077243 WO2019170114A1 (en) 2018-03-09 2019-03-07 Method for sending packet, network device, and computer-readable storage medium

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/077243 Continuation WO2019170114A1 (en) 2018-03-09 2019-03-07 Method for sending packet, network device, and computer-readable storage medium

Publications (2)

Publication Number Publication Date
US20200403914A1 true US20200403914A1 (en) 2020-12-24
US11546259B2 US11546259B2 (en) 2023-01-03

Family

ID=67846925

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/015,503 Active US11546259B2 (en) 2018-03-09 2020-09-09 Packet sending method, network device, and computer-readable storage medium

Country Status (4)

Country Link
US (1) US11546259B2 (en)
EP (1) EP3758339A4 (en)
CN (2) CN113824642B (en)
WO (1) WO2019170114A1 (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113596184B (en) * 2020-04-30 2023-08-08 华为云计算技术有限公司 Hybrid cloud system, gatekeeper, network access method and storage medium
CN111901364A (en) * 2020-08-17 2020-11-06 扬州网桥软件技术有限公司 Firewall penetration method and system
CN112333169B (en) * 2020-10-28 2023-05-23 锐捷网络股份有限公司 Message processing method, device, electronic equipment and computer readable medium
CN112653740A (en) * 2020-12-11 2021-04-13 北京金山云网络技术有限公司 Load balancing method and device supporting QUIC connection migration and computer product
CN112838966A (en) * 2021-04-22 2021-05-25 北京拓课网络科技有限公司 UDP link monitoring method and system and electronic equipment
CN115801675A (en) * 2021-09-09 2023-03-14 华为技术有限公司 Message processing method and related device
CN114285805A (en) * 2021-12-28 2022-04-05 赛尔网络有限公司 QUIC message filtering method, system, equipment and medium
CN114500379A (en) * 2022-02-18 2022-05-13 吉旗(成都)科技有限公司 Message transmission method, device, equipment and storage medium
CN115361268A (en) * 2022-08-19 2022-11-18 湖北天融信网络安全技术有限公司 Redirection method, redirection device, electronic equipment and computer-readable storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018149684A1 (en) * 2017-02-20 2018-08-23 Nokia Solutions And Networks Oy A method of distributing load and a network apparatus
US10412159B1 (en) * 2014-02-07 2019-09-10 Amazon Technologies, Inc. Direct load balancing using a multipath protocol

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7161947B1 (en) * 2002-07-30 2007-01-09 Cisco Technology, Inc. Methods and apparatus for intercepting control and data connections
US8230067B2 (en) * 2003-10-31 2012-07-24 Ericsson Ab DHCP proxy in a subscriber environment
US8145908B1 (en) * 2004-10-29 2012-03-27 Akamai Technologies, Inc. Web content defacement protection system
CN101697633B (en) * 2009-11-10 2011-12-28 西安西电捷通无线网络通信股份有限公司 IP adaptation-based load balancing method and system thereof
CN102075445B (en) * 2011-02-28 2013-12-25 杭州华三通信技术有限公司 Load balancing method and device
US9026783B2 (en) 2013-03-07 2015-05-05 Google Inc. Low latency server-side redirection of UDP-based transport protocols traversing a client-side NAT firewall
US10298712B2 (en) * 2013-12-11 2019-05-21 Telefonaktiebolaget Lm Ericsson (Publ) Proxy interception
US9419889B2 (en) * 2014-03-07 2016-08-16 Nicira, Inc. Method and system for discovering a path of network traffic
US9985896B2 (en) * 2014-03-31 2018-05-29 Nicira, Inc. Caching of service decisions
CN106330715A (en) * 2015-06-30 2017-01-11 杭州华三通信技术有限公司 Message processing method and device
US9807016B1 (en) * 2015-09-29 2017-10-31 Juniper Networks, Inc. Reducing service disruption using multiple virtual IP addresses for a service load balancer
US9882972B2 (en) * 2015-10-30 2018-01-30 International Business Machines Corporation Packet forwarding optimization without an intervening load balancing node
CN107026890B (en) * 2016-02-02 2020-10-09 华为技术有限公司 Message generation method based on server cluster and load balancer
CN111866064B (en) * 2016-12-29 2021-12-28 华为技术有限公司 Load balancing method, device and system
US10305973B2 (en) * 2017-01-09 2019-05-28 International Business Machines Corporation Distributed load-balancing for software defined networks
CN106878193B (en) * 2017-02-10 2022-03-22 新华三技术有限公司 Load sharing method and device
US10298711B2 (en) * 2017-06-09 2019-05-21 Hewlett Packard Enterprise Development Lp Service function chains using correlation cookies
US10785020B2 (en) * 2018-01-19 2020-09-22 Microsoft Technology Licensing, Llc Hardware offload for QUIC connections

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10412159B1 (en) * 2014-02-07 2019-09-10 Amazon Technologies, Inc. Direct load balancing using a multipath protocol
WO2018149684A1 (en) * 2017-02-20 2018-08-23 Nokia Solutions And Networks Oy A method of distributing load and a network apparatus

Also Published As

Publication number Publication date
CN113824642A (en) 2021-12-21
WO2019170114A1 (en) 2019-09-12
CN110247848B (en) 2021-08-20
EP3758339A4 (en) 2021-04-21
CN110247848A (en) 2019-09-17
CN113824642B (en) 2023-02-03
US11546259B2 (en) 2023-01-03
EP3758339A1 (en) 2020-12-30

Similar Documents

Publication Publication Date Title
US11546259B2 (en) Packet sending method, network device, and computer-readable storage medium
TWI437863B (en) Peer-to-peer mobility management in heterogeneous ipv4 networks
US9155032B2 (en) System, method, and apparatus for determining a network interface preference policy
US11277313B2 (en) Data transmission method and corresponding device
JP4642777B2 (en) Providing location-specific services to mobile nodes
US8654735B2 (en) IP mobility multihoming
US20060034256A1 (en) System and method for service discovery during connection setup in a wireless environment
US20160344688A1 (en) Communicating via IPv6-only Networks Using IPv4 Literal Identifiers
US10051607B2 (en) Data processing method, apparatus and system
US11784963B2 (en) NAT traversal method, device, and system
US9648646B2 (en) Method and device for establishing packet data network connection
US20100278099A1 (en) Methods for Transmitting and Receiving Data and Communication Devices
US11419171B2 (en) Method for establishing subflow of multipath connection, apparatus, and system
US20110087789A1 (en) Subscription based network routing tables and enforcement for overlay networks
US11438942B2 (en) Method for establishing tunnel between local gateways, and gateway
CN105101176A (en) Session binding method, device and system in roaming scene
WO2021217869A1 (en) Method and system for transferring real ip address of client
US20230291706A1 (en) Method for accessing network, media gateway, electronic device and storage medium
CA3173928A1 (en) Communication method and apparatus
US20230216798A1 (en) Data Steering Method and Apparatus
WO2017166038A1 (en) Communication method and terminal
EP3073691B1 (en) Method and device for improving qos
WO2022218194A1 (en) Service routing method and device
EP3852412B1 (en) Roaming
CN107211010B (en) data transmission method and device

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED

AS Assignment

Owner name: HUAWEI TECHNOLOGIES CO., LTD, CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LIU, BINGYANG;XIA, CHENGGUI;LI, GUANGPENG;REEL/FRAME:054147/0989

Effective date: 20201022

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction