EP3725056B1 - Offloading communication security operations to a network interface controller - Google Patents
Offloading communication security operations to a network interface controller Download PDFInfo
- Publication number
- EP3725056B1 EP3725056B1 EP18830310.1A EP18830310A EP3725056B1 EP 3725056 B1 EP3725056 B1 EP 3725056B1 EP 18830310 A EP18830310 A EP 18830310A EP 3725056 B1 EP3725056 B1 EP 3725056B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- cryptographic security
- nic
- security protocol
- packet
- data packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 title description 5
- 238000000034 method Methods 0.000 claims description 20
- 238000005538 encapsulation Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 238000013467 fragmentation Methods 0.000 claims description 6
- 238000006062 fragmentation reaction Methods 0.000 claims description 6
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000008878 coupling Effects 0.000 claims 1
- 238000010168 coupling process Methods 0.000 claims 1
- 238000005859 coupling reaction Methods 0.000 claims 1
- 230000006870 function Effects 0.000 description 29
- 238000010586 diagram Methods 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 7
- 239000012634 fragment Substances 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000005641 tunneling Effects 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001152 differential interference contrast microscopy Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/52—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
- G06F21/53—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow by executing in a restricted environment, e.g. sandbox or secure virtual machine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
- H04L63/0485—Networking architectures for enhanced packet encryption processing, e.g. offloading of IPsec packet processing or efficient security association look-up
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/16—Implementing security features at a particular protocol layer
- H04L63/164—Implementing security features at a particular protocol layer at the network layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0894—Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
- H04L9/0897—Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage involving additional devices, e.g. trusted platform module [TPM], smartcard or USB
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/14—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using a plurality of keys or algorithms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3226—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using a predetermined code, e.g. password, passphrase or PIN
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45587—Isolation or security of virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/12—Details relating to cryptographic hardware or logic circuitry
Definitions
- the present invention relates generally to computer network communications, and particularly to apparatus and methods for performing security-related operations on data packets transmitted and received over a network.
- IPsec Internet Protocol Security
- IP Internet Protocol Security
- Layer 3 the Internet Layer 3 of the Internet Protocol (IP) suite, and can automatically secure applications and data transmitted in IP packets.
- IPsec uses cryptographic security services to support network-level peer authentication, data-origin authentication, data integrity, data confidentiality (encryption), and replay protection.
- IPsec also includes protocols for establishing mutual authentication between agents at the beginning of the session and negotiation of cryptographic keys for use during the session.
- the IPsec architecture and operational features are specified in a series of Requests for Comments (RFCs) published on line by the Internet Engineering Task Force (IETF), include RFC 4301, RFC 4303, and RFC 4106. Specific features of IPsec that are used in popular implementations include the following:
- U.S. Patent 8,006,297 describes a method and system for combined security protocol and packet filter offload and onload.
- This patent describes a NIC that includes a security association database (SADE) comprising a plurality of security associations (SAs), a cryptographic offload engine configured to decrypt a packet using one of the plurality of SAs, a security policy database (SPD) comprising a plurality of security policies (SPs) and a plurality of filter policies, and a policy engine configured to determine an admittance of the packet using one of the plurality of SPs from the SPD and apply one of the plurality of filter policies to the packet.
- SADE security association database
- SPD security policy database
- SPs security policies
- policy engine configured to determine an admittance of the packet using one of the plurality of SPs from the SPD and apply one of the plurality of filter policies to the packet.
- U.S. Patent Application Publication 2010/0228962 describes offloading cryptographic protection processing of packet data sent according to a security protocol between a first computer and a second computer via a forwarding device.
- the forwarding device performs a portion of the processing, and forwards the packet data to a third computer, connected to the forwarding device, for other processing.
- the third computer may support non-standard extensions to the security protocol, such as extensions used in authorizing and establishing a connection over the secure protocol.
- the third computer sends the results of its processing, such as a cryptographic key, or a detected access control policy, to the forwarding device.
- EP1657878A1 describes mechanisms for transferring processor control of secure Internet Protocol (IPSec) security association (SA) functions between a host and a target processing devices of a computerized system, such as processors in a host CPU and a NIC.
- IPSec Internet Protocol
- SA security association
- the computation associated with authentication and/or encryption is offloaded while the host maintains control of when SA functions are offloaded, uploaded, invalidated, and re-keyed.
- the devices coordinate to maintain metrics for the SA, including support for both soft and hard limits on SA expiration. Timer requirements are minimized for the target.
- the offloaded SA function may be embedded in other offloaded state objects of intermediate software layers of a network stack.
- US2013125125A1 describes a computer system comprising a virtual machine operating on a physical machine; and a management block operating on the physical machine and managing the virtual machine.
- the virtual machine has a specific function processing module that performs specific function processing with respect to a packet for transmission and a received packet.
- the management block has a virtual switch that relays a packet transmitted and received by the virtual machine.
- the virtual switch has an offload processing block that performs the specific function processing if the specific function processing is offloaded to the management block. If the specific function processing is offloaded from the virtual machine to the management block, the specific function processing module notifies the management block of processing information required for the specific function processing, and the offload processing block executes the specific function processing based on the processing information received from the virtual machine.
- Embodiments of the present invention that are described hereinbelow provide improved apparatus and methods for offload of security-related functions to hardware logic.
- computing apparatus including a host processor, which is configured to run a virtual machine monitor (VMM), which supports a plurality of virtual machines running on the host processor, and which includes a cryptographic security software module configured to apply a cryptographic security protocol to data packets transmitted and received by one or more of the virtual machines.
- VMM virtual machine monitor
- a network interface controller is configured to link the host processor to a network so as to transmit and receive the data packets from and to the virtual machines over the network, and includes a cryptographic security hardware logic module, which is configured, when invoked by the VMM, to apply the cryptographic security protocol to the data packets transmitted and received by the one or more of the virtual machines while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the virtual machines.
- the NIC is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given virtual machine, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given virtual machine, to the cryptographic security software module of the VMM, which processes the data packet using the state context and passes the data packet, after processing, to the given virtual machine.
- the VMM is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the virtual machines in invocation or implementation of the cryptographic security protocol.
- the VMM is configured, when the NIC has encountered the exception, to acquire the state context of the cryptographic security protocol with respect to the given virtual machine by performing a predefined handshake with the NIC.
- the predefined handshake includes, for example, querying and receiving packet sequence number information from the NIC and updating replay protection information used in the cryptographic security protocol.
- the exception includes a fragmentation of the data packet following application of the cryptographic security protocol by a sender of the data packet, and wherein the VMM is configured to defragment the data packet.
- the VMM is configured, after processing the data packet, to pass the processed data packet to the given virtual machine by looping the processed data packet through the NIC to the given virtual machine.
- the NIC is configured to apply an encapsulation, using the state context of the cryptographic security protocol, to the data packets transmitted from the given virtual machine to a specified destination, while maintaining a count of the data transmitted using the state context, and when the count reaches a predefined limit, to stop transmitting the data packets to the specified destination and transfer the state context to the cryptographic security software module of the VMM for update of the state context.
- the NIC is configured to apply a decapsulation, using the state context of the cryptographic security protocol, to encapsulated data packets received from the network, and upon receiving instructions from the VMM to terminate the decapsulation, to loop the received data packets back to the network.
- the cryptographic security protocol includes an IPsec protocol.
- computing apparatus including a network interface, configured to be connected to a network, and a host interface, configured to be connected to a peripheral component bus of a host computer.
- An embedded controller is configured to run a cryptographic security software module, which applies a cryptographic security protocol to data packets transmitted and received by applications running on the host computer.
- Packet processing hardware logic is coupled between the host interface and the network interface so as to transmit and receive data packets over the network from and to the applications running on the host computer, and includes a cryptographic security hardware logic module, which is configured, when invoked by the embedded controller, to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the applications.
- the packet processing hardware logic is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given application, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module of the embedded controller, which processes the data packet using the state context and passes the data packet, after processing, to the given application.
- the embedded controller is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the applications in invocation or implementation of the cryptographic security protocol.
- a method for computing which includes running on a programmable processor a cryptographic security software module configured to apply a cryptographic security protocol to data packets transmitted and received by applications running on a host computer.
- a network interface controller (NIC) is coupled between the host processor and a network so as to transmit and receive the data packets from and to the applications over the network.
- the cryptographic security software module invokes a cryptographic security hardware logic module in the NIC, thereby causing to the cryptographic security hardware logic module to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the applications.
- the data packet Upon encountering in the cryptographic security hardware logic module an exception in applying the cryptographic security protocol to a data packet directed to a given application, the data packet is transferred, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module running on the programmable processor.
- the data packet is processed on the programmable processor using the state context and the cryptographic security software module, which passes the data packet, after the processing, to the given application.
- IPsec In view of the heavy computational burden involved in cryptographic security protocols, such as IPsec, offloading the processing tasks to a NIC is a desirable solution.
- the core computational functions of protocols such as IPsec can be implemented efficiently in a cryptographic security hardware logic module, which can be designed in such a way that the necessary packet processing is performed without reducing the data throughput of the NIC.
- IP networks for example, packets to which an AH or ESP was applied at the sending node may be fragmented into smaller packets on the way to their destination. The receiving node must first defragment these packets before AH authentication and decapsulation. Implementing this sort of defragmentation in the NIC requires substantial amounts of memory and re-ordering logic, which increase chip size and processing latency.
- IPsec requires that a given SA context be used for no more than a certain number of packets or volume of data.
- the IPsec endpoints are supposed to maintain a count of data transmitted using the current SA, and then negotiate a new SA when the count reaches a predefined limit. In this case, too, implementing this sort of functionality in hardware logic in the NIC is impractical.
- Embodiments of the present invention address these problems by means of a novel collaboration between a cryptographic security hardware logic module in the NIC and a cryptographic security software module running on a processor in a privileged domain.
- the domain is "privileged" in that it has access to and is able to make changes in the state context of the cryptographic security protocol, wherein this context includes parameters that are used in encryption and authentication, such as packet sequence numbers, counters, and cryptographic keys.
- Examples of such privileged domains include a virtual machine monitor (VMM, also referred to as the hypervisor) in a host computer running virtual machines, or an embedded programmable controller in a smart NIC.
- VMM virtual machine monitor
- the hypervisor an embedded programmable controller in a smart NIC.
- the privileged software module is able to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the (non-privileged) user-domain applications that transmit and receive the packets via the NIC.
- the user-domain applications need not even be aware that packet encryption or authentication is being applied to the packets that they transmit and receive.
- the cryptographic security software module invokes the cryptographic security hardware logic module in the NIC.
- the hardware logic module will then apply the appropriate security operations to the data packets transmitted and received by these applications, while maintaining the state context of the cryptographic security protocol with respect to each of the applications in question.
- the security hardware logic module encounters an exception in applying the cryptographic security protocol to a data packet directed to a given application, such as a fragmented packet, it transfers the data packet, together with the applicable state context, to the cryptographic security software module.
- This software module processes the data packet, using the state context, and passes the processed data packet to the appropriate application.
- IPsec IP Security
- authentication and encapsulation The principles of the present invention, however, are by no means limited to IPsec and may alternatively be applied in implementing other cryptographic security protocols that are known in the art, particularly datagram-based packet encryption, authentication and encapsulation protocols. Examples of such protocols include Media Access Control Security (MACsec), as defined by IEEE standard 802.1AE, and Datagram Transport Layer Security (DTLS), specified in RFC 4347.
- MACsec Media Access Control Security
- DTLS Datagram Transport Layer Security
- Fig. 1 is a block diagram that schematically illustrates a networked computer system 20, in accordance with an embodiment of the present invention.
- System 20 comprises multiple host computers 22, 24, 26, ... (also referred to simply as "hosts"), which communicate over a packet data network 28.
- data network 28 is a Layer-3 network, such as an IP network, and thus comprises Layer-3 routers 36, as well as switches 34, through which hosts 22, 24, 26, ..., may connect to the network.
- the principles of the present invention are similarly applicable over other sorts of data networks, such as InfiniBand networks; and the methods and circuits described herein can be used to support various sorts of packet-level cryptographic security protocols, including both Layer-2 and Layer-3 protocols.
- Each host 22, 24, 26 in this example comprises a central processing unit (CPU) 30, which typically comprises one or more processing cores (not shown), with a system memory 31 and a network interface controller (NIC) 32.
- NIC 32 is connected by a bus 33 to CPU 30 and memory 31, and is connected via one of switches 34 to network 28.
- Bus 33 may comprise, for example, a peripheral component bus, such as PCI Express ® (PCIe ® ) bus, or a dedicated system bus of the CPU.
- a cryptographic security hardware logic module 44 in NIC 32 can be invoked to apply a cryptographic security protocol, such as IPsec, to outgoing and incoming data packets to and from network 28, as described further hereinbelow.
- Hosts 22, 24, 26 support a virtual machine environment, in which multiple virtual machines 38 (labeled VM1, VM2, VM3 in Fig. 1 ) may run on any given CPU 30.
- a virtual machine monitor (VMM) 40 in the CPU native domain interacts with the kernels of the guest operating systems of virtual machines 38 in a manner that emulates the host processor and allows the virtual machines to share the resources of the CPU.
- VMM virtual machine monitor
- NIC 32 comprises packet processing circuitry, which is configured to appear to the programs running on CPU 30 as multiple virtual NICs (vNICs) 42.
- vNICs virtual NICs
- SR-IOV single-root I/O virtualization
- each virtual machine 38 interacts with NIC 32 as though the NIC was dedicated to that virtual machine, linking the virtual machine to other machines (virtual and/or physical) on network 28.
- NIC 32 acts as a virtual switch, connecting each of the virtual machines to a particular tenant network while allowing vNICs 42 to share the same physical port to underlying data network 28.
- VM2 in host 22 and VM3 in host 24 reside on the same tenant network.
- VM2 submits the packet to VMM 40 in host 22; and the VMM adds an AH, encapsulates the packet and transmits it to the VMM in host 24, which then authenticates, decapsulates, and passes the packet to VM3.
- VM2 in host 22 communicates with VM3 in host 24 via an IPsec tunnel 46 between the respective NICs 32, without necessarily even being aware that IPsec authentication or encapsulation is taking place.
- Fig. 2 is a block diagram that schematically shows details of NIC 32 and software running on CPU 30, in accordance with an embodiment of the present invention.
- the software running on CPU 30, including both operating system and application programs, may be downloaded to the CPU in electronic form, over a network for example. Additionally or alternatively, the software may be stored on tangible, non-transitory computer-readable media, such as optical, magnetic or electronic memory media, which may be embodied in memory 31.
- CPU 30 operates a native domain 48, with a host operating system 50 and other privileged functions, including an IPsec software module 54.
- the CPU concurrently runs one or more virtual machines 38, as noted above, each with its own guest operating system 52 and guest user applications 56.
- VMM 40 in native domain 48 interacts with the kernels of guest operating systems 52 in a manner that emulates the host processor and allows the virtual machines to share the resources of CPU 30.
- a wide range of virtual machine software of this sort is available commercially, and further description is beyond the scope of the present disclosure.
- Client processes such as user applications 56 communicate with the transport layer of network 28 by manipulating a transport service instance, known as a "queue pair" (QP).
- QP transport service instance
- a client submits work items, called work queue elements (WQEs), to the appropriate queues for execution by the NIC.
- WQEs work queue elements
- NIC 32 appears to each virtual machine 38 to be a dedicated I/O device, or vNIC, for use by that virtual machine in communicating directly over its assigned virtualized tenant network. This configuration minimizes the burden of communication on VMM 40 and on host operating system 50.
- NIC 32 comprises a host interface, such as a PCIe interface, which connects to bus 33 of host computer 22, and a network interface, comprising one or more ports 62 connected to network 28.
- Packet processing hardware logic 64 in NIC 32 is coupled between host interface 60 and network ports 62 and comprises a transmit (Tx) pipe 66 and a receive (Rx) pipe 68, which transmit and receive data packets to and from network 28 in response to the WQEs posted by applications 56.
- Tx pipe 66 executes WQEs by composing packet headers, reading specified data from memory 31 into the packet payloads, and then transmitting the packets to network 28.
- Rx pipe 68 receives incoming packets, writes the packet data to memory 31, and notifies the destination application (for example by posting a completion queue item in an appropriate queue in memory 31), as well as returning acknowledgments over network 28 to the senders of the packets.
- These basic packet transmission and reception operations are well known in the art. Further details of the virtualization functions associated with NIC 32 are described, for example, in U.S. Patent 9,462,047 ,.
- Packet processing hardware logic 64 also comprises cryptographic security hardware logic module 44, which is configured, when invoked by IPsec software module 54, to apply IPsec security functions to the data packets transmitted and received by Tx pipe 66 and Rx pipe 68.
- These security functions typically include adding IPsec authentication headers to transmitted packets and using these headers to authenticate received packets and protect against replay attacks, as well as ESP encryption, decryption, encapsulation, decapsulation, and tunneling, as described in the above-mentioned RFCs.
- Hardware logic module 44 can be introduced, for example, as a stage in flow steering within pipes 66 and 68.
- Such flow steering typically uses a packet steering table, containing steering instructions keyed by header field values, as described, for example, in U.S. Patent Application Publications 2013/0114599 and 2016/0359768 ,.
- the flow steering entries can indicate which flows are subject to IPsec handling and which IPsec operations should be applied to each flow.
- the flow steering mechanism in pipes 66 and 68 is governed by flow steering entries that are configurable by software.
- Flow steering processing begins from a single flow steering root entry for Tx pipe 66 and another for Rx pipe 68.
- Flow steering entries are added to the root entry to form a flow steering tree, which is an acyclic graph.
- IPsec flow steering entries can specify one or more actions to be performed by hardware logic module 44, for example:
- the flow steering tree for a received packet could include the following sequence of entries and corresponding actions:
- NIC 32 After all steering stages have been successfully completed, NIC 32 passes the packet to its destination process. In case of an exception in one of the IPsec processing stages, however, NIC 32 will pass the packet to VMM 40 for handling in software. On the other hand, if a packet does not satisfy the IPsec protection requirements dictated by the applicable security policy, NIC 32 will block the packet altogether.
- security policy rules are included in the steering tree to select the entry in the SAD that is to be used for each outgoing packet flow, which in turn will determine whether the packet is to be referred to hardware logic module 44 for IPsec processing and, if so, which IPsec functions to apply.
- the SAD entry is selected for each packet depending on certain header fields, such as the IP header, a transport header, and/or an encapsulation header.
- the flow steering tree for an outgoing packet could include the following sequence of entries and corresponding actions:
- the steering logic in Tx pipe 66 may decide to bypass further IPsec processing or possibly to drop the packet.
- Rx pipe 68 can be directed by the flow steering entries to decapsulate and handle an IPsec packet that is encapsulated in a Virtual Extensible LAN (VXLAN) packet, or a VXLAN packet that is encapsulated inside an IPsec packet.
- VXLAN Virtual Extensible LAN
- Tx pipe 66 can similarly be directed to perform this sort of multi-level encapsulation.
- Hardware logic module 44 maintains an IPsec state context 70 with respect to each of applications 56 or virtual machines 38 for which IPsec software module 54, under the control of VMM 40, has invoked IPsec services. If module 44 is required to handle a large number of packet flows, state context data can be stored in memory 31 and cached in NIC 32 as needed. Context 70 contains an SA database, which holds keys and encryption parameters for use in authenticating and encapsulating packets. Typically, context 70 also includes counters 72, which keep track of packet serial numbers, replay protection windows, and numbers of transmitted bytes and/or packets, as required by the IPsec protocol. Further details of the information maintained in context 70 can be found in the above-mentioned RFC 4301.
- hardware logic module 44 Upon encountering an exception in applying the mandated IPsec processing to a given packet (or flow of packets) to or from a given virtual machine 38 or application 56, hardware logic module 44 transfers the packet or flow to IPsec software module 54 for further handling. Hardware logic module 44 also transfers the corresponding state context 70 for the given virtual machine or application to software module 54. VMM 40 uses this state context in software module 54 to continue processing the packet or flow, in a manner that is transparent to the virtual machine or application.
- hardware logic module 44 continues handling subsequent packets in the flow.
- all further offload of this flow (or specifically, handling of this IPsec SA) is terminated, and all subsequent packets in this flow are processed completely by the VMM.
- termination is only possible after fully handling the exception to allow correct identification of the flow.
- the hardware logic module 44 when hardware logic module 44 encounters an exception in applying the required IPsec operations to a data packet that is directed to a given virtual machine 38, the hardware logic module transfers the data packet, together with IPsec state context 70 with respect to the given virtual machine, to IPsec software module 54.
- This software module processes the data packet using the state context and passes the data packet, after processing, to the given virtual machine while updating the state context (including replay protection data and sequence numbers, for example).
- state context including replay protection data and sequence numbers, for example.
- hardware logic module 44 may encounter an exception when a packet that should be encrypted is transmitted before any cryptographic information has been configured for handling this packet. VMM 40 will take over and perform a handshake to configure the cryptographic information after receiving this packet.
- Fig. 3 is a ladder diagram showing communications exchanged between NIC 32 and software running on CPU 30, in accordance with an embodiment of the invention.
- Actions taken by VMM 40 in this description include functions carried out by IPsec software module 54.
- NIC 32 receives IPsec packets 80 from network 28 that are destined for a given VM 38, and processes packets 80 in hardware logic module 44 to authenticate and decapsulate the packets as appropriate.
- NIC 32 then writes corresponding packet data 82 to memory 31 and notifies the appropriate VM 38.
- the VM is uninvolved in IPsec functions and may be unaware that such functions are even being applied to packets that the VM transmits and receives.
- NIC 32 When NIC 32 receives an IPsec fragment 84, however, hardware logic module 44 recognizes that the packet has been fragmented and notifies VMM 40 that an exception 86 has occurred. (This sort of fragmentation can occur, for example, when the sender of the packet, such as host 24, applied IPsec authentication and/or encapsulation to a large packet, and one of routers 36 broke the original large packet into smaller IP packets for transmission on to host 22.) Specifically, NIC 32 typically writes packet fragments to memory 31 for handling by software and places an event report in a queue for handling by VMM 40. Upon receiving and parsing the event report, VMM 40 reads and reassembles the pieces of the original packet from the fragments in memory 31, at a defragmentation step 88.
- IPsec software module 54 carries out a handshake 90 with hardware logic module 44 in order to retrieve and update the parameters in state context 70 that are needed in order to process the defragmented packet. Details of handshake 90 are shown in Fig. 4 . Handshake 90 can take place concurrently with or before defragmentation step 88, rather than after defragmentation as shown in Fig. 3 .
- IPsec software module 54 has completed defragmentation step 88 and handshake 90, it is able to carry out the required IPsec processing operations on the received packets.
- NIC 32 passes the current IPsec extended sequence number (ESN) to VMM 40, for use in decrypting the defragmented packet data.
- ESN IPsec extended sequence number
- VMM 40 updates context information used by hardware logic module 44 in replay protection, and module 44 is thus able to continue processing subsequent packets in this flow.
- VMM 40 After completion of the IPsec processing, VMM 40 then passes the processed data packet to the destination VM 38 by looping the processed data packet through NIC 32.
- VMM 40 writes a recovered packet 92 to NIC 32 as though it were transmitting the packet over network 28 to any destination VM.
- NIC 32 applies its usual SR-IOV virtual switching functionality to recovered packet 92 in order to write corresponding packet data 94 to VM 38.
- VM 38 remains unaware of the chain of IPsec processing and exception handling that was applied.
- Fig. 4 is a flow chart that schematically shows details of handshake 90, in accordance with an embodiment of the invention.
- VMM 40 uses this handshake to acquire IPsec state context 70 with respect to the VM 38 to which the packet or flow is destined.
- the steps in the handshake are carried out by reading and writing instructions and data between VMM 40 and NIC 32 over bus 33.
- Fig. 4 shows one example of such a handshake, but alternative implementations can also be used, depending on the protocol and the context information that is exchanged.
- VMM 40 queries NIC 32 for packet sequence number information that is used in IPsec processing, at a query step 100.
- VMM 40 may request the most significant bits (MSB) of the IPsec extended sequence number (ESN), which are needed for authentication and decryption of received packets.
- MSB most significant bits
- ESN IPsec extended sequence number
- NIC 32 returns this information from IPsec state context 70 to VMM 40, at a query response step 102.
- VMM 40 also performs atomic read and update operations on replay protection information used in IPsec, at an atomic update step 104. This step fixes the value of the sliding window that is used in IPsec to prevent replay attacks, and thus ensures that VMM 40 will pass no replayed packets to VM 38.
- VMM 40 takes over processing the exception that has occurred (for example, packet fragmentation) in this IPsec SA, at a handover step 106.
- VMM 40 updates IPsec state context 70, thus enabling hardware logic module 44 in NIC 32 to continue processing subsequent packets in the flow.
- hardware logic module 44 in NIC 32 can apply IPsec ESP encapsulation, decapsulation and tunneling to outgoing and incoming packets, using the appropriate state context 70 for each virtual machine 38.
- module uses counters 72 to maintain counts of data transferred using a given SA, for example, data transmitted or received through a particular IPsec tunnel. When the count reaches a predefined limit, NIC 32 will stop transmitting or receiving the data packets and will transfer the corresponding state context 70 to VMM 40 for update of the state context. At this stage, VMM 40 may negotiate a new SA over network 28 with a remote host, whereupon the ESP operations can resume.
- hardware logic module 44 in NIC 32 may decapsulate incoming packets from network 28 using state context 70. Upon receiving instructions from VMM 40 to terminate the decapsulation, NIC 32 can loop the received data packets back to network 28, and may also perform additional IPsec functions on the outgoing packets.
- Fig. 5 is a block diagram that schematically illustrates a host computer 110 with a "smart NIC" 112 comprising an embedded controller 114, in accordance with an embodiment of the invention.
- This embodiment handles IPsec offload in similar fashion to the embodiments described above, except that IPsec hardware logic module 44 in NIC 112 interacts with an IPsec software module 116 running on embedded controller 114, rather than on CPU 30.
- IPsec offload with transfer of state context to an IPsec software module that were described above with reference to NIC 32 and VMM 40 can likewise be applied, mutatis mutandis, in smart NIC 112.
- Applications and virtual machines running on CPU 30 in computer host 110 can similarly be unaware of the IPsec functions applied by NIC 112.
- NIC 112 comprises network ports 62, connected to network 28, and host interface 60, connected to bus 33 of computer 110.
- Packet processing hardware logic 64 is coupled between host interface 60 and network ports 62 so as to transmit and receive data packets over the network from and to the applications running on CPU 30.
- IPsec hardware logic module 44 when invoked by embedded controller 114, applies IPsec processing to the data packets transmitted and received by one or more of the applications running on the CPU, while maintaining an IPsec state context (as shown in Fig. 2 ) with respect to each of these applications.
- IPsec software module 116 running on embedded controller 114 is configured to apply IPsec processing and to invoke hardware logic module 44 as appropriate.
- hardware logic module 44 encounters an exception in applying IPsec to a data packet directed to a given application, module 44 transfers the data packet, together with the corresponding IPsec state context, to IPsec software module 116.
- Embedded controller 114 then processes the data packet using the state context and passes the data packet, after processing, to the given application on CPU 30.
- computers 22 and 110 were described above specifically with reference to IPsec, the hardware and software architectures and methods of operation of these computers can similarly be applied, mutatis mutandis, in offload of other cryptographic security protocols.
Description
- The present invention relates generally to computer network communications, and particularly to apparatus and methods for performing security-related operations on data packets transmitted and received over a network.
- Internet Protocol Security (IPsec) is a network protocol suite that authenticates and encrypts data packets sent over a network. IPsec operates the Internet Layer (referred to generically as the network layer, or Layer 3) of the Internet Protocol (IP) suite, and can automatically secure applications and data transmitted in IP packets. IPsec uses cryptographic security services to support network-level peer authentication, data-origin authentication, data integrity, data confidentiality (encryption), and replay protection. IPsec also includes protocols for establishing mutual authentication between agents at the beginning of the session and negotiation of cryptographic keys for use during the session. The IPsec architecture and operational features are specified in a series of Requests for Comments (RFCs) published on line by the Internet Engineering Task Force (IETF), include RFC 4301, RFC 4303, and RFC 4106. Specific features of IPsec that are used in popular implementations include the following:
- Authentication Headers (AH) are added to IPsec packets to provide connectionless data integrity and data origin authentication for IP datagrams, along with protection against replay attacks. The AH contains a 32-bit sequence number and an integrity check value. To protect against replay attacks, the sequence number is never reused in a given Security Association, and when it reaches its maximum value, a new Security Association is negotiated.
- Encapsulating Security Payload (ESP) is an encrypted payload format that provides confidentiality, data-origin authentication, connectionless integrity, an anti-replay service, and limited traffic-flow confidentiality. In Tunnel Mode, the entire original IP packet is encapsulated with a new packet header added, and ESP protection is applied to the whole inner IP packet (including the header), while an outer header for network routing remains unprotected.
- Security Associations (SA) provide the algorithms and data that are used in deriving and negotiating the parameters necessary for AH and/or ESP operations between a pair of IPsec endpoints. A security association database (SAD) defines the parameters associated with each SA.
- Because IPsec is computation-intensive, some authors have suggested offloading IPsec processing from the host processor to a network interface controller (NIC). For example,
U.S. Patent 8,006,297 describes a method and system for combined security protocol and packet filter offload and onload. This patent describes a NIC that includes a security association database (SADE) comprising a plurality of security associations (SAs), a cryptographic offload engine configured to decrypt a packet using one of the plurality of SAs, a security policy database (SPD) comprising a plurality of security policies (SPs) and a plurality of filter policies, and a policy engine configured to determine an admittance of the packet using one of the plurality of SPs from the SPD and apply one of the plurality of filter policies to the packet. - As another example,
U.S. Patent Application Publication 2010/0228962 describes offloading cryptographic protection processing of packet data sent according to a security protocol between a first computer and a second computer via a forwarding device. The forwarding device performs a portion of the processing, and forwards the packet data to a third computer, connected to the forwarding device, for other processing. The third computer may support non-standard extensions to the security protocol, such as extensions used in authorizing and establishing a connection over the secure protocol. The third computer sends the results of its processing, such as a cryptographic key, or a detected access control policy, to the forwarding device. -
EP1657878A1 describes mechanisms for transferring processor control of secure Internet Protocol (IPSec) security association (SA) functions between a host and a target processing devices of a computerized system, such as processors in a host CPU and a NIC. In one aspect of the invention, the computation associated with authentication and/or encryption is offloaded while the host maintains control of when SA functions are offloaded, uploaded, invalidated, and re-keyed. The devices coordinate to maintain metrics for the SA, including support for both soft and hard limits on SA expiration. Timer requirements are minimized for the target. The offloaded SA function may be embedded in other offloaded state objects of intermediate software layers of a network stack. -
US2013125125A1 describes a computer system comprising a virtual machine operating on a physical machine; and a management block operating on the physical machine and managing the virtual machine. The virtual machine has a specific function processing module that performs specific function processing with respect to a packet for transmission and a received packet. The management block has a virtual switch that relays a packet transmitted and received by the virtual machine. The virtual switch has an offload processing block that performs the specific function processing if the specific function processing is offloaded to the management block. If the specific function processing is offloaded from the virtual machine to the management block, the specific function processing module notifies the management block of processing information required for the specific function processing, and the offload processing block executes the specific function processing based on the processing information received from the virtual machine. - The invention is defined by the independent claims. Dependent claims define preferred embodiments.
- Embodiments of the present invention that are described hereinbelow provide improved apparatus and methods for offload of security-related functions to hardware logic.
- There is therefore provided, in accordance with an embodiment of the invention, computing apparatus, including a host processor, which is configured to run a virtual machine monitor (VMM), which supports a plurality of virtual machines running on the host processor, and which includes a cryptographic security software module configured to apply a cryptographic security protocol to data packets transmitted and received by one or more of the virtual machines. A network interface controller (NIC) is configured to link the host processor to a network so as to transmit and receive the data packets from and to the virtual machines over the network, and includes a cryptographic security hardware logic module, which is configured, when invoked by the VMM, to apply the cryptographic security protocol to the data packets transmitted and received by the one or more of the virtual machines while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the virtual machines. The NIC is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given virtual machine, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given virtual machine, to the cryptographic security software module of the VMM, which processes the data packet using the state context and passes the data packet, after processing, to the given virtual machine.
- In the disclosed embodiments, the VMM is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the virtual machines in invocation or implementation of the cryptographic security protocol.
- In some embodiments, the VMM is configured, when the NIC has encountered the exception, to acquire the state context of the cryptographic security protocol with respect to the given virtual machine by performing a predefined handshake with the NIC. The predefined handshake includes, for example, querying and receiving packet sequence number information from the NIC and updating replay protection information used in the cryptographic security protocol.
- In one embodiment, the exception includes a fragmentation of the data packet following application of the cryptographic security protocol by a sender of the data packet, and wherein the VMM is configured to defragment the data packet.
- In a disclosed embodiment, the VMM is configured, after processing the data packet, to pass the processed data packet to the given virtual machine by looping the processed data packet through the NIC to the given virtual machine.
- In some embodiments, the NIC is configured to apply an encapsulation, using the state context of the cryptographic security protocol, to the data packets transmitted from the given virtual machine to a specified destination, while maintaining a count of the data transmitted using the state context, and when the count reaches a predefined limit, to stop transmitting the data packets to the specified destination and transfer the state context to the cryptographic security software module of the VMM for update of the state context. Additionally or alternatively, the NIC is configured to apply a decapsulation, using the state context of the cryptographic security protocol, to encapsulated data packets received from the network, and upon receiving instructions from the VMM to terminate the decapsulation, to loop the received data packets back to the network.
- In an example embodiment, the cryptographic security protocol includes an IPsec protocol.
- There is also provided, in accordance with an embodiment of the invention, computing apparatus, including a network interface, configured to be connected to a network, and a host interface, configured to be connected to a peripheral component bus of a host computer. An embedded controller is configured to run a cryptographic security software module, which applies a cryptographic security protocol to data packets transmitted and received by applications running on the host computer. Packet processing hardware logic is coupled between the host interface and the network interface so as to transmit and receive data packets over the network from and to the applications running on the host computer, and includes a cryptographic security hardware logic module, which is configured, when invoked by the embedded controller, to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the applications. The packet processing hardware logic is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given application, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module of the embedded controller, which processes the data packet using the state context and passes the data packet, after processing, to the given application.
- In the disclosed embodiments, the embedded controller is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the applications in invocation or implementation of the cryptographic security protocol.
- There is additionally provided, in accordance with an embodiment of the invention, a method for computing, which includes running on a programmable processor a cryptographic security software module configured to apply a cryptographic security protocol to data packets transmitted and received by applications running on a host computer. A network interface controller (NIC) is coupled between the host processor and a network so as to transmit and receive the data packets from and to the applications over the network. The cryptographic security software module invokes a cryptographic security hardware logic module in the NIC, thereby causing to the cryptographic security hardware logic module to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the applications. Upon encountering in the cryptographic security hardware logic module an exception in applying the cryptographic security protocol to a data packet directed to a given application, the data packet is transferred, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module running on the programmable processor. The data packet is processed on the programmable processor using the state context and the cryptographic security software module, which passes the data packet, after the processing, to the given application.
- The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
-
-
Fig. 1 is a block diagram that schematically illustrates a networked computer system, in accordance with an embodiment of the present invention; -
Fig. 2 is a block diagram that schematically shows details of a network interface controller (NIC) and software running on a host processor, in accordance with an embodiment of the present invention; -
Fig. 3 is a ladder diagram showing communications exchanged between a NIC and software running on a host processor, in accordance with an embodiment of the invention; -
Fig. 4 is a flow chart that schematically illustrates a method for performing a security-related handshake between a NIC and a virtual machine monitor (VMM), in accordance with an embodiment of the invention; and -
Fig. 5 is a block diagram that schematically illustrates a host computer with a NIC having an embedded controller, in accordance with an embodiment of the invention. - In view of the heavy computational burden involved in cryptographic security protocols, such as IPsec, offloading the processing tasks to a NIC is a desirable solution. The core computational functions of protocols such as IPsec can be implemented efficiently in a cryptographic security hardware logic module, which can be designed in such a way that the necessary packet processing is performed without reducing the data throughput of the NIC.
- Almost every protocol, however, has its exceptions, which fall outside the core logical functions. In IP networks, for example, packets to which an AH or ESP was applied at the sending node may be fragmented into smaller packets on the way to their destination. The receiving node must first defragment these packets before AH authentication and decapsulation. Implementing this sort of defragmentation in the NIC requires substantial amounts of memory and re-ordering logic, which increase chip size and processing latency.
- As another example, for reasons of replay protection, IPsec requires that a given SA context be used for no more than a certain number of packets or volume of data. The IPsec endpoints are supposed to maintain a count of data transmitted using the current SA, and then negotiate a new SA when the count reaches a predefined limit. In this case, too, implementing this sort of functionality in hardware logic in the NIC is impractical.
- One solution to the sorts of problems described above is for the NIC simply to drop packets when exceptions occur and rely on higher-level software to recover and invoke retransmission when needed. These sorts of solutions generally require that the software applications that transmit and receive the packets be aware of and involved in the cryptographic security protocol. This software involvement adds to the processing burden that is imposed on the CPU, as well as increasing packet latency and reducing throughput.
- Embodiments of the present invention that are described herein address these problems by means of a novel collaboration between a cryptographic security hardware logic module in the NIC and a cryptographic security software module running on a processor in a privileged domain. The domain is "privileged" in that it has access to and is able to make changes in the state context of the cryptographic security protocol, wherein this context includes parameters that are used in encryption and authentication, such as packet sequence numbers, counters, and cryptographic keys. Examples of such privileged domains, as illustrated in the embodiments described below, include a virtual machine monitor (VMM, also referred to as the hypervisor) in a host computer running virtual machines, or an embedded programmable controller in a smart NIC. The privileged software module is able to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the (non-privileged) user-domain applications that transmit and receive the packets via the NIC. In fact, the user-domain applications need not even be aware that packet encryption or authentication is being applied to the packets that they transmit and receive.
- In the disclosed embodiments, when cryptographic security is to be applied to data packets transmitted and received by one or more applications (including applications running on a given virtual machine), the cryptographic security software module invokes the cryptographic security hardware logic module in the NIC. The hardware logic module will then apply the appropriate security operations to the data packets transmitted and received by these applications, while maintaining the state context of the cryptographic security protocol with respect to each of the applications in question. When the security hardware logic module encounters an exception in applying the cryptographic security protocol to a data packet directed to a given application, such as a fragmented packet, it transfers the data packet, together with the applicable state context, to the cryptographic security software module. This software module processes the data packet, using the state context, and passes the processed data packet to the appropriate application.
- The embodiments that are described below relate specifically, for the sake of clarity and concreteness, to IPsec encryption, authentication and encapsulation. The principles of the present invention, however, are by no means limited to IPsec and may alternatively be applied in implementing other cryptographic security protocols that are known in the art, particularly datagram-based packet encryption, authentication and encapsulation protocols. Examples of such protocols include Media Access Control Security (MACsec), as defined by IEEE standard 802.1AE, and Datagram Transport Layer Security (DTLS), specified in RFC 4347.
-
Fig. 1 is a block diagram that schematically illustrates anetworked computer system 20, in accordance with an embodiment of the present invention.System 20 comprisesmultiple host computers packet data network 28. Typically, although not necessarily,data network 28 is a Layer-3 network, such as an IP network, and thus comprises Layer-3routers 36, as well as switches 34, through which hosts 22, 24, 26, ..., may connect to the network. The principles of the present invention, however, are similarly applicable over other sorts of data networks, such as InfiniBand networks; and the methods and circuits described herein can be used to support various sorts of packet-level cryptographic security protocols, including both Layer-2 and Layer-3 protocols. - Each
host system memory 31 and a network interface controller (NIC) 32.NIC 32 is connected by abus 33 toCPU 30 andmemory 31, and is connected via one ofswitches 34 tonetwork 28.Bus 33 may comprise, for example, a peripheral component bus, such as PCI Express® (PCIe®) bus, or a dedicated system bus of the CPU. A cryptographic securityhardware logic module 44 inNIC 32 can be invoked to apply a cryptographic security protocol, such as IPsec, to outgoing and incoming data packets to and fromnetwork 28, as described further hereinbelow. -
Hosts Fig. 1 ) may run on any givenCPU 30. A virtual machine monitor (VMM) 40 in the CPU native domain interacts with the kernels of the guest operating systems ofvirtual machines 38 in a manner that emulates the host processor and allows the virtual machines to share the resources of the CPU. -
NIC 32 comprises packet processing circuitry, which is configured to appear to the programs running onCPU 30 as multiple virtual NICs (vNICs) 42. In a model that is known as single-root I/O virtualization (SR-IOV), eachvirtual machine 38 interacts withNIC 32 as though the NIC was dedicated to that virtual machine, linking the virtual machine to other machines (virtual and/or physical) onnetwork 28. In this regard,NIC 32 acts as a virtual switch, connecting each of the virtual machines to a particular tenant network while allowingvNICs 42 to share the same physical port tounderlying data network 28. - This virtualization of NIC functions is also supported by offloading of IPsec functions from
VMM 40 toNIC 32, as explained below in greater detail. In the example shown inFig. 1 , VM2 inhost 22 and VM3 inhost 24 reside on the same tenant network. In network virtualization systems that are known in the art, to send an IPsec packet to VM3, VM2 submits the packet toVMM 40 inhost 22; and the VMM adds an AH, encapsulates the packet and transmits it to the VMM inhost 24, which then authenticates, decapsulates, and passes the packet to VM3. By contrast, in the present embodiment, VM2 inhost 22 communicates with VM3 inhost 24 via anIPsec tunnel 46 between therespective NICs 32, without necessarily even being aware that IPsec authentication or encapsulation is taking place. -
Fig. 2 is a block diagram that schematically shows details ofNIC 32 and software running onCPU 30, in accordance with an embodiment of the present invention. The software running onCPU 30, including both operating system and application programs, may be downloaded to the CPU in electronic form, over a network for example. Additionally or alternatively, the software may be stored on tangible, non-transitory computer-readable media, such as optical, magnetic or electronic memory media, which may be embodied inmemory 31. -
CPU 30 operates anative domain 48, with ahost operating system 50 and other privileged functions, including anIPsec software module 54. In addition, the CPU concurrently runs one or morevirtual machines 38, as noted above, each with its ownguest operating system 52 andguest user applications 56. (Only one guest OS is shown inFig. 2 for the sake of simplicity.)VMM 40 innative domain 48 interacts with the kernels ofguest operating systems 52 in a manner that emulates the host processor and allows the virtual machines to share the resources ofCPU 30. A wide range of virtual machine software of this sort is available commercially, and further description is beyond the scope of the present disclosure. - Client processes, such as
user applications 56, communicate with the transport layer ofnetwork 28 by manipulating a transport service instance, known as a "queue pair" (QP). To send and receive messages over thenetwork using NIC 32, a client submits work items, called work queue elements (WQEs), to the appropriate queues for execution by the NIC. As illustrated inFig. 1 ,NIC 32 appears to eachvirtual machine 38 to be a dedicated I/O device, or vNIC, for use by that virtual machine in communicating directly over its assigned virtualized tenant network. This configuration minimizes the burden of communication onVMM 40 and onhost operating system 50. -
NIC 32 comprises a host interface, such as a PCIe interface, which connects tobus 33 ofhost computer 22, and a network interface, comprising one ormore ports 62 connected to network 28. Packetprocessing hardware logic 64 inNIC 32 is coupled betweenhost interface 60 andnetwork ports 62 and comprises a transmit (Tx)pipe 66 and a receive (Rx)pipe 68, which transmit and receive data packets to and fromnetwork 28 in response to the WQEs posted byapplications 56.Tx pipe 66 executes WQEs by composing packet headers, reading specified data frommemory 31 into the packet payloads, and then transmitting the packets to network 28.Rx pipe 68 receives incoming packets, writes the packet data tomemory 31, and notifies the destination application (for example by posting a completion queue item in an appropriate queue in memory 31), as well as returning acknowledgments overnetwork 28 to the senders of the packets. These basic packet transmission and reception operations are well known in the art. Further details of the virtualization functions associated withNIC 32 are described, for example, inU.S. Patent 9,462,047 - Packet
processing hardware logic 64 also comprises cryptographic securityhardware logic module 44, which is configured, when invoked byIPsec software module 54, to apply IPsec security functions to the data packets transmitted and received byTx pipe 66 andRx pipe 68. These security functions typically include adding IPsec authentication headers to transmitted packets and using these headers to authenticate received packets and protect against replay attacks, as well as ESP encryption, decryption, encapsulation, decapsulation, and tunneling, as described in the above-mentioned RFCs. -
Hardware logic module 44 can be introduced, for example, as a stage in flow steering withinpipes U.S. Patent Application Publications 2013/0114599 and2016/0359768 ,. - The flow steering entries can indicate which flows are subject to IPsec handling and which IPsec operations should be applied to each flow.
- The flow steering mechanism in
pipes Tx pipe 66 and another forRx pipe 68. Flow steering entries are added to the root entry to form a flow steering tree, which is an acyclic graph. IPsec flow steering entries can specify one or more actions to be performed byhardware logic module 44, for example: - 1. Encrypt/decrypt and authenticate.
- 2. Check replay protection against a replay-window.
- 3. Add/remove IPsec headers.
- These functions can be combined to provide full IPsec offload, and can be interleaved with flow steering entries unrelated to IPsec, such as various header matching functions.
- For example, in
Rx pipe 68, the flow steering tree for a received packet could include the following sequence of entries and corresponding actions: - 1. Check destination medium access control (MAC) address.
- 2. Check IP address.
- 3. Check Security Parameters Index (SPI) of IPsec ESP.
- 4. IPsec decryption and authentication.
- 5. Check and update IPsec replay protection.
- 6. Remove ESP header (decapsulate packet).
- 7. Parse packet following decapsulation.
- 8. Check IP address and drop if no match to existing list.
- 9. Check transport protocol port and drop if no match.
- 10. Apply high-level offload functions.
- After all steering stages have been successfully completed,
NIC 32 passes the packet to its destination process. In case of an exception in one of the IPsec processing stages, however,NIC 32 will pass the packet toVMM 40 for handling in software. On the other hand, if a packet does not satisfy the IPsec protection requirements dictated by the applicable security policy,NIC 32 will block the packet altogether. - In
Tx pipe 66, security policy rules are included in the steering tree to select the entry in the SAD that is to be used for each outgoing packet flow, which in turn will determine whether the packet is to be referred tohardware logic module 44 for IPsec processing and, if so, which IPsec functions to apply. The SAD entry is selected for each packet depending on certain header fields, such as the IP header, a transport header, and/or an encapsulation header. Thus, the flow steering tree for an outgoing packet could include the following sequence of entries and corresponding actions: - 1. Check destination MAC address.
- 2. Check IP address.
- 3. Check transport header.
- 4. Select SAD entry for this flow.
- 5. Add ESP header.
- 6. Encrypt and authenticate packet.
- Alternatively, depending on the SAD entry that is selected at step 4, the steering logic in
Tx pipe 66 may decide to bypass further IPsec processing or possibly to drop the packet. - More complex use cases can arise when other actions are combined with IPsec. For example,
Rx pipe 68 can be directed by the flow steering entries to decapsulate and handle an IPsec packet that is encapsulated in a Virtual Extensible LAN (VXLAN) packet, or a VXLAN packet that is encapsulated inside an IPsec packet.Tx pipe 66 can similarly be directed to perform this sort of multi-level encapsulation. -
Hardware logic module 44 maintains anIPsec state context 70 with respect to each ofapplications 56 orvirtual machines 38 for whichIPsec software module 54, under the control ofVMM 40, has invoked IPsec services. Ifmodule 44 is required to handle a large number of packet flows, state context data can be stored inmemory 31 and cached inNIC 32 as needed.Context 70 contains an SA database, which holds keys and encryption parameters for use in authenticating and encapsulating packets. Typically,context 70 also includescounters 72, which keep track of packet serial numbers, replay protection windows, and numbers of transmitted bytes and/or packets, as required by the IPsec protocol. Further details of the information maintained incontext 70 can be found in the above-mentioned RFC 4301. - Upon encountering an exception in applying the mandated IPsec processing to a given packet (or flow of packets) to or from a given
virtual machine 38 orapplication 56,hardware logic module 44 transfers the packet or flow toIPsec software module 54 for further handling.Hardware logic module 44 also transfers thecorresponding state context 70 for the given virtual machine or application tosoftware module 54.VMM 40 uses this state context insoftware module 54 to continue processing the packet or flow, in a manner that is transparent to the virtual machine or application. - In some embodiments, after
VMM 40 has resolved the exception,hardware logic module 44 continues handling subsequent packets in the flow. Alternatively, after the VMM has handled the exception, all further offload of this flow (or specifically, handling of this IPsec SA) is terminated, and all subsequent packets in this flow are processed completely by the VMM. In some cases, such as exceptions encountered in non-initial IP fragments, such termination is only possible after fully handling the exception to allow correct identification of the flow. - As explained above, when
hardware logic module 44 encounters an exception in applying the required IPsec operations to a data packet that is directed to a givenvirtual machine 38, the hardware logic module transfers the data packet, together withIPsec state context 70 with respect to the given virtual machine, toIPsec software module 54. This software module processes the data packet using the state context and passes the data packet, after processing, to the given virtual machine while updating the state context (including replay protection data and sequence numbers, for example). One example of this sort of process with respect to a fragmented packet will be described below with reference toFig. 3 . - Similar sorts of handovers from hardware to software take place in other cases of states and rules that are too complex for
hardware logic module 44 to maintain and identify, such as complex firewall rules. For example, an exception may occur when acertain VM 38 asksNIC 32 to transmit a packet that is too large to be transmitted after encapsulation, and IP fragmentation is needed.Hardware logic module 44 identifies this exception after encryption, authentication and encapsulation and passes the packet toVMM 40 for retransmission after fragmentation. - As another example,
hardware logic module 44 may encounter an exception when a packet that should be encrypted is transmitted before any cryptographic information has been configured for handling this packet.VMM 40 will take over and perform a handshake to configure the cryptographic information after receiving this packet. -
Fig. 3 is a ladder diagram showing communications exchanged betweenNIC 32 and software running onCPU 30, in accordance with an embodiment of the invention. Actions taken byVMM 40 in this description include functions carried out byIPsec software module 54. In normal operation,NIC 32 receivesIPsec packets 80 fromnetwork 28 that are destined for a givenVM 38, and processespackets 80 inhardware logic module 44 to authenticate and decapsulate the packets as appropriate.NIC 32 then writes correspondingpacket data 82 tomemory 31 and notifies theappropriate VM 38. As noted earlier, the VM is uninvolved in IPsec functions and may be unaware that such functions are even being applied to packets that the VM transmits and receives. - When
NIC 32 receives anIPsec fragment 84, however,hardware logic module 44 recognizes that the packet has been fragmented and notifiesVMM 40 that anexception 86 has occurred. (This sort of fragmentation can occur, for example, when the sender of the packet, such ashost 24, applied IPsec authentication and/or encapsulation to a large packet, and one ofrouters 36 broke the original large packet into smaller IP packets for transmission on tohost 22.) Specifically,NIC 32 typically writes packet fragments tomemory 31 for handling by software and places an event report in a queue for handling byVMM 40. Upon receiving and parsing the event report,VMM 40 reads and reassembles the pieces of the original packet from the fragments inmemory 31, at adefragmentation step 88. - In addition,
IPsec software module 54 carries out ahandshake 90 withhardware logic module 44 in order to retrieve and update the parameters instate context 70 that are needed in order to process the defragmented packet. Details ofhandshake 90 are shown inFig. 4 .Handshake 90 can take place concurrently with or beforedefragmentation step 88, rather than after defragmentation as shown inFig. 3 . - Once
IPsec software module 54 has completeddefragmentation step 88 andhandshake 90, it is able to carry out the required IPsec processing operations on the received packets. As part of the handshake,NIC 32 passes the current IPsec extended sequence number (ESN) toVMM 40, for use in decrypting the defragmented packet data. (Assuming the decryption is successful,VMM 40 updates context information used byhardware logic module 44 in replay protection, andmodule 44 is thus able to continue processing subsequent packets in this flow.) After completion of the IPsec processing,VMM 40 then passes the processed data packet to thedestination VM 38 by looping the processed data packet throughNIC 32. In other words,VMM 40 writes a recoveredpacket 92 toNIC 32 as though it were transmitting the packet overnetwork 28 to any destination VM.NIC 32 applies its usual SR-IOV virtual switching functionality to recoveredpacket 92 in order to writecorresponding packet data 94 toVM 38. Thus,VM 38 remains unaware of the chain of IPsec processing and exception handling that was applied. -
Fig. 4 is a flow chart that schematically shows details ofhandshake 90, in accordance with an embodiment of the invention. As explained above, whenNIC 32 encounters an exception in IPsec handling of a given packet or flow,VMM 40 uses this handshake to acquireIPsec state context 70 with respect to theVM 38 to which the packet or flow is destined. The steps in the handshake are carried out by reading and writing instructions and data betweenVMM 40 andNIC 32 overbus 33.Fig. 4 shows one example of such a handshake, but alternative implementations can also be used, depending on the protocol and the context information that is exchanged. - VMM 40 (or IPsec software module 54) queries
NIC 32 for packet sequence number information that is used in IPsec processing, at aquery step 100. For example,VMM 40 may request the most significant bits (MSB) of the IPsec extended sequence number (ESN), which are needed for authentication and decryption of received packets.NIC 32 returns this information fromIPsec state context 70 toVMM 40, at aquery response step 102. -
VMM 40 also performs atomic read and update operations on replay protection information used in IPsec, at anatomic update step 104. This step fixes the value of the sliding window that is used in IPsec to prevent replay attacks, and thus ensures thatVMM 40 will pass no replayed packets toVM 38. Once the handshake is complete,VMM 40 takes over processing the exception that has occurred (for example, packet fragmentation) in this IPsec SA, at ahandover step 106.VMM 40 updatesIPsec state context 70, thus enablinghardware logic module 44 inNIC 32 to continue processing subsequent packets in the flow. - As another example (not shown explicitly in the figures),
hardware logic module 44 inNIC 32 can apply IPsec ESP encapsulation, decapsulation and tunneling to outgoing and incoming packets, using theappropriate state context 70 for eachvirtual machine 38. In accordance with IPsec requirements, module usescounters 72 to maintain counts of data transferred using a given SA, for example, data transmitted or received through a particular IPsec tunnel. When the count reaches a predefined limit,NIC 32 will stop transmitting or receiving the data packets and will transfer thecorresponding state context 70 toVMM 40 for update of the state context. At this stage,VMM 40 may negotiate a new SA overnetwork 28 with a remote host, whereupon the ESP operations can resume. - As another example,
hardware logic module 44 inNIC 32 may decapsulate incoming packets fromnetwork 28 usingstate context 70. Upon receiving instructions fromVMM 40 to terminate the decapsulation,NIC 32 can loop the received data packets back tonetwork 28, and may also perform additional IPsec functions on the outgoing packets. -
Fig. 5 is a block diagram that schematically illustrates ahost computer 110 with a "smart NIC" 112 comprising an embeddedcontroller 114, in accordance with an embodiment of the invention. This embodiment handles IPsec offload in similar fashion to the embodiments described above, except that IPsechardware logic module 44 inNIC 112 interacts with anIPsec software module 116 running on embeddedcontroller 114, rather than onCPU 30. The features of IPsec offload with transfer of state context to an IPsec software module that were described above with reference toNIC 32 andVMM 40 can likewise be applied, mutatis mutandis, insmart NIC 112. Applications and virtual machines running onCPU 30 incomputer host 110 can similarly be unaware of the IPsec functions applied byNIC 112. - As in the preceding embodiments,
NIC 112 comprisesnetwork ports 62, connected to network 28, andhost interface 60, connected tobus 33 ofcomputer 110. Packetprocessing hardware logic 64 is coupled betweenhost interface 60 andnetwork ports 62 so as to transmit and receive data packets over the network from and to the applications running onCPU 30. IPsechardware logic module 44, when invoked by embeddedcontroller 114, applies IPsec processing to the data packets transmitted and received by one or more of the applications running on the CPU, while maintaining an IPsec state context (as shown inFig. 2 ) with respect to each of these applications. - In the present embodiment,
IPsec software module 116 running on embeddedcontroller 114 is configured to apply IPsec processing and to invokehardware logic module 44 as appropriate. Whenhardware logic module 44 encounters an exception in applying IPsec to a data packet directed to a given application,module 44 transfers the data packet, together with the corresponding IPsec state context, toIPsec software module 116. Embeddedcontroller 114 then processes the data packet using the state context and passes the data packet, after processing, to the given application onCPU 30. - Although the features of
computers
Claims (12)
- Computing apparatus, comprising:a host processor (30), which is configured to run a virtual machine monitor, VMM (40), which supports a plurality of virtual machines (38) running on the host processor, and which comprises a cryptographic security software module (54) configured to apply a cryptographic security protocol to data packets transmitted and received by one or more of the virtual machines; anda network interface controller, NIC (32), which is configured to link the host processor to a network (28) so as to transmit and receive the data packets from and to the virtual machines over the network, and which comprises a cryptographic security hardware logic module (44), which is configured, when invoked by the VMM, to apply the cryptographic security protocol to the data packets transmitted and received by the one or more of the virtual machines while maintaining a state context (70) of the cryptographic security protocol with respect to each of the one or more of the virtual machines,wherein the NIC is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given virtual machine, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given virtual machine, to the cryptographic security software module of the VMM, which processes the data packet using the state context and passes the data packet, after processing, to the given virtual machine.
- The apparatus according to claim 1, wherein the cryptographic security software module is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the virtual machines in invocation or implementation of the cryptographic security protocol.
- The apparatus according to claim 1, wherein the VMM is configured, when the NIC has encountered the exception, to acquire the state context of the cryptographic security protocol with respect to the given virtual machine by performing a predefined handshake with the NIC.
- The apparatus according to claim 3, wherein the predefined handshake comprises querying and receiving packet sequence number information from the NIC and updating replay protection information used in the cryptographic security protocol.
- The apparatus according to claim 1, wherein the exception comprises a fragmentation of the data packet following application of the cryptographic security protocol by a sender of the data packet, and wherein the VMM is configured to defragment the data packet.
- The apparatus according to any of claims 1 to 5, wherein the NIC is configured to apply an encapsulation, using the state context of the cryptographic security protocol, to the data packets transmitted from the given virtual machine to a specified destination, while maintaining a count of the data transmitted using the state context, and when the count reaches a predefined limit, to stop transmitting the data packets to the specified destination and transfer the state context to the cryptographic security software module of the VMM for update of the state context.
- The apparatus according to any of claims 1 to 5, wherein the NIC is configured to apply a decapsulation, using the state context of the cryptographic security protocol, to encapsulated data packets received from the network, and upon receiving instructions from the VMM to terminate the decapsulation, to loop the received data packets back to the network.
- The apparatus according to any preceding claim, wherein the VMM is configured to pass the processed data packet to the given virtual machine by looping the processed data packet through the NIC to the given virtual machine.
- A network interface controller (112), comprising:a network interface (62), configured to be connected to a network (28);a host interface (60), configured to be connected to a peripheral component bus of a host computer;an embedded controller (114), which is configured to run a cryptographic security software module (116), which applies a cryptographic security protocol to data packets transmitted and received by applications running on the host computer; andpacket processing hardware logic (64), which is coupled between the host interface and the network interface so as to transmit and receive data packets over the network from and to the applications running on the host computer, and which comprises a cryptographic security hardware logic module (44), which is configured, when invoked by the embedded controller, to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context of the cryptographic security protocol with respect to each of the one or more of the applications,wherein the packet processing hardware logic is configured, upon encountering an exception in applying the cryptographic security protocol to a data packet directed to a given application, to transfer the data packet, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module of the embedded controller, which processes the data packet using the state context and passes the data packet, after processing, to the given application.
- The apparatus according to claim 9, wherein the embedded controller is configured to apply the cryptographic security protocol and to invoke the cryptographic security hardware logic module without involvement by the applications in invocation or implementation of the cryptographic security protocol.
- The apparatus according to any of claims 1 to 10, wherein the cryptographic security protocol comprises an IPsec protocol.
- A method for computing, comprising:running on a programmable processor (114, 30) a cryptographic security software module (116, 54) configured to apply a cryptographic security protocol to data packets transmitted and received by applications running on a host computer;coupling a network interface controller, NIC (112, 32), between a host processor of the host computer and a network (28) so as to transmit and receive the data packets from and to the applications over the network;invoking, by the cryptographic security software module, a cryptographic security hardware logic module (44) in the NIC, thereby causing to the cryptographic security hardware logic module to apply the cryptographic security protocol to the data packets transmitted and received by one or more of the applications while maintaining a state context (70) of the cryptographic security protocol with respect to each of the one or more of the applications;upon encountering in the cryptographic security hardware logic module an exception in applying the cryptographic security protocol to a data packet directed to a given application, transferring the data packet, together with the state context of the cryptographic security protocol with respect to the given application, to the cryptographic security software module running on the programmable processor; andprocessing the data packet on the programmable processor using the state context and the cryptographic security software module, and passing the data packet, after the processing, to the given application, and either:the programmable processor comprises a controller (114) embedded in the NIC; orthe programmable processor is comprised in a central processing unit, CPU (30), of the host computer and runs a virtual machine monitor, VMM, which supports a plurality of virtual machines running on the host computer and comprises the cryptographic security software module, and wherein the one or more of the applications run on one or more of the virtual machines.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/841,339 US10708240B2 (en) | 2017-12-14 | 2017-12-14 | Offloading communication security operations to a network interface controller |
PCT/IB2018/059824 WO2019116195A1 (en) | 2017-12-14 | 2018-12-10 | Offloading communication security operations to a network interface controller |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3725056A1 EP3725056A1 (en) | 2020-10-21 |
EP3725056B1 true EP3725056B1 (en) | 2022-01-26 |
Family
ID=64959385
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18830310.1A Active EP3725056B1 (en) | 2017-12-14 | 2018-12-10 | Offloading communication security operations to a network interface controller |
Country Status (4)
Country | Link |
---|---|
US (2) | US10708240B2 (en) |
EP (1) | EP3725056B1 (en) |
CN (1) | CN111480328B (en) |
WO (1) | WO2019116195A1 (en) |
Families Citing this family (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11005771B2 (en) | 2017-10-16 | 2021-05-11 | Mellanox Technologies, Ltd. | Computational accelerator for packet payload operations |
US11502948B2 (en) | 2017-10-16 | 2022-11-15 | Mellanox Technologies, Ltd. | Computational accelerator for storage operations |
US11095617B2 (en) | 2017-12-04 | 2021-08-17 | Nicira, Inc. | Scaling gateway to gateway traffic using flow hash |
US10708240B2 (en) * | 2017-12-14 | 2020-07-07 | Mellanox Technologies, Ltd. | Offloading communication security operations to a network interface controller |
CN110300064A (en) * | 2018-03-22 | 2019-10-01 | 华为技术有限公司 | A kind of data traffic processing method, equipment and system |
US11347561B1 (en) | 2018-04-30 | 2022-05-31 | Vmware, Inc. | Core to resource mapping and resource to core mapping |
US20210092103A1 (en) * | 2018-10-02 | 2021-03-25 | Arista Networks, Inc. | In-line encryption of network data |
US11184439B2 (en) | 2019-04-01 | 2021-11-23 | Mellanox Technologies, Ltd. | Communication with accelerator via RDMA-based network adapter |
US11277343B2 (en) | 2019-07-17 | 2022-03-15 | Vmware, Inc. | Using VTI teaming to achieve load balance and redundancy |
US11416435B2 (en) * | 2019-09-03 | 2022-08-16 | Pensando Systems Inc. | Flexible datapath offload chaining |
CN114667499A (en) * | 2019-09-11 | 2022-06-24 | 艾锐势有限责任公司 | Password and policy based device independent authentication |
US11336629B2 (en) * | 2019-11-05 | 2022-05-17 | Vmware, Inc. | Deterministic load balancing of IPSec packet processing |
US11509638B2 (en) | 2019-12-16 | 2022-11-22 | Vmware, Inc. | Receive-side processing for encapsulated encrypted packets |
IL276538B2 (en) | 2020-08-05 | 2023-08-01 | Mellanox Technologies Ltd | Cryptographic data communication apparatus |
CN114095153A (en) | 2020-08-05 | 2022-02-25 | 迈络思科技有限公司 | Cipher data communication device |
US11841985B2 (en) * | 2020-09-03 | 2023-12-12 | Pensando Systems Inc. | Method and system for implementing security operations in an input/output device |
CN112822164B (en) * | 2020-12-29 | 2023-11-03 | 北京八分量信息科技有限公司 | Method, system and related product for safely accessing data in big data system |
US11934333B2 (en) | 2021-03-25 | 2024-03-19 | Mellanox Technologies, Ltd. | Storage protocol emulation in a peripheral device |
US11934658B2 (en) | 2021-03-25 | 2024-03-19 | Mellanox Technologies, Ltd. | Enhanced storage protocol emulation in a peripheral device |
CN116069695A (en) | 2021-11-03 | 2023-05-05 | 迈络思科技有限公司 | Memory access tracking using peripheral devices |
US11863514B2 (en) | 2022-01-14 | 2024-01-02 | Vmware, Inc. | Performance improvement of IPsec traffic using SA-groups and mixed-mode SAs |
US20220231991A1 (en) * | 2022-03-28 | 2022-07-21 | Intel Corporation | Method, system and apparatus for inline decryption analysis and detection |
US11956213B2 (en) | 2022-05-18 | 2024-04-09 | VMware LLC | Using firewall policies to map data messages to secure tunnels |
CN115529180A (en) * | 2022-09-28 | 2022-12-27 | 芯启源(南京)半导体科技有限公司 | IPSec encryption and decryption unloading method |
Family Cites Families (87)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6904519B2 (en) * | 1998-06-12 | 2005-06-07 | Microsoft Corporation | Method and computer program product for offloading processing tasks from software to hardware |
US7600131B1 (en) | 1999-07-08 | 2009-10-06 | Broadcom Corporation | Distributed processing in a cryptography acceleration chip |
US9444785B2 (en) | 2000-06-23 | 2016-09-13 | Cloudshield Technologies, Inc. | Transparent provisioning of network access to an application |
US20040039940A1 (en) | 2002-08-23 | 2004-02-26 | Koninklijke Philips Electronics N.V. | Hardware-based packet filtering accelerator |
US7269171B2 (en) | 2002-09-24 | 2007-09-11 | Sun Microsystems, Inc. | Multi-data receive processing according to a data communication protocol |
US6901496B1 (en) | 2002-10-04 | 2005-05-31 | Adaptec, Inc. | Line rate buffer using single ported memories for variable length packets |
US20050102497A1 (en) | 2002-12-05 | 2005-05-12 | Buer Mark L. | Security processor mirroring |
US7587587B2 (en) * | 2002-12-05 | 2009-09-08 | Broadcom Corporation | Data path security processing |
US7290134B2 (en) | 2002-12-31 | 2007-10-30 | Broadcom Corporation | Encapsulation mechanism for packet processing |
US7734844B2 (en) * | 2003-08-19 | 2010-06-08 | General Dynamics Advanced Information Systems, Inc. | Trusted interface unit (TIU) and method of making and using the same |
US8549345B1 (en) * | 2003-10-31 | 2013-10-01 | Oracle America, Inc. | Methods and apparatus for recovering from a failed network interface card |
US7783880B2 (en) * | 2004-11-12 | 2010-08-24 | Microsoft Corporation | Method and apparatus for secure internet protocol (IPSEC) offloading with integrated host protocol stack management |
US7716730B1 (en) * | 2005-06-24 | 2010-05-11 | Oracle America, Inc. | Cryptographic offload using TNICs |
US8341237B2 (en) | 2006-10-23 | 2012-12-25 | International Business Machines Corporation | Systems, methods and computer program products for automatically triggering operations on a queue pair |
US7657659B1 (en) | 2006-11-30 | 2010-02-02 | Vmware, Inc. | Partial copying of data to transmit buffer for virtual network device |
KR20090087119A (en) | 2006-12-06 | 2009-08-14 | 퓨전 멀티시스템즈, 인크.(디비에이 퓨전-아이오) | Apparatus, system, and method for managing data in a storage device with an empty data token directive |
US8006297B2 (en) * | 2007-04-25 | 2011-08-23 | Oracle America, Inc. | Method and system for combined security protocol and packet filter offload and onload |
US20090086736A1 (en) | 2007-09-28 | 2009-04-02 | Annie Foong | Notification of out of order packets |
US8244826B2 (en) | 2007-10-23 | 2012-08-14 | International Business Machines Corporation | Providing a memory region or memory window access notification on a system area network |
US8103785B2 (en) | 2007-12-03 | 2012-01-24 | Seafire Micros, Inc. | Network acceleration techniques |
US8689292B2 (en) * | 2008-04-21 | 2014-04-01 | Api Technologies Corp. | Method and systems for dynamically providing communities of interest on an end user workstation |
US8584250B2 (en) * | 2008-09-23 | 2013-11-12 | Rite-Solutions, Inc. | Methods and apparatus for information assurance in a multiple level security (MLS) combat system |
US8572251B2 (en) | 2008-11-26 | 2013-10-29 | Microsoft Corporation | Hardware acceleration for remote desktop protocol |
US20100228962A1 (en) | 2009-03-09 | 2010-09-09 | Microsoft Corporation | Offloading cryptographic protection processing |
US9038073B2 (en) | 2009-08-13 | 2015-05-19 | Qualcomm Incorporated | Data mover moving data to accelerator for processing and returning result data based on instruction received from a processor utilizing software and hardware interrupts |
EP2306322A1 (en) | 2009-09-30 | 2011-04-06 | Alcatel Lucent | Method for processing data packets in flow-aware network nodes |
US9015268B2 (en) | 2010-04-02 | 2015-04-21 | Intel Corporation | Remote direct storage access |
US9755947B2 (en) | 2010-05-18 | 2017-09-05 | Intel Corporation | Hierarchical self-organizing classification processing in a network switch |
US8824492B2 (en) | 2010-05-28 | 2014-09-02 | Drc Computer Corporation | Accelerator system for remote data storage |
JP5772946B2 (en) | 2010-07-21 | 2015-09-02 | 日本電気株式会社 | Computer system and offloading method in computer system |
US9736116B2 (en) * | 2014-07-28 | 2017-08-15 | Intel Corporation | Cooperated approach to network packet filtering |
US9003053B2 (en) | 2011-09-22 | 2015-04-07 | Solarflare Communications, Inc. | Message acceleration |
US8996644B2 (en) | 2010-12-09 | 2015-03-31 | Solarflare Communications, Inc. | Encapsulated accelerator |
US8774213B2 (en) | 2011-03-30 | 2014-07-08 | Amazon Technologies, Inc. | Frameworks and interfaces for offload device-based packet processing |
CN103051510B (en) * | 2011-09-07 | 2016-04-13 | 微软技术许可有限责任公司 | The method and apparatus that network strategy unloads to the safety and efficiently of network interface unit |
US9397960B2 (en) | 2011-11-08 | 2016-07-19 | Mellanox Technologies Ltd. | Packet steering |
US9619406B2 (en) | 2012-05-22 | 2017-04-11 | Xockets, Inc. | Offloading of computation for rack level servers and corresponding methods and systems |
US9286472B2 (en) | 2012-05-22 | 2016-03-15 | Xockets, Inc. | Efficient packet handling, redirection, and inspection using offload processors |
US8964554B2 (en) | 2012-06-07 | 2015-02-24 | Broadcom Corporation | Tunnel acceleration for wireless access points |
US9571507B2 (en) * | 2012-10-21 | 2017-02-14 | Mcafee, Inc. | Providing a virtual security appliance architecture to a virtual cloud infrastructure |
US20140129741A1 (en) | 2012-11-07 | 2014-05-08 | Mellanox Technologies Ltd. | Pci-express device serving multiple hosts |
US10341263B2 (en) | 2012-12-10 | 2019-07-02 | University Of Central Florida Research Foundation, Inc. | System and method for routing network frames between virtual machines |
US9008097B2 (en) | 2012-12-31 | 2015-04-14 | Mellanox Technologies Ltd. | Network interface controller supporting network virtualization |
US9094219B2 (en) | 2013-03-08 | 2015-07-28 | Intel Corporation | Network processor having multicasting protocol |
US9335886B2 (en) * | 2013-03-13 | 2016-05-10 | Assured Information Security, Inc. | Facilitating user interaction with multiple domains while preventing cross-domain transfer of data |
US9582320B2 (en) | 2013-03-14 | 2017-02-28 | Nxp Usa, Inc. | Computer systems and methods with resource transfer hint instruction |
JP2015076643A (en) | 2013-10-04 | 2015-04-20 | 富士通株式会社 | Control program, control device, and control method |
US9678818B2 (en) | 2014-01-30 | 2017-06-13 | Mellanox Technologies, Ltd. | Direct IO access from a CPU's instruction stream |
US10078613B1 (en) | 2014-03-05 | 2018-09-18 | Mellanox Technologies, Ltd. | Computing in parallel processing environments |
US10218645B2 (en) | 2014-04-08 | 2019-02-26 | Mellanox Technologies, Ltd. | Low-latency processing in a network node |
US10120832B2 (en) | 2014-05-27 | 2018-11-06 | Mellanox Technologies, Ltd. | Direct access to local memory in a PCI-E device |
US9207979B1 (en) | 2014-05-28 | 2015-12-08 | Freescale Semiconductor, Inc. | Explicit barrier scheduling mechanism for pipelining of stream processing algorithms |
US9733981B2 (en) | 2014-06-10 | 2017-08-15 | Nxp Usa, Inc. | System and method for conditional task switching during ordering scope transitions |
US10423414B2 (en) | 2014-11-12 | 2019-09-24 | Texas Instruments Incorporated | Parallel processing in hardware accelerators communicably coupled with a processor |
US9787605B2 (en) | 2015-01-30 | 2017-10-10 | Nicira, Inc. | Logical router with multiple routing components |
IL238690B (en) | 2015-05-07 | 2019-07-31 | Mellanox Technologies Ltd | Network-based computational accelerator |
US10152441B2 (en) | 2015-05-18 | 2018-12-11 | Mellanox Technologies, Ltd. | Host bus access by add-on devices via a network interface controller |
US10027601B2 (en) | 2015-06-03 | 2018-07-17 | Mellanox Technologies, Ltd. | Flow-based packet modification |
US20160378529A1 (en) | 2015-06-29 | 2016-12-29 | Fortinet, Inc. | Utm integrated hypervisor for virtual machines |
US10114792B2 (en) | 2015-09-14 | 2018-10-30 | Cisco Technology, Inc | Low latency remote direct memory access for microservers |
US10929189B2 (en) | 2015-10-21 | 2021-02-23 | Intel Corporation | Mobile edge compute dynamic acceleration assignment |
US9912774B2 (en) * | 2015-12-22 | 2018-03-06 | Intel Corporation | Accelerated network packet processing |
US10831547B2 (en) | 2016-01-29 | 2020-11-10 | Nec Corporation | Accelerator control apparatus for analyzing big data, accelerator control method, and program |
US10552205B2 (en) | 2016-04-02 | 2020-02-04 | Intel Corporation | Work conserving, load balancing, and scheduling |
US10417174B2 (en) | 2016-06-24 | 2019-09-17 | Vmware, Inc. | Remote direct memory access in a virtualized computing environment |
US10318737B2 (en) | 2016-06-30 | 2019-06-11 | Amazon Technologies, Inc. | Secure booting of virtualization managers |
EP3340064B1 (en) | 2016-08-03 | 2020-12-02 | Huawei Technologies Co., Ltd. | Network interface card, computer device and data packet processing method |
US10891253B2 (en) | 2016-09-08 | 2021-01-12 | Microsoft Technology Licensing, Llc | Multicast apparatuses and methods for distributing data to multiple receivers in high-performance computing and cloud-based networks |
US20180109471A1 (en) | 2016-10-13 | 2018-04-19 | Alcatel-Lucent Usa Inc. | Generalized packet processing offload in a datacenter |
DE102016124383B4 (en) * | 2016-10-18 | 2018-05-09 | Fujitsu Technology Solutions Intellectual Property Gmbh | Computer system architecture and computer network infrastructure comprising a plurality of such computer system architectures |
US10642972B2 (en) | 2016-10-20 | 2020-05-05 | Intel Corporation | Extending packet processing to trusted programmable and fixed-function accelerators |
CN112506568A (en) | 2016-12-31 | 2021-03-16 | 英特尔公司 | System, method and apparatus for heterogeneous computing |
EP3574442A4 (en) * | 2017-01-26 | 2021-02-24 | Semper Fortis Solutions, LLC | Multiple single levels of security (msls) in a multi-tenant cloud |
US10250496B2 (en) | 2017-01-30 | 2019-04-02 | International Business Machines Corporation | Router based maximum transmission unit and data frame optimization for virtualized environments |
US11032248B2 (en) * | 2017-03-07 | 2021-06-08 | Nicira, Inc. | Guest thin agent assisted host network encryption |
US10210125B2 (en) | 2017-03-16 | 2019-02-19 | Mellanox Technologies, Ltd. | Receive queue with stride-based data scattering |
US11157422B2 (en) * | 2017-03-31 | 2021-10-26 | Intel Corporation | Shared memory for intelligent network interface cards |
US10402341B2 (en) | 2017-05-10 | 2019-09-03 | Red Hat Israel, Ltd. | Kernel-assisted inter-process data transfer |
CN110892380B (en) | 2017-07-10 | 2023-08-11 | 芬基波尔有限责任公司 | Data processing unit for stream processing |
US10423774B1 (en) * | 2017-08-22 | 2019-09-24 | Parallels International Gmbh | System and method for establishing secure communication channels between virtual machines |
US10382350B2 (en) | 2017-09-12 | 2019-08-13 | Mellanox Technologies, Ltd. | Maintaining packet order in offload of packet processing functions |
US11005771B2 (en) | 2017-10-16 | 2021-05-11 | Mellanox Technologies, Ltd. | Computational accelerator for packet payload operations |
US20190163364A1 (en) | 2017-11-30 | 2019-05-30 | Eidetic Communications Inc. | System and method for tcp offload for nvme over tcp-ip |
US10938784B2 (en) * | 2017-12-05 | 2021-03-02 | Assured Information Security, Inc. | Dedicating hardware devices to virtual machines in a computer system |
US10708240B2 (en) * | 2017-12-14 | 2020-07-07 | Mellanox Technologies, Ltd. | Offloading communication security operations to a network interface controller |
US10956336B2 (en) | 2018-07-20 | 2021-03-23 | International Business Machines Corporation | Efficient silent data transmission between computer servers |
US11036650B2 (en) | 2019-09-19 | 2021-06-15 | Intel Corporation | System, apparatus and method for processing remote direct memory access operations with a device-attached memory |
-
2017
- 2017-12-14 US US15/841,339 patent/US10708240B2/en active Active
-
2018
- 2018-12-10 WO PCT/IB2018/059824 patent/WO2019116195A1/en unknown
- 2018-12-10 CN CN201880079802.2A patent/CN111480328B/en active Active
- 2018-12-10 EP EP18830310.1A patent/EP3725056B1/en active Active
-
2020
- 2020-04-27 US US16/858,874 patent/US10958627B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
EP3725056A1 (en) | 2020-10-21 |
US10708240B2 (en) | 2020-07-07 |
WO2019116195A1 (en) | 2019-06-20 |
CN111480328B (en) | 2022-08-02 |
US20200259803A1 (en) | 2020-08-13 |
US20190190892A1 (en) | 2019-06-20 |
US10958627B2 (en) | 2021-03-23 |
CN111480328A (en) | 2020-07-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3725056B1 (en) | Offloading communication security operations to a network interface controller | |
EP3605976B1 (en) | Message sending method and network device | |
US8055895B2 (en) | Data path security processing | |
US10757138B2 (en) | Systems and methods for storing a security parameter index in an options field of an encapsulation header | |
US9015467B2 (en) | Tagging mechanism for data path security processing | |
US10250571B2 (en) | Systems and methods for offloading IPSEC processing to an embedded networking device | |
WO2021207231A1 (en) | Application aware tcp performance tuning on hardware accelerated tcp proxy services | |
US20110271096A1 (en) | Loosely-Coupled Encryption Functionality for Operating Systems | |
WO2020063528A1 (en) | Method, apparatus and system for communication between virtual machines in data center | |
US9467471B2 (en) | Encrypted communication apparatus and control method therefor | |
Raumer et al. | Efficient serving of VPN endpoints on COTS server hardware | |
US20110271097A1 (en) | Loosely-Coupled Encryption Functionality for Operating Systems | |
CN117254976B (en) | National standard IPsec VPN realization method, device and system based on VPP and electronic equipment | |
US20230403260A1 (en) | Computer and Network Interface Controller Offloading Encryption Processing to the Network Interface Controller and Using Derived Encryption Keys | |
US20240106647A1 (en) | Methods and systems of a packet orchestration to provide data encryption at the ip layer, utilizing a data link layer encryption scheme | |
Pismenny et al. | Securitization of cloud, edge and IoT communications through hardware accelerations/offloadings | |
WO2011139440A2 (en) | Loosely-coupled encryption functionality for operating systems | |
CN117811787A (en) | Information configuration method, device, equipment and storage medium | |
Rosen et al. | IPsec | |
Luniya et al. | SmartX--Advanced Network Security for Windows Opearating System |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20200504 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20210819 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1466168 Country of ref document: AT Kind code of ref document: T Effective date: 20220215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602018030225 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20220126 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1466168 Country of ref document: AT Kind code of ref document: T Effective date: 20220126 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220526 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220426 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220426 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220427 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220526 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602018030225 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20221027 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230518 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220126 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20221231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221210 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221210 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231124 Year of fee payment: 6 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231122 Year of fee payment: 6 Ref country code: DE Payment date: 20231121 Year of fee payment: 6 |