US20200014625A1 - Router to send a request from a first subnet to a second subnet - Google Patents
Router to send a request from a first subnet to a second subnet Download PDFInfo
- Publication number
- US20200014625A1 US20200014625A1 US16/571,609 US201916571609A US2020014625A1 US 20200014625 A1 US20200014625 A1 US 20200014625A1 US 201916571609 A US201916571609 A US 201916571609A US 2020014625 A1 US2020014625 A1 US 2020014625A1
- Authority
- US
- United States
- Prior art keywords
- request
- router
- memory address
- subnet
- target node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/74—Address processing for routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
- H04L45/04—Interdomain routing, e.g. hierarchical routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
Definitions
- a system such as a computer or other type of system, can include various types of devices.
- Such devices can include a processor, an input/output device, a memory device, and other devices.
- the processor or I/O device can issue memory access commands over a memory bus to the memory device.
- the memory device can respond by sending data to the requesting processor or I/O device over the memory bus.
- FIG. 1 is a block diagram of an example arrangement including a transparent router connected between subnets, in accordance with some implementations.
- FIG. 2 is a flow diagram of an example process performed by a transparent router according to some implementations.
- FIG. 3 is a schematic diagram of an example transaction packet according to some implementations.
- FIG. 4 is a schematic diagram illustrating an example of a translation of a source node request to a transparent router request, according to some implementations.
- FIG. 5 is a block diagram of an example transparent router according to further implementations.
- FIG. 6A is a block diagram of another example arrangement including a transparent router connected between subnets, according to further implementations.
- FIG. 6B is a block diagram of a further example arrangement that includes multiple redundant transparent routers, according to alternative implementations.
- FIG. 7 is a message flow diagram of an example process involving a source node, a transparent router, and a target node, according to further implementations.
- FIG. 8 is a block diagram of an example router according to some implementations.
- FIG. 9 is a block diagram of example machine-readable instructions stored in at least one machine-readable storage medium, according to further implementations.
- a communication fabric can support memory semantic operations.
- a “communication fabric” can refer to communication links and communication nodes used to communicate data between source and target nodes that are coupled to the communication links and communication nodes.
- a “memory semantic operation” can refer to an operation that employs memory requests to communicate between a source node and a target node.
- the source node can be a processor
- the target node can be a memory (volatile memory or non-volatile memory).
- a memory request can include a read request, a write request, an atomic operation request, or other type of request, which can be issued by the source node to access data in the target node.
- the target node responds to a memory request with a response (or multiple responses).
- a source node can include a different type of electronic component, such as a graphics controller, an input/output (I/O) controller, a network interface controller, a computer, and so forth, while a target node can also include any of the foregoing devices. More generally, a source node or target node can include an entire machine (e.g. a computer, storage system, communication system, a handheld device such as a smartphone or personal digital assistant, a game appliance, etc.), or alternatively, a source node or target node can include one or a combination of electronic components within a machine (e.g. a processor, a memory device, a persistent storage device, a graphics controller, an I/O controller, a network interface controller, etc.).
- a processor e.g. a processor, a memory device, a persistent storage device, a graphics controller, an I/O controller, a network interface controller, etc.
- a communication fabric that provides communications using memory semantic operations can be associated with a lower overhead than a communication fabric that provides communications based on use of traditional network protocols, such as the Transmission Control Protocol (TCP) and/or the Internet Protocol (IP).
- Network protocols can define formats and procedures for communication of data over communication fabrics.
- the defined formats can include protocol headers with specified fields, which can take up a relatively large amount of space and thus can lead to increased overhead.
- the larger overhead of network protocols can result in increased latency and reduced bandwidth in network communications.
- a network environment can include multiple subnets.
- a “subnet” can refer to a logical subdivision of a larger network. Different subnets can employ separate address spaces. Each subnet can include a respective group of endpoint nodes. Traditionally, use of memory semantic operations in a network environment that includes multiple subnets may not be practical.
- a router 102 is provided between subnets (e.g. subnet 1 and subnet 2 ) to support memory semantic operations between endpoint nodes located in different subnets.
- subnets e.g. subnet 1 and subnet 2
- more than two subnets may be present in a network environment that includes communication fabric(s) according to the present disclosure.
- Communications between endpoint nodes in different subnets e.g. an endpoint node 104 in subnet 1 and an endpoint node 106 in subnet 2
- the router 102 is referred to as a transparent router, which enables interconnection of multiple subnets without endpoint nodes in one subnet being aware of endpoint nodes in another subnet.
- FIG. 1 depicts just one endpoint node in each subnet, it is noted that a subnet can include multiple endpoint nodes.
- the endpoint node 104 is the node that initiates a particular transaction with a request, and the target node 106 responds to the request with a response.
- the endpoint node 104 can be a target node, while the endpoint node 106 can be a source node.
- the transparent router 102 appears as a target node in a first subnet and as a source node in a second subnet.
- the router 102 appears as the target node to the source node 104 in subnet 1
- the router 102 appears as a source node to the target node 106 in subnet 2 .
- the source node 104 submits a request to the transparent router 102 as the target
- the transparent router 102 is the source that submits the request to the target node 106 .
- the transparent router 102 maintains the illusion that an endpoint node is communicating with a local subnet endpoint node, even though a transaction initiated by a source node (e.g. source node 104 ) involves endpoint nodes in multiple subnets (e.g. source node 104 and target node 106 in subnets 1 and 2 , respectively).
- a source node e.g. source node 104
- endpoint nodes in multiple subnets e.g. source node 104 and target node 106 in subnets 1 and 2 , respectively.
- Subnet 1 includes a communication fabric 108 that allows endpoint nodes in subnet 1 to communicate with each other and with the transparent router 102 .
- subnet 2 includes a communication fabric 110 that allows endpoint nodes in subnet 2 to communicate with each other and with the transparent router 102 .
- the communication fabrics of the subnets support memory semantic operations, as discussed above.
- FIG. 2 is a flow diagram of a process performed by the transparent router 102 , according to some implementations.
- the transparent router 102 receives (at 202 ) a request including a first memory address from the source node 104 in subnet 1 .
- a memory address is used to identify a storage location in a target node, such as a memory device, a persistent storage device, or another type of target node.
- the transparent router 102 validates (at 204 ), based on information in the request, that the request corresponds to a valid operation in subnet 2 .
- the validating is based on performance of a validation procedure that includes validating an access key (if present) in the request and an operation code (OpCode) in the request.
- the validation procedure can validate other field(s) in the request, discussed further below.
- a request can include an access key field to store an access key.
- An access key can be an identifier used to provide access privileges to a specified resource (or resources) associated with a target node.
- a resource can include any or some combination of the following: a memory region of a memory device (or memory devices), a storage region of a persistent storage device (or storage devices), an electronic component or a portion of an electronic component (e.g. a processor, a switch, a programmable gate array (PGA), a digital signal processor (DSP), a general processing unit (GPU), etc.), an accelerator (which is an entity that is used to perform computations on behalf of another device), a shared I/O component (e.g. a network or storage controller, a graphics controller, etc.), an ingress or egress interface, and so forth.
- a shared I/O component e.g. a network or storage controller, a graphics controller, etc.
- any request targeting the given resource that does not contain the access key in the access key field of the request will violate resource permissions and can trigger an access violation error.
- the access key in the access key field of the request is compared to the access key associated with the given resource, to determine whether access of the given resource can be granted.
- a validation procedure determines whether the access key in the request authorizes an operation in a subnet that is the target of the request. Including the access key in the access key field provides an explicit specification of the access key. In other examples, an access key can be implicitly indicated; if a transaction packet does not contain an access key field, then the transparent router 102 can either apply a subnet-local access key or can specify that use of a default access key is permitted at the target node.
- An operation code can specify a specific operation to be performed.
- the operation code can be included in an OpCode field of a request. Examples of operations that can be specified by an operation code include a read operation, a write operation, and so forth.
- a validation procedure can validate whether the operation code included in the request is for a valid operation. In some examples, this can be based on comparing the operation code included in the request with a data structure (referred to as an “OpCode set structure”) that contains a set of supported operation codes. Any operation code that does not match the set of supported operation codes in the OpCode set structure is marked as an un-supported operation code, and the validation procedure indicates an error.
- the OpCode set structure can be in the form of a table, a list, or any other type of data structure.
- the validation procedure performed by the transparent router 102 can include other types of validations.
- the validation procedure can validate an address in the address field to ensure that the address maps to an address supported in the peer subnet.
- the transparent router 102 can perform error processing in response to the validation error, where the error processing can include taking one or multiple actions to address the error.
- the transparent router 102 translates (at 206 ) the first memory address in the request to a second memory address that is different from the first memory address.
- the first memory address is within an address space of subnet 1
- the second memory address is within an address space of subnet 2 , where the address spaces of subnets 1 and 2 are different from each other.
- the transparent router 102 sends (at 208 ) the request that includes the second memory address to the target node 106 in subnet 2 .
- the request received by the target node 106 causes the target node 106 to perform an operation (or operations) as specified by the request, where the operation(s) can include a read operation, a write operation, both read and write operations, and so forth.
- a request or a response to the request can be communicated in a transaction packet through a communication fabric of a subnet.
- a transaction packet includes specified fields.
- FIG. 3 shows an example arrangement of fields that can be included in a transaction packet 300 . Although example fields are shown in the transaction packet 300 , it is noted that in other examples, other fields or other arrangements of fields can be included.
- the transaction packet 300 includes a source component identifier (SCID) field 302 that contains an SCID.
- SCID component identifier
- nodes on a communication fabric are uniquely identified using component identifiers (CIDs). Each node on the communication fabric is assigned one or multiple CIDs.
- the source node is identified using a source SID (SCID), while the target node is identified using a destination CID (DCID).
- SCID source SID
- DCID destination CID
- the source node and the target node can be identified using other types of identifiers, such as IP addresses, TCP ports, and so forth.
- the transaction packet 300 further includes a DCID field 304 that contains the DCID of the target node.
- the transaction packet 300 includes an access key field 306 that contains an access key, and an OpCode field 308 that contains an operation code.
- an access key can be omitted, such as in scenarios where a transaction involves a resource that is not protected using an access key.
- the transaction packet 300 also includes a payload field 310 for carrying a data payload when appropriate.
- the payload field 310 of a request packet can include write data (that is to be written to the target node).
- the payload field 310 of a response packet carries read data retrieved from the target node.
- the transaction packet 300 also includes a tag field 312 to store a tag.
- a tag is an identifier that is used for associating a request with a response.
- the response can be a read response that carries the read data.
- the response can be a positive acknowledgement that the write was successfully completed.
- a tag is included in the tag field 312 of a request and also included in the response to the request, such that the request and the response can be correlated to each other.
- the combination of an SCID and a tag provides an identifier of a transaction.
- an SCID and a tag provides an identifier of a transaction.
- the transaction packet 300 can also include an address field 314 to carry a memory address, such as the first memory address or the second memory address discussed in connection with the process of FIG. 2 .
- FIG. 4 is a schematic diagram showing an example translation of fields of a request received from a source node, as performed by the transparent router 102 .
- the request received from the source node is depicted as a source node request 402 .
- the source node request 402 is translated by the transparent router 102 to a transparent router request 404 , which is sent by the transparent router 102 to a target node (e.g. 106 ) in response to the source node request 402 .
- a target node e.g. 106
- the source node request 402 includes an SCID 104 (the SCID of the source node 104 ), a DCID 102 (the DCID of the transparent router 102 ), memory address X, and tag A (the tag for the transaction initiated by the source node request 402 in subnet 1 ).
- the source node request 402 becomes the transparent router request 404 , which includes an SCID 102 (the SCID of the transparent router 102 ), a DCID 106 (the DCID of the target node 106 ), memory address Y, and tag B (the tag for the transaction in subnet 2 ).
- memory address X is translated to memory address Y
- tag A is translated to tag B.
- the transparent router 102 includes an address translation engine 502 and a transaction tracking engine 504 .
- the term “engine” can refer to a combination of machine-executable instructions and processing hardware, such as a processor (or multiple processors). Alternatively, an “engine” can refer to just processing hardware.
- a processor can include a microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, a core of a microprocessor, or any other type of processing hardware.
- the translation engine 502 performs translation of various fields of a request, while the transaction tracking engine 504 tracks a transaction to correlate a request with one or multiple responses of the transaction.
- the translation performed by the translation engine 502 is based on a memory address translation (MAT) table 506 , which includes a number of entries. Each entry of the MAT 506 maps a memory address of a first subnet to a corresponding address of a second subnet.
- the entries of the MAT 506 can be populated by an administrative entity of a network, such as a human network administrator or a network controller.
- each entry of the MAT 506 can map a memory address contained in the address field 314 of the transaction packet 300 for a request to the following information of the target subnet: [DCID, Memory Address, Access Key] (assuming that an access key is used). If an access key is not used, then the address contained in the address field 314 is translated to [DCID, Memory Address] in the target subnet. The DCID mapped from the memory address identifies the target node in the target subnet.
- the transaction tracking engine 504 adds (creates) entries 509 for the respective transactions to a transaction tracking table 508 .
- Each entry 509 of the transaction tracking table 508 can include various values, including a requester CID 510 (the CID of the requester, such as the source node 104 ), a requester tag 512 (the tag included in the request received from the requester, such as tag A in the source node request 402 of FIG. 4 ), a responder CID 514 (to identify the responder, such as the target node 106 ), and the responder tag 516 (the tag included in the transparent router request, such as tag B in the transparent router request 404 in FIG. 4 ). Also, if the request from the source node includes an access key, the access key is also added to the corresponding entry 509 in the transaction tracking table 508 .
- the transaction tracking engine 504 can correlate a response from the responder to the request received from a requester.
- the response from the responder would include the CID of the responder, along with the responder tag, which can be used to retrieve the respective entry 509 from the transaction tracking table 508 to correlate to the transaction identified by the requester CID and the requester tag.
- the combination of the requester CID and the requester tag uniquely identifies a respective transaction in subnet 1 .
- the access key is also used in correlating the response to the transaction.
- MAT 506 and transaction tracking table 508 can be implemented using other types of data structures.
- each entry 509 of the transaction tracking table 508 includes or is associated with a failsafe timer 510 .
- the failsafe timer 510 can be part of each respective entry 509 , or alternatively, the failsafe timer 510 can be separate from the entry 509 .
- the failsafe timer 510 of a respective entry 509 is started in response to the transmission of a transparent router request by the transparent router 102 to the target node. Assuming that the transparent router request is validated at the target node and the target node executes an operation specified by the transparent router request, the transparent router 102 receives a response sent by the target node. The transparent router 102 validates the received response, locates the respective entry 509 of the transaction tracking table 508 , and produces a response to be sent from the transparent router 102 to the source node. Also, the transparent router 102 resets the failsafe timer 510 .
- the failsafe timer 510 expires before the transparent router 102 receives a response to the transparent router request sent by the transparent router 102 to the target node, then the respective entry 509 of the transaction tracking table 508 is removed, and an error handling procedure can be performed.
- the error handling procedure can include sending an error message to the source node.
- FIG. 6A shows the transparent router 102 divided into two logical nodes 602 and 604 .
- the logic node 602 is part of subnet 1
- the logical node 604 is part of subnet 2 .
- the transparent router logical node 602 in subnet 1 can communicate, over a communication fabric 607 that supports memory semantic operations, with endpoint nodes 606 in subnet 1
- the transparent router logical node 604 in subnet 2 can communicate, over a communication fabric 609 that supports memory semantic operations, with endpoint nodes 608 in subnet 2 .
- a subnet 1 MAT 610 and subnet 1 transaction tracking table 612 are used for requests originated in subnet 1
- a subnet 2 MAT 614 and subnet 2 transaction tracking table 616 are used for requests originated in subnet 2 .
- FIG. 6B shows the use of redundant transparent routers TR_A and TR_A′ to provide resiliency in case of failure of one of the redundant transparent routers.
- Subnet 1 in FIG. 6B includes a processor 620 that can access a logical memory component 624 (that includes the redundant transparent routers TR_A and TR_A′).
- the logical memory component 624 can be identified by a single component identifier in subnet 1 .
- a memory request issued by the processor 620 to the logical memory component 624 over a communication fabric 622 is translated by a transparent router (TR_A or TR_A′, depending on which is the primary transparent router and which is the standby transparent router) into a translated memory request that is issued to a physical memory component (or components) in subnet 2 .
- TR_A or TR_A′ depending on which is the primary transparent router and which is the standby transparent router
- transparent router TR_A is the primary transparent router
- failure of the transparent router TR_A will cause the processor 620 to be notified of an event that causes the processor 620 to initiate transaction recovery.
- the transaction recovery performed by the processor 620 can include the processor 620 retransmitting outstanding requests on an alternative path that leads to the transparent router TR_A′.
- the transparent router TR_A′ then performs the translation/correlation tasks discussed above for the requests sent by the processor 620 .
- the processor 620 has multiple redundant paths to the logical memory component 624 , which provides resiliency while still maintaining transparency of components in subnet 2 to the processor 620 .
- the logical memory component 624 can be implemented with a single hardware component, or multiple hardware components that include the respective transparent routers TR_A and TR_A′.
- FIG. 7 is a message flow diagram that shows the communication of requests and responses according to some implementations of the present disclosure.
- the source node 104 transmits (at 702 ) a request REQ 0 to the transparent router 102 , which translates (at 704 ) field values (e.g. memory address, tag) of the received request REQ 0 to respective field values to be included in a request TR_REQ 0 .
- field values e.g. memory address, tag
- the transparent router 102 In response to the request REQ 0 , the transparent router 102 also adds (at 706 ) an entry to a transaction tracking table (e.g. 508 in FIG. 5 ). The transparent router 102 sends (at 708 ) the request TR_REQ 0 to the target node 106 . In response to the request TR_REQ 0 , the target node 106 performs the requested operation(s).
- a transaction tracking table e.g. 508 in FIG. 5
- the transparent router 102 sends (at 708 ) the request TR_REQ 0 to the target node 106 .
- the target node 106 performs the requested operation(s).
- the target node 106 sends (at 712 ) a response TR_RSP 0 to the transparent router 102 , where the response TR_RSP 0 relates to the operation(s) performed (at 710 ).
- the response TR_RSP 0 can include read data or a positive acknowledgment of a write operation.
- the transparent router 102 retrieves a corresponding entry of the transaction tracking table to correlate the response TR_RSP 0 to the respective transaction corresponding to the request REQ 0 .
- the transparent router 102 then produces (at 716 ) a response RSP 0 corresponding to TR_RSP 0 , and the transparent router 102 sends (at 718 ) the response RSP 0 back to the source node 104 .
- the transparent router 102 does not have to store the received source node request (REQ 0 ), which reduces resource consumption at the transparent router 102 .
- the transparent router 102 is able to support advanced transactions.
- a specific address range or specific address ranges can indicate an advanced transaction.
- other types of indicators of advanced transactions can be used.
- One type of advanced transaction is a transaction in which the transparent router 102 transforms a single operation into multiple operations (e.g. multiple smaller operations). For example, a read request to read data of a particular size is transformed into multiple smaller read requests to read data of smaller sizes, which can be performed by multiple target nodes in parallel to enhance parallelism.
- Another type of advanced transaction is a transaction in which a given request received from a source node is translated to operations performed by multiple different types of target nodes.
- the transparent router 102 can initiate the performance of the advanced transaction by sending a first request to a first target node to perform a first operation. This first operation can produce result data that is sent from the first target node back to the transparent router 102 .
- the transparent router 102 can then send a second request to a second target node, where the second request can include the result data from the first operation performed by the first target node.
- the second request specifies a second operation at the second target node.
- the first target node can be an accelerator to perform a specific computation.
- An accelerator is a component that is used to perform computations on behalf of another component. Examples of computations that can be performed by the accelerator include deduplication (to deduplicate data, in which identical copies of data are identified and only one of the identified identical copies is stored), malware detection (to detect malware, such as a virus, a worm, or any unauthorized code), data analysis (to perform analysis of data), an auditing service (to verify if a requester has permission to perform an operation, to create an audit trail, etc.), mathematical calculations (such as computing a running total or running average), or other computations.
- deduplication to deduplicate data, in which identical copies of data are identified and only one of the identified identical copies is stored
- malware detection to detect malware, such as a virus, a worm, or any unauthorized code
- data analysis to perform analysis of data
- an auditing service to verify if a requester has permission to perform an operation, to create an audit trail, etc.
- Result data produced by the computation performed at the accelerator can then be sent to the second target node, such as a memory, for storage.
- use of a transparent router can also allow for support of more robust servicing or management of components in a network. For example, components in a given subnet can be serviced or replaced without downtime of an application running in components of another subnet. Also, components can be power managed to conserve power by comprehending their access patterns and adjusting power accordingly. In addition, adding components to a subnet (or adding a new subnet) can enable the provisioning of additional components to processors or applications. In addition, transparent migration of memory components between different tiers of memory is possible.
- FIG. 8 is a block diagram of a router 800 (e.g. a transparent router as discussed above) according to some implementations.
- the router includes one or multiple processors 802 .
- the processor(s) 802 can execute various machine-executable instructions, including request receiving instructions 804 (to receive a request including a first memory address from a source node in a first subnet), request determining instructions 806 (to determine whether the request corresponds to a transaction involving multiple target nodes), first request sending instructions 808 (to send a first request to a target node to perform a first operation in response to determining that the request corresponds to a transaction involving multiple target nodes), translation instructions 810 (to translate the first memory address in the received request to a second memory address that is different from the first memory address), and second request sending instructions 812 (to send a second request including the second memory address to a second target node).
- request receiving instructions 804 to receive a request including a first memory address from a source node in a first subnet
- FIG. 9 is a block diagram of a non-transitory computer-readable or machine-readable storage medium (or storage media) 902 according to some implementations.
- the storage medium (or storage media) 902 can be part of a transparent router as discussed above.
- the storage medium (or storage media) 902 can include a removable storage medium (or storage media) that is removably connectable to the transparent router.
- the storage medium (or storage media) 902 can be at a remote site (e.g. remote server 0 that is accessible by the transparent router to download the machine-executable instructions of the storage medium (or storage media) 902 from the remote site to the transparent router.
- a remote site e.g. remote server 0 that is accessible by the transparent router to download the machine-executable instructions of the storage medium (or storage media) 902 from the remote site to the transparent router.
- the storage medium (or storage media) 902 can store request receiving instructions 904 (to receive a request including a first memory address from a source node in a first subnet), validation instructions 906 (to perform a validation procedure), translation instructions 908 (to translate the first memory address in the request to a second memory address that is different from the first memory address), and a request sending instructions 910 (to send the request including the second memory address to a target node in the second subnet).
- the machine-executable instructions stored in the storage medium (or storage media) 902 are executable by at least one processor, such as at least one processor in the router 102 , to perform respective tasks.
- the validation instructions 906 include access key determining instructions 912 to determine that an access key associated with the received request authorizes an operation in the second subnet requested by the received request, and operation code determining instructions 914 to determine that the received request has a valid operation code.
- Data and instructions are stored in respective storage devices, which are implemented as one or multiple computer-readable or machine-readable storage media.
- the storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices.
- DRAMs or SRAMs dynamic or static random access memories
- EPROMs erasable and programmable read-only memories
- EEPROMs electrically erasable and programmable read-only memories
- flash memories such as fixed, floppy and removable disks
- magnetic media such as fixed, floppy and removable disks
- optical media such as compact disks (CDs) or digital video disks (DVDs); or other
- the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes.
- Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture).
- An article or article of manufacture can refer to any manufactured single component or multiple components.
- the storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- A system, such as a computer or other type of system, can include various types of devices. Such devices can include a processor, an input/output device, a memory device, and other devices. The processor or I/O device can issue memory access commands over a memory bus to the memory device. The memory device can respond by sending data to the requesting processor or I/O device over the memory bus.
- Some implementations are described with respect to the following figures.
-
FIG. 1 is a block diagram of an example arrangement including a transparent router connected between subnets, in accordance with some implementations. -
FIG. 2 is a flow diagram of an example process performed by a transparent router according to some implementations. -
FIG. 3 is a schematic diagram of an example transaction packet according to some implementations. -
FIG. 4 is a schematic diagram illustrating an example of a translation of a source node request to a transparent router request, according to some implementations. -
FIG. 5 is a block diagram of an example transparent router according to further implementations. -
FIG. 6A is a block diagram of another example arrangement including a transparent router connected between subnets, according to further implementations. -
FIG. 6B is a block diagram of a further example arrangement that includes multiple redundant transparent routers, according to alternative implementations. -
FIG. 7 is a message flow diagram of an example process involving a source node, a transparent router, and a target node, according to further implementations. -
FIG. 8 is a block diagram of an example router according to some implementations. -
FIG. 9 is a block diagram of example machine-readable instructions stored in at least one machine-readable storage medium, according to further implementations. - To achieve relatively low latency and high bandwidth, a communication fabric can support memory semantic operations. A “communication fabric” can refer to communication links and communication nodes used to communicate data between source and target nodes that are coupled to the communication links and communication nodes.
- A “memory semantic operation” can refer to an operation that employs memory requests to communicate between a source node and a target node. For example, the source node can be a processor, while the target node can be a memory (volatile memory or non-volatile memory). A memory request can include a read request, a write request, an atomic operation request, or other type of request, which can be issued by the source node to access data in the target node. The target node responds to a memory request with a response (or multiple responses).
- In other examples, a source node can include a different type of electronic component, such as a graphics controller, an input/output (I/O) controller, a network interface controller, a computer, and so forth, while a target node can also include any of the foregoing devices. More generally, a source node or target node can include an entire machine (e.g. a computer, storage system, communication system, a handheld device such as a smartphone or personal digital assistant, a game appliance, etc.), or alternatively, a source node or target node can include one or a combination of electronic components within a machine (e.g. a processor, a memory device, a persistent storage device, a graphics controller, an I/O controller, a network interface controller, etc.).
- A communication fabric that provides communications using memory semantic operations can be associated with a lower overhead than a communication fabric that provides communications based on use of traditional network protocols, such as the Transmission Control Protocol (TCP) and/or the Internet Protocol (IP). Network protocols can define formats and procedures for communication of data over communication fabrics. The defined formats can include protocol headers with specified fields, which can take up a relatively large amount of space and thus can lead to increased overhead. The larger overhead of network protocols can result in increased latency and reduced bandwidth in network communications.
- A network environment can include multiple subnets. A “subnet” can refer to a logical subdivision of a larger network. Different subnets can employ separate address spaces. Each subnet can include a respective group of endpoint nodes. Traditionally, use of memory semantic operations in a network environment that includes multiple subnets may not be practical.
- In accordance with some implementations of the present disclosure, as shown in
FIG. 1 , arouter 102 is provided between subnets (e.g. subnet 1 and subnet 2) to support memory semantic operations between endpoint nodes located in different subnets. Although just two subnets are depicted inFIG. 1 , it is noted that in other examples, more than two subnets may be present in a network environment that includes communication fabric(s) according to the present disclosure. Communications between endpoint nodes in different subnets (e.g. anendpoint node 104 insubnet 1 and anendpoint node 106 in subnet 2) can pass through therouter 102. In some examples, therouter 102 is referred to as a transparent router, which enables interconnection of multiple subnets without endpoint nodes in one subnet being aware of endpoint nodes in another subnet. - Although
FIG. 1 depicts just one endpoint node in each subnet, it is noted that a subnet can include multiple endpoint nodes. In the ensuing discussion, reference is made to theendpoint node 104 as being a “source node,” and theendpoint node 106 as being a “target node.” Thesource node 104 is the node that initiates a particular transaction with a request, and thetarget node 106 responds to the request with a response. Note that for other transactions, theendpoint node 104 can be a target node, while theendpoint node 106 can be a source node. - For a given transaction, the
transparent router 102 appears as a target node in a first subnet and as a source node in a second subnet. In the example ofFIG. 1 , therouter 102 appears as the target node to thesource node 104 insubnet 1, while therouter 102 appears as a source node to thetarget node 106 insubnet 2. For the given transaction, thesource node 104 submits a request to thetransparent router 102 as the target, while for the same given transaction, thetransparent router 102 is the source that submits the request to thetarget node 106. - The
transparent router 102 maintains the illusion that an endpoint node is communicating with a local subnet endpoint node, even though a transaction initiated by a source node (e.g. source node 104) involves endpoint nodes in multiple subnets (e.g. source node 104 andtarget node 106 insubnets -
Subnet 1 includes acommunication fabric 108 that allows endpoint nodes insubnet 1 to communicate with each other and with thetransparent router 102. Similarly,subnet 2 includes acommunication fabric 110 that allows endpoint nodes insubnet 2 to communicate with each other and with thetransparent router 102. The communication fabrics of the subnets support memory semantic operations, as discussed above. -
FIG. 2 is a flow diagram of a process performed by thetransparent router 102, according to some implementations. Thetransparent router 102 receives (at 202) a request including a first memory address from thesource node 104 insubnet 1. A memory address is used to identify a storage location in a target node, such as a memory device, a persistent storage device, or another type of target node. - The
transparent router 102 validates (at 204), based on information in the request, that the request corresponds to a valid operation insubnet 2. The validating is based on performance of a validation procedure that includes validating an access key (if present) in the request and an operation code (OpCode) in the request. In other examples, the validation procedure can validate other field(s) in the request, discussed further below. - A request can include an access key field to store an access key. An access key can be an identifier used to provide access privileges to a specified resource (or resources) associated with a target node. A resource can include any or some combination of the following: a memory region of a memory device (or memory devices), a storage region of a persistent storage device (or storage devices), an electronic component or a portion of an electronic component (e.g. a processor, a switch, a programmable gate array (PGA), a digital signal processor (DSP), a general processing unit (GPU), etc.), an accelerator (which is an entity that is used to perform computations on behalf of another device), a shared I/O component (e.g. a network or storage controller, a graphics controller, etc.), an ingress or egress interface, and so forth.
- If a given resource is associated with an access key, then any request targeting the given resource that does not contain the access key in the access key field of the request will violate resource permissions and can trigger an access violation error. If an access key is included in the access key field of the request, the access key in the access key field of the request is compared to the access key associated with the given resource, to determine whether access of the given resource can be granted. Thus, using the access key in the request, a validation procedure determines whether the access key in the request authorizes an operation in a subnet that is the target of the request. Including the access key in the access key field provides an explicit specification of the access key. In other examples, an access key can be implicitly indicated; if a transaction packet does not contain an access key field, then the
transparent router 102 can either apply a subnet-local access key or can specify that use of a default access key is permitted at the target node. - An operation code (or OpCode) can specify a specific operation to be performed. The operation code can be included in an OpCode field of a request. Examples of operations that can be specified by an operation code include a read operation, a write operation, and so forth.
- A validation procedure can validate whether the operation code included in the request is for a valid operation. In some examples, this can be based on comparing the operation code included in the request with a data structure (referred to as an “OpCode set structure”) that contains a set of supported operation codes. Any operation code that does not match the set of supported operation codes in the OpCode set structure is marked as an un-supported operation code, and the validation procedure indicates an error. The OpCode set structure can be in the form of a table, a list, or any other type of data structure.
- The validation procedure performed by the
transparent router 102 can include other types of validations. For example, the validation procedure can validate an address in the address field to ensure that the address maps to an address supported in the peer subnet. There are other fields that can be validated as part of transaction validation, such as data integrity fields, etc. - If the validation procedure produces an error, the
transparent router 102 can perform error processing in response to the validation error, where the error processing can include taking one or multiple actions to address the error. - The
transparent router 102 translates (at 206) the first memory address in the request to a second memory address that is different from the first memory address. The first memory address is within an address space ofsubnet 1, while the second memory address is within an address space ofsubnet 2, where the address spaces ofsubnets - The
transparent router 102 sends (at 208) the request that includes the second memory address to thetarget node 106 insubnet 2. The request received by thetarget node 106 causes thetarget node 106 to perform an operation (or operations) as specified by the request, where the operation(s) can include a read operation, a write operation, both read and write operations, and so forth. - A request or a response to the request can be communicated in a transaction packet through a communication fabric of a subnet. A transaction packet includes specified fields.
FIG. 3 shows an example arrangement of fields that can be included in atransaction packet 300. Although example fields are shown in thetransaction packet 300, it is noted that in other examples, other fields or other arrangements of fields can be included. - The
transaction packet 300 includes a source component identifier (SCID)field 302 that contains an SCID. In some implementations, nodes on a communication fabric are uniquely identified using component identifiers (CIDs). Each node on the communication fabric is assigned one or multiple CIDs. In a given transaction, the source node is identified using a source SID (SCID), while the target node is identified using a destination CID (DCID). In other examples, the source node and the target node can be identified using other types of identifiers, such as IP addresses, TCP ports, and so forth. - The
transaction packet 300 further includes aDCID field 304 that contains the DCID of the target node. In addition, thetransaction packet 300 includes an accesskey field 306 that contains an access key, and anOpCode field 308 that contains an operation code. In some cases, an access key can be omitted, such as in scenarios where a transaction involves a resource that is not protected using an access key. - The
transaction packet 300 also includes apayload field 310 for carrying a data payload when appropriate. For a write transaction, thepayload field 310 of a request packet can include write data (that is to be written to the target node). For a read transaction, thepayload field 310 of a response packet carries read data retrieved from the target node. - The
transaction packet 300 also includes atag field 312 to store a tag. A tag is an identifier that is used for associating a request with a response. For a read transaction, the response can be a read response that carries the read data. For a write transaction, the response can be a positive acknowledgement that the write was successfully completed. A tag is included in thetag field 312 of a request and also included in the response to the request, such that the request and the response can be correlated to each other. - In some implementations, the combination of an SCID and a tag provides an identifier of a transaction. Thus, within a subnet, there can only be one outstanding request with a given [SCID, Tag] combination.
- The
transaction packet 300 can also include anaddress field 314 to carry a memory address, such as the first memory address or the second memory address discussed in connection with the process ofFIG. 2 . -
FIG. 4 is a schematic diagram showing an example translation of fields of a request received from a source node, as performed by thetransparent router 102. The request received from the source node is depicted as asource node request 402. Thesource node request 402 is translated by thetransparent router 102 to atransparent router request 404, which is sent by thetransparent router 102 to a target node (e.g. 106) in response to thesource node request 402. - The
source node request 402 includes an SCID 104 (the SCID of the source node 104), a DCID 102 (the DCID of the transparent router 102), memory address X, and tag A (the tag for the transaction initiated by thesource node request 402 in subnet 1). After translation by thetransparent router 102, thesource node request 402 becomes thetransparent router request 404, which includes an SCID 102 (the SCID of the transparent router 102), a DCID 106 (the DCID of the target node 106), memory address Y, and tag B (the tag for the transaction in subnet 2). - In the translation performed according to
FIG. 4 , memory address X is translated to memory address Y, and tag A is translated to tag B. - As shown in
FIG. 5 , in accordance with some implementations, thetransparent router 102 includes anaddress translation engine 502 and atransaction tracking engine 504. As used here, the term “engine” can refer to a combination of machine-executable instructions and processing hardware, such as a processor (or multiple processors). Alternatively, an “engine” can refer to just processing hardware. A processor can include a microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, a core of a microprocessor, or any other type of processing hardware. - The
translation engine 502 performs translation of various fields of a request, while thetransaction tracking engine 504 tracks a transaction to correlate a request with one or multiple responses of the transaction. The translation performed by thetranslation engine 502 is based on a memory address translation (MAT) table 506, which includes a number of entries. Each entry of theMAT 506 maps a memory address of a first subnet to a corresponding address of a second subnet. The entries of theMAT 506 can be populated by an administrative entity of a network, such as a human network administrator or a network controller. - More specifically, in some examples, each entry of the
MAT 506 can map a memory address contained in theaddress field 314 of thetransaction packet 300 for a request to the following information of the target subnet: [DCID, Memory Address, Access Key] (assuming that an access key is used). If an access key is not used, then the address contained in theaddress field 314 is translated to [DCID, Memory Address] in the target subnet. The DCID mapped from the memory address identifies the target node in the target subnet. - As requests for respective transactions are received, the
transaction tracking engine 504 adds (creates)entries 509 for the respective transactions to a transaction tracking table 508. Eachentry 509 of the transaction tracking table 508 can include various values, including a requester CID 510 (the CID of the requester, such as the source node 104), a requester tag 512 (the tag included in the request received from the requester, such as tag A in thesource node request 402 ofFIG. 4 ), a responder CID 514 (to identify the responder, such as the target node 106), and the responder tag 516 (the tag included in the transparent router request, such as tag B in thetransparent router request 404 inFIG. 4 ). Also, if the request from the source node includes an access key, the access key is also added to thecorresponding entry 509 in the transaction tracking table 508. - Using a given transaction tracking
table entry 509, thetransaction tracking engine 504 can correlate a response from the responder to the request received from a requester. The response from the responder would include the CID of the responder, along with the responder tag, which can be used to retrieve therespective entry 509 from the transaction tracking table 508 to correlate to the transaction identified by the requester CID and the requester tag. Note that in subnet 1 (the subnet that the source node is part of), the combination of the requester CID and the requester tag uniquely identifies a respective transaction insubnet 1. - If the given transaction tracking
table entry 509 also includes an access key, then the access key is also used in correlating the response to the transaction. - Although reference is made to tables (506 and 508 in
FIG. 5 ), it is noted that in other examples, theMAT 506 and transaction tracking table 508 can be implemented using other types of data structures. - In some examples, as further shown in
FIG. 5 , eachentry 509 of the transaction tracking table 508 includes or is associated with a failsafe timer 510. The failsafe timer 510 can be part of eachrespective entry 509, or alternatively, the failsafe timer 510 can be separate from theentry 509. - The failsafe timer 510 of a
respective entry 509 is started in response to the transmission of a transparent router request by thetransparent router 102 to the target node. Assuming that the transparent router request is validated at the target node and the target node executes an operation specified by the transparent router request, thetransparent router 102 receives a response sent by the target node. Thetransparent router 102 validates the received response, locates therespective entry 509 of the transaction tracking table 508, and produces a response to be sent from thetransparent router 102 to the source node. Also, thetransparent router 102 resets the failsafe timer 510. - If the failsafe timer 510 expires before the
transparent router 102 receives a response to the transparent router request sent by thetransparent router 102 to the target node, then therespective entry 509 of the transaction tracking table 508 is removed, and an error handling procedure can be performed. The error handling procedure can include sending an error message to the source node. By using the failsafe timer 510, stranded resources in the transaction tracking table can be avoided. -
FIG. 6A shows thetransparent router 102 divided into twological nodes logic node 602 is part ofsubnet 1, while thelogical node 604 is part ofsubnet 2. The transparent routerlogical node 602 insubnet 1 can communicate, over acommunication fabric 607 that supports memory semantic operations, withendpoint nodes 606 insubnet 1, while the transparent routerlogical node 604 insubnet 2 can communicate, over acommunication fabric 609 that supports memory semantic operations, withendpoint nodes 608 insubnet 2. - For any transaction involving endpoint nodes in
subnets endpoint node 606 - As further shown in
FIG. 6A , two sets of MATs and transaction tracking tables (TTTs) are included in thetransparent router 102. Asubnet 1MAT 610 andsubnet 1 transaction tracking table 612 are used for requests originated insubnet 1, while asubnet 2MAT 614 andsubnet 2 transaction tracking table 616 are used for requests originated insubnet 2. -
FIG. 6B shows the use of redundant transparent routers TR_A and TR_A′ to provide resiliency in case of failure of one of the redundant transparent routers.Subnet 1 inFIG. 6B includes aprocessor 620 that can access a logical memory component 624 (that includes the redundant transparent routers TR_A and TR_A′). Thelogical memory component 624 can be identified by a single component identifier insubnet 1. A memory request issued by theprocessor 620 to thelogical memory component 624 over acommunication fabric 622 is translated by a transparent router (TR_A or TR_A′, depending on which is the primary transparent router and which is the standby transparent router) into a translated memory request that is issued to a physical memory component (or components) insubnet 2. - Assuming that transparent router TR_A is the primary transparent router, then failure of the transparent router TR_A will cause the
processor 620 to be notified of an event that causes theprocessor 620 to initiate transaction recovery. In some examples, the transaction recovery performed by theprocessor 620 can include theprocessor 620 retransmitting outstanding requests on an alternative path that leads to the transparent router TR_A′. The transparent router TR_A′ then performs the translation/correlation tasks discussed above for the requests sent by theprocessor 620. - According to the arrangement of
FIG. 6B , theprocessor 620 has multiple redundant paths to thelogical memory component 624, which provides resiliency while still maintaining transparency of components insubnet 2 to theprocessor 620. - The
logical memory component 624 can be implemented with a single hardware component, or multiple hardware components that include the respective transparent routers TR_A and TR_A′. -
FIG. 7 is a message flow diagram that shows the communication of requests and responses according to some implementations of the present disclosure. Thesource node 104 transmits (at 702) a request REQ0 to thetransparent router 102, which translates (at 704) field values (e.g. memory address, tag) of the received request REQ0 to respective field values to be included in a request TR_REQ0. - In response to the request REQ0, the
transparent router 102 also adds (at 706) an entry to a transaction tracking table (e.g. 508 inFIG. 5 ). Thetransparent router 102 sends (at 708) the request TR_REQ0 to thetarget node 106. In response to the request TR_REQ0, thetarget node 106 performs the requested operation(s). - The
target node 106 sends (at 712) a response TR_RSP0 to thetransparent router 102, where the response TR_RSP0 relates to the operation(s) performed (at 710). For example, the response TR_RSP0 can include read data or a positive acknowledgment of a write operation. - Using a CID and tag included in the response TR_RSP0, the
transparent router 102 retrieves a corresponding entry of the transaction tracking table to correlate the response TR_RSP0 to the respective transaction corresponding to the request REQ0. - The
transparent router 102 then produces (at 716) a response RSP0 corresponding to TR_RSP0, and thetransparent router 102 sends (at 718) the response RSP0 back to thesource node 104. - Note that the
transparent router 102 does not have to store the received source node request (REQ0), which reduces resource consumption at thetransparent router 102. - In further implementations, the
transparent router 102 is able to support advanced transactions. In some examples, a specific address range or specific address ranges can indicate an advanced transaction. In other examples, other types of indicators of advanced transactions can be used. - One type of advanced transaction is a transaction in which the
transparent router 102 transforms a single operation into multiple operations (e.g. multiple smaller operations). For example, a read request to read data of a particular size is transformed into multiple smaller read requests to read data of smaller sizes, which can be performed by multiple target nodes in parallel to enhance parallelism. - Another type of advanced transaction is a transaction in which a given request received from a source node is translated to operations performed by multiple different types of target nodes. For example, in response to the given request, the
transparent router 102 can initiate the performance of the advanced transaction by sending a first request to a first target node to perform a first operation. This first operation can produce result data that is sent from the first target node back to thetransparent router 102. Thetransparent router 102 can then send a second request to a second target node, where the second request can include the result data from the first operation performed by the first target node. The second request specifies a second operation at the second target node. - In some examples, the first target node can be an accelerator to perform a specific computation. An accelerator is a component that is used to perform computations on behalf of another component. Examples of computations that can be performed by the accelerator include deduplication (to deduplicate data, in which identical copies of data are identified and only one of the identified identical copies is stored), malware detection (to detect malware, such as a virus, a worm, or any unauthorized code), data analysis (to perform analysis of data), an auditing service (to verify if a requester has permission to perform an operation, to create an audit trail, etc.), mathematical calculations (such as computing a running total or running average), or other computations.
- Result data produced by the computation performed at the accelerator can then be sent to the second target node, such as a memory, for storage.
- In some implementations, use of a transparent router can also allow for support of more robust servicing or management of components in a network. For example, components in a given subnet can be serviced or replaced without downtime of an application running in components of another subnet. Also, components can be power managed to conserve power by comprehending their access patterns and adjusting power accordingly. In addition, adding components to a subnet (or adding a new subnet) can enable the provisioning of additional components to processors or applications. In addition, transparent migration of memory components between different tiers of memory is possible.
-
FIG. 8 is a block diagram of a router 800 (e.g. a transparent router as discussed above) according to some implementations. The router includes one ormultiple processors 802. The processor(s) 802 can execute various machine-executable instructions, including request receiving instructions 804 (to receive a request including a first memory address from a source node in a first subnet), request determining instructions 806 (to determine whether the request corresponds to a transaction involving multiple target nodes), first request sending instructions 808 (to send a first request to a target node to perform a first operation in response to determining that the request corresponds to a transaction involving multiple target nodes), translation instructions 810 (to translate the first memory address in the received request to a second memory address that is different from the first memory address), and second request sending instructions 812 (to send a second request including the second memory address to a second target node). -
FIG. 9 is a block diagram of a non-transitory computer-readable or machine-readable storage medium (or storage media) 902 according to some implementations. The storage medium (or storage media) 902 can be part of a transparent router as discussed above. Alternatively, the storage medium (or storage media) 902 can include a removable storage medium (or storage media) that is removably connectable to the transparent router. As yet another example, the storage medium (or storage media) 902 can be at a remote site (e.g. remote server0 that is accessible by the transparent router to download the machine-executable instructions of the storage medium (or storage media) 902 from the remote site to the transparent router. - The storage medium (or storage media) 902 can store request receiving instructions 904 (to receive a request including a first memory address from a source node in a first subnet), validation instructions 906 (to perform a validation procedure), translation instructions 908 (to translate the first memory address in the request to a second memory address that is different from the first memory address), and a request sending instructions 910 (to send the request including the second memory address to a target node in the second subnet). The machine-executable instructions stored in the storage medium (or storage media) 902 are executable by at least one processor, such as at least one processor in the
router 102, to perform respective tasks. - The
validation instructions 906 include accesskey determining instructions 912 to determine that an access key associated with the received request authorizes an operation in the second subnet requested by the received request, and operationcode determining instructions 914 to determine that the received request has a valid operation code. - Data and instructions are stored in respective storage devices, which are implemented as one or multiple computer-readable or machine-readable storage media. The storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.
- In the foregoing description, numerous details are set forth to provide an understanding of the subject disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the appended claims cover such modifications and variations.
Claims (15)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/571,609 US20200014625A1 (en) | 2015-01-22 | 2019-09-16 | Router to send a request from a first subnet to a second subnet |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2015/012344 WO2016118128A1 (en) | 2015-01-22 | 2015-01-22 | Router to send a request from a first subnet to a second subnet |
US201715500078A | 2017-01-30 | 2017-01-30 | |
US16/571,609 US20200014625A1 (en) | 2015-01-22 | 2019-09-16 | Router to send a request from a first subnet to a second subnet |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2015/012344 Continuation WO2016118128A1 (en) | 2015-01-22 | 2015-01-22 | Router to send a request from a first subnet to a second subnet |
US15/500,078 Continuation US10419339B2 (en) | 2015-01-22 | 2015-01-22 | Router to send a request from a first subnet to a second subnet |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200014625A1 true US20200014625A1 (en) | 2020-01-09 |
Family
ID=56417507
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/500,078 Active 2035-02-15 US10419339B2 (en) | 2015-01-22 | 2015-01-22 | Router to send a request from a first subnet to a second subnet |
US16/571,609 Abandoned US20200014625A1 (en) | 2015-01-22 | 2019-09-16 | Router to send a request from a first subnet to a second subnet |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/500,078 Active 2035-02-15 US10419339B2 (en) | 2015-01-22 | 2015-01-22 | Router to send a request from a first subnet to a second subnet |
Country Status (2)
Country | Link |
---|---|
US (2) | US10419339B2 (en) |
WO (1) | WO2016118128A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11734430B2 (en) | 2016-04-22 | 2023-08-22 | Hewlett Packard Enterprise Development Lp | Configuration of a memory controller for copy-on-write with a resource controller |
US10817176B2 (en) * | 2018-06-18 | 2020-10-27 | Intel Corporation | Compute offload in block storage using virtual objects |
US12021838B2 (en) * | 2021-11-28 | 2024-06-25 | Uab 360 It | Authentication procedure in a virtual private network |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7010633B2 (en) * | 2003-04-10 | 2006-03-07 | International Business Machines Corporation | Apparatus, system and method for controlling access to facilities based on usage classes |
US7334071B2 (en) | 2005-05-25 | 2008-02-19 | Integrated Device Technology, Inc. | Expansion of cross-domain addressing for PCI-express packets passing through non-transparent bridge |
US7937435B2 (en) * | 2006-02-21 | 2011-05-03 | Strangeloop Networks, Inc. | Identifying, storing, and retrieving context data for a network message |
US20100299421A1 (en) * | 2007-10-04 | 2010-11-25 | Robby Gurdan | Digital multimedia network with hierarchical parameter control protocol |
GB2458154B (en) * | 2008-03-07 | 2012-06-27 | Hewlett Packard Development Co | Routing across a virtual network |
US8713128B2 (en) * | 2009-07-24 | 2014-04-29 | Broadcom Corporation | Method and system for utilizing native ethernet as a virtual memory interconnect |
US8521941B2 (en) | 2010-12-28 | 2013-08-27 | Plx Technology, Inc. | Multi-root sharing of single-root input/output virtualization |
US20140149729A1 (en) * | 2011-07-18 | 2014-05-29 | Ted A. Hadley | Reset vectors for boot instructions |
EP2605481A1 (en) * | 2011-12-13 | 2013-06-19 | Siemens Aktiengesellschaft | Device and method for filtering network traffic |
US8793411B1 (en) | 2013-03-05 | 2014-07-29 | Apple Inc. | Bridge circuit reorder buffer for transaction modification and translation |
US9223737B1 (en) * | 2013-03-14 | 2015-12-29 | Google Inc. | Computer interconnect isolation |
US20150261709A1 (en) * | 2014-03-14 | 2015-09-17 | Emilio Billi | Peripheral component interconnect express (pcie) distributed non- transparent bridging designed for scalability,networking and io sharing enabling the creation of complex architectures. |
US9693219B2 (en) * | 2014-10-24 | 2017-06-27 | Ibasis, Inc. | User profile conversion to support roaming |
-
2015
- 2015-01-22 US US15/500,078 patent/US10419339B2/en active Active
- 2015-01-22 WO PCT/US2015/012344 patent/WO2016118128A1/en active Application Filing
-
2019
- 2019-09-16 US US16/571,609 patent/US20200014625A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
US20170250909A1 (en) | 2017-08-31 |
WO2016118128A1 (en) | 2016-07-28 |
US10419339B2 (en) | 2019-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10055561B2 (en) | Identity risk score generation and implementation | |
US10803027B1 (en) | Method and system for managing file system access and interaction | |
US20200014625A1 (en) | Router to send a request from a first subnet to a second subnet | |
EP3543853A1 (en) | Providing microservice information | |
EP3005648B1 (en) | Terminal identification method, and method, system and apparatus of registering machine identification code | |
US20130326494A1 (en) | System and method for distributed patch management | |
US20150046971A1 (en) | Method and system for access control in cloud computing service | |
US10171463B1 (en) | Secure transport layer authentication of network traffic | |
US11552953B1 (en) | Identity-based authentication and access control mechanism | |
US9654483B1 (en) | Network communication rate limiter | |
US20100235539A1 (en) | System and method for reduced cloud ip address utilization | |
US11252190B1 (en) | Limited access policy bypass | |
CN108073823B (en) | Data processing method, device and system | |
KR20140064850A (en) | Clustered client failover | |
US10270759B1 (en) | Fine grained container security | |
US20190373016A1 (en) | Providing cross site request forgery protection at an edge server | |
Song et al. | DS‐ARP: A New Detection Scheme for ARP Spoofing Attacks Based on Routing Trace for Ubiquitous Environments | |
US10031863B2 (en) | Access controlled memory region | |
US10491513B2 (en) | Verifying packet tags in software defined networks | |
CN113315706A (en) | Private cloud flow control method, device and system | |
JP7452813B2 (en) | Techniques for accelerated hierarchical key caching in edge systems | |
US10924452B1 (en) | Auditing IP address assignments | |
US20140007197A1 (en) | Delegation within a computing environment | |
WO2016172862A1 (en) | Memory management method, device and system | |
US7933962B1 (en) | Reducing reliance on a central data store while maintaining idempotency in a multi-client, multi-server environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KRAUSE, MICHAEL R.;REEL/FRAME:051489/0262 Effective date: 20150121 |
|
AS | Assignment |
Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.;REEL/FRAME:051737/0762 Effective date: 20151027 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |