JP5175200B2 - Apparatus and method used in a system for transmitting and receiving resource command messages - Google Patents

Description

  The present invention relates to the field of computing resource command messaging and resource devices.

  The resource device manages computing resources. There are countless examples of such computing resources ranging from data storage on disk drives constituting a storage array to connection processes in a large-capacity content server included in a server farm. Usually, since a resource device includes a plurality of resource nodes, it constitutes a distributed resource that is displayed as the only logical resource device from the viewpoint of a resource consumer.

  The resource node responsible for managing the physical resource needs to function efficiently regardless of whether the resource node manages a data storage resource or an individual web server that manages connection and content resources. This is especially true in complex environments with many resource consumers and resource devices. Efficiency depends on desirable characteristics including scalability, high performance, load balancing, short response time (responsiveness), or other attributes that can require optimization. A resource node is typically managed by an external management system or comprises a management layer imposed on it. This unfortunately creates extra overhead that goes beyond the basic responsibilities of resource management. In a resource device having a plurality of resource nodes, the problem of external management becomes serious. For example, when a load is assigned to a resource node, performance is usually maximized by notifying resource consumers of its status, migrating resource requests to other resource nodes, or performing other out-of-band management communications. As a result, excessive out-of-band communication occurs. In these systems, as new resource nodes are added to the environment, scalability issues arise because the increased management “chatter” is a significant part of the communication and bandwidth processing, adversely affecting performance. . This coarse scalability increases the cost of gradually increasing system capacity, and the cost of replicating the entire system is prohibitive.

  In order to reduce costs and achieve other desirable results, it is advantageous to design a system in which each individual resource node functions independently of other resource nodes and external management systems. In other words, resource nodes should not need information about other resource nodes to perform the basic responsibilities of resource management. In order to serve a resource consumer who wishes to access a resource, the timing at which the autonomous resource node processes a command transmitted from the resource consumer is determined by: (a) information related to the resource node; and (b) resource. It needs to be determined based on any information provided in the message containing the command by the consumer, or at least as a factor of such information.

  In addition to providing command information, the resource consumer provides information about the desired urgency or importance for the command to be processed. If a resource consumer understands the behavior of an autonomous resource node, the resource consumer can adjust its resource command messages based on interactions with all resource nodes to achieve higher performance, thus providing additional functionality. Inevitably, scalability, performance, or responsiveness can be achieved without introduction.

  Current related technologies try to provide efficient access to computing resources by focusing on activities other than the inevitable behaviors caused by interactions between resource consumers and resource nodes. . In the related art, the desired properties are optimized by introducing additional functions. For example, related technologies may need to manage communication between resource consumers and other resource consumers, communication between resource nodes and other resource nodes, or communication using communication paths. .

  Sun Microsystems US Pat. No. 5,506,969, “Method and apparatus for bus bandwidth management,” teaches how to efficiently schedule bus access on a high-speed bus from multiple applications to peripheral modules. . Although this patent describes how the bus between the application and the module is managed, a bus management system is required and the desired performance is not obtained in the operation of the individual modules and applications. This application does not use resource command messages with information on urgency or importance to allow a module to determine when to process a request.

  Hewlett-Packet Development Company, US Pat. No. 6,886,035, “Dynamic load balancing of a network and server computer” uses a redirection between the client computer and the resource itself. Teaches how to optimize Redirection requires a host on the network to be more familiar with itself than other hosts on the network. As each additional element introduced into the system must be managed and integrated into the system to ensure sufficient information for redirection, the scalability of the system is reduced. Load balancing is not achieved automatically because it uses information about urgency or importance in the command message.

  EMC Corporation, US Pat. No. 6,904,470, “Device selection by a disk adapter scheduler” teaches how to efficiently schedule resource I / O requests based on urgency and priority of requests. ing. This patent describes how the main scheduler determines which type of scheduler should be used to manage various I / O tasks directed to logical volumes managed by a disk adapter. . A logical volume is associated with a physical data storage resource, but is a coherent virtual device. Thus, each disk adapter manages devices, not resources, and by introducing additional management functions, load balancing, performance, and other qualities of the entire logical volume are guaranteed. Patent No. 6,904,470 does not correspond to the autonomous operation of the resource node, and the operation uses the information on the urgency and importance and the information of the resource node to execute the task. It is inevitably determined from the decision.

There is no related technology that addresses the need for autonomous resource node behavior to necessarily provide desirable attributes including scalability, high performance, load balancing, or responsiveness. In order to fully realize the benefits of autonomous resource nodes, the solution preferably has the following characteristics:
The resource node determines when to process the command transmitted from the resource consumer based on information related to the command and information related to the information of the resource node itself.
A message sent from a resource consumer to a resource node represents the resource consumer's awareness of the urgency or importance associated with processing the message or command within the message.

  The ability of the resource node to determine when to process a command sent to the resource node provides several advantages. First, because individual resource nodes are focused on their primary responsibilities (rather than other non-resource-intensive tasks), these resource nodes function more than similar resource nodes performing additional management tasks. It is efficient. Second, multiple resource consumers can interact with multiple resource nodes of a single resource device that does not have an external arbitrator. This allows each resource node to independently determine which resource consumer is noteworthy (if applicable), thereby reducing response time. Third, a group of resource nodes because multiple resource nodes provide access to redundant resources, and when these resource nodes are addressed simultaneously, each resource device takes advantage of its maximum capacity. Is automatically load balanced. If the load on one redundant resource node is maximum, another redundant resource node can serve the request without external intervention. Furthermore, since any of the redundant resource nodes can provide a valid response to the resource consumer, the responsiveness of the system is higher than a resource device without a redundant resource node. Fourth, each resource node is independent and can be easily integrated into the environment without requiring additional information from resource consumers or other resource nodes, so the scalability of such environments is high. Although only some advantages are shown, it will be appreciated that other advantages are envisaged in the present subject matter.

  Thus, a method and apparatus for providing a resource command message, and one or more resource nodes that autonomously determine when to process the resource command message based on the content of the command message and information associated with the resource node There remains a high need for resource devices comprising:

  One aspect of the present invention is directed to a resource command message comprising command parameters including a command and command urgency or command importance designation. A resource consumer interacts with resource nodes that compose a resource command message to configure one resource device. The resource node processes the resource command message based on the urgency or importance of the resource command message and the resource node-centric information. Furthermore, the resource device can include a plurality of resource nodes, and each resource node has a function of operating independently from all other nodes, and each resource node can receive a resource command message. The urgency or importance of the resource command message includes a relative value or an absolute value.

  In another aspect, the present invention is directed to a method for processing a resource command message. The method includes the steps of interpreting command urgency or command importance information contained in a resource command message and combining this information with resource node information to determine when a command in the resource command message is processed. A step of determining. The method further includes determining the order in which commands in the command queue are processed based on the timing at which the commands are processed. Such a determination can cause the command to be processed immediately, delayed processing, not processed, or change the order of processing relative to other commands sent before or after. Further, the method includes processing a resource command message by a plurality of resource nodes that make up the resource device.

  In yet another aspect, the present invention is directed to a method of accessing a resource device by creating a resource command message that includes a command and a command parameter comprising at least one of command urgency or command importance. . The method also includes sending a resource command message to the resource device and determining when to process a command in the resource command message. If the resource device comprises a plurality of resource nodes, sending the resource command message includes sending to at least some resource nodes by multicast.

  In a preferred embodiment, resource nodes included in a resource device can operate as autonomous entities, with each resource node managing its own resource. Resource consumers acquire resources from resource nodes and execute their own functions. Resource consumers are also autonomous entities. When a resource consumer needs a resource, the resource consumer sends a resource command message with a command urgency or command importance specification to the resource device to obtain a resource, reserve a resource, use a resource, Or interact with resources in other ways. Resource nodes are autonomous and serve requests from multiple resource consumers, so resource nodes interpret information about their state, history, capabilities, or other relevant information and information about its urgency or importance. In combination, determine how or when to process the command. As used herein, the expression “timing to process” means that the command is processed autonomously, including time-based processing, processing order, or other process processing concepts. Need to be interpreted broadly.

  It is assumed that resource consumers and resource nodes can communicate via paths that are outside the control of the consumer or node. In a preferred embodiment, to ensure high performance or reliability, the resource device comprises a plurality of resource nodes, each resource node manages all or part of the resource, and all other nodes, Functions independently of both devices and consumers. If resource nodes provide redundant resources, resource consumers are most likely to respond if they send resource command messages to some or all resource nodes and specify the current state of the network or load The resource node responds. In addition, other resource nodes interpret additional resource command messages or resource command responses as instructions that suspend or stop processing of previously unprocessed commands and reduce multiple responses. By combining the autonomous operation of resource nodes and the urgency or importance of resource commands, an overall load-balanced system is realized, and there is no need for out-of-band communication.

  The following is a description of terms used within this specification. The following terms are provided to ensure clarity when describing various aspects of the present subject matter without implied limitation.

  “Resource device” means a logical device that can be addressed in whole or in part on a communication path and provides access to products used as computing resources by resource consumers. The logical resource device is assumed to include a physical device or a virtual device. Physical resource devices include computers, monitors, hard disk drives, power supplies, or other physical elements. Virtual resource devices include addressable video displays, logical storage volumes, web server farms with URLs, or other abstractions of physical elements. A resource consumer interprets each resource device as a coherent integrated device, regardless of whether it is an actual physical or virtual architecture.

  “Resource consumer” means an entity that requests access or control to a product that performs a desired function. Resource consumers include computers, applications, users, web server gateways, or other entities that can communicate with a resource node via a communication path. Thus, resource consumers are also addressable. A resource device may be able to function as a resource consumer in some cases.

  “Resource node” means a part of a resource device and represents a fragment of a larger resource device. The maximum is the entire resource device, including the entire resource device. A resource node can also operate as an independent addressable entity on the communication path. Possible resource nodes include logical partitions (combined with other logical partitions to form logical volumes from the perspective of resource consumers), addressable video frames, individual web servers in a server farm, or others Are included.

  The content taught herein is used by developers or manufacturers of computing resources, including storage devices or media content servers, to build efficient and scalable systems that provide high performance and quick response. It would be advantageous to do so.

  Various objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments of the invention and the accompanying drawings. However, in these drawings, like numbers represent like components.

  The following detailed description of the embodiment refers to an example based on disks in a storage array and a web server in a server farm, illustrating the applicability of the subject matter of the present invention. The following two examples are used, but countless other examples can be shown, and no implicit limitations are derived from these examples.

  FIG. 1 illustrates an environment in which a resource consumer interacts with a resource device that includes one or more resource nodes. The resource device 110 includes one or a plurality of resource nodes 100A to 100N. Each resource node is communicatively connected to one or more resource consumers 120A to 120P via a communication path 115, respectively. In the preferred embodiment, many resource consumers interact with many resource devices.

Resource Consumers Resource consumers 120A to 120P operate independently of each other and do not require information from entities other than resource nodes 100A to 100N to interact with any resource managed by resource nodes 100A to 100N.

  Resource consumers 120A-120P interact with resource device 110 and include a combination of hardware, software, or firmware that includes instructions in computer-readable memory that are programmed to access resources managed by resource nodes 100A-100N. I have. In the preferred embodiment, the resource consumer comprises a computer running an application or operating system that attempts to access the resource. In a further preferred embodiment, the resource consumer comprises a workstation, which comprises a driver that provides communication between the workstation operating system and the resource nodes 100A to 100N. The driver also provides sufficient information about the resource device 100 to the operating system, and the resource device 100 is displayed as a locally connected device. For example, a Windows (registered trademark) computer attempts to mount a logical volume for storage. The Windows (registered trademark) computer includes a driver that receives an I / O command from the file system and converts it into a message. The converted message is transmitted via the network to the logical partition that configures the logical volume in a manner that is transparent to both the file system and the application that accesses the logical volume. Logical volumes are displayed as locally connected disk drives.

  On the other hand, the resource consumers 120A to 120P are considered to have applications that directly interact with the resource nodes 100A to 100N. For example, a gateway to a web site can represent a resource consumer accessing a distributed web server farm, and each web server represents a resource node.

  Resource consumers 120A to 120P function independently of each other, but interact with resource nodes 100A to 100N collectively or individually. Furthermore, resource consumers 120A-120P do not need to request information from resource consumers 120A-120P or systems external to resource nodes 100A-100N (including name servers, metadata servers, or other external systems). In the preferred embodiment, it is assumed that resource consumers 120A to 120P have the capability to detect resource nodes 100A to 100N. The detection function includes transmission of a broadcast message via the communication path 115, and the resource nodes 100A to 100N respond to this using their respective names. Further, in the preferred embodiment, resource consumers 120A-120P use name resolution to translate responses from resource nodes 100A-100N to addresses on communication path 115. Those skilled in the art of network programming will appreciate that there are various ways to perform detection and name identification, including SSDP, DNS, WINS, and so forth.

  When resource consumers 120A to 120P establish communication with resource nodes 100A to 100N, resource consumers 120A to 120P transmit resource command messages to resource device 110. The resource command message can be addressed to the resource device 110 in whole or in part. In a preferred embodiment, resource command messages are sent collectively from resource nodes 100A to 100N using multicast and are addressed as a whole, but resource device 110 is also partially addressed unicast messaging. It is assumed that it is possible. In this context, “multicast” sends a single message over the communication path 115, so that the resource consumer sends the bandwidth on the communication path 115 by sending two or more copies of the message to each resource node. It means that two or more resource nodes 100A to 100N receive the message without having to consume the width. It is contemplated that the resource device 110 can also be addressed simultaneously using multicast and unicast messaging.

  Each resource consumer 120A-120P builds a resource command message with command parameters relating to their unique needs. At least a portion of the resource command message is assumed to be in memory when the message is constructed. As used herein, the term “memory” refers to any hardware that stores information, and does not matter where the memory resides or how the information is stored. Command parameters include the resource consumer's awareness of the urgency or importance necessary to satisfy the resource consumer's needs. Urgency indicates the perception of timing constraints, and importance indicates the perception of priority required by individual resource consumers. Resource nodes 100A-100N use urgency or importance command parameters and other command parameters to assist in determining when to process resource command messages. In a preferred embodiment, a resource consumer determines its urgency or importance based on its own internal information or based on information collected from responses from resource nodes. Further, in a more preferred embodiment, the command parameters include a command identifier that is used to correlate groups of related resource command messages.

  Each of the resource consumers 120A to 120P has a function of receiving two or more responses from one resource command message. When the resource device 110 includes redundant resources managed by the resource nodes 100A to 100N, two or more of the resource nodes 100A to 100N respond to the message. Since each resource node functions independently of each other and does not know whether a response has already been generated, multiple responses are expected. However, multiple responses are reduced by properly processing urgency or importance information.

  In a preferred embodiment, resource consumers 120A-120P use a slow start algorithm to avoid congestion and ensure efficient use of bandwidth while simultaneously receiving multiple responses from resource nodes. Decrease. By first sending a small resource command message at a low speed, resource consumers 120A-120P determine which of resource nodes 100A-100N is likely to respond first, and then individual resource consumer 120A. 120P can help reduce multiple responses by independently adjusting each urgency and importance information. For example, the slow start algorithm can split a large command message into smaller command messages and send the small command message at a lower speed. When a response is received, the algorithm starts sending a larger message at a faster rate. Slow start ensures that flooding due to large packets is not performed if the network device has a small buffer. When flooding is performed on such devices, network performance is degraded. In addition, slow start provides the resource consumer with an opportunity to first detect which resource node is responsive. A window is provided in which multiple responses from the resource node are allowed when the packet is initially transmitted at low speed. A resource consumer can use multiple responses to determine a preferred provider of the resource. Further, when the communication speed is improved, information on the preferred provider is used, and a plurality of responses are deleted.

Resource Device Resource device 110 comprises one or more resource nodes (shown as resource nodes 100A to 100N). Although FIG. 1 shows only one resource device, it is conceivable that a plurality of resource devices coexist on the communication path 115.

  Since resource device 110 is accessible from one or more resource consumers 120A-120P, it is envisioned that resource device 110 may be a shared resource. In the preferred embodiment, resource device 110 includes information that resides on resource nodes 100A through 100N and is either privately owned or shared between resource consumers 120A through 120P. Shows the case.

  Resource device 110 comprises an identifier that is used by resource consumers 120A-120P to distinguish resource device 110 from other resource devices on communication path 115. In a preferred embodiment, these identifiers include names stored in the memory of resource nodes 100A through 100N, and addresses on communication path 115 can be determined by name. When resource consumers 120A to 120P issue detection requests, resource nodes 100A to 100N respond using names. These names include the name of the resource device 110, indicating that the resource node belongs to the resource device 110. In a particularly preferred embodiment, the IP address (which can include a unicast or multicast address) is identified by name identification. It is assumed that resource consumers 120A to 120P can specify the address of resource device 110 (preferably an IP multicast address) from one address.

  In a preferred environment, the resource device 110 includes redundant resource nodes, and two or more resource nodes 100A to 100N manage overlapping resources. For example, when resource device 110 represents a logical volume used to store data by resource consumers 120A-120P, resource node 100A and resource node 100B can represent logical partitions that mirror the same stored data. In yet another example, the resource device 110 is a logical Web server, and the resource nodes 100A to 100N are individual servers and have equivalent capabilities to process incoming connection request content.

  As an example of a resource device having redundant resource nodes, storage implemented based on Zetera (registered trademark) technology and virtualized as a plurality of logical partitions (resource nodes) in which logical volumes (resource devices) can specify IP addresses Consider an array. The logical volume represents one virtual disk to which logical block addresses (LBA: logical block addresses) from 1 to the maximum value MAX are assigned. Each logical partition manages a set of LBAs and does not necessarily have to be contiguous, but the set of logical partitions covers the entire range from LBA 1 to MAX. Furthermore, if two or more logical partitions manage the same set of LBAs, they are redundant, thus providing data mirroring. The workstation mounts the logical volume in the same way as a locally attached disk drive. The driver handles all communication with the logical partition that uses multicast to send command messages to all logical partitions over the network using a single address.

  Another example of a resource device with redundant nodes is a web server farm where each server can display the same content in a browser. The gateway collectively sends requests by command messages that arrive via the Internet to the server (s). The first responding server handles the connection.

  Resource device 110 represents processor bandwidth, display, memory, displayable content (servable content), connection handling, network bandwidth, or other computing resources including other related computing resources It is considered possible.

Communication Path Communication path 115 supports addressing and data transfer between resource consumers 120A to 120P and resource nodes 100A to 100N. Although it is assumed that the communication path 115 is not directly managed from either the resource node or the resource consumer, it is assumed that the resource consumers 120A to 120P or the resource nodes 100A to 100N can change the operation of the communication path 115. Furthermore, it is considered that the communication path 115 has a characteristic that lowers the reliability of the communication path.

  In the preferred embodiment, the communication path 115 comprises a packet switched network with Ethernet communications carrying the internet protocol. In the preferred embodiment, resource consumers 120A-120P and resource nodes 100A-100N use DHCP to obtain IP addresses.

Resource Node FIG. 2 shows a physical embodiment of the expected resource node. The resource node 200 receives a resource command message from the resource consumer via the communication path 115. The processing unit 210 receives the resource command message and processes the command in the message using the command queue 230 stored in the memory 220. Commands in the message are arranged in a command queue 230 represented by commands 233A to 233N. The processing unit 210 processes the commands 233A to 233N according to the resource node information stored in the memory 220 including the command queue 230 or the resource node data 240. When the processing unit 210 processes the commands 233A to 233N, the processing unit 210 accesses the resources 260A to 260M via the resource communication path 115.

  The resource node information stored in the memory 220 is assumed to provide sufficient information so that the resource node 200 can function independently of other resource nodes and can be aimed at a set of key responsibilities. Yes. In a preferred embodiment, one element of hardware comprising processing unit 210 and memory 220 provides service to one or more resource nodes. For example, a disk drive with data storage resources can be configured to provide a number of logical partitions using memory and processing units, each assigned a unique IP address, and each managing an LBA set. On the other hand, a rack mount enclosure that supports a plurality of disk drives can include one or more CPUs that constitute the processing unit 210 and one or more RAM modules that constitute the memory 220. The rack mount enclosure can now provide many logical partitions that manage multiple disk drives. It is also envisioned that the resource node 200 can represent a single resource. For example, one logical partition assigned an address can manage the entire disk drive.

  Resource communication path 215 provides addressing and data transfer between processing unit 210 and resources 260A-260M. In the preferred embodiment, the resource communication path 215 comprises a disk drive communication bus. Examples of disk buses include ATA, SCSI, Fiber Channel, USB, or others that exist or have not yet been invented. Further, it is contemplated that the resource communication path 215 can include a packet switched network. For example, when the resource node 200 is a content server, the resource communication path 200 may be an IP network to a storage array that stores content.

  The resource node 200 determines the timing of processing the commands 233A to 233N based on the interpretation of the urgency or importance information included in each resource command message and the interpretation of the resource node information stored in the memory 220. decide. The resource node 200 uses information about itself to determine an appropriate way to process commands autonomously. Information about the resource node 200 includes the ability to process commands, capacity, load, command queue order, preceding commands placed in the command queue, or others that affect the service of resource command messages sent from resource consumers Related information. For example, if the resource node 200 is functioning at 100% capacity and serving many resource consumers, it will silently discard the current resource command message while processing the current load. Can decide not to do this service. When a resource consumer message is discarded, the resource consumer may send another command, possibly adjusting the urgency or importance of the message, or wait for another resource node response.

  The information associated with the resource node 200 stored in the memory 220 can advantageously include instructions and data that determine the operation of the resource node 200. In a particularly preferred embodiment, the resource node data 240 contains information used by the resource consumer to establish an understanding of the entire resource device (the name of the resource device to which the resource node belongs, the name of the resource node, the resource node Role, attribute, or other information about the resource node). This means that the resource node data 240 also represents resource device information.

  In the preferred embodiment, the resource node 200 can enhance the desired characteristics of the resource device by focusing on its responsibility processing and not performing any other work. Thus, the resource node 200 can fully utilize its function with respect to the service request, and does not adversely affect performance and responsiveness. Furthermore, the replication of the resource node 200 provides advanced functions from the resource consumer point of view.

Redundant resource nodes Redundant resource nodes are resource nodes that provide access to almost equivalent resources. Redundant resource nodes can be distinguished by, for example, resource node data 240, name, or address. However, each redundant resource node manages the same type of resource and has the same capability with respect to load, function, or other capabilities in servicing resource command messages. Examples of redundant resource nodes include logical partitions that manage the same set of LBAs on separate disks or logical volumes on two Web servers that can perform equivalent content services. In the preferred embodiment, redundant resource nodes can join the same multicast group and resource consumers can address them simultaneously.

  In a preferred embodiment, the resource consumer sends a resource command message to the resource node of the resource device regardless of which resource node actually processes the resource command message. In the case of redundant resource nodes, the resource command message is likely to be processed substantially in parallel by the redundant resource node. As used herein, the expression “substantially in parallel” means that at least two resource nodes process resource command messages one after another within 10 seconds, depending on the communication path and the timing characteristics of the resource nodes. To do. Timing characteristics include latency, node load, or other parameters that affect processing time, including those specified directly by resource consumers or resource nodes.

  In redundant resource nodes, multiple responses may be generated for resource command messages, which is assumed to consume bandwidth in some cases. In the preferred embodiment, resource nodes and resource consumers interact in an attempt to erase multiple responses. It is envisioned that a resource consumer can initiate the exchange of multiple resource command messages that are expected to have multiple responses. In the preferred embodiment, the resource consumer selects a preferred provider from among the responding resource nodes and further inserts preferred provider information into the urgency of subsequent resource command messages. If the resource node is a preferred provider, the resource command message is processed normally. If the resource node is not a preferred provider, the process is delayed. If the preferred provider responds, the resource consumer sends the next message. A resource node other than the preferred provider receives the next message and cancels the previously sent pending command. Further, it is assumed that the current command takes over the position of the preceding command in the command queue.

  It is envisioned that the resource command message can comprise a command identifier that is used to identify a group of related commands. In such a situation, if the resource node holds a command in its command queue and receives an associated additional command, the resource node will process this preceding event (including deleting the command). ) Can be interpreted as interrupting instructions, reducing the number of expected multiple responses.

  The resource node 200 can execute a command or reserve a resource for future use based on the command and command parameters included in the resource command message. Executing a command actually provides a service for resource command messages. By reserving resources, resource consumers can aggregate the capabilities of multiple resource nodes.

Resource Command Message FIG. 3 shows an outline of an expected resource command message. The resource command message 300 comprises a command 320 with a command parameter 330 that is processed by the resource node. In the preferred embodiment, a resource consumer sends a resource command message 300 to a resource device or resource node using a resource destination address 310. Resource command message 300 also optionally includes data 340. For example, when the command 320 designates a write command to the disk drive, the data 340 exists. However, data 340 represents data to be written. In the preferred embodiment, the resource command 320 comprises command urgency information 335 or command importance information 337. In a further preferred embodiment, the resource command 320 comprises a command identifier 333. The term “indicates”, as used herein, means that something is resolved to something else. Therefore, “command 320 specifies a write command” means “command 320 is resolved and a write command is obtained”.

  The resource consumer builds the resource command message 300 in computer readable memory, and at least a portion of the resource command message 300 exists in the memory. Once constructed, the resource command message 300 is transmitted via a communication path connecting the resource consumer and the resource node. It is also envisioned that the resource command message 300 can be sent while it is being constructed. In the preferred embodiment, the resource command message 300 is encapsulated in a datagram and sent over a packet switched network. In a particularly preferred embodiment, the resource command message 300 is transmitted as a transport using a user datagram protocol (UDP). UDP has a reduced processing overhead as compared with a transmission control protocol (TCP), and is suitable for an atomic command structure in which information of one command is not necessary for processing of another command. Possible commands include: I / O processing execution, data reading, data writing, resource allocation, resource reservation, resource management, resource status check, resource inventory management, resource event Includes recording, locking resources, or other operations related to resources. The resource node uses the command parameter 330 in combination with its own information to determine when to process the command 320.

Command identifier The command identifier 333 includes information for grouping two or more related commands. The command identifier 333 is assumed to include a unique value for each group of commands. Commands are grouped for various reasons. For example, if the file system requests file data with many LBAs that are read from a logical volume with multiple mirrored logical partitions, the driver may request a separate resource command message for each LBA or a group of associated LBAs. Divide into resource command messages. Each mirrored logical partition may generate multiple responses in response to individual resource command messages. However, if the resource node detects a new read command in the command group identified by the command identifier 333, the resource node interrupts processing of the preceding command and reduces the possibility of multiple responses to the preceding resource command message Let It is also conceivable that the resource node can stop processing the currently executing command or interrupt the response of the processed command. In the preferred embodiment, the command identifier 333 comprises an ID number or sequence number.

  Command identifier 333 may also be considered to represent a series of bid-response transactions. For example, there are many connections requiring attention exceeding the capacity of one Web server in the Web server gateway. The gateway transmits a resource command message 300 with data 340 indicating the number of connections and a command identifier 333 to all Web servers operating as resource nodes. Each of the responding Web servers reserves a capacity and transmits a response. The gateway consolidates these responses and instructs the participating web server to handle the connection by sending a subsequent command with the same command identifier 333. Further, the non-participating Web server interprets the subsequent command as an instruction to stop processing the command with the same command identifier 333.

Urgency Urgency 335 (when used as a noun herein) comprises information related to the timing of processing command 320. The resource node is assumed to infer from the urgency 335 the timing at which commands are actually processed and the order of commands in the command queue. Expected urgency includes relative timing information or absolute timing information. The relative timing information includes a designation that processing must be performed within the time frame. The absolute timing information includes a specification that processing is performed at a specific time from the viewpoint of a resource consumer or a resource node.

  The resource node also combines urgency 335 with its own information. In the preferred embodiment, urgency 335 includes a preferred provider of resource consumers. A resource node that matches the preferred provider is inferred to be more urgent than a resource node that does not match the preferred provider. For example, a preferred provider resource node processes a command normally, but a non-preferred provider resource node delays processing of the command. This approach saves bandwidth by reducing multiple responses, and allows another resource node to take over as the preferred provider if the original preferred provider cannot respond immediately. There are several advantages such as high responsiveness is guaranteed.

Importance Importance 337 (when used as a noun herein) comprises information related to the priority with which the command 320 is processed. Importance is assumed to include relative priority or absolute priority. The relative priority includes quality of service (QoS) information. Absolute priority includes individual levels often associated with command queues. It is assumed that the resource node processes resource command messages transmitted from a plurality of resource consumers, and uses the importance information to resolve the command processing order.

  The resource node uses command parameters including urgency 335 or importance 337 to determine the final processing order of the commands.

Command Queue FIG. 4 shows an overview of the expected resource node command queue. The command queue 400 includes one or more command locations 415A through 415Z, the number of locations being determined by the resource node implementation.

  Although FIG. 4 shows a general representation of the command queue, those skilled in the art will appreciate that many ways of determining the order of processing a series of commands are possible without such a data structure. As used herein, “command queue” is to be interpreted broadly to include any order of command processing. Examples of command queues include chronological order, priority order, first-come-first-served order, only pending and executing commands, or any other order determined by the resource node .

  The resource node determines the order of commands or the order change based on the timing of processing the commands. Once the order is determined based on the resource node information, command urgency or importance, the resource node reorders the queue by placing the command at an appropriate location in the command queue 400. “Position”, as used herein, shall be interpreted broadly to include the concept of the relative order of commands relative to other commands that are pending or being executed. The resource node has a function of operating the command queue 400. Further, in ordering, it can be specified that the resource node does not process the command, and therefore the command is not queued. This concept also includes situations where the resource node cannot process incoming messages due to the heavy load on the resource node. Thus, the concept of determining the “timing” at which a resource node processes a command includes ignoring the resource command message.

  In the preferred embodiment, the command queue 400 is generally a first-come-first-served queue, and the resource node changes the location of commands based on QoS, preferred provider information, or command identifiers.

Resource Command Message Processing FIG. 5 illustrates the expected sequence of steps that a resource node uses to process a command queue message. A resource consumer sends a resource command message to one or more resource nodes. Thus, when multiple resource nodes (preferably redundant nodes) receive a resource command message, the steps shown in FIG. 5 are performed substantially in parallel.

  In step 500, the resource node receives a resource command message. Resource command messages can be addressed to individual nodes or collectively addressed to a set of resource nodes. In a preferred embodiment, the resource node receives a resource command message with a unicast or multicast IP address. It is assumed that a resource node cannot receive a resource command message when its load is large. In this case, another resource node handles this or the resource consumer tries to resend the resource message.

  In step 505, the resource node starts evaluating the resource command message. The resource node interprets urgency information in the resource command message when applicable. Urgent information includes direct or indirect information. Direct information includes a reference to the time at which the command should be processed. For example, the direct information includes a resource consumer's desired urgency designation as an absolute or relative time. The indirect information includes a reference for the resource node to estimate the time based on the urgency information. For example, the resource node can change the timing at which the command is processed when the resource command message includes information on a provider that is prioritized.

  At step 510, the resource node continues to evaluate the resource command message and interprets the importance information if applicable. Similar to the step of interpreting urgency information, importance information includes direct or indirect information. Direct information includes information on absolute priority or relative priority. Indirect information includes QoS information. The QoS information instructs the resource node to process the command preferentially over other commands to improve performance.

  At step 515, the resource node collects information related to the node itself and makes a final decision as to when the command in the resource command message should be processed. The assumed resource node information includes load information, functions, preceding commands, commands in the command queue, or other information about the resource nodes.

  Those skilled in the art will appreciate that the order of the preceding steps can be varied and does not necessarily follow the order presented.

  In step 520, the resource node combines its resource node information with information interpreted from the urgency or importance information to determine when the command in the resource command message should be processed. At step 533, the resource node determines whether the command should really be processed. If not, in step 535 the resource node discards the command message without reporting. In the preferred embodiment, the resource node autonomously determines whether the resource command message is discarded and the resource consumer is responsible for ensuring that the needs for that resource are met. If the resource node's load is high, the resource node's command queue is full, the resource node's resources are reserved, or the resource node does not want to process the command for other reasons, the resource It is assumed that the node will discard the command. If so, at step 500, the resource node waits again to receive another resource command message.

  If the resource node determines that the command is to be processed, step 543 determines whether the command is to be delayed. The command can be delayed for several reasons, such as the resource node is not the preferred provider, or the resource consumer explicitly requests the time that the command is processed. If the command is delayed, at step 545 the resource node determines the time that the command is delayed. It is also considered that the resource node can speed up command processing by canceling a command being executed with priority over the current command.

  After manipulating the command processing conditions, at step 553 the resource node determines whether to suspend the pending command. If the command is no longer valid (determined by information from the command parameter of the command), the pending command is interrupted. If the current command identifies itself as part of the group to which the pending command belongs by the command identifier, at step 555 the resource node can interpret the current command as an instruction to suspend the pending command. Suspension includes the result of further delaying or stopping the processing of the pending command, deleting the pending command from the command queue, deleting the pending command, or changing the processing time of the pending command. Other actions that are included.

  In step 565, the resource node completes the determination of when the command should be processed and the resource node places the command in the command queue. In the preferred embodiment, the order of the command queue is changed by the resource node based on priority, urgency, or command identifier. Those skilled in the art will appreciate that there are many ways to implement the command queue other than those described herein. The resource node is assumed to insert a command at an absolute or relative position in the command queue. For example, if the number of command queue positions is set, the absolute position represents a unique index into the standard queue. Examples of absolute positions include the command position currently being executed, the first position, or the last position. Relative position often represents a position ordered by time or priority, relative to other commands in the queue.

  At step 570, the resource node executes the command if applicable. Further, if applicable, at step 575, the resource node sends a resource command response message to the resource consumer. In the preferred embodiment, this response includes confirmation that the command is processed, requested data, or specification of the ability to process the command. In a further preferred embodiment, the resource node reserves at least part of the resource allocation requested by the resource consumer and informs the resource consumer of its capability specification. For example, if a resource consumer requests storage of 100 gigabytes of data, the resource node can respond with a designation that it can store 50 gigabytes. A resource node can reserve 50 gigabytes to allow resource consumers to consolidate the capabilities of other resource nodes to ensure 100 gigabytes.

  When the resource command response message is received by another resource node, it is also assumed that it can be interpreted as an instruction to interrupt the processing of the command in the resource command message. Those skilled in the art of software or firmware development will appreciate that step 570 can be performed as a thread or task parallel to the message processing step.

  In the preferred embodiment, the steps of the resource node shown in FIG. 5 are stored on a computer readable medium as a series of instructions executed by the processing unit. One skilled in the art of firmware or software development will understand that there are many possible ways to implement these steps, but all such methods do not depart from the scope of the inventive subject matter. In yet another preferred embodiment, it is assumed that multiple resource nodes process resource command messages substantially in parallel. In a further preferred embodiment, multiple resource nodes process resource command messages one after another within 3 seconds.

Accessing Resource Devices FIG. 6 shows the expected sequence of steps used to allow resource consumers and resource nodes to access resources. A resource consumer sends a resource command message to a resource device comprising one or more resource nodes. In the preferred embodiment, it is assumed that one or more resource consumers perform steps independently of each other and possibly interact with the same resource node.

  At step 600, the resource consumer begins a process of constructing a resource command message in computer readable memory. The resource consumer sets the sense desired by the resource consumer for the urgency associated with the command in the resource command message. In step 605, the resource consumer sets the importance of the command. If applicable, both steps 600 and 605 are performed for the current resource command message. At step 610, the resource consumer optionally assigns a command identifier that indicates how the current command relates to the preceding or subsequent command. Steps 600, 605, or 610 can be performed in any order.

  At step 615, the resource consumer constructs a resource command message based on the command, command parameters including a command identifier, urgency, or importance.

  At step 620, the resource consumer sends a resource command message to the resource device. In a preferred embodiment, the resource command message is configured as one or more packets and is sent over a packet switched network. In a particularly preferred embodiment, packets are transmitted using UDP. Further, when a resource consumer sends a resource command message, the resource consumer preferably sends the message collectively to a group of resource nodes or all resource nodes. In a preferred embodiment, the resource command message is sent using multicast, and each resource node is a member of a multicast group whose address indicates a resource device. It is assumed that the resource command message is initially transmitted at a low speed in order to avoid congestion of a communication path connecting the resource consumer and the resource node. Those skilled in the art of network protocols including TCP will understand that congestion is avoided by slow start.

  At step 625, the resource node receives the resource command message and begins processing the message. In the preferred embodiment, multiple resource nodes can receive the same resource message. Furthermore, in a more preferred embodiment, multiple resource nodes can process commands equally and return responses to the resource consumer that sent the resource command message.

  At step 630, the resource node uses the urgency, importance, or command identifier information and information about the resource node itself to determine when the command should be processed. In step 633, the resource node determines whether processing of the preceding command should be interrupted. If so, the preceding command is interrupted at step 635, otherwise the current command is inserted into the command queue at step 640. When the commands are processed, the resource node executes the command at step 645 and sends an appropriate response at step 650.

  In a preferred embodiment, at step 655, the resource consumer can receive multiple responses from multiple resource nodes. In this case, the plurality of resource nodes provide a redundant function. In this case, the resource consumer can select a preferred resource node from among a plurality of nodes. In a particularly preferred embodiment, the preferred resource node is selected on the basis of the first responding node among the redundant nodes. Each resource consumer interacting with a resource device comprising multiple resource nodes can specify a completely different preferred provider. Furthermore, as the environmental conditions change, the preferred providers can change. Thus, at any given time, resource consumers inevitably achieve some performance, load balancing, or responsiveness without imposing external management.

  In a preferred embodiment, the steps shown in FIG. 6 are stored on a computer readable medium in the form of instructions executed by a processing unit.

Benefits A resource device comprising a resource consumer and one or more resource nodes can realize many benefits as a necessary result of using resource command messages.

  As additional resource devices or resource nodes are added to the system, the size of the resource inevitably increases. Because individual resource nodes are focused on their primary responsibilities and resource command message processing, resource nodes are autonomous and scale the system at an atomic level to the ability of the communication path to process resource command messages. Can be enlarged. The bandwidth of the communication path is used more efficiently. This is because all traffic is related to access to resources, not system management or maintenance. Furthermore, when the resource system requires further functions, an increase in cost is suppressed because individual resource nodes need only be added as opposed to copying the entire resource system.

  As additional redundant nodes are added to the system, both resource performance and responsiveness improve. Since the resource consumer collectively sends resource command messages to a plurality of resource nodes, the plurality of resource nodes can respond. Since different loads are assigned to individual resource nodes, the resource node with the highest response responds first, resulting in the shortest response time. Furthermore, multiple resource nodes (not necessarily redundant nodes) process resource command messages substantially in parallel, providing high performance to resource consumers. The resource consumer uses the importance information to specify a priority for the resource node to consider for processing the command. Importance information supports the processing of QoS data. Multiple responses are reduced by slow start, so congestion is avoided and bandwidth consumption is limited. In addition, resource consumers have a unique view for each resource node, ensuring the shortest response time and the reduction of multiple messages by uniquely selecting the preferred provider when operating on redundant resource nodes. To help.

  Each node functions independently and can handle only the traffic that each node is designed to handle, so load balancing is realized as a necessary result of redundant resource nodes. The resource consumer has no prior priority as to which resource node will service the request, but the resource consumer can bias which node prioritizes to reduce multiple responses. Even if there is a preferred provider to the resource consumer, continuous interaction can change the preferred provider based on how other resource nodes respond. Therefore, the load is distributed among the nodes. As additional nodes are added to the system, the load is reduced, so resource consumers can effectively share resource nodes by cycling through the preferred nodes as needed.

  As described above, specific configurations and methods of resource command messages are disclosed. However, it will be apparent to those skilled in the art that many more modifications besides the foregoing disclosure may be made without departing from the inventive concepts set forth herein. Accordingly, the subject matter of the present invention should not be limited except in the spirit of the present disclosure. Further, in interpreting the present disclosure, all terms shall be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprising”, “comprising” refer to elements, components, or steps non-exclusively, and the mentioned elements, components, or steps are other elements, components that are not explicitly mentioned. Or co-exist with, combined with, or combined with steps.

FIG. 2 is a diagram illustrating an environment in which a resource consumer interacts with one resource device comprising a plurality of resource nodes. FIG. 3 is a schematic diagram illustrating an expected physical embodiment of a resource node. FIG. 6 is a schematic diagram illustrating expected resource command messages stored in computer readable memory. FIG. 6 is a schematic diagram illustrating an expected resource node command queue. FIG. 6 is a schematic diagram illustrating expected steps for processing a resource command message. FIG. 6 is a schematic diagram illustrating expected steps for accessing a resource device.

Claims (16)

A computer readable memory;
An apparatus having instructions in said computer readable memory comprising :
Said computer-readable instructions in memory to perform the steps in the apparatus, The procedure,
Dividing the first resource command message More command message groups of a small size,
In the first multicast message transmitted to a plurality of resource nodes via the communications path said command message group,
Receive a response to the previous Kiko command message group from the plurality of resource nodes,
Selecting a preferred provider of the plurality of resource nodes based on at least some of the responses received from the plurality of resource nodes;
Transmitting a second resource command message to the plurality of resource nodes in a second multicast message;
The second resource command message includes a command and a designation of the preferred provider selected from among the plurality of resource nodes, wherein the preferred provider is more urgent than a second urgency. An apparatus for processing the command according to a first urgency, wherein a non-preferred provider of the plurality of resource nodes processes the command according to the second urgency.   The apparatus of claim 1, wherein each of the responses from the plurality of resource nodes includes the same content.   The apparatus of claim 1, wherein the plurality of resource nodes comprises a plurality of redundant resource nodes that manage overlapping resources.   The apparatus of claim 1, wherein the second resource command message includes a command urgency including a designation of the preferred provider.   The apparatus of claim 1, wherein the first resource command message includes a command urgency that specifies a relative time to process a command in the first resource command message.   The apparatus of claim 1, wherein the first resource command message includes a command urgency that specifies an absolute time to process a command in the first resource command message.   The apparatus of claim 1, wherein the first resource command message includes an importance of a command that specifies an absolute priority for processing a command in the first resource command message.   The apparatus of claim 1, wherein the second resource command message includes a command identifier that associates the second resource command message with the first resource command message. Dividing the first resource command message into smaller sized command messages ;
In the first multicast message, and transmitting the command message group to a plurality of resource nodes,
Receiving a response to the previous Kiko command message group from the plurality of resource nodes,
Selecting a preferred provider of the plurality of resource nodes based on at least some of the responses received from the plurality of resource nodes;
Transmitting a second resource command message to the plurality of resource nodes in a second multicast message, wherein the second resource command message is a preferred provider selected from the plurality of resource nodes. The preferred provider processes the command according to a first urgency that is more urgent than a second urgency, and the non-preferred provider of the plurality of resource nodes is the second urgency. Processing the command according to:   The method of claim 9, wherein each of the responses from the plurality of resource nodes includes the same content.   The method of claim 9, wherein the plurality of resource nodes comprises a plurality of redundant resource nodes that manage overlapping resources.   The method of claim 9, further comprising providing the second resource command message with a command urgency including designation of the preferred provider.   The method of claim 9, further comprising: providing a first resource command message with an urgency of a command that specifies a relative time for processing a command in the first resource command message.   The method of claim 9, further comprising: providing a first resource command message with an urgency of a command that specifies an absolute time for processing a command in the first resource command message.   The method of claim 9, further comprising: providing a first resource command message with the importance of a command that specifies an absolute priority for processing a command in the first resource command. The method of claim 9, further comprising: providing a second resource command message with a command identifier that associates the second resource command message with the first resource command message.

Images