JP2017184195A - Communication management device, communication management method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a management technology capable of flexibly switching a path at high speed while reducing a management load in distributed type storage.SOLUTION: A communication management device includes: generation means 12 for generating settings, including a first rule for determining a transfer destination of an object based on an identifier uniquely given to the object, of a plurality of communication equipment in a route to send the object to one of a plurality of actual storage nodes based on a first assignment relation in which the identifier is assigned to one of the plurality of actual storage nodes and connection relation between the plurality of communication equipment and the plurality of actual storage nodes; and application means 13 for applying the settings to the communication equipment.SELECTED DRAWING: Figure 27

Description

  The present disclosure relates to a technique for managing a network.

  In a distributed file system (hereinafter referred to as “distributed FS” (File System)) and a distributed object storage, node management is required. As a node management method, (1) a centralized management method using a metadata server, or (2) a management method in which each node belonging to the system has a calculation algorithm or a management table and solves a communication destination Has been used.

  Patent Documents 1 to 9 disclose SDN (Software-Defined Network) typified by OpenFlow (registered trademark) and techniques related to packet transfer.

JP2015-154210A Japanese Patent Laying-Open No. 2015-095730 JP 2010-109791 A Special table 2015-537754 gazette JP 2003-153239 A European Patent Application No. 2731304 European Patent Application Publication No. 2782312 US Patent Application Publication No. 2014/0229945 US Patent Application Publication No. 2014/0269288

  When the centralized management of (1) is performed, if a failure occurs in the control server that performs management, there is a problem that the influence of the failure spreads to the whole. Therefore, it is necessary to take measures such as making the control server redundant. Further, since the access to the control server is excessively concentrated, there is a problem that the performance of the control server becomes a bottleneck and the performance of the distributed FS or the distributed object storage is not obtained.

  In the distributed management type method (2), nodes participating in the system must grasp each node. This method has an advantage that even if a certain node goes down, the remaining nodes can continue to operate, and the system can continue to operate. However, since each node must grasp the responsibility of participating nodes and objects, the burden of node management is large. In addition, when a change is made to a node participating in the system, such as addition of a node, there is a drawback that it takes time until the management information is transmitted to the whole.

  One of the objects of the present invention is to provide a management technique capable of switching routes at high speed and flexibly while reducing the management load in the distributed storage.

  The communication management apparatus according to an aspect of the present invention provides a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. A setting including a first rule for determining a first assignment relationship for assigning the identifier to any of the plurality of real storage nodes, and a connection relationship between the plurality of communication devices and the plurality of real storage nodes. Generating means for generating the information based on the information, and applying means for applying the setting to the communication device.

  The communication management method according to an aspect of the present invention provides a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. A setting including a first rule for determining a first assignment relationship for assigning the identifier to any of the plurality of real storage nodes, and a connection relationship between the plurality of communication devices and the plurality of real storage nodes. And the setting is applied to the communication device.

  The program according to an aspect of the present invention is a program for transferring an object to a computer based on an identifier uniquely assigned to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. A first allocation relationship for assigning the identifier to any one of the plurality of real storage nodes, and a connection relationship between the plurality of communication devices and the plurality of real storage nodes. Generation processing to be generated based on the above, and application processing to apply the setting to the communication device.

  According to the present invention, there is an effect that a flexible path switching can be performed at high speed while reducing the management load in the distributed storage.

FIG. 1 is a diagram showing the configuration of a distributed storage system according to the first embodiment of the present invention. FIG. 2 is a diagram schematically showing the OpenFlow network according to the first embodiment of the present invention. FIG. 3 is a diagram schematically illustrating the configuration of the OpenFlow switch. . FIG. 4 is a diagram schematically illustrating the structure of the Flow Table. FIG. 5 is a diagram showing an outline of the structure of a network packet. FIG. 6 is a diagram illustrating an example of hash space division according to the first embodiment of the present invention. FIG. 7 is a diagram schematically showing the structure of the Group Table. FIG. 8 is a flowchart showing the overall operation of the system according to the first embodiment of the present invention. FIG. 9 is a flowchart showing an operation of designing and constructing an overlay network for object storage in the system according to the first embodiment of the present invention. FIG. 10 is a diagram illustrating an example of a Flow Table according to the first embodiment of the present invention. FIG. 11 is a diagram illustrating an example of a flow table according to the first embodiment of the present invention. FIG. 12 is a diagram illustrating an example of a Flow Table according to the first embodiment of the present invention. FIG. 13 is a flowchart showing an example of an operation when a packet arrives in the OpenFlow switch according to the first embodiment of the present invention. FIG. 14 is a flowchart showing an example of the overall operation of the client according to the first embodiment of the present invention. FIG. 15 is a flowchart showing an example of the operation of the object storage node according to the first embodiment of the present invention. FIG. 16 is a sequence diagram illustrating an example of an operation for exchanging data in the system according to the first embodiment of the present invention. FIG. 17 is a diagram schematically illustrating data in which a fragment is generated. FIG. 18 is a sequence diagram illustrating an example of an operation using a connection request in the system according to the present embodiment. FIG. 19 is a sequence diagram illustrating an example of an operation to which a solution is applied in the system according to the first embodiment of the present invention. FIG. 20 is a diagram illustrating an example of a configuration in which a format conversion function is incorporated in a library according to the system of the first embodiment of the present invention. FIG. 21 is a sequence diagram illustrating an example of an operation of requesting the communication module to explain the communication in the system according to the present embodiment. FIG. 22 is a sequence diagram illustrating an example of object storage communication using the communication module according to the embodiment. FIG. 23 is a block diagram showing the configuration of a storage system according to the second embodiment of the present invention. FIG. 24 is a block diagram illustrating an example of a configuration of a communication device according to the second embodiment of the present invention. FIG. 25 is a flowchart showing an example of the operation of the communication management apparatus according to the second embodiment of the present invention. FIG. 26 is a flowchart illustrating an example of the operation of the communication device according to the second embodiment of the present invention. FIG. 27 is a block diagram illustrating an example of a configuration of a communication management apparatus according to the third embodiment of the present invention. FIG. 28 is a flowchart showing an example of the operation of the communication management apparatus according to the third embodiment of the present invention. FIG. 29 is a diagram illustrating an example of a hardware configuration of a computer 1000 that can implement the apparatus according to each embodiment of the present invention.

  Next, embodiments of the invention will be described in detail with reference to the drawings.

<< First Embodiment >>
First, features of the present embodiment will be briefly described.

  In the present invention, a new system is constructed in which the node management function is transferred from the storage system side to the network side.

In this embodiment, an overlay network for a storage network is constructed and used. In this embodiment, the OpenFlow is not only used but also extended for realization. And this embodiment is provided with the feature shown below, for example.
1. The OpenFlow controller or a cooperative server of the OpenFlow controller has a function of managing the distributed storage arrangement information, and appropriately manages the resource sharing of the distributed object storage. In FIG. 1 described below and the following description, the function for managing the distributed storage arrangement information is also referred to as a “distributed storage arrangement information management function”.
2. The function to manage the distribution information of the distributed storage and the OpenFlow controller work together to construct an overlay network for the distributed storage on the OpenFlow network.
3. In the constructed overlay network, the OpenFlow switch delivers a request to an appropriate storage node using a hash value (resource identifier) associated with an object (resource) as a clue.

  Next, an outline of a technique for realizing the present embodiment will be described.

  When the resource is transmitted to the representative address of the storage, an overlay network for the distributed storage is constructed so that the network delivers the object to the node in charge of the resource, and the constructed overlay network is used. As a result, the storage client can use the distributed storage without directly grasping the node in charge of the resource.

Use OpenFlow to build an overlay network for distributed storage. OpenFlow can build an overlay network, but it cannot build a network for object storage with the current specifications. At the time of construction, for example, at least one of the following solutions is performed.
(1) Extend the OpenFlow specification to handle resource identifiers and requests used in object storage.
(2) The transmission module maps necessary information such as resource identifiers and requests to a protocol header area that can be handled by OpenFlow so that OpenFlow can be used.

  In order to be able to manage which resource (corresponding to an object for object storage) each node of the distributed storage is in charge of, a function for managing the arrangement information of the distributed storage is prepared. The function to manage the distribution information of distributed storage makes it possible to handle resources even with OpenFlow switches that use limited memory. In addition, the function for managing the distribution information of the distributed storage is used to construct an overlay network for the distributed storage by linking with the OpenFlow controller and setting the Flow Table of the OpenFlow switch.

<Configuration>
Next, the configuration of the present embodiment will be described.

  In this embodiment, node management is realized on the network side by constructing an overlay network of distributed object storage. The client and the object storage configuration node are additionally provided with a communication function for performing the exchange described below in the function module for object storage.

  First, the object storage will be briefly described.

  An object storage is a storage device that accesses a series of data called an object using an object identifier associated with the object. In general, some hash value is often used as the object identifier. When storing data, a flat arrangement having no hierarchical structure is often adopted, but the storage method is not strictly defined.

  An interface specification called REST (Representational State Transfer) I / F may be used for data exchange with the object storage instead of CRUD (Create, Read, Update, Delete) I / F (Interface). In the following description, REST I / F is also referred to as “Rest I / F”. The REST I / F uses two attributes: a resource identifier that specifies a target resource, and a request that represents an operation on the resource. Object storage can be easily used as a REST I / F by replacing a resource with an object (a series of data) and a resource identifier with an object identifier.

Describe requests for resources. Requests include GET, HEAD, POST, PUT, and DELETE. These requests are mainly used in the following meanings.
GET: Retrieves the resource with the specified resource identifier.
HEAD: Takes out the meta resource with the specified resource identifier.
POST: Creates a new resource with the specified resource identifier.
PUT: Updates the resource to the specified resource identifier.
DELETE: Deletes the resource with the specified resource identifier.

  In this embodiment, an identification number called a request ID (Identifier) is assigned to the request. For example, it is assumed that the request IDs assigned to the above requests are 1, 2,. The reason for assigning numerical values is to reduce the amount of information and fix the description position on the packet. However, it is only necessary to identify the request, so another expression such as a character string may be used as the request ID.

  The above request almost corresponds to CRUD (Create, Read, Update, Delete) except HEAD and POST. Therefore, if the interface specification can be realized by Rest I / F, I / F using a request based on a resource identifier and CRUD instead of Rest I / F can be easily realized.

  In this embodiment, the maximum size of an object is set to 1460 (header length used in this embodiment) or less (Related 1). This limitation is to reduce the size of the object to a standard MTU (Maximum Transmission Unit) size (1500) of Ethernet (registered trademark) used for communication. The size expansion method will be described in << Related Technology >>.

  In addition, an object storage program held by an object storage server or client is an example of preparing data according to the network protocol used in this embodiment. The method of providing the conversion to the communication format as an external library or OS (Operating System) instead of the object storage program is described in << Related Technology >> (Related 2).

  This embodiment will be described with reference to FIGS. FIG. 1 is a diagram showing the configuration of the distributed storage system of this embodiment. FIG. 2 is a diagram schematically showing the OpenFlow network of the present embodiment. FIG. 2 is a more specific example of the OpenFlow network portion (FIG. 1, 101) of FIG.

  The network handled in this embodiment is not a simple Ethernet device but a network using OpenFlow compatible devices. In contrast to OpenFlow, simple Ethernet is sometimes referred to as a legacy network.

  The OpenFlow network is composed of one or more OpenFlow controllers called network control controllers and one or more OpenFlow switches. OpenFlow switches are distinguished by identifiers called datapath IDs. In this embodiment, priority is given to ease of understanding, and identifiers of OpenFlow switches are sometimes expressed as OFS1, OFS2,... Instead of numerical values.

  The OpenFlow controller and OpenFlow switch are connected by a control network connected by an Ethernet network, and control communication is performed using the OpenFlow protocol.

  A network configured by connections between OpenFlow switches is used for communication of devices such as PCs (Personal Computers). Here, such a network is referred to as an OpenFlow network.

  In the network configuration shown in FIGS. 1 and 2, one OpenFlow controller (FIGS. 1 and 103) and three OpenFlow switches (FIGS. 2 and 201 to 203) are used. In the description of this embodiment, the OpenFlow controller is abbreviated as OFC, and the OpenFlow switch is abbreviated as OFS. In particular, if there is no need to explain the OFC, the OFC is omitted in the following figures for simplicity.

In this embodiment, the OpenFlow controller is equipped with a function for constructing a storage overlay network called distributed storage arrangement information management means (FIG. 1, 104). The distributed storage arrangement information management means may be mounted on a server that cooperates with the OpenFlow controller. (Specific functions of the distributed storage arrangement information management means will be described later.)
Client (FIG. 1, 106) is a device that is a client of object storage. Usually, the client is a device having a communication function such as a personal computer. In the present embodiment, the client additionally has a function for using the object storage.

  A computer (object storage node) (Fig. 1, 105) that configures object storage is connected to OFS.

  In addition to the object storage node, a personal computer or a server (FIG. 1, 107) may be connected to the OFS. And the apparatus connected to OFS may use OFS for each communication. That is, communication other than object storage may be performed via OFS.

  In FIG. 2, the port number is described in the portion where the connection between OFS is desired to be specified. Specifically, the connection relationship between OSFs is as follows.

OFS1 port 1 and OFS2 port 0 are cabled,
Cable connection is made between port 2 of OFS1 and port 0 of OFS3.

  The connection relationship between the OSF and the object storage node is as follows.

OFS2 port 1 and object storage node 1 are cabled,
OFS2 port 2 and object storage node 2 are cabled.

OFS3 port 1 and object storage node 3 are cabled,
OFS3 port 2 and object storage node 4 are cabled.

  The internal configuration of the OpenFlow switch is shown in FIG. FIG. 3 is a diagram schematically illustrating the configuration of the OpenFlow switch. According to the current convention, the OpenFlow switch includes a communication means (Secure Channel) for communicating with the OpenFlow controller, a Group Table for storing a certain type of Action list, and a plurality of Flow Tables starting from 0. The Group Table and each Flow Table may be stored in a storage device such as a memory, for example.

  The Flow Table will be described with reference to FIG. FIG. 4 is a diagram schematically illustrating the structure of the Flow Table. Each Flow Table is composed of a plurality of flow entries having Match Fields, Counters, and Instructions in one entry. Note that the components of the Flow Table may differ depending on the OpenFlow version.

Match Fields holds the condition of the packet corresponding to this entry. The Match Field can specify Ethernet, TCP / IP (Transmission Control Protocol / Internet Protocol) header information, etc., in addition to special values corresponding to all values ANY. For reference, the following can be specified in OpenFlow version 1.1.
1. Packet input port (Ingress Port),
2. Meta information such as statistical information (Metadata),
3. Ether source MAC address,
4. Ether destination MAC address,
5. Ether type,
6. VLAN ID,
7. VLAN priority,
8. MPLS label,
9. MPLS traffic class,
10. IPv4 source address,
11. IPv4 destination address,
12. IPv4 proto / ARP opcode,
13. IPv4 ToS bits,
14. TCP / UDP / SCTP source port or ICMP type,
15. TCP / UDP / SCTP destination port or ICMP code
The traffic specified by these attributes is called a flow. In the present embodiment, in addition to the attributes in the OpenFlow specification, attributes for object storage (resource identifier and request) can be specified in the flow. (If necessary, other attributes such as request number may be added.)
A network packet is composed of a protocol header including information such as a destination and a transmission source, and a payload for storing data. This can be represented as shown in FIG. FIG. 5 is a diagram showing an outline of the structure of a network packet. In the object storage exchange of this embodiment, a request and a resource identifier are placed at the beginning of the payload so that it can be handled as an extension of the protocol header. (There is also a direction to expand in response to the related 1 fragment). In this embodiment, it is assumed that the size of the protocol header and UDP (User Datagram Protocol) datagram does not exceed the MTU value so that no fragment occurs. The standard size for Ethernet is 1500 bytes, which is 1460 bytes if the IP and UDP header sizes are reduced from there. Therefore, in the present embodiment, the maximum size of the object is set to be within 1460 (header size used in the present embodiment).

  Counters stores statistical information that is updated when the entry corresponds to this Counters entry.

  Instructions holds processing instructions. In Instructions, an action for processing a packet corresponding to Match Fields is registered.

  The method for realizing the flow for Rest I / F will be described in more detail.

  The resource identifier handled in this embodiment is assumed to be a hash value obtained by SHA-1 (Secure Hash Algorithm 1). However, the resource identifier may be a value obtained by another method as long as the resource can be appropriately expressed. In selecting the method, it is desirable in terms of operation performance on the OpenFlow switch that the hash value has a fixed length.

  The hash value obtained by SHA-1 ranges from 0 to “(2 to the 160th power) −1”. When applying to OpenFlow, if one hash value is simply associated with one flow entry, more memory must be allocated to the Flow Table. If the number of flow entries increases, the number of flow confirmations increases, which affects the processing performance of the OpenFlow switch. Due to these two problems, the present embodiment performs the following devices.

  Let n be the maximum number of nodes in the object storage that can be handled by the system according to this embodiment.

  First, divide the hash space into n. As an example of hash space division, the hash value is divided by a power of 2. For example, in the case of 16 (= 2 ^ 4) partitioning, hash space partitioning is as shown in FIG. FIG. 6 is a diagram illustrating an example of hash space division. This makes it possible to realize a simple discrimination method that can discriminate an area including a hash value by looking at only the first bit (4 bits in 16 divisions) of the value representing the hash value in binary.

  Next, the assigned charge of the hash space is associated with a virtual object storage (ie, virtual object storage) as follows, for example.

Hash space area 0: Virtual object storage 0,
Hash space area 1: Virtual object storage 1,
......
Virtual object storage has the meaning of securing a frame in advance on the Flow Table.

  Finally, the association between the virtual object storage and the actually existing object storage is performed. The association described so far is performed by the distributed storage arrangement information management means installed in the OpenFlow controller or a server cooperating with the controller. The distributed storage arrangement information management means appropriately performs maintenance management by appropriately associating at the timing such as addition and deletion of nodes. When there are four configuration nodes of the object storage of this embodiment, allocation is performed as follows based on the consistent hashing method.

Virtual object storage 0 to 3: Object Storage 1
Virtual object storage 4-7: Object Storage 2
Virtual object storage 8-11: Object Storage 3
Virtual object storage 12-15: Object Storage 4
Object Storage 1 to Object Storage 4 are the above-described object storage nodes. Hereinafter, the object storage node is also referred to as “real node”. When the real node is determined, the port to be transferred by the switch is known, so that it can be described in the Flow Table. If no real node exists, the packet is dropped.

  As described above, once a frame is provided as a virtual node (that is, virtual object storage; hereinafter also referred to as a virtual storage node), the number of flow entries in the Flow Table can be increased even if the actual object storage node increases or decreases. It does not change. Therefore, the impact on the processing performance of the OpenFlow switch can be prevented.

  Until the OpenFlow specification v1.1, only the mask part of the IP address could be specified as a flow match condition. However, in v1.2, the target that can be specified by the range has been expanded. Also in this embodiment, the resource identifier (hash value) match condition is extended so that it can be specified by a range. That is, the value of the first few bits of the resource identifier can be handled as a flow match condition to prepare for an increase or decrease in the number of divisions in the resource ID space.

  As described above, in this embodiment, the object storage in charge can be determined by the 4-bit value at the head of the resource ID, so that the description of Storage Overlay Network can be implemented in the form of Flow Table.

  The Group Table will be described with reference to FIG. FIG. 7 is a diagram schematically showing the structure of the Group Table. The Group Table is composed of a plurality of group entries each having a Group Identifier, Group Type, Counters, and Action Buckets. By using Group Table, operations such as rewriting and transferring the protocol field can be realized.

  The Group Identifier is an unsigned 32-bit integer and is a unique identifier for identifying a group entry.

  Group Type represents the meaning of this group entry and represents the usage of Action Buckets. In the OpenFlow v1.1 specification, there are four group types: all, select, indirect, and fast failover.

  If the value is all, all action buckets registered in Action Buckets are executed. When executing each action bucket, the packet is efficiently copied and passed.

  If the value is select, execute one of the action buckets registered in Action Buckets.

  Counters stores statistical information of packets processed by this group entry.

  Action Buckets is a list of action buckets. Each action bucket stores a list of actions that execute, change parameters, etc.

  The OpenFlow controller sets the Flow Table of the OpenFlow switch through a secure communication path. For packets input to the OpenFlow switch, the OpenFlow switch performs processing such as forwarding and discarding according to the set Flow Table.

  Specifically, when an OpenFlow switch receives a packet, it executes pipeline processing using the Flow Table. Pipeline processing starts from the 0th table of the Flow Table and ends when it is executed up to the last flow entry of the Table without the GoTo instruction. The Goto command can move processing to another specified Flow Table, but cannot return to the Flow Table. In other words, the Goto command can only be used in the direction of always increasing the table number.

<Description of operation>
Next, the overall operation of the system according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing the overall operation of the system according to the present embodiment.

  First, the system constructs a base OpenFlow network using normal OpenFlow technology (step S0101).

  Next, the system designs and constructs an overlay network for object storage (step S0102). Specific contents will be described later.

  When the network design and construction are completed, the nodes constituting the object storage start the storage service (step S0103). Object storage client and object storage are exchanged.

  The design and construction of the overlay network for object storage (step S0102) will be described in more detail with reference to FIG. FIG. 9 is a flowchart showing an operation of designing and constructing an overlay network for object storage. The information that needs to be grasped in advance depends on how the overlay network is implemented. In this embodiment, the network designer knows the IP address used by the object storage (representative object storage address), the OFS to which the connected object storage node is connected, and the connected port up to this point. Need to be.

  First, the divided storage arrangement information management means (hereinafter also referred to as “management unit”) of the OpenFlow controller or the cooperative server divides a hash space that the object storage handles as a resource identifier (step S0201). When the management unit divides the hash space, the management unit determines a virtual object storage in charge of each divided space. The specific method is described in the description part of the resource identifier.

  Next, the management unit determines an actual delivery destination when the REST I / F is used (step S0202). Following the above association, the management unit associates the virtual object storage node with the real storage node. When the real storage node is determined, the physical port number to be transmitted to the OFS can be specifically determined. In other words, the port to which the packet should be transferred is known. In this way, a flow entry for object storage can be created (step S0203).

  Since the distribution route of the entire network can be specified, the OpenFlow controller sets the Flow Table of the OpenFlow switch existing on the network so that it becomes the designed network. In the present embodiment, the OFS1 Flow Table is as shown in FIG. 10, the OFS2 Flow Table is as shown in FIG. 11, and the OFS3 Flow Table is as shown in FIG. become. 10, FIG. 11 and FIG. 12 are diagrams showing examples of a flow table.

  In Flow Table 0 to be executed first, a flow entry for determining whether or not the object storage is exchanged is described. Specifically, when the destination IP address of the packet corresponds to a specific IP address, the OpenFlow switch of the present embodiment interprets the communication using the received packet as the communication for object storage. This specific IP address is the above-described representative object storage address (hereinafter also referred to as “representative address”). Then, the OpenFlow switch of this embodiment advances the processing to Flow Table 1 describing the flow processing for object storage.

  If the destination IP address of the packet is not the representative object storage address, the destination IP address matches the flow entry of ANY. Therefore, the OpenFlow switch of this embodiment advances the processing to Flow Table2. The management unit sets an entry for the OpenFlow network in Flow Table2. The OpenFlow switch that has proceeded to Flow Table2 handles normal communication (other than object storage). In this way, by assigning the representative address at the beginning of the process, even when building an overlay network for another object storage, it is only necessary to insert the object storage flow created in the same way before ANY. Easy to expand.

  A description will be given of Flow Table 1 describing processing when the destination IP address is the representative object storage address. In Flow Table1, a flow for specifying a resource identifier is set in Match Fields, and an action for transmitting a packet corresponding to the condition to an appropriate port is set in Instructions. That is, Flow Table 1 describes a flow entry to be sent to the node in charge of the resource identifier.

  In the OpenFlow switches (OFS2 and OSF3) directly connected to the object storage node, the directly connected ports become transmission ports (FIG. 11: upper two flow entries, FIG. 12: lower two flow entries). In other entries, a port that is appropriately transferred to the OpenFlow switch to which the node in charge of the resource identifier is connected is specified.

  The OFC distributes the finally designed flow table to the OFS on the network (step S0204). By distributing the flow table, the designed OpenFlow network starts operating.

  The operation of the OpenFlow switch when a packet arrives will be described with reference to FIG. FIG. 13 is a flowchart showing an example of the operation of the OpenFlow switch when a packet arrives.

  When a packet arrives, the OpenFlow switch processes the arrived packet according to the Flow Table. As described above, the Flow Table is basically set by the OFC before the OFS operates.

  The OpenFlow switch first processes sequentially from Flow Table0. The OpenFlow switch checks whether it is a packet for object storage based on a flow entry on condition that the destination IP address matches the representative object storage address (step S0301). When the destination IP address of the packet corresponds to the representative object storage address (Yes in step S0301), the OpenFlow switch moves the process to Flow Table 1 (step S0302).

  Flow Table 1 describes a flow for checking resource identifiers. With the operation extension in this embodiment, the OpenFlow switch reads the resource identifier at the head of the payload. The OpenFlow switch checks whether the resource identifier corresponds to the Match Fields of the flow entry. When the resource identifier corresponds to the Match Fields of the flow entry, the OpenFlow switch transmits the packet to a predetermined port according to the output action set in the Instructions of the flow entry.

  In step S0301, when the resource identifier does not match the Match Fields of the flow entry (No in step S0301), the OpenFlow switch processes the packet as a normal packet (S0303). This is achieved when the OpenFlow switch advances the process to Flow Table 2 that stores normal OpenFlow processing entries by the flow of ANY in Flow Table 0.

  A client that uses object storage will be described. FIG. 14 is a flowchart illustrating an example of the overall operation of the client.

  The client exchanges data with the object storage through the Rest I / F. The client has a function of exchanging in the format for this network.

  In order to communicate with the object storage, the client creates a UDP socket (communication end) at the preparation stage (step S0401).

  A UDP socket can specify a destination address for each transmission. In communication with the object storage, the client designates the representative object storage address as the destination IP address. The client requests transmission of data according to the format of this network storing the request ID and resource identifier (step S0402).

  When the client stops using the object storage, in addition to terminating the created UDP socket, the client also terminates the function for object storage (step S0403).

  The operation of the object storage node will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of the operation of the object storage node.

  The node constituting the object storage (that is, the object storage node) creates a standby socket in the service preparation stage (step S0501). This socket is used for communication with clients.

  When the object storage node is ready to provide the object storage service, the object storage node starts providing the data service using the standby socket (step S0502).

  The object storage node ends the standby socket to finish providing the service. In addition, the object storage node sequentially ends the functions used for providing the service (step S0503).

  An operation in which a client exchanges object storage and data (that is, a resource in the present embodiment) using the Rest I / F via the network according to the present embodiment will be described with reference to FIG. FIG. 16 is a sequence diagram illustrating an example of an operation for exchanging data in the system according to the present embodiment.

  The generated resource identifier is assigned to the operation target resource (existing technology). The resource identifier may be generated by calculating a hash value of data held by the resource, or may be generated based on a path name for storing the resource.

  In the present embodiment, the above-described identifier called a request ID using numerical values is used for a request representing an operation on a resource.

  As described above, the client creates transmission data (for example, a payload portion) as follows. The client stores the request ID and resource identifier in the payload part in order from the top. Depending on the request, the client may include necessary data later in the payload portion. In the following description, the number in parentheses after the request ID is a numerical value given as a request ID to each operation in the present embodiment.

  For example, in the case of a GET operation, the client makes a transmission request using the GET request ID (1) and the resource identifier as transmission data.

  In the HEAD operation, the client makes a transmission request using the HEAD request ID (2) and the resource identifier as transmission data.

  In the case of the POST operation, the client makes a transmission request using the POST request ID (3), the resource identifier, and the data to be stored in the object storage as transmission data.

  In the case of the PUT operation, the client makes a transmission request using transmission data that includes the PUT request ID (4) and the resource identifier followed by data to be stored in the object storage.

  In the case of a DELETE operation, the client makes a transmission request using the DELETE request ID (5) and the resource identifier as data.

  OFS1 and OFS2 transfer the packet according to the transfer destination of the resource identifier after confirming whether the destination IP address of the packet is the representative object storage address. Eventually, the responsible node receives the packet. Here, it is assumed that the object storage 1 receives a request.

  The object storage node that receives the packet processes the request based on the request ID and the resource identifier. The technology of the Rest I / F execution part other than the technology that handles the data format specific to the network according to the present embodiment may be the same as the normal technology.

  An example of the operation is shown below. In the following description, the numerical value in parentheses represents the numerical value set as the request ID. Hereinafter, for example, “when the request ID is GET (1)” represents “the request ID is 1 and the type of operation indicated by the request ID is GET”.

  When the request ID is GET (1), the object storage 1 retrieves the data of the resource specified by the resource identifier from the storage device and transmits the data to the client.

  When the request ID is HEAD (2), the object storage 1 extracts the meta information of the resource specified by the resource identifier from the storage device or the file system, and transmits the extracted data to the client.

  When the request ID is POST (3), the object storage 1 writes the data simultaneously passed as the resource specified by the resource identifier as an object to the storage device. When the writing is completed, the object storage 1 transmits a writing completion notification to the client.

  When the request ID is PUT (4), the object storage 1 writes the data passed at the same time as the resource specified by the resource identifier as an object to the storage device. When the writing is completed, the object storage 1 transmits a writing completion notification to the client.

  When the request ID is DELETE (5), the object storage 1 deletes the resource specified by the resource identifier. When the deletion is completed, the object storage 1 transmits a completion notification to the client.

  When responding to the client, if the source information is an IP address used by the storage node and is different from the representative address, a problem (for example, being caught by a network filter) may occur.

  There are the following solutions for this.

Solution 1. Expansion of communication format (A) An identifier updated every time a communication request is made in the Rest I / F format or communication format, and the storage node returns an identifier delayed when responding. This mechanism makes it possible to recognize a request even if the transmission source address is different from the representative storage address. (Application response 1: Network File System / Remote Procedure Call (NFS / RPC) mechanism)
Solution 2. Address Conversion (A) The storage node has an address translation mechanism (NAT, Network Address Translation), and when transmitting, converts the source address to a representative storage address and transmits it. (Use of OS mechanism)
(B) The raw socket is used for the response of the storage node, and the storage node transmits the source address as the representative storage address. (Application response 2)
(C) In the OpenFlow switch, when the port number used between the object storage and the client and the destination address are not the representative storage address, the OpenFlow switch converts the source address to the representative storage address and transfers it. (Corresponding on the network route)
You may also go into the OS platform or communication library to solve this problem.

<Description of effects>
An ordinary distributed storage has a node management function because it needs to manage nodes participating in the object storage. According to this embodiment, node management can be performed in the network, so that the management functions that the distributed object storage must have can be greatly reduced.

  Speeding up communication processing with distributed storage: Generally, it is necessary to find a node in charge of a resource and communicate with the node in charge of the resource. In this embodiment, the resource may be transmitted to a representative address. After that, resources are delivered sequentially by the OpenFlow switch.

  In addition, the communication performance is improved because the OpenFlow switch can perform delivery processing at extremely high speed.

The node in charge of the resource can be determined more flexibly: It is possible to assign the actual node in charge by the distributed storage arrangement information management means. In other words, it has the flexibility to determine the responsible node of the resource more directly than the general method. (For example, there are no restrictions such as having to divide the space equally.)
The node in charge can be switched quickly: If the connection partner is managed by a client or node, they must be changed sequentially. However, in this embodiment, it is only necessary to change the Flow Table of the OpenFlow switch, so that switching can be performed quickly. The effect increases as the number of nodes and clients involved increases.

  Also, with the centralized management method using a metadata server, there is a problem that the failure of the metadata server is spread to the entire system. Unlike this, this embodiment has the advantage that the service can be continued even if the OpenFlow controller goes down. . In this embodiment, even if the OpenFlow controller goes down, the path cannot be changed. Communication will not stop immediately because the OpenFlow switch that has been set will continue processing.

<< Related technology >>
In the present embodiment, there is no limit on the number of OpenFlow controllers and the number of OpenFlow switches. The present embodiment can also be implemented in a network constituted by OpenFlow controllers and OpenFlow switches, which is different from the number of OpenFlow controllers and the number of OpenFlow switches.

  The configuration of the system according to the present embodiment may be configured to make data redundant by duplicating a packet in the OpenFlow switch and delivering the packet to a plurality of object storage nodes. Further, the configuration of the system according to the present embodiment may be extended to a configuration having two or more overlay networks. It is also conceivable to support the addition and deletion of storage nodes by having as a generation a plurality of states of the storage network that have changed due to the addition and deletion of storage nodes.

  The flow table of the OpenFlow switch may be automatically assembled by a module that extends the OpenFlow controller or a module that operates in cooperation with the OpenFlow controller.

<Related technologies>
Related 1: Implementation example for expanding data length (for problems involving fragments)
For UDP datagrams and TCP streams, a size larger than the MTU can be specified for a single transmission request. Depending on the size of the transmission data, network processing called fragmentation is performed in the lower layer. In the present embodiment, for simplification, the data size performed in one exchange is limited to MTU or less. However, considering the handling of data larger than the MTU, such as image data, it is necessary to expand the data length so that it can handle a data length larger than the MTU.

  The problem when a fragment occurs will be described with reference to FIG. In the description of this embodiment, by passing data with a resource request ID and resource identifier at the beginning at the time of a transmission request, the UDP data graph immediately follows the protocol header (UDP header), that is, at the beginning of the payload, the resource request and resource identifier. Is stored. When a fragment is generated here, the resource request and the resource identifier are stored only in the first divided packet, and thus there arises a problem that the transfer destination cannot be specified only by the subsequent packet.

  There are several solutions to this problem.

(Solution 1)
Solution 1 extends the format used in object storage. That is, in addition to the resource ID and request identifier, a data offset (length from the beginning of a series of data) and data size are added.

  If the transmission side has a data size that cannot be handled by one packet, it is divided into a size that fits in one packet, the starting point of the divided data is stored as an offset, and the data length from the offset is stored as a data size. You can solve this problem by sending it with the appropriate data.

(Solution 2)
Solution 2 is to store resource identifier fragment information (first bit) in a protocol header that is not used, such as a VLAN tag, and expand it so that OFS can determine only by looking at the relevant area of the packet.

  A normal operating system mounted on a computer performs protocol processing using a network module inside the kernel. Therefore, the above-described Solution 2 can be realized by modifying the client network module so as to correspond to the network of the present embodiment. In addition, in order for OFS to check whether the resource is a resource by communication with object storage, a flow that also sees the corresponding area is added to Flow Table 0, and if applicable, it branches to Flow Table 1 that handles communication for object storage It is necessary to set as follows.

(Solution 3)
Solution 3 is to once assemble the fragment packet inside the OFS, recognize the transfer destination, and then divide and transfer again.

  It is possible to determine a fragmented packet by referring to the fragment bit of the IP header. If the packet is fragmented, OFS only needs to obtain the resource identifier of the packet group by waiting for transfer of the packet and returning it to the datagram. After that, OFS may divide the datagram again and send the packet to the transfer destination obtained from the resource identifier.

  This method has the disadvantage that the memory usage inside the OFS increases and the performance decreases. However, there is an advantage that the function addition to the client and the object storage node can be suppressed.

(Solution 4)
Solution 4 is extended to detect the communication partner, and the actual exchange is performed using information obtained from the representative object storage address instead of the representative object storage address. This method is possible not only for UDP but also for communication using TCP.

  The case of using UDP will be described. In Solution 4, a connection request is added to the resource request. The client and the object storage node need to know the connection request. FIG. 18 is a sequence diagram illustrating an example of an operation using a connection request in the system according to the present embodiment.

  The client transmits an operation target resource identifier and a connection request to the representative object storage address.

  OFS refers to the resource identifier and delivers the request to the object storage node in charge of the resource identifier.

  The responsible object storage node confirms the request ID, and if the request is a connection request, it sends communication information such as its own IP address and port number to the client.

  The client that has received the communication information transmits using the communication information instead of the representative object storage address. The client creates a socket if necessary for communication.

  In communication using TCP, sequence numbers and ack numbers are managed for each socket (communication end), and it is not possible to communicate with different parties using the same socket as in UDP. As a result, communication using TCP has the problem that the same socket cannot be used when communicating with distributed nodes. This problem can also be solved by adding a connection request to the resource request and calling it back. FIG. 19 is a sequence diagram illustrating an example of an operation to which a solution is applied in the system according to the present embodiment.

  At the time of the initial request, the client stores the connection request and the resource ID to be operated in the request payload, and transmits the request to the object storage network.

  OFS sends a request from the resource ID to the object storage node in charge.

  The object storage node confirms the request ID, and if the request is a connection request, creates a TCP socket having the client IP address as the destination address. Once the client and object storage node have established a TCP session, they exchange a Rest I / F and delete the socket after the exchange.

  The session may be established from the object storage node side. It may be stretched from the client side. In addition, if the creation of a socket is a burden, it is possible to manage the socket so that it can be reused.

Related 2: Constructing Method for Object Storage Overlay Network In this embodiment, OpenFlow is extended so that information for object storage can be read as construction of an overlay network for object storage. Here, as another method, a method of utilizing an existing protocol field is described. According to this method, since the existing field is used, the OpenFlow switch itself has an advantage that the current product can be used as it is.

In the present embodiment, no header for object storage is provided on the payload. Used by unused areas in the protocol header. As an example, a VLAN (Virtual Local Area Network) header is used. In OpenFlow, the following fields related to VLANs are already available for flow matching conditions.
VLAN ID: 12bit
VLAN priority: 3bit
In the present embodiment, the VLAN priority field is used as the request identifier, and a value not used in the VLAN ID is used as the resource identifier. Since the bit length of the field is small, the entire resource identifier cannot be stored. Therefore, only information that can be identified by the virtual object storage is stored in the field. In the present embodiment, the first 4 bits of the resource identifier are used. Similarly, the first 4 bits are used and assigned to (1000 to 1016) in this example. (VLAN ID 0 cannot be used due to restrictions. In this example, it is assigned in order from 1000.)
In designing the overlay network, the OpenFlow controller divides the resource identifier space and associates the area with the virtual object storage as described in the present embodiment. Thereafter, the OpenFlow controller associates the virtual object storage with the real storage. In this embodiment, the value indicated by the first bit of the resource identifier may be associated with the VLAN ID.

  When the association is completed, the OpenFlow controller creates a Flow Table using the VLAN ID as a matching condition so that it is next delivered to the real storage node in charge. That is, the OpenFlow controller creates an overlay network for object storage based on the VLAN ID and priority. (The resource identifier flow matching condition in Flow Table 1 uses VLAN ID and priority).

  At the time of transmission, the object storage client sets the first few bits of the resource identifier (depending on the number of virtual storage nodes to be used) to the VLAN ID, and sets the request identifier to the VLAN priority. By doing this, the OpenFlow switch can finally transfer the packet to the machine responsible for the resource only by transferring the packet based on the representative object storage address, VLAN ID, and priority. Become.

  Note that the details of the TCP / IP protocol cannot be set by application programs, and are often set inside the OS. Therefore, if such a communication mechanism is provided as a function of the OS, the above method can be implemented more easily than the case where the application handles it. When implemented on the application program side, implementation using an existing communication mechanism called raw socket is possible. If you use raw socket, you can set the protocol header freely. A configuration that mixes OS and application programs is also conceivable.

(Support for various object storage formats)
There are various data formats used in the object storage even if they are called by the same name “object storage”. That is, the data format used is not unified. Here, as a further contrivance, a method of using a communication module corresponding to various formats of object storage will be described. The matter described immediately before is also taken into consideration.

  In the present embodiment described so far, the communication module in the object storage converts the format according to the network to be used. The mechanism in which the object storage itself is not involved in the format used and an external communication module converts to a communication format in a timely manner will be described.

  As the form, a form used in a library form, a form providing format conversion as a function of the OS, and a composite form thereof are conceivable.

  Below, the format which incorporates the function which converts a format as a library is demonstrated as an example. Other forms can be realized without much change. FIG. 20 is a diagram illustrating an example of a configuration in which a format conversion function is incorporated in a library according to the system of the present embodiment.

  The communication management unit in FIG. 20 is a mechanism for managing communication. Information necessary for communication is stored in a management structure called a communication handler. In addition, an ID that can be uniquely specified is assigned to a communication handler when it is created. The communication management unit has a role of performing handler management operations such as creation and disposal of handlers and a function of passing information held by a handler with a specified ID.

  At the time of initialization, the object storage requests the communication management unit to create a communication handler (FIG. 21). FIG. 21 is a sequence diagram illustrating an example of an operation of requesting the communication module to explain the communication in the system according to the present embodiment. When the object storage requests creation of a communication handler, it also passes information for transmission such as a representative object storage address as an argument. The communication management unit records the communication information passed at the time of the handler creation request in the communication handler, and then creates a socket used for communication. When the handler is created, the handler ID is returned to the requesting application. The object storage makes a request by specifying the handler ID received during the subsequent communication.

  Next, object storage communication using the communication module will be described. FIG. 22 is a sequence diagram illustrating an example of object storage communication using a communication module.

  When transmission is performed using this communication module, a transmission request is issued from the transmission request I / F prepared by the input / output receiving unit. The transmission request I / F passes the communication handler ID and the data formatted in the object storage format to the input / output receiving unit. When passing data to the input / output receiving unit, both the resource ID and the request ID may be passed. In this way, format conversion described later can be performed more easily.

  When receiving a transmission request, the communication module obtains a resource identifier and a request ID from the format of the passed data. Next, the communication module converts the format of the passed data into a communication format used in this embodiment.

  In this embodiment, a request ID and a resource identifier are described before the data format.

  In another embodiment, a part of the request ID may be processed and stored in the VLAN ID, and the resource identifier may be stored in the VLAN priority.

  The communication module describes the network header such as IP and UDP necessary for communication after converting the format for the overlay network. Next, the communication module makes a transmission request to the communication layer of the OS using the raw socket of the communication handler, and transmits it to the network.

  When the transmission is completed, the communication module returns a transmission completion notification to the transmission request source.

  Next, the reception operation will be described. The object storage program executes the reception interface prepared by the input / output reception unit with the handler ID and the memory address for the reception buffer as arguments. The reception interface checks whether the socket of the handler specified by the handler ID is received, and if received, converts the received data to a format used by the host.

  In this embodiment, since the header used in the overlay network is added to the beginning of the data, the object storage program deletes the added header.

  In other embodiments (for example, an example using VLAN), there is no added header, so the data area may be used as it is.

  The object storage program writes the converted data in the reception buffer passed as an argument. Finally, the reception function is returned, and the reception mechanism on the object storage program side operates.

(Other realization methods: network side)
This embodiment can also be realized by extending only the network side.

  1. When a packet of a specific network port arrives, it is determined as object storage communication and the packet is analyzed.

(A) If there is information in the protocol field used in (B), (B) is not implemented. (A packet with information in the protocol field is a packet transferred from another switch.)
(B) If there is no information in the protocol field, format analysis is performed to obtain information such as a necessary resource ID.

  As the format analysis method, there are (1) a method for analyzing a fixed length range from the head, and (2) a form that is greatly expanded so that the format of the object storage can be analyzed.

As in previous implementations, unused protocol fields are used and the virtual object storage information is written into the protocol fields. This writing eliminates the need to always perform an analysis operation with a switch. (This is realized by the mechanism of Group Table)
2. Based on the information in the protocol field, transfer is performed in the overlay network (as described above). Finally, deliver the packet to the destination.

  This embodiment can also be realized by extending only the network side as described above. As described above, the present embodiment has been described by taking OpenFlow as an example. However, the technology used for constructing the overlay network for the storage network is not limited to OpenFlow, and may be another technology for realizing SDN, for example.

<< Second Embodiment >>
Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

<Configuration>
FIG. 23 is a block diagram showing the configuration of the storage system 1 according to this embodiment. The storage system 1 of this embodiment schematically represents the first embodiment.

  Referring to FIG. 23, the storage system 1 according to the present embodiment includes a communication management device 10, a plurality of communication devices 20, and a plurality of real storage nodes 30. The setting terminal device 50 may be connected to the communication management device 10 so as to be communicable.

  The communication device 20 is, for example, an exchange that also operates as a general OpenFlow switch. Each communication device 20 is communicably connected to at least one other communication device 20. The real storage node 30 is communicably connected to any one of the communication devices 20. The communication device 20 implements a communication network 40 that mediates communication between each real storage node 30 and a device that can communicate with any one of the communication devices 20 (for example, the information processing device 60). In other words, the communication device 20 realizes a communication network 40 in which a connected device can communicate via at least one communication device 20. The real storage node 30 is connected to the communication network 40. The information processing device 60 is also connected to the communication network 40. The communication network 40 is a storage overlay network in the first embodiment.

  The communication management device 10 and the communication device 20 are further communicably connected via a route independent of the communication network 40.

  The communication management device 10 includes a determination unit 11, a generation unit 12, an application unit 13, an input / output unit 14, and a storage unit 15. The communication management device 10 is a control device that operates as a general OpenFlow controller. These units of the communication management apparatus 10 mainly correspond to a function (that is, a distributed storage arrangement information management unit) that manages the arrangement information of the distributed storage in the first embodiment.

  For example, the setting terminal device 50 communicates the connection relationship between the plurality of communication devices 20 and the plurality of real storage nodes 30 and the allocation of the virtual storage nodes to the real storage nodes 30 input by the administrator of the storage system 1. It transmits to the management apparatus 10.

  Each communication device 20 includes a plurality of ports. Another communication device 20 or real storage node 30 can be connected to one port. The connection relationship between the plurality of communication devices 20 and the plurality of real storage nodes 30 includes, for example, a port of each communication device 20 to which another device is connected and another device connected to the port. Information to identify. The above-mentioned “other device” refers to either one of the other communication devices 20 or one of the real storage nodes 30. The assignment of the virtual storage node to the real storage node 30 is information that can identify the virtual storage node assigned to each real storage node 30, for example. A plurality of virtual storage nodes may be assigned to one real storage node 30.

  The input / output unit 14 of the communication management device 10 receives, from the setting terminal device 50, the connection relationship of the communication device 20, the connection relationship between the communication device 20 and the real storage node 30, and the allocation of the virtual storage node to the real storage node 30. Receive. The communication management apparatus 10 stores the received connection relationship of the communication device 20, the connection relationship between the communication device 20 and the real storage node 30, and the allocation of the virtual storage node to the real storage node 30 in the storage unit 15. . In the following description, the allocation of the virtual storage node to the real storage node 30 is also expressed as “third allocation relationship”.

  The storage unit 15 stores, for example, the connection relationship between the communication devices 20, the connection relationship between the communication device 20 and the real storage node 30, and the allocation of virtual storage nodes to the real storage node 30.

  The information processing device 60 is a device that interacts with the object storage. When the information processing apparatus 60 interacts with the object storage, the information processing apparatus 60 transmits the contents of the object to the storage system 1 depending on the resource identifier and the type of request. When the information processing apparatus 60 stores an object in the storage system 1, for example, the information processing apparatus 60 may transmit an object to which a resource identifier is uniquely assigned to the storage system 1. The object transmitted to the storage system 1 may be given the representative address of the storage system 1 as the destination address. The representative address may be determined in advance. The communication device 20 that has received the object replaces the device connected to one of the ports (that is, the other communication device 20 or the real storage node 30) with the object in accordance with the settings applied to the communication device 20. Is determined as the transfer destination. The communication device 20 transfers the received object to the determined transfer destination. When the transfer destination is the real storage node 30, the real storage node 30 receives the object and stores the received object. The settings applied to the communication device 20 are generated as described below, for example. The setting may be represented as the above-described Flow Table, for example.

  The determination unit 11 determines a first allocation relationship in which the resource identifier is allocated to any of the plurality of real storage nodes. Specifically, the determination unit 11 may determine a second allocation relationship in which the resource identifier is allocated to one of the plurality of virtual storage nodes based on the used part, for example, by the above-described consistent hashing. When the resource identifier is assigned as a hash value, a second assignment relationship assigned to either a part of the resource identifier (for example, a predetermined number of bits at the beginning, hereinafter also referred to as “used portion”) and a virtual storage node is set. You may decide. The determination unit 11 may determine a used portion (the number of leading bits in the above example) based on the number of virtual storage nodes. The determination unit 11 may determine the minimum exponent of 2 that is not smaller than the number of virtual storage nodes as the number of bits in the used portion. The relationship between the resource identifier and the real storage node 30 is specified by the second allocation relationship and the third allocation relationship described above. The determination unit 11 may determine the first allocation relationship based on the second allocation relationship and the third allocation relationship described above.

  The generation unit 12 stores data representing a setting including a rule for determining the transfer destination of the object based on a resource identifier uniquely given to the object, in the first allocation relationship described above, for example, in the storage unit 15. It is generated based on the connection relationship of the communication device 20. In the following description, data representing settings is simply referred to as “settings”. A rule for determining a transfer destination of an object based on a resource identifier uniquely assigned to the object is referred to as a “first rule”.

  A plurality of real storage nodes 30 may be assigned to one virtual storage node. In that case, a plurality of real storage nodes 30 are allocated to one resource identifier in the first allocation relationship. In other words, the first allocation relationship may include an allocation that allocates resource identifiers to two or more real storage nodes. This represents that one object is transmitted to two or more real storage nodes. In that case, at least one of the communication devices 20 transfers the same object to two or more transfer destinations. If the generation unit 12 generates a setting including a first rule that determines two or more transfer destinations as a transfer destination of an object as a setting of the communication device 20 that transfers the same object to two or more transfer destinations. Good. In that case, the communication device 20 may copy the objects so that the number of the same objects is two or more. The generation unit 12 further sets a rule (hereinafter referred to as “second rule”) for copying the object so as to have the same number as the number of transfer destinations as the setting of the communication device 20 in which the number of transfer destinations is two or more. It is only necessary to generate a setting including this.

  The generation unit 12 may generate a setting including the third rule that determines the transfer destination of the object according to the first rule when the representative address is specified as the address of the object transmission destination. Good. The third rule is that the communication device 20 is set to operate as a normal OpenFlow switch based on the Flow Table when the above-described representative address is not designated as the destination address of the object. Just do it.

  The application unit 13 applies the setting generated by the generation unit 12 for each communication device 20 to the communication device 20. Specifically, the application unit 13 transmits settings (that is, data representing settings) to the communication device 20.

  Next, the configuration of the communication device 20 will be described in detail with reference to the drawings.

  FIG. 24 is a block diagram illustrating an example of a configuration of the communication device 20 according to the present embodiment.

  Referring to FIG. 24, the communication device 20 includes a setting reception unit 21, a transfer destination determination unit 22, and an input / output unit 23.

  The setting reception unit 21 of the communication device 20 receives the setting from the application unit 13 of the communication management apparatus 10. The setting receiving unit 21 transmits the received setting to the transfer destination determining unit 22.

  The input / output unit 23 receives an object from a device (the information processing device 60, another communication device 20, or the real storage node 30) connected to any port.

  The transfer destination determination unit 22 determines the transfer destination of the received object according to the received setting.

  The input / output unit 23 transfers the received object to the transfer destination determined by the transfer destination determination unit 22.

<Operation>
Next, the operation of the communication management apparatus 10 of this embodiment will be described in detail with reference to the drawings.

  FIG. 25 is a flowchart showing an example of the operation of the communication management apparatus 10 of the present embodiment.

  In the operation illustrated in FIG. 25, the input / output unit 14 determines the connection relationship between the plurality of communication devices 20 and the plurality of real storage nodes 30 and the allocation of the virtual storage nodes to the real storage nodes (that is, the third allocation relationship). Receive (step S101). The input / output unit 14 stores the received connection relationship and the third allocation relationship in the storage unit 15. The real storage node corresponds to the object storage node of the first embodiment.

  For example, the determination unit 11 determines the used part of the resource identifier based on the number of virtual storage nodes (step S102). For example, the determination unit 11 determines the allocation of resource identifiers to the virtual storage node (that is, the second allocation relationship) based on the determined used portion (step S103). The determination unit 11 determines the allocation of resource identifiers to the real storage nodes (that is, the first allocation relationship) based on the second allocation relationship and the third allocation relationship (step S104).

  Next, the generation unit 12 uniquely identifies the object based on the connection relationship received by the input / output unit 14 and stored in the storage unit 15 and the first allocation relationship determined by the determination unit 11. A setting including the first rule for determining the transfer destination of the object based on the given resource identifier is generated (step S105). As described above, the first rule is a rule that determines a transfer destination of an object based on a resource identifier uniquely given to the object. The generation unit 12 performs a setting including a first rule for determining two or more transfer destinations and the above-described second rule for an object assigned a resource identifier to two or more real storage nodes. It may be generated. As described above, the second rule is a rule for copying an object so that the number of objects is the same as the number of transfer destinations, and transferring the objects to each of the transfer destinations. The generation unit 12 determines the transfer destination of the object according to the first rule when the transmission destination of the object is the above-described representative address. Otherwise, the generation unit 12 determines the transfer destination based on the flow table of the normal Open Flow Network. A third rule to be determined may be generated.

  Next, the application unit 13 applies the generated setting to the communication device 20 (step S106). For example, the application unit 13 transmits the generated setting to the communication device 20. The setting receiving unit 21 of the communication device 20 receives the setting. The transfer destination determination unit 22 determines a transfer destination according to the received setting.

  Next, the operation of the communication device 20 of the present embodiment will be described in detail with reference to the drawings.

  FIG. 26 is a flowchart illustrating an example of the operation of the communication device 20 according to the present embodiment. The example of the operation illustrated in FIG. 26 is an example of the operation of the communication device 20 that operates according to the setting including the above-described first, second, and third rules.

  According to FIG. 26, first, the input / output unit 23 receives an object to which a resource identifier that is unique for the object is assigned (step S201). The destination address is further given to the object.

  The transfer destination determination unit 22 determines whether or not the destination address is a representative address (step S202). If the destination address is not a representative address (NO in step S202), the transfer destination determining unit 22 operates as a general OpenFlow switch, thereby determining the transfer destination of the received object based on the destination address. (Step S207). Then, the input / output unit 23 transfers the received object to the determined transfer destination (step S207).

  When the destination address is a representative address (YES in step S202), the transfer destination determination unit 22 determines the transfer destination of the received object based on the resource identifier in accordance with the setting received from the communication management apparatus 10 ( Step S203).

  When two or more transfer destinations are not determined (NO in step S206), the input / output unit 23 transfers the received object to the determined transfer destination (step S207).

  When two or more transfer destinations are determined (YES in step S206), the input / output unit 23 duplicates the objects so that the number of objects is equal to the number of transfer destinations (step S205). Then, the input / output unit 23 transfers the object to the determined transfer destination (step S207).

<Effect>
The present embodiment described above has an effect that the path can be switched at high speed and flexibly while reducing the management load in the distributed storage. The reason is that the application unit 13 applies the setting for determining the transfer destination of the object generated by the generation unit 12 to the communication device 20. For this reason, the communication management apparatus 10 does not need to determine the transfer destination each time an object is transferred in the communication device 20, so that the load on the communication management apparatus 10 is reduced. In addition, since it is not necessary for each communication device 20 to grasp the responsibility of the participating node and the object, the burden on the communication device 20 is reduced. Further, when a change is made to the communication device 20 that participates in the system, such as addition of the communication device 20, the setting is transmitted to each communication device 20 by the communication management apparatus 10, so that the time is shortened until the setting is fully transmitted Is done.

<< Third Embodiment >>
Next, a third embodiment of the present invention will be described in detail with reference to the drawings.

<Configuration>
FIG. 27 is a block diagram illustrating an example of the configuration of the communication management apparatus 10A of the present embodiment.

  Referring to FIG. 27, the communication management device 10 </ b> A includes a generation unit 12 and an application unit 13.

  The generation unit 12 sets the setting of the plurality of communication devices in the path for transmitting the object to any one of the plurality of real storage nodes, to the first allocation relationship and the connection relationship between the plurality of communication devices and the plurality of real storage nodes. Generate based on. The setting includes a first rule that determines a transfer destination of the object based on an identifier uniquely given to the object. The first allocation relationship represents an allocation in which the identifier is allocated to any of the plurality of real storage nodes.

  The application unit 13 applies the setting to the communication device.

<Operation>
Next, the operation of the communication management apparatus 10A of the present embodiment will be described in detail with reference to the drawings.

  FIG. 28 is a flowchart illustrating an example of the operation of the communication management apparatus 10A according to the present embodiment.

  Referring to FIG. 28, the generation unit 12 generates the communication device setting described above (step S301). The production | generation part 12 should just produce | generate a setting for every communication apparatus. Next, the application unit 13 applies the setting to the communication device (step S302). The application unit 13 may apply the generated setting for each communication device.

<Effect>
The present embodiment described above has the same effect as the second embodiment. The reason is the same as the reason for the effect of the second embodiment.

<< Other Embodiments >>
The apparatus according to each of the embodiments described above can be realized by a computer and a program for controlling the computer, dedicated hardware, or a combination of the computer and the program for controlling the computer and dedicated hardware. The apparatus according to each of the above embodiments includes, for example, the OFS, OFC, and cooperative server according to the first embodiment, the communication management apparatus according to the second and third embodiments, and the communication according to the second embodiment. Equipment.

  In other words, the device according to each of the above-described embodiments can be realized by hardware such as a circuit configuration. The circuit configuration may be, for example, a processor and a memory included in the computer. In that case, the program only needs to be loaded into the memory. The program can be executed by a processor, and the computer may be operated as an apparatus according to each of the above-described embodiments. The circuit configuration may be, for example, a plurality of computers that are communicably connected. The circuit configuration may be, for example, a circuit. The circuit configuration may be, for example, a plurality of circuits that are communicably connected. The circuit configuration may be a combination of one or more computers and one or more circuits that are communicably connected.

  FIG. 29 is a diagram illustrating an example of a hardware configuration of a computer 1000 that can implement the apparatus according to each embodiment of the present invention. Referring to FIG. 29, a computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, and an I / O (Input / Output) interface 1004. The computer 1000 can access the recording medium 1005. The memory 1002 and the storage device 1003 are storage devices such as a RAM (Random Access Memory) and a hard disk, for example. The recording medium 1005 is, for example, a storage device such as a RAM or a hard disk, a ROM (Read Only Memory), or a portable recording medium. The storage device 1003 may be the recording medium 1005. The processor 1001 can read and write data and programs from and to the memory 1002 and the storage device 1003. The processor 1001 can access other devices via the I / O interface 1004. The processor 1001 can access the recording medium 1005. The recording medium 1005 stores a program that causes the computer 1000 to operate as the communication management apparatus 10. The recording medium 1005 may store a program that causes the computer 1000 to operate as the communication management apparatus 10A. The recording medium 1005 may store a program that causes the computer 1000 to operate as the communication device 20. The recording medium 1005 may store a program that causes the computer 1000 to operate as the OFC of the first embodiment. The recording medium 1005 may store a program that causes the computer 1000 to operate as the OFS of the first embodiment. The recording medium 1005 may store a program that causes the computer 1000 to operate as the cooperative server of the first embodiment.

  The processor 1001 loads the above-described program stored in the recording medium 1005 into the memory 1002. Then, when the processor 1001 executes the program loaded in the memory 1002, the computer 1000 operates as one of the above-described devices.

  The determination unit 11, the generation unit 12, the application unit 13, the input / output unit 14, the setting reception unit 21, the transfer destination determination unit 22, and the input / output unit 23 are, for example, a memory 1002 and a dedicated load loaded in the memory 1002. It can be realized by a processor 1001 that executes a program. Further, the storage unit 15 can be realized by a memory 1002 included in the computer 1000 or a storage device 1003 such as a hard disk device. Alternatively, the determination unit 11, the generation unit 12, the application unit 13, the input / output unit 14, the storage unit 15, the setting reception unit 21, the transfer destination determination unit 22, and the input / output unit 23 may be partially or entirely It can also be realized by a dedicated circuit for realizing the function of each part.

  Moreover, although a part or all of said embodiment can be described also as the following additional remarks, it is not restricted to the following.

(Appendix 1)
A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generating means for generating the identifier based on a first allocation relationship for allocating to any of the plurality of real storage nodes and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
Applying means for applying the setting to the communication device;
A communication management device comprising:

(Appendix 2)
The first allocation relationship includes an allocation in which the identifier is allocated to two or more real storage nodes among the plurality of real storage nodes;
The generation means determines the two or more transfer destinations for the object in which the identifier is allocated to two or more real storage nodes as the setting of any of the communication devices. The setting including a rule and a second rule that duplicates the object so that the number of the object is the same as the number of the transfer destinations and transfers the object to each of the transfer destinations is generated. The communication management device described.

(Appendix 3)
The object transmitted to one of the plurality of real addresses is given a representative address as a destination address,
The generation unit further includes a third rule for determining the transfer destination of the object according to the first rule when an address given to the object as a transmission destination is the representative address. The communication management device according to attachment 1 or 2.

(Appendix 4)
The generation unit sets the setting including the first rule for determining a transfer destination of the object based on a used part which is a part of the identifier, and sets the used part to any of the plurality of real storage nodes. The communication management device according to any one of supplementary notes 1 to 3, wherein the communication management device is generated based on the first assignment relationship to be assigned.

(Appendix 5)
Determining the used portion in the identifier based on the number of virtual storage nodes; determining a second allocation relationship for assigning the identifier to any of the plurality of virtual storage nodes based on the used portion; Determining means for determining the first allocation relationship based on the second allocation and a third allocation relationship in which each of the plurality of virtual storage nodes is allocated to any of the plurality of real storage nodes; The communication management apparatus according to appendix 4.

(Appendix 6)
The communication management device according to any one of appendices 1 to 5,
The communication device transferring the object according to the setting;
A communication system including:

(Appendix 7)
Transfer destination determining means for determining a transfer destination of the object in accordance with the setting generated by the communication management device according to any one of appendices 1 to 5;
Input / output means for receiving the object and transmitting the object to the transfer destination determined by the transfer destination determination means;
A communication device comprising:

(Appendix 8)
A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generating the identifier based on a first allocation relationship for allocating to any of the plurality of real storage nodes and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
Applying the setting to the communication device;
Communication management method.

(Appendix 9)
The first allocation relationship includes an allocation in which the identifier is allocated to two or more real storage nodes among the plurality of real storage nodes;
The first rule for determining two or more transfer destinations for the object assigned with two or more real storage nodes as the setting of any of the communication devices, and the object 9. The communication management method according to claim 8, wherein the object is copied so that the number of transfer destinations is equal to the number of transfer destinations, and the setting including the second rule for transferring the object to each of the transfer destinations .

(Appendix 10)
The object transmitted to one of the plurality of real addresses is given a representative address as a destination address,
The setting is generated further including a third rule for determining the transfer destination of the object according to the first rule when the address given to the object as the transmission destination is the representative address. The communication management method described in 1.

(Appendix 11)
Assigning the setting including the first rule for determining a transfer destination of the object based on a used part that is a part of the identifier to assign the used part to any of the plurality of real storage nodes The communication management method according to any one of appendices 8 to 10, wherein the communication management method is generated based on an allocation relationship.

(Appendix 12)
Determining the used portion in the identifier based on the number of virtual storage nodes; determining a second allocation relationship for assigning the identifier to any of the plurality of virtual storage nodes based on the used portion; The first allocation relationship is determined based on the second allocation and a third allocation relationship in which each of the plurality of virtual storage nodes is allocated to any of the plurality of real storage nodes. Communication management method.

(Appendix 13)
On the computer,
A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generation processing for generating the identifier based on a first allocation relationship for assigning to any of the plurality of real storage nodes, and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
An application process for applying the setting to the communication device;
A program that executes

(Appendix 14)
The first allocation relationship includes an allocation in which the identifier is allocated to two or more real storage nodes among the plurality of real storage nodes;
The generation process determines the two or more transfer destinations for the object assigned the identifier to two or more real storage nodes as the setting of any of the communication devices. The setting including the rule and the second rule for duplicating the object so that the number of the object is the same as the number of the transfer destinations and transferring the object to each of the transfer destinations is generated. The program described.

(Appendix 15)
The object transmitted to one of the plurality of real addresses is given a representative address as a destination address,
The generation process further includes a third rule for determining the transfer destination of the object according to the first rule when an address given to the object as a transmission destination is the representative address. The program according to appendix 13 or 14.

(Appendix 16)
In the generation process, the setting including the first rule for determining a transfer destination of the object based on a used part which is a part of the identifier is set to one of the plurality of real storage nodes. The program according to any one of appendices 13 to 15, which is generated based on the first allocation relationship to be allocated.

(Appendix 17)
On the computer,
Determining the used portion in the identifier based on the number of virtual storage nodes; determining a second allocation relationship for assigning the identifier to any of the plurality of virtual storage nodes based on the used portion; A determination process for determining the first allocation relationship based on the second allocation and a third allocation relationship in which each of the plurality of virtual storage nodes is allocated to any of the plurality of real storage nodes; The program according to supplementary note 16 to be executed.

  The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Storage system 10 Communication management apparatus 10A Communication management apparatus 11 Determination part 12 Generation part 13 Application part 14 Input / output part 15 Storage part 20 Communication apparatus 30 Real storage node 40 Communication network 50 Setting terminal apparatus 60 Information processing apparatus 101 OpenFlow Network
102 Storage Overlay Network
103 OpenFlow controller
104 Distributed storage arrangement information management means 105 Object Storage
106 Client
107 Other Server
201 OSF
202 OSF
203 OSF
1000 Computer 1001 Processor 1002 Memory 1003 Storage Device 1004 I / O Interface 1005 Recording Medium

Claims (10)

A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generating means for generating the identifier based on a first allocation relationship for allocating to any of the plurality of real storage nodes and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
Applying means for applying the setting to the communication device;
A communication management device comprising: The first allocation relationship includes an allocation in which the identifier is allocated to two or more real storage nodes among the plurality of real storage nodes;
The generation means determines the two or more transfer destinations for the object in which the identifier is allocated to two or more real storage nodes as the setting of any of the communication devices. 2. The setting including a rule and a second rule that duplicates the object so that the number of the object is the same as the number of the transfer destinations and transfers the object to each of the transfer destinations is generated. The communication management device according to 1. The object transmitted to one of the plurality of real addresses is given a representative address as a destination address,
The generation unit further includes a third rule for determining the transfer destination of the object according to the first rule when an address given to the object as a transmission destination is the representative address. The communication management device according to claim 1 or 2. The generation unit sets the setting including the first rule for determining a transfer destination of the object based on a used part which is a part of the identifier, and sets the used part to any of the plurality of real storage nodes. The communication management device according to any one of claims 1 to 3, wherein the communication management device is generated based on the first assignment relationship to be assigned. Determining the used portion in the identifier based on the number of virtual storage nodes; determining a second allocation relationship for assigning the identifier to any of the plurality of virtual storage nodes based on the used portion; Determining means for determining the first allocation relationship based on the second allocation and a third allocation relationship in which each of the plurality of virtual storage nodes is allocated to any of the plurality of real storage nodes; The communication management device according to claim 4 provided. The communication management device according to any one of claims 1 to 5,
The communication device transferring the object according to the setting;
A communication system including: Transfer destination determination means for determining a transfer destination of the object according to the setting generated by the communication management device according to claim 1;
An input / output unit that receives the object and transmits the object to the transfer destination determined by the transfer destination determination unit. A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generating the identifier based on a first allocation relationship for allocating to any of the plurality of real storage nodes and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
Applying the setting to the communication device;
Communication management method. The first allocation relationship includes an allocation in which the identifier is allocated to two or more real storage nodes among the plurality of real storage nodes;
The first rule for determining two or more transfer destinations for the object assigned with two or more real storage nodes as the setting of any of the communication devices, and the object The communication management according to claim 8, wherein the object is duplicated so that the number of objects is the same as the number of the transfer destinations, and the setting including a second rule for transferring the object to each of the transfer destinations is generated. Method. On the computer,
A setting including a first rule for determining a transfer destination of the object based on an identifier uniquely given to the object of a plurality of communication devices in a path for transmitting the object to any of the plurality of real storage nodes. Generation processing for generating the identifier based on a first allocation relationship for assigning to any of the plurality of real storage nodes, and a connection relationship between the plurality of communication devices and the plurality of real storage nodes;
An application process for applying the setting to the communication device;
A program that executes

Images