JP2014032578A - Distribution storage system, distribution storage data allocation control method and program for distribution storage data allocation control - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distribution storage system which can perform autonomous load distribution control as a whole system while adjusting data allocation on the basis of requirements of a user.SOLUTION: A distribution storage system is constructed by connecting a plurality of constitution nodes which comprise storage means for storing data, through a network. Each constitution node comprises: data transfer means which transfers data to arbitrary number of logical connection destination nodes which are logically connected with each other; status acquisition means which acquires data storage status information of the storage means and load information of the arbitrary number of logical connection destination nodes and the construction node itself as load information; control content calculation means which calculates data amount to be transferred based on the acquired load status for each transfer destination node; transfer exception rule storage means which stores a transfer exception rule holding a condition of data in which the data is not transferred to the outside; and data selection/rule check means for selecting transfer data which does not match the transfer exception rule from the calculated data amount to be transferred.

Description

  The present invention relates to a distributed storage system, a distributed storage data placement control method, and a distributed storage data placement control program, and more particularly to a distributed storage system, a distributed storage data placement control method, and a distributed storage data placement control that can protect conditions related to data placement. Regarding the program.

  The distributed storage system has a large number of computers connected via a network, and stores data and provides data using a hard disk drive (HDD: Hard Disk Drive), a memory, and the like of these computers. In a distributed storage system, software or special hardware decides which computer will place data and which computer will process the data. In addition, it is required to dynamically change the operation of the distributed storage system to adjust the resource usage in the system and improve the performance for the client terminal and its users.

  In a distributed storage system, data is distributed and stored in a plurality of storage nodes (hereinafter simply referred to as “nodes”). As in the technique described in Non-Patent Document 1, there is known a distributed storage system that maintains high availability by holding data copies in a plurality of nodes. Japanese Patent Application Laid-Open No. 2004-228561 describes a technique for continuing a service even if a part of a distributed system cannot be accessed due to a disaster or the like by arranging a copy of data at a remote location. Patent Document 2 and the like also describe that in a wide-area distributed storage system, access performance is improved by placing a copy of data close to a user.

JP 2006-146580 A JP 2006-172217 A

Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: amazon's highly available key-value store. In Proceedings of twenty-first ACM SIGOPS sposium Operating systems principles (SOSP '07). ACM, New York, NY, USA, 205-220. DOI = 10.1145 / 1294261.1294281 http://doi.acm.org/10.1145/1294261.1294281

  The first problem is that it is difficult to arbitrarily adjust data placement according to the requirements of users and system administrators by performing data placement control based on any existing autonomous load balancing algorithm That is. This is because the load balancing algorithm may be developed without considering the requirements of users and system administrators. For this reason, in order to adjust the data arrangement in accordance with the demands of the system administrator and the user with respect to the load distribution algorithms that have already been proven, it is necessary to modify those load distribution algorithms. Since the correction work requires advanced technical knowledge, application as a widely used system becomes difficult.

  For example, in a distributed storage system that distributes and distributes replicated data over a wide area for reasons such as disaster countermeasures, each node constituting the system transfers data in an autonomous distributed manner to achieve load distribution The control algorithm shall be operated. In that case, each node transfers data at random for load distribution, so all replicated data is placed in geographically close nodes, or power supply sources are stored in nodes in the same area. Or may be stored in the same rack or the same node. If such a data arrangement state occurs, access to all copies of the data when some areas are damaged becomes impossible, making it meaningless as a disaster countermeasure system. End up.

  In the case of a use case where a data center (operating some constituent nodes of a distributed storage system) is placed at each geographically dispersed base and a user's file is stored, for example, a certain user is connected to the base A. When file access is mainly performed in B, it is advantageous in terms of access performance to store a copy of the user's file in the bases A and B. However, such an arrangement adjustment is difficult in an autonomous load balancing algorithm that transfers data at random.

  The second problem is that it takes time to shift to a data arrangement state for satisfying requirements for disaster countermeasures and improvement of user access performance. In an autonomous load distribution system in which data is transferred autonomously only by load monitoring with a logical connection destination node having a limited number of constituent nodes, which is the object of the present invention, the data is probable. This is because there is no choice but to wait for it to be propagated and transferred.

  An object of the present invention is to provide a distributed storage system that can perform autonomous load distribution control as a whole system while adjusting data arrangement based on user requirements.

  According to the first aspect of the present invention, there is provided a distributed storage system in which a plurality of constituent nodes having storage means for holding data are connected via a network, and each constituent node is logically connected to each other. Data transfer means for transferring data to an arbitrary number of logical connection destination nodes, data holding state information of the storage means, and load information of the arbitrary number of logical connection destination nodes and constituent nodes themselves State acquisition means for acquiring the state, control content calculation means for calculating the amount of data to be transferred from the acquired load state for each transfer destination node, and transfer for storing a transfer exclusion rule that holds data conditions not to be transferred to the outside Exclusion rule holding means and data selection / rule check for selecting transfer data not corresponding to the transfer exclusion rule from the calculated data amount to be transferred Distributed storage system comprising: the click means, is provided.

  By adopting the configuration as described above, by selecting data that is not included in the transfer exclusion rule when determining transfer data, and by operating to continue storing data that satisfies the transfer exclusion rule in the configuration node, The object of the present invention can be achieved.

  According to the second aspect of the present invention, there is provided a data placement control method in a distributed storage system in which a plurality of constituent nodes each having a storage means for holding data are connected via a network, Transfer exclusion rules that hold data conditions that are not transferred to the outside are stored in advance in the transfer exclusion rule holding means, and each component node can also be connected to any number of logical connection destination nodes that are logically connected to each other. A data transfer step, a step of acquiring the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the configuration node itself as a load state, and a transfer from the acquired load state Calculating the amount of data to be transferred for each transfer destination node, and determining the transfer exclusion rule from the calculated amount of data to be transferred. Distributed Storage data allocation control method characterized by performing the step of selecting the transfer data does not, is provided.

  According to the third aspect of the present invention, there is provided a distributed storage system in which a plurality of constituent nodes having storage means for holding data are connected via a network, and each constituent node does not transfer data to the outside. Is a data placement control program applied to a distributed storage system in which transfer exclusion rules that hold the above conditions are stored in advance in the transfer exclusion rule holding means, and optionally connected logically to the computer of each component node Transferring data to the number of logical connection destination nodes, and obtaining the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves as load states Calculating the amount of data to be transferred from the acquired load state for each transfer destination node; Distributed Storage data allocation control program for to execute the steps of: selecting a transfer data does not correspond to the transfer exclusion rule is provided from the should do data amount.

  The first effect of the present invention is that data placement adjustment can be realized on an autonomous load balancing control algorithm that does not consider it based on user requirements (disaster countermeasures, access performance improvement, etc.). The reason is that, when each constituent node selects transfer data, data not included in the transfer exclusion condition is selected and transferred, so that data necessary for arrangement adjustment is not transferred outside the constituent node.

It is a block diagram which shows the outline | summary of a structure of the 1st Embodiment of this invention. It is a block diagram which shows the logical node connection form in 1st Embodiment shown by FIG. FIG. 2 is a block diagram showing in detail a configuration of a control unit 2 with a configuration node 100 as a center in the configuration of the first embodiment shown in FIG. 1. It is a figure for demonstrating the evaluation function in the Example of the control content calculation means 22 shown by FIG. It is a figure for demonstrating the parameter | index of a start / stop determination in the Example of the control content calculation means 22 shown by FIG. It is a flowchart for demonstrating operation | movement of the 1st Embodiment of this invention. It is a flowchart for demonstrating the transfer data selection process operation | movement among the operation | movement of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the 3rd Embodiment of this invention.

  Next, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram showing an outline of a system according to the first embodiment of the present invention. Referring to FIG. 1, the first embodiment of the present invention is composed of configuration nodes 100 to 108 coupled by a network 200. Hereinafter, when the configuration node 10X is described, it means that X can take a value from 0 to 8. Although the number of constituent nodes is nine in FIG. 1, it goes without saying that the number of constituent nodes is an arbitrary plural number. Further, it is assumed that some of the plurality of component nodes are located at remote locations (for example, data centers). In the case of disaster countermeasures, one or more bases must be installed at a location that is not far away from other bases at the same time, or where power supply sources (for example, power plants) are separated. To do. However, in the case of a purpose other than disaster countermeasures (for example, a file sharing system between bases), it is not necessarily limited in this way.

  The configuration node 10X (that is, 100 to 108) is a storage node that configures the distributed storage system. The configuration node 10X is realized by a computer (computer) including a storage device (eg, HDD, semiconductor memory, etc.) for holding data and an arithmetic device (eg, CPU: Central Processing Unit) for realizing a storage function. .

  The configuration node 10X includes a data store (storage unit) 1, a control unit 2 realized by a CPU or the like, a data reception unit 3, and a load monitoring unit 4. The data store 1 functions to store data stored in the distributed storage system. The data store 1 is realized by a storage device held by the configuration node 10X and software that implements storage processing for the storage device, and is realized by computer resources of the configuration node. The control unit 2 performs arrangement control (migration or increase in the number of replicas) of arbitrary data among other data stored in the data store 1 of each configuration node 10X. The data receiving unit 3 receives the transfer data from the configuration nodes other than itself, which is transferred by the arrangement control process performed by the control unit 2, and holds it in the data store 1. The load monitoring unit 4 obtains node load information of the configuration node 10X (for example, data access amount, computer load amount of the configuration node 10X, such as CPU usage rate and IO (Input-Output) amount) by using existing monitoring software or the like. The control means 2 and the function of transmitting to a part of the configuration node 10X other than itself are fulfilled.

  The network 200 is used to connect the configuration nodes 100 to 108 and exchange data and control messages. The network 200 includes not only a network in one site (for example, a network in a data center) but also a wide area network that connects a plurality of sites. The network 200 is, for example, Ethernet (registered trademark), Fiber Channel or FCoE (Fibre Channel over Ethernet, InfiniBand, or TCP / IP (Transmission Control Protocol / Internet Protocol), RDMA (Remote Direct Memory Access) using these. However, the implementation method is not limited to these, and it is possible to connect geographically distant bases, or connect bases via the Internet or a dedicated line. .

  Data stored in the distributed storage system is a set of fixed-length or semantically separated data fragments (hereinafter referred to as “objects”) in one or a plurality of data stores 1 of each of the configuration nodes 10X. Stored. When stored in a plurality of nodes, it is stored as a copy of the object. A client of a distributed storage system accesses a single object or a set with a unique identifier (key). In addition to each object, redundant code information calculated based on the object may be stored as a duplicate in another node. Redundant code information is used to prevent data loss in the event of a node failure.

  Examples of objects include, for example, a file system file, a set of metadata related to the file, a tuple or table of a relational database, data of an object database, a value of a key-value data storage system, and JSON (Java Script (registered trademark)). Documents described in Object Notation) and BSON (Binary JSON). However, the objects in the present invention are not limited to these objects. In addition, in these examples, a data fragment further fixed length or semantically divided may be used as a storage unit in the data store 1.

[Outline of control method]
The distributed storage data arrangement control method according to the first embodiment of the present invention is a method in which the control means 2 of each component node 10X determines and executes the necessity of data transfer, the data transfer amount, and the transfer destination. As shown in FIG. 1, each constituent node 10X is coupled via a network, and therefore can transfer data to any node. However, calculating the data transfer amount with all nodes as the transfer destination increases the calculation cost. Therefore, each component node is logically connected in an arbitrary topology, and only the logical connection destination node is connected to each node. The control unit 2 uses the transfer destination candidate node as a transfer destination candidate node, and determines the necessity of data transfer to the transfer destination candidate node and performs a calculation process for calculating the data transfer amount and the transfer destination, thereby reducing the calculation amount.

  As the logical connection format of each component node 10X, a mesh type, a tree type, a full connect type, a star type, a cube type, and the like, or a plurality of these connection methods are connected hierarchically or combined. There is a format. In this embodiment, in order to simplify the description, it is assumed that they are connected with a two-dimensional mesh structure as shown in FIG. In this case, the logical connection nodes (transfer destination candidate nodes) in the configuration node 100 are the configuration nodes 101 and 103. Information regarding this connection form is held by each component node 10X as logical connection information, and is notified to each component node when it is changed.

  In addition, the logical connection form is free, but basically, it is fundamental to have a topology configuration in which a large number of connections are made with neighboring configuration nodes and a limited number of connections are established with remote configuration nodes ( (For example, a tree structure). This depends on the control algorithm of the control content calculation means 22 (FIG. 3), which will be described later. However, since many data transfers occur between the connected constituent nodes, a data transfer bottleneck occurs when connected to a remote node. This increases the size of the computer and causes problems in storage performance.

[Constitution]
FIG. 3 is a block diagram showing the configuration of the control unit 2 in more detail with the configuration node 100 as a center in the configuration of the first embodiment. The control unit 2 includes a state acquisition unit 21, a control content calculation unit 22, a data selection / rule check unit 23, a data movement control unit (data transfer unit) 24, a transfer exclusion rule holding unit 25, a logical connection form management unit 26, The rule update means 5 is comprised. Each means operates as software using a hardware device that implements the configuration node 10X. Further, special hardware on the configuration node 10X may perform the function.

  The status acquisition means 21 of the configuration node 100 includes data retention status information (for example, data retention amount and type) of the data store 1 and the configuration node 100 and its logical connection destination nodes (configuration nodes 101 and 103 in the case of FIG. 3). Is obtained from the load monitoring means 4 of the constituent nodes 100, 101, 103 as a load state. The detailed contents of the data holding state information and the load information are determined by the algorithm of the control content calculation means 22. The logical connection destination node is acquired from the logical connection form management means 26.

  The control content calculation unit 22 uses the data holding state information and load information acquired by the state acquisition unit 21 based on the algorithm described below, and calculates and determines the amount of data to be transferred for each transfer destination candidate node. Fulfill.

First, an evaluation function related to the data store 1 as shown in FIG. 4 is introduced. In this case, the horizontal axis λ corresponds to the load amount of the data store 1. The vertical axis f _i (λ) is an index related to some efficiency or profit. In the case of the data store 1, an index relating to response efficiency and energy efficiency is set. According to the response efficiency, the response of the entire system can be optimized, and according to the energy efficiency, the energy efficiency of the entire system can be optimized. The evaluation function related to this efficiency is expressed by a convex function. The use of a convex function is also a feature of this control method. As shown in FIG. 4, the upward convex function is sometimes called a concave function, and the downward convex function is sometimes called a convex function. It will be expressed as a convex function.

  The problem of optimizing the entire function by linking elements whose evaluation function is a convex function (a state in which the sum of the evaluation function values of each element is maximized) is known as a “convex programming problem”. It is mathematically shown that optimization is achieved in the situation where the differential values of the evaluation function are equal at the operation levels. The present embodiment applies this principle. This is the reason why the convex function is used as the evaluation function in the present embodiment.

Considering this principle, the state change of each element (load distribution in the case of the server of the present embodiment) is controlled according to the following equation (1).

Here, λ _i is the load amount of the data store 1, and K ₁ is a coefficient corresponding to the gain of state change. By applying this control, each element attempts to change its state so that the differential value df _i / dλ _i of the evaluation function is equal. This is equivalent to controlling the state (load amount) of each element at the point that maximizes the total profit, as described in the section “convex planning problem”. That is, the efficiency set on the vertical axis of the evaluation function is maximized for the entire system by the control based on the above formula (1). k means the number of an element adjacent to the element i (when the element i is the configuration node 100 when logically joined in the configuration of FIG. 2, the configuration nodes 101 and 103). When there are a plurality of adjacent elements in this way, the control based on the above formula (1) may be repeated sequentially.

  Although the load placement problem can be solved in real time by the methods described above, in some cases, it may be more efficient if the data store 1 (and the configuration node 10X) is stopped. In some cases, it may be more efficient to start the stopped data store 1 (configuration node 10X). Since only the above equation (1) cannot be used to determine whether to start or stop, the following index is used to determine whether or not to start and stop the data store 1.

First, an index is defined between the element itself and the evaluation function of the node adjacent to the element. FIG. 5 is a diagram for explaining the evaluation function. In FIG. 5, when the load amount λ is 0, the meaning that the efficiency or profit (vertical axis) f _i (λ) takes a negative value means that a cost is generated by activation. Considering the vertical axis as profit, it is an image that if there is no demand even though the vehicle is operating (starting up), the operating cost exceeds the recovery (amount collected from the customer) and the profit becomes negative. When the value of the evaluation function becomes 0, the operating cost and the recovery are balanced, and the positive value of the evaluation function means that a profit is generated.

Here, the zero cross point of the node i (self) is λ _{0, i} , the point having the same slope as the zero cross point of the node i in the evaluation function of the adjacent node j is Z _ij , and the current load of the adjacent node j is λ _j To do. In this case, Z _ij can be expressed by the following equation (2).

Then, an index Si of the following formula (3) is defined here.

This index Si means how much the load (total) of the current adjacent node is larger than its own zero-cross point. If the node i is stopped, the condition for starting the node i is whether or not the node i is in charge of a load greater than or equal to the zero cross point by starting. Then, profit does not become negative (the evaluation function value is negative), so there is no disadvantage in starting node i. The index Si is a numerical value of it. The first term on the right-hand side of Equation (3) indicates how much the adjacent node is responsible for the load from Z _ij equivalent to the evaluation function differential value at the zero-cross point of node i _{. By} subtracting with _i (the second term on the right side), it becomes an index indicating whether or not there is a load around its zero-cross point. If the index Si is greater than 0, the load that the adjacent node is responsible for when the node i is activated will be handled by the node i, and the load assigned to the node i is equal to or greater than the zero cross point. On the other hand, if the index Si is smaller than 0, even if the node i newly bears the load assigned to the adjacent node, the node i becomes below the zero cross point, and the profit is negative due to the activation of the node i. The index Si is like this, and it can be seen whether the node i should be started or stopped by looking at the sign of the index Si. When the index Si is negative, the load of the node i is in a region where the evaluation function takes a negative value at that time, and the active node stops when the evaluation function becomes 0 or less You may think.

  As described above, a suitable load amount of the data store 1 is calculated in real time by monitoring the index Si in the above equation (3) while controlling the state change of each element in the above equation (1), and the node Can be determined whether to start or stop. This is an epoch-making algorithm as compared with the conventional algorithm that is scheduled and controlled in advance in a prediction scenario. This is because even if there is an unexpected disturbance or a change in supply and demand, it can respond immediately in real time.

  In addition, since this algorithm operates independently in an autonomous and distributed manner, even if some part fails, the other element autonomously recovers as much as the signal from the failed element stops. . Moreover, even if the number of elements is suddenly increased or decreased, an appropriate operation can be gradually and autonomously performed. In other words, it is very robust against disturbances and has scalability that allows elements to be freely increased or decreased.

  Conventional systems have had to be powerless with respect to failure or have prepared various error sequences. In addition, there is no guarantee that the stability of the entire system will be maintained if elements (resources) are increased or decreased without permission, and the program and processing must be reviewed each time. The algorithm of this embodiment can solve all of these problems by autonomous distributed adaptive control.

  The data selection / rule check unit 23 actually selects data from the node for each connection destination, the amount of data to be transferred, and information on the transfer destination node determined by the control content calculation unit 22. At that time, the data selection / rule check means 23 refers to the rules stored in the transfer exclusion rule holding means 25, does not select data that matches the conditions registered as the transfer exclusion rules as transfer data, Transfer data is selected from the data.

  The data movement control unit 24 acquires the data selected and designated by the data selection / rule check unit 23 from the data store 1 and transfers the data to the transfer destination node. The data receiving means 3 of the transfer destination node receives the data and reflects it in the data store 1 of the transfer destination node.

  The transfer exclusion rule holding unit 25 holds data conditions that are not transferred from the configuration node to the outside when stored in the configuration node (the configuration node 100 in this case), and holds one or more conditions. . Of course, conditions that may be transferred may be stored, and data that does not meet the conditions may be disabled.

[Example of forwarding exclusion rule]
Examples of holding constraint rules include a list of identifiers {key, file path and file name, ID (identifier assigned to each object or record)} indicating data not to be transferred, and a list of metadata attached to the data (For example, file owner, group name, etc.). Recording this information in a file, database, etc. can be considered as a simple implementation method.

  The logical connection form management means 26 is means for holding and managing the logical connection destination nodes of the constituent nodes and responding to requests. When the logical connection form of the configuration node is as shown in FIG. 2, the logical connection form management means 26 of the configuration node 100 responds with the configuration nodes 101 and 103 as logical connection destination nodes. The logical connection form management means 26 may be changed by any external means. In the first embodiment, since the logical connection destination node is statically determined, information determined in advance for each node may be returned.

  As an example of the logical connection form management means 26, a connection destination node list (information indicating 101 and 103 in the case of the configuration node 100) is recorded in a database or a file, or all connection relationships are recorded. (That is, information corresponding to FIG. 2) may be held in each constituent node.

  The rule update means 5 is means for changing / updating / adding a rule held in the transfer exclusion rule holding means 25 on one or a plurality of constituent nodes 10X. The rule for only one configuration node may be changed, or the rules for two or more or all configuration nodes may be changed. The rule update unit 5 may operate on an arbitrary configuration node 10X, or may operate on a node other than the configuration node of the distributed storage system (for example, a personal computer used by a file system user). Good).

  For example, when the user or system administrator wants to hold user A's data in the configuration nodes 100 and 108, the rule update unit 5 sets the exclusion rule of the transfer exclusion rule holding unit 25 of the configuration nodes 100 and 108 as the exclusion rule of the user A. Add retained data information. For example, when targeting a file system, the owner of metadata adds user A to the exclusion list.

  In addition, for example, the configuration node 100 is a node in the vicinity of the base used by the user A, and the reason for holding the copy in the configuration node 108 is a use where the copy is desired to be held at a location far enough away for disaster countermeasures In case, it may operate as follows. A user or a system administrator causes the configuration node 100 to add an exclusion setting. On the other hand, the rule update means 5 is a configuration node at a location that is not damaged at the same time as the configuration node 100 (at a geographically sufficient distance, or where power supply is divided, or has another network route). The same condition is added to the configuration node 108 by selecting one from 10X. At this time, the rule update unit 5 may recognize the position of the configuration node (or the power supply state and the system configuration state).

  Also, a condition description such that certain data should not be transferred from the configuration node 100 to a node other than the configuration nodes 101 and 102 may be used. In this case, when the transfer destination node is the configuration node 103, the corresponding data cannot be transferred, but when the transfer destination node is the configuration node 101, transfer is possible. For example, such a condition specification is performed when the configuration nodes 100 to 102 exist at the same base and the purpose can be achieved if one of the nodes has one or more replicas.

[Overall operation]
Next, the overall operation of the first embodiment will be described in detail with reference to the flowcharts of FIGS. FIG. 6 is a data transfer control processing flow in each component node 10X. By periodically executing this flow, the system is stably distributed and operated. FIG. 7 is a flowchart for explaining step S103 in FIG. 6 in detail, and is a process of selecting a data fragment to be actually transferred to a certain transfer destination node.

  First, a basic data transfer control processing flow will be described with reference to FIGS. The status acquisition means 21 acquires data holding status information from the data store 1, and acquires load information from the load monitoring means 4 of the configuration nodes 100, 101, and 103, which are itself and logical connection destination nodes. The status is passed to the control content calculation means 22 (step S101). The control content calculation means 22 calculates the control content (transfer data amount for each transfer destination) using the received data holding state information and load information (step S102). The specific method of step S102 is shown in the section of control content calculation means 22.

  Subsequently, the data selection / rule check unit 23 selects a transfer data fragment for each transfer destination node from data other than the data corresponding to the rule stored in the transfer exclusion rule holding unit 25, and designates the selected transfer data fragment. The information to be transferred is passed to the data movement control means 24, and execution of the data movement control is requested (step S103). Details of step S103 will be described later with reference to the flowchart of FIG.

  The data movement control unit 24 acquires the data fragment to be transferred from the data store 1 based on the information specifying the transfer data fragment from the data selection / rule check unit 23 and transfers the data to the data receiving unit 3 as the transfer destination. (Step S104).

[Transfer data selection in step S103]
Next, the operation of step S103 by the data selection / rule check means 23 will be described with reference to FIG. As described above, the data selection / rule checking unit 23 determines the amount of data that satisfies the data transfer amount from the node, data transfer amount, and information on the transfer destination node for each connection destination determined by the control content calculation unit 22. A fragment is selected from the data store 1 via the state acquisition means 21 (step S201). For example, if the determined data transfer amount corresponds to 100 data fragments, the data selection / rule check means 23 selects 100 data fragments stored in the data store 1. Various methods such as selecting in random order, selecting in order of FIFO (First in First out), selecting in order of FIFO (First in Last out) can be considered, but any method may be used here.

  Next, the data selection / rule check unit 23 takes out the exclusion rule from the transfer exclusion rule holding unit 25 (step S202). Since steps S201 and S202 can be executed independently, they may be executed in any order.

  The data selection / rule checking means 23 further determines whether each data fragment corresponds to the transfer exclusion rule from the constraint rule acquired in step S202 and the data fragment selected in step S201 (being a transfer candidate) (step S203).

  Next, the data selection / rule check unit 23 checks whether all the transfer candidate data fragments selected in step S201 do not correspond to the transfer exclusion rule in the determination in step S203 (step S204). If all the transfer candidate data fragments do not correspond to the transfer exclusion rule (YES in step S204), the list of transfer data fragments of the designated data amount has been completed, so this processing ends, and FIG. The process proceeds to step S104.

  If there is data corresponding to the transfer exclusion rule among the transfer candidate data fragments (NO in step S204), the data selection / rule check means 23 creates new data fragments corresponding to the data amount corresponding to the transfer exclusion rule. It again selects from the data fragments stored in the data store 1 (step S205). A transfer candidate data fragment that does not correspond to the transfer exclusion rule is used as a transfer candidate as it is. The basic selection method in step S205 may be the same as in step S201, but data fragments that have been previously selected and correspond to the exclusion rule should be excluded from the selection candidates. After selecting a new transfer data fragment candidate, the data selection / rule check means 23 returns to step S203 and performs the rule determination process again.

  Needless to say, the transfer candidate data may be selected while determining the transfer exclusion rule at the time of step S201 in this operation flow.

[Second Embodiment: Change in Logical Joining Mode]
Next, a second embodiment according to the present invention will be described. In 1st Embodiment, the subject was achieved by giving the transfer exclusion rule to each component node. However, in this case, the arrangement state cannot be satisfied until the corresponding data is accidentally transferred to the target constituent node by the autonomous distributed control algorithm. In order to solve this problem, a mechanism for increasing the connection destinations of the constituent nodes at the time when the rule update means 5 updates the rule is introduced.

  For example, when a condition for preventing the user A data from being transferred to the configuration node 100 is added, when the user A data is transferred to the configuration node 100, the logical connection destination node (the configuration node in FIG. 2) 101 and 103) need to wait for data to be transferred. In order to increase the probability of this data transfer, the number of logical connection destination nodes may be increased. That is, one or more configuration nodes among the configuration nodes 102 and 104 to 108 may be added as logical connection destination nodes.

  The control algorithm described in the present invention tends to increase the calculation cost as the number of logical connection destination nodes increases and the network use bandwidth also increases. This is a trade-off between speeding up the transition to the state and calculation cost. Therefore, it is desirable to insert a process of returning to the original state when the time specified in advance is exceeded.

  Further, the number of connection destination nodes to be increased may be changed according to the time required for shifting to the data arrangement state. For example, if data migration in a short time is necessary, an interface that increases the number of connection destination nodes may be provided.

  Next, a second embodiment of the present invention will be described with reference to FIG. The second embodiment shown in FIG. 8 is almost the same as the first embodiment shown in FIG. 3 except that the transfer exclusion rule holding means 25 in the control means 2 is the rule from the rule update means 5. The difference is that it has a function of changing the information of the logical connection form management means 26 based on the update request. Also, the logical connection form management means 26 needs to request an update from the logical connection form management means 26 of the logical connection destination node. That is, in the connection form of FIG. 2, when adding a connection between the configuration node 100 and the configuration node 108, it is necessary to change the logical connection form management means 26 of the configuration node 108 (however, in FIG. Omitted). Note that it is not always necessary to change the logical connection form management unit 26 via the transfer exclusion rule holding unit 25. For example, a configuration in which the rule update unit 5 directly changes may be adopted.

[Third Embodiment]
As a third embodiment of the present invention, a configuration provided with a mechanism for automatically adding a transfer exclusion rule to user access will be described. In the first embodiment, it is necessary for a system administrator or a user to add a transfer exclusion rule based on information on the access destination base (or the configuration node) of the user A using the rule update unit 5.

  In the third embodiment of the present invention, when a certain user accesses a configuration node of the distributed storage system, if the access is detected by the configuration node, an operation is performed to add a data transfer exclusion rule for that user. .

  FIG. 9 shows the configuration of the third exemplary embodiment of the present invention. The third embodiment is almost the same as the configuration of the first and second embodiments except that it includes an access detection means 7 for detecting access from the user client 6.

  The user client 6 is a client that accesses a distributed storage system that is a service for realizing the present invention, and is realized by an arbitrary terminal (a computer, a portable terminal, or the like). The access detection means 7 is software that operates on the configuration node 10X, and identifies the user of the access request. As an embodiment, it may be realized in software that receives an access request of a distributed storage system. After the access detection unit 7 identifies the access source user, the transfer detection condition is added to the transfer exclusion rule holding unit 25 in order to control the data related to the user so that the data is stored in the accessed configuration node. (For example, the owner adds the access user's data to the exclusion condition).

  Further, the access detection means 7 may hold an access history. By holding the access history, if there is no user access to the corresponding node for a certain period, the transfer exclusion condition for the user is deleted. By operating in this way, user access performance can be improved. In addition, efficient load distribution control can be maintained by automatically deleting conditions.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

"Appendix"
Appendix 1
A distributed storage system in which a plurality of configuration nodes having storage means for holding data are connected via a network,
Data transfer means for transferring data to an arbitrary number of logical connection destination nodes each logically connected to each component node;
State acquisition means for acquiring the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves;
Control content calculation means for calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
A transfer exclusion rule holding means for storing a transfer exclusion rule for holding a condition of data not transferred to the outside;
Data selection / rule checking means for selecting transfer data not corresponding to the transfer exclusion rule from the calculated data amount to be transferred;
A distributed storage system characterized by comprising:

Appendix 2
Each of the constituent nodes receives data transferred from constituent nodes other than itself, and stores data in the storage means;
The distributed storage according to appendix 1, further comprising: load information of the plurality of constituent nodes, and load monitoring means for acquiring the state acquisition means and transmitting the load information to the arbitrary number of logical connection destination nodes. system.

Appendix 3
The distributed storage system according to appendix 1 or 2, further comprising rule updating means for assigning a transfer exclusion condition for arbitrary data to the transfer exclusion rule holding means of a single or a plurality of constituent nodes.

Appendix 4
When a transfer exclusion condition for the arbitrary data is given to the transfer exclusion rule holding unit of a single or a plurality of configuration nodes, select one or a plurality of configuration nodes that are not damaged at the same time as a specified configuration node, The distributed storage system according to appendix 3, wherein the same transfer exclusion condition is assigned to the selected configuration node.

Appendix 5
The distributed storage system according to appendix 4, wherein the condition of the configuration node that is not damaged at the same time as the designated configuration node includes a condition that a physical distance is more than a certain distance.

Appendix 6
The distributed storage system according to appendix 4, wherein the condition of the configuration node that is not damaged simultaneously with the designated configuration node includes a condition that the power supply source is a separate system.

Appendix 7
When a transfer exclusion condition for the arbitrary data is given to the transfer exclusion rule holding unit of a single or a plurality of configuration nodes, the logical connection form of the configuration nodes to which the conditions are given is changed. The distributed storage system according to Supplementary Note 3.

Appendix 8
A data placement control method in a distributed storage system in which a plurality of constituent nodes having storage means for holding data are connected via a network,
Each configuration node stores in advance a transfer exclusion rule that holds data conditions that are not transferred to the outside in the transfer exclusion rule holding means.
Each configuration node also
Transferring data to any number of logically connected nodes, each logically connected;
Obtaining the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves as load states;
Calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
Selecting transfer data not corresponding to the transfer exclusion rule from the calculated amount of data to be transferred;
A distributed storage data arrangement control method characterized by executing:

Appendix 9
9. The distributed storage according to appendix 8, wherein each of the configuration nodes further executes a step of assigning a transfer exclusion condition for arbitrary data to the transfer exclusion rule holding unit of a single or a plurality of configuration nodes. Data placement control method.

Appendix 10
Each of the configuration nodes includes a configuration node that is not damaged at the same time as a specified configuration node when a transfer exclusion condition for the arbitrary data is given to the transfer exclusion rule holding unit of the single or multiple configuration nodes. 10. The distributed storage data placement control method according to appendix 9, wherein one or a plurality are selected and the same transfer exclusion condition is assigned to the selected configuration node.

Appendix 11
The distributed storage data arrangement control method according to appendix 10, wherein the condition of the configuration node that is not damaged simultaneously with the designated configuration node includes a condition that a physical distance is more than a certain distance.

Appendix 12
The distributed storage data placement control method according to appendix 10, wherein the condition of the configuration node that is not damaged simultaneously with the designated configuration node includes a condition that the power supply source is a separate system.

Appendix 13
Each configuration node is a logical connection form of configuration nodes to which conditions are added when transfer exclusion conditions for the arbitrary data are added to the transfer exclusion rule holding unit of the single or multiple configuration nodes. 10. The distributed storage data arrangement control method according to appendix 9, wherein:

Appendix 14
A distributed storage system in which a plurality of configuration nodes having storage means for storing data are connected via a network, and each configuration node transfers a transfer exclusion rule that holds data conditions that are not transferred to the outside in advance. A data placement control program applied to the distributed storage system stored in the exclusion rule holding means,
Transferring data to any number of logically connected nodes logically connected to the computers of each constituent node;
Obtaining the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves as load states;
Calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
Selecting transfer data not corresponding to the transfer exclusion rule from the calculated amount of data to be transferred;
Distributed storage data placement control program for executing

  According to the present invention, the present invention can be applied to uses such as a distributed storage system in which a storage system is realized by a plurality of nodes. In particular, the present invention can be applied to disaster-resistant systems and distributed file systems that share files between sites.

DESCRIPTION OF SYMBOLS 1 Data store 2 Control means 3 Data reception means 4 Load monitoring means 5 Rule update means 6 User client 21 Status acquisition means 22 Control content calculation means 23 Data selection / rule check means 24 Data movement control means 25 Transfer exclusion rule holding means 26 Logic Joint form management means 100 to 108 Component node 200 Network

Claims (10)

A distributed storage system in which a plurality of configuration nodes having storage means for holding data are connected via a network,
Data transfer means for transferring data to an arbitrary number of logical connection destination nodes each logically connected to each component node;
State acquisition means for acquiring the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves;
Control content calculation means for calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
A transfer exclusion rule holding means for storing a transfer exclusion rule for holding a condition of data not transferred to the outside;
Data selection / rule checking means for selecting transfer data not corresponding to the transfer exclusion rule from the calculated data amount to be transferred;
A distributed storage system characterized by comprising: Each of the constituent nodes receives data transferred from constituent nodes other than itself, and stores data in the storage means;
The distribution according to claim 1, further comprising: load information of the plurality of constituent nodes, and load monitoring means for acquiring the state acquisition means and transmitting it to the arbitrary number of logical connection destination nodes. Storage system.   The distributed storage system according to claim 1, further comprising a rule update unit that assigns a transfer exclusion condition for arbitrary data to the transfer exclusion rule holding unit of a single or a plurality of constituent nodes.   When a transfer exclusion condition for the arbitrary data is given to the transfer exclusion rule holding unit of a single or a plurality of configuration nodes, select one or a plurality of configuration nodes that are not damaged at the same time as a specified configuration node, 4. The distributed storage system according to claim 3, wherein the same transfer exclusion condition is assigned to the selected configuration node.   The distributed storage system according to claim 4, wherein the condition of the configuration node that is not damaged at the same time as the designated configuration node includes a condition that a physical distance is more than a certain distance.   The distributed storage system according to claim 4, wherein the condition of the configuration node that is not damaged simultaneously with the designated configuration node includes a condition that the power supply source is a separate system.   When a transfer exclusion condition for the arbitrary data is given to the transfer exclusion rule holding unit of a single or a plurality of configuration nodes, the logical connection form of the configuration nodes to which the conditions are given is changed. The distributed storage system according to claim 3. A data placement control method in a distributed storage system in which a plurality of constituent nodes having storage means for holding data are connected via a network,
Each configuration node stores in advance a transfer exclusion rule that holds data conditions that are not transferred to the outside in the transfer exclusion rule holding means.
Each configuration node also
Transferring data to any number of logically connected nodes, each logically connected;
Obtaining the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves as load states;
Calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
Selecting transfer data not corresponding to the transfer exclusion rule from the calculated amount of data to be transferred;
A distributed storage data arrangement control method characterized by executing:   9. The distribution according to claim 8, wherein each of the configuration nodes further executes a step of assigning a transfer exclusion condition for arbitrary data to the transfer exclusion rule holding unit of one or a plurality of configuration nodes. Storage data placement control method. A distributed storage system in which a plurality of configuration nodes having storage means for storing data are connected via a network, and each configuration node transfers a transfer exclusion rule that holds data conditions that are not transferred to the outside in advance. A data placement control program applied to the distributed storage system stored in the exclusion rule holding means,
Transferring data to any number of logically connected nodes logically connected to the computers of each constituent node;
Obtaining the data holding state information of the storage means and the load information of the arbitrary number of logical connection destination nodes and the constituent nodes themselves as load states;
Calculating the amount of data to be transferred from the acquired load state for each transfer destination node;
Selecting transfer data not corresponding to the transfer exclusion rule from the calculated amount of data to be transferred;
Distributed storage data placement control program for executing

Images