WO2015081750A1 - 一种存储数据副本的方法和终端装置以及服务器装置 - Google Patents

一种存储数据副本的方法和终端装置以及服务器装置 Download PDF

Info

Publication number
WO2015081750A1
WO2015081750A1 PCT/CN2014/087621 CN2014087621W WO2015081750A1 WO 2015081750 A1 WO2015081750 A1 WO 2015081750A1 CN 2014087621 W CN2014087621 W CN 2014087621W WO 2015081750 A1 WO2015081750 A1 WO 2015081750A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
node
stored
copy
storage location
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2014/087621
Other languages
English (en)
French (fr)
Inventor
罗文彬
张波
杨万元
杨小龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to US15/039,342 priority Critical patent/US10165048B2/en
Priority to RU2016126187A priority patent/RU2637434C1/ru
Publication of WO2015081750A1 publication Critical patent/WO2015081750A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/44—Star or tree networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a method and a terminal device for storing a copy of data, and a server device.
  • FIG. 1 is a schematic diagram of the physical topology of a storage device in a distributed data storage system in connection with the present invention.
  • the distributed data storage system is disposed in the data center 10.
  • the data center 10 is composed of three computer rooms, and each of the equipment rooms has multiple racks.
  • the machine room 2 has a rack 1 and a rack 2.
  • each rack there are multiple hosts (computers) on each rack, for example, rack 1, host 1, host 2, ..., host N2; each host has multiple storage medium devices (usually a hard disk), for example, the host 2 has a hard disk 1, a hard disk 2, ..., a hard disk N3.
  • the distributed data storage system is a tree structure, the storage medium device is located on the leaf node, and the host, the rack, and the machine room are intermediate nodes.
  • Distributed data storage systems can be divided into two types: central nodes and non-central nodes.
  • the client, the central node, and the storage node are generally included, and the data is processed into blocks and stored in multiple copies, and the distributed position of the data is determined by the central node according to the load of the storage node.
  • the storage policy of the copy determines where the copy of the data is stored.
  • the central node can be either two server primary or secondary configurations or a server cluster.
  • each node is connected to each other, and data is randomly distributed on the storage devices of the respective nodes, and the storage location can be calculated by the node using a hash function.
  • a distributed data storage system without a central node can avoid the above bottleneck, but its reliability is still insufficient. For example, if a rack is powered off or a network failure occurs, and all copies of a piece of data are on the hard disk of the host of the rack, the data is not available.
  • a distributed data storage system without a central node is bound to have extensive data migration. This is illustrated by a simple example.
  • the hash value of a data to be stored calculated by the hash function is 13, and the modulus is determined according to the number of nodes. 3, then the data to be saved is saved on the node 3; when the device is added, the number of nodes is 6, when the data is read, the hash value 13 is modulo 2 to the node number 6, that is, read from the node 2 Take the data, at which point the data will first migrate from node 3 to node 2. Since the modulo result is inevitably different when the number of nodes changes, data migration must occur when the data saved before the number of nodes is changed after the number of nodes changes. Therefore, once the number of nodes changes, data migration becomes a common phenomenon, which will lead to the system. Reduced efficiency and reduced storage media life.
  • the present invention provides a method for storing a data copy, a terminal device, and a server device, which contribute to improving the reliability of a distributed data storage system without a central node and reducing data migration when the number of nodes changes. the amount.
  • a method of storing a copy of data is provided.
  • the method for storing a data copy of the present invention is applied to a distributed data storage system having a tree structure without a central node, the tree structure including a leaf node and an intermediate node, and the storage medium device is located on the leaf node,
  • the method includes: Step A: according to the tree structure, for each data copy of the data to be stored, starting from the pre-selected nodes in the tree structure, selecting the child nodes step by step according to a preset child node selection manner Until the selected child node is a leaf node, and then determining the leaf node as a target storage location of the data copy; wherein the child node is selected by: combining the identifier of the data to be stored and the identifier of the child node as an original image Deriving a mapping value associated with the child node according to a preset mapping manner, and then selecting a child node of the intermediate node according to a size relationship between the plurality of mapping values of each child node associated with the same intermediate node; And for each
  • the intermediate node in the process of determining a target storage location of all data copies of the to-be-stored data, if the intermediate node is selected again, then the intermediate node is Re-select all of the sibling nodes.
  • the target storage location determined for the current data copy is the same as the determined target storage location of another data copy, The target storage location is re-determined for the current data copy according to step A, so that the target storage locations of all data copies of the to-be-stored data are different.
  • the mapping manner includes: calculating a unique mapping value by using a hash algorithm after adjusting the original image.
  • the mapping manner includes: calculating a hash value by using a hash algorithm after adjusting the original image; multiplying the hash value by a preset weight value to obtain the mapping value; wherein the pre-predetermined Let the weight value be the weight of all leaf nodes connected to the child nodes in the original image The sum of the values, the weight value of the leaf node is positively correlated with the storage capacity of the storage medium device on the leaf node.
  • the step of resetting the mapping relationship comprises: changing a parameter used when adjusting the original image.
  • a terminal device is provided.
  • the terminal device of the present invention is configured to save a plurality of data copies of data to be stored into a distributed data storage system having a tree structure without a central node, the tree structure including leaf nodes and intermediate nodes, and the storage medium device is located
  • the terminal device includes: a selecting module, configured to: according to the tree structure, each data copy of the data to be stored is preset from the pre-selected node in the tree structure
  • the child node selection method selects the child node step by step until the selected child node is a leaf node, and then determines the leaf node as a target storage location of the data copy; wherein the child node is selected by: the data to be saved
  • the combination of the identifier and the identifier of the child node is the original image, and the mapping value associated with the child node is obtained according to a preset mapping manner, and then the size between the plurality of mapping values associated with each child node of the same intermediate node is determined.
  • Relationship selects one child node of the intermediate node; and for each intermediate node, resets the mapping relationship each time its child node is selected; uploading mode And after the selection module determines a target storage location of all data copies of the to-be-stored data, storing all data copies of the to-be-stored data and target storage locations of all data copies of the to-be-stored data Information is sent to a computer in the distributed data storage system for the computer to save a copy of all data of the data to be stored to a target storage location of each data copy.
  • the selecting module is further configured to, in the process of determining a target storage location of all data copies of the to-be-stored data, for the specified one or more of the intermediate nodes, if the intermediate node is selected again Then, at this time, the selection is re-selected from all the sibling nodes of the intermediate node.
  • the selecting module is further configured to: when determining a target storage location of all data copies of the to-be-stored data, if the target storage location determined for the current data copy is determined with another data copy If the target storage locations are the same, the target storage location is re-determined for the current data copy, so that the target storage locations of all the data copies of the to-be-stored data are different.
  • the mapping manner includes: calculating a unique mapping value by using a hash algorithm after adjusting the original image.
  • the mapping manner includes: calculating a hash value by using a hash algorithm after adjusting the original image; multiplying the hash value by a preset weight value to obtain the mapping value; wherein the pre-predetermined
  • the weighted value is a sum of weight values of all leaf nodes that are in communication with the child nodes in the original image, and the weight value of the leaf node is positively correlated with the storage capacity of the storage medium device on the leaf node.
  • a server device is provided.
  • the server device of the present invention is disposed in a server in a distributed data storage system having a tree structure and having no central node, and is configured to save a plurality of data copies of the data to be stored into the distributed data storage system.
  • the tree structure includes a leaf node and an intermediate node, and the storage medium device is located on the leaf node, and the server device includes: a selecting module, configured to respectively perform, for each data copy of the data to be stored, according to the tree structure After the pre-selected node in the tree structure, the child node is selected step by step according to a preset child node selection manner until the selected child node is a leaf node, and then the leaf node is determined as a target storage location of the data copy; The sub-node is selected in the following manner: the combination of the identifier of the data to be saved and the identifier of the child node is used as the original image, and the mapping value associated with the child node is obtained according to a preset mapping manner, and then according to the
  • the size relationship between the plurality of mapping values of each child node of the intermediate node selects one child node of the intermediate node; and for each intermediate node, Resetting times before selecting its child nodes to the mapping relationship; storing module, for storing a target in said selection module determines that the data to be stored copies of all data After the location, all copies of the data of the pending data are saved to a target storage location of each data copy.
  • the selecting module is further configured to, in the process of determining a target storage location of all data copies of the to-be-stored data, for the specified one or more of the intermediate nodes, if the intermediate node is selected again Then, at this time, the selection is re-selected from all the sibling nodes of the intermediate node.
  • the selecting module is further configured to: when determining a target storage location of all data copies of the to-be-stored data, if the target storage location determined for the current data copy is determined with another data copy If the target storage locations are the same, the target storage location is re-determined for the current data copy, so that the target storage locations of all the data copies of the to-be-stored data are different.
  • the mapping manner includes: calculating a unique mapping value by using a hash algorithm after adjusting the original image.
  • the mapping manner includes: calculating a hash value by using a hash algorithm after adjusting the original image; multiplying the hash value by a preset weight value to obtain the mapping value; wherein the pre-predetermined
  • the weighted value is a sum of weight values of all leaf nodes that are in communication with the child nodes in the original image, and the weight value of the leaf node is positively correlated with the storage capacity of the storage medium device on the leaf node.
  • the method further includes a confirmation module, configured to trigger, when the node in the distributed data storage system changes, a respective pre-selected copy of the plurality of data stored in the current distributed data storage system
  • the selection module in the server is further configured to: when the trigger module is triggered by the confirmation module, use the pre-selected copy and other copies that are the same as the copy as the to-be-stored data, and determine the to-be-stored data. A new target storage location for each copy of the data, and then migrating the data copy in the respective data copy that was not the original target storage location to the new target storage location.
  • the selection method selects the child node step by step until the selected child node is a leaf node, and then determines the leaf node as the target storage location of the data copy, that is, selects the child node from the selected intermediate node until the storage medium
  • the change can Dispersing the target storage locations of multiple copies as much as possible helps to improve the robustness of the distributed data storage system in the event of equipment failure, thereby improving the reliability of the distributed data storage system.
  • one or more intermediate nodes can be specified, so that only one copy of the data to be stored is on the storage device of the intermediate node, that is, the data copies are distributed on different intermediate nodes such as a rack or a host, so that when When the intermediate node fails, for example, when the rack or a host cannot be serviced, other copies can still be served, thereby reducing the risk and probability of data loss and non-acquisition, and improving the reliability of the distributed data storage system.
  • new storage devices in distributed data storage systems there are quite a few copies of data that do not need to be migrated. The amount of data migration is relatively small, and the impact on system efficiency is low, making the distributed data storage system easier to expand.
  • FIG. 1 is a schematic diagram of a physical topology of a storage device in a distributed data storage system related to the present invention
  • FIG. 2 is a schematic diagram of a connection state between a terminal device and a distributed data storage system according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a method of storing a copy of data in accordance with an embodiment of the present invention
  • FIG. 4 is a schematic diagram of a basic structure of a terminal device according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a basic structure of a server device according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of a connection state between a terminal device and a distributed data storage system according to an embodiment of the present invention.
  • a plurality of terminal devices 21, 22, ..., 2M are connected to the distributed data storage system 20.
  • the physical topology of the distributed data storage system 20 is also in the form of a tree as shown in FIG. structure.
  • Each terminal device holds information of the physical topology such that the terminal device can calculate a target storage location of each data copy to be stored in system 20.
  • the terminal device sequentially calculates a target storage location of each data copy in the system 20 for a plurality of data copies of the data to be stored in the system 20, that is, determines a leaf node in the system 20 for each data copy.
  • a copy of the data is saved in a storage medium device located at the leaf node.
  • the child node is selected step by step from the one of the intermediate nodes specified by the user in the system 20 until the selected child node is a leaf node.
  • the process of determining the target storage location of a data copy is shown in Figure 3.
  • 3 is a schematic diagram of a method of storing a copy of data in accordance with an embodiment of the present invention.
  • Step S31 Acquire an identifier of the initial intermediate node set by the user. From this step, through the loop process, the target storage location of each copy of the data to be stored is determined, and a result set is obtained, which is a set of target storage locations of all the copies of the data to be saved.
  • the initial intermediate node is taken as an example of the equipment room 2.
  • Step S32 Select one of the plurality of child nodes of the current node.
  • the current node is the starting intermediate node set by the user.
  • the score of each child node is first obtained, and the preferred method is based on the standard of the data to be saved.
  • the identification (such as the file name of the data) and the identifier of the child node are obtained according to the mapping manner in the policy, and the mapping value is the score of the child node for the data to be saved.
  • a hash algorithm may be specifically used for calculation.
  • a hash function is used to calculate a hash for the combination of the identifier of the data to be stored and the identifier of the child node and an adjustable parameter. Value, the hash value is taken as the above score. After deriving the score of each child node of the starting intermediate node, one child node is selected according to the size of each score, for example, the child node with the highest score is selected.
  • mapping Another way of mapping, that is, calculating the above-mentioned score is to consider storing data in a storage medium device with a large capacity as much as possible, and therefore assigning weight to each leaf node, the weight and the storage medium device on the leaf node
  • the storage capacity is positively related.
  • a 1TB hard disk has a weight of 1
  • a 3TB hard disk has a weight of 3.
  • the weight of the intermediate node is the sum of the weights of all its child nodes.
  • the score of the node is multiplied by the weight of the node based on the above hash value.
  • Step S33 It is judged whether the node selected in step S32 is a leaf node. If yes, go to step S34, otherwise go to step S38. Returning to step S32 after step S38, the current node in step S32 is the node judged in step S33.
  • Step S34 It is judged whether the leaf node selected in step S32 already exists in the result set. In order to improve reliability, multiple copies of a piece of data to be stored are distributed among different storage medium devices. Therefore, if the result of the determination in this step is YES, the selection of the leaf node is discarded, and the process proceeds to step S37, and the target storage location of the current copy is re-determined in step S32 executed again; otherwise, the current leaf is reserved. The selection of the node proceeds to step S35.
  • Step S35 Put the leaf nodes selected in step S32 into the result set. After this step, the process proceeds to step S36.
  • Step S36 It is judged whether all copies of the current data to be stored have determined the target storage location, and if so, the process ends, otherwise the process proceeds to step S37.
  • Step S37 setting the initial intermediate node set by the user as the current node. Then it proceeds to step S38.
  • Step S38 Modify the mapping mode in step S32. That is, the adjustable parameters in step S32 are changed. The purpose of this is to change the score of the node to change the selection result of the node in step S32. There are many ways to adjust the adjustable parameters. As long as the final score is changed, the generated random number can be used as the parameter, or the parameter can be added with a value greater than zero for each adjustment. Fixed or random value. Of course, the method of changing the hash function may also be used to modify the mapping mode in step S32. This step returns to step S32.
  • the sub-node is selected from the selected intermediate node until the leaf node where the storage medium device is located, and for each intermediate node, each time
  • the selection result is changed as much as possible.
  • the change can make the target storage locations of the multiple copies as scattered as possible, which helps to improve
  • the robustness of distributed data storage systems in the event of equipment failures increases the reliability of distributed data storage systems.
  • a preferred mode is to designate one or more intermediate nodes.
  • a step is added between step S32 and step S33, in which step S32 is determined. Whether the selected child node has been selected. There are two cases here.
  • One case is that for a data copy, after determining the target storage location for the first time, it is found that there are other data copies on the target storage location (ie, the result of the determination in step S34 is YES), In the process of redetermining the target storage location of the data copy, the selected child node is the same as the child node selected when determining the target storage location of the data copy for the first time. The probability of this situation is lower because the middle goes through step S38.
  • Another case is that for different data copies, the same child node is selected in the process of determining their target storage location, for example, the target storage location of the data copy 1 is determined to be the hard disk 2 in the host 2, and the target of the data copy 2 is determined.
  • the host 2 is also selected at the time of executing step S32.
  • it is judged whether or not the currently selected intermediate node has been selected and if the result of the determination is "YES", the process proceeds directly to step S38, and in step S32 which is re-executed next.
  • the parent node of the intermediate node that is repeatedly selected is taken as the current node, that is, reselected in the sibling node of the repeatedly selected intermediate node. In the above example, if it is confirmed that the host 2 has been selected, the other host is reselected in the rack where the host 2 is located.
  • a data to be saved have only one copy on a designated intermediate node such as a rack, a host, etc., that is, distribute the data copies on different racks or hosts, so that when a rack or When a host cannot be serviced, there are still other copies that can be serviced, thereby reducing the risk and probability of data loss and non-acquisition, and improving the reliability of the distributed data storage system.
  • a designated intermediate node such as a rack, a host, etc.
  • the flow shown in FIG. 3 is executed by the terminal device, and the target storage location of each data copy of a piece of data to be stored in the distributed data storage system can be determined.
  • new storage devices are often needed in the system, such as adding a hard disk, adding a host, or adding a rack, etc., that is, the node will change. And need to migrate some of the data from the existing device to the new storage device.
  • distributed data The host in the storage system also has the function of determining the target storage location of the data copy that the terminal device has, so that the new data in the distributed data storage system can be added to the existing data in the distributed data storage system. The storage location is determined.
  • each host performs calculations to make the determination.
  • this calculation is only performed when a new storage device appears in the system, and it can be seen from the flow of FIG. 3 that the calculation is relatively simple and consumes less CPU resources, so the host re-determines the new storage of the stored data.
  • the work of the location does not have a big impact on system performance.
  • only one thread of a host performs the above calculation, which is a host that holds a copy of the stored data, and can determine each of the stored data in the terminal device.
  • the host is set when the destination of the data copy is stored.
  • the terminal device sequentially determines the target storage locations of the first to eighth copies of the data A to be stored, wherein the first copy is saved on one hard disk of the fourth host of the second rack, and then the data is transmitted.
  • the host that saves the first copy of the data A that is, the first copy of the target storage location is determined by the terminal device
  • the second machine is the second machine.
  • the fourth host of the rack calculates a new target storage location of the eight copies of the data A.
  • the fourth host can execute according to the flow of FIG.
  • the new target storage location may be the same as the original storage location, especially in the physical topology where the storage location is far away from the newly added storage device.
  • the possibility of a change in storage location is quite low. That is to say, when a new storage device is added to a distributed data storage system, a considerable number of data copies do not need to be migrated, the data migration amount is relatively small, and the impact on system efficiency is low, so that the distributed data storage system is relatively easy to expand.
  • the terminal device is provided in a terminal device such as a personal computer and is connected to a distributed data storage system having a tree structure via the Internet.
  • 4 is a schematic diagram of a basic structure of a terminal device according to an embodiment of the present invention. As shown in FIG. 4, the terminal device 40 mainly includes a selection module 41 and an uploading module 42.
  • the selecting module 41 is configured to select, according to the tree structure of the distributed data storage system, each data copy of the data to be stored, from the pre-selected nodes in the tree structure, according to a preset sub-node selection manner.
  • the child node is determined to be the target storage location of the data copy until the selected child node is the leaf node; wherein the child node is selected by the combination of the identifier of the data to be stored and the identifier of the child node as the original image, Presetting the mapping manner to obtain a mapping value associated with the child node, and then selecting a child node of the intermediate node according to a size relationship between the plurality of mapping values of each child node associated with the same intermediate node; and for each The intermediate node resets the mapping relationship each time its child nodes are selected.
  • the uploading module 42 is configured to send, after the selection module 41 determines the target storage location of all data copies of the data to be stored, the information of the target storage location of all data copies of the data to be stored and all data copies of the data to be stored to the distributed A computer in a data storage system for the computer to save a copy of all data of the data to be stored to a destination storage location for each data copy.
  • the selection module 41 is further configured to, in the process of determining a target storage location of all data copies of the data to be stored, for the specified one or more intermediate nodes, if the intermediate node is selected again, then the intermediate node is Reselect in all sibling nodes.
  • the selection module 41 is further configured to: in determining a target storage location of all data copies of the data to be stored, if the target storage location determined for the current data copy is the same as the determined target storage location of another data copy, Then, the target storage location is re-determined for the current data copy, so that the target storage locations of all data copies of the data to be saved are different.
  • FIG. 5 is a schematic diagram of a basic structure of a server device according to an embodiment of the present invention.
  • the server device is disposed in a server in a distributed data storage system having a tree structure for storing a plurality of copies of data of the data to be stored in the distributed data storage system.
  • the server device 50 mainly includes a selection module 51 and a saving module 52. Its
  • the middle selection module 51 may adopt a similar structure to the selection module 51, and the save module 52 is configured to save all data copies of the data to be saved to each after the selection module 51 determines the target storage location of all data copies of the data to be stored. The target storage location for the data copy.
  • the server device 50 may further include a confirmation module (not shown) for triggering each of the plurality of data stored in the current distributed data storage system in the event of a change in a node in the distributed data storage system
  • the selection module 51 in the server where a pre-selected copy is located in this way, the selection module 51 is further configured to use the pre-selected copy and the other copy identical to the copy as the data to be saved, and determine the pending A new target storage location for each data copy in the data, and then the data copy in the original data copy that is not the original target storage location is migrated to the new target storage location.
  • the objects of the invention can also be achieved by running a program or a set of programs on any computing device.
  • the computing device can be a well-known general purpose device.
  • the object of the present invention can also be achieved by merely providing a program product comprising program code for implementing the method or apparatus. That is to say, such a program product also constitutes the present invention, and a storage medium storing such a program product also constitutes the present invention. It will be apparent that the storage medium may be any known storage medium or any storage medium developed in the future.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

提供一种存储数据副本的方法和终端装置以及服务器装置，有助于提高无中心节点的分布式数据存储系统的可靠性，并降低其在节点数目发生改变时的数据迁移量。该方法包括：根据无中心节点的分布式数据存储系统的树形结构，对于待存数据的每个数据副本，分别从预选中间节点起逐级选择子节点直至叶节点，将该叶节点确定为数据副本的目标存储位置；子节点选择方式为：以待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据映射值选择该中间节点的一个子节点；并且每次选择子节点之前重新设定所述映射关系；将待存数据的所有数据副本保存到各数据副本的目标存储位置。

Description

一种存储数据副本的方法和终端装置以及服务器装置 技术领域
本发明涉及计算机技术领域，特别地涉及一种存储数据副本的方法和终端装置以及服务器装置。
背景技术
分布式数据存储系统通常采用多副本的方式存储数据，以提高数据存储的可靠性。存储设备的物理拓扑结构一般是层级式的，参考图1，图1是与本发明有关的分布式数据存储系统中的存储设备的物理拓扑结构的示意图。如图1所示，分布式数据存储系统设置在数据中心10中，数据中心10由3个机房组成，每个机房中有多个机架，例如机房2中有机架1、机架2、……、机架N1；每个机架上又有多个主机(计算机)，例如机架1上有主机1、主机2、……、主机N2；每个主机中又有多个存储介质设备(通常是硬盘)，例如主机2中有硬盘1、硬盘2、……、硬盘N3。为示意清晰起见，图1中仅示出了部分设备。可以看出分布式数据存储系统是树形结构，存储介质设备位于叶节点上，而主机、机架、机房则是中间节点。
分布式数据存储系统可以分为带有中心节点和无中心节点两种形式。在带有中心节点的分布式数据存储系统中，一般含有客户端、中心节点及存储节点，同时将数据分块处理，多副本存储，而数据的分布位置由中心节点根据存储节点的负载情况、副本的存储策略来决定数据副本的存储位置。中心节点可以是两台服务器主备配置，也可以是服务器集群。在无中心节点的分布式数据存储系统中，各个节点彼此互相连接，数据随机的分布在各个节点的存储设备上，存储位置可以由节点使用哈希函数计算得到。
对于带有中心节点的分布式数据存储系统来说，从其中读取数据 时，需先访问中心节点以获取数据的存储位置，这使得中心节点负荷过重，处理效率降低，从而形成整个系统的瓶颈，降低了系统性能。并且在中心节点出现故障的情况下，该瓶颈会进一步加剧甚至使整个分布式数据存储系统不可用，可靠性较低。
无中心节点的分布式数据存储系统能够避免上述瓶颈，但其可靠性仍显不足。例如某个机架断电或出现网络故障，并且某项数据的所有副本都在该机架的主机的硬盘上，则该项数据不可获取。另外，在设备数量发生改变，例如增加一块硬盘，或者增加一台主机等情况，无中心节点的分布式数据存储系统必然出现广泛的数据迁移，以下举一个简单的例子对此加以说明。
例如无中心节点的分布式数据存储系统中有5个节点(实际上远不止5个)，由哈希函数计算得到一项待存数据的哈希值为13，按节点数取模，则为3，于是该待存数据被保存在节点3上；当增加设备，使节点数为6，则在读取数据时，将哈希值13对节点数6取模得2，即从节点2上读取数据，此时数据首先会从节点3迁移到节点2。因为节点数改变时取模结果必然不同，所以在节点数变化之后读取节点数变化之前保存的数据时必然发生数据迁移，因此一旦节点数发生变化，数据迁移就成为普遍现象，这将导致系统效率降低以及缩短存储介质寿命。
发明内容
有鉴于此，本发明提供一种存储数据副本的方法和终端装置以及服务器装置，有助于提高无中心节点的分布式数据存储系统的可靠性，并降低其在节点数目发生改变时的数据迁移量。
为实现上述目的，根据本发明的一个方面，提供了一种存储数据副本的方法。
本发明的存储数据副本的方法，应用于具有树形结构的无中心节点的分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，该方法包括：步骤A：根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；步骤B：在确定了所述待存数据的所有数据副本的目标存储位置之后，将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
可选地，对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
可选地，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本按所述步骤A重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出唯一的映射值。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出哈希值；将所述哈希值与预设权重值相乘得到所述映射值；其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重 值之和，所述叶节点的权重值与该叶节点上的存储介质设备的存储容量正相关。
可选地，所述重新设定所述映射关系的步骤包括：改变将所述原像调整时采用的参数。
根据本发明的另一方面，提供了一种终端装置。
本发明的终端装置用于将待存数据的多个数据副本保存到具有树形结构的无中心节点的分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，该终端装置包括：选择模块，用于根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；上传模块，用于在所述选择模块确定了所述待存数据的所有数据副本的目标存储位置之后，将所述待存数据的所有数据副本和所述待存数据的所有数据副本的目标存储位置的信息发送给所述分布式数据存储系统中的计算机，以供所述计算机将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
可选地，所述选择模块还用于：对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
可选地，所述选择模块还用于：在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出唯一的映射值。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出哈希值；将所述哈希值与预设权重值相乘得到所述映射值；其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重值之和，所述叶节点的权重值与该叶节点上的存储介质设备的存储容量正相关。
根据本发明的又一方面，提供了一种服务器装置。
本发明的服务器装置，设置在具有树形结构的无中心节点的分布式数据存储系统中的服务器内，用于将待存数据的多个数据副本保存到所述分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，该服务器装置包括：选择模块，用于根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；保存模块，用于在所述选择模块确定了所述待存数据的所有数据副本的目标存储 位置之后，将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
可选地，所述选择模块还用于：对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
可选地，所述选择模块还用于：在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出唯一的映射值。
可选地，所述映射方式包括：将所述原像调整之后使用哈希算法计算出哈希值；将所述哈希值与预设权重值相乘得到所述映射值；其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重值之和，所述叶节点的权重值与该叶节点上的存储介质设备的存储容量正相关。
可选地，还包括确认模块，用于在所述分布式数据存储系统中的节点发生改变的情况下触发当前所述分布式数据存储系统中保存的多项数据的各自的一个预选副本所在的服务器中的所述选择模块；所述选择模块还用于在被所述确认模块触发的情况下将所述预选副本以及与该副本相同的其他副本作为所述待存数据，确定该待存数据中的各个数据副本的新的目标存储位置，然后将该各个数据副本中原存储位置不为该新的目标存储位置的数据副本迁移到该新的目标存储位置。
根据本发明的技术方案，在确定副本的目标存储位置时，对于待存数据的每个数据副本，分别从分布式数据存储系统的树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置，也就是说，从选定的中间节点起层层选择子节点直至存储介质设备所在的叶节点，并且对于每个中间节点来说，每次选择其子节点时的选择方式有变化从而使选择结果尽可能地改变，在层层选择子节点的情况下，这种改变能够使多个副本的目标存储位置尽可能地分散，有助于提高分布式数据存储系统在设备出现故障时的鲁棒性，从而提高了分布式数据存储系统的可靠性。并且可以指定一个或多个中间节点，使一项待存数据只有一个副本在该中间节点的存储设备上，也就是说将数据副本分布在不同的中间节点例如机架或主机上，这样当一个中间节点失效例如机架或者一台主机不能服务时，仍有其他副本能够服务，从而减少数据丢失以及不可获取性的风险及概率，提高分布式数据存储系统的可靠性。在分布式数据存储系统新增存储设备的情况下，有相当多的数据副本并不需要迁移，数据迁移量相当小，对系统效率的影响也较低，使得分布式数据存储系统比较易于扩容。
附图说明
附图用于更好地理解本发明，不构成对本发明的不当限定。其中：
图1是与本发明有关的分布式数据存储系统中的存储设备的物理拓扑结构的示意图；
图2是根据本发明实施例的终端设备与分布式数据存储系统连接状态的示意图；
图3是根据本发明实施例的存储数据副本的方法的示意图；
图4是根据本发明实施例的终端装置的基本结构的示意图；
图5是根据本发明实施例的服务器装置的基本结构的示意图。
具体实施方式
以下结合附图对本发明的示范性实施例做出说明，其中包括本发明实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本发明的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。
图2是根据本发明实施例的终端设备与分布式数据存储系统连接状态的示意图。如图2所示，多个终端设备21、22、……、2M与分布式数据存储系统20连接，本实施例中该分布式数据存储系统20的物理拓扑同样采用图1所示的树状结构。各个终端设备保存有该物理拓扑的信息，从而终端设备能够计算要存储到系统20中的各数据副本的目标存储位置。
终端设备对于待存到系统20中的数据的多个数据副本，依次计算每个数据副本在系统20中的目标存储位置，也就是说为每个数据副本确定出系统20中的一个叶节点，数据副本保存在位于该叶节点的存储介质设备中。在确定叶节点时，从用户在系统20中指定的一个中间节点起，逐级选择子节点直至被选子节点为叶节点。确定一个数据副本的目标存储位置的流程如图3所示。图3是根据本发明实施例的存储数据副本的方法的示意图。
步骤S31：获取用户设定的起始中间节点的标识。从本步骤起，通过循环的流程，对一项待存数据的各个副本的目标存储位置加以确定，得到一个结果集，该结果集是一项待存数据的所有副本的目标存储位置的集合。在以下的步骤中，参考图1，以起始中间节点为机房2为例进行说明。
步骤S32：从当前节点的多个子节点中选择一个子节点。在第一次执行本步骤时，该当前节点为用户设定的起始中间节点。在选择子节点时，先得出每个子节点的得分，优选的方式是根据待存数据的标 识(例如数据的文件名)和子节点的标识按照策略中的映射方式得出一个映射值，该映射值即为该子节点对于本次待存数据的得分。在上述的映射方式中，具体可以采用哈希算法进行计算。因为在后面的步骤S36中需要改变映射方式，所以在本步骤中，对于待存数据的标识和子节点的标识以及一个可调整的参数这三者的组合，采用哈希函数进行计算得到一个哈希值，将该哈希值作为上述得分。在得出起始中间节点的每个子节点的得分之后，根据各个得分的大小选择一个子节点，例如选择得分最高的子节点。
另一种映射方式即计算上述的得分的方式是考虑到尽量把数据保存到容量较大的存储介质设备中，因此对每个叶节点赋予权重，该权重与该叶节点上的存储介质设备的存储容量正相关，例如1TB的硬盘的权重为1，3TB的硬盘的权重为3。中间节点的权重是其所有子节点权重之和。节点的得分即在上述的哈希值的基础上再乘以节点的权重。这样，在按照得分最高来选择一个中间节点的子节点时，存储容量更大的节点就有更大的可能性被选取。
步骤S33：判断步骤S32中选择的节点是否为叶节点。若是，则进入步骤S34，否则进入步骤S38。在步骤S38之后返回步骤S32，此时步骤S32中的当前节点为步骤S33中被判断的节点。
步骤S34：判断步骤S32中选择的叶节点是否已存在于结果集中。为了提高可靠性，要使一项待存数据的多个副本分布在不同的存储介质设备中。所以本步骤的判断结果若为是，则舍弃本次对叶节点的选择，并且进入步骤S37，并在再次执行的步骤S32中对当前副本的目标存储位置重新加以确定；否则保留本次对叶节点的选择，进入步骤S35。
步骤S35：将步骤S32中选择的叶节点放入结果集中。本步骤之后进入步骤S36。
步骤S36：判断当前待存数据的所有副本是否都已确定目标存储位置，若是，结束流程，否则进入步骤S37。
步骤S37：将用户设定的起始中间节点设定为当前节点。然后进入步骤S38。
步骤S38：修改步骤S32中的映射方式。即改变步骤S32中的可调整的参数。这样做的目的是改变节点的得分从而改变步骤S32中对节点的选择结果。对该可调整的参数进行调整的方式可以有很多，只要达到了改变最终的得分即可，可以是将生成的随机数作为该参数，也可以是每次调整时对该参数累加一个大于零的固定或随机值。当然，也可以采用改变哈希函数的方法来修改步骤S32中的映射方式。本步骤之后返回步骤S32。
从以上的流程可以看出，在确定副本的目标存储位置时，是从选定的中间节点起层层选择子节点直至存储介质设备所在的叶节点，并且对于每个中间节点来说，每次选择其子节点时的选择方式有变化从而使选择结果尽可能地改变，在层层选择子节点的情况下，这种改变能够使多个副本的目标存储位置尽可能地分散，有助于提高分布式数据存储系统在设备出现故障时的鲁棒性，从而提高了分布式数据存储系统的可靠性。
当然，如上所述，在以上的流程中每次选择其子节点时的选择方式有变化从而使选择结果尽可能地改变，但这种改变并不是绝对的，在实际中仍以一个较小的概率会出现两次选择结果相同的情况。例如副本1被确定在机架1下的某个主机的硬盘上，副本2也被确定在机架1下的某个主机的硬盘上，这样这两个副本就会因为机架1的整体故障(例如断电、网络故障等)变得不可用。另外在实际的分布式数据存储系统中，更为理想的方式是将每个副本都分布在某一层级的不同存储位置上，例如有8个副本，系统中有10个机架，则最好是这8 个副本分布在不同的机架上。或者因某些实际情况，有些机架的可靠性较低，则最好不要使多个副本都处于这样的机架上的存储设备中。所以在本实施例中，一个优选的方式是指定一个或多个中间节点，当执行图3所示的流程时，向步骤S32和步骤S33之间增加一个步骤，在该步骤中判断步骤S32中选择的子节点是否曾经被选择过。这里分两种情况，一种情况是对于一个数据副本，在第一次确定其目标存储位置之后发现该目标存储位置上已有其他数据副本(即步骤S34中判断结果为是的情形)，在该重新确定该数据副本的目标存储位置的过程中，选择的子节点与第一次确定该数据副本的目标存储位置时选择的子节点相同。这种情况的概率较低，因为中间要经过步骤S38。另一种情况是对于不同的数据副本，在确定它们的目标存储位置的过程中选择了同一子节点，例如确定数据副本1的目标存储位置是主机2中的硬盘2，确定数据副本2的目标存储位置的过程中，在一次执行步骤S32时也选中了主机2。此时在步骤S32和步骤S33之间增加的步骤中判断当前选择的中间节点是否曾经被选择过，并且判断结果为“是”，则直接进入步骤S38，并且在接下来重新执行的步骤S32中，将被重复选择的中间节点的父节点作为当前节点，也就是说在该被重复选择的中间节点的兄弟节点中重新选择。如上例，确认主机2曾经被选择过，则在主机2所在机架中重新选择另一台主机。通过以上的处理，能够使一项待存数据在指定的中间节点例如机架、主机等上只有一个副本，也就是说将数据副本分布在不同的机架或主机上，这样当一个机架或者一台主机不能服务时，仍有其他副本能够服务，从而减少数据丢失以及不可获取性的风险及概率，提高分布式数据存储系统的可靠性。
由终端设备执行图3所示的流程，能够对一项待存数据的各个数据副本在分布式数据存储系统中的目标存储位置得以确定。对于该分布式数据存储系统来说，由于数据量的不断增加，系统中常常需要新增存储设备，例如增加一个硬盘、增加一台主机、或者增加一个机架等，也就是说节点会发生改变；并且需要将已有设备中的一些数据迁移到该新增存储设备中。对于这种情况，在本实施例中，分布式数据 存储系统中的主机也具备上述的终端设备所具有的确定数据副本的目标存储位置的功能，从而能够在分布式数据存储系统新增存储设备的情况下，对该系统中的已存数据的新的存储位置加以确定，需要说明的是，只要系统中出现新增存储设备的情况，各个主机都会进行计算以做出该确定。当然这种计算仅是在系统中出现新增存储设备时进行，而且从图3的流程可以看出这种计算比较简单，占用的CPU资源较少，所以主机重新确定已存数据的新的存储位置的工作并不会对系统性能带来太大的影响。
另外，对于一项已存数据，只需由一台主机的一个线程来执行上述计算，该是保存有该项已存数据的一个副本的主机，可以在终端设备确定该项已存数据的各个数据副本的目标存储位置时就设定该主机。例如，终端设备依次确定了待存数据A的第1个至第8个副本的目标存储位置，其中第1个副本将保存在第2机架的第4主机的一个硬盘上，然后将数据发送给各个目标存储位置所在的主机，当分布式数据存储系统新增存储设备时，由保存数据A的第1个副本(即最先被终端设备确定目标存储位置的副本)的主机即第2机架的第4主机计算数据A的8个副本的新的目标存储位置，此时该第4主机可以按图3的流程来执行。从图3的流程可以看出，对于这8个副本来说，新的目标存储位置有可能与原有存储位置相同，特别是在物理拓扑上存储位置与新增存储设备距离较远的数据副本，存储位置发生改变的可能性相当低。也就是说分布式数据存储系统新增存储设备时，有相当多的数据副本并不需要迁移，数据迁移量相当小，对系统效率的影响也较低，使得分布式数据存储系统比较易于扩容。
以下对于本发明实施例中的终端装置的基本结构做出说明。该终端装置设置在终端设备例如个人计算机中，通过互联网与具有树形结构的分布式数据存储系统连接。图4是根据本发明实施例的终端装置的基本结构的示意图。如图4所示，终端装置40主要包括选择模块41和上传模块42。
选择模块41用于根据分布式数据存储系统的树形结构，对于待存数据的每个数据副本，分别从该树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；其中子节点选择方式为：以待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定映射关系。
上传模块42用于在选择模块41确定了待存数据的所有数据副本的目标存储位置之后，将待存数据的所有数据副本和待存数据的所有数据副本的目标存储位置的信息发送给分布式数据存储系统中的计算机，以供该计算机将待存数据的所有数据副本保存到各数据副本的目标存储位置。
选择模块41还可用于：对于指定的一个或多个中间节点，在确定待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
选择模块41还可用于：在确定待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本重新确定目标存储位置，使待存数据的所有数据副本的目标存储位置各不相同。
图5是根据本发明实施例的服务器装置的基本结构的示意图。该服务器装置设置在具有树形结构的分布式数据存储系统中的服务器内，用于将待存数据的多个数据副本保存到该分布式数据存储系统中。如图5所示，服务器装置50主要包括选择模块51和保存模块52。其 中选择模块51可以采用与选择模块51类似的结构，而保存模块52用于在选择模块51确定了待存数据的所有数据副本的目标存储位置之后，将待存数据的所有数据副本保存到各数据副本的目标存储位置。
服务器装置50还可以包括确认模块(图中未示出)，该确认模块用于在分布式数据存储系统中的节点发生改变的情况下触发当前分布式数据存储系统中保存的多项数据的各自的一个预选副本所在的服务器中的选择模块51；这样，选择模块51还用于在被确认模块触发的情况下将该预选副本以及与该副本相同的其他副本作为待存数据，确定该待存数据中的各个数据副本的新的目标存储位置，然后将该各个数据副本中原存储位置不为该新的目标存储位置的数据副本迁移到该新的目标存储位置。
以上结合具体实施例描述了本发明的基本原理，但是，需要指出的是，对本领域的普通技术人员而言，能够理解本发明的方法和设备的全部或者任何步骤或者部件，可以在任何计算装置(包括处理器、存储介质等)或者计算装置的网络中，以硬件、固件、软件或者它们的组合加以实现，这是本领域普通技术人员在阅读了本发明的说明的情况下运用他们的基本编程技能就能实现的。
因此，本发明的目的还可以通过在任何计算装置上运行一个程序或者一组程序来实现。所述计算装置可以是公知的通用装置。因此，本发明的目的也可以仅仅通过提供包含实现所述方法或者装置的程序代码的程序产品来实现。也就是说，这样的程序产品也构成本发明，并且存储有这样的程序产品的存储介质也构成本发明。显然，所述存储介质可以是任何公知的存储介质或者将来开发出的任何存储介质。
还需要指出的是，在本发明的装置和方法中，显然，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本发明的等效方案。并且，执行上述系列处理的步骤可以自然地按照说明 的顺序按时间顺序执行，但是并不需要一定按照时间顺序执行。某些步骤可以并行或彼此独立地执行。
上述具体实施方式，并不构成对本发明保护范围的限制。本领域技术人员应该明白的是，取决于设计要求和其他因素，可以发生各种各样的修改、组合、子组合和替代。任何在本发明的精神和原则之内所作的修改、等同替换和改进等，均应包含在本发明保护范围之内。

Claims (17)

  1. 一种存储数据副本的方法，应用于具有树形结构的无中心节点的分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，其特征在于，该方法包括：
    步骤A：根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；
    其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；
    步骤B：在确定了所述待存数据的所有数据副本的目标存储位置之后，将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
  2. 根据权利要求1所述的方法，其特征在于，对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
  3. 根据权利要求1所述的方法，其特征在于，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本按所述步骤A重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
  4. 根据权利要求1所述的方法，其特征在于，所述映射方式包括： 将所述原像调整之后使用哈希算法计算出唯一的映射值。
  5. 根据权利要求1所述的方法，其特征在于，所述映射方式包括：
    将所述原像调整之后使用哈希算法计算出哈希值；
    将所述哈希值与预设权重值相乘得到所述映射值；
    其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重值之和，所述叶节点的权重值与该叶节点上的存储介质设备的存储容量正相关。
  6. 根据权利要求4或5所述的方法，其特征在于，所述重新设定所述映射关系的步骤包括：改变将所述原像调整时采用的参数。
  7. 一种终端装置，用于将待存数据的多个数据副本保存到具有树形结构的无中心节点的分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，其特征在于，该终端装置包括：
    选择模块，用于根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定为该数据副本的目标存储位置；其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；
    上传模块，用于在所述选择模块确定了所述待存数据的所有数据副本的目标存储位置之后，将所述待存数据的所有数据副本和所述待存数据的所有数据副本的目标存储位置的信息发送给所述分布式数据存储系统中的计算机，以供所述计算机将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
  8. 根据权利要求7所述的终端装置，其特征在于，所述选择模块还用于：对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
  9. 根据权利要求7所述的终端装置，其特征在于，所述选择模块还用于：在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
  10. 根据权利要求7所述的终端装置，其特征在于，所述映射方式包括：将所述原像调整之后使用哈希算法计算出唯一的映射值。
  11. 根据权利要求7所述的终端装置，其特征在于，所述映射方式包括：
    将所述原像调整之后使用哈希算法计算出哈希值；
    将所述哈希值与预设权重值相乘得到所述映射值；
    其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重值之和，所述叶节点的权重值与该叶节点上的存储介质设备的存储容量正相关。
  12. 一种服务器装置，设置在具有树形结构的无中心节点的分布式数据存储系统中的服务器内，用于将待存数据的多个数据副本保存到所述分布式数据存储系统中，所述树形结构包括叶节点和中间节点，存储介质设备位于所述叶节点上，其特征在于，该服务器装置包括：
    选择模块，用于根据所述树形结构，对于待存数据的每个数据副本，分别从所述树形结构中的预选中间节点起，按预设的子节点选择方式逐级选择子节点直至被选子节点为叶节点，然后将该叶节点确定 为该数据副本的目标存储位置；其中所述子节点选择方式为：以所述待存数据的标识和子节点的标识的组合为原像，按预设的映射方式得出关联于该子节点的映射值，再根据关联于同一所述中间节点的各个子节点的多个映射值之间的大小关系选择该中间节点的一个子节点；并且对于每个中间节点，在每次选择其子节点之前重新设定所述映射关系；
    保存模块，用于在所述选择模块确定了所述待存数据的所有数据副本的目标存储位置之后，将所述待存数据的所有数据副本保存到各数据副本的目标存储位置。
  13. 根据权利要求12所述的服务器装置，其特征在于，所述选择模块还用于：对于指定的一个或多个所述中间节点，在确定所述待存数据的所有数据副本的目标存储位置的过程中，若该中间节点被再次选择，则此时从该中间节点的所有兄弟节点中重新进行选择。
  14. 根据权利要求12所述的服务器装置，其特征在于，所述选择模块还用于：在确定所述待存数据的所有数据副本的目标存储位置的过程中，若对当前数据副本确定出的目标存储位置与另一数据副本的已确定出的目标存储位置相同，则对当前数据副本重新确定目标存储位置，使所述待存数据的所有数据副本的目标存储位置各不相同。
  15. 根据权利要求12所述的服务器装置，其特征在于，所述映射方式包括：将所述原像调整之后使用哈希算法计算出唯一的映射值。
  16. 根据权利要求12所述的服务器装置，其特征在于，所述映射方式包括：
    将所述原像调整之后使用哈希算法计算出哈希值；
    将所述哈希值与预设权重值相乘得到所述映射值；
    其中所述预设权重值是与所述原像中的子节点连通的所有叶节点的权重值之和，所述叶节点的权重值与该叶节点上的存储介质设备的 存储容量正相关。
  17. 根据权利要求12至16中任一项所述的服务器装置，其特征在于，
    还包括确认模块，用于在所述分布式数据存储系统中的节点发生改变的情况下触发当前所述分布式数据存储系统中保存的多项数据的各自的一个预选副本所在的服务器中的所述选择模块；
    所述选择模块还用于在被所述确认模块触发的情况下将所述预选副本以及与该副本相同的其他副本作为所述待存数据，确定该待存数据中的各个数据副本的新的目标存储位置，然后将该各个数据副本中原存储位置不为该新的目标存储位置的数据副本迁移到该新的目标存储位置。
PCT/CN2014/087621 2013-12-03 2014-09-26 一种存储数据副本的方法和终端装置以及服务器装置 Ceased WO2015081750A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/039,342 US10165048B2 (en) 2013-12-03 2014-09-26 Data copy storage method, terminal apparatus, and server apparatus
RU2016126187A RU2637434C1 (ru) 2013-12-03 2014-09-26 Способ, устройство терминала и устройство сервера для хранения копий данных

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310641751.7A CN103634401B (zh) 2013-12-03 2013-12-03 一种存储数据副本的方法和终端装置以及服务器装置
CN201310641751.7 2013-12-03

Publications (1)

Publication Number Publication Date
WO2015081750A1 true WO2015081750A1 (zh) 2015-06-11

Family

ID=50215017

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/087621 Ceased WO2015081750A1 (zh) 2013-12-03 2014-09-26 一种存储数据副本的方法和终端装置以及服务器装置

Country Status (4)

Country Link
US (1) US10165048B2 (zh)
CN (1) CN103634401B (zh)
RU (1) RU2637434C1 (zh)
WO (1) WO2015081750A1 (zh)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103634401B (zh) 2013-12-03 2015-11-25 北京京东尚科信息技术有限公司 一种存储数据副本的方法和终端装置以及服务器装置
CN105991705B (zh) * 2015-02-10 2020-04-28 中兴通讯股份有限公司 一种分布式存储系统及其实现资源硬亲和性的方法
CN105573679B (zh) * 2015-12-18 2019-10-11 国云科技股份有限公司 一种适用于分布式存储系统存储池资源映射规则的方法
CN105630973A (zh) * 2015-12-25 2016-06-01 深圳市中博科创信息技术有限公司 集群文件系统文件存储的方法及集群文件系统
US10638259B2 (en) * 2017-08-09 2020-04-28 Dell Products, L.P. Systems and method for mapping systems in a server rack based on neighboring systems
CN107657027B (zh) * 2017-09-27 2021-09-21 北京小米移动软件有限公司 数据存储方法及装置
CN107592368B (zh) * 2017-10-10 2020-06-16 武汉斗鱼网络科技有限公司 分布式数据同步路由选择方法、存储介质、设备及系统
CN108491478A (zh) * 2018-03-09 2018-09-04 深圳市瑞驰信息技术有限公司 一种改进型的分布式存储系统的数据分布方法及系统
CN109471894B (zh) * 2018-10-29 2021-12-24 深圳市瑞驰信息技术有限公司 一种新型分散式文件与对象统一存储的系统及方法
CN109740593B (zh) * 2018-12-18 2020-11-13 全球能源互联网研究院有限公司 样本中至少一个预定目标的位置确定方法及装置
CN109992206B (zh) * 2019-03-27 2022-05-10 新华三技术有限公司成都分公司 数据分布存储方法及相关装置
CN110099112B (zh) * 2019-04-28 2022-03-29 平安科技（深圳）有限公司 基于点对点网络的数据存储方法、装置、介质及终端设备
CN112511578B (zh) * 2019-09-16 2022-06-24 大唐移动通信设备有限公司 数据存储方法及装置
CN112036772A (zh) * 2020-09-27 2020-12-04 上海松鼠课堂人工智能科技有限公司 一种学生数据的追溯分析方法
CN114650296B (zh) * 2020-12-18 2023-03-24 中国科学院声学研究所 一种信息中心网络副本选择方法
CN113849307B (zh) * 2021-09-22 2025-11-21 网易（杭州）网络有限公司 一种数据存储和读取方法、系统、装置、设备及介质
CN115352835A (zh) * 2022-08-19 2022-11-18 广东南牧机械设备有限公司 多层级多单元的干料输送系统控制方法和装置
CN115357768B (zh) * 2022-08-19 2026-02-03 浪潮电子信息产业股份有限公司 分布式存储集群的管理方法及相关组件
CN115510292B (zh) * 2022-11-18 2023-03-24 四川汉唐云分布式存储技术有限公司 一种分布式存储系统树搜索管理方法、装置、设备及介质
CN116166202B (zh) * 2023-03-30 2023-07-21 中国华能集团清洁能源技术研究院有限公司 一种大数据环境下的副本放置方法、装置、设备及介质
CN118409867B (zh) * 2024-06-26 2024-10-29 济南浪潮数据技术有限公司 数据均衡方法、计算机程序产品、设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100146128A1 (en) * 2006-10-05 2010-06-10 National Ict Australia Limited Decentralised multi-user online environment
CN102571991A (zh) * 2012-03-27 2012-07-11 电子科技大学 基于多级映射的大规模多副本分布式存储系统及其应用方法
CN102916811A (zh) * 2012-10-18 2013-02-06 中国科学院信息工程研究所 一种多元实体身份凭证信息存储方法
CN103634401A (zh) * 2013-12-03 2014-03-12 北京京东尚科信息技术有限公司 一种存储数据副本的方法和终端装置以及服务器装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5132573B2 (ja) * 2005-11-29 2013-01-30 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 分散型記憶システムを管理する方法
US7647329B1 (en) * 2005-12-29 2010-01-12 Amazon Technologies, Inc. Keymap service architecture for a distributed storage system
US7945751B2 (en) * 2006-12-18 2011-05-17 International Business Machines Corporation Disk image inheritance
CN101252603B (zh) * 2008-04-11 2011-03-30 清华大学 基于存储区域网络san的集群分布式锁管理方法
CA2894623C (en) * 2012-12-27 2019-09-10 Huawei Technologies Co., Ltd. Partition extension method and apparatus
US9418131B1 (en) * 2013-09-24 2016-08-16 Emc Corporation Synchronization of volumes

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100146128A1 (en) * 2006-10-05 2010-06-10 National Ict Australia Limited Decentralised multi-user online environment
CN102571991A (zh) * 2012-03-27 2012-07-11 电子科技大学 基于多级映射的大规模多副本分布式存储系统及其应用方法
CN102916811A (zh) * 2012-10-18 2013-02-06 中国科学院信息工程研究所 一种多元实体身份凭证信息存储方法
CN103634401A (zh) * 2013-12-03 2014-03-12 北京京东尚科信息技术有限公司 一种存储数据副本的方法和终端装置以及服务器装置

Also Published As

Publication number Publication date
US20170048313A1 (en) 2017-02-16
US10165048B2 (en) 2018-12-25
RU2637434C1 (ru) 2017-12-04
HK1195832A1 (zh) 2014-11-21
CN103634401B (zh) 2015-11-25
CN103634401A (zh) 2014-03-12

Similar Documents

Publication Publication Date Title
WO2015081750A1 (zh) 一种存储数据副本的方法和终端装置以及服务器装置
CN108780386B (zh) 一种数据存储的方法、装置和系统
US9665428B2 (en) Distributing erasure-coded fragments in a geo-distributed storage system
CN104781795B (zh) 存储层的动态选择
CN103929454B (zh) 一种云计算平台中负载均衡存储的方法和系统
US10616134B1 (en) Prioritizing resource hosts for resource placement
CN106445677A (zh) 负载均衡方法及设备
CN105187547A (zh) 一种负载均衡方法与系统
CN108769123B (zh) 一种数据系统及数据处理方法
CN107463342B (zh) 一种cdn边缘节点文件的存储方法及装置
WO2019148841A1 (zh) 一种分布式存储系统、数据处理方法和存储节点
CN104063501B (zh) 基于hdfs的副本平衡方法
CN106527985A (zh) 一种基于ceph的存储交互装置及存储系统
CN106293492A (zh) 一种存储管理方法及分布式文件系统
JP2020173727A (ja) ストレージ管理装置、情報システム、及びストレージ管理方法
CN107729514A (zh) 一种基于hadoop的副本放置节点确定方法及装置
CN115834587A (zh) 一种选择目标存储服务器的方法、装置及电子设备
KR20170045928A (ko) 인메모리 데이터베이스를 이용한 데이터 관리 방법 및 그 장치
US12141105B2 (en) Data placement selection among storage devices associated with nodes of a distributed file system cluster
US11971902B1 (en) Data retrieval latency management system
CN105915626B (zh) 一种面向云存储的数据副本初始放置方法
JP2019527414A (ja) 所望のストレージ・ユニット・セットを特定するためのディストリビューテッド・アグリーメント・プロトコルの操作
CN106506647A (zh) 一种客户端具有数据备份装置的智慧社区云存储系统
CN107656980A (zh) 应用于分布式数据库系统中的方法及分布式数据库系统
US10678462B2 (en) Rebalancing data storage in a dispersed storage network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14867987

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15039342

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2016126187

Country of ref document: RU

Kind code of ref document: A

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC ( EPO FORM 1205A DATED 05/09/2016 )

122 Ep: pct application non-entry in european phase

Ref document number: 14867987

Country of ref document: EP

Kind code of ref document: A1