WO2024148824A1 - 数据处理方法、系统、装置、存储介质及电子设备 - Google Patents

数据处理方法、系统、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2024148824A1
WO2024148824A1 PCT/CN2023/114305 CN2023114305W WO2024148824A1 WO 2024148824 A1 WO2024148824 A1 WO 2024148824A1 CN 2023114305 W CN2023114305 W CN 2023114305W WO 2024148824 A1 WO2024148824 A1 WO 2024148824A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
storage node
hash
disk
data
Prior art date
Application number
PCT/CN2023/114305
Other languages
English (en)
French (fr)
Inventor
尹明俊
Original Assignee
苏州元脑智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州元脑智能科技有限公司 filed Critical 苏州元脑智能科技有限公司
Publication of WO2024148824A1 publication Critical patent/WO2024148824A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the embodiments of the present application relate to the field of computers, and more specifically, to a data processing method, system, device, non-volatile readable storage medium, and electronic device.
  • Software-defined storage refers to the integration of physical storage resources through abstraction and pooling, and the management of storage resources through intelligent software, so as to achieve the decoupling of the control plane and the data plane, and finally provide storage services to applications to meet the application's on-demand storage needs (such as capacity, performance, quality of service, service level agreement, etc.).
  • software-defined storage uses a replica mechanism to ensure high data reliability, and uses a replica replication module to ensure the consistency of replicas and data synchronization between replicas.
  • the replica indexing mechanism is implemented based on a consistent hashing algorithm.
  • the software-defined storage cluster cannot ensure that data replicas are evenly distributed in multiple computer rooms in cross-computer room deployment, making it impossible to achieve local disaster recovery and active-active.
  • the embodiments of the present application provide a data processing method, system, device, non-volatile readable storage medium and electronic device to at least solve the problem that it is difficult to achieve local data disaster recovery and dual active in the related art.
  • a data processing method including: connecting to a target storage node in a distributed storage cluster through a client, wherein the target storage node is any storage node in the storage cluster, and each storage node in the storage cluster stores a distributed hash table, which is used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring, and the disk hash ring; in the case of data writing demand, indexing the storage nodes where the copies of the data to be written are located according to the distributed hash table stored in the target storage node, and controlling the storage nodes where the copies are located to write the data Disk; when there is a demand for data reading, the storage node storing the data to be read is indexed according to the distributed hash table stored in the target storage node, and the data is read from the disk in the storage node storing the data to be read.
  • a distributed hash table is established in the following manner: multiple storage nodes in a storage cluster are divided into different storage domains, characteristic values of each storage domain are obtained and the number of virtual nodes of each storage domain is set, and a storage domain hash ring is generated according to the characteristic values of each storage domain and the number of virtual nodes, wherein the storage domain hash ring is distributed with virtual nodes of each storage domain, and each storage domain contains at least one virtual node; for each storage domain, characteristic values of each storage node are obtained and the number of virtual nodes of each storage node is set, and a storage node hash ring is generated according to the characteristic values of each storage node and the number of virtual nodes, wherein the storage node ...
  • Virtual nodes of various storage nodes are distributed, and each storage node contains at least one virtual node; for each storage node, characteristic values of various disks are obtained and the number of virtual nodes of each disk is set, and a disk hash ring is generated according to the characteristic values of various disks and the number of virtual nodes, wherein virtual nodes of various disks are distributed on the disk hash ring, and each disk contains at least one virtual node; a mapping relationship between hash rings of various storage nodes and hash rings of storage domains is established according to the belonging relationship between storage nodes and storage domains, and a mapping relationship between hash rings of various disks and hash rings of storage nodes is established according to the belonging relationship between disks and storage nodes, so as to obtain a distributed hash table.
  • generating a storage domain hash ring based on the characteristic values of each storage domain and the number of virtual nodes includes: when the number of virtual nodes in the storage domain is one, calculating a hash value for the characteristic value of the storage domain according to a preset hash algorithm to obtain a hash value of the storage domain; when the number of virtual nodes in the storage domain is multiple, calculating a hash value for the characteristic value of the storage domain according to a preset hash algorithm, and calculating a hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated hash value calculations reaches the number of virtual nodes in the storage domain, thereby obtaining multiple hash values for one storage domain; and respectively setting the virtual nodes of the storage domain on the initial hash ring according to the hash values of each storage domain to obtain a storage domain hash ring.
  • generating a storage node hash ring based on the characteristic values of each storage node and the number of virtual nodes includes: when the number of virtual nodes of the storage node is one, calculating a hash value for the characteristic value of the storage node according to a preset hash algorithm to obtain a hash value of the storage node; when the number of virtual nodes of the storage node is multiple, calculating a hash value for the characteristic value of the storage node according to a preset hash algorithm, and calculating the hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated hash value calculations reaches the number of virtual nodes of the storage node, thereby obtaining multiple hash values of a storage node; and setting the virtual nodes of the storage nodes on the initial hash ring respectively according to the hash values of the storage nodes to obtain a storage node hash ring.
  • generating a disk hash ring according to the characteristic values of each disk and the number of virtual nodes includes: when the number of virtual nodes of the disk is one, calculating a hash value for the characteristic value of the disk according to a preset hash algorithm to obtain a hash value of the disk; when the number of virtual nodes of the disk is multiple, calculating a hash value for the characteristic value of the disk according to a preset hash algorithm, and calculating a hash value again according to the preset hash algorithm based on the obtained hash value until the number of hash value calculations reaches the number of virtual nodes of the disk, thereby obtaining multiple hash values of one disk; and generating a hash ring according to the hash values of each disk in the initial hash ring. Set the virtual nodes of the disks respectively to obtain the disk hash ring.
  • the number of virtual nodes of a storage domain is determined by the importance of the storage domain, and the number of virtual nodes of each storage domain is positively correlated with the importance of the storage domain.
  • the number of virtual nodes of a storage node is determined by the storage space of the storage node, and the number of virtual nodes of each storage node is positively correlated with the storage space of the storage node.
  • the number of virtual nodes of a disk is determined by the storage space of the disk, and the number of virtual nodes of each disk is positively correlated with the storage space of the disk.
  • a storage node storing data to be read is indexed according to a distributed hash table stored in a target storage node, and data is read from a disk in the storage node storing data to be read, including: determining whether the target storage node stores data to be read, and if the target storage node stores data to be read, determining the target storage node as a destination storage node, and reading data from the disk of the destination storage node; when the target storage node does not store data to be read, determining whether there is a storage node storing data to be read in the storage domain where the target storage node is located, and if so, determining the storage node storing data to be read as the destination storage node, and reading data from the disk of the destination storage node; when there is no storage node storing data to be read in the storage domain where the target storage node is located, determining the destination storage domain that is closest to the storage domain where the target storage node is located, and
  • the method further includes: in the event that a storage domain where the destination storage node is located fails, redetermining the destination storage node from a storage domain other than the storage domain where the destination storage node is located, and reading data from a disk of the redetermined destination storage node.
  • the storage nodes where the copies of the data to be written are located are indexed according to the distributed hash table stored in the target storage node, and the storage nodes where the copies are located are controlled to write the data to the disk, including: determining a preset number of copies of the data to be written, determining a preset number of virtual nodes clockwise starting from the first virtual node of the storage domain hash ring, and writing the data to be written to the disk of the storage node of the storage domain indicated by the preset number of virtual nodes.
  • the method also includes: in the event that a storage domain in a distributed storage cluster fails, if a preset number of virtual nodes includes a virtual node of the faulty storage domain, skipping the virtual node of the faulty storage domain and continuing to determine the virtual nodes until a preset number of virtual nodes that do not include the virtual node of the faulty storage domain are obtained.
  • the method also includes: in the event that a storage node in the distributed storage cluster fails, performing a data recovery operation in the storage domain where the storage node is located; in the event that a disk in the distributed storage cluster fails, performing a data recovery operation in the storage domain where the disk is located.
  • the method also includes: in the event of a storage node failure in a distributed storage cluster, rebuilding the storage node hash ring and the disk hash ring, and establishing a mapping relationship between the reconstructed storage node hash ring and the storage domain hash ring before reconstruction, and establishing a mapping relationship between the reconstructed disk hash ring and the reconstructed storage node hash ring to obtain an updated distributed hash table.
  • the method further includes: in the case where there is a need to reduce the capacity of the disk in the distributed storage cluster, rebuilding the disk hash ring, deleting the disk hash before the reconstruction on the distributed hash table A mapping relationship between the reconstructed disk hash ring and the storage node hash ring before reconstruction is established, and a mapping relationship between the reconstructed disk hash ring and the storage node hash ring before reconstruction is established to obtain an updated distributed hash table.
  • a data processing system including: multiple computer rooms; a distributed storage cluster, wherein multiple storage nodes in the distributed storage cluster are deployed in multiple computer rooms, wherein the multiple storage nodes are divided into different storage domains, and the storage nodes in the same storage domain are deployed in the same computer room, and each storage node stores a distributed hash table, which is used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring and the disk hash ring; a client, connected to any storage node in the distributed storage cluster, and used to perform data read and write operations through the connected storage node.
  • virtual nodes of various storage domains are distributed on the storage domain hash ring, each storage domain includes at least one virtual node, and the number of virtual nodes of each storage domain is positively correlated with the importance of the storage domain.
  • virtual nodes of each storage node are distributed on the storage node hash ring, each storage node includes at least one virtual node, and the number of virtual nodes of each storage node is positively correlated with the storage space of the storage node.
  • virtual nodes of various disks are distributed on the disk hash ring, each disk includes at least one virtual node, and the number of virtual nodes of each disk is positively correlated with the storage space of the disk.
  • a data processing device including: a connection unit, configured to connect to a target storage node in a distributed storage cluster through a client, wherein the target storage node is any storage node in the storage cluster, and each storage node in the storage cluster stores a distributed hash table, which is used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring and the disk hash ring; a writing unit, configured to, when there is a demand for data writing, index the storage nodes where each copy of the data to be written is located according to the distributed hash table stored in the target storage node, and control the storage nodes where the copies are located to write the data to the disk; a reading unit, configured to, when there is a demand for data reading, index the storage node where the data to be read is stored according to the distributed hash table stored in the target storage node, and read the data from the disk in the storage node where the data to be read is stored.
  • a computer non-volatile readable storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
  • an electronic device including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • a storage domain hash ring, a node hash ring and a disk hash ring are constructed by using a hash algorithm, and a mapping relationship between the three-level hash rings is constructed, thereby obtaining a three-level data index relationship.
  • the data is indexed to a distributed storage cluster according to the three-level data index relationship. Therefore, the problem of difficulty in realizing local data disaster recovery and dual active can be solved, thereby achieving local data disaster recovery and dual active, and improving data reliability.
  • FIG1 is a hardware structure block diagram of a mobile terminal of a data processing method according to an embodiment of the present application
  • FIG2 is a flow chart of a data processing system according to an embodiment of the present application.
  • FIG3 is a schematic diagram of a distributed storage cluster according to an embodiment of the present application.
  • FIG4 is a flow chart of a data processing method according to an embodiment of the present application.
  • FIG5 is a schematic diagram 1 of performing a data write operation based on a storage domain hash ring according to an embodiment of the present application
  • FIG6 is a second schematic diagram of performing a data write operation based on a storage domain hash ring according to an embodiment of the present application
  • FIG. 7 is a flow chart of a data processing device according to an embodiment of the present application.
  • FIG. 1 is a hardware structure block diagram of a mobile terminal of a data processing method of an embodiment of the present application.
  • the mobile terminal may include one or more (only one is shown in FIG. 1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input/output device 108 for communication functions.
  • processors 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA
  • a memory 104 for storing data
  • the mobile terminal may also include a transmission device 106 and an input/output device 108 for communication functions.
  • FIG. 1 is only for illustration and does not limit the structure of the mobile terminal.
  • the mobile terminal may also include more or fewer components than those shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 .
  • the memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data processing method in the embodiment of the present application.
  • the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method.
  • the memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the transmission device 106 is used to receive or send data via a network.
  • the specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal.
  • the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
  • RF radio frequency
  • FIG. 2 is a flow chart of a data processing system according to an embodiment of the present application. As shown in FIG. 2 , the system includes:
  • multiple storage devices are set in the computer room 202 to provide storage support for data reading and writing on the client 206 side.
  • a distributed storage cluster 204 wherein multiple storage nodes in the distributed storage cluster 204 are deployed in multiple computer rooms 202, wherein multiple storage nodes are divided into different storage domains, and storage nodes in the same storage domain are deployed in the same computer room 202, and each storage node stores a distributed hash table, which is used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring, and the disk hash ring.
  • the distributed storage cluster 204 refers to aggregating the storage space in multiple storage devices into a storage pool that can provide a unified access interface and management interface to the application server.
  • the application can transparently access and utilize the disks on all storage devices through the access interface, and can give full play to the performance of the storage devices and the disk utilization.
  • the data will be stored and read from multiple storage devices according to certain rules to obtain higher concurrent access performance.
  • FIG 3 is a schematic diagram of a distributed storage cluster according to an embodiment of the present application. As shown in Figure 3, multiple storage nodes in the distributed storage cluster 204 are divided into multiple storage domains, each storage domain contains at least one storage node, and the storage nodes in the same storage domain are deployed in the same computer room 202 to achieve storage isolation between different computer rooms and realize local disaster recovery and active-active data.
  • each storage node stores a distributed hash table.
  • the distributed hash table can be used to map the data to be read from the storage domain hash ring to the storage node hash ring, and then from the storage node hash ring to the disk hash ring, so as to index to a specific location and perform data reading and writing at this location without the need to additionally build index metadata, which saves metadata storage space and reduces metadata complexity.
  • the client 206 is connected to any storage node in the distributed storage cluster 204 and is used to perform data read and write operations through the connected storage node.
  • the client 206 performs data read and write operations by connecting to the nodes of the distributed storage cluster 204. Since each storage node stores the distributed hash table of the entire distributed storage cluster 202, the client 206 can connect to any storage node and index to the location of the data to be read or the location of the data to be written through the distributed hash table.
  • virtual nodes of each storage domain are distributed on the storage domain hash ring, each storage domain includes at least one virtual node, and the number of virtual nodes of each storage domain is positively correlated with the importance of the storage domain.
  • the number of virtual nodes of the storage domain can be set to achieve more concentrated distribution of data in a certain storage domain.
  • storage domain A and storage domain B are included.
  • the number of virtual nodes of storage domain A can be set to 15 and the number of virtual nodes of storage domain B can be set to 10 in the storage domain hash ring.
  • virtual nodes of each storage node are distributed on the storage node hash ring, and each storage node virtual node includes at least one.
  • the number of virtual nodes of a storage node is positively correlated with the storage space of the storage node.
  • the number of virtual nodes is allocated according to the data capacity memory of the storage node to achieve uniform distribution of data. For example, if the data capacity memory ratio of storage node A to storage node B is 1.5 to 1, the number of virtual nodes of storage node A and the number of virtual nodes of storage node B can be set to 15 and 10 in the storage node hash ring.
  • virtual nodes of each disk are distributed on the disk hash ring, each disk contains at least one virtual node, and the number of virtual nodes of each disk is positively correlated with the storage space of the disk.
  • the number of virtual nodes is allocated according to the data capacity and memory of the disk to achieve uniform distribution of data. For example, if the data capacity and memory ratio of disk A to disk B is 1.5 to 1, the number of virtual nodes in disk A and the number of virtual nodes in disk B can be set to 15 and 10 in the disk hash ring.
  • FIG. 4 is a flow chart of the data processing method according to an embodiment of the present application. As shown in FIG. 4 , the flow chart includes the following steps:
  • Step S402 connecting to the target storage node in the distributed storage cluster through the client, wherein the target storage node is any storage node in the storage cluster, and each storage node in the storage cluster stores a distributed hash table, which is used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring and the disk hash ring.
  • a distributed hash table is a type of distributed computing system that is used to distribute a set of keys to all nodes in the distributed system, where a node refers to a storage location in a hash table.
  • the distributed hash table includes three levels of hash rings, including a hash ring of a storage domain, a hash ring of a node, and a disk hash ring, which are used to characterize the mapping relationship between the storage domain hash ring, the storage node hash ring, and the disk hash ring.
  • the hash value uniquely corresponding to the characteristic value of each storage domain is calculated, and the virtual nodes of the storage domain are arranged on the hash ring according to the hash value to obtain the storage domain hash ring; the hash value uniquely corresponding to the characteristic value of each storage node is calculated, and the virtual nodes of the storage node are arranged on the hash ring according to the hash value to obtain the storage node hash ring; the hash value uniquely corresponding to the characteristic value of each disk is calculated, and the virtual nodes of the disk are arranged on the hash ring according to the hash value to obtain the disk hash ring.
  • each storage node in this embodiment contains a distributed hash table
  • the client can access any storage node and index to the storage node in the distributed storage cluster through the distributed hash table.
  • the storage node contains a storage unit, that is, a storage engine, which is used to write data to the disk or read the content on the disk.
  • Step S404 when there is a need to write data, the storage nodes where the copies of the data to be written are located are indexed according to the distributed hash table stored in the target storage node, and the storage nodes where the copies are located are controlled to write the data to the disk.
  • the client when the client sends a data write request, the client connects to any node in the distributed storage cluster, where the node is used as the target storage node, and the data is stored in the target storage node.
  • the one-to-one mapping relationship of the three-level hash ring is used to index the locations of all copies of the data for which the write request is made, and then the write request is forwarded to the storage nodes where all copies are located.
  • the local storage unit of each node completes the write operation and writes the data to the disk.
  • Step S406 when there is a demand for data reading, the storage node storing the data to be read is indexed according to the distributed hash table stored in the target storage node, and the data is read from the disk in the storage node storing the data to be read.
  • the client when the client issues a data read request, the client connects to any node in the distributed storage cluster, where the node serves as the target storage node, and indexes the locations of all replicas closest to the client that store the data to be read based on the one-to-one mapping relationship of the three-level hash ring stored in the target storage node, and then forwards the read request to the storage node where the replica is located, and performs a data read operation from the storage unit of the storage node that stores the read request data.
  • the execution subject of the above steps may be a server, a terminal, etc., but is not limited thereto.
  • step S404 and step S406 are interchangeable, that is, step S406 may be executed first, and then step S404.
  • a distributed hash table is established in the following manner: multiple storage nodes in a storage cluster are divided into different storage domains, characteristic values of each storage domain are obtained and the number of virtual nodes of each storage domain is set, and a storage domain hash ring is generated according to the characteristic values of each storage domain and the number of virtual nodes, wherein the storage domain hash ring is distributed with virtual nodes of each storage domain, and each storage domain contains at least one virtual node; for each storage domain, characteristic values of each storage node are obtained and the number of virtual nodes of each storage node is set, and a storage node hash ring is generated according to the characteristic values of each storage node and the number of virtual nodes, wherein the storage node The virtual nodes of each storage node are distributed on the point hash ring, and each storage node contains at least one virtual node; for each storage node, the characteristic values of each disk are obtained and the number of virtual nodes of each disk is set, and
  • the storage domain contains multiple storage nodes, each storage node contains a unique characteristic value.
  • the unique hash value corresponding to the storage node is calculated according to the storage node characteristic value.
  • a virtual node is established according to the storage domain hash value, and the virtual nodes are arranged in a hash ring to obtain a storage node hash ring. It should be noted that To evenly distribute data across storage nodes in the same storage domain, different storage nodes may be provided with different numbers of virtual nodes according to storage capacity.
  • the storage node contains multiple disks, each of which contains a unique characteristic value.
  • the unique hash value corresponding to the disk is calculated based on the disk characteristic value.
  • a virtual node is established based on the storage domain hash value.
  • the virtual nodes are arranged in a hash ring to obtain a disk hash ring. It should be noted that in order to evenly distribute data on each disk in the same storage domain, different disks can be set with different numbers of virtual nodes according to the storage capacity.
  • mapping relationship between storage domain hash ring and storage node hash ring According to the subordinate relationship among storage domain, storage node and disk, the mapping relationship between storage domain hash ring and storage node hash ring, the mapping relationship between storage node hash ring and disk hash ring can be further obtained, thereby obtaining distributed hash.
  • generating a storage domain hash ring according to the characteristic values of each storage domain and the number of virtual nodes includes: when the number of virtual nodes in the storage domain is one, calculating a hash value for the characteristic value of the storage domain according to a preset hash algorithm to obtain a hash value of the storage domain; when the number of virtual nodes in the storage domain is multiple, calculating a hash value for the characteristic value of the storage domain according to a preset hash algorithm, and calculating a hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated calculations of the hash value reaches the number of virtual nodes in the storage domain, thereby obtaining multiple hash values for one storage domain; and setting the virtual nodes of the storage domain on the initial hash ring respectively according to the hash values of each storage domain to obtain a storage domain hash ring.
  • the hash value of the storage domain is determined by calculating the number of virtual nodes and characteristic values of the storage domain using a hash algorithm.
  • the number of virtual nodes of the storage domain is determined by the importance of the storage domain, and the number of virtual nodes of each storage domain is positively correlated with the importance of the storage domain.
  • the number of virtual nodes can represent the importance of the storage domain, that is, the storage domain with the largest number of virtual nodes is the primary storage domain, and the remaining storage domains are backup storage domains, that is, the number of virtual nodes is positively correlated with the primary and secondary relationship of the storage domains.
  • a distributed cluster includes storage domain A and storage domain B.
  • Storage domain B is a primary storage domain and is set with three virtual nodes.
  • Storage domain A is a backup storage domain and is set with one virtual node.
  • the number of virtual nodes in storage domain A is one.
  • the unique characteristic value of the storage domain is hashed once using a hash algorithm to obtain the hash value of storage domain A.
  • the number of virtual nodes in storage domain B is three.
  • the first hash value is calculated according to the hash algorithm and the unique storage domain characteristic value.
  • the first hash value is calculated using the hash algorithm to obtain the second hash value.
  • the second hash value is substituted into the hash algorithm to obtain the third hash value.
  • the hash value of storage domain B includes three: the first hash value, the second hash value, and the third hash value.
  • the hash algorithm refers to mapping a binary value string of any length to a binary value string of fixed length.
  • the rule of this mapping is the hash algorithm, and the binary value string obtained after mapping the original data is represented as a hash value.
  • the virtual nodes are set on the hash ring, that is, the storage domain hash ring is obtained according to the positions of the hash values mapped on the storage domain hash ring.
  • generating a storage node hash ring according to the characteristic values of each storage node and the number of virtual nodes includes: when the number of virtual nodes of the storage node is one, calculating the hash value of the characteristic value of the storage node according to a preset hash algorithm to obtain the hash value of the storage node; when the number of virtual nodes of the storage node is multiple, calculating the hash value of the characteristic value of the storage node according to the preset hash algorithm, and calculating the hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated calculations of the hash value reaches the number of virtual nodes of the storage node, and obtaining multiple hash values of a storage node; and setting the virtual nodes of the storage node on the initial hash ring according to the hash values of each storage node to obtain a storage node hash ring.
  • the storage domain includes multiple storage nodes, and the storage domain hash ring and the storage node hash ring in the three-level hash ring are subordinate to each other.
  • the storage node hash ring associated with each storage domain is determined by the unique characteristic values of each storage node in the storage domain and the number of virtual nodes of the storage node.
  • the number of virtual nodes of the storage node is determined by the storage space of the storage node, and the number of virtual nodes of each storage node is positively correlated with the storage space of the storage node.
  • the number of virtual nodes of each storage node in the storage node hash ring is obtained by normalizing the total node storage amount.
  • the storage domain includes storage node A and storage node B, and the ratio of the storage space memory size of storage node A to the storage space memory size of storage node B is 3 to 1. Then, when the number of virtual nodes of storage node A is 3, the number of virtual nodes of storage node B is 1. The larger the storage space, the more virtual nodes there are, that is, the greater the degree of importance.
  • the number of virtual nodes of storage node B is one, and the hash value of storage node B is obtained by performing a hash calculation according to the hash algorithm and the characteristic value of the storage node;
  • the number of virtual nodes of storage node A is three, and the first storage node hash value is first calculated using the hash algorithm and the characteristic value of the storage node, and the first storage node hash value is further calculated using the hash algorithm and the storage node characteristic value to obtain the second storage node hash value;
  • the third storage node hash value is calculated using the hash algorithm and the storage node characteristic value, so that the number of hash value calculations is three times, which is consistent with the number of virtual nodes, and the hash value of a storage node includes: the first storage node hash value, the second storage node hash value, and the third storage node hash value.
  • the virtual nodes are set on the hash ring, that is, the storage node hash ring is obtained according to the position of the hash value mapping on the storage node hash ring.
  • generating a disk hash ring according to the characteristic values of each disk and the number of virtual nodes includes: when the number of virtual nodes of the disk is one, calculating a hash value for the characteristic value of the disk according to a preset hash algorithm to obtain a hash value of the disk; when the number of virtual nodes of the disk is multiple, calculating a hash value for the characteristic value of the disk according to a preset hash algorithm, and calculating the hash value again according to the preset hash algorithm based on the obtained hash value until the number of hash value calculations reaches the number of virtual nodes of the disk, and obtaining multiple hash values for one disk; and setting the virtual nodes of the disk on the initial hash ring according to the hash values of each disk to obtain a disk hash ring.
  • the storage node contains multiple disks, and the storage node hash ring and the disk hash ring in the three-level hash ring are subordinate to each other.
  • the disk hash ring associated with each storage node is determined by the unique characteristic values of each disk in the storage node and the number of virtual nodes of the disk.
  • the number of virtual nodes of the disk is determined by the storage space of the disk, and the number of virtual nodes of each disk is positively correlated with the storage space of the disk.
  • the number of virtual nodes of each disk in the disk hash ring is obtained by normalizing the total disk storage.
  • the storage node includes disk A and disk B, and the ratio of the storage space memory size of disk A to the storage space memory size of disk B is 3 to 1. Then, when the number of virtual nodes of disk A is 3, the number of virtual nodes of disk B is 1. The larger the storage space of the disk, the more virtual nodes it has, that is, the greater its importance.
  • the number of virtual nodes of disk B is one, and the hash value of disk B is obtained by performing a hash calculation according to the hash algorithm and the characteristic value of the disk;
  • the number of virtual nodes of disk A is three, and the first disk hash value is first calculated using the hash algorithm and the characteristic value of the disk, and the second disk hash value is obtained by continuing to calculate the first disk hash value using the hash algorithm and the disk characteristic value;
  • the third disk hash value is calculated using the hash algorithm and the disk characteristic value, so that the number of hash value calculations is three times, which is consistent with the number of disks, and the hash value of one disk includes: the first disk hash value, the second disk hash value, and the third disk hash value.
  • the virtual nodes After obtaining the hash values of the four virtual nodes of disk A and disk B, the virtual nodes are set on the hash ring according to the calculated disk hash values. That is, the mapping relationship between data and the disk hash ring can be established according to the position of the hash value mapping on the disk hash ring.
  • the storage node storing the data to be read is indexed according to the distributed hash table stored in the target storage node, and the data is read from the disk in the storage node storing the data to be read, including: judging whether the target storage node stores the data to be read, and when the target storage node stores the data to be read, determining the target storage node as the destination storage node, and reading the data from the disk of the destination storage node; when the target storage node does not store the data to be read, judging whether there is a storage node storing the data to be read in the storage domain where the target storage node is located, and when it does, determining the storage node storing the data to be read as the destination storage node, and reading the data from the disk of the destination storage node; when there is no storage node storing the data to be read in the storage domain where the target storage node is located, determining the destination storage domain that is
  • mapping relationship of the three-level hash ring stored in the target node that is, read according to the mapping relationship between the data and the storage node and storage domain.
  • the copy of the nearest node in the topological relationship of the replicas is first determined whether the target storage node stores the data to be read. If it is determined that the data exists, the target storage node is determined as the destination storage node, and the data is read from the destination storage node. If the data is not found in the target storage node, it is necessary to determine whether the storage domain where the target storage node is located contains the data.
  • the storage node containing the required data is found and the data is read from the storage node; when the required data does not exist in the storage domain, that is, the storage domain closest to the storage domain where the target storage node is located is determined, and the data is read from the storage node containing the required data in the storage domain. Get data.
  • the method also includes: in the event that a storage domain where the destination storage node is located fails, redetermining the destination storage node from a storage domain other than the storage domain where the destination storage node is located, and reading data from the disk of the redetermined destination storage node.
  • the storage domain where the destination storage node is located fails, it is necessary to redetermine the storage node from a copy of the nearest node in the topological relationship among all copies of the data outside the storage domain where the storage node is located, that is, from the nearest storage domain outside the storage domain where the destination storage node is located. For example, if a data copy exists in a storage node in a storage domain adjacent to the storage domain where the storage node is located, the storage node in the adjacent storage domain is determined as the destination storage node, and data is read from the disk of the newly determined destination storage node.
  • the storage nodes where the copies of the data to be written are located are indexed according to the distributed hash table stored in the target storage node, and the storage nodes where the copies are located are controlled to write the data to the disk, including: determining a preset number of copies of the data to be written, determining a preset number of virtual nodes clockwise from the first virtual node of the storage domain hash ring, and writing the data to be written to the disk of the storage node of the storage domain indicated by the preset number of virtual nodes.
  • the storage domain can isolate multiple nodes in a single cluster and implement local data disaster recovery with a multi-copy mechanism, that is, multiple copies are stored in a certain node or cluster, and the storage domains are copies of each other, that is, the copies are divided according to the storage domain.
  • the storage domain includes two, namely storage domain A and storage domain B, and the data stored in each node in storage domain A has a corresponding copy in storage domain B.
  • the number of copies of the written data is determined, starting from the initial position of the storage domain hash ring and moving clockwise, the first virtual node "encountered” is determined as the first copy of the written data, and the data to be written is written to multiple copies of the hash ring, that is, the data is written to the disk of the storage node in the storage domain hash ring.
  • FIG5 is a schematic diagram of performing a data write operation based on a storage domain hash ring according to an embodiment of the present application.
  • the data to be written is the number a, in clockwise order
  • the first storage domain (storage domain A) found clockwise from the initial position is the first copy
  • the next storage domain (storage domain B) found clockwise after the first copy is the second copy position.
  • the data to be written is the number b, in clockwise order
  • the first storage domain (storage domain C) found clockwise from the initial position is the first copy
  • the next storage domain (storage domain A) where the non-first copy is located found clockwise after the first copy is the second copy position.
  • FIG6 is a second schematic diagram of performing a data write operation based on a storage domain hash ring according to an embodiment of the present application.
  • the data to be written is the number a, in clockwise order
  • the first storage domain (storage domain A) found clockwise from the initial position is the first copy
  • the next storage domain (storage domain B) found clockwise after the first copy is the second copy position
  • the next storage domain (storage domain C) found clockwise after the second copy is the third copy position.
  • the first storage domain (storage domain C) found clockwise from the initial position is the first copy
  • the next storage domain (storage domain A) found clockwise after the first copy is the second copy position
  • the next storage domain (storage domain C) found clockwise after the second copy is the third copy position.
  • the storage domain where the first copy is not located is the location of the third copy.
  • the method also includes: in the event that there is a fault in a storage domain in a distributed storage cluster, if a preset number of virtual nodes includes a virtual node of the faulty storage domain, skipping the virtual node of the faulty storage domain and continuing to determine the virtual nodes until a preset number of virtual nodes that do not include the virtual node of the faulty storage domain are obtained.
  • the fault domain can be represented as a division of a single cluster across storage domains.
  • the storage domain in the distributed storage cluster includes the fault domain
  • all virtual nodes containing the read or write demand are retrieved to see whether they exist in the fault domain. If a virtual node exists in the fault domain, the node is skipped and a new virtual node for writing or reading data is re-determined until all virtual nodes containing the read or write demand no longer exist in the fault domain.
  • the method also includes: in the event that a storage node in a distributed storage cluster fails, performing a data recovery operation in the storage domain where the storage node is located; in the event that a disk in a distributed storage cluster fails, performing a data recovery operation in the storage domain where the disk is located.
  • the method also includes: in the event of a storage node failure in the distributed storage cluster, rebuilding the storage node hash ring and the disk hash ring, and establishing a mapping relationship between the reconstructed storage node hash ring and the storage domain hash ring before reconstruction, establishing a mapping relationship between the reconstructed disk hash ring and the reconstructed storage node hash ring, and obtaining an updated distributed hash table.
  • the distributed storage cluster needs to rebuild a failed storage node, it needs to rebuild the storage node hash ring and the disk hash ring.
  • the storage node hash ring is rebuilt according to the characteristic value of the failed storage node, and the disk hash ring is rebuilt according to the characteristic values of each disk of the failed storage node.
  • the data in the reconstructed storage node hash ring is mapped to the storage domain hash ring before reconstruction to form a mapping relationship.
  • the data in the reconstructed disk hash ring is mapped to the reconstructed storage node hash ring to form a one-to-one mapping relationship, and an updated distributed hash table can be obtained.
  • the method also includes: when there is a need to shrink the disk in the distributed storage cluster, rebuilding the disk hash ring, deleting the mapping relationship between the disk hash ring before reconstruction and the storage node hash ring before reconstruction on the distributed hash table, establishing a mapping relationship between the reconstructed disk hash ring and the storage node hash ring before reconstruction, and obtaining an updated distributed hash table.
  • the disk in the node of the distributed storage cluster cannot store more data
  • the disk needs to be expanded.
  • the free space in the disk is too large, the disk needs to be reduced.
  • the distributed hash table the mapping relationship between the storage node and the corresponding hash ring of the disk before reconstruction is deleted, a new hash ring mapping relationship is established, and the distributed hash table is updated according to the new hash ring mapping relationship.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.
  • the technical solution of the present application, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a non-volatile readable storage medium (such as ROM/RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
  • a non-volatile readable storage medium such as ROM/RAM, magnetic disk, optical disk
  • a terminal device which can be a mobile phone, computer, server, or network device, etc.
  • a data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated.
  • the term "module” can implement a combination of software and/or hardware of a predetermined function.
  • the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
  • FIG. 7 is a flow chart of a data processing device according to an embodiment of the present application. As shown in FIG. 7 , the device includes a connecting unit 70, a writing unit 71 and a reading unit 72:
  • the connection unit 70 is configured to connect to a target storage node in a distributed storage cluster through a client, wherein the target storage node is any storage node in the storage cluster, and each storage node in the storage cluster stores a distributed hash table, which is used to represent the mapping relationship between the storage domain hash ring, the storage node hash ring, and the disk hash ring.
  • the writing unit 71 is configured to, when there is a demand for data writing, index the storage nodes where the copies of the data to be written are located according to the distributed hash table stored in the target storage node, and control the storage nodes where the copies are located to write the data to the disk.
  • the reading unit 72 is configured to, when there is a demand for data reading, index the storage node storing the data to be read according to the distributed hash table stored in the target storage node, and read the data from the disk in the storage node storing the data to be read.
  • the connection unit 70 includes: a distributed hash table is established in the following manner: a division module, configured to divide multiple storage nodes in the storage cluster into different storage domains, obtain characteristic values of each storage domain and set the number of virtual nodes of each storage domain, and generate a storage domain hash ring according to the characteristic values of each storage domain and the number of virtual nodes, wherein the storage domain hash ring is distributed with virtual nodes of each storage domain, and each storage domain contains at least one virtual node; a first acquisition module, configured to obtain the characteristic values of each storage node and set the virtual node of each storage node for each storage domain.
  • a division module configured to divide multiple storage nodes in the storage cluster into different storage domains, obtain characteristic values of each storage domain and set the number of virtual nodes of each storage domain, and generate a storage domain hash ring according to the characteristic values of each storage domain and the number of virtual nodes, wherein the storage domain hash ring is distributed with virtual nodes of each storage domain, and each storage domain contains at least one virtual node
  • a storage node hash ring is generated according to the characteristic values of each storage node and the number of virtual nodes, wherein the storage node hash ring is distributed with virtual nodes of each storage node, and each storage node includes at least one virtual node;
  • a second acquisition module is configured to, for each storage node, obtain the characteristic values of each disk and set the number of virtual nodes of each disk, and generate a disk hash ring according to the characteristic values of each disk and the number of virtual nodes, wherein the disk hash ring is distributed with virtual nodes of each disk, and each disk includes at least one virtual node;
  • an establishment module is configured to establish each storage node according to the ownership relationship between the storage node and the storage domain.
  • mapping relationship between the storage node hash ring and the storage domain hash ring is established, and the mapping relationship between each disk hash ring and the storage node hash ring is established according to the ownership relationship between the disk and the storage node to obtain a distributed hash table.
  • the connection unit 70 includes: a first calculation module, configured to calculate a hash value for the characteristic value of the storage domain according to a preset hash algorithm when the number of virtual nodes in the storage domain is one, to obtain a hash value of the storage domain; a second calculation module, configured to calculate a hash value for the characteristic value of the storage domain according to a preset hash algorithm when the number of virtual nodes in the storage domain is multiple, and calculate the hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated calculations of the hash value reaches the number of virtual nodes in the storage domain, to obtain multiple hash values of a storage domain; a first setting module, configured to respectively set the virtual nodes of the storage domain on the initial hash ring according to the hash values of each storage domain, to obtain a storage domain hash ring.
  • the connection unit 70 includes: a third calculation module, configured to calculate a hash value for the characteristic value of the storage node according to a preset hash algorithm when the number of virtual nodes of the storage node is one, to obtain the hash value of the storage node; a fourth calculation module, configured to calculate a hash value for the characteristic value of the storage node according to a preset hash algorithm when the number of virtual nodes of the storage node is multiple, and calculate the hash value again according to the preset hash algorithm based on the obtained hash value until the number of repeated calculations of the hash value reaches the number of virtual nodes of the storage node, to obtain multiple hash values of a storage node; a second setting module, configured to set the virtual nodes of the storage nodes on the initial hash ring respectively according to the hash values of each storage node, to obtain a storage node hash ring.
  • a third calculation module configured to calculate a hash value for the characteristic value of the storage node according to a preset has
  • the connection unit 70 includes: a fifth calculation module, configured to calculate a hash value for the characteristic value of the disk according to a preset hash algorithm when the number of virtual nodes of the disk is one, so as to obtain a hash value of the disk; a sixth calculation module, configured to calculate a hash value for the characteristic value of the disk according to a preset hash algorithm when the number of virtual nodes of the disk is multiple, and calculate the hash value again according to the preset hash algorithm based on the obtained hash value until the number of hash value calculations reaches the number of virtual nodes of the disk, so as to obtain multiple hash values of one disk; a third setting module, configured to set the virtual nodes of the disk on the initial hash ring respectively according to the hash values of each disk, so as to obtain a disk hash ring.
  • a fifth calculation module configured to calculate a hash value for the characteristic value of the disk according to a preset hash algorithm when the number of virtual nodes of the disk is one, so as to obtain
  • connection unit 70 includes: a first determination module, configured such that the number of virtual nodes of the storage domain is determined by the importance of the storage domain, and the number of virtual nodes of each storage domain is positively correlated with the importance of the storage domain.
  • the write unit 71 includes: a second determination module, the number of virtual nodes configured as storage nodes is determined by the storage space of the storage nodes, and the number of virtual nodes of each storage node is positively correlated with the storage space of the storage node.
  • the write unit 71 includes: a third determination module, which is configured such that the number of virtual nodes of the disk is determined by the storage space of the disk, and the number of virtual nodes of each disk is positively correlated with the storage space of the disk.
  • the reading unit 72 includes: a first judgment module configured to judge whether the target storage node stores the data to be read, and When the target storage node stores data to be read, the target storage node is determined as the destination storage node, and the data is read from the disk of the destination storage node; the second judgment module is configured to, when the target storage node does not store the data to be read, determine whether there is a storage node storing the data to be read in the storage domain where the target storage node is located, and if so, determine the storage node storing the data to be read as the destination storage node, and read the data from the disk of the destination storage node; the fourth determination module is configured to, when the storage domain where the target storage node is located does not store the data to be read, determine the destination storage domain that is closest to the storage domain where the target storage node is located, determine the storage node storing the data to be read in the destination storage domain as the destination storage node, and read the
  • the reading unit 72 includes: a fifth determination module, configured to redetermine the destination storage node from a storage domain outside the storage domain where the destination storage node is located when a failure occurs in the storage domain where the destination storage node is located, and read data from the disk of the redetermined destination storage node.
  • a fifth determination module configured to redetermine the destination storage node from a storage domain outside the storage domain where the destination storage node is located when a failure occurs in the storage domain where the destination storage node is located, and read data from the disk of the redetermined destination storage node.
  • the write unit 71 includes: a sixth determination module, configured to determine a preset number of copies of the data to be written, determine a preset number of virtual nodes clockwise starting from the first virtual node of the storage domain hash ring, and write the data to be written to the disk of the storage node of the storage domain indicated by the preset number of virtual nodes.
  • the write unit 71 includes: a skipping module, which is configured to, when a storage domain in a distributed storage cluster fails, skip the virtual nodes of the faulty storage domain and continue to determine the virtual nodes until a preset number of virtual nodes that do not include the virtual nodes of the faulty storage domain are obtained.
  • a skipping module which is configured to, when a storage domain in a distributed storage cluster fails, skip the virtual nodes of the faulty storage domain and continue to determine the virtual nodes until a preset number of virtual nodes that do not include the virtual nodes of the faulty storage domain are obtained.
  • the write unit 71 includes: an execution module, configured to perform data recovery operations in the storage domain where the storage node is located when a storage node in the distributed storage cluster fails; and to perform data recovery operations in the storage domain where the disk is located when a disk in the distributed storage cluster fails.
  • the write unit 71 includes: a first reconstruction module, configured to reconstruct the storage node hash ring and the disk hash ring when a storage node failure occurs in the distributed storage cluster, and to establish a mapping relationship between the reconstructed storage node hash ring and the storage domain hash ring before reconstruction, and to establish a mapping relationship between the reconstructed disk hash ring and the reconstructed storage node hash ring to obtain an updated distributed hash table.
  • a first reconstruction module configured to reconstruct the storage node hash ring and the disk hash ring when a storage node failure occurs in the distributed storage cluster, and to establish a mapping relationship between the reconstructed storage node hash ring and the storage domain hash ring before reconstruction, and to establish a mapping relationship between the reconstructed disk hash ring and the reconstructed storage node hash ring to obtain an updated distributed hash table.
  • the device includes: a second reconstruction module, configured to reconstruct the disk hash ring when there is a need to shrink the disk in the distributed storage cluster, delete the mapping relationship between the disk hash ring before reconstruction and the storage node hash ring before reconstruction on the distributed hash table, establish a mapping relationship between the reconstructed disk hash ring and the storage node hash ring before reconstruction, and obtain an updated distributed hash table.
  • a second reconstruction module configured to reconstruct the disk hash ring when there is a need to shrink the disk in the distributed storage cluster, delete the mapping relationship between the disk hash ring before reconstruction and the storage node hash ring before reconstruction on the distributed hash table, establish a mapping relationship between the reconstructed disk hash ring and the storage node hash ring before reconstruction, and obtain an updated distributed hash table.
  • the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
  • An embodiment of the present application further provides a computer non-volatile readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
  • the above-mentioned computer non-volatile readable storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
  • An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
  • modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation.
  • the present application is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Hardware Redundancy (AREA)

Abstract

一种数据处理方法、系统、装置、非易失性可读存储介质及电子设备，其中，方法包括：通过客户端（206）连接分布式存储集群（204）中的目标存储节点（S402）；在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘（S404）；在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据（S406）。解决了分布式存储系统难以实现数据本地容灾及双活的问题，进而达到了实现数据本地容灾及双活，提升数据可靠性的效果。

Description

数据处理方法、系统、装置、存储介质及电子设备
相关申请的交叉引用
本申请要求于2023年01月09日提交中国专利局，申请号为202310029731.8，申请名称为“数据处理方法、系统、装置、存储介质及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机领域，具体而言，涉及一种数据处理方法、系统、装置、非易失性可读存储介质及电子设备。
背景技术
随着云计算等技术产业的发展，企业的数字化转型成为战略核心，而多数企业在数字化转型中面临着海量数据的存放、管理、优化和利用等挑战，当前存储中心的响应速度难以满足企业及时性、灵活性的存储需求。在传统存储中，存储过于依赖设备，需根据设备性能响应需求，而软件定义存储可按需求进行动态配置实现存储与应用更紧密的联系，提供更高的敏捷性，以策略管理驱动的高级功能匹配企业业务需求，用户管理数据请求直接传达至自动化软件，软件直接对应请求分配存储资源，无需人工干预。
软件定义存储(SDS，Software-defined Storage)指将存储物理资源通过抽象、池化整合，并通过智能软件实现存储资源的管理，实现控制平面和数据平面的解耦，最终以存储服务的形式提供给应用，满足应用按需(如容量、性能、服务质量、服务等级协议等)使用存储的需求。通常软件定义存储利用副本机制保证数据的高可靠，通过副本复制模块来保证副本的一致性和副本之间的数据同步，副本的索引机制依据一致性哈希算法实现，但是当业务有本地容灾或者本地双活需求时，在跨机房部署中软件定义存储集群无法保障数据副本均匀分布在多个机房间，使得不能良好实现本地容灾及双活。
针对相关技术中难以实现数据本地容灾及双活的问题，目前尚未提出有效的解决方案。
发明内容
本申请实施例提供了一种数据处理方法、系统、装置、非易失性可读存储介质及电子设备，以至少解决相关技术中难以实现数据本地容灾及双活的问题。
根据本申请的一个实施例，提供了一种数据处理方法，包括：通过客户端连接分布式存储集群中的目标存储节点，其中，目标存储节点是存储集群中的任意存储节点，存储集群中的每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入 磁盘；在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据。
在一个示例性实施例中，分布式哈希表通过以下方式建立：将存储集群中的多个存储节点划分至不同的存储域，获取各个存储域的特征值并设置每个存储域的虚拟节点个数，根据各个存储域的特征值以及虚拟节点个数生成存储域哈希环，其中，存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个；对于每个存储域，获取各个存储节点的特征值并设置每个存储节点的虚拟节点个数，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环，其中，存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个；对于每个存储节点，获取各个磁盘的特征值并设置每个磁盘的虚拟节点个数，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环，其中，磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个；根据存储节点与存储域之间的归属关系建立各个存储节点哈希环与存储域哈希环之间的映射关系，根据磁盘与存储节点之间的归属关系建立各个磁盘哈希环与存储节点哈希环之间的映射关系，得到分布式哈希表。
在一个示例性实施例中，根据各个存储域的特征值以及虚拟节点个数生成存储域哈希环包括：在存储域的虚拟节点个数为一个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，得到存储域的哈希值；在存储域的虚拟节点个数为多个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储域的虚拟节点个数，得到一个存储域的多个哈希值；根据各个存储域的哈希值在初始哈希环上分别设置存储域的虚拟节点，得到存储域哈希环。
在一个示例性实施例中，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环包括：在存储节点的虚拟节点个数为一个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，得到存储节点的哈希值；在存储节点的虚拟节点个数为多个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储节点的虚拟节点个数，得到一个存储节点的多个哈希值；根据各个存储节点的哈希值在初始哈希环上分别设置存储节点的虚拟节点，得到存储节点哈希环。
在一个示例性实施例中，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环包括：在磁盘的虚拟节点个数为一个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，得到磁盘的哈希值；在磁盘的虚拟节点个数为多个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至计算哈希值的次数达到磁盘的虚拟节点个数，得到一个磁盘的多个哈希值；根据各个磁盘的哈希值在初始哈希环 上分别设置磁盘的虚拟节点，得到磁盘哈希环。
在一个示例性实施例中，存储域的虚拟节点个数由存储域的重要程度确定，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
在一个示例性实施例中，存储节点的虚拟节点个数由存储节点的存储空间确定，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
在一个示例性实施例中，磁盘的虚拟节点个数由磁盘的存储空间确定，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
在一个示例性实施例中，在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据包括：判断目标存储节点是否存储有待读取数据，并在目标存储节点存储有待读取数据的情况下，将目标存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；在目标存储节点未存储待读取数据的情况下，判断目标存储节点所在存储域是否存在存储有待读取数据的存储节点，并在存在的情况下，将存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；在目标存储节点所在存储域不存在存储有待读取数据的存储节点的情况下，确定与目标存储节点所在存储域距离最近的目的存储域，并目的存储域中存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据。
在一个示例性实施例中，该方法还包括：在目的存储节点所在存储域出现故障的情况下，从目的存储节点所在存储域以外的存储域重新确定目的存储节点，并从重新确定的目的存储节点的磁盘读取数据。
在一个示例性实施例中，在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘包括：确定待写入数据的副本的预设数量，从存储域哈希环的第一个虚拟节点开始顺时针确定预设数量的虚拟节点，并将待写入数据写入预设数量的虚拟节点指示的存储域的存储节点的磁盘中。
在一个示例性实施例中，该方法还包括：在分布式存储集群中的存储域存在故障的情况下，若预设数量的虚拟节点中包含故障存储域的虚拟节点，则跳过故障存储域的虚拟节点继续确定虚拟节点，直至得到不包含故障存储域的虚拟节点的预设数量的虚拟节点。
在一个示例性实施例中，该方法还包括：在分布式存储集群中的存储节点存在故障的情况下，在存储节点所在存储域执行数据恢复操作；在分布式存储集群中的磁盘存在故障的情况下，在磁盘所在存储域执行数据恢复操作。
在一个示例性实施例中，该方法还包括：在分布式存储集群中存在存储节点故障的情况下，重建存储节点哈希环和磁盘哈希环，并建立重建后的存储节点哈希环与重建前的存储域哈希环之间的映射关系，建立重建后的磁盘哈希环和重建后的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
在一个示例性实施例中，该方法还包括：在分布式存储集群中存在磁盘缩容需求的情况下，重建磁盘哈希环，在分布式哈希表上删除重建前的磁盘哈希 环和重建前的存储节点哈希环之间的映射关系，建立重建后的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
根据本申请的一个实施例，提供了一种数据处理系统，包括：多个机房；分布式存储集群，分布式存储集群中的多个存储节点部署在多个机房内，其中，多个存储节点划分至不同的存储域，同一存储域中的存储节点设部署在同一机房内，每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；客户端，与分布式存储集群中的任意一个存储节点连接，用于通过连接的存储节点执行数据读写操作。
在一个示例性实施例中，存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
在一个示例性实施例中，存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
在一个示例性实施例中，磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
根据本申请的另一个实施例，提供了一种数据处理装置，包括：连接单元，被配置为通过客户端连接分布式存储集群中的目标存储节点，其中，目标存储节点是存储集群中的任意存储节点，存储集群中的每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；写入单元，被配置为在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘；读取单元，被配置为在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据。
根据本申请的又一个实施例，还提供了一种计算机非易失性可读存储介质，计算机非易失性可读存储介质中存储有计算机程序，其中，计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据本申请的又一个实施例，还提供了一种电子设备，包括存储器和处理器，存储器中存储有计算机程序，处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
通过本申请，由于利用哈希算法构建了存储域哈希环、节点哈希环与磁盘哈希环，并构建了三级哈希环之间的映射关系，得到了三级数据索引关系，在数据读取和写入时根据三级数据索引关系索引至分布式存储集群中，因此，可以解决难以实现数据本地容灾及双活的问题，达到实现数据本地容灾及双活，提升数据可靠性的效果。
附图说明
图1是本申请实施例的一种数据处理方法的移动终端的硬件结构框图；
图2是根据本申请实施例的数据处理系统的流程图；
图3是根据本申请实施例的分布式存储集群的示意图；
图4是根据本申请实施例的数据处理方法的流程图；
图5是根据本申请实施例的基于存储域哈希环执行数据写入操作的示意图一；
图6是根据本申请实施例的基于存储域哈希环执行数据写入操作的示意图二；
图7是根据本申请实施例的数据处理装置的流程图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本申请的实施例。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语″第一″、″第二″等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
本申请实施例中所提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例，图1是本申请实施例的一种数据处理方法的移动终端的硬件结构框图。如图1所示，移动终端可以包括一个或多个(图1中仅示出一个)处理器102(处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)和用于存储数据的存储器104，其中，上述移动终端还可以包括用于通信功能的传输设备106以及输入输出设备108。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述移动终端的结构造成限定。例如，移动终端还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。
存储器104可用于存储计算机程序，例如，应用软件的软件程序以及模块，如本申请实施例中的数据处理方法对应的计算机程序，处理器102通过运行存储在存储器104内的计算机程序，从而执行各种功能应用以及数据处理，即实现上述的方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可进一步包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至移动终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输设备106用于经由一个网络接收或者发送数据。上述的网络具体实例可包括移动终端的通信供应商提供的无线网络。在一个实例中，传输设备106包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输设备106可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
图2是根据本申请实施例的数据处理系统的流程图，如图2所示，系统包括：
多个机房202。
可选的，机房202中设置多个存储设备，为客户端206侧的数据读写提供存储支持。
分布式存储集群204，分布式存储集群204中的多个存储节点部署在多个机房202内，其中，多个存储节点划分至不同的存储域，同一存储域中的存储节点设部署在同一机房202内，每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系。
可选的，分布式存储集群204是指将多台存储设备中的存储空间聚合成一个能够给应用服务器提供统一访问接口和管理界面的存储池，应用可以通过该访问接口透明地访问和利用所有存储设备上的磁盘，可以充分发挥存储设备的性能和磁盘利用率，数据将会按照一定的规则从多台存储设备上存储和读取，以获得更高的并发访问性能。
图3是根据本申请实施例的分布式存储集群的示意图，如图3所示，将布式存储集群204中的多个存储节点划分为多个存储域，每个存储域中包含至少一个存储节点，同一存储域的存储节点部署在同一机房202内，实现不同机房之间的存储隔离，实现数据本地容灾及双活。
其中，每个存储节点均存储有分布式哈希表，在数据读写时，可以通过分布式哈希表将待读取的数据从存储域哈希环映射至存储节点哈希环，再从存储节点哈希环映射至磁盘哈希环之间，从而索引至具体的位置，在该位置执行数据读写，无需额外构建索引元数据，节省元数据存储空间的同时，降低元数据复杂度。
客户端206，与分布式存储集群204中的任意一个存储节点连接，用于通过连接的存储节点执行数据读写操作。
可选的，客户端206通过与分布式存储集群204的节点连接进行数据的读写操作，由于每个存储节点均存储有整个分布式存储集群202的分布式哈希表，客户端206连接任意一个存储节点，即可通过分布式哈希表索引至待读取数据的位置、或待写入数据的位置。
可选地，在本申请实施例提供的数据处理系统中，存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
可选的，存储域的虚拟节点的个数可以设置，以实现数据更集中分布在某个存储域。例如，在分布式集群202中包含存储域A与存储域B，为了实现数据更集中分布在存储域A，则可以在存储域哈希环设置存储域A的虚拟节点个数为15，存储域B的虚拟节点个数为10。
可选地，在本申请实施例提供的数据处理系统中，存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个，各个存 储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
可选的，按照存储节点的数据容量内存分配虚拟节点的个数，以实现数据的均匀分布。例如，存储节点A与存储节点B的数据容量内存比例为1.5比1，则可以在存储节点哈希环设置存储节点A的虚拟节点个数为15，存储节点B的虚拟节点个数为10。
可选地，在本申请实施例提供的数据处理系统中，磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
可选的，按照磁盘的数据容量内存分配虚拟节点的个数，以实现数据的均匀分布。例如，磁盘A与磁盘B的数据容量内存比例为1.5比1，则可以在磁盘哈希环设置磁盘A中的虚拟节点个数为15，磁盘B中的虚拟节点个数为10。
在本实施例中提供了一种运行于移动终端的数据处理方法，图4是根据本申请实施例的数据处理方法的流程图，如图4所示，该流程包括如下步骤：
步骤S402，通过客户端连接分布式存储集群中的目标存储节点，其中，目标存储节点是存储集群中的任意存储节点，存储集群中的每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系。
需要说明的是，分布式哈希表，也即DHT(distributed hash table)，是分布式计算系统中的一类，用来将一个键的集合分散到所有在分布式系统中的节点，其中，节点是指哈希表中的存储位置。本实施例中分布式哈希表包含三级哈希环，包括存储域的哈希环、节点的哈希环以及磁盘哈希环三类，用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系。
其中，计算各个存储域的特征值唯一对应的哈希值，并根据哈希值将存储域的虚拟节点排布在哈希环上，得到存储域哈希环；计算各个存储节点的特征值唯一对应的哈希值，并根据哈希值将存储节点的虚拟节点排布在哈希环上，得到存储节点哈希环；计算各个磁盘的特征值唯一对应的哈希值，并根据哈希值将磁盘的虚拟节点排布在哈希环上，得到磁盘哈希环。
可选的，在对客户端信息进行响应时，需要通过客户端连接分布式存储集群中的存储节点，其中，分布式存储集群的作用为应用可以通过该访问接口透明地访问和利用所有存储设备上的磁盘，可以充分发挥存储设备的性能和磁盘利用率，由于本实施例中每个存储节点中包含分布式哈希表，客户端接入任意一个存储节点，即可通过分布式哈希表索引至分布式存储集群中的存储节点，存储节点中含有存储单元，也即存储引擎，用于将数据写入磁盘或读取磁盘中的内容。
步骤S404，在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘。
可选的，当客户端侧发出数据写入申请时，客户端连接分布式存储集群中的任意一个节点，其中，该节点作为目标存储节点，依据目标存储节点中存储 的三级哈希环的一一映射关系，索引到存在写入申请的数据所有副本的位置，然后将写请求转发给所有副本所在的存储节点中，由各个节点的本地存储单元完成写入操作，将数据写入磁盘。
步骤S406，在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据。
可选的，当客户端侧发出数据读取申请时，客户端连接分布式存储集群中的任意一个节点，其中，该节点作为目标存储节点，依据目标存储节点中存储的三级哈希环的一一映射关系，索引到存储有待读取数据的距离客户端最近的所有副本的位置，然后将读请求转发给该副本所在的存储节点中，从存储有读取申请数据的存储节点的存储单元进行读取数据操作。
通过上述步骤，解决了难以实现数据本地容灾及双活的问题，达到了实现数据本地容灾及双活，提升数据可靠性的效果。
其中，上述步骤的执行主体可以为服务器、终端等，但不限于此。
步骤S404和步骤S406的执行顺序是可以互换的，即可以先执行步骤S406，然后再执行S404。
可选地，在本申请实施例提供的数据处理方法中，分布式哈希表通过以下方式建立：将存储集群中的多个存储节点划分至不同的存储域，获取各个存储域的特征值并设置每个存储域的虚拟节点个数，根据各个存储域的特征值以及虚拟节点个数生成存储域哈希环，其中，存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个；对于每个存储域，获取各个存储节点的特征值并设置每个存储节点的虚拟节点个数，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环，其中，存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个；对于每个存储节点，获取各个磁盘的特征值并设置每个磁盘的虚拟节点个数，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环，其中，磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个；根据存储节点与存储域之间的归属关系建立各个存储节点哈希环与存储域哈希环之间的映射关系，根据磁盘与存储节点之间的归属关系建立各个磁盘哈希环与存储节点哈希环之间的映射关系，得到分布式哈希表。
可选的，分布式存储集群包含多个存储域，每个存储域包含唯一的特征值，例如，UUID值(通用唯一识别码，Universally Unique Identifier)，根据存储域特征值计算得到存储域唯一对应的哈希值，根据存储域哈希值建立虚拟节点，将虚拟节点排布在哈希环，即可得到存储域哈希环，需要说明的是，为使得数据更集中分布在某个存储域，不同存储域可以设置有不同数量的虚拟节点。
存储域中包含多个存储节点，每个存储节点包含唯一的特征值，根据存储节点特征值计算得到存储节点唯一对应的哈希值，根据存储域哈希值建立虚拟节点，将虚拟节点排布在哈希环，即可得到存储节点哈希环，需要说明的是， 为使得数据均匀分布在同一存储域的各个存储节点，不同存储节点可以根据存储容量设置有不同数量的虚拟节点。
存储节点中包含多个磁盘，每个磁盘包含唯一的特征值，根据磁盘特征值计算得到磁盘唯一对应的哈希值，根据存储域哈希值建立虚拟节点，将虚拟节点排布在哈希环，即可得到磁盘哈希环，需要说明的是，为使得数据均匀分布在同一存储域的各个磁盘，不同磁盘可以根据存储容量设置有不同数量的虚拟节点。
根据存储域、存储节点以及磁盘的从属关系可以进一步得到存储域哈希环和存储节点哈希环的映射关系，存储节点哈希环和磁盘哈希环的映射关系，从而得到分布式哈希。
可选地，在本申请实施例提供的数据处理方法中，根据各个存储域的特征值以及虚拟节点个数生成存储域哈希环包括：在存储域的虚拟节点个数为一个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，得到存储域的哈希值；在存储域的虚拟节点个数为多个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储域的虚拟节点个数，得到一个存储域的多个哈希值；根据各个存储域的哈希值在初始哈希环上分别设置存储域的虚拟节点，得到存储域哈希环。
可选的，存储域哈希值的确定由存储域的虚拟节点的个数以及特征值利用哈希算法计算得到。
可选地，在本申请实施例提供的数据处理方法中，存储域的虚拟节点个数由存储域的重要程度确定，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
可选的，虚拟节点的数量可以表示存储域的重要性程度，也即当虚拟节点个数最多的存储域即为主存储域，其余存储域为备存储域，也即虚拟节点个数与存储域的主次关系呈现正相关。
例如，分布式集群包含存储域A和存储域B，存储域B为主存储域，设置3个虚拟节点，存储域A为备存储域，设置一个虚拟节点。
存储域A的虚拟节点个数为一个，对存储域的唯一特征值通过哈希算法计算进行一次哈希计算，得到存储域A的哈希值。存储域B的虚拟节点个数为三个，根据哈希算法以及唯一的存储域特征值计算得到第一哈希值，将第一哈希值再利用哈希算法计算得到第二哈希值，最后利用哈希算法将第二哈希值进行代入计算得到第三哈希值，则存储域B的哈希值包括三个：第一哈希值、第二哈希值以及第三哈希值。其中，哈希算法是指将任意长度的二进制值串映射为固定长度的二进制值串，这个映射的规则就是哈希算法，而通过原始数据映射之后得到的二进制值串表示为哈希值。
进一步的，在得到存储域A和存储域B的四个虚拟节点的哈希值后，在哈希环上对虚拟节点进行设置，也即根据哈希值映射在存储节域哈希环上的位置得到存储域哈希环。
可选地，在本申请实施例提供的数据处理方法中，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环包括：在存储节点的虚拟节点个数为一个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，得到存储节点的哈希值；在存储节点的虚拟节点个数为多个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储节点的虚拟节点个数，得到一个存储节点的多个哈希值；根据各个存储节点的哈希值在初始哈希环上分别设置存储节点的虚拟节点，得到存储节点哈希环。
可选的，存储域包含多个存储节点，三级哈希环中存储域哈希环与存储节点哈希环存在从属关系，每个存储域关联的存储节点哈希环由该存储域中的各个存储节点的唯一特征值以及存储节点的虚拟节点个数确定。
可选地，在本申请实施例提供的数据处理方法中，存储节点的虚拟节点个数由存储节点的存储空间确定，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
可选的，存储节点哈希环中各个存储节点的虚拟节点个数由节点存储总量归一化处理后得出，例如，存储域中包含存储节点A和存储节点B，存储节点A的存储空间内存大小与存储节点B的存储空间内存大小比例为3比1，则当存储节点A的虚拟节点个数为3时，存储节点B的虚拟节点的个数1个，存储空间越大，虚拟节点的个数越多，也即重要性程度越大。
例如，存储节点B的虚拟节点个数为一个，存储节点B的哈希值按照哈希算法以及存储节点的特征值进行一次哈希计算得到；存储节点A的虚拟节点个数为三个，首先利用哈希算法以及该存储节点的特征值计算得到第一存储节点哈希值，将第一存储节点哈希值继续利用哈希算法以及存储节点特征值计算得到第二存储节点哈希值；基于第二存储节点哈希值，并利用哈希算法以及存储节点特征值计算得到第三存储节点哈希值，使得计算哈希值的次数为三次，与虚拟节点的个数保持一致，得到一个存储节点的哈希值包括：第一存储节点哈希值、第二存储节点哈希值以及第三存储节点哈希值。进一步的，在得到存储节点A和存储节点B的四个虚拟节点的哈希值后，在哈希环上对虚拟节点进行设置，也即根据哈希值映射在存储节点哈希环上的位置，得到存储节点哈希环。
可选地，在本申请实施例提供的数据处理方法中，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环包括：在磁盘的虚拟节点个数为一个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，得到磁盘的哈希值；在磁盘的虚拟节点个数为多个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至计算哈希值的次数达到磁盘的虚拟节点个数，得到一个磁盘的多个哈希值；根据各个磁盘的哈希值在初始哈希环上分别设置磁盘的虚拟节点，得到磁盘哈希环。
可选的，存储节点中包含多个磁盘，三级哈希环中存储节点哈希环与磁盘哈希环存在从属关系，每个存储节点关联的磁盘哈希环由该存储节点中的各个磁盘的唯一特征值以及磁盘的虚拟节点个数确定。
可选地，在本申请实施例提供的数据处理方法中，磁盘的虚拟节点个数由磁盘的存储空间确定，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
可选的，磁盘哈希环中各个磁盘的虚拟节点个数由磁盘存储总量归一化处理后得出，例如，存储节点中包含磁盘A和磁盘B，磁盘A的存储空间内存大小与磁盘B的存储空间内存大小比例为3比1，则当磁盘A的虚拟节点个数为3时，磁盘B的虚拟节点的个数1个，磁盘的存储空间越大，虚拟节点的个数越多，也即重要性程度越大。
例如，磁盘B的虚拟节点的个数为一个，磁盘B的哈希值按照哈希算法以及磁盘的特征值进行一次哈希计算得到；磁盘A的虚拟节点的个数为三个，首先利用哈希算法以及该磁盘的特征值计算得到第一磁盘哈希值，将第一磁盘哈希值继续利用哈希算法以及磁盘特征值计算得到第二磁盘哈希值；基于第二磁盘哈希值，并利用哈希算法以及磁盘特征值计算得到第三磁盘哈希值，使得计算哈希值的次数为三次，与磁盘的个数保持一致，得到一个磁盘的哈希值包括：第一磁盘哈希值、第二磁盘哈希值以及第三磁盘哈希值。
在得到磁盘A和磁盘B的四个虚拟节点的哈希值后，根据计算所得的磁盘哈希值，在哈希环上对虚拟节点进行设置，也即根据哈希值映射在磁盘哈希环上的位置，可以建立数据与磁盘哈希环的映射关系。
可选地，在本申请实施例提供的数据处理方法中，在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据包括：判断目标存储节点是否存储有待读取数据，并在目标存储节点存储有待读取数据的情况下，将目标存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；在目标存储节点未存储待读取数据的情况下，判断目标存储节点所在存储域是否存在存储有待读取数据的存储节点，并在存在的情况下，将存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；在目标存储节点所在存储域不存在存储有待读取数据的存储节点的情况下，确定与目标存储节点所在存储域距离最近的目的存储域，并目的存储域中存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据。
可选的，当需要从软件定义存储中读取数据时，需要按照目标节点中存储的三级哈希环的映射关系进行读取，也即，按照数据与存储节点、存储域的映射关系进行读取。
可选的，在客户端侧发送读取需求时，按照副本的拓扑关系中最近节点的副本，首先判断目标存储节点是否存储需要读取的数据，当确定存在时，将该目标存储节点确定为目的存储节点，并从目的存储节点中读取数据；当未在目标存储节点中查找到，需要判断目标存储节点所在的存储域中是否包含数据。
当该存储域中包含所需数据时，查找到包含所需数据的存储节点，从该存储节点中读取数据；当该存储域中不存在所需数据时，也即确定与目标存储节点所在存储域距离最近的存储域，从该存储域中含有所需数据的存储节点中读 取数据。
可选地，在本申请实施例提供的数据处理方法中，该方法还包括：在目的存储节点所在存储域出现故障的情况下，从目的存储节点所在存储域以外的存储域重新确定目的存储节点，并从重新确定的目的存储节点的磁盘读取数据。
可选的，当目的存储节点所在的存储域出现故障时，需要从该存储节点所在的存储域之外数据所有副本中拓扑关系中最近节点的副本，也即目的存储节点所在存储域以外的最近的存储域中重新确定存储节点，例如，若该存储节点所在的存储域相邻的存储域中的存储节点存在数据副本，则将相邻的存储域中的存储节点确定为目的存储节点，从新确定的目的存储节点的磁盘中读取数据。
可选地，在本申请实施例提供的数据处理方法中，在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘包括：确定待写入数据的副本的预设数量，从存储域哈希环的第一个虚拟节点开始顺时针确定预设数量的虚拟节点，并将待写入数据写入预设数量的虚拟节点指示的存储域的存储节点的磁盘中。
可选的，存储域可将单一集群中多个节点间进行隔离，以多副本机制实现数据本地容灾，也即在某一节点或者集群中存储多个副本，存储域与存储域之间互为对方副本也即按照存储域划分副本，例如，存储域包含两个，分别是A存储域以及B存储域，A存储域中各个节点存储的数据在B存储域中有对应的副本。可选的，确定写入数据的副本的数量，从存储域哈希环初始位置开始顺时针运动，″遇到″的第一个虚拟节点开始确定为写入数据的第一个副本，将待写入的数据写入到哈希环的多个副本中，也即将数据到存储域哈希环中的存储节点的磁盘中。
图5是根据本申请实施例的基于存储域哈希环执行数据写入操作的示意图一，如图5所示，当哈希环上设置有两个存储域时，在待写入数据为数a时，按照顺时针顺序，初始位置顺时针寻找到的第一个存储域(存储域A)即为第一副本，在第一副本后按照顺时针找到的下一存储域(存储域B)即为第二副本位置。在待写入数据为数b时，按照顺时针顺序，初始位置顺时针寻找到的第一个存储域(存储域C)即为第一副本，在第一副本后按照顺时针找到的下一个非第一副本所在的存储域(存储域A)即为第二副本位置。
图6是根据本申请实施例的基于存储域哈希环执行数据写入操作的示意图二，如图6所示，当哈希环上设置有三个存储域时，在待写入数据为数a时，按照顺时针顺序，初始位置顺时针寻找到的第一个存储域(存储域A)即为第一副本，在第一副本后按照顺时针找到的下一存储域(存储域B)即为第二副本位置，在第二副本后按照顺时针找到的下一存储域(存储域C)即为第三副本位置。在待写入数据为数b时，按照顺时针顺序，初始位置顺时针寻找到的第一个存储域(存储域C)即为第一副本，在第一副本后按照顺时针找到的下一存储域(存储域A)即为第二副本位置，在第二副本后按照顺时针找到下一个 非第一副本所在的存储域(存储域C)即为第三副本位置。
可选地，在本申请实施例提供的数据处理方法中，该方法还包括：在分布式存储集群中的存储域存在故障的情况下，若预设数量的虚拟节点中包含故障存储域的虚拟节点，则跳过故障存储域的虚拟节点继续确定虚拟节点，直至得到不包含故障存储域的虚拟节点的预设数量的虚拟节点。
可选的，故障域可以表示为单一集群跨存储域的划分，当分布式存储集群中的存储域包含了故障域，当存在读写需求时，检索含有读写需求的所有虚拟节点是否存在于故障域中，若有虚拟节点存在于故障域，则跳过该节点重新确定新的写入或读取数据的虚拟节点，直至含有读写需求的所有虚拟节点均不存在于故障域中。
可选地，在本申请实施例提供的数据处理方法中，该方法还包括：在分布式存储集群中的存储节点存在故障的情况下，在存储节点所在存储域执行数据恢复操作；在分布式存储集群中的磁盘存在故障的情况下，在磁盘所在存储域执行数据恢复操作。
例如，当数据a应落在存储域中的存储节点A和B中，存储域包含三个存储节点，分别为A、B、C，当存储域中的存储节点A出现故障时，或者存储节点A中的磁盘出现故障时，需要将存储域的存储节点A的各个磁盘中的数据进行转移，移到下一存储域的存储节点中，也即转移至存储域的存储节点C中，其余部分保持不动，也即存储节点B中的数据保持不动。从而达到了不影响存储域哈希环的重建的情况下，保证数据符合不同存储域分布的可靠性的效果。
可选地，在本申请实施例提供的数据处理方法中，该方法还包括：在分布式存储集群中存在存储节点故障的情况下，重建存储节点哈希环和磁盘哈希环，并建立重建后的存储节点哈希环与重建前的存储域哈希环之间的映射关系，建立重建后的磁盘哈希环和重建后的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
可选的，当分布式存储集群需要重建故障存储节点时，需要重建存储节点哈希环和磁盘哈希环，根据故障存储节点的特征值重建存储节点哈希环，根据故障存储节点的各个磁盘的特征值重建磁盘哈希环，将重建后的存储节点哈希环中的数据对应到重建前的存储域哈希环中，形成映射关系，将重建后的磁盘哈希环中的数据对应到重建后的存储节点哈希环中，形成一一映射关系，进而可以得到更新后的分布式哈希表。
可选地，在本申请实施例提供的数据处理方法中，该方法还包括：在分布式存储集群中存在磁盘缩容需求的情况下，重建磁盘哈希环，在分布式哈希表上删除重建前的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，建立重建后的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
可选的，当分布式存储集群的节点中的磁盘不能存放更多的数据时，需要对磁盘进行扩容，另一方面当磁盘内空闲空间占用过大，需要对磁盘进行缩容。在扩容和缩容的需求下，需要将存储节点以及磁盘进行更新，也即在分布 式哈希表中将重建前的存储节点以及磁盘的对应哈希环的映射关系删除，建立新的哈希环映射关系，根据新的哈希环映射关系，更新分布式哈希表。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个非易失性可读存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
在本实施例中还提供了一种数据处理装置，该装置用于实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语″模块″可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图7是根据本申请实施例的数据处理装置的流程图，如图7所示，该装置包括连接单元70、写入单元71和读取单元72：
连接单元70，被配置为通过客户端连接分布式存储集群中的目标存储节点，其中，目标存储节点是存储集群中的任意存储节点，存储集群中的每个存储节点均存储有分布式哈希表，分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系。
写入单元71，被配置为在存在数据写入需求的情况下，根据目标存储节点存储的分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘。
读取单元72，被配置为在存在数据读取需求的情况下，根据目标存储节点存储的分布式哈希表索引存储有待读取数据的存储节点，并从存储有待读取数据的存储节点中的磁盘读取数据。
可选地，在本申请实施例提供的镜像的数据处理装置中，连接单元70包括：分布式哈希表通过以下方式建立：划分模块，被配置为将存储集群中的多个存储节点划分至不同的存储域，获取各个存储域的特征值并设置每个存储域的虚拟节点个数，根据各个存储域的特征值以及虚拟节点个数生成存储域哈希环，其中，存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个；第一获取模块，被配置为对于每个存储域，获取各个存储节点的特征值并设置每个存储节点的虚拟节点个数，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环，其中，存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个；第二获取模块，被配置为对于每个存储节点，获取各个磁盘的特征值并设置每个磁盘的虚拟节点个数，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环，其中，磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个；建立模块，被配置为根据存储节点与存储域之间的归属关系建立各个存 储节点哈希环与存储域哈希环之间的映射关系，根据磁盘与存储节点之间的归属关系建立各个磁盘哈希环与存储节点哈希环之间的映射关系，得到分布式哈希表。
可选地，在本申请实施例提供的镜像的数据处理装置中，连接单元70包括：第一计算模块，被配置为在存储域的虚拟节点个数为一个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，得到存储域的哈希值；第二计算模块，被配置为在存储域的虚拟节点个数为多个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储域的虚拟节点个数，得到一个存储域的多个哈希值；第一设置模块，被配置为根据各个存储域的哈希值在初始哈希环上分别设置存储域的虚拟节点，得到存储域哈希环。
可选地，在本申请实施例提供的镜像的数据处理装置中，连接单元70包括：第三计算模块，被配置为在存储节点的虚拟节点个数为一个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，得到存储节点的哈希值；第四计算模块，被配置为在存储节点的虚拟节点个数为多个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至重复计算哈希值的次数达到存储节点的虚拟节点个数，得到一个存储节点的多个哈希值；第二设置模块，被配置为根据各个存储节点的哈希值在初始哈希环上分别设置存储节点的虚拟节点，得到存储节点哈希环。
可选地，在本申请实施例提供的镜像的数据处理装置中，连接单元70包括：第五计算模块，被配置为在磁盘的虚拟节点个数为一个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，得到磁盘的哈希值；第六计算模块，被配置为在磁盘的虚拟节点个数为多个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，并基于得到的哈希值再次根据预设哈希算法计算哈希值，直至计算哈希值的次数达到磁盘的虚拟节点个数，得到一个磁盘的多个哈希值；第三设置模块，被配置为根据各个磁盘的哈希值在初始哈希环上分别设置磁盘的虚拟节点，得到磁盘哈希环。
可选地，在本申请实施例提供的镜像的数据处理装置中，连接单元70包括：第一确定模块，被配置为存储域的虚拟节点个数由存储域的重要程度确定，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：第二确定模块，被配置为存储节点的虚拟节点个数由存储节点的存储空间确定，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：第三确定模块，被配置为磁盘的虚拟节点个数由磁盘的存储空间确定，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
可选地，在本申请实施例提供的镜像的数据处理装置中，读取单元72包括：第一判断模块，被配置为判断目标存储节点是否存储有待读取数据，并在 目标存储节点存储有待读取数据的情况下，将目标存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；第二判断模块，被配置为在目标存储节点未存储待读取数据的情况下，判断目标存储节点所在存储域是否存在存储有待读取数据的存储节点，并在存在的情况下，将存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据；第四确定模块，被配置为在目标存储节点所在存储域不存在存储有待读取数据的存储节点的情况下，确定与目标存储节点所在存储域距离最近的目的存储域，并目的存储域中存储有待读取数据的存储节点确定为目的存储节点，从目的存储节点的磁盘读取数据。
可选地，在本申请实施例提供的镜像的数据处理装置中，读取单元72包括：第五确定模块，被配置为在目的存储节点所在存储域出现故障的情况下，从目的存储节点所在存储域以外的存储域重新确定目的存储节点，并从重新确定的目的存储节点的磁盘读取数据。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：第六确定模块，被配置为确定待写入数据的副本的预设数量，从存储域哈希环的第一个虚拟节点开始顺时针确定预设数量的虚拟节点，并将待写入数据写入预设数量的虚拟节点指示的存储域的存储节点的磁盘中。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：跳过模块，被配置为在分布式存储集群中的存储域存在故障的情况下，若预设数量的虚拟节点中包含故障存储域的虚拟节点，则跳过故障存储域的虚拟节点继续确定虚拟节点，直至得到不包含故障存储域的虚拟节点的预设数量的虚拟节点。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：执行模块，被配置为在分布式存储集群中的存储节点存在故障的情况下，在存储节点所在存储域执行数据恢复操作；在分布式存储集群中的磁盘存在故障的情况下，在磁盘所在存储域执行数据恢复操作。
可选地，在本申请实施例提供的镜像的数据处理装置中，写入单元71包括：第一重建模块，被配置为在分布式存储集群中存在存储节点故障的情况下，重建存储节点哈希环和磁盘哈希环，并建立重建后的存储节点哈希环与重建前的存储域哈希环之间的映射关系，建立重建后的磁盘哈希环和重建后的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
可选地，在本申请实施例提供的镜像的数据处理装置中，装置包括：第二重建模块，被配置为在分布式存储集群中存在磁盘缩容需求的情况下，重建磁盘哈希环，在分布式哈希表上删除重建前的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，建立重建后的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
需要说明的是，上述各个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述各个模块以任意组合的形式分别位于不同的处理器中。
本申请的实施例还提供了一种计算机非易失性可读存储介质，该计算机非易失性可读存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述计算机非易失性可读存储介质可以包括但不限于：U盘、只读存储器(Read-Only Memory，简称为ROM)、随机存取存储器(Random Access Memory，简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
本申请的实施例还提供了一种电子设备，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述电子设备还可以包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
本实施例中的具体示例可以参考上述实施例及示例性实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本申请的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本申请不限制于任何特定的硬件和软件结合。
以上所述仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (22)

  1. 一种数据处理方法，其特征在于，包括：
    通过客户端连接分布式存储集群中的目标存储节点，其中，所述目标存储节点是所述存储集群中的任意存储节点，所述存储集群中的每个存储节点均存储有分布式哈希表，所述分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；
    在存在数据写入需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘；
    在存在数据读取需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引存储有待读取数据的存储节点，并从所述存储有待读取数据的存储节点中的磁盘读取数据。
  2. 根据权利要求1所述的方法，其特征在于，所述分布式哈希表通过以下方式建立：
    将所述存储集群中的多个存储节点划分至不同的存储域，获取各个存储域的特征值并设置每个存储域的虚拟节点个数，根据各个存储域的特征值以及虚拟节点个数生成所述存储域哈希环，其中，所述存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个；
    对于每个存储域，获取各个存储节点的特征值并设置每个存储节点的虚拟节点个数，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环，其中，所述存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个；
    对于每个存储节点，获取各个磁盘的特征值并设置每个磁盘的虚拟节点个数，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环，其中，所述磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个；
    根据存储节点与存储域之间的归属关系建立各个存储节点哈希环与存储域哈希环之间的映射关系，根据磁盘与存储节点之间的归属关系建立各个磁盘哈希环与存储节点哈希环之间的映射关系，得到所述分布式哈希表。
  3. 根据权利要求2所述的方法，其特征在于，根据各个存储域的特征值以及虚拟节点个数生成所述存储域哈希环包括：
    在存储域的虚拟节点个数为一个的情况下，根据预设哈希算法对存储域的特征值计算哈希值，得到存储域的哈希值；
    在存储域的虚拟节点个数为多个的情况下，根据所述预设哈希算法对存储域的特征值计算哈希值，并基于得到的哈希值再次根据所述预设哈希算法计算哈希值，直至重复计算哈希值的次数达到所述存储域的虚拟节点个数，得到一个存储域的多个哈希值；
    根据各个存储域的哈希值在初始哈希环上分别设置存储域的虚拟节点，得到所述存储域哈希环。
  4. 根据权利要求2所述的方法，其特征在于，根据各个存储节点的特征值以及虚拟节点个数生成存储节点哈希环包括：
    在存储节点的虚拟节点个数为一个的情况下，根据预设哈希算法对存储节点的特征值计算哈希值，得到存储节点的哈希值；
    在存储节点的虚拟节点个数为多个的情况下，根据所述预设哈希算法对存储节点的特征值计算哈希值，并基于得到的哈希值再次根据所述预设哈希算法计算哈希值，直至重复计算哈希值的次数达到所述存储节点的虚拟节点个数，得到一个存储节点的多个哈希值；
    根据各个存储节点的哈希值在初始哈希环上分别设置存储节点的虚拟节点，得到所述存储节点哈希环。
  5. 根据权利要求2所述的方法，其特征在于，根据各个磁盘的特征值以及虚拟节点个数生成磁盘哈希环包括：
    在磁盘的虚拟节点个数为一个的情况下，根据预设哈希算法对磁盘的特征值计算哈希值，得到磁盘的哈希值；
    在磁盘的虚拟节点个数为多个的情况下，根据所述预设哈希算法对磁盘的特征值计算哈希值，并基于得到的哈希值再次根据所述预设哈希算法计算哈希值，直至计算哈希值的次数达到所述磁盘的虚拟节点个数，得到一个磁盘的多个哈希值；
    根据各个磁盘的哈希值在初始哈希环上分别设置磁盘的虚拟节点，得到所述磁盘哈希环。
  6. 根据权利要求2所述的方法，其特征在于，存储域的虚拟节点个数由存储域的重要程度确定，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
  7. 根据权利要求2所述的方法，其特征在于，存储节点的虚拟节点个数由存储节点的存储空间确定，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
  8. 根据权利要求2所述的方法，其特征在于，磁盘的虚拟节点个数由磁盘的存储空间确定，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
  9. 根据权利要求1所述的方法，其特征在于，在存在数据读取需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引存储有待读取数据的存储节点，并从所述存储有待读取数据的存储节点中的磁盘读取数据包括：
    判断所述目标存储节点是否存储有所述待读取数据，并在所述目标存储节点存储有所述待读取数据的情况下，将所述目标存储节点确定为目的存储节点，从所述目的存储节点的磁盘读取数据；
    在所述目标存储节点未存储所述待读取数据的情况下，判断所述目标存储节点所在存储域是否存在存储有所述待读取数据的存储节点，并在存在的情况下，将存储有所述待读取数据的存储节点确定为目的存储节点，从所述目的存储节点的磁盘读取数据；
    在所述目标存储节点所在存储域不存在存储有所述待读取数据的存储节点的情况下，确定与所述目标存储节点所在存储域距离最近的目的存储域，并将所述目的存储域中存储有所述待读取数据的存储节点确定为目的存储节点，从所述目的存储节点的磁盘读取数据。
  10. 根据权利要求9所述的方法，其特征在于，还包括：
    在所述目的存储节点所在存储域出现故障的情况下，从所述目的存储节点所在存储域以外的存储域重新确定目的存储节点，并从重新确定的目的存储节点的磁盘读取数据。
  11. 根据权利要求1所述的方法，其特征在于，在存在数据写入需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘包括：
    确定所述待写入数据的副本的预设数量，从所述存储域哈希环的第一个虚拟节点开始顺时针确定所述预设数量的虚拟节点，并将所述待写入数据写入所述预设数量的虚拟节点指示的存储域的存储节点的磁盘中。
  12. 根据权利要求11所述的方法，其特征在于，还包括：
    在所述分布式存储集群中的存储域存在故障的情况下，若所述预设数量的虚拟节点中包含故障存储域的虚拟节点，则跳过所述故障存储域的虚拟节点继续确定虚拟节点，直至得到不包含所述故障存储域的虚拟节点的所述预设数量的虚拟节点。
  13. 根据权利要求1所述的方法，其特征在于，还包括：
    在所述分布式存储集群中的存储节点存在故障的情况下，在所述存储节点所在存储域执行数据恢复操作；在所述分布式存储集群中的磁盘存在故障的情况下，在所述磁盘所在存储域执行数据恢复操作。
  14. 根据权利要求2所述的方法，其特征在于，还包括：
    在所述分布式存储集群中存在存储节点故障的情况下，重建存储节点哈希环和磁盘哈希环，并建立重建后的存储节点哈希环与重建前的存储域哈希环之间的映射关系，建立重建后的磁盘哈希环和重建后的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
  15. 根据权利要求2所述的方法，其特征在于，还包括：
    在所述分布式存储集群中存在磁盘缩容需求的情况下，重建磁盘哈希环，在所述分布式哈希表上删除重建前的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，建立重建后的磁盘哈希环和重建前的存储节点哈希环之间的映射关系，得到更新后的分布式哈希表。
  16. 一种数据处理系统，其特征在于，包括：
    多个机房；
    分布式存储集群，所述分布式存储集群中的多个存储节点部署在所述多个机房内，其中，所述多个存储节点划分至不同的存储域，同一存储域中的存储节点设部署在同一机房内，每个存储节点均存储有分布式哈希表，所述分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；
    客户端，与所述分布式存储集群中的任意一个存储节点连接，用于通过连接的存储节点执行数据读写操作。
  17. 根据权利要求16所述的数据处理系统，其特征在于，所述存储域哈希环上分布有各个存储域的虚拟节点，每个存储域的虚拟节点至少包含一个，各个存储域的虚拟节点个数与存储域的重要程度呈正相关关系。
  18. 根据权利要求16所述的数据处理系统，其特征在于，所述存储节点哈希环上分布有各个存储节点的虚拟节点，每个存储节点的虚拟节点至少包含一个，各个存储节点的虚拟节点个数与存储节点的存储空间呈正相关关系。
  19. 根据权利要求16所述的数据处理系统，其特征在于，所述磁盘哈希环上分布有各个磁盘的虚拟节点，每个磁盘的虚拟节点至少包含一个，各个磁盘的虚拟节点个数与磁盘的存储空间呈正相关关系。
  20. 一种数据处理装置，其特征在于，包括：
    连接单元，被配置为通过客户端连接分布式存储集群中的目标存储节点，其中，所述目标存储节点是所述存储集群中的任意存储节点，所述存储集群中的每个存储节点均存储有分布式哈希表，所述分布式哈希表用于表征存储域哈希环、存储节点哈希环以及磁盘哈希环之间的映射关系；
    写入单元，被配置为在存在数据写入需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引待写入数据的各个副本所在存储节点，并控制副本所在存储节点将数据写入磁盘；
    读取单元，被配置为在存在数据读取需求的情况下，根据所述目标存储节点存储的所述分布式哈希表索引存储有待读取数据的存储节点，并从所述存储有待读取数据的存储节点中的磁盘读取数据。
  21. 一种计算机非易失性可读存储介质，其特征在于，所述计算机非易失性可读存储介质中存储有计算机程序，其中，所述计算机程序被处理器执行时实现所述权利要求1至15任一项中所述的方法的步骤。
  22. 一种电子设备，包括存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现所述权利要求1至15任一项中所述的方法的步骤。
PCT/CN2023/114305 2023-01-09 2023-08-22 数据处理方法、系统、装置、存储介质及电子设备 WO2024148824A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310029731.8 2023-01-09
CN202310029731.8A CN115878046B (zh) 2023-01-09 2023-01-09 数据处理方法、系统、装置、存储介质及电子设备

Publications (1)

Publication Number Publication Date
WO2024148824A1 true WO2024148824A1 (zh) 2024-07-18

Family

ID=85758380

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/114305 WO2024148824A1 (zh) 2023-01-09 2023-08-22 数据处理方法、系统、装置、存储介质及电子设备

Country Status (2)

Country Link
CN (1) CN115878046B (zh)
WO (1) WO2024148824A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115878046B (zh) * 2023-01-09 2023-05-12 苏州浪潮智能科技有限公司 数据处理方法、系统、装置、存储介质及电子设备
CN118377835A (zh) * 2024-06-21 2024-07-23 支付宝(杭州)信息技术有限公司 用于分布式图数据库的数据处理方法和装置

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016189105A (ja) * 2015-03-30 2016-11-04 日本電気株式会社 ストレージシステム
CN110058822A (zh) * 2019-04-26 2019-07-26 北京计算机技术及应用研究所 一种磁盘阵列横向拓展方法
CN110096227A (zh) * 2019-03-28 2019-08-06 北京奇艺世纪科技有限公司 数据存储方法、数据处理方法、装置、电子设备及计算机可读介质
US20200073765A1 (en) * 2018-09-03 2020-03-05 Ashish Tanwer Distributed backup and precise recovery for consistent hashing systems
CN113326006A (zh) * 2021-06-17 2021-08-31 上海天玑科技股份有限公司 一种基于纠删码的分布式块存储系统
CN113806298A (zh) * 2021-08-09 2021-12-17 中国地质大学（武汉） 分布式文件系统存储优化方法、计算机设备及存储介质
CN114816225A (zh) * 2021-01-28 2022-07-29 北京金山云网络技术有限公司 存储集群的管理方法、装置、电子设备及存储介质
CN114879907A (zh) * 2022-04-28 2022-08-09 济南浪潮数据技术有限公司 一种数据分布确定方法、装置、设备及存储介质
CN115878046A (zh) * 2023-01-09 2023-03-31 苏州浪潮智能科技有限公司 数据处理方法、系统、装置、存储介质及电子设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115113893A (zh) * 2021-03-22 2022-09-27 腾讯科技（深圳）有限公司 一种数据处理方法、装置、存储介质和计算机设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016189105A (ja) * 2015-03-30 2016-11-04 日本電気株式会社 ストレージシステム
US20200073765A1 (en) * 2018-09-03 2020-03-05 Ashish Tanwer Distributed backup and precise recovery for consistent hashing systems
CN110096227A (zh) * 2019-03-28 2019-08-06 北京奇艺世纪科技有限公司 数据存储方法、数据处理方法、装置、电子设备及计算机可读介质
CN110058822A (zh) * 2019-04-26 2019-07-26 北京计算机技术及应用研究所 一种磁盘阵列横向拓展方法
CN114816225A (zh) * 2021-01-28 2022-07-29 北京金山云网络技术有限公司 存储集群的管理方法、装置、电子设备及存储介质
CN113326006A (zh) * 2021-06-17 2021-08-31 上海天玑科技股份有限公司 一种基于纠删码的分布式块存储系统
CN113806298A (zh) * 2021-08-09 2021-12-17 中国地质大学（武汉） 分布式文件系统存储优化方法、计算机设备及存储介质
CN114879907A (zh) * 2022-04-28 2022-08-09 济南浪潮数据技术有限公司 一种数据分布确定方法、装置、设备及存储介质
CN115878046A (zh) * 2023-01-09 2023-03-31 苏州浪潮智能科技有限公司 数据处理方法、系统、装置、存储介质及电子设备

Also Published As

Publication number Publication date
CN115878046A (zh) 2023-03-31
CN115878046B (zh) 2023-05-12

Similar Documents

Publication Publication Date Title
US10956601B2 (en) Fully managed account level blob data encryption in a distributed storage environment
US10185497B2 (en) Cluster federation and trust in a cloud environment
US9906598B1 (en) Distributed data storage controller
WO2024148824A1 (zh) 数据处理方法、系统、装置、存储介质及电子设备
US9405781B2 (en) Virtual multi-cluster clouds
US20180373459A1 (en) Redistributing encoded data slices in a dispersed storage network
US9794337B2 (en) Balancing storage node utilization of a dispersed storage network
US8930364B1 (en) Intelligent data integration
US9031906B2 (en) Method of managing data in asymmetric cluster file system
US20170177229A1 (en) Storage of data with verification in a dispersed storage network
US10712964B2 (en) Pre-forking replicas for efficient scaling of a distributed data storage system
JP2017531857A (ja) 分散型能動ハイブリッドストレージシステム
CN109407975B (zh) 写数据方法与计算节点以及分布式存储系统
US20200401340A1 (en) Distributed storage system
KR101527634B1 (ko) 샤딩 서비스를 제공하는 방법 및 장치
JP6671708B2 (ja) バックアップリストアシステム及びバックアップリストア方法
US11334456B1 (en) Space efficient data protection
WO2016206392A1 (zh) 数据读写方法及装置
JP5956364B2 (ja) クラスタシステム
JP6291977B2 (ja) 分散ファイルシステム、バックアップファイル取得方法、制御装置及び管理装置
JP2006155398A (ja) データ仲介方法およびデータ仲介装置
US20180107398A1 (en) Vault synchronization within a dispersed storage network
JP2015088109A (ja) ストレージシステム、ストレージ制御方法、およびストレージ制御プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23915585

Country of ref document: EP

Kind code of ref document: A1