WO2026020992A1 - 分布式系统负载均衡方法、装置、电子设备及介质 - Google Patents
分布式系统负载均衡方法、装置、电子设备及介质Info
- Publication number
- WO2026020992A1 WO2026020992A1 PCT/CN2025/098749 CN2025098749W WO2026020992A1 WO 2026020992 A1 WO2026020992 A1 WO 2026020992A1 CN 2025098749 W CN2025098749 W CN 2025098749W WO 2026020992 A1 WO2026020992 A1 WO 2026020992A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- offset
- storage node
- compute
- computing
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
Definitions
- This disclosure relates to the field of data network technology, and in particular to a method, apparatus, electronic device and medium for load balancing of distributed systems.
- a distributed system consists of a scheduling node and multiple independent computing and storage nodes.
- An index is a multi-field data body (such as a data table) to be computed and stored.
- the scheduling node divides the index into shards and stores them on various computing and storage nodes for computation and storage.
- a load balancing algorithm is generally used to determine the number of shards allocated to each computing and storage unit.
- One existing load balancing algorithm allocates shards to each computing storage unit based on its disk utilization. That is, when allocating shards for an index, shards are preferentially assigned to computing storage units with lower disk utilization.
- this method can easily lead to a concentration of larger shards in a particular computing storage unit, which can reduce its request processing efficiency due to the concentrated storage of large shards. In severe cases, this can even cause the entire distributed system to fail. How to achieve load balancing in a distributed system while improving the request processing efficiency of computing storage units is a problem that urgently needs to be discussed and solved.
- This disclosure provides a distributed system load balancing method, apparatus, electronic device, and medium.
- a distributed system load balancing method applied to a scheduling node in a distributed system, the distributed system including the scheduling node and multiple computing and storage nodes, wherein an index stored in the distributed system is decomposed into shards and stored in the multiple computing and storage nodes, the index being a multi-field data body to be computed and stored, and the distributed system load balancing method includes:
- an index change is performed on the plurality of compute storage nodes
- the first offset is the offset of the number of shards of a single index stored on the compute storage node relative to the average number of shards of the single index stored on each compute storage node;
- the second offset is the offset of the total number of shards stored by the compute storage node relative to the average total number of shards stored by each compute storage node;
- the fragments on the plurality of computing and storage nodes are migrated, such that after migration, the first offset on each computing and storage node is less than a first threshold, the second offset is less than a second threshold, and the third offset is less than a third threshold.
- a distributed system load balancing device applied to a scheduling node in a distributed system, the distributed system including the scheduling node and multiple computing and storage nodes, wherein an index stored in the distributed system is decomposed into shards and stored in the multiple computing and storage nodes, the index being a multi-field data body to be computed and stored, and the distributed system load balancing device includes:
- a response unit is configured to perform an index change on the plurality of compute storage nodes in response to an index change request
- a first determining unit is configured to: determine a first offset for each compute storage node after the index change, wherein the first offset is the offset of the number of shards of a single index stored on the compute storage node relative to the average number of shards of a single index stored on all compute storage nodes; determine a second offset for each compute storage node after the index change, wherein the second offset is the offset of the total number of shards stored on the compute storage node relative to the average total number of shards stored on all compute storage nodes; and determine a third offset for each compute storage node after the index change, wherein the third offset is the offset of the storage space utilization rate of the compute storage node relative to the average storage space utilization rate of all compute storage nodes; and
- the migration unit is used to migrate the fragments on the plurality of computing and storage nodes based on the first offset, the second offset and the third offset, such that after migration, the first offset on each computing and storage node is less than a first threshold, the second offset is less than a second threshold and the third offset is less than a third threshold.
- an electronic device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the distributed system load balancing method as described above.
- a computer-readable storage medium storing a computer program that, when executed by a processor, implements the distributed system load balancing method described above.
- a computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the distributed system load balancing method as described above.
- Figures 1A and 1B are architecture diagrams of a distributed system load balancing method according to embodiments of the present disclosure
- Figures 2A and 2B are schematic diagrams illustrating application scenarios of the distributed system load balancing method according to embodiments of the present disclosure
- Figure 3 is a general flowchart of a distributed system load balancing method according to an embodiment of the present disclosure
- Figure 4A is a schematic diagram of the sharding of various indexes in a distributed system according to an embodiment of the present disclosure
- Figure 4B is a schematic diagram of the index being decomposed into fragments and stored in multiple computing storage nodes according to an embodiment of the present disclosure
- Figure 5A is a schematic diagram of the migration of shards on a computing storage node according to an embodiment of the present disclosure
- Figure 5B is a schematic diagram of the shards stored by the storage node after performing shard migration according to an embodiment of the present disclosure
- Figure 6 is a flowchart illustrating the specific process of performing index changes on multiple compute storage nodes in response to an index change request according to an embodiment of the present disclosure
- Figures 7A and 7B are schematic diagrams illustrating the execution of index changes on multiple compute storage nodes in response to an index allocation request for a target index, according to an embodiment of the present disclosure.
- Figures 8A-8B are schematic diagrams illustrating the execution of index changes on multiple compute storage nodes in response to an index deletion request for a target index, according to an embodiment of the present disclosure.
- Figure 9 is a detailed flowchart of dividing a target index into multiple fragments according to an embodiment of the present disclosure.
- Figure 10 is a schematic diagram of determining the number of fragments based on the target index size, average storage capacity, and average storage space utilization according to an embodiment of the present disclosure.
- Figure 11 is a detailed flowchart of the migration of fragments on multiple computing and storage nodes based on a first offset, a second offset, and a third offset according to an embodiment of the present disclosure.
- Figure 12 is a schematic diagram of the migration of fragments on multiple computing and storage nodes based on a first offset, a second offset, and a third offset according to an embodiment of the present disclosure.
- Figure 13 is a detailed flowchart of the migration of fragments on multiple computing and storage nodes based on a first offset, a second offset, and a third offset for the traversed index, according to an embodiment of the present disclosure.
- Figure 14 is a detailed flowchart of performing a first sorting of multiple computing storage nodes based on a first offset, a second offset, and a third offset according to an embodiment of the present disclosure
- Figure 15 is a schematic diagram of a first sorting of multiple computing storage nodes based on a first offset, a second offset, and a third offset according to an embodiment of the present disclosure
- Figure 16 is a detailed flowchart of a first sorting of multiple computing and storage nodes based on a weighted and descending order according to an embodiment of the present disclosure
- Figure 17 is a schematic diagram of a first sorting of multiple computing storage nodes based on a weighted and descending order according to an embodiment of the present disclosure
- Figure 18 is a detailed flowchart of migrating a fragment on a traversed compute storage node to a compute storage node with a first offset smaller than that of the traversed compute storage node, according to an embodiment of the present disclosure.
- Figure 19 is a schematic diagram of initializing the first target migration compute storage node as the first compute storage node in the second sorting according to an embodiment of the present disclosure
- Figures 20A-20B are schematic diagrams illustrating the migration of a first target fragment from a traversed compute-storage node to a first target migration compute-storage node according to an embodiment of the present disclosure.
- Figure 21 is a detailed flowchart of selecting a first target fragment on the traversed computing and storage nodes according to an embodiment of the present disclosure
- Figure 22 is a schematic diagram of obtaining the fragment size of each fragment on the traversed computing storage node according to an embodiment of the present disclosure
- Figure 23 is a flowchart illustrating the migration of fragments on traversed compute storage nodes to compute storage nodes with a first offset smaller than that of the traversed compute storage nodes, according to an embodiment of the present disclosure.
- Figure 24 is a schematic diagram of other computing storage nodes that match the second multiple with the first difference as target computing storage nodes according to an embodiment of the present disclosure
- Figure 25 is a detailed flowchart of determining the target fragment according to an embodiment of the present disclosure.
- Figure 26 is a flowchart illustrating the migration of a fragment on a traversed compute storage node to a compute storage node with a second offset smaller than that of the traversed compute storage node, according to an embodiment of the present disclosure.
- Figure 27 is a schematic diagram of initializing the second target migration computation storage node as the first computation storage node in the third sorting according to an embodiment of the present disclosure
- Figures 28A-28B are schematic diagrams illustrating the migration of a second target fragment from the traversed compute-storage node to the second target migration compute-storage node according to an embodiment of the present disclosure.
- Figure 29 is a flowchart illustrating the migration of a fragment on a traversed compute storage node to a compute storage node with a second offset smaller than that of the traversed compute storage node, according to an embodiment of the present disclosure.
- Figure 30 is a schematic diagram of initializing the third target migration computation storage node as the first computation storage node in the fourth sorting according to an embodiment of the present disclosure
- Figures 31A-31B are schematic diagrams illustrating the migration of a third target fragment from the traversed compute-storage node to the third target migration compute-storage node according to an embodiment of the present disclosure.
- Figure 32 is an implementation detail diagram of a distributed system load balancing method according to an embodiment of the present disclosure.
- Figure 33 is a block diagram of a distributed system load balancing device according to an embodiment of the present disclosure.
- Figure 34 is a terminal structure diagram of a distributed system load balancing method implemented according to an embodiment of the present disclosure
- Figure 35 is a server architecture diagram of a distributed system load balancing method implemented according to an embodiment of the present disclosure.
- Distributed systems consist of multiple independent computing and storage units. Data is distributed across each independent computing and storage unit. Each computing and storage unit is responsible for processing a portion of the data in a distributed request, thus efficiently completing a distributed request in a parallel manner.
- Load balancing algorithms are methods for evenly distributing network traffic across a pool of resources supporting an application. Modern applications must handle millions of users simultaneously and return the correct text, video, images, and other data to each user quickly and reliably. To handle such high traffic, most applications have numerous resource servers, which contain a lot of duplicate data. Load balancing algorithms ensure that all resource servers are used equally.
- Elasticsearch is a very popular open-source distributed search and analytics engine.
- a compute storage unit (Node) can be part of a cluster or standalone.
- a compute storage unit can contain one or more indexes.
- An index is a logical data storage unit in Elasticsearch that contains a set of documents with similar characteristics.
- a document is a data unit in Elasticsearch, containing fields and field values.
- a distributed system consists of a scheduling node and multiple independent computing and storage nodes.
- An index is a multi-field data body (such as a data table) to be computed and stored.
- the scheduling node divides the index into shards and stores them on various computing and storage nodes for computation and storage.
- a load balancing algorithm is generally used to determine the number of shards allocated to each computing and storage unit.
- One existing load balancing algorithm allocates shards to each computing storage unit based on its disk utilization. That is, when allocating shards for an index, shards are preferentially assigned to computing storage units with lower disk utilization.
- this method can easily lead to a concentration of larger shards in a particular computing storage unit, which can reduce its request processing efficiency due to the concentrated storage of large shards. In severe cases, this can even cause the entire distributed system to fail. How to achieve load balancing in a distributed system while improving the request processing efficiency of computing storage units is a problem that urgently needs to be discussed and solved.
- embodiments of this disclosure provide a distributed system load balancing method, apparatus, electronic device, and medium.
- the distributed system load balancing method provided by embodiments of this disclosure can migrate shards across multiple computing and storage nodes based on a first offset, a second offset, and a third offset after the computing and storage nodes have completed index changes. This ensures that after migration, each computing and storage node achieves load balancing in terms of the number of shards for a single index stored on the computing and storage node, the total number of shards stored on the computing and storage node, and the storage space utilization of the computing and storage node. This achieves load balancing of the distributed system while improving the efficiency of the computing and storage nodes in processing requests.
- Figures 1A and 1B are system architecture diagrams of the distributed system load balancing method applied according to embodiments of the present disclosure.
- Figure 1A includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.
- Terminal 140 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.
- PDAs personal digital assistants
- Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data.
- Server 110 refers to a computer system capable of providing certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). In a distributed system, multiple servers 110 are interconnected as computing and storage units to transmit data and coordinate task processing using shared communication lines.
- Gateway 120 also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.
- the scheduling node and computing/storage node of the distributed system load balancing method in this embodiment can be either a terminal 140 or a server 110. That is, the distributed system load balancing method in this embodiment can be implemented entirely on the terminal 140, entirely on the server 110, or partially on the terminal 140 and partially on the server 110.
- a cluster of a distributed system includes multiple scheduling nodes and multiple compute and storage nodes.
- Different scheduling nodes can synchronize metadata to ensure data consistency within the distributed system.
- scheduling nodes can distribute metadata or requests to compute and storage nodes, enabling them to store the metadata or forward requests to other compute and storage nodes.
- Metadata synchronization refers to the data synchronization operation between different scheduling nodes in a distributed system. The purpose is to ensure that the data in the distributed system remains consistent, so that the system metadata information held by each scheduling node is the same, so as to accurately send metadata or requests to the computing and storage nodes.
- the computing and storage nodes in a distributed system bear the functions of data storage and computation.
- Data in the cluster is stored in the form of indexes.
- An index is a multi-field data body to be computed and stored; it can also be understood as a logical data storage mechanism.
- An index will be split into multiple shards and distributed across various computing and storage nodes in the distributed system.
- Each index consists of a primary shard and multiple replica shards to improve data availability and performance.
- the primary shard is one of the components of the split index.
- Each index consists of a primary shard and multiple replica shards.
- the primary shard plays a crucial role in data storage and processing, serving as the primary storage location to improve data availability and performance.
- Replica shards are also components of the split index, forming an index together with the primary shard.
- Replica shards are copies of the primary shard, used to improve data availability and performance. When the primary shard fails or becomes unavailable, the replica shard can replace it to provide data services.
- compute storage node 1 can store shards 1, 2, and 3; compute storage node 2 can store shards 4, 5, and 6; and compute storage node 3 can store shards 7, 8, and 9.
- the distributed system load balancing method of this disclosure can be applied to a distributed search and analysis engine to improve the efficiency of the engine in processing requests.
- the distributed system load balancing method of this disclosure can also be applied to distributed applications to improve their data processing capabilities.
- FIGS 2A and 2B are schematic diagrams illustrating application scenarios of the distributed system load balancing method according to embodiments of the present disclosure.
- the scheduling node in the distributed retrieval and analysis engine will divide the target index E into multiple shards, which will be stored in various computing and storage nodes respectively.
- the scheduling node after the scheduling node completes the index allocation operation on each computing and storage node, it will also perform load balancing on the multiple shards in each computing and storage node using the distributed load balancing method of the present disclosure embodiments, so as to improve the efficiency of the computing and storage nodes in processing requests, thereby improving the efficiency of the distributed retrieval and analysis engine in processing requests.
- the scheduling node in the distributed retrieval and analysis engine upon receiving the request, will delete the shards of index A already stored on each compute storage node until no shard of index A remains on any compute storage node. Furthermore, after executing the index deletion operations on each compute storage node, the scheduling node will also perform load balancing on the multiple shards in each compute storage node using the distributed load balancing method of this embodiment, thereby improving the efficiency of the distributed retrieval and analysis engine in processing requests.
- one load balancing algorithm allocates shards to each computing storage unit based on its disk utilization. That is, when allocating index shards, shards are preferentially allocated to computing storage units with lower disk utilization.
- this method can easily lead to some larger shards being concentrated in a particular computing storage unit, which can reduce its request processing efficiency due to the concentrated storage of large shards, and in severe cases, even cause the entire distributed system to fail.
- an index change After an index change is performed on a computing storage node, it can migrate shards across multiple computing storage nodes based on a first offset, a second offset, and a third offset. This ensures that after migration, each computing storage node achieves load balancing in terms of the number of shards for a single index stored on the computing storage node, the total number of shards stored on the computing storage node, and the storage space utilization of the computing storage node, while also improving the efficiency of the computing storage node in processing requests.
- Load balancing refers to distributing shards across multiple compute and storage nodes to achieve a balanced load across each node. Maintaining a balanced load across compute and storage nodes ensures optimal cluster performance, maximizes the utilization of hardware resources, and guarantees cluster stability.
- a distributed system load balancing method is provided. This method can be applied to the distributed retrieval and analysis engine shown in Figures 2A-2B, and can also be applied to distributed databases, distributed applications, and other application scenarios.
- a scheduling node is applied in a distributed system.
- the distributed system includes a scheduling node and multiple computing and storage nodes.
- the index stored in the distributed system is decomposed into fragments and stored in multiple computing and storage nodes.
- the index is a multi-field data body to be computed and stored.
- the distributed system load balancing method includes:
- Step 310 In response to the index change request, perform index changes on multiple compute storage nodes.
- Step 320 Determine the first offset of each compute storage node after the index change.
- the first offset is the offset of the number of shards of a single index stored on the compute storage node relative to the average number of shards of a single index stored on all compute storage nodes.
- the second offset is the offset of the total number of shards stored on the compute storage node relative to the average total number of shards stored on all compute storage nodes.
- the third offset is the offset of the storage space utilization of the compute storage node relative to the average storage space utilization of all compute storage nodes.
- the order of determining the first offset, the second offset, and the third offset can be arbitrary, including sequential execution in a specific order or parallel execution.
- Step 330 Based on the first offset, the second offset, and the third offset, migrate the fragments on multiple computing and storage nodes so that after migration, the first offset on each computing and storage node is less than the first threshold, the second offset is less than the second threshold, and the third offset is less than the third threshold.
- step 310 in response to the index change request, index changes are performed on multiple compute storage nodes.
- the scheduling node and computing/storage node in the distributed system can be servers, wherein the scheduling node refers to a server used to schedule the shards stored by the computing/storage node, and the computing/storage node refers to a server used to store and process the shards.
- An index refers to a multi-field data body to be computed and stored. It contains a group of documents with similar characteristics, each with a unique identifier. This identifier can be used to reference the document corresponding to that identifier.
- an index can be understood as a database used to store and manage documents.
- an index can be understood as a data table containing multiple fields, and the data type of each field can be specified.
- an index is decomposed into multiple shards and stored across multiple compute storage nodes.
- a shard refers to a data partition used to store and manage a document.
- the distributed system includes indexes A, B, C, D, and E.
- Index A is decomposed into 12 shards for storage across multiple compute storage nodes, index B into 9 shards, index C into 2 shards, index D into 2 shards, and index E into 2 shards.
- compute storage node 1 stores 4 shards of index A, 3 shards of index B, 1 shard of index C, and 1 shard of index D.
- Compute storage node 2 stores 4 shards of index A, 3 shards of index B, 1 shard of index C, and 1 shard of index E.
- the compute storage node 3 stores 4 shards of index A, 3 shards of index B, 1 shard of index D, and 1 shard of index E.
- an index change request refers to a request to change the number of shards stored on multiple computing and storage nodes in a distributed system.
- An index change request can be a request to add a new index to the distributed system, a request to delete an existing index in the distributed system, or a request to migrate a shard from one computing and storage node to another.
- the scheduling node upon receiving the request, will respond by dividing the new index into multiple shards and allocating these shards to multiple compute-storage nodes.
- the scheduling node upon receiving the request, will respond by deleting the shard of that index stored in the compute-storage nodes.
- the scheduling node upon receiving the request, will respond by migrating shard H from compute-storage node A to compute-storage node B.
- an index change request will also be triggered, causing the scheduling node to respond to the index change request and migrate some shards in the computing storage node to another computing storage node.
- a first offset for each compute storage node after the index change is determined, where the first offset is the offset of the number of shards of a single index stored on the compute storage node relative to the average number of shards of a single index stored on all compute storage nodes; a second offset for each compute storage node after the index change is determined, where the second offset is the offset of the total number of shards stored on the compute storage node relative to the average total number of shards stored on all compute storage nodes; and a third offset for each compute storage node after the index change is determined, where the third offset is the offset of the storage space utilization rate of the compute storage node relative to the average storage space utilization rate of all compute storage nodes.
- the first offset refers to the offset of the number of shards of a single index stored by a compute storage node after an index change relative to the average number of shards of a single index stored by all compute storage nodes. For example, assuming index A is divided into 10 shards, compute storage node 1 in the distributed system stores 2 shards of index A, compute storage node 2 stores 3 shards of index A, compute storage node 3 stores 1 shard of index A, and compute storage node 4 stores 4 shards of index A. Calculations show that the average number of shards of index A stored by each compute storage node is 2.5. Therefore, the first offset of compute storage node 1 is -0.5, the first offset of compute storage node 2 is 0.5, the first offset of compute storage node 3 is -1.5, and the first offset of compute storage node 4 is 1.5.
- the second offset refers to the offset of the total number of shards stored on the compute storage node after the index change relative to the average total number of shards stored on all compute storage nodes.
- the distributed system includes compute storage node 1, compute storage node 2, and compute storage node 3.
- Compute storage node 1 stores a total of 1500 shards
- compute storage node 2 stores a total of 1400 shards
- compute storage node 3 stores a total of 1600 shards. Calculations show that the average total number of shards stored on all compute storage nodes is 1500. Therefore, the second offset for compute storage node 1 is 0, the second offset for compute storage node 2 is -100, and the second offset for compute storage node 3 is 100.
- the third offset refers to the offset of the storage space utilization rate of the compute storage node after an index change relative to the average storage space utilization rate of all compute storage nodes.
- the distributed system includes compute storage node 1, compute storage node 2, and compute storage node 3.
- the storage space utilization rate of compute storage node 1 is 65%, that of compute storage node 2 is 60%, and that of compute storage node 3 is 70%. Calculations show that the average storage space utilization rate of all compute storage nodes is 65%. Therefore, the third offset of compute storage node 1 is 0, the third offset of compute storage node 2 is -5%, and the third offset of compute storage node 3 is 5%.
- step 330 based on the first offset, the second offset, and the third offset, the fragments on multiple computing and storage nodes are migrated, such that after the migration, the first offset on each computing and storage node is less than the first threshold, the second offset is less than the second threshold, and the third offset is less than the third threshold.
- the first threshold refers to a critical value for an allowable first offset.
- the second threshold refers to a critical value for an allowable second offset.
- the second threshold can be set to 5% of the average total number of shards stored by all computing storage nodes.
- the third threshold refers to the critical value of the third allowable offset.
- the first threshold of 1 is based on extensive experimental and practical experience. When the offset of the number of shards for a single index stored on a compute storage node relative to the average number of shards reaches 1, it will significantly affect the system's load balancing and processing efficiency.
- the second threshold is 5% of the average total number of shards stored on all compute storage nodes. This is a comprehensive consideration of system stability and resource utilization. When the total number of shards stored on a compute storage node exceeds 5% of the average total number of shards, it may lead to excessive load on that node.
- the third threshold is 5% of the average storage space utilization of all compute storage nodes. This is to avoid excessive storage space utilization on a single compute storage node, which could affect the overall performance of the system. This value was determined after multiple tests and optimizations.
- the embodiments of this disclosure require the migration of shards across multiple computing storage nodes, so that after the migration, the first offset on each computing storage node is less than the first threshold, the second offset is less than the second threshold, and the third offset is less than the third threshold.
- the distributed system includes compute-storage node 1, compute-storage node 2, and compute-storage node 3.
- Compute-storage node 1 stores 4 shards of index A, 3 shards of index B, 1 shard of index C, and 1 shard of index D.
- Compute-storage node 2 stores 4 shards of index A, 3 shards of index B, 1 shard of index C, and 1 shard of index E.
- the first threshold is 1, if one shard of index A stored in compute-storage node 2 is migrated to compute-storage node 1 during the shard migration process, then after the migration, compute-storage node 1 will store 5 shards of index A, and compute-storage node 2 will store 3 shards of index A. This will cause the first offset of the shard of index A in compute-storage node 1 to reach the first threshold. Therefore, migrating one shard of index A stored in compute-storage node 2 to compute-storage node 1 is not feasible.
- the scheduling node receives an index change request indicating that the index in the distributed system needs to be changed.
- the scheduling node will perform the index change on multiple compute storage nodes. After the index change is performed on multiple compute storage nodes, the number of shards stored on these compute storage nodes will also change.
- the scheduling node will also determine a first offset, a second offset, and a third offset for each compute storage node after the index change.
- the first offset indicates the offset of the number of shards for a single index stored by the compute storage node relative to the average number of shards for a single index stored across all compute storage nodes.
- the second offset indicates the offset of the total number of shards stored by the compute storage node relative to the average total number of shards stored across all compute storage nodes.
- the third offset indicates the offset of the storage space utilization rate of the compute storage node relative to the average storage space utilization rate of all compute storage nodes. Therefore, the deviation between each computing storage node and other computing storage nodes in the distributed system can be determined based on the first offset, second offset, and third offset.
- shard migration across multiple computing storage nodes can be performed based on the first offset, second offset, and third offset, ensuring that the first offset of each migrated computing storage node is less than a first threshold, the second offset is less than a second threshold, and the third offset is less than a third threshold. It is understood that the embodiments of this disclosure, by making the first offset of each computing storage node less than the first threshold, achieve a balance in the dimension of the number of shards for a single index stored on the computing storage node, reducing the occurrence of some relatively large shards being concentrated on a single computing storage node, and improving the efficiency of request processing by each computing storage node in the distributed system.
- the embodiments of this disclosure can also avoid the situation where computing storage nodes reduce the efficiency of processing requests due to storing too many fragments when there is a lot of free disk space on some computing storage nodes, thereby improving the operating efficiency of the distributed system.
- step 320 has been described in sufficient detail above, the specific implementation process of steps 310 and 330 will be described in detail below.
- step 310 Detailed description of step 310
- Step 310 In response to the index change request, perform index changes on multiple compute storage nodes.
- step 310 includes:
- Step 610 Divide the target index into multiple shards
- Step 620 Distribute multiple shards across multiple compute storage nodes such that, for the target index, the first offset on each compute storage node is less than the first threshold.
- Steps 610 to 620 are described in detail below:
- step 610 the target index is divided into multiple shards.
- a target index refers to a new index added to the distributed system by an external request when the index change request becomes an index allocation request.
- the scheduling node needs to allocate and process this new index, dividing it into multiple shards and distributing them to multiple computing and storage nodes.
- an index change request can be an index allocation request for a target index.
- An index allocation request refers to an external request to add a new index in a distributed system, and the scheduling node, upon receiving the allocation request, needs to allocate the new index in response. Therefore, after receiving an index allocation request for a target index, the scheduling node will, in response, split the target index into multiple shards to facilitate subsequent allocation of these shards to multiple compute and storage nodes.
- scheduling node 2 After receiving an index allocation request for target index E, scheduling node 2 will respond by dividing target index E into 3 shards, so that target index E can be subsequently allocated to compute-storage node 1, compute-storage node 2, and compute-storage node 3.
- step 620 multiple shards are allocated across multiple compute storage nodes such that, for the target index, the first offset on each compute storage node is less than a first threshold.
- the scheduling node in order to ensure a balanced number of shards for a single index stored on computing storage nodes, the scheduling node needs to consider the number of shards allocated to each computing storage node when allocating shards for the target index, so as to ensure that the number of shards for the target index is balanced across all computing storage nodes as much as possible.
- embodiments of this disclosure when allocating multiple shards across multiple computing storage nodes, need to ensure that after the allocation of shards for the target index is completed, the first offset of the shards allocated to each computing storage node is less than a first threshold.
- scheduling node 2 After receiving an index allocation request for the target index E, scheduling node 2 will respond by dividing the target index E into 3 shards. To ensure a balanced distribution of the number of shards across computing storage nodes 1, 2, and 3, scheduling node 2 allocates one shard of the target index E to computing storage node 1, one shard to computing storage node 2, and one shard to computing storage node 3. This allocation method ensures that the first offset of the target index E allocated to each computing storage node is less than the first threshold, achieving load balancing in the distributed system.
- an index change request can be an index deletion request for the target index.
- An index deletion request refers to an external request to delete an index stored in the distributed system, and the scheduling node, upon receiving the index deletion request, needs to respond by deleting the specified index.
- the distributed system includes scheduling node 1, scheduling node 2, scheduling node 3, compute-storage node 1, compute-storage node 2, and compute-storage node 3.
- scheduling node 2 After receiving an index deletion request for index E, scheduling node 2 will respond by deleting one shard of index E stored in compute-storage node 1, one shard of index E stored in compute-storage node 2, and one shard of index E stored in compute-storage node 3.
- FIG 8B after scheduling node 2 deletes index E in the distributed system in response to the index deletion request, there are no shards of index E in compute-storage nodes 1, 2, and 3.
- steps 610 to 620 described above can, in response to an index allocation request for the target index, divide the target index into multiple shards, and take into account the number of target index shards allocated to each compute storage node when allocating the target index shards, so that after the target index shards are allocated, the first offset on each compute storage node is less than a first threshold.
- step 610 includes:
- Step 910 Obtain the storage capacity and storage space utilization of each computing storage node
- Step 920 Calculate the average storage capacity of each computing storage node based on the storage capacity of each computing storage node;
- Step 930 Calculate the average storage space utilization of each computing storage node based on the storage space utilization of each computing storage node;
- Step 940 Determine the number of shards based on the target index's index size, average storage capacity, and average storage space utilization.
- Step 950 Based on the number of shards, split the target index to obtain multiple shards.
- Steps 910 to 950 are described in detail below:
- step 910 the storage capacity and storage space utilization of each computing storage node are obtained.
- storage capacity refers to the space used by a computing storage node to store shards.
- the computing storage node uses this storage capacity to store shards for each index.
- a larger storage capacity allows the computing storage node to store more shards; conversely, a smaller storage capacity allows the computing storage node to store fewer shards.
- Storage space utilization refers to the ratio of the storage capacity occupied by the shards already stored by the computing storage node to the total storage capacity of the computing storage node. A higher storage space utilization indicates that the computing storage node has used more storage capacity; conversely, a lower storage space utilization indicates that the computing storage node has used less storage capacity.
- step 920 the average storage capacity of each computing storage node is calculated based on the storage capacity of each computing storage node.
- average storage capacity refers to the average storage capacity of the computing storage nodes in a distributed system. Assume the distributed system includes computing storage node 1, computing storage node 2, computing storage node 3, and computing storage node 4. Computing storage node 1 has a storage capacity of 15GB, computing storage node 2 has a storage capacity of 10GB, computing storage node 3 has a storage capacity of 20GB, and computing storage node 4 has a storage capacity of 15GB. Calculations show that the average storage capacity of each computing storage node in this distributed system is 15GB.
- step 930 the average storage space utilization of each computing storage node is calculated based on the storage space utilization of each computing storage node.
- average storage space utilization refers to the average storage space utilization of computing storage nodes in a distributed system. Assume the distributed system includes computing storage node 1, computing storage node 2, computing storage node 3, and computing storage node 4. The storage space utilization of computing storage node 1 is 50%, that of computing storage node 2 is 40%, that of computing storage node 3 is 44%, and that of computing storage node 4 is 46%. Calculations show that the average storage space utilization of all computing storage nodes in this distributed system is 45%.
- step 940 the number of shards is determined based on the index size, average storage capacity, and average storage space utilization of the target index.
- the index size of the target index refers to the capacity occupied by the target index
- the number of shards refers to the number of shards corresponding to the target index after the target index is split.
- the number of shards is determined based on the target index size, average storage capacity, and average storage space utilization, including: determining a first fraction based on the average storage capacity; determining a second fraction based on the average storage space utilization; determining the shard size based on the first and second fractions; and determining the number of shards based on the shard size and the target index size.
- determining the first score based on average storage capacity can be done using a lookup table.
- the table maps the average storage capacity to the first score in a pre-defined table, yielding the corresponding score.
- the first score can be determined using a substitution method.
- the average storage capacity is substituted into a pre-defined function relating average storage capacity and the first score to obtain the corresponding score.
- determining the second score based on average storage space utilization can be done using a lookup table.
- the table maps the average storage space utilization to the second score in a pre-defined table, yielding the corresponding second score.
- the second score can be determined using a substitution method.
- the average storage space utilization is substituted into a pre-defined function relating it to the second score, resulting in the second score corresponding to the average storage space utilization.
- a first score is determined based on average storage capacity. Specifically, this involves looking up a pre-set table of correspondence between average storage capacity and first score to obtain the first score corresponding to the average storage capacity. For example, when the average storage capacity is less than 10GB, the first score is 2; when the average storage capacity is between 10GB and 20GB, the first score is 5; and when the average storage capacity is greater than 20GB, the first score is 8.
- a second score is determined based on average storage space utilization.
- the partition size is determined based on the first and second scores. This can be achieved by calculating the weighted sum of the first and second scores, which serves as the basis for partitioning the partitions. For example, after determining the weighted sum of the first and second scores as the basis for partitioning the partitions, a pre-set table corresponding to the weighted sum of the first and second scores and their corresponding partition sizes can be consulted to obtain the partition size corresponding to the weighted sum of the first and second scores. Alternatively, the weighted sum of the first and second scores can be substituted into a pre-set function related to the weighted sum of the first and second scores and their corresponding partition sizes to obtain the partition size corresponding to the weighted sum of the first and second scores.
- the number of partitions is determined based on the partition size and the index size of the target index. After determining the partition size, the index size of the target index can be divided by the partition size to determine the number of partitions corresponding to the target index.
- a weighted sum of the first and second scores can be calculated as the basis for determining the fragment size.
- the sum or average of the first and second scores can also be used as the basis for determining the fragment size.
- the advantage of the weighted sum method is that by adjusting the weights assigned to the first and second scores, the influence of average storage capacity and average storage space utilization on the fragment size determination can be flexibly set.
- the slice size can be determined by substitution.
- the weighted sum of the first and second scores is substituted into a pre-set function related to the weighted sum of the first and second scores and the slice size to obtain the slice size corresponding to the weighted sum of the first and second scores.
- the number of shards is determined based on the shard size and the target index size. After determining the shard size, the target index size can be divided by the shard size to determine the number of shards corresponding to the target index.
- the first and second scores are determined using a lookup table, and the sum of the first and second scores is used as the basis for dividing the shard size.
- the first score corresponding to the average storage capacity is determined to be 8
- the second score corresponding to the average storage space utilization is 6,
- the sum of the first and second scores is 14.
- the shard size corresponding to the sum of the first and second scores can be determined to be 200MB. Assuming the target index size is 3GB, the number of shards corresponding to the target index can be determined to be 15.
- step 950 the target index is split based on the number of shards to obtain multiple shards.
- the target index in order to divide a target index into multiple shards to facilitate the subsequent allocation of the target index shards to multiple compute storage nodes, after determining the number of shards, the target index can be evenly divided into multiple shards based on the number of shards and the shard size, and the multiple shards can be allocated to each compute storage node, so as to achieve a balance in the dimension of the number of shards of a single index stored on the compute storage node when allocating shards to the compute storage node.
- steps 910 to 950 described above can determine the average storage capacity and average storage space utilization of each computing storage node based on the storage capacity and storage space utilization of each computing storage node.
- the average storage capacity of each computing storage node reflects the distributed system's ability to store indexes
- the average storage space utilization of each computing storage node reflects the proportion of the distributed system's total storage space that has been used.
- the average storage capacity and average storage space utilization of each computing storage node provide a better understanding of the current state of the distributed system.
- the number of shards can be determined based on the target index's index size, average storage capacity, and average storage space utilization.
- the target index is then evenly divided into multiple shards based on the number of shards, achieving a balance in the number of shards for a single index stored on each computing storage node when allocating shards.
- step 330 Detailed description of step 330
- Step 330 Based on the first offset, the second offset, and the third offset, migrate the fragments on multiple computing and storage nodes so that after migration, the first offset on each computing and storage node is less than the first threshold, the second offset is less than the second threshold, and the third offset is less than the third threshold.
- step 330 includes:
- Step 1110 Traverse the indexes in the distributed system
- Step 1120 For the traversed index, migrate the shards on multiple computing and storage nodes based on the first offset, the second offset, and the third offset.
- Steps 1110 to 1120 are described in detail below:
- step 1110 the indexes in the distributed system are traversed.
- "traversing the indexes in a distributed system” means using a fixed algorithm to calculate each index in the distributed system sequentially until all calculations for the distributed system are completed.
- "traversing the indexes in the distributed system” can employ a sequential traversal algorithm, where each index is calculated sequentially according to its storage order in the system. For example, if the indexes in the distributed system are stored in lexicographical order, then each index is calculated sequentially in lexicographical order until all indexes in the distributed system have been calculated. Alternatively, a random traversal algorithm can be used, where indexes in the distributed system are randomly selected for calculation until all indexes have been calculated. A hierarchical traversal algorithm can also be used; if the indexes in the distributed system have a hierarchical structure, the first-level indexes are traversed first, then the next level of indexes are traversed sequentially until all indexes have been calculated.
- a distributed system includes computing storage node 1, computing storage node 2, and computing storage node 3, and the distributed system stores multiple shards of index A, multiple shards of index B, and multiple shards of index C.
- the process of traversing the indexes of the distributed system is as follows: First, for the multiple shards of index A, computing storage node 1, computing storage node 2, and computing storage node 3 are calculated sequentially. Then, for the multiple shards of index B, computing storage node 1, computing storage node 2, and computing storage node 3 are calculated sequentially. Finally, for the multiple shards of index C, computing storage node 1, computing storage node 2, and computing storage node 3 are calculated sequentially.
- step 1120 for the traversed index, the fragments on multiple computing storage nodes are migrated based on the first offset, the second offset, and the third offset.
- each computing storage node may store a large number of indexes, with varying numbers of shards for each index. Furthermore, the shards of the indexes stored on each computing storage node are mostly different, easily leading to unbalanced loads on the computing storage nodes. Therefore, embodiments of this disclosure can sequentially traverse multiple computing storage nodes in the distributed system for each traversed index, and migrate the shards on computing storage nodes with unbalanced loads based on a first offset, a second offset, and a third offset to achieve load balancing.
- each computing storage node also has a different first offset for each index.
- the embodiments of this disclosure by sequentially traversing each compute storage node for each traversed index and migrating shards on multiple compute storage nodes based on the first offset, the second offset, and the third offset, accurately achieve a balance in the dimensions of the number of shards for a single index stored on the compute storage node, the total number of shards stored on the compute storage node, and the storage space utilization of the compute storage node.
- the distributed system includes computing storage node 1, computing storage node 2, and computing storage node 3, and stores indexes A, B, C, and D.
- index A computing storage nodes 1, 2, and 3 are traversed sequentially, and the shards of index A on multiple computing storage nodes are migrated based on the first, second, and third offsets of each computing storage node.
- index B computing storage nodes 1, 2, and 3 are traversed sequentially, and the shards of index B on multiple computing storage nodes are migrated based on the first, second, and third offsets of each computing storage node.
- index C computing storage nodes 1, 2, and 3 are traversed sequentially, and the shards of index C on multiple computing storage nodes are migrated based on the first, second, and third offsets of each computing storage node.
- index D the compute storage nodes 1, 2 and 3 are traversed in sequence, and the shards of index D on multiple compute storage nodes are migrated based on the first offset, second offset and third offset of each compute storage node.
- step 1120 includes:
- Step 1310 For the traversed indexes, perform a first sort on the multiple computational storage nodes based on the first offset, the second offset, and the third offset;
- Step 1320 Traverse multiple computing storage nodes according to the first sorting. For the traversed computing storage node, when the first offset is greater than or equal to the first threshold, migrate the fragments on the traversed computing storage node to the computing storage node with the first offset less than the traversed computing storage node.
- Step 1330 For the traversed computing storage node, if the second offset is greater than or equal to the second threshold, migrate the fragments on the traversed computing storage node to the computing storage node with the second offset less than the traversed computing storage node.
- Step 1340 For the traversed computation storage node, if the third offset is greater than or equal to the third threshold, migrate the fragments on the traversed computation storage node to the computation storage node with a third offset less than the traversed computation storage node.
- Steps 1310 to 1340 are described in detail below:
- step 1310 for the traversed indexes, the multiple computational storage nodes are sorted in the first order based on the first offset, the second offset, and the third offset.
- the first sort refers to the order obtained by sorting multiple compute storage nodes based on the first offset, second offset, and third offset for the traversed index. This order determines the traversal order of compute storage nodes during shard migration, so that compute storage nodes with unbalanced loads can be migrated shards first.
- each traversed index multiple computing and storage nodes in the distributed system need to be traversed sequentially, and shard migration on the multiple computing and storage nodes is performed based on the first offset, second offset, and third offset of each computing and storage node. It is conceivable that when traversing multiple computing and storage nodes in the distributed system, the traversal order of the computing and storage nodes determines whether that computing and storage node can be prioritized for shard migration.
- embodiments of this disclosure require sorting the multiple computing and storage nodes based on the first offset, second offset, and third offset for each traversed index.
- the first sort refers to the traversal order of the multiple computing and storage nodes in the distributed system for each traversed index.
- step 1320 multiple computing storage nodes are traversed according to the first sorting. For each traversed computing storage node, if the first offset is greater than or equal to the first threshold, the fragments on the traversed computing storage node are migrated to a computing storage node with a first offset less than the traversed computing storage node.
- embodiments of this disclosure in order to balance the dimensions of the number of shards for a single index stored by a compute storage node, the total number of shards stored by a compute storage node, and the storage space utilization of the compute storage node, embodiments of this disclosure prioritize the balancing of the number of shards for a single index stored by a compute storage node. This is because the shard sizes allocated to each compute storage node by the scheduling node in response to an index allocation request for the target index are similar. In this case, if the number of shards stored by each compute storage node is balanced, then the total number of shards stored by each compute storage node and the storage space utilization of the compute storage node will also tend to be balanced.
- the first offset of a compute storage node if the first offset of a compute storage node is less than a first threshold, it indicates that the number of shards for a single index on that compute storage node is balanced. Conversely, if the first offset is greater than or equal to the first threshold, it indicates that the number of shards for a single index on that compute storage node is unbalanced. Therefore, if the first offset of a compute storage node is greater than or equal to the first threshold, shards on that compute storage node need to be migrated to compute storage nodes with a first offset less than that node, until the first offset of the unbalanced compute storage node is less than the first threshold.
- the first offset of the compute storage node with a first offset less than that node is less than the first threshold.
- the first threshold is 1, for the traversed index A, if the first offset of computation storage node 1 is 1.5, there exists a computation storage node 2 with a first offset of 0.2. In this case, the shard of index A in computation storage node 1 can be migrated to computation storage node 2 until the first offset of computation storage node 1 for index A is less than 1.
- step 1330 for the traversed computation storage node, if the second offset is greater than or equal to the second threshold, the fragment on the traversed computation storage node is migrated to the computation storage node with the second offset less than the traversed computation storage node.
- the priority of balancing the total number of shards stored by the compute storage node can be set second, lower than the priority of balancing the number of shards for a single index stored by the compute storage node. Because the number and size of shards allocated to each compute storage node by the scheduling node in response to an index allocation request for the target index are similar, if the total number of shards stored by the compute storage node is balanced, the storage space utilization of the compute storage node will also tend to be balanced, provided that the number of shards for a single index stored by the compute storage node is balanced.
- the second offset of a compute storage node if the second offset of a compute storage node is less than the second threshold, it indicates that the total number of shards stored on that compute storage node is balanced. Conversely, if the second offset is greater than or equal to the second threshold, it indicates that the total number of shards stored on that compute storage node is unbalanced. Therefore, if the second offset of a compute storage node is greater than or equal to the second threshold, the shards on that compute storage node need to be migrated to compute storage nodes with a second offset less than that node, until the second offset of the unbalanced compute storage node is less than the second threshold.
- the second threshold is 50
- the second offset of compute storage node 1 is 100
- the shard of index A in compute storage node 1 can be migrated to compute storage node 2 until the second offset of compute storage node 1 for index A is less than 50.
- step 1340 for the traversed computation storage node, if the third offset is greater than or equal to the third threshold, the fragment on the traversed computation storage node is migrated to the computation storage node with the third offset less than the traversed computation storage node.
- the priority of balancing the storage space utilization dimension of computational storage nodes can be set last. It is understood that, for a traversed index, if the third offset of a computational storage node is less than a third threshold, it indicates that the storage space utilization of that computational storage node is in a balanced state. For a traversed index, if the third offset is greater than or equal to the third threshold, it indicates that the storage space utilization of that computational storage node is in an unbalanced state.
- the shards on that computational storage node need to be migrated to computational storage nodes with a third offset less than that of the computational storage node, until the third offset of the unbalanced computational storage node is less than the third threshold. Furthermore, when migrating shards on a traversed computational storage node to a computational storage node with a third offset less than that of the traversed computational storage node, it is also necessary to ensure that after the shard migration, the third offset of the computational storage node with a third offset less than that of the traversed computational storage node is less than the third threshold.
- the third threshold is 30, for the traversed index A, if the third offset of compute storage node 1 is 75, there exists a compute storage node 2 with a third offset of -50. In this case, the shard of index A from compute storage node 1 can be migrated to compute storage node 2 until the third offset of compute storage node 1 for index A is less than 30.
- multiple compute storage nodes are first sorted based on a first offset, a second offset, and a third offset, so that compute storage nodes with unbalanced loads can be migrated with priority. Then, the multiple compute storage nodes are traversed according to the first sort. For a traversed compute storage node, if the first offset of the traversed compute storage node reaches a first threshold, the shards on the compute storage node are migrated to a compute storage node with a first offset smaller than that compute storage node, thus accurately achieving a dimensional balance in the number of shards for a single index stored on the compute storage node.
- a traversed compute storage node For a traversed compute storage node, if the second offset of the traversed compute storage node reaches a second threshold, the shards on the traversed compute storage node are migrated to a compute storage node with a second offset smaller than that compute storage node, thus accurately achieving a dimensional balance in the total number of shards stored on the compute storage node.
- the third offset of the compute storage node reaches the third threshold, the fragments on the compute storage node are migrated to a compute storage node with a third offset smaller than that compute storage node, thus accurately achieving a dimensional balance in the storage space utilization of compute storage nodes.
- step 1310 includes:
- Step 1410 Sort the multiple computing storage nodes in descending order of the first offset
- Step 1420 For computational storage nodes with equal first offsets, sort them in descending order based on the second offset;
- Step 1430 For computational storage nodes with equal second offsets, sort them in descending order based on the third offset to form a first sort.
- Steps 1410 to 1430 are described in detail below.
- step 1410 multiple computing storage nodes are sorted in descending order of the first offset.
- multiple computational storage nodes can first be sorted in descending order based on the first offset. After sorting the multiple computational storage nodes based on the first offset, for each traversed index, the computational storage node storing the most shards for that index will be ranked first, and correspondingly, the computational storage node storing the fewest shards for that index will be ranked last.
- This sorting facilitates the migration of shards on computational storage nodes with a first offset greater than or equal to a first threshold to computational storage nodes with a first offset less than that node when traversing the first offset.
- the distributed system includes computing storage nodes A, B, C, D, E, and F.
- the first offset of computing storage node A is -0.9
- the first offset of computing storage node B is -0.3
- the first offset of computing storage node C is 0.3
- the first offset of computing storage node D is 1.5
- the first offset of computing storage node E is -0.9
- the first offset of computing storage node F is 0.3.
- step 1420 for computational storage nodes with equal first offsets, they are sorted in descending order based on the second offset.
- computing storage nodes with equal first offsets can be sorted in descending order of the second offset. Furthermore, after sorting computing storage nodes with equal first offsets in descending order of the second offset, for computing storage nodes with equal first offsets, the computing storage node with the largest total number of stored fragments will be ranked first, and correspondingly, the computing storage node with the smallest total number of stored fragments will be ranked last.
- This sorting facilitates the migration of fragments from computing storage nodes with a second offset greater than or equal to the second threshold to computing storage nodes with a second offset less than that node when traversing the current computing storage node.
- step 1430 for computational storage nodes with equal second offsets, they are sorted in descending order based on the third offset to form a first sort.
- the third offset of computational storage node A is -2%
- the third offset of computational storage node B is 0
- the third offset of computational storage node C is 1%
- the third offset of computational storage node D is 2%
- the third offset of computational storage node E is -4%
- the third offset of computational storage node F is 1%.
- the sorting order is as follows: Computational storage node D > Computational storage node C > Computational storage node F > Computational storage node B > Computational storage node A > Computational storage node E.
- steps 1410 to 1430 sort multiple computing storage nodes based on a descending order of the first offset.
- the computing storage node with the most fragments containing the traversed indexes is placed first, and the computing storage node with the fewest fragments containing the traversed indexes is placed last. This facilitates the migration of fragments from a computing storage node with a first offset greater than or equal to a first threshold to a computing storage node with a first offset less than that node when traversing that node.
- they are sorted based on a descending order of the second offset.
- the computing storage node with the largest total number of stored fragments is placed first, and the computing storage node with the fewest total number of stored fragments is placed last. This facilitates the migration of fragments from a computing storage node with a second offset greater than or equal to a second threshold to a computing storage node with a second offset less than that node when traversing that node. Furthermore, for computation storage nodes with equal second offsets, they are sorted in descending order of third offset, with the computation storage node with the highest storage space utilization placed first and the computation storage node with the lowest storage space utilization placed last. This facilitates the migration of fragments on computation storage nodes with third offsets greater than or equal to the third threshold to computation storage nodes with third offsets less than the current computation storage node when traversing the network.
- step 1310 includes:
- Step 1610 Obtain the first weight of the first offset, the second weight of the second offset, and the third weight of the third offset;
- Step 1630 Sort the multiple computing and storage nodes in the first order based on weighted sum and descending order.
- Steps 1610 to 1630 are described in detail below.
- step 1610 the first weight of the first offset, the second weight of the second offset, and the third weight of the third offset are obtained.
- a first weight can be used to indicate the importance of a first offset; the more important the first offset, the greater the first weight.
- a second weight can be used to indicate the importance of a second offset; the more important the second offset, the greater the second weight.
- a third weight can be used to indicate the importance of a third offset; the more important the third offset, the greater the third weight.
- the value ranges of the first weight, the second weight, and the third weight can be adjusted according to the actual system performance and load conditions.
- the first weight can be in the range of [50, 100], the second weight in the range of [1, 10], and the third weight in the range of [0.1, 1].
- multiple simulation tests can be conducted on the system to observe the load balancing effect and processing efficiency of the system under different weight combinations, and the optimal weight combination can be selected.
- the weight values can be determined based on the experience of the system administrator and combined with the system's historical operating data. The specific method for determining these weights through simulation testing is as follows: First, set different weight combinations in the simulation environment.
- the first weight starts at 50 and increases by 10 each time; the second weight starts at 1 and increases by 1 each time; and the third weight starts at 0.1 and increases by 0.1 each time, generating a series of weight combinations.
- simulate different load scenarios in the simulation environment such as high-concurrency requests and different index change frequencies.
- Run the system for a period of time and record the system's load balancing effect (such as the degree of balance of the first, second, and third offsets of each compute and storage node) and processing efficiency (such as the average response time of requests and throughput).
- the system's load balancing effect such as the degree of balance of the first, second, and third offsets of each compute and storage node
- processing efficiency such as the average response time of requests and throughput.
- the priority for balancing the dimension of the number of shards for a single index stored on a storage node is set first
- the priority for balancing the dimension of the total number of shards stored on a storage node is set second
- the priority for balancing the storage space utilization of a storage node is set third.
- the first weight should be set to a value greater than the second weight to indicate the importance of the first offset
- the second weight should be set to a value greater than the third weight to indicate the importance of the second offset.
- a weighted sum is calculated based on a first weight, a second weight, and a third weight.
- the resulting weighted sum of the computing and storage nodes comprehensively reflects the importance of the first, second, and third offsets, accurately representing the balance of the computing and storage nodes. Specifically, a larger weighted sum indicates that the computing and storage node needs to migrate more shards. Conversely, a smaller weighted sum indicates that the computing and storage node can accommodate more space for shards migrated from other computing and storage nodes.
- the weighted sum corresponding to computing storage node A can be determined to be -76.01. If there is a computing storage node B in the distributed system, and the first offset, second offset, and third offset of computing storage node B are -0.3, -2, and 0 respectively, the weighted sum corresponding to computing storage node B can be determined to be -17. It is conceivable that the weighted sums of computing storage nodes A and B are both less than 0, indicating that computing storage nodes A and B can also receive shards migrated from other computing storage nodes.
- step 1630 the multiple computing and storage nodes are sorted in a first order based on a weighted sum and from largest to smallest.
- a first sorting of multiple compute storage nodes based on a weighted sum and descending order ensures that the compute storage node requiring the most shards to be migrated is placed at the front, while the compute storage node capable of accommodating the most shards from other compute storage nodes is placed at the back.
- This sorting facilitates the subsequent migration of shards stored on the compute storage node requiring shard migration to a compute storage node capable of accommodating the shards from other compute storage nodes.
- steps 1610 to 1630 described above perform a weighted sum of the first offset, second offset, and third offset based on a first weight, a second weight, and a third weight.
- the resulting weighted sum of the computing and storage nodes integrates the importance of the first offset, the second offset, and the third offset, accurately reflecting the balance of the computing and storage nodes. Furthermore, it allows for a first sorting of multiple computing and storage nodes based on the weighted sum in descending order, placing the computing and storage node requiring the most shards to be migrated at the front, and placing the computing and storage node capable of accommodating the most shards migrated from other computing and storage nodes at the back.
- This sorting method facilitates the subsequent migration of shards stored on the computing and storage nodes requiring shard migration to computing and storage nodes capable of accommodating the shards migrated from other computing and storage nodes.
- step 1320 includes:
- Step 1810 Sort the computing and storage nodes in ascending order of the first offset
- Step 1820 Initialize the first target migration computation storage node as the first computation storage node in the second sorting
- Step 1830 On the traversed computing and storage nodes, select the first target fragment
- Step 1840 Migrate the first target fragment from the traversed computation storage node to the first target migration computation storage node. If the first offset is still greater than or equal to the first threshold after migration, update the first target migration computation storage node with the next computation storage node in the second sorting, and return to the step of migrating the first target fragment from the traversed computation storage node to the first target migration computation storage node until the first offset is less than the first threshold after migration.
- the first target migration computing storage node refers to the computing storage node that receives the migrated fragment from the traversed computing storage node when the first offset is greater than or equal to the first threshold. It is initialized as the first computing storage node in the second sorting and is updated as needed during the migration process.
- the first target fragment refers to the fragment that needs to be migrated from the computing and storage node whose first offset reaches the first threshold. It is selected through a specific method so that its size matches the first offset, thereby improving the efficiency of fragment migration.
- Steps 1810 to 1840 are described in detail below.
- step 1810 the computing storage nodes are sorted in the second order according to the first offset from smallest to largest.
- the second sorting refers to sorting the compute storage nodes in ascending order based on the first offset, such that the node with the smallest first offset is placed first, and the node with the largest first offset is placed last.
- the second sorting refers to the order obtained after sorting the compute storage nodes in ascending order based on the first offset, such that the compute storage node with the smallest first offset is placed first, and the node with the largest first offset is placed last. This is used to determine the selection order of target compute storage nodes to be migrated out of the traversed compute storage nodes when the first offset is greater than or equal to a first threshold.
- the smallest first offset means that the compute storage node can accommodate the maximum number of shards migrated from other compute storage nodes.
- the largest first offset means that the compute storage node needs to migrate out the maximum number of shards.
- the distributed system includes computing storage nodes A, B, C, D, E, and F.
- the first offset of computing storage node A is -0.9
- the first offset of computing storage node B is -0.3
- the first offset of computing storage node C is 0.3
- the first offset of computing storage node D is 1.5
- the first offset of computing storage node E is -0.8
- the first offset of computing storage node F is 0.5.
- the second sort is: computing storage node A, computing storage node E, computing storage node B, computing storage node C, computing storage node F, and computing storage node D.
- step 1820 the first target migration compute storage node is initialized as the first compute storage node in the second sort.
- the first target migration compute storage node refers to a compute storage node that receives the fragments migrated from compute storage nodes whose traversed first offset reaches a first threshold. It is understood that the first compute storage node in the second sort can accommodate the maximum number of fragments migrated from other compute storage nodes. Determining the first target migration compute storage node as the first compute storage node in the second sort allows it to receive as many fragments as possible. Furthermore, when determining the first target migration compute storage node, it is necessary to ensure that after receiving the fragments migrated from the traversed compute storage nodes, the first offset of the first target migration compute storage node remains less than the first threshold.
- the first target migration compute storage node can be initialized to compute storage node A.
- step 1830 the first target shard is selected on the traversed compute storage nodes.
- the first target shard refers to the shard that needs to be migrated from the computational storage node whose first offset reaches a first threshold.
- a computational storage node may store multiple shards of that index. Different shards have different loads, and the loads of shards also affect the first offset of the computational storage node. Storing shards with higher loads may result in a larger first offset for the computational storage node; naturally, migrating this shard out of the computational storage node will significantly reduce the first offset of the computational storage node. Therefore, when migrating shards in computational storage nodes whose first offset reaches the first threshold, it is first necessary to select a first target shard of appropriate size.
- step 1840 the first target fragment is migrated from the traversed computation storage node to the first target migration computation storage node. If the first offset is still greater than or equal to the first threshold after migration, the first target migration computation storage node is updated with the next computation storage node in the second sort. The step of migrating the first target fragment from the traversed computation storage node to the first target migration computation storage node is returned until the first offset is less than the first threshold after migration.
- the first target fragment can be migrated from the traversed compute storage node to the first target migration compute storage node.
- the first offset of the traversed compute storage node D reaches a first threshold
- the first target migration compute storage node has been determined to be compute storage node A
- the first target fragment is a fragment of index A
- FIG20B is a schematic diagram of compute storage node A and compute storage node D after the fragment migration is completed. It can be seen that at this time, compute storage node D no longer stores the first target fragment, while compute storage node A stores the first target fragment.
- the first offsets of both the traversed compute-storage node and the first target migration compute-storage node will change.
- the first offset of the traversed compute-storage node after migration is still greater than or equal to the first threshold, it indicates that the traversed compute-storage node needs to select another first target fragment for migration.
- the first target migration compute-storage node also needs to be updated along with the first target fragment.
- both the first target fragment and the first target migration compute-storage node need to be updated and selected.
- the next compute-storage node before the second sorting relay update is generally determined as the updated first target migration compute-storage node.
- the size of the updated first target fragment is similar to the size of the first target fragment before the update.
- the first target shard and the first target migration compute storage node after updating the first target shard and the first target migration compute storage node, it is necessary to return to the step of migrating the first target shard from the traversed compute storage node to the first target migration compute storage node, so as to migrate the updated first target shard from the traversed compute storage node to the updated first target migration compute storage node, until the first offset of the traversed compute storage node after migration is less than a first threshold.
- a first threshold when the first offset of the traversed compute storage node is less than the first threshold, it indicates that the compute storage node has achieved dimensional balance in the number of shards for that index.
- the fragment on the traversed computing storage node is migrated to a computing storage node with a first offset less than the traversed computing storage node.
- the method further includes: when the first target migration computing storage node is the last computing storage node in the second sort, and the first offset is still greater than or equal to the first threshold after migration, the first target fragment is reselected from the fragments that have not been selected on the traversed computing storage node, and the first target migration computing storage node is updated with the first computing storage node in the second sort, and the step of migrating the first target fragment from the traversed computing storage node to the first target migration computing storage node is returned.
- the first target migration compute storage node is the last compute storage node in the second sort, and after migrating the first target shard from the traversed compute storage node to the first target migration compute storage node, the first offset of the traversed compute storage node is still greater than or equal to the first threshold, it indicates that the compute storage node has not yet achieved dimensional balance in the number of index shards. The most likely reason is that the size of the first target shard is too small. Even if multiple shard migration operations are performed on the traversed compute storage node, the first offset of the traversed compute storage node will still be greater than or equal to the first threshold if the first target shard size is too small. Therefore, it is necessary to reselect a larger first target shard on the traversed compute storage node and update the second sort of each compute storage node, using the first compute storage node in the updated second sort to update the first target migration compute storage node.
- the first target migration compute-storage node needs to be updated with the next compute-storage node in the updated second sort.
- the operations of shard migration and updating the first target migration compute-storage node and the first target shard are performed cyclically until the first offset of the traversed compute-storage node is less than the first threshold.
- the first target fragment needs to be reselected only if the first target migration computing storage node is the last computing storage node in the second sort, and the first offset of the traversed computing storage node after migration is still greater than or equal to the first threshold.
- the first offset of the traversed compute storage node D reaches a first threshold
- the second sorting is compute storage node A, compute storage node B, and compute storage node C.
- the first target fragment is first migrated from compute storage node D to compute storage node A. If the first offset of compute storage node D after migration is still greater than or equal to the first threshold, the first target migration compute storage node is updated with compute storage node B. Then, the first target fragment is updated, and the first target fragment is migrated from compute storage node D to compute storage node B.
- the first target migration compute storage node is updated with compute storage node C. After this, the first target fragment is updated, and the first target fragment is migrated from compute storage node D to compute storage node C. If the first offset of compute storage node D after migration is still greater than or equal to the first threshold, the first target fragment is reselected from the fragments not selected on compute storage node D, and the reselected first target fragment is 20MB. Assuming the updated second sorting remains compute storage node A, compute storage node B, and compute storage node C, the newly selected first target shard can be migrated from compute storage node D to compute storage node A. This migration process, along with updating the first target shard's compute storage node and the first target shard, is repeated until the first offset of compute storage node D is less than a first threshold.
- steps 1810 to 1840 described above by initializing the first target migration compute storage node as the first compute storage node in the second sort, enable the first target migration compute storage node to receive as many shards as possible. Furthermore, by selecting a suitable first target shard and migrating it from the traversed compute storage nodes to the first target migration compute storage node, a precise balance is achieved in the dimension of the number of shards for a single index stored on the compute storage node.
- the embodiments of this disclosure will use the next compute storage node of the first target migration compute storage node before the second sort relay update as the updated first target migration compute storage node to perform shard migration, and repeat this step until the first offset is less than the first threshold after migration.
- step 1830 includes:
- Step 2110 Obtain the size of each fragment on the traversed computing storage node
- Step 2120 Determine the average fragment size based on the fragment size of each fragment
- Step 2130 Determine the first multiple of the size of each fragment relative to the average fragment size
- Step 2140 Select the fragment that matches the first multiple and the first offset as the first target fragment.
- step 2110 the size of each shard on the traversed computing storage node is obtained.
- the shards in those nodes need to be migrated out.
- the computational storage node stores multiple shards of different sizes. To accurately determine which shards to migrate, the shard size of each shard on the traversed computational storage node must first be obtained.
- each compute storage node stores the fragment size of that index's fragments.
- the scheduling node can directly send a request to the compute storage node to obtain the fragment size of each fragment on that compute storage node.
- the request can take the form of a specific message format, such as including the request type (identified as a request to retrieve shard sizes), the index identifier, and the identifier of the scheduling node.
- the compute and storage node parses the request content, looks up the shard sizes of each shard under that index based on the index identifier, and encapsulates this shard size information according to a pre-agreed response format, such as JSON, containing the index identifier and a list of shard sizes for each shard.
- the response message is then returned to the scheduling node.
- the fragment sizes of each fragment of index A on compute storage node D are 50MB, 53MB, 60MB and 55MB respectively.
- step 2120 the average fragment size is determined based on the fragment size of each fragment.
- the average fragment size is the average of the sizes of the individual fragments.
- step 2130 the first multiple of the fragment size of each fragment relative to the average fragment size is determined.
- the first multiple refers to the quotient of the fragment size divided by the average fragment size.
- each fragment of index A on compute storage node D is 50MB, 53MB, 60MB, and 55MB, respectively, and the average fragment size of index A stored on compute storage node D is 54.5MB. Based on this, we can determine that the first multiples of each fragment size relative to the average fragment size are 100/109, 106/109, 120/109, and 110/109, respectively.
- step 2140 the fragment that matches the first multiple and the first offset is selected as the first target fragment.
- the first offset is too large (i.e., a large number of shards need to be migrated), but the first multiple is too small (i.e., the shard is too small).
- the first offset of the traversed compute storage node is still greater than or equal to the first threshold, reducing the efficiency of shard migration.
- a shard whose first multiple matches the first offset can be selected as the first target shard to reduce the number of shard migrations performed, improve the efficiency of shard migration, and also increase the speed of achieving dimensional balance in the number of shards for a single index of the compute storage node.
- the first offset and the first multiple are directly proportional. Based on this, a function for the first offset and the first multiple can be constructed in advance according to the actual operation of the computing and storage nodes. After determining the first offset, the first offset is substituted into the function for the first offset and the first multiple to determine the first multiple that matches the first offset. Then, the corresponding shard can be selected based on the first multiple that matches the first offset, and this shard can be used as the first target shard.
- a first multiple matching the first offset can also be selected by looking up a table, and the corresponding fragment can be selected based on the first multiple matching the first offset.
- the first multiple corresponding to the first offset can be obtained. Based on this, the fragment matching the first multiple and the first offset can be found quickly.
- the first offset and the first multiple can be directly proportional.
- the first offset is substituted into this function to determine the first multiple that matches the first offset.
- the corresponding shard can be selected based on the first multiple that matches the first offset, and this shard can be used as the first target shard.
- a lookup table can be used to select the first multiple that matches the first offset.
- steps 2110 to 2140 described above can determine the average fragment size based on the fragment size of each fragment on the traversed compute storage node, and determine a first multiple of the fragment size of each fragment relative to the average fragment size.
- the number of fragment migrations can be reduced, the efficiency of fragment migration can be improved, and the speed of achieving dimensional balance of the number of fragments for a single index on the compute storage node can also be improved.
- step 1320 includes:
- Step 2310 Obtain the size of each fragment on the traversed computing storage node
- Step 2320 Determine the average fragment size based on the fragment size of each fragment
- Step 2330 Determine the second multiple of the target fragment size in each fragment relative to the average fragment size
- Step 2340 Calculate the first difference between the first offset of the traversed computation storage node and the first offset of each computation storage node to be examined.
- the computation storage node to be examined is the computation storage node whose first offset is less than that of the traversed computation storage node.
- Step 2350 Select other computing storage nodes that match the second multiple with the first difference as target computing storage nodes, and migrate the target fragment to the target computing storage node.
- Steps 2310 to 2350 are described in detail below:
- step 2110 has already explained the method of "obtaining the fragment size of each fragment on the traversed computing storage node", and will not be repeated here.
- step 2320 the average fragment size is determined based on the fragment size of each fragment.
- step 2120 has already explained the method of "determining the average fragment size based on the fragment size of each fragment", and will not be repeated here.
- step 2330 the size of the target fragment in each fragment is determined as a second multiple of the average fragment size.
- a target fragment refers to a fragment that needs to be migrated from a computing storage node whose first offset reaches a first threshold.
- the target fragment can be randomly selected from the fragments storing that index on the traversed computing storage nodes, or it can be determined by a preset method. The specific method for "determining the target fragment" will be described in detail below.
- the second multiple refers to the quotient of the target fragment's fragment size divided by the average fragment size.
- step 2130 above is similar to the implementation method of "determining the second multiple of the target fragment size in each fragment relative to the average fragment size", and will not be elaborated here.
- step 2340 the first difference between the first offset of the traversed computational storage node and the first offset of each computational storage node to be examined is calculated.
- the computational storage node to be examined is the computational storage node whose first offset is less than that of the traversed computational storage node.
- this compute storage node can be defined as a compute storage node to be examined.
- a compute storage node to be examined refers to a compute storage node whose first offset is less than the traversed compute storage node when the first offset is greater than or equal to a first threshold. The difference between the first offset of the traversed compute storage node and the first offset of the traversed compute storage node is calculated to determine a suitable target compute storage node to receive the migrated shards.
- the first difference is the difference between the first offset of the traversed compute storage node and the first offset of each compute storage node to be examined.
- the reason for calculating the first difference between the first offset of the traversed compute storage node and the first offset of each compute storage node to be examined is that it is necessary to determine the difference between the number of shards stored by the traversed compute storage node and the number of shards stored by each compute storage node to be examined. Through this difference, the size of the shards migrated from the traversed compute storage node that the compute storage node to be examined can receive can be more clearly determined.
- computing storage nodes A, B, C, D, E, and F are all computing storage nodes to be examined.
- the first difference between the first offset of computing storage node D and the first offset of computing storage node A is -0.9
- the first difference between the first offset of computing storage node D and the first offset of computing storage node B is -0.3
- the first difference between the first offset of computing storage node D and the first offset of computing storage node C is 0.3
- the first difference between the first offset of computing storage node D and the first offset of computing storage node E is -0.9
- the first difference between the first offset of computing storage node D and the first offset of computing storage node F is 0.3.
- step 2350 other computing storage nodes that match the second multiple with the first difference are taken as target computing storage nodes, and the target fragment is migrated to the target computing storage node.
- the target computing storage node refers to other computing storage nodes whose second multiple matches the first difference when the first offset is greater than or equal to the first threshold. These nodes are then used as the nodes to receive the target fragment migration, thereby improving the efficiency of fragment migration.
- a larger second multiple indicates a larger target shard size.
- a larger first difference indicates a larger shard size that the compute-storage node under consideration can receive from the migrated compute-storage nodes.
- determining the first difference based on the second multiple can be done using a lookup table. A pre-set table of correspondence between the second multiple and the first difference is consulted to obtain the first difference corresponding to the second multiple.
- determining the first difference based on the second multiple can also be done by substituting into a formula. The second multiple is substituted into a pre-set formula for the second multiple and the first difference to obtain the first difference corresponding to the second multiple.
- determining the first difference based on the second multiple can be done using a lookup table.
- a pre-defined table showing the correspondence between the second multiple and the first difference might look like this: when the second multiple is in [1, 1.2), the corresponding first difference is in the range [1, 2); when the second multiple is in [1.2, 1.4), the corresponding first difference is in the range [2, 3); when the second multiple is in [1.4, 1.6), the corresponding first difference is in the range [3, 4), and so on.
- you can determine the first difference based on the second multiple by substituting into a formula.
- the second multiple of the target fragment size of compute storage node D relative to the average fragment size is 1.2, then according to the correspondence table between the second multiple and the first difference, the range of the first difference matching the second multiple is (2,3). Based on this, compute storage node A or compute storage node E can be used as the target compute storage node.
- steps 2310 to 2350 above calculate the average fragment size based on the fragment size of each fragment, and determine the second multiple of the target fragment size relative to the average fragment size in each fragment, thus determining the difference between the number of fragments stored in the traversed computing storage nodes and the number of fragments stored in each computing storage node to be examined. Further, by calculating the first difference between the first offset of the traversed computing storage nodes and the first offset of each computing storage node to be examined, the computing storage node whose second multiple matches the first difference is selected as the target computing storage node, and the target fragment is migrated to the target computing storage node. This reduces the number of fragment migrations and improves the efficiency of fragment migration.
- the target fragment in step 2330 is determined from the individual fragments in the following manner:
- Step 2510 Determine the storage capacity of the compute storage nodes traversed
- Step 2520 Determine the storage space utilization of the traversed computing storage nodes
- Step 2530 Obtain the size of each fragment
- Step 2540 Based on storage capacity, storage space utilization and the size of each partition, determine the target partition in each partition.
- Steps 2510 to 2540 are described in detail below:
- step 2510 the storage capacity of the compute storage nodes being traversed is determined.
- the target shards that need to be migrated from the traversed compute storage nodes can be determined by multiple factors, one of which is the storage capacity of the traversed compute storage nodes. It is conceivable that the larger the storage capacity of the traversed compute storage node, the more likely that the compute storage node is to store multiple heavily loaded shards.
- step 2520 the storage space utilization of the traversed computing storage nodes is determined.
- the remaining storage space of the traversed computing storage nodes can be determined.
- the remaining storage space of the computing storage node determines the computing storage node's ability to receive fragments in subsequent operating conditions, as well as the size of the target fragment that the computing storage node needs to migrate out.
- compute storage node 1 For example, suppose the storage capacity of compute storage node 1 that is being traversed is determined to be 900MB. If the current storage space utilization of compute storage node 1 is 80%, then the remaining storage space of compute storage node 1 is 180MB. If the current storage space utilization of compute storage node 1 is 50%, then the remaining storage space of compute storage node 1 is 450MB.
- step 2530 the fragment size of each fragment is obtained.
- the size of the remaining storage space of the computing storage node determines the size of the target fragment to be migrated, in order to accurately select the target fragment from the fragments stored in the traversed computing storage node, it is also necessary to obtain the fragment size of each fragment.
- the compute storage node 1 that is traversed stores shard 1, shard 2, and shard 3.
- the shard size of shard 1 is 10MB
- the shard size of shard 2 is 15MB
- the shard size of shard 3 is 13MB.
- the target fragment is determined in each fragment based on the storage capacity, storage space utilization, and fragment size of each fragment.
- the 80MB shard can be selected as the target shard due to the smaller remaining storage space. If the remaining storage space is 500MB, the 20MB shard can be selected as the target shard.
- the remaining storage space of the visited computing storage nodes can be determined based on their storage capacity and storage space utilization. It is conceivable that if the remaining storage space of the computing storage node is small, a target fragment with a larger fragment size can be selected to migrate out of the visited computing storage node. Conversely, if the remaining storage space of the computing storage node is large, a target fragment with a smaller fragment size can be selected to migrate out of the visited computing storage node.
- compute storage node 1 For example, suppose the storage capacity of compute storage node 1, which is being traversed, is determined to be 500MB. For index A, compute storage node 1 stores shard 1, shard 2, shard 3, and shard 4. We can obtain the shard size as follows: shard 1 is 50MB, shard 2 is 45MB, shard 3 is 47MB, and shard 4 is 52MB. If the current storage space utilization of compute storage node 1 is 80%, it means that compute storage node 1 has 100MB of remaining storage space. Shard 4 can be used as the target shard to migrate out of compute storage node 1. If the current storage space utilization of compute storage node 1 is 50%, it means that compute storage node 1 has 250MB of remaining storage space. Shard 2 can be used as the target shard to migrate out of compute storage node 1.
- steps 2510 to 2540 described above can determine the remaining storage space of the traversed computing storage nodes based on their storage capacity and storage space utilization. Furthermore, the size of the target fragment to be migrated is further determined based on the size of the remaining storage space of the computing storage node. If the remaining storage space of the computing storage node is small, a target fragment with a larger fragment size can be selected to be migrated from the traversed computing storage node. If the remaining storage space of the computing storage node is large, a target fragment with a smaller fragment size can be selected to be migrated from the traversed computing storage node. This method accurately determines the target fragment that needs to be migrated from the traversed computing storage nodes, reducing the number of fragment migration operations and improving the efficiency of fragment migration.
- step 1330 includes:
- Step 2610 Sort the computing and storage nodes in ascending order of the second offset
- Step 2620 Initialize the second target migration computation storage node as the first computation storage node in the third sorting
- Step 2630 On the traversed computing and storage nodes, select the second target fragment
- Step 2640 Migrate the second target fragment from the traversed compute storage node to the second target migration compute storage node. If the second offset is still greater than or equal to the second threshold after migration, update the second target migration compute storage node with the next migration compute storage node in the third sort. Return to the step of migrating the second target fragment from the traversed compute storage node to the second target migration compute storage node until the second offset is less than the second threshold after migration.
- the second target migration compute storage node refers to the compute storage node that receives the migrated fragment from the traversed compute storage node when the second offset is greater than or equal to the second threshold. It is initialized as the first compute storage node in the third sort and is updated as needed during the migration process.
- the second target shard refers to the shard that needs to be migrated from the computing and storage node whose second offset reaches the second threshold.
- Steps 2610 to 2640 are described in detail below:
- step 2610 the computing storage nodes are sorted in the third order according to the second offset from smallest to largest.
- the third sorting refers to sorting the computing storage nodes in ascending order based on the second offset, such that the node with the smallest second offset is placed first, and the node with the largest second offset is placed last.
- the third sorting refers to the order obtained after sorting the computing storage nodes in ascending order based on the second offset, such that the computing storage node with the smallest second offset is placed first, and the node with the largest second offset is placed last. This is used to determine the selection order of target computing storage nodes for migrating out shards from the traversed computing storage nodes when the second offset is greater than or equal to a second threshold.
- the smallest second offset means that the computing storage node currently stores the fewest shards, and can accommodate the most shards migrated from other computing storage nodes.
- the largest second offset means that the computing storage node currently stores the most shards, and needs to migrate the most shards.
- the distributed system includes computing storage nodes A, B, C, D, E, and F.
- the second offset of computing storage node A is -31
- the second offset of computing storage node B is -2
- the second offset of computing storage node C is 33
- the second offset of computing storage node D is 60
- the second offset of computing storage node E is -28
- the second offset of computing storage node F is 25.
- the third sort is: computing storage node A, computing storage node E, computing storage node B, computing storage node F, computing storage node C, and computing storage node D.
- step 2620 the second target migration compute storage node is initialized as the first compute storage node in the third sort.
- the second target migration compute storage node refers to the compute storage node that receives the fragments migrated from the compute storage node whose traversed second offset reaches the second threshold. It is understood that the first compute storage node in the third sort can accommodate the maximum number of fragments migrated from other compute storage nodes. Determining the second target migration compute storage node as the first compute storage node in the third sort allows the second target migration compute storage node to receive as many fragments as possible. Furthermore, when determining the second target migration compute storage node, it is necessary to ensure that after receiving the fragments migrated from the traversed compute storage nodes, the first offset of the second target migration compute storage node is less than the first threshold, and the second offset is still less than the second threshold.
- the second target migration compute storage node can be initialized to compute storage node A.
- step 2630 a second target shard is selected on the compute storage node that has been traversed.
- the second target shard refers to the shard that needs to be migrated from the computation storage node whose traversed second offset reaches the second threshold.
- a computation storage node stores multiple shards with different indices. The sizes of these shards are not the same, and the sizes of shards with the same index are also not identical. Storing a heavily loaded shard may cause the second offset of the computation storage node to be large; naturally, migrating this shard out of the computation storage node will significantly reduce the second offset of the computation storage node. Therefore, when migrating shards in computation storage nodes whose traversed second offset reaches the second threshold, it is first necessary to select a second target shard of appropriate size.
- step 2640 the second target fragment is migrated from the traversed compute storage node to the second target migration compute storage node. If the second offset is still greater than or equal to the second threshold after migration, the second target migration compute storage node is updated with the next migration compute storage node in the third sort. The step of migrating the second target fragment from the traversed compute storage node to the second target migration compute storage node is returned until the second offset is less than the second threshold after migration.
- the second target fragment can be migrated from the traversed compute storage node to the second target migration compute storage node.
- the second offset of the traversed compute storage node D reaches a second threshold
- the second target migration compute storage node has been determined to be compute storage node A
- the second target fragment is a fragment of index B
- a fragment of index B can be migrated from compute storage node D to compute storage node A.
- FIG28B is a schematic diagram of compute storage node A and compute storage node D after the fragment migration is completed. It can be seen that at this time, compute storage node D no longer stores the second target fragment, while compute storage node A stores the second target fragment.
- the second offsets of both the traversed and second target migration compute-storage nodes will change.
- the second offset of the traversed compute-storage node after migration is still greater than or equal to the second threshold, it indicates that the traversed compute-storage node needs to select another second target fragment for migration.
- the second target migration compute-storage node also needs to be updated along with the second target fragment.
- both the second target fragment and the second target migration compute-storage node need to be updated and selected.
- the next compute-storage node after the third sorting relay update is generally determined as the updated second target migration compute-storage node.
- the size of the updated second target fragment is similar to the size of the original second target fragment.
- the second target shard and the second target migration compute storage node after updating the second target shard and the second target migration compute storage node, it is necessary to return to the step of migrating the second target shard from the traversed compute storage node to the second target migration compute storage node, so as to migrate the updated second target shard from the traversed compute storage node to the updated second target migration compute storage node, until the second offset of the traversed compute storage node after migration is less than the second threshold.
- the second offset of the traversed compute storage node is less than the second threshold, it indicates that the compute storage node has achieved dimensional balance of the total number of shards.
- the fragment on the traversed compute storage node is migrated to a compute storage node with a second offset less than the traversed compute storage node.
- the second target migration compute storage node is the last compute storage node in the third sort, and after migrating the second target shard from the traversed compute storage node to the second target migration compute storage node, the second offset of the traversed compute storage node is still greater than or equal to the second threshold, it indicates that the compute storage node has not yet achieved dimensional balance in the total number of shards. The most likely reason is that the size of the second target shard is too small. Even if multiple shard migration operations are performed on the traversed compute storage node, the second offset of the traversed compute storage node will still be greater than or equal to the second threshold if the second target shard size is too small. Therefore, it is necessary to reselect a larger second target shard on the traversed compute storage node and update the third sort of each compute storage node, using the first compute storage node in the updated third sort to update the second target migration compute storage node.
- the second target migration compute storage node needs to be updated with the next compute storage node in the updated third sort. After this, the operations of shard migration and updating the second target migration compute storage node and the second target shard are performed cyclically until the second offset of the traversed compute storage node is less than the second threshold.
- the second target fragment needs to be reselected only if the second target migration compute storage node is the last compute storage node in the third sort, and the second offset of the compute storage node traversed after migration is still greater than or equal to the second threshold.
- the second offset of the compute storage node D reaches the second threshold, and the third order is compute storage node A, compute storage node B, compute storage node C.
- the second target fragment is first migrated from compute storage node D to compute storage node A. If the second offset of compute storage node D after migration is still greater than or equal to the second threshold, the second target migration compute storage node is updated with compute storage node B. Then, the second target fragment is updated and migrated from compute storage node D to compute storage node B. If the second offset of compute storage node D after migration is still greater than or equal to the second threshold, the second target migration compute storage node is updated with compute storage node C.
- the second target fragment is updated and migrated from compute storage node D to compute storage node C. If the second offset of compute storage node D after migration is still greater than or equal to the second threshold, the second target fragment is reselected from the fragments not selected on compute storage node D, and the reselected second target fragment is 20MB. Assuming the updated third order remains compute storage node A, compute storage node B, and compute storage node C, the newly selected second target shard can be migrated from compute storage node D to compute storage node A. This migration process, along with updating the second target shard and the compute storage node to be migrated, is repeated until the second offset of compute storage node D is less than the second threshold.
- steps 2610 to 2640 described above by initializing the second target migration computational storage node as the first computational storage node in the third sort, enable the second target migration computational storage node to receive as many fragments as possible. Furthermore, by selecting a suitable second target fragment and migrating it from the traversed computational storage node to the second target migration computational storage node, a precise balance of the dimensionality of the total number of fragments across the computational storage nodes is achieved.
- the embodiments of this disclosure will use the next computational storage node of the second target migration computational storage node before the third sort relay update as the updated second target migration computational storage node to perform fragment migration, and repeat this step until the second offset is less than the second threshold after migration.
- step 1340 includes:
- Step 2910 Sort the computing and storage nodes in ascending order of the third offset, and then perform the fourth sorting.
- Step 2920 Initialize the third target migration computation storage node as the first computation storage node in the fourth sorting
- Step 2930 On the traversed computing and storage nodes, select the third target fragment
- Step 2940 Migrate the third target fragment from the traversed computation storage node to the third target migration computation storage node. If the third offset is still greater than or equal to the third threshold after migration, update the third target migration computation storage node with the next migration computation storage node in the fourth sort. Return to the step of migrating the third target fragment from the traversed computation storage node to the third target migration computation storage node until the third offset is less than the third threshold after migration.
- the third target migration computing storage node refers to the computing storage node that receives the migrated fragment from the traversed computing storage node when the third offset is greater than or equal to the third threshold. It is initialized as the first computing storage node in the fourth sort and is updated as needed during the migration process.
- the third target shard refers to the shard that needs to be migrated from the computing and storage node whose third offset reaches the third threshold during traversal.
- Steps 2910 to 2940 are described in detail below.
- step 2910 the computing storage nodes are sorted in the fourth order according to the third offset from smallest to largest.
- the fourth sorting refers to sorting the compute storage nodes in ascending order based on the third offset, such that the node with the smallest third offset is placed first, and the node with the largest third offset is placed last.
- the fourth sorting refers to the order obtained after sorting the compute storage nodes in ascending order based on the third offset, such that the compute storage node with the smallest third offset is placed first, and the node with the largest third offset is placed last. This is used to determine the selection order of target compute storage nodes to be migrated out of the traversed compute storage nodes when the third offset is greater than or equal to a third threshold.
- the smallest third offset means that the compute storage node currently has the lowest storage space utilization, and can accommodate the most fragments migrated from other compute storage nodes.
- the largest third offset means that the compute storage node currently has the highest storage space utilization, and needs to migrate the most fragments.
- the distributed system includes computing storage nodes A, B, C, D, E, and F.
- the third offset of computing storage node A is -2%
- the third offset of computing storage node B is 0%
- the third offset of computing storage node C is 1%
- the third offset of computing storage node D is 3%
- the third offset of computing storage node E is -4%
- the third offset of computing storage node F is 2%. Sort the computing storage nodes in ascending order of their third offsets, resulting in the following fourth sort: computing storage node E, computing storage node A, computing storage node B, computing storage node C, computing storage node F, and computing storage node D.
- step 2920 the third target migration computation storage node is initialized as the first computation storage node in the fourth sort.
- the third target migration computation storage node refers to the computation storage node that receives the fragments migrated from the computation storage node whose traversed third offset reaches the third threshold. It is understood that the first computation storage node in the fourth sort can accommodate the maximum number of fragments migrated from other computation storage nodes. Determining the third target migration computation storage node as the first computation storage node in the fourth sort allows it to receive as many fragments as possible.
- the third target migration computation storage node when determining the third target migration computation storage node, it is necessary to ensure that after receiving the fragments migrated from the traversed computation storage nodes, the third target migration computation storage node's first offset is less than the first threshold, its second offset is less than the second threshold, and its third offset is still less than the third threshold.
- compute storage node E compute storage node A
- compute storage node B compute storage node C
- compute storage node F compute storage node D
- the third target migration compute storage node can be initialized to compute storage node E.
- step 2930 a third target shard is selected on the traversed compute storage nodes.
- the third target shard refers to the shard that needs to be migrated from the computation storage node whose traversed third offset reaches the third threshold.
- a computation storage node stores multiple shards with different indices. The sizes of these shards are not the same, and the sizes of shards with the same index are also not identical. Storing a heavily loaded shard may cause the third offset of the computation storage node to be large; naturally, migrating this shard out of the computation storage node will significantly reduce the third offset of the computation storage node. Therefore, when migrating shards in computation storage nodes whose traversed third offset reaches the third threshold, it is first necessary to select a third target shard of appropriate size.
- step 2940 the third target fragment is migrated from the traversed computation storage node to the third target migration computation storage node. If the third offset is still greater than or equal to the third threshold after migration, the third target migration computation storage node is updated with the next migration computation storage node in the fourth sort. The step of migrating the third target fragment from the traversed computation storage node to the third target migration computation storage node is returned until the third offset is less than the third threshold after migration.
- the third target fragment can be migrated from the traversed compute storage node to the third target migration compute storage node.
- the third target migration compute storage node For example, referring to FIG31A, assuming that for the traversed index C, the third offset of the traversed compute storage node D reaches the third threshold, and the third target migration compute storage node has been determined to be compute storage node E, and the third target fragment is a fragment of index C, then a fragment of index C can be migrated from compute storage node D to compute storage node E.
- FIG31B is a schematic diagram of compute storage node E and compute storage node D after the fragment migration is completed. It can be seen that at this time, compute storage node D no longer stores the third target fragment, while compute storage node E stores the third target fragment.
- the third offset of both the traversed compute-storage node and the third target migration compute-storage node will change.
- the third offset of the traversed compute-storage node after migration is still greater than or equal to the third threshold, it indicates that the traversed compute-storage node needs to select another third target fragment for migration.
- the third target migration compute-storage node also needs to be updated along with the third target fragment.
- both the third target fragment and the third target migration compute-storage node need to be updated and selected.
- the next compute-storage node before the fourth sorting relay update is generally determined as the updated third target migration compute-storage node.
- the size of the updated third target fragment is similar to the size of the original third target fragment.
- the third target fragment and the third target migration compute storage node after updating the third target fragment and the third target migration compute storage node, it is necessary to return to the step of migrating the third target fragment from the traversed compute storage node to the third target migration compute storage node, so as to migrate the updated third target fragment from the traversed compute storage node to the updated third target migration compute storage node, until the third offset of the traversed compute storage node after migration is less than the third threshold.
- the third offset of the traversed compute storage node is less than the third threshold, it indicates that the compute storage node has achieved a balance in the dimension of storage space utilization.
- the fragment on the traversed computing storage node is migrated to a computing storage node with a third offset less than the traversed computing storage node.
- the method further includes: if the third target migration computing storage node is the last migration computing storage node in the fourth sort, and the third offset is still greater than or equal to the third threshold after migration, the third target fragment is reselected from the fragments that have not been selected on the traversed computing storage node, and the third target migration computing storage node is updated with the first computing storage node in the fourth sort, and the step of migrating the third target fragment from the traversed computing storage node to the third target migration computing storage node is returned.
- the third target migration compute storage node is the last compute storage node in the fourth sort, and after migrating the third target fragment from the traversed compute storage node to the third target migration compute storage node, the third offset of the traversed compute storage node is still greater than or equal to the third threshold, it indicates that the compute storage node has not yet achieved a balance in storage space utilization. The most likely reason is that the size of the third target fragment is too small. Even if multiple fragment migration operations are performed on the traversed compute storage node, the third offset of the traversed compute storage node will still be greater than or equal to the third threshold if the third target fragment size is too small. Therefore, it is necessary to reselect a larger third target fragment on the traversed compute storage node and update the fourth sort of each compute storage node, using the first compute storage node in the updated fourth sort to update the third target migration compute storage node.
- the third target migration compute-storage node needs to be updated with the next compute-storage node in the updated fourth sort.
- the operations of shard migration and updating the third target migration compute-storage node and the third target shard are performed cyclically until the third offset of the traversed compute-storage node is less than the third threshold.
- the third target fragment needs to be reselected only if the third target migration compute storage node is the last compute storage node in the fourth sort, and the third offset of the compute storage node traversed after migration is still greater than or equal to the third threshold.
- the third offset of the traversed compute storage node D reaches the third threshold, and the fourth order is compute storage node A, compute storage node B, compute storage node C.
- the third target fragment is first migrated from compute storage node D to compute storage node A. If the third offset of compute storage node D after migration is still greater than or equal to the third threshold, the third target migration compute storage node is updated with compute storage node B. Then, the third target fragment is updated, and the third target fragment is migrated from compute storage node D to compute storage node B.
- the third target migration compute storage node is updated with compute storage node C. After this, the third target fragment is updated, and the third target fragment is migrated from compute storage node D to compute storage node C. If the third offset of compute storage node D after migration is still greater than or equal to the third threshold, a new third target fragment is selected from the fragments that have not been selected on compute storage node D. The newly selected third target fragment is 60MB. Assume the updated fourth order is still compute storage node A, compute storage node B, and compute storage node C.
- the newly selected third target fragment can be migrated from compute storage node D to compute storage node A, and the fragment migration and updating of the third target fragment and the compute storage node and third target fragment are performed iteratively until the third offset of compute storage node D is less than the third threshold.
- steps 2910 to 2940 described above by initializing the third target migration compute storage node as the first compute storage node in the fourth sort, enable the third target migration compute storage node to receive as many fragments as possible. Furthermore, by selecting a suitable third target fragment and migrating it from the traversed compute storage nodes to the third target migration compute storage node, a precise balance in the storage space utilization of the compute storage nodes is achieved. Moreover, if the third offset after migration is still greater than or equal to the third threshold, the embodiments of this disclosure will use the next compute storage node of the third target migration compute storage node before the fourth sort relay update as the updated third target migration compute storage node to perform fragment migration, and repeat this step until the third offset after migration is less than the third threshold. By updating the third target migration compute storage node, the situation where the third offset reaches the third threshold due to the same compute storage node continuously receiving fragments is avoided, thus precisely balancing the storage space utilization of multiple compute storage nodes.
- step 3210 in response to the index change request, index changes are performed on multiple compute storage nodes.
- the index change request is an index allocation request for the target index; in response to the index change request, performing an index change on multiple compute storage nodes includes: dividing the target index into multiple shards; and allocating the multiple shards on multiple compute storage nodes such that, for the target index, a first offset on each compute storage node is less than a first threshold.
- dividing the target index into multiple shards includes: obtaining the storage capacity and storage space utilization of each computing storage node; calculating the average storage capacity of each computing storage node based on the storage capacity of each computing storage node; calculating the average storage space utilization of each computing storage node based on the storage space utilization of each computing storage node; determining the number of shards based on the index size, average storage capacity, and average storage space utilization of the target index; and splitting the target index into multiple shards based on the number of shards.
- a first offset for each compute storage node after the index change is determined, where the first offset is the offset of the number of shards of a single index stored on the compute storage node relative to the average number of shards of a single index stored on all compute storage nodes; a second offset for each compute storage node after the index change is determined, where the second offset is the offset of the total number of shards stored on the compute storage node relative to the average total number of shards stored on all compute storage nodes; and a third offset for each compute storage node after the index change is determined, where the third offset is the offset of the storage space utilization rate of the compute storage node relative to the average storage space utilization rate of all compute storage nodes.
- step 3231 the indexes in the distributed system are traversed.
- step 3232 for the traversed indexes, the multiple computing storage nodes are sorted in the first order based on the first offset, the second offset, and the third offset.
- a first sorting is performed on a plurality of computing storage nodes based on a first offset, a second offset, and a third offset, including: sorting the plurality of computing storage nodes in descending order based on the first offset; sorting computing storage nodes with equal first offsets in descending order based on the second offset; and sorting computing storage nodes with equal second offsets in descending order based on the third offset, to form a first sort.
- a first sorting is performed on multiple computing and storage nodes based on a first offset, a second offset, and a third offset, including: obtaining a first weight of the first offset, a second weight of the second offset, and a third weight of the third offset; performing a weighted sum on the first offset, the second offset, and the third offset based on the first weight, the second weight, and the third weight, wherein the first weight is greater than the second weight, and the second weight is greater than the third weight; and performing a first sorting on the multiple computing and storage nodes based on the order of the weighted sum from largest to smallest.
- step 3233 multiple computing storage nodes are traversed according to the first sorting. For each traversed computing storage node, if the first offset is greater than or equal to the first threshold, the fragments on the traversed computing storage node are migrated to a computing storage node with a first offset less than the traversed computing storage node.
- migrating the fragment on the traversed computing storage node to a computing storage node with a first offset less than the traversed computing storage node includes: performing a second sorting of computing storage nodes in ascending order of the first offset; initializing the first target migration computing storage node as the first computing storage node in the second sort; selecting a first target fragment on the traversed computing storage node; migrating the first target fragment from the traversed computing storage node to the first target migration computing storage node; if the first offset is still greater than or equal to the first threshold after migration, updating the first target migration computing storage node with the next computing storage node in the second sort, and returning to the step of migrating the first target fragment from the traversed computing storage node to the first target migration computing storage node, until the first offset is less than the first threshold after migration.
- selecting a first target fragment on the traversed computing storage nodes includes: obtaining the fragment size of each fragment on the traversed computing storage nodes; determining an average fragment size based on the fragment size of each fragment; determining a first multiple of the fragment size of each fragment relative to the average fragment size; and selecting a fragment whose first multiple matches a first offset as the first target fragment.
- the fragment on the traversed computing storage node is migrated to a computing storage node with a first offset less than the traversed computing storage node.
- the method further includes: when the first target migration computing storage node is the last migration computing storage node in the second sort, and the first offset is still greater than or equal to the first threshold after migration, the first target fragment is reselected from the fragments that have not been selected on the traversed computing storage node, and the first target migration computing storage node is updated with the first computing storage node in the second sort, and the step of migrating the first target fragment from the traversed computing storage node to the first target migration computing storage node is returned.
- migrating the fragments on the traversed computing storage nodes to computing storage nodes with a first offset less than the traversed computing storage nodes includes: obtaining the fragment size of each fragment on the traversed computing storage nodes; determining the average fragment size based on the fragment size of each fragment; determining the second multiple of the fragment size of the target fragment in each fragment relative to the average fragment size; calculating the first difference between the first offset of the traversed computing storage nodes and the first offset of each computing storage node to be examined, wherein the computing storage nodes to be examined are computing storage nodes with a first offset less than the traversed computing storage nodes; and selecting other computing storage nodes whose second multiple matches the first difference as target computing storage nodes, and migrating the target fragments to the target computing storage nodes.
- the target fragment is determined from each fragment by: determining the storage capacity of the compute storage node to be traversed; determining the storage space utilization of the compute storage node to be traversed; obtaining the fragment size of each fragment; and determining the target fragment from each fragment based on the storage capacity, storage space utilization and fragment size of each fragment.
- step 3234 for the traversed computing storage node, if the second offset is greater than or equal to the second threshold, the fragment on the traversed computing storage node is migrated to the computing storage node with the second offset less than the traversed computing storage node.
- migrating the fragment on the traversed computing storage node to a computing storage node with a second offset less than the traversed computing storage node includes: sorting the computing storage nodes in ascending order of the second offset; initializing the second target migration computing storage node as the first computing storage node in the third sort; selecting the second target fragment on the traversed computing storage node; migrating the second target fragment from the traversed computing storage node to the second target migration computing storage node; if the second offset is still greater than or equal to the second threshold after migration, updating the second target migration computing storage node with the next migration computing storage node in the third sort, and returning to the step of migrating the second target fragment from the traversed computing storage node to the second target migration computing storage node, until the second offset is less than the second threshold after migration.
- the fragment on the traversed compute storage node is migrated to a compute storage node with a second offset less than the traversed compute storage node.
- step 3235 for the traversed computation storage node, if the third offset is greater than or equal to the third threshold, the fragment on the traversed computation storage node is migrated to the computation storage node with the third offset less than the traversed computation storage node.
- migrating the fragment on the traversed computing storage node to a computing storage node with a third offset less than the traversed computing storage node includes: sorting the computing storage nodes in ascending order of the third offset; initializing the third target migration computing storage node as the first computing storage node in the fourth sort; selecting the third target fragment on the traversed computing storage node; migrating the third target fragment from the traversed computing storage node to the third target migration computing storage node; if the third offset is still greater than or equal to the third threshold after migration, updating the third target migration computing storage node with the next migration computing storage node in the fourth sort, and returning to the step of migrating the third target fragment from the traversed computing storage node to the third target migration computing storage node, until the third offset is less than the third threshold after migration.
- the fragment on the traversed computing storage node is migrated to a computing storage node with a third offset less than the traversed computing storage node.
- the method further includes: if the third target migration computing storage node is the last migration computing storage node in the fourth sort, and the third offset is still greater than or equal to the third threshold after migration, the third target fragment is reselected from the fragments that have not been selected on the traversed computing storage node, and the third target migration computing storage node is updated with the first computing storage node in the fourth sort, and the step of migrating the third target fragment from the traversed computing storage node to the third target migration computing storage node is returned.
- steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.
- Figure 33 is a schematic diagram of the structure of a distributed system load balancing device 3300 provided in an embodiment of this disclosure.
- the distributed system load balancing device 3300 is applied to a scheduling node in a distributed system.
- the distributed system includes a scheduling node and multiple computing and storage nodes.
- the index stored in the distributed system is decomposed into fragments and stored in multiple computing and storage nodes.
- the index is a multi-field data body to be computed and stored.
- the distributed system load balancing device 3300 includes:
- the response unit 3310 is used to perform index changes on multiple compute storage nodes in response to an index change request
- the first determining unit 3320 is used to determine the first offset of each compute storage node after the index change, wherein the first offset is the offset of the number of shards of a single index stored in the compute storage node relative to the average number of shards of a single index stored in all compute storage nodes; determine the second offset of each compute storage node after the index change, wherein the second offset is the offset of the total number of shards stored in the compute storage node relative to the average total number of shards stored in all compute storage nodes; and determine the third offset of each compute storage node after the index change, wherein the third offset is the offset of the storage space utilization rate of the compute storage node relative to the average storage space utilization rate of all compute storage nodes.
- the migration unit 3330 is used to migrate fragments on multiple computing and storage nodes based on a first offset, a second offset, and a third offset, such that after migration, the first offset on each computing and storage node is less than a first threshold, the second offset is less than a second threshold, and the third offset is less than a third threshold.
- an index change request is an index allocation request for the target index
- the response unit 3310 is specifically used for:
- Multiple shards are distributed across multiple compute storage nodes such that, for the target index, the first offset on each compute storage node is less than a first threshold.
- response unit 3310 is specifically used for:
- the number of shards is determined based on the target index size, average storage capacity, and average storage space utilization.
- the target index is split into multiple shards.
- the migration unit 3330 is specifically used for:
- the shards on multiple computing and storage nodes are migrated based on the first offset, the second offset, and the third offset.
- the migration unit 3330 is specifically used for:
- the multiple computational storage nodes are sorted in the first order based on the first offset, the second offset, and the third offset;
- the first sorting multiple computing storage nodes are traversed. For the computing storage node that is traversed, if the first offset is greater than or equal to the first threshold, the fragment on the traversed computing storage node is migrated to the computing storage node whose first offset is less than the traversed computing storage node.
- the fragment on the compute storage node that has been traversed will be migrated to the compute storage node whose second offset is less than the compute storage node that has been traversed.
- the fragments on the traversed compute storage node are migrated to the compute storage node whose third offset is less than the traversed compute storage node.
- the migration unit 3330 is specifically used for:
- the multiple computing and storage nodes are sorted in descending order of their first offset.
- the migration unit 3330 is specifically used for:
- the first offset, the second offset, and the third offset are weighted and summed, where the first weight is greater than the second weight, and the second weight is greater than the third weight.
- the migration unit 3330 is specifically used for:
- the computing storage nodes are sorted in ascending order of the first offset
- the first target fragment is migrated from the traversed computation storage node to the first target migration computation storage node. If the first offset is still greater than or equal to the first threshold after migration, the first target migration computation storage node is updated with the next computation storage node in the second sort. The step of migrating the first target fragment from the traversed computation storage node to the first target migration computation storage node is returned until the first offset is less than the first threshold after migration.
- the migration unit 3330 is specifically used for:
- the migration unit 3330 is specifically used for:
- the first target migration compute storage node is the last compute storage node in the second sort, and the first offset after migration is still greater than or equal to the first threshold, the first target fragment is reselected from the fragments that have not been selected on the traversed compute storage nodes, and the first target migration compute storage node is updated with the first compute storage node in the second sort, and the step of migrating the first target fragment from the traversed compute storage node to the first target migration compute storage node is returned.
- the migration unit 3330 is specifically used for:
- the computational storage node to be examined is the computational storage node whose first offset is less than that of the traversed computational storage node.
- Other computing and storage nodes that match the second multiple with the first difference are taken as target computing and storage nodes, and the target fragments are migrated to the target computing and storage nodes.
- the migration unit 3330 is specifically used for:
- the target partition is determined in each partition.
- the migration unit 3330 is specifically used for:
- the computing storage nodes are then sorted in ascending order of the second offset
- the second target fragment is migrated from the traversed compute storage node to the second target migration compute storage node. If the second offset is still greater than or equal to the second threshold after migration, the second target migration compute storage node is updated with the next migration compute storage node in the third sort. The step of migrating the second target fragment from the traversed compute storage node to the second target migration compute storage node is returned until the second offset is less than the second threshold after migration.
- the migration unit 3330 is specifically used for:
- the second target migration compute storage node is the last compute storage node in the third sort, and the second offset after migration is still greater than or equal to the second threshold, then among the unselected fragments on the traversed compute storage nodes, the second target fragment is reselected, and the second target migration compute storage node is updated with the first compute storage node in the third sort. Then, the step of migrating the second target fragment from the traversed compute storage node to the second target migration compute storage node is returned.
- the migration unit 3330 is specifically used for:
- the computing storage nodes are sorted in ascending order of the third offset
- the migration unit 3330 is specifically used for:
- the third target migration compute storage node is the last compute storage node in the fourth sort, and the third offset after migration is still greater than or equal to the third threshold, select the third target fragment again from the fragments that have not been selected on the traversed compute storage nodes, update the third target migration compute storage node with the first compute storage node in the fourth sort, and return to the step of migrating the third target fragment from the traversed compute storage nodes to the third target migration compute storage node.
- the terminal includes: a radio frequency (RF) circuit 3410, a memory 3415, an input unit 3430, a display unit 3440, a sensor 3450, an audio circuit 3460, a wireless fidelity (WiFi) module 3470, a processor 3480, and a power supply 3490, etc.
- RF radio frequency
- the terminal structure shown in Figure 34 does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
- the RF circuit 3410 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 3480; in addition, it transmits uplink data to the base station.
- the memory 3415 can be used to store software programs and modules, and the processor 3480 executes various functional applications and data processing of the content terminal by running the software programs and modules stored in the memory 3415.
- the input unit 3430 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the content terminal.
- the input unit 3430 may include a touch panel 3431 and other input devices 3432.
- Display unit 3440 can be used to display input or provided information, as well as various menus of the content terminal.
- Display unit 3440 may include display panel 3441.
- Audio circuitry 3460, speaker 3461, and microphone 3462 provide an audio interface.
- the processor 3480 included in the terminal can execute the distributed system load balancing method of the previous embodiment.
- the terminals disclosed in this embodiment include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft.
- the embodiments of this invention can be applied to various scenarios, including but not limited to distributed applications and distributed search and analysis engines.
- FIG. 35 is a partial structural block diagram of a server implementing the distributed system load balancing method of this disclosure.
- the server can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 3522 (e.g., one or more processors) and memory 3532, and one or more storage media 3530 (e.g., one or more mass storage devices) for storing application programs 3542 or data 3544.
- the memory 3532 and storage media 3530 can be temporary or persistent storage.
- the program stored in the storage media 3530 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the server.
- the CPU 3522 may be configured to communicate with the storage media 3530 and execute the series of instruction operations in the storage media 3530 on the server.
- the server may also include one or more power supplies 3526, one or more wired or wireless network interfaces 3550, one or more input/output interfaces 3558, and/or one or more operating systems 3541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
- the central processing unit 3522 in the server can be used to execute the distributed system load balancing method of the present disclosure embodiments.
- This disclosure also provides a computer-readable storage medium for storing program code for executing the distributed system load balancing methods of the foregoing embodiments.
- This disclosure also provides a computer program product comprising a computer program.
- a processor of a computer device reads and executes the computer program, causing the computer device to perform the distributed system load balancing method described above.
- this application provides a distributed system load balancing method, apparatus, electronic device, computer-readable storage medium, and computer program product.
- the scheduling node plays a crucial coordinating role.
- the scheduling node receives an index change request, it means that the index data in the system needs to be adjusted, which may involve adding or deleting indexes, or migrating shards.
- the scheduling node will perform index changes on multiple computing and storage nodes. After the index change, the number of shards stored on each computing and storage node will change, which may disrupt the original load balancing state.
- the scheduling node determines a first offset, a second offset, and a third offset for each compute storage node.
- the first offset reflects the difference between the number of fragments for a single index stored on the compute storage node and the average number of fragments for that single index stored across all compute storage nodes.
- the second offset reflects the deviation between the total number of fragments stored on the compute storage node and the average total number of fragments stored across all compute storage nodes.
- the third offset indicates the degree of deviation between the storage space utilization of the compute storage node and the average storage space utilization of all compute storage nodes.
- the scheduling nodes perform shard migration, ensuring that after migration, the first offset of each compute-storage node is less than a first threshold, the second offset is less than a second threshold, and the third offset is less than a third threshold.
- the number of shards for a single index when the first offset is less than the first threshold, it avoids some compute-storage nodes storing a large number of shards for a particular index, resulting in a more balanced load across compute-storage nodes when processing requests related to that index. This reduces processing latency caused by excessive processing pressure on a single node and improves request processing efficiency.
- the second offset being less than the second threshold ensures overall load balance across compute-storage nodes, preventing some nodes from becoming system bottlenecks due to storing too many shards, thus improving the overall system processing capacity.
- the third offset being less than the third threshold ensures reasonable use of storage space across compute-storage nodes, preventing some nodes from overusing storage space while others have a large amount of idle space, thereby improving system resource utilization.
- this load balancing method avoids centralized storage of large shards, improves the efficiency of each computing and storage node in processing requests, and also avoids the situation where some nodes become less efficient due to storing too many shards, thus improving the operating efficiency of the distributed system.
- an index change request is an index allocation request for the target index
- the scheduling node divides the target index into multiple shards and allocates them across multiple compute and storage nodes, ensuring that the first offset on each compute and storage node for the target index is less than a first threshold.
- the allocation of the target index is a critical operation; improper allocation can lead to some compute and storage nodes bearing excessive load.
- This allocation method ensures that the load on each compute and storage node is relatively balanced when processing the target index. From a technical perspective, this allows each compute and storage node to complete computational and storage tasks related to the target index at a similar speed and efficiency, preventing the overall system's performance from being affected by an overloaded node. It also ensures a balanced dimension in the number of shards for a single index stored on each compute and storage node, achieving load balancing in the distributed system and improving system response speed and processing capacity.
- the scheduling node When dividing the target index into multiple shards, the scheduling node obtains the storage capacity and storage space utilization of each compute storage node, calculates the average storage capacity and average storage space utilization of each compute storage node, and then determines the number of shards based on the target index size, average storage capacity, and average storage space utilization. Finally, the target index is split into multiple shards based on the number of shards. Obtaining the storage capacity and storage space utilization of each compute storage node allows the scheduling node to understand the actual storage capacity and usage of each node. Calculating the average storage capacity and average storage space utilization provides a reference for determining a reasonable number of shards.
- the number of shards determined based on this information can better adapt to the actual situation of each compute storage node, allowing the shards of the target index to be more evenly distributed across the nodes. From a technical perspective, this approach provides a better understanding of the current state of the distributed system. When allocating shards to compute storage nodes, it achieves a balance in the number of shards for a single index stored on each compute storage node, improving the system's resource utilization and data processing efficiency, and avoiding problems such as excessive storage pressure on some nodes or idle resources on others due to unreasonable shard allocation.
- the scheduling node When migrating shards across multiple compute-storage nodes based on the first, second, and third offsets, the scheduling node traverses the indexes in the distributed system. For each index encountered, the scheduling node migrates the shards across multiple compute-storage nodes based on these three offsets. In a distributed system, there are numerous indexes distributed across different compute-storage nodes, and the sharding configuration of each index varies. By traversing the indexes, the scheduling node can comprehensively examine the distribution of each index across various compute-storage nodes. Shard migration based on the three offsets allows for precise adjustments based on the actual load of each compute-storage node.
- this approach precisely achieves a balance across three dimensions: the number of shards for a single index, the total number of shards, and storage space utilization on the compute-storage nodes.
- the individual index level ensures that the distribution of each index's shards across various compute and storage nodes is more reasonable, improving the processing efficiency of individual indexes.
- the total number of shards level it makes the overall load of each compute and storage node more balanced, avoiding performance degradation caused by some nodes bearing too many shards.
- the storage space utilization level it optimizes the storage space usage of each compute and storage node, improving the system's resource utilization.
- first offset For each index traversed, multiple compute storage nodes are sorted based on a first offset, a second offset, and a third offset.
- the scheduling node then traverses these nodes according to this first sort.
- the first offset of a compute storage node is greater than or equal to a first threshold, the shards on that node are migrated to compute storage nodes with a first offset less than that node.
- the purpose of this first sort is to determine the priority and order of shard migration, allowing compute storage nodes with unbalanced loads to be adjusted first.
- the first offset is greater than or equal to the first threshold, it indicates a significant deviation in the number of shards for a single index on that node, potentially affecting the processing efficiency of that index.
- Migrating the shards on that node to compute storage nodes with a first offset less than that node quickly adjusts the distribution of the index across the compute storage nodes, achieving a balance in the number of shards for a single index stored on each compute storage node. From a technical perspective, this approach can promptly identify and resolve issues related to uneven distribution of shards for a single index, improving the system's processing performance for individual indexes and reducing the impact of processing delays for individual indexes on the entire system.
- one approach is to sort them in descending order of their first offset. For compute and storage nodes with equal first offsets, they are then sorted in descending order of their second offset. Similarly, for compute and storage nodes with equal second offsets, they are sorted in descending order of their third offset, forming the initial sort.
- This sorting method fully considers the importance and priority of the three offsets. Sorting by the first offset prioritizes nodes with larger deviations in the number of shards for a single index, as this has the most direct impact on the processing efficiency of individual indexes. For nodes with equal first offsets, sorting by the second offset further adjusts the balance of the total number of shards.
- sorting by the third offset optimizes storage space utilization. From a technical perspective, this sorting method facilitates the migration of shards to suitable compute and storage nodes when different offsets reach thresholds, improving the efficiency of shard migration. It allows scheduling nodes to perform more targeted load balancing adjustments, resulting in a more balanced load across multiple dimensions and improving the overall performance and stability of the system.
- Another approach to initially sorting multiple compute-storage nodes involves obtaining the first weight of the first offset, the second weight of the second offset, and the third weight of the third offset. These three offsets are then weighted and summed, with the first weight greater than the second, and the second weight greater than the third. The compute-storage nodes are then sorted in descending order of the weighted sum.
- This method by setting different weights, more flexibly considers the impact of the three offsets on system load balancing.
- the first weight being greater than both the second and third weights highlights the importance of the number of shards for a single index in load balancing, as it directly affects the processing efficiency of a single index.
- the weighted sum approach comprehensively considers the information from the three offsets, resulting in an indicator that better reflects the overall load of the compute-storage nodes. Sorting based on the weighted sum places the compute-storage node requiring the most shards to be migrated at the top, and the compute-storage node capable of accommodating the most shards from other compute-storage nodes at the bottom. From a technical perspective, this sorting method facilitates the subsequent migration of shards stored on compute and storage nodes that need to be migrated to appropriate nodes, improving the accuracy and efficiency of shard migration and enabling more effective load balancing of the system.
- the scheduling node When the first offset is greater than or equal to the first threshold, and a shard on a visited compute-storage node is migrated to a compute-storage node with a first offset less than that node, the scheduling node will perform a second sorting of the compute-storage nodes according to the ascending order of the first offset.
- the first target compute-storage node is initialized as the first compute-storage node in the second sort.
- a first target shard is selected from the visited compute-storage nodes and migrated to the first target compute-storage node.
- the first target compute-storage node is updated with the next compute-storage node in the second sort, until the first offset is less than the first threshold after migration.
- the purpose of the second sorting is to determine the priority of the compute-storage nodes receiving shards, placing nodes with smaller first offsets at the top, allowing these nodes to receive shards preferentially, thus better balancing the number of individual index shards on each compute-storage node. Selecting an appropriate first target shard for migration can more effectively reduce the first offset of the visited compute-storage nodes.
- this approach precisely achieves a dimensional balance in the number of shards for a single index stored on the compute storage node, improving the system's processing efficiency for a single index and reducing processing latency caused by uneven distribution of shards for a single index.
- the scheduling node When selecting the first target shard on the compute and storage nodes being traversed, the scheduling node obtains the shard size of each shard on that node, determines the average shard size based on these shard sizes, and determines the first multiple of each shard size relative to the average shard size. The shard whose first multiple matches the first offset is selected as the first target shard. Obtaining the shard size of each shard allows the scheduling node to understand the actual load on each shard. Determining the average shard size and the first multiple provides a quantitative basis for selecting a suitable shard.
- this selection method reduces the number of shard migrations, improves shard migration efficiency, and also increases the speed of achieving a dimensional balance in the number of shards for a single index on the compute and storage nodes. Because selecting a shard of appropriate size for migration allows for a faster reduction of the first offset, unnecessary migration operations are reduced, and system processing efficiency is improved.
- the scheduling node When the first target compute storage node to be migrated is the last compute storage node in the second sort, and the first offset after migration is still greater than or equal to the first threshold, the scheduling node will reselect the first target shard from the shards that have not been selected on the traversed compute storage nodes, and update the first target migration compute storage node with the first compute storage node in the second sort, continuing the shard migration until the first offset after migration is less than the first threshold. This situation indicates that the previously selected first target shard may be too small to effectively reduce the first offset of the traversed compute storage nodes.
- this approach ensures a balance in the number of shards for a single index stored on the compute storage node under various conditions, improving the system's fault tolerance and stability, and avoiding load balancing adjustment failures caused by improper shard selection.
- Another way to migrate shards from traversed compute storage nodes to compute storage nodes with a first offset less than the first threshold is as follows: the scheduling node obtains the shard size of each shard on the traversed compute storage nodes, determines the average shard size based on these shard sizes, determines the second multiple of the target shard size relative to the average shard size, calculates the first difference between the first offset of the traversed compute storage nodes and the first offset of each compute storage node under consideration, and selects other compute storage nodes whose second multiple matches the first difference as target compute storage nodes, migrating the target shards to these target compute storage nodes.
- Obtaining shard sizes, determining the average shard size, and the second multiple allow for quantitative analysis of the load on each shard. Calculating the first difference reveals the difference in the number of shards per index between the traversed compute storage nodes and other nodes. Matching the second multiple with the first difference allows for the selection of the most suitable compute storage node to receive the target shards. Technically, this method reduces the number of shard migrations and improves the efficiency of shard migration. Because precise matching allows the target shard to be migrated to the most suitable node in one go, avoiding multiple migrations and adjustments, and improving the system's processing efficiency and load balancing effect.
- the target shard is determined as follows: the scheduling node determines the storage capacity and storage space utilization of the compute storage nodes it visits, obtains the shard size of each shard, and, based on the storage capacity, storage space utilization, and shard size, identifies the target shard among the various shards. If the remaining storage space of the compute storage node is small, a target shard with a larger size is selected and migrated from the visited compute storage node; conversely, if the remaining storage space of the compute storage node is large, a target shard with a smaller size is selected and migrated from the visited compute storage node. Determining the storage capacity and storage space utilization provides insight into the actual storage situation of the compute storage nodes.
- this method accurately identifies the target shard that needs to be migrated from the visited compute storage nodes, reducing the number of shard migrations and improving migration efficiency. It avoids storage space waste or uneven node load caused by inappropriately sized shards, thus improving system resource utilization and stability.
- the scheduling node When the second offset is greater than or equal to the second threshold, the scheduling node will sort the compute-storage nodes in a third order according to the second offset from smallest to largest.
- the second target migration compute-storage node will be initialized as the first compute-storage node in the third order.
- a second target fragment will be selected from the traversed compute-storage nodes and migrated from the traversed compute-storage nodes to the second target migration compute-storage node. If the second offset is still greater than or equal to the second threshold after migration, the second target migration compute-storage node will be updated with the next migration compute-storage node in the third order, until the second offset is less than the second threshold after migration.
- the purpose of the third sorting is to determine the priority of the compute-storage nodes receiving fragments, placing nodes with smaller second offsets at the top, allowing these nodes to receive fragments preferentially, thus better balancing the total number of fragments across the compute-storage nodes. Selecting appropriate second target fragments for migration can more effectively reduce the second offset of the traversed compute-storage nodes. By continuously updating the second target migration compute-storage node, the situation where the second offset of the same compute-storage node reaches the second threshold due to continuous receiving of fragments is avoided. From a technical perspective, this approach precisely balances the total number of shards across computing and storage nodes, improving the overall processing capacity of the system and preventing some nodes from becoming system bottlenecks due to handling too many shards.
- the scheduling node will reselect the second target shard from the shards that have not been selected on the traversed compute storage nodes, and update the second target migrate compute storage node with the first compute storage node in the third sorting, continuing the shard migration until the second offset after migration is less than the second threshold.
- This situation indicates that the previously selected second target shard may be too small to effectively reduce the second offset of the traversed compute storage nodes.
- this approach ensures a balance in the total number of shards on compute storage nodes under various conditions, improving the system's fault tolerance and stability, and avoiding load balancing adjustment failures caused by improper shard selection.
- the scheduling node When the third offset is greater than or equal to the third threshold, the scheduling node will sort the compute-storage nodes in a fourth order according to the third offset from smallest to largest.
- the third target migration compute-storage node will be initialized as the first compute-storage node in the fourth sort.
- a third target fragment will be selected from the traversed compute-storage nodes and migrated from the traversed compute-storage nodes to the third target migration compute-storage node. If the third offset is still greater than or equal to the third threshold after migration, the third target migration compute-storage node will be updated with the next migration compute-storage node in the fourth sort, until the third offset is less than the third threshold after migration.
- the purpose of the fourth sort is to determine the priority of the compute-storage nodes receiving fragments, placing nodes with smaller third offsets at the top, allowing these nodes to receive fragments preferentially, thereby better balancing the storage space utilization of each compute-storage node. Selecting appropriate third target fragments for migration can more effectively reduce the third offset of the traversed compute-storage nodes. By continuously updating the third target migration compute-storage node, the situation where the third offset of the same compute-storage node reaches the third threshold due to continuous receiving of fragments is avoided. From a technical perspective, this approach precisely balances the storage space utilization of computing storage nodes, improves system resource utilization, and avoids the problem of some nodes affecting system performance due to excessive use of storage space.
- the scheduling node will reselect the third target fragment from the fragments not selected on the traversed compute storage nodes, update the third target migration compute storage node with the first compute storage node in the fourth sort, and continue fragment migration until the third offset after migration is less than the third threshold.
- This situation indicates that the previously selected third target fragment may be too small to effectively reduce the third offset of the traversed compute storage nodes.
- By reselecting the third target fragment and updating the third target migration compute storage node it is possible to continue attempting to adjust the storage space utilization of that node. From a technical perspective, this approach ensures a balance in the dimension of storage space utilization of compute storage nodes under various conditions, improves the fault tolerance and stability of the system, and avoids the problem of uneven storage space usage caused by improper fragment selection.
- At least one item refers to one or more items, and “more than one item” refers to two or more items.
- “And/or” is used to describe the relationship between related content, indicating that three relationships can exist. For example, “A and/or B” can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character “/” generally indicates that the preceding and following related content are in an “or” relationship. "At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
- At least one of a, b, or c can represent: a, b, c, "a and b", “a and c", “b and c", or "a and b and c", where a, b, and c can be single or multiple.
- the disclosed systems, apparatuses, and methods can be implemented in other ways.
- the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods.
- multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
- the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
- the units described as separate components may or may not be physically separate.
- the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
- the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
- the integrated unit can be implemented in hardware or as a software functional unit.
- the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
- This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server 130, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure.
- the aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
一种分布式系统负载均衡方法,所述分布式系统负载均衡方法应用于分布式系统中的调度节点,分布式系统包括调度节点和多个计算存储节点,分布式系统中存储的索引分解成分片,存储在多个计算存储节点中,分布式系统负载均衡方法包括:响应于索引变更请求,在多个计算存储节点上执行索引变更(310);确定索引变更后每个计算存储节点的第一偏移量,第一偏移量为计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量;确定索引变更后每个计算存储节点的第二偏移量,第二偏移量为计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定索引变更后每个计算存储节点的第三偏移量,第三偏移量为计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量(320);及基于第一偏移量、第二偏移量和第三偏移量,进行多个计算存储节点上分片的迁移,使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值(330)。
Description
相关申请
本申请要求2024年07月24日申请的,申请号为202410997429.6,名称为“分布式系统负载均衡方法、装置、电子设备及介质”的中国专利申请的优先权,在此将其全文引入作为参考。
本公开涉及数据网络技术领域,特别是涉及一种分布式系统负载均衡方法、装置、电子设备及介质。
分布式系统包括调度节点和多个独立的计算存储节点。索引是待进行计算和存储的多字段数据体(如数据表)。由调度节点将索引分成分片存储在各个计算存储节点上,进行计算存储。为保障分布式系统的性能状态,当前一般采用负载均衡算法来决定每个计算存储单元所分配到的分片数量。
现有技术中有一种负载均衡算法是基于计算存储单元的磁盘使用率来为每个计算存储单元分配分片的。也就是说,在对索引的分片进行分配时,会优先将分片分配给磁盘使用率较小的计算存储单元。然而,这种方法容易导致某些比较大的分片相对集中在某个计算存储单元中,而该计算存储单元也会因为集中存储有比较大的分片而降低处理请求的效率,严重的情况下甚至会导致整个分布式系统出现故障。如何实现分布式系统的负载均衡的同时提高计算存储单元处理请求的效率,是当下亟待讨论和解决的问题。
本公开实施例提供了一种分布式系统负载均衡方法、装置、电子设备及介质。
根据本公开的一方面,提供了一种分布式系统负载均衡方法,应用于分布式系统中的调度节点,所述分布式系统包括所述调度节点和多个计算存储节点,所述分布式系统中存储的索引分解成分片,存储在所述多个计算存储节点中,所述索引是待进行计算和存储的多字段数据体,所述分布式系统负载均衡方法包括:
响应于索引变更请求,在所述多个计算存储节点上执行索引变更;
确定所述索引变更后每个所述计算存储节点的第一偏移量,所述第一偏移量为所述计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的所述单个索引的平均分片数量的偏移量;
确定所述索引变更后每个所述计算存储节点的第二偏移量,所述第二偏移量为所述计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;
确定所述索引变更后每个所述计算存储节点的第三偏移量,所述第三偏移量为所述计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量;及
基于所述第一偏移量、所述第二偏移量和所述第三偏移量,进行所述多个计算存储节点上所述分片的迁移,使得迁移后每个所述计算存储节点上的所述第一偏移量小于第一阈值,所述第二偏移量小于第二阈值,所述第三偏移量小于第三阈值。
根据本公开的一方面,提供了一种分布式系统负载均衡装置,应用于分布式系统中的调度节点,所述分布式系统包括所述调度节点和多个计算存储节点,所述分布式系统中存储的索引分解成分片,存储在所述多个计算存储节点中,所述索引是待进行计算和存储的多字段数据体,所述分布式系统负载均衡装置包括:
响应单元,用于响应于索引变更请求,在所述多个计算存储节点上执行索引变更;
第一确定单元,用于确定所述索引变更后每个所述计算存储节点的第一偏移量,所述第一偏移量为所述计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的所述单个索引的平均分片数量的偏移量;确定所述索引变更后每个所述计算存储节点的第二偏移量,所述第二偏移量为所述计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定所述索引变更后每个所述计算存储节点的第三偏移量,所述第三偏移量为所述计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量;及
迁移单元,用于基于所述第一偏移量、所述第二偏移量和所述第三偏移量,进行所述多个计算存储节点上所述分片的迁移,使得迁移后每个所述计算存储节点上的所述第一偏移量小于第一阈值,所述第二偏移量小于第二阈值,所述第三偏移量小于第三阈值。
根据本公开的一方面,提供了一种电子设备,包括存储器和处理器,所述存储器存储有计算机程序,所述处理器执行所述计算机程序时实现如上所述的分布式系统负载均衡方法。
根据本公开的一方面,提供了一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序被处理器执行时实现如上所述的分布式系统负载均衡方法。
根据本公开的一方面,提供了一种计算机程序产品,该计算机程序产品包括计算机程序,所述计算机程序被计算机设备的处理器读取并执行,使得该计算机设备执行如上所述的分布式系统负载均衡方法。
本公开的其他特征和优点将在随后的说明书中阐述,并且,部分地从说明书中变得显而易见,或者通过实施本公开而了解。本公开的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据公开的附图获得其他的附图。
图1A-图1B是根据本公开的实施例的分布式系统负载均衡方法的体系架构图;
图2A-图2B是根据本公开的实施例的分布式系统负载均衡方法的应用场景示意图;
图3是根据本公开的实施例的分布式系统负载均衡方法的总体流程图;
图4A是根据本公开的实施例的分布式系统中的各个索引的分片的示意图;
图4B是根据本公开的实施例的索引分解成分片并存储在多个计算存储节点中的示意图;
图5A是根据本公开的实施例的计算存储节点上分片的迁移的示意图;
图5B是根据本公开的实施例的执行分片迁移后计算存储节点所存储的分片的示意图;
图6是根据本公开的实施例的响应于索引变更请求在多个计算存储节点上执行索引变更的具体流程图;
图7A-图7B是根据本公开的实施例的响应于对目标索引的索引分配请求在多个计算存储节点上执行索引变更的示意图;
图8A-图8B是根据本公开的实施例的响应于对目标索引的索引删除请求在多个计算存储节点上执行索引变更的示意图;
图9是根据本公开的实施例的将目标索引分成多个分片的具体流程图;
图10是根据本公开的实施例的基于目标索引的索引大小、平均存储容量和平均存储空间利用率确定分片数目的示意图;
图11是根据本公开的实施例的基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移的具体流程图;
图12是根据本公开的实施例的基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移的示意图;
图13是根据本公开的实施例的针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移的具体流程图;
图14是根据本公开的实施例的基于第一偏移量、第二偏移量和第三偏移量对多个计算存储节点进行第一排序的具体流程图;
图15是根据本公开的实施例的基于第一偏移量、第二偏移量和第三偏移量对多个计算存储节点进行第一排序的示意图;
图16是根据本公开的实施例的基于加权和由大到小的顺序对多个计算存储节点进行第一排序的具体流程图;
图17是根据本公开的实施例的基于加权和由大到小的顺序对多个计算存储节点进行第一排序的示意图;
图18是根据本公开的实施例的将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点的具体流程图;
图19是根据本公开的实施例的初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点的示意图;
图20A-图20B是根据本公开的实施例的将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的示意图;
图21是根据本公开的实施例的在遍历到的计算存储节点上选择第一目标分片的具体流程图;
图22是根据本公开的实施例的获取遍历到的计算存储节点上各个分片的分片大小的示意图;
图23是根据本公开的实施例的将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点的具体流程图;
图24是根据本公开的实施例的将第二倍数与第一差匹配的其它计算存储节点作为目标计算存储节点的示意图;
图25是根据本公开的实施例的确定目标分片的具体流程图;
图26是根据本公开的实施例的将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点的具体流程图;
图27是根据本公开的实施例的初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点的示意图;
图28A-图28B是根据本公开的实施例的将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的示意图;
图29是根据本公开的实施例的将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点的具体流程图;
图30是根据本公开的实施例的初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点的示意图;
图31A-图31B是根据本公开的实施例的将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的示意图;
图32是根据本公开的实施例的分布式系统负载均衡方法的一个实施细节图;
图33是根据本公开的实施例的分布式系统负载均衡装置的模块框图;
图34是根据本公开的实施例执行分布式系统负载均衡方法的终端结构图;
图35是根据本公开的实施例执行分布式系统负载均衡方法的服务器结构图。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
对本公开实施例进行进一步详细说明之前,对本公开实施例中涉及的名词和术语进行说明,本公开实施例中涉及的名词和术语适用于如下的解释:
分布式系统:分布式系统由多个独立的计算存储单元组成,数据会以分布式的形式打散在每个独立的计算存储单元中,每个计算存储单元负责一个分布式请求的其中一部分数据的处理,以并行化的方式高效完成一个分布式请求。
负载均衡算法:负载均衡算法是在支持应用程序的资源池中平均分配网络流量的一种方法。现代应用程序必须同时处理数百万用户,并以快速、可靠的方式将正确的文本、视频、图像和其他数据返回给每个用户。为了处理如此高的流量,大多数应用程序都有许多资源服务器,它们之间包含很多重复数据,负载均衡算法能够确保均等使用所有资源服务器。
Elasticsearch:Elasticsearch是一款目前非常流行的开源分布式检索分析引擎。在Elasticsearch中,一个计算存储单元(Node)可以是集群的一部分,也可以是独立的。一个计算存储单元可以包含一个或多个索引(Index)。索引是Elasticsearch中一个数据存储的逻辑,它包含了一组有相似特征的文档。文档是Elasticsearch中的数据单元,且文档包含了字段和字段值。
Elasticsearch的读写速度遵循木桶原理,即某台计算存储单元如果表现不佳的话会拖慢整个请求的时延,严重时甚至出现线上故障。因此,尽可能的保持计算存储单元间负载的均衡,既能保证集群性能处于最优状态,又能最大限度利用硬件资源,保障集群稳定性。
分布式系统包括调度节点和多个独立的计算存储节点。索引是待进行计算和存储的多字段数据体(如数据表)。由调度节点将索引分成分片存储在各个计算存储节点上,进行计算存储。为保障分布式系统的性能状态,当前一般采用负载均衡算法来决定每个计算存储单元所分配到的分片数量。
现有技术中有一种负载均衡算法是基于计算存储单元的磁盘使用率来为每个计算存储单元分配分片的。也就是说,在对索引的分片进行分配时,会优先将分片分配给磁盘使用率较小的计算存储单元。然而,这种方法容易导致某些比较大的分片相对集中在某个计算存储单元中,而该计算存储单元也会因为集中存储有比较大的分片而降低处理请求的效率,严重的情况下甚至会导致整个分布式系统出现故障。如何实现分布式系统的负载均衡的同时提高计算存储单元处理请求的效率,是当下亟待讨论和解决的问题。
基于此,本公开实施例提供了一种分布式系统负载均衡方法、装置、电子设备及介质。本公开实施例提供的分布式系统负载均衡方法能够在计算存储节点执行完索引变更之后,基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移,以使得迁移后每个计算存储节点实现计算存储节点存储的单个索引的分片数量的维度的均衡、计算存储节点存储的分片总数量的维度的均衡和计算存储节点的存储空间利用率的维度的均衡,实现了分布式系统的负载均衡的同时提高了计算存储节点处理请求的效率。
本公开实施例应用的系统体系构架及场景说明
图1A-图1B是根据本公开的实施例的分布式系统负载均衡方法所应用的系统构架图。图1A包括终端140、互联网130、网关120、服务器110等。
终端140包括桌面电脑、膝上型电脑、PDA(个人数字助理)、手机、车载终端、家庭影院终端、专用终端等多种形式。另外,它可以是单台设备,也可以是多台设备组成的集合。终端140可以以有线或无线的方式与互联网130进行通信,交换数据。
服务器110是指能对终端140提供某些服务的计算机系统。相对于普通终端140来说,服务器110在稳定性、安全性、性能等方面都要求更高。服务器110可以是网络平台中的一台高性能计算机、多台高性能计算机的集群、一台高性能计算机中划出的一部分(例如虚拟机)、多台高性能计算机中划出的一部分(例如虚拟机)的组合等。在分布式系统中,多台服务器110作为计算存储单元互相连接,以利用共享的通信线路来传送数据,协调任务处理。
网关120又称网间连接器、协议转换器。网关在传输层上实现网络互连,是一种充当转换作用的计算机系统或设备。在使用不同的通信协议、数据格式或语言,甚至体系结构完全不同的两种系统之间,网关是一个翻译器。同时,网关也可以提供过滤和安全功能。终端140向服务器110发送的消息要通过网关120发送到相应的服务器110。服务器110向终端140发送的消息也要通过网关120发送到相应的终端140。
本公开实施例的分布式系统负载均衡方法的调度节点和计算存储节点可以为终端140,也可以为服务器110。即本公开实施例的分布式系统均衡方法可以完全在终端140实现,可以完全在服务器110实现,也可以一部分在终端140实现,另一部分在服务器110实现。
参照图1B,分布式系统的一个集群包括多个调度节点和多个计算存储节点,不同的调度节点之间可以实现元数据同步,以确保分布式系统中的数据保持一致。且调度节点可以将元数据或请求下发给计算存储节点,以使得计算存储节点对元数据进行存储,或是将请求转发给其他计算存储节点。
元数据同步是指分布式系统中不同调度节点之间进行的数据同步操作,目的是确保分布式系统中的数据保持一致,使得各个调度节点所掌握的系统元数据信息相同,以便准确地将元数据或请求下发给计算存储节点。
进一步的,分布式系统中的计算存储节点承载着数据的存储与计算的功能。集群中的数据以索引的形式被存储,索引是待进行计算和存储的多字段数据体,也可以将索引理解成一个数据存储的逻辑。一个索引将会被切分成多个分片,分散在分布式系统中的各个计算存储节点上存储。其中,每个索引由一个主分片和多个副本分片组成,来提高数据的可用性和性能。主分片是索引切分后的组成部分之一,每个索引由一个主分片和多个副本分片组成,主分片在数据存储和处理中起到关键作用,是数据的主要存储位置,用于提高数据的可用性和性能。副本分片是索引切分后的组成部分之一,与主分片共同构成一个索引。副本分片是主分片的复制,用于提高数据的可用性和性能,当主分片出现故障或不可用时,副本分片可以替代主分片提供数据服务。
示例性的,假设索引A被切分为分片1、分片4和分片7,索引B被切分为分片2、分片5和分片8,索引C被切分为分片3、分片6和分片9。那么在分布式系统中,计算存储节点1可以存储有分片1、分片2和分片3.计算存储节点2可以存储有分片4、分片5和分片6,计算存储节点3可以存储有分片7、分片8和分片9。
本公开实施例的分布式系统负载均衡方法可应用于分布式检索分析引擎,以提高分布式检索分析引擎处理请求的效率。本公开实施例的分布式系统负载均衡方法还可以应用于分布式应用程序,以提高分布式应用程序的数据处理能力。
图2A-图2B是根据本公开的实施例的分布式系统负载均衡方法的应用场景示意图,参照图2A,示例性的,假设用户请求上传目标索引E,分布式检索分析引擎中的调度节点在接收到上传目标索引E的请求后,将会把目标索引E分成多个分片,以分别存储在各个计算存储节点中。而且,调度节点在执行完各个计算存储节点的索引分配的操作之后,还将通过本公开实施例的分布式负载均衡方法来对各个计算存储节点中的多个分片进行负载均衡,以提高计算存储节点处理请求的效率,从而提高分布式检索分析引擎的处理请求的效率。
参照图2B,示例性的,假设用户请求删除索引A,分布式检索分析引擎中的调度节点在接收到删除索引A的请求后,将会删除每个计算存储节点已经存储的索引A的分片,直到每个计算存储节点中都不存在索引A的分片。而且,调度节点在执行完各个计算存储节点的索引删除的操作之后,还将通过本公开实施例的分布式负载均衡方法来对各个计算存储节点中的多个分片进行负载均衡,以提高分布式检索分析引擎的处理请求的效率。
应理解,以上内容仅示出了对本公开部分应用场景的说明。本公开能够应用的业务场景可以包括但不限于上述举出的具体实施例。
本公开实施例的总体说明
需要强调的是,本公开实施例能够适用于多种应用场景,例如分布式检索分析引擎、分布式应用程序等场景。相关技术中,有一种负载均衡算法是基于计算存储单元的磁盘使用率来为每个计算存储单元分配分片的。也就是说,在对索引的分片进行分配时,会优先将分片分配给磁盘使用率较小的计算存储单元。然而,这种方法容易导致某些比较大的分片相对集中在某个计算存储单元中,而该计算存储单元也会因为集中存储有比较大的分片而降低处理请求的效率,严重的情况下甚至会导致整个分布式系统出现故障。本公开的一些实施例,提供了一种分布式系统负载均衡方法、装置、电子设备及介质,它能够在计算存储节点执行完索引变更之后,基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移,以使得迁移后每个计算存储节点实现计算存储节点存储的单个索引的分片数量的维度的均衡、计算存储节点存储的分片总数量的维度的均衡和计算存储节点的存储空间利用率的维度的均衡,同时还提高了计算存储节点处理请求的效率。
负载均衡是指将分片分摊到多个计算存储节点上进行存储,以使得每个计算存储节点的负载达到均衡的状态。尽可能地保持计算存储节点间的负载的均衡,既能保证集群的性能处于最优的状态,又能最大限度地利用集群中的硬件资源,保障集群的稳定性。
根据本公开的一个实施例,提供了一种分布式系统负载均衡方法。该方法可以应用于如图2A-图2B的分布式检索分析引擎,还可以应用于分布式数据库、分布式应用程序等应用场景。
如图3所示,根据本公开的一个实施例,应用于分布式系统中的调度节点,分布式系统包括调度节点和多个计算存储节点,分布式系统中存储的索引分解成分片,存储在多个计算存储节点中,索引是待进行计算和存储的多字段数据体,分布式系统负载均衡方法包括:
步骤310、响应于索引变更请求,在多个计算存储节点上执行索引变更。
步骤320、确定索引变更后每个计算存储节点的第一偏移量,第一偏移量为计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量;确定索引变更后每个计算存储节点的第二偏移量,第二偏移量为计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定索引变更后每个计算存储节点的第三偏移量,第三偏移量为计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量。其中,确定第一偏移量、第二偏移量和确定第三偏移量的步骤的顺序可以是任意顺序,包括按照特定顺序依次执行,也可以是并行执行。
步骤330、基于第一偏移量、第二偏移量和第三偏移量,进行多个计算存储节点上分片的迁移,使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。
下面对上述步骤310-330进行简要描述。
在步骤310中,响应于索引变更请求,在多个计算存储节点上执行索引变更。
根据本公开的实施例,分布式系统中的调度节点和计算存储节点可以为服务器,其中,调度节点是指用于调度计算存储节点所存储的分片的服务器,计算存储节点是指用于对分片进行存储和处理的服务器。
索引是指待计算存储的多字段数据体,且索引包含了一组具有相似特征的文档,每个文档都有一个唯一的标识,可以使用标识来引用标识所对应的文档。示例性的:在分布式检索分析引擎中,可以将索引理解为一个数据库,该数据库用于存储和管理文档。在分布式数据库中,可以将索引理解为一个数据表格,数据表格中包含了多个字段,且可以指定各个字段的类型。
在分布式系统中,索引被分解成多个分片,存储在多个计算存储节点中,分片是指用于存储和管理文档的数据分片。示例性的,参照图4A,假设分布式系统中包括索引A、索引B、索引C、索引D和索引E,索引A被分解成12个分片以存储在多个计算存储节点中,索引B被分解成9个分片以存储在多个计算存储节点中,索引C被分解成2个分片以存储在多个计算存储节点中,索引D被分解成2个分片以存储在多个计算存储节点中,索引E被分解成2个分片以存储在多个计算存储节点中。参照图4B,计算存储节点1存储有索引A的4个分片、索引B的3个分片、索引C的1个分片和索引D的1个分片。计算存储节点2存储有索引A的4个分片、索引B的3个分片、索引C的1个分片和索引E的1个分片。计算存储节点3存储有索引A的4个分片、索引B的3个分片、索引D的1个分片和索引E的1个分片。
根据本公开的实施例,索引变更请求是指请求对分布式系统中多个计算存储节点所存储的分片的数量进行变更。索引变更请求可以为请求往分布式系统中增加新的索引,也可以为请求删除分布式系统中已经存在的索引,索引变更请求也可以为请求将某个计算存储节点中的某个分片迁移到另一个计算存储节点中。
示例性的,在索引变更请求为请求往分布式系统中增加新的索引的情况下,调度节点在接收到索引变更请求后将响应于索引变更请求,将新的索引分成多个分片并给多个计算存储节点分配新的索引的分片。在索引变更请求为请求删除分布式系统中已经存在的索引的情况下,调度节点在接收到索引变更请求后将响应于索引变更请求,对计算存储节点中存储的该索引的分片进行删除。当索引变更请求为请求将计算存储节点A中的分片H迁移到计算存储节点B中,调度节点在接收到索引变更请求后将响应于索引变更请求,将计算存储节点A中的分片H迁移到计算存储节点B中。
需要说明的是,当分布式系统中的某个计算存储节点的存储空间利用率超出了预设的水位线的情况下,也会触发索引变更请求,使得调度节点响应于索引变更请求将该计算存储节点中的部分分片迁移到另一个计算存储节点中。
在步骤320中,确定索引变更后每个计算存储节点的第一偏移量,第一偏移量为计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量;确定索引变更后每个计算存储节点的第二偏移量,第二偏移量为计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定索引变更后每个计算存储节点的第三偏移量,第三偏移量为计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量。
根据本公开的实施例,第一偏移量是指索引变更后计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量。示例性的,假设索引A被切分成10个分片,分布式系统中的计算存储节点1存储有索引A的2个分片,计算存储节点2存储有索引A的3个分片,计算存储节点3存储有索引A的1个分片,计算存储节点4存储有索引A的4个分片。通过计算可以得到,各个计算存储节点存储的索引A的平均分片数量为2.5个,由此可得计算存储节点1的第一偏移量为-0.5,计算存储节点2的第一偏移量为0.5,计算存储节点3的第一偏移量为-1.5,计算存储节点4的第一偏移量为1.5。
第二偏移量是指索引变更后计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量。示例性的,参照图4B,假设分布式系统包括计算存储节点1、计算存储节点2和计算存储节点3,计算存储节点1存储的分片总数量为1500,计算存储节点2存储的分片总数量为1400,计算存储节点3存储的分片总数量为1600。通过计算可以得到,各个计算存储节点存储的平均分片总数量为1500,由此可得计算存储节点1的第二偏移量为0,计算存储节点2的第二偏移量为-100,计算存储节点3的第二偏移量为100。
第三偏移量是指索引变更后计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量。示例性的,参照图4B,假设分布式系统包括计算存储节点1、计算存储节点2和计算存储节点3,计算存储节点1的存储空间利用率是65%,计算存储节点2的存储空间利用率是60%,计算存储节点3的存储空间利用率是70%。通过计算可以得到,各个计算存储节点的平均存储空间利用率为65%,由此可得计算存储节点1的第三偏移量为0,计算存储节点2的第三偏移量为-5%,计算存储节点3的第三偏移量为5%。
在步骤330中,基于第一偏移量、第二偏移量和第三偏移量,进行多个计算存储节点上分片的迁移,使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。
根据本公开的实施例,第一阈值是指可以允许的第一偏移量的临界值,当某个计算存储节点的某个索引的分片数量的第一偏移量达到第一阈值,说明这个计算存储节点所存储的这个索引的分片数量远远超出了分布式系统中的其它计算存储节点所存储的该索引的分片数量。此时分布式系统中并没有达到负载均衡的状态,在这种情况下这个计算存储节点的处理请求的效率将大大降低。在实际应用中,第一阈值可以取值为1。第二阈值是指可以允许的第二偏移量的临界值,当某个计算存储节点的第二偏移量达到第二阈值,说明这个计算存储节点所存储的分片总数量远远超出了分布式系统中的其它计算存储节点所存储的分片总数量。此时分布式系统中并没有达到负载均衡的状态,在这种情况下这个计算存储节点的处理请求的效率将大大降低。在实际应用中,第二阈值可以取值为各个计算存储节点存储的平均分片总数量的5%。第三阈值是指可以允许的第三偏移量的临界值,当某个计算存储节点的第三偏移量达到第三阈值,说明这个计算存储节点的存储空间利用率远远超出了分布式系统中的其它计算存储节点的存储空间利用率。此时分布式系统中并没有达到负载均衡的状态,在这种情况下这个计算存储节点的处理请求的效率将大大降低。在实际应用中,第三阈值可以取值为各个计算存储节点的平均存储空间利用率的5%。
第一阈值取值为1是基于大量实验和实际应用经验,当计算存储节点存储的单个索引的分片数量相对于平均分片数量的偏移量达到1时,会明显影响系统的负载均衡和处理效率。第二阈值取值为各个计算存储节点存储的平均分片总数量的5%,是综合考虑系统的稳定性和资源利用率,当某个计算存储节点存储的分片总数量超出平均分片总数量的5%时,可能会导致该节点负载过重。第三阈值取值为各个计算存储节点的平均存储空间利用率的5%,是为了避免某个计算存储节点的存储空间利用率过高,影响系统的整体性能,经过多次测试和优化确定了该取值。
可以理解的是,当分布式系统中存在某个计算存储节点的第一偏移量达到第一阈值,或是某个计算存储节点的第二偏移量达到第二阈值,或是某个计算存储节点的第三偏移量达到第三阈值,说明此时分布式系统并没有达到负载均衡状态。为了实现分布式系统的负载均衡,提高计算存储节点处理请求的效率,本公开的实施例需要进行多个计算存储节点上分片的迁移,以使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。
此处结合图5A和图5B对多个计算存储节点上分片的迁移过程进行说明:参照图5A,分布式系统包括计算存储节点1、计算存储节点2和计算存储节点3,计算存储节点1存储有索引A的4个分片、索引B的3个分片、索引C的1个分片和索引D的1个分片,计算存储节点2存储有索引A的4个分片、索引B的3个分片、索引C的1个分片和索引E的1个分片。假设第一阈值为1,如果在分片迁移过程中将计算存储节点2所存储的索引A的1个分片迁移到计算存储节点1,那么在迁移之后计算存储节点1将存储有索引A的5个分片,计算存储节点2将存储有索引A的3个分片,这将导致计算存储节点1的索引A的分片的第一偏移量达到第一阈值。因此,将计算存储节点2所存储的索引A的1个分片迁移到计算存储节点1是不可行的。如果在分片迁移过程中将计算存储节点2所存储的索引E的1个分片迁移到计算存储节点1,参照图5B,在迁移之后计算存储节点1将存储有索引E的1个分片,计算存储节点2没有存储有索引E的分片。此时计算存储节点1的索引E的分片的第一偏移量和计算存储节点2的索引E的分片的第一偏移量均小于第一阈值。因此,将计算存储节点2所存储的索引E的1个分片迁移到计算存储节点1是可行的。
上述步骤310至步骤330的实施例中,调度节点接收到索引变更请求说明分布式系统中的索引需要变更,在分布式系统中的索引需要变更的情况下调度节点将在多个计算存储节点上执行索引变更。而在多个计算存储节点执行索引变更之后,这些计算存储节点所存储的分片的数目也将发生变化。为了降低多个计算存储节点执行索引变更之后某个计算存储节点因为集中存储某些比较大的分片而降低处理请求的效率的情况发生,调度节点还将确定索引变更后每个计算存储节点的第一偏移量、第二偏移量和第三偏移量。其中,第一偏移量表明了计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量,第二偏移量表明了计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量,第三偏移量表明了计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量。因此,基于第一偏移量、第二偏移量和第三偏移量可以确定得到分布式系统中的每个计算存储节点和分布式系统中其它计算存储节点的偏差。进一步的,还能够基于第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上的分片的迁移,以使得迁移后的每个计算存储节点的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。可以理解的是,本公开的实施例通过使每个计算存储节点的第一偏移量小于第一阈值,实现了计算存储节点存储的单个索引的分片数量的维度的均衡,降低了某些比较大的分片相对集中地存储在某个计算存储节点的情况发生,提高了分布式系统中各个计算存储节点处理请求的效率。而且,还通过使每个计算存储节点的第二偏移量小于第二阈值,实现了计算存储节点存储的分片总数量的维度的均衡,并通过使每个计算存储节点的第三偏移量小于第三阈值,实现了计算存储节点的存储空间利用率的维度的均衡。基于此,本公开实施例还能够避免在某些计算存储节点存在大量空闲的磁盘空间的情况下存在计算存储节点因为存储过多分片而降低处理请求的效率的情况发生,提高了分布式系统的运行效率。
以上是对步骤310至步骤330的总体描述。由于上面已经对步骤320描述得足够详细,下面仅对步骤310和步骤330的具体实施过程进行详细描述。
步骤310的详细描述
步骤310、响应于索引变更请求,在多个计算存储节点上执行索引变更。
在一实施例中,索引变更请求是对目标索引的索引分配请求,参照图6,步骤310包括:
步骤610、将目标索引分成多个分片;
步骤620、将多个分片在多个计算存储节点上进行分配,使得针对目标索引,每个计算存储节点上的第一偏移量小于第一阈值。
下面对步骤610至步骤620进行详细描述:
在步骤610中,将目标索引分成多个分片。
目标索引是指在分布式系统负载均衡方法中,当索引变更请求为索引分配请求时,外部请求在分布式系统中增加的新索引,调度节点需要对其进行分配和处理,将其切分成多个分片并分配给多个计算存储节点。
根据本公开的实施例,索引变更请求可以为对目标索引的索引分配请求,索引分配请求是指外部请求在分布式系统中增加新的索引,而调度节点在接收到索引分配请求之后需要响应于索引分配请求对新的索引进行分配。基于此,调度节点在接收到对目标索引的索引分配请求之后,将响应于对目标索引的索引分配请求,把目标索引切分成多个分片,以便于后续将目标索引的分片分配给多个计算存储节点。
示例性的,参照图7A和图7B,假设分布式系统包括调度节点1、调度节点2、调度节点3、计算存储节点1、计算存储节点2和计算存储节点3。调度节点2在接收到对目标索引E的索引分配请求之后,将响应于对目标索引E的索引分配请求,将目标索引E分成3个分片,以便于后续将目标索引E分配给计算存储节点1、计算存储节点2和计算存储节点3。
“将目标索引分成多个分片”的具体方法将在下文详细描述。
在步骤620中,将多个分片在多个计算存储节点上进行分配,使得针对目标索引,每个计算存储节点上的第一偏移量小于第一阈值。
根据本公开的实施例,为了保障计算存储节点存储的单个索引的分片数量的维度的均衡,调度节点在对目标索引的分片进行分配时,需要考虑到每个计算存储节点所分配到的目标索引的分片数量,以尽可能地保证目标索引在各个计算存储节点上的分片数量是均衡的。基于此,本公开的实施例在将多个分片在多个计算存储节点上进行分配时,需要确保在完成目标索引的分片分配之后,每个计算存储节点被分配到的目标索引的分片的第一偏移量小于第一阈值。
示例性的,参照图7B,假设第一阈值为1。调度节点2在接收到对目标索引E的索引分配请求之后,将响应于对目标索引E的索引分配请求,将目标索引E分成3个分片。为了尽可能地保证目标索引在计算存储节点1、计算存储节点2和计算存储节点3上的分片数量是均衡的,调度节点2将目标索引E的1个分片分配给了计算存储节点1,将目标索引E的1个分片分配给了计算存储节点2,将目标索引E的1个分片分配给了计算存储节点3。通过这种分配方式,保障了每个计算存储节点被分配到的目标索引E的分片的第一偏移量小于第一阈值,实现了分布式系统的负载均衡。
可以理解的是,索引变更请求可以为对目标索引的索引删除请求,索引删除请求是指外部请求删除分布式系统中存储的索引,而调度节点在接收到索引删除请求之后需要响应于索引删除请求对指定的索引进行删除。示例性的,参照图8A,分布式系统包括调度节点1、调度节点2、调度节点3、计算存储节点1、计算存储节点2和计算存储节点3。调度节点2在接收到对索引E的索引删除请求之后,将响应于对索引E的索引删除请求,对计算存储节点1存储的索引E的1个分片、计算存储节点2存储的索引E的1个分片和计算存储节点3存储的索引E的1个分片进行删除。参照图8B可知,在调度节点2响应于对索引E的索引删除请求对分布式系统中的索引E进行删除之后,计算存储节点1、计算存储节点2和计算存储节点3中不存在索引E的分片。
上述步骤610至步骤620的实施例能够响应于对目标索引的索引分配请求,将目标索引分成多个分片,并在对目标索引的分片进行分配时考虑到了每个计算存储节点所分配到的目标索引的分片数量,使得目标索引的分片分配完毕之后每个计算存储节点上的第一偏移量小于第一阈值。通过确保每个计算存储节点被分配到的目标索引的分片的第一偏移量小于第一阈值,实现了计算存储节点存储的单个索引的分片数量的维度的均衡。
在一实施例中,参照图9,步骤610包括:
步骤910、获取每个计算存储节点的存储容量和存储空间利用率;
步骤920、基于每个计算存储节点的存储容量,计算各个计算存储节点的平均存储容量;
步骤930、基于每个计算存储节点的存储空间利用率,计算各个计算存储节点的平均存储空间利用率;
步骤940、基于目标索引的索引大小、平均存储容量和平均存储空间利用率,确定分片数目;
步骤950、基于分片数目,将目标索引进行切分,获得多个分片。
下面对步骤910至步骤950进行详细描述:
在步骤910中,获取每个计算存储节点的存储容量和存储空间利用率。
根据本公开的实施例,存储容量是指计算存储节点用于存储分片的空间的容量,计算存储节点使用存储容量来存储各个索引的分片。存储容量越大,该计算存储节点可以存储的分片的数量越多,对应的,存储容量越小,该计算存储节点可以存储的分片的数量越少。存储空间利用率是指计算存储节点已经存储的分片所占用的存储容量和计算存储节点的存储容量的比值。存储空间利用率越大,说明该计算存储节点已经使用的存储容量越多,对应的,存储空间利用率越小,说明该计算存储节点已经使用的存储容量越少。
在步骤920中,基于每个计算存储节点的存储容量,计算各个计算存储节点的平均存储容量。
根据本公开的实施例,平均存储容量是指分布式系统中的计算存储节点的存储容量的平均值。假设分布式系统包括计算存储节点1、计算存储节点2、计算存储节点3和计算存储节点4,计算存储节点1的存储容量为15GB,计算存储节点2的存储容量为10GB,计算存储节点3的存储容量为20GB,计算存储节点4的存储容量为15GB。通过计算可以确定,该分布式系统中各个计算存储节点的平均存储容量为15GB。
在步骤930中,基于每个计算存储节点的存储空间利用率,计算各个计算存储节点的平均存储空间利用率。
根据本公开的实施例,平均存储空间利用率是指分布式系统中的计算存储节点的存储空间利用率的平均值。假设分布式系统包括计算存储节点1、计算存储节点2、计算存储节点3和计算存储节点4,计算存储节点1的存储空间利用率为50%,计算存储节点2的存储空间利用率为40%,计算存储节点3的存储空间利用率为44%,计算存储节点4的存储空间利用率为46%。通过计算可以确定,该分布式系统中各个计算存储节点的平均存储空间利用率为45%。
在步骤940中,基于目标索引的索引大小、平均存储容量和平均存储空间利用率,确定分片数目。
根据本公开的实施例,目标索引的索引大小是指目标索引所占的容量,分片数目是指对目标索引进行切分之后,目标索引所对应的分片的数目。
可以理解的是,基于目标索引的索引大小、平均存储容量和平均存储空间利用率,确定分片数目,包括:基于平均存储容量确定第一分数;基于平均存储空间利用率确定第二分数;基于第一分数和第二分数确定分片大小;基于分片大小和目标索引的索引大小确定分片数目。
可以理解的是,基于平均存储容量确定第一分数,可以采用查表的方式。根据平均存储容量查找预先设置的平均存储容量与第一分数对应关系表,得到平均存储容量对应的第一分数。基于平均存储容量确定第一分数,还可以采用代入的方式。将平均存储容量代入预先设置的平均存储容量和第一分数相关的函数,得到平均存储容量对应的第一分数。
可以理解的是,基于平均存储空间利用率确定第二分数,可以采用查表的方式。根据平均存储空间利用率查找预先设置的平均存储空间利用率与第二分数对应关系表,得到平均存储空间利用率对应的第二分数。基于平均存储空间利用率确定第二分数,还可以采用代入的方式。将平均存储空间利用率代入预先设置的平均存储空间利用率和第二分数相关的函数,得到平均存储空间利用率对应的第二分数。
在一些实施例中,基于平均存储容量确定第一分数,具体为根据平均存储容量查找预先设置的平均存储容量与第一分数对应关系表,得到平均存储容量对应的第一分数。例如当平均存储容量小于10GB时,第一分数为2;当平均存储容量在10GB-20GB之间时,第一分数为5;当平均存储容量大于20GB时,第一分数为8。也可以将平均存储容量代入预先设置的平均存储容量和第一分数相关的函数y=ax+b(其中x代表平均存储容量,y是第一分数,a为斜率,b为截距),得到平均存储容量对应的第一分数。基于平均存储空间利用率确定第二分数,可根据平均存储空间利用率查找预先设置的平均存储空间利用率与第二分数对应关系表,得到平均存储空间利用率对应的第二分数,如当平均存储空间利用率小于30%时,第二分数为2;当平均存储空间利用率在30%-60%之间时,第二分数为5;当平均存储空间利用率大于60%时,第二分数为8。也可将平均存储空间利用率代入预先设置的平均存储空间利用率和第二分数相关的函数n=cm+d(其中m表示平均存储空间利用率,n是第二分数,c为斜率,d为截距),得到平均存储空间利用率对应的第二分数。
基于第一分数和第二分数确定分片大小,可求第一分数和第二分数的加权和,作为划分分片大小的依据,例如将第一分数和第二分数的加权和确定为划分分片大小的依据后,根据第一分数和第二分数的加权和查找预先设置的根据第一分数和第二分数的加权和与分片大小对应关系表,得到第一分数和第二分数的加权和对应的分片大小。也可将第一分数和第二分数的加权和代入预先设置的第一分数和第二分数的加权和与分片大小相关的函数,得到第一分数和第二分数的加权和对应的分片大小。基于分片大小和目标索引的索引大小确定分片数目,在确定得到分片大小之后,便可以将目标索引的索引大小除以分片大小,以确定得到目标索引所对应的分片数目。
在确定第一分数和第二分数之后,可以求第一分数和第二分数的加权和,作为划分分片大小的依据。当然,也可以把第一分数和第二分数的和或平均数,作为划分分片大小的依据。相比于求和或求平均数的方式,加权和的方式的优点在于,通过调节为第一分数和第二分数设置的权重,可以灵活设置平均存储容量和平均存储空间利用率对划分分片大小的依据的影响程度。
示例性的,假设将第一分数和第二分数的加权和确定为划分分片大小的依据,在这之后,可以采用查表的方式,来确定分片大小。根据第一分数和第二分数的加权和查找预先设置的根据第一分数和第二分数的加权和与分片大小对应关系表,得到第一分数和第二分数的加权和对应的分片大小。基于第一分数和第二分数的加权和确定分片大小,还可以采用代入的方式。将第一分数和第二分数的加权和代入预先设置的第一分数和第二分数的加权和与分片大小相关的函数,得到第一分数和第二分数的加权和对应的分片大小。
可以理解的是,基于分片大小和目标索引的索引大小确定分片数目,在确定得到分片大小之后,便可以将目标索引的索引大小除以分片大小,以确定得到目标索引所对应的分片数目。
示例性的,参照图10,假设通过采用查表的方式来确定第一分数和第二分数,将第一分数和第二分数的和作为划分分片大小的依据。在分布式系统中各个计算存储节点的平均存储容量是35GB,平均存储空间利用率是63%的情况下,可以确定得到平均存储容量所对应的第一分数是8,平均存储空间利用率所对应的第二分数是6,第一分数和第二分数的和是14。进一步的,可以确定得到第一分数和第二分数的和所对应的分片大小为200MB。假设目标索引的大小为3GB,此时可以确定得到目标索引所对应的分片数目为15。
在步骤950中,基于分片数目,将目标索引进行切分,获得多个分片。
根据本公开的实施例,为了将目标索引切分成多个分片,以便于后续将目标索引的分片分配给多个计算存储节点。在确定分片数目之后,便可以依据分片数目和分片大小,将目标索引均衡地切分成多个分片,将多个分片分配给各个计算存储节点,以在给计算存储节点分配分片时实现计算存储节点存储的单个索引的分片数量的维度的均衡。
上述步骤910至步骤950的实施例能够基于每个计算存储节点的存储容量和存储空间利用率分别确定得到各个计算存储节点的平均存储容量和平均存储空间利用率。其中,各个计算存储节点的平均存储容量体现了该分布式系统存储索引的能力,各个计算存储节点的平均存储空间利用率体现了该分布式系统已经使用的存储空间占分布式系统的总存储空间的比例,可以通过各个计算存储节点的平均存储容量和平均存储空间利用率更好地了解到当前分布式系统的状态。在这之后,还可以基于目标索引的索引大小、平均存储容量和平均存储空间利用率来确定分片数目,并基于分片数目将目标索引均衡地分成多个分片,在给计算存储节点分配分片时,实现了计算存储节点存储的单个索引的分片数量的维度的均衡。
步骤330的详细描述
步骤330、基于第一偏移量、第二偏移量和第三偏移量,进行多个计算存储节点上分片的迁移,使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。
在一实施例中,参照图11,步骤330包括:
步骤1110、对分布式系统中的索引进行遍历;
步骤1120、针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点上的分片进行迁移。
下面对步骤1110至步骤1120进行详细描述:
在步骤1110中,对分布式系统中的索引进行遍历。
根据本公开的实施例,“对分布式系统中的索引进行遍历”是指使用某种固定的算法依次对分布式系统中的每一个索引进行计算,直到对分布式系统的所有计算完毕。
在一些实施例中,“对分布式系统中的索引进行遍历”可以采用顺序遍历的算法,即按照索引在系统中的存储顺序依次对每个索引进行计算。例如,若分布式系统中的索引按照索引名称的字典序进行存储,那么就按照字典序依次对每个索引进行计算,直到对分布式系统中的所有索引计算完毕。也可以采用随机遍历的算法,即随机选择分布式系统中的索引进行计算,直到所有索引都被计算过。还可以采用分层遍历的算法,若分布式系统中的索引存在分层结构,先遍历第一层的索引,再依次遍历下一层的索引,直至完成所有索引的计算。”
示例性的,假设分布式系统中包括计算存储节点1、计算存储节点2和计算存储节点3,且分布式系统中存储有索引A的多个分片、索引B的多个分片和索引C的多个分片。则对分布式系统的索引进行遍历的过程为:首先,针对索引A的多个分片,依次对计算存储节点1、计算存储节点2和计算存储节点3进行计算。然后,针对索引B的多个分片,依次对计算存储节点1、计算存储节点2和计算存储节点3进行计算。最后,针对索引C的多个分片,依次对计算存储节点1、计算存储节点2和计算存储节点3进行计算。
在步骤1120中,针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点上的分片进行迁移。
根据本公开的实施例,在分布式系统的运行实况中,各个计算存储节点所存储的索引可能非常多,这些索引的分片数目各有差异,且每个计算存储节点所存储的索引的分片大多不相同,很容易导致计算存储节点负载不均衡的情况发生。基于此,本公开的实施例可以针对每一个遍历到的索引,依次对分布式系统中的多个计算存储节点进行遍历,并基于第一偏移量、第二偏移量和第三偏移量对存在负载不均衡的计算存储节点上的分片进行迁移,以实现负载均衡。具体的是,针对每一个索引来说,每个计算存储节点所存储的该索引的分片不完全相同,自然的,每个计算存储节点的第一偏移量也不完全相同。而且,从分布式系统存储的多个索引的角度来说,每个计算存储节点针对于每一个索引也都有不同的第一偏移量。本公开的实施例通过针对每一个遍历到的索引,依次对每个计算存储节点进行遍历,并基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移,精确地实现了计算存储节点存储的单个索引的分片数量的维度的均衡、计算存储节点存储的分片总数量的维度的均衡和计算存储节点的存储空间利用率的维度的均衡。
示例性的,参照图12,假设分布式系统包括计算存储节点1、计算存储节点2和计算存储节点3,且分布式系统中存储有索引A、索引B、索引C和索引D。在对分布式系统中的多个计算存储节点进行负载均衡的过程中,首先针对索引A,依次对计算存储节点1、计算存储节点2和计算存储节点3进行遍历,并基于每个计算存储节点的第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上索引A的分片的迁移。然后针对索引B,依次对计算存储节点1、计算存储节点2和计算存储节点3进行遍历,并基于每个计算存储节点的第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上索引B的分片的迁移。在这之后,针对索引C,依次对计算存储节点1、计算存储节点2和计算存储节点3进行遍历,并基于每个计算存储节点的第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上索引C的分片的迁移。最后,针对索引D,依次对计算存储节点1、计算存储节点2和计算存储节点3进行遍历,并基于每个计算存储节点的第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上索引D的分片的迁移。
“针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点上的分片进行迁移”的具体方法将在下文详细描述。
上述步骤1110至1120的实施例通过针对每一个遍历到的索引,依次对每个计算存储节点进行遍历,并基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移,精确地实现了计算存储节点存储的单个索引的分片数量的维度的均衡、计算存储节点存储的分片总数量的维度的均衡和计算存储节点的存储空间利用率的维度的均衡。
在一实施例中,参照图13,步骤1120包括:
步骤1310、针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序;
步骤1320、按照第一排序对多个计算存储节点进行遍历,以针对遍历到的计算存储节点,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点;
步骤1330、针对遍历到的计算存储节点,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点;
步骤1340、针对遍历到的计算存储节点,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点。
下面对步骤1310至步骤1340进行详细描述:
在步骤1310中,针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序。
第一排序是指针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量对多个计算存储节点进行排序后得到的顺序,该顺序决定了计算存储节点在进行分片迁移时的遍历顺序,以便负载不均衡的计算存储节点能够优先进行分片的迁移。
根据本公开的实施例,针对每一个遍历到的索引,都需要依次对分布式系统中的多个计算存储节点进行遍历,并基于每个计算存储节点的第一偏移量、第二偏移量和第三偏移量来进行多个计算存储节点上分片的迁移。可以想到的是,对分布式系统中的多个计算存储节点进行遍历时,计算存储节点所对应的遍历顺序决定着该计算存储节点是否能够优先进行分片的迁移。因此,为了确定计算存储节点所对应的遍历顺序,以便于负载不均衡的计算存储节点能够优先进行分片的迁移,本公开的实施例针对每一个遍历到的索引,都需要基于第一偏移量、第二偏移量和第三偏移量来对多个计算存储节点进行排序,而第一排序便是指针对每一个遍历到的索引,分布式系统中多个计算存储节点所对应的遍历顺序。
“针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序”的具体方法将在下文详细描述。
在步骤1320中,按照第一排序对多个计算存储节点进行遍历,以针对遍历到的计算存储节点,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点。
根据本公开的实施例,为了兼顾计算存储节点存储的单个索引的分片数量的维度的均衡、计算存储节点存储的分片总数量的维度的均衡和计算存储节点的存储空间利用率的维度的均衡,本公开的实施例将计算存储节点存储的单个索引的分片数量的维度的均衡的优先级定在第一。这是因为调度节点响应于对目标索引的索引分配请求给每个计算存储节点所分配到的分片大小是相近的,在这种情况下如果每个计算存储节点所存储的分片数量是均衡的,那么各个计算存储节点存储的分片总数量和计算存储节点的存储空间利用率也将趋于均衡。
可以理解的是,针对遍历到的索引,如果计算存储节点的第一偏移量小于第一阈值,说明该计算存储节点的单个索引的分片数量处于均衡状态。针对遍历到的索引,当第一偏移量大于或等于第一阈值,说明该计算存储节点的单个索引的分片数量处于不均衡的状态。因此,如果某个计算存储节点的第一偏移量大于或等于第一阈值,需要将该计算存储节点上的分片迁移到第一偏移量小于该计算存储节点的计算存储节点,直到这个处于不均衡状态的计算存储节点的第一偏移量小于第一阈值。进一步的,在将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点时,同时也需要保证分片迁移之后,第一偏移量小于遍历到的计算存储节点的计算存储节点的第一偏移量小于第一阈值。
示例性的,假设第一阈值是1,针对遍历到的索引A,如果计算存储节点1的第一偏移量为1.5,存在一计算存储节点2的第一偏移量为0.2。此时可以将计算存储节点1的索引A的分片迁移到计算存储节点2中,直到针对索引A,计算存储节点1的第一偏移量小于1。
“按照第一排序对多个计算存储节点进行遍历,以针对遍历到的计算存储节点,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点”的具体方法将在下文详细描述。
在步骤1330中,针对遍历到的计算存储节点,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点。
根据本公开的实施例,可以将计算存储节点存储的分片总数量的维度的均衡的优先级定在第二,次于计算存储节点存储的单个索引的分片数量的维度的均衡的优先级。因为调度节点响应于对目标索引的索引分配请求给每个计算存储节点所分配到的分片数目和分片大小是相近的,在计算存储节点存储的单个索引的分片数量达到均衡的前提下,如果实现计算存储节点存储的分片总数量的均衡,计算存储节点的存储空间利用率也将趋于均衡。
可以理解的是,针对遍历到的索引,如果计算存储节点的第二偏移量小于第二阈值,说明该计算存储节点存储的分片总数量处于均衡状态。针对遍历到的索引,当第二偏移量大于或等于第二阈值,说明该计算存储节点的存储的分片总数量处于不均衡的状态。因此,如果某个计算存储节点的第二偏移量大于或等于第二阈值,需要将该计算存储节点上的分片迁移到第二偏移量小于该计算存储节点的计算存储节点,直到这个处于不均衡状态的计算存储节点的第二偏移量小于第二阈值。进一步的,在将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点时,同时也需要保证分片迁移之后,第二偏移量小于遍历到的计算存储节点的计算存储节点的第二偏移量小于第二阈值。
示例性的,假设第二阈值是50,针对遍历到的索引A,如果计算存储节点1的第二偏移量为100,存在一计算存储节点2的第二偏移量为-50。此时可以将计算存储节点1的索引A的分片迁移到计算存储节点2中,直到针对索引A,计算存储节点1的第二偏移量小于50。
“针对遍历到的计算存储节点,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点”的具体方法将在下文详细描述。
在步骤1340中,针对遍历到的计算存储节点,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点。
根据本公开的实施例,可以将计算存储节点的存储空间利用率的维度的均衡的优先级定在最后。可以理解的是,针对遍历到的索引,如果计算存储节点的第三偏移量小于第三阈值,说明该计算存储节点的存储空间利用率处于均衡状态。针对遍历到的索引,当第三偏移量大于或等于第三阈值,说明该计算存储节点的存储空间利用率处于不均衡的状态。因此,如果某个计算存储节点的第三偏移量大于或等于第三阈值,需要将该计算存储节点上的分片迁移到第三偏移量小于该计算存储节点的计算存储节点,直到这个处于不均衡状态的计算存储节点的第三偏移量小于第三阈值。进一步的,在将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点时,同时也需要保证分片迁移之后,第三偏移量小于遍历到的计算存储节点的计算存储节点的第三偏移量小于第三阈值。
示例性的,假设第三阈值是30,针对遍历到的索引A,如果计算存储节点1的第三偏移量为75,存在一计算存储节点2的第三偏移量为-50。此时可以将计算存储节点1的索引A的分片迁移到计算存储节点2中,直到针对索引A,计算存储节点1的第三偏移量小于30。
“针对遍历到的计算存储节点,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点”的具体方法将在下文详细描述。
需要说明的是,针对每一个遍历到的索引,都需要依次对分布式系统中的多个计算存储节点进行遍历,而每遍历一个计算存储节点,都需要执行步骤1320至步骤1340这三个步骤。
上述步骤1310至步骤1340的实施例针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序,使得负载不均衡的计算存储节点能够优先进行分片的迁移。然后,按照第一排序对多个计算存储节点进行遍历,针对遍历到的计算存储节点,如果该计算存储节点的第一偏移量达到第一阈值,便将该计算存储节点上的分片迁移到第一偏移量小于该计算存储节点的计算存储节点,精确地实现了计算存储节点存储的单个索引的分片数量的维度的均衡。针对遍历到的计算存储节点,如果该计算存储节点的第二偏移量达到第二阈值,便将该计算存储节点上的分片迁移到第二偏移量小于该计算存储节点的计算存储节点,精确地实现了计算存储节点存储的分片总数量的维度的均衡。针对遍历到的计算存储节点,如果该计算存储节点的第三偏移量达到第三阈值,便将该计算存储节点上的分片迁移到第三偏移量小于该计算存储节点的计算存储节点,精确地实现了计算存储节点的存储空间利用率的维度的均衡。
在一实施例中,“针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序”存在着两种确定第一排序的方式,此处对第一种确定第一排序的方式进行说明,参照图14,步骤1310包括:
步骤1410、基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序;
步骤1420、对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序;
步骤1430、对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,以形成第一排序。
下面对步骤1410至步骤1430进行详细描述。
在步骤1410中,基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序。
可以理解的是,由于本公开的实施例将计算存储节点存储的单个索引的分片数量的维度的均衡的优先级定在第一,所以首先可以基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序。在基于第一偏移量对多个计算存储节点之后,针对遍历到的索引,存储有该索引的分片数目最多的计算存储节点将排在第一,对应的,存储有该索引的分片数目最少的计算存储节点将排在最后。通过这样排序,便于后续在遍历到第一偏移量大于或等于第一阈值的计算存储节点时,将该计算存储节点上的分片迁移到第一偏移量小于该计算存储节点的计算存储节点。
示例性的,参照图15,假设分布式系统中包括计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点D、计算存储节点E、计算存储节点F。其中,计算存储节点A的第一偏移量为-0.9,计算存储节点B的第一偏移量为-0.3,计算存储节点C的第一偏移量为0.3,计算存储节点D的第一偏移量为1.5,计算存储节点E的第一偏移量为-0.9,计算存储节点F的第一偏移量为0.3。在基于第一偏移量由大到小的顺序对多个计算存储节点进行排序之后,得到的排序顺序为:计算存储节点D>计算存储节点C=计算存储节点F>计算存储节点B>计算存储节点A=计算存储节点E。
在步骤1420中,对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序。
可以理解的是,由于本公开的实施例将计算存储节点存储的分片总数量的维度的均衡的优先级定在第二,所以在基于第一偏移量由大到小的顺序对多个计算存储节点进行排序之后,对于第一偏移量相等的计算存储节点,可以基于第二偏移量对其由大到小进行排序。进一步的,在基于第二偏移量由大到小的顺序对第一偏移量相等的计算存储节点进行排序之后,针对第一偏移量相等的计算存储节点,存储的分片总数量最多的计算存储节点将排在第一,对应的,存储的分片总数量最少的计算存储节点将排在最后。通过这样排序,便于后续在遍历到第二偏移量大于或等于第二阈值的计算存储节点时,将该计算存储节点上的分片迁移到第二偏移量小于该计算存储节点的计算存储节点。
示例性的,参照图15,假设计算存储节点A的第二偏移量为-31,计算存储节点B的第二偏移量为-2,计算存储节点C的第二偏移量为33,计算存储节点D的第二偏移量为60,计算存储节点E的第二偏移量为-31,计算存储节点F的第一偏移量为25。在基于第一偏移量由大到小的顺序对多个计算存储节点进行排序之后,得到的排序顺序为:计算存储节点D>计算存储节点C=计算存储节点F>计算存储节点B>计算存储节点A=计算存储节点E。基于此,可以基于第二偏移量由大到小的顺序对第一偏移量相等的计算存储节点进行排序,以得到排序顺序为:计算存储节点D>计算存储节点C>计算存储节点F>计算存储节点B>计算存储节点A=计算存储节点E。
在步骤1430中,对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,以形成第一排序。
可以理解的是,由于本公开的实施例将计算存储节点的存储空间利用率的优先级定在第三,所以在基于第二偏移量由大到小的顺序对第一偏移量相等的计算存储节点进行排序之后,对于第二偏移量相等的计算存储节点,可以基于第三偏移量对其由大到小进行排序。进一步的,在基于第三偏移量由大到小的顺序对第二偏移量相等的计算存储节点进行排序之后,针对第二偏移量相等的计算存储节点,存储空间利用率最高的计算存储节点将排在第一,对应的,存储空间利用率最低的计算存储节点将排在最后。通过这样排序,便于后续在遍历到第三偏移量大于或等于第三阈值的计算存储节点时,将该计算存储节点上的分片迁移到第三偏移量小于该计算存储节点的计算存储节点。
示例性的,参照图15,假设计算存储节点A的第三偏移量为-2%,计算存储节点B的第三偏移量为0,计算存储节点C的第三偏移量为1%,计算存储节点D的第三偏移量为2%,计算存储节点E的第三偏移量为-4%,计算存储节点F的第三偏移量为1%。在基于第二偏移量由大到小的顺序对第一偏移量相等的计算存储节点进行排序之后,得到的排序顺序为:计算存储节点D>计算存储节点C>计算存储节点F>计算存储节点B>计算存储节点A=计算存储节点E。基于此,可以基于第三偏移量由大到小的顺序对第二偏移量相等的计算存储节点进行排序
,以得到排序顺序为:计算存储节点D>计算存储节点C>计算存储节点F>计算存储节点B>计算存储节点A>计算存储节点E。
上述步骤1410至步骤1430的实施例基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序,将存储有遍历到的索引的分片数目最多的计算存储节点将排在第一,将存储有遍历到的索引的分片数目最少的计算存储节点将排在最后。方便了后续在遍历到第一偏移量大于或等于第一阈值的计算存储节点时,将该计算存储节点上的分片迁移到第一偏移量小于该计算存储节点的计算存储节点。进一步的,对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序,将存储的分片总数量最多的计算存储节点将排在第一,将存储的分片总数量最少的计算存储节点将排在最后。方便了后续在遍历到第二偏移量大于或等于第二阈值的计算存储节点时,将该计算存储节点上的分片迁移到第二偏移量小于该计算存储节点的计算存储节点。而且,对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,将存储空间利用率最高的计算存储节点将排在第一,将存储空间利用率最低的计算存储节点将排在最后。方便了后续在遍历到第三偏移量大于或等于第三阈值的计算存储节点时,将该计算存储节点上的分片迁移到第三偏移量小于该计算存储节点的计算存储节点。
在一实施例中,此处对第二种确定第一排序的方式进行说明,参照图16,步骤1310包括:
步骤1610、获取第一偏移量的第一权重、第二偏移量的第二权重和第三偏移量的第三权重;
步骤1620、基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,其中,第一权重大于第二权重,第二权重大于第三权重;
步骤1630、基于加权和由大到小的顺序,对多个计算存储节点进行第一排序。
下面对步骤1610至步骤1630进行详细描述。
在步骤1610中,获取第一偏移量的第一权重、第二偏移量的第二权重和第三偏移量的第三权重。
根据本公开的实施例,第一权重可以用于表明第一偏移量的重要程度,第一偏移量越重要,第一权重越大。同理,第二权重可以用于表明第二偏移量的重要程度,第二偏移量越重要,第二权重越大;第三权重可以用于表明第三偏移量的重要程度,第三偏移量越重要,第三权重越大。
在本公开实施例中,第一权重、第二权重和第三权重的取值范围可根据实际系统的性能和负载情况进行调整。一般情况下,第一权重可取值范围为[50,100],第二权重可取值范围为[1,10],第三权重可取值范围为[0.1,1]。确定这些权重时,可通过对系统进行多次模拟测试,观察不同权重组合下系统的负载均衡效果和处理效率,选择最优的权重组合。也可以根据系统管理员的经验,结合系统的历史运行数据来确定权重值。”修改成:“在本公开实施例中,第一权重、第二权重和第三权重的取值范围可根据实际系统的性能和负载情况进行调整。一般情况下,第一权重可取值范围为[50,100],第二权重可取值范围为[1,10],第三权重可取值范围为[0.1,1]。确定这些权重时,进行模拟测试的具体方法如下:首先,在模拟环境中设置不同的权重组合,例如第一权重从50开始,每次增加10,第二权重从1开始,每次增加1,第三权重从0.1开始,每次增加0.1,生成一系列的权重组合。然后,针对每个权重组合,在模拟环境中模拟不同的负载场景,例如高并发请求、不同的索引变更频率等,运行系统一段时间,记录系统的负载均衡效果(如各个计算存储节点的第一偏移量、第二偏移量和第三偏移量的均衡程度)和处理效率(如请求的平均响应时间、吞吐量等)。最后,对比不同权重组合下的记录数据,选择使系统负载均衡效果最佳且处理效率最高的权重组合作为最终的权重值。结合系统的历史运行数据确定权重值时,系统管理员可以分析历史数据中不同时间段内系统的负载情况、各个计算存储节点的性能表现等信息。例如,如果历史数据显示在某些时间段内,计算存储节点存储的单个索引的分片数量不均衡对系统性能影响较大,那么可以适当提高第一权重的值;如果分片总数量的不均衡对系统性能影响较大,则提高第二权重的值;如果存储空间利用率的不均衡对系统性能影响较大,则提高第三权重的值。
在步骤1620中,基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,其中,第一权重大于第二权重,第二权重大于第三权重;
可以理解的是,由于本公开实施例将计算存储节点存储的单个索引的分片数量的维度的均衡的优先级定在第一,将计算存储节点存储的分片总数量的维度的均衡的优先级定在第二,将计算存储节点的存储空间利用率的优先级定在第三。对应的,第一权重也应该设置为大于第二权重的数值,以表明第一偏移量的重要程度,第二权重应该设置为大于第三权重的数值,以表明第二偏移量的重要程度。具体的是,实际的运行工况中,为凸显计算存储节点存储的单个索引的分片数量的维度的均衡的优先级,一般需要将第一权重设置的远大于第二权重。
根据本公开的实施例,基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,所得到的计算存储节点的加权和综合了第一偏移量的重要程度、第二偏移量的重要程度和第三偏移量的重要程度,能够准确地体现计算存储节点的均衡状态。具体的是,计算存储节点的加权和越大,则表明该计算存储节点需要进行迁移的分片越多。相对应的,算存储节点的加权和越小,则表明该计算存储节点可以容纳其它计算存储节点迁移出的分片的空间越大。
示例性的,参照图17,假设第一权重为50,第二权重为1,第三权重为0.5。如果分布式系统中存在一计算存储节点A,且计算存储节点A的第一偏移量、第二偏移量、第三偏移量分别为-0.9、-31、-2%,可以确定得到计算存储节点A所对应的加权和为-76.01。如果分布式系统中存在一计算存储节点B,且计算存储节点B的第一偏移量、第二偏移量、第三偏移量分别为-0.3、-2、0,可以确定得到计算存储节点B所对应的加权和为-17。可以想到的是,计算存储节点A和计算存储节点B的加权和都小于0,这表明了计算存储节点A和计算存储节点B还可以接收其它计算存储节点迁移出的分片。
在步骤1630中,基于加权和由大到小的顺序,对多个计算存储节点进行第一排序。
根据本公开的实施例,基于加权和由大到小的顺序对多个计算存储节点进行第一排序,可以使得需要迁移出最多分片的计算存储节点排在最前面,使可以容纳其它计算存储节点迁移出的最多分片的计算存储节点排在最后面。通过这样排序,便于后续将需要迁移分片的计算存储节点所存储的分片迁移到可以容纳其它计算存储节点迁移出的分片的计算存储节点。
示例性的,参照图17,假设第一权重为50,第二权重为1,第三权重为0.5。通过计算确定得到计算存储节点A所对应的加权和为-76.01,计算存储节点B所对应的加权和为-17,计算存储节点C所对应的加权和为48.005,计算存储节点D所对应的加权和为125.01,计算存储节点E所对应的加权和为-17,计算存储节点F所对应的加权和为40.005。在基于加权和由大到小的顺序对多个计算存储节点进行第一排序后,可以得到排序顺序为:计算存储节点D>计算存储节点C>计算存储节点F>计算存储节点B>计算存储节点A>计算存储节点E。
上述步骤1610至步骤1630的实施例基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,所得到的计算存储节点的加权和综合了第一偏移量的重要程度、第二偏移量的重要程度和第三偏移量的重要程度,能够准确地体现计算存储节点的均衡状态。而且,还能够基于加权和由大到小的顺序对多个计算存储节点进行第一排序,使得需要迁移出最多分片的计算存储节点排在最前面,使可以容纳其它计算存储节点迁移出的最多分片的计算存储节点排在最后面。这种排序方式便于后续将需要迁移分片的计算存储节点所存储的分片迁移到可以容纳其它计算存储节点迁移出的分片的计算存储节点。
在一实施例中,“当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点”的实施方式有两种,此处对第一种实施方式进行说明,参照图18,步骤1320包括:
步骤1810、将计算存储节点按照第一偏移量由小到大的顺序,进行第二排序;
步骤1820、初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点;
步骤1830、在遍历到的计算存储节点上,选择第一目标分片;
步骤1840、将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点,当迁移后第一偏移量仍然大于或等于第一阈值,用第二排序中的下一个计算存储节点更新第一目标迁移计算存储节点,返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤,直到迁移后第一偏移量小于第一阈值。
其中,第一目标迁移计算存储节点是指在第一偏移量大于或等于第一阈值时,接收遍历到的计算存储节点迁移出的分片的计算存储节点,初始化为第二排序中第一个计算存储节点,在迁移过程中根据情况进行更新。
第一目标分片是指遍历到的第一偏移量达到第一阈值的计算存储节点需要迁移出去的分片,通过特定方法选择,使得其大小与第一偏移量相匹配,以提高分片迁移的效率。
下面对步骤1810至步骤1840进行详细描述。
在步骤1810中,将计算存储节点按照第一偏移量由小到大的顺序,进行第二排序。
根据本公开的实施例,第二排序是指基于第一偏移量对计算存储节点进行升序排序,以使第一偏移量最小的排在第一,第一偏移量最大的排在最后。第二排序具体是指基于第一偏移量对计算存储节点进行升序排序后得到的顺序,使第一偏移量最小的计算存储节点排在第一,第一偏移量最大的排在最后,用于在第一偏移量大于或等于第一阈值时,确定接收遍历到的计算存储节点迁移出分片的目标计算存储节点的选择顺序。其中,第一偏移量最小意味着该计算存储节点可以容纳其它计算存储节点迁移出的最多分片。第一偏移量最大意味着该计算存储节点需要迁移出最多分片。
示例性的,参照图19,假设分布式系统包括计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点D、计算存储节点E、计算存储节点F。计算存储节点A的第一偏移量为-0.9,计算存储节点B的第一偏移量为-0.3,计算存储节点C的第一偏移量为0.3,计算存储节点D的第一偏移量为1.5,计算存储节点E的第一偏移量为-0.8,计算存储节点F的第一偏移量为0.5。将计算存储节点按照第一偏移量从小到大的顺序进行第二排序,可以得到第二排序为:计算存储节点A、计算存储节点E、计算存储节点B、计算存储节点C、计算存储节点F、计算存储节点D。
在步骤1820中,初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点。
根据本公开的实施例,第一目标迁移计算存储节点是指接收遍历到的第一偏移量达到第一阈值的计算存储节点迁移出的分片的计算存储节点。可以理解的是,第二排序中的第一个计算存储节点可以容纳其它计算存储节点迁移出的最多分片,将第一目标迁移计算存储节点确定为第二排序中第一个计算存储节点,可以使第一目标迁移计算存储节点接收尽可能多的分片。进一步的,在确定第一目标迁移计算存储节点时,需要保证第一目标迁移计算存储节点在接收到遍历到的计算存储节点迁移出的分片之后,第一偏移量仍小于第一阈值。
示例性的,参照图19,假设第二排序为:计算存储节点A、计算存储节点E、计算存储节点B、计算存储节点C、计算存储节点F、计算存储节点D,由此可以初始化第一目标迁移计算存储节点为计算存储节点A。
在步骤1830中,在遍历到的计算存储节点上,选择第一目标分片。
根据本公开的实施例,第一目标分片是指遍历到的第一偏移量达到第一阈值的计算存储节点需要迁移出去的分片。参照图20A可知,针对同一个索引,计算存储节点可能存储有该索引的多个分片,不同的分片负载也不相同,且不同负载的分片也影响着该计算存储节点的第一偏移量。存储有负载较大的分片会可能使得该计算存储节点的第一偏移量偏大,自然的,将该分片迁移出计算存储节点也将使得该计算存储节点的第一偏移量明显减小。基于此,在对遍历到的第一偏移量达到第一阈值的计算存储节点中的分片进行迁移时,首先需要选择合适大小的第一目标分片。
“在遍历到的计算存储节点上,选择第一目标分片”的具体方法将在下文详细描述。
在步骤1840中,将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点,当迁移后第一偏移量仍然大于或等于第一阈值,用第二排序中的下一个计算存储节点更新第一目标迁移计算存储节点,返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤,直到迁移后第一偏移量小于第一阈值。
根据本公开的实施例,在确定了第一目标分片和第一目标迁移计算存储节点之后,便可以将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点上。示例性的,参照图20A,假设针对遍历到的索引A,遍历到的计算存储节点D的第一偏移量达到第一阈值,且已经确定得到第一目标迁移计算存储节点为计算存储节点A,第一目标分片为索引A的一个分片。那么,可以将索引A的一个分片从计算存储节点D迁移到计算存储节点A中。图20B为完成分片迁移后的计算存储节点A和计算存储节点D的示意图,可以得到的是,此时计算存储节点D中不再存储有第一目标分片,计算存储节点A中存储有第一目标分片。
进一步的,在将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点之后,遍历到的计算存储节点和第一目标迁移计算存储节点的第一偏移量都将发生变化。此时,当迁移后遍历到的计算存储节点的第一偏移量仍然大于或等于第一阈值,则说明遍历到的计算存储节点还需要再选择第一目标分片来迁移出去。同时,为避免同一个计算存储节点因为持续地接收分片而导致第一偏移量达到第一阈值的情况发生,第一目标迁移计算存储节点也需要随着第一目标分片一同更新。因此,在每次将第一目标分片从遍历到的计算存储节点中迁移出去之后,都需要对第一目标分片和第一目标迁移计算存储节点进行更新和选择。其中,当对第一目标迁移计算存储节点进行更新时,一般会将第二排序中继更新前的第一目标迁移计算存储节点的下一个计算存储节点确定为更新后的第一目标迁移计算存储节点。当对第一目标分片进行更新时,更新后得到的第一目标分片的大小和更新前的第一目标分片的大小是相近的。
根据本公开的实施例,在更新完第一目标分片和第一目标迁移计算存储节点之后,还需返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤,以将更新后的第一目标分片从遍历到的计算存储节点迁移到更新后的第一目标迁移计算存储节点,直到遍历到的计算存储节点迁移后的第一偏移量小于第一阈值。其中,针对某个索引来说,当遍历到的计算存储节点的第一偏移量小于第一阈值,则说明该计算存储节点已经实现了这个索引的分片数量的维度的均衡。
在一实施例中,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点,还包括:当第一目标迁移计算存储节点为第二排序中的最后一个计算存储节点,且迁移后第一偏移量仍然大于或等于第一阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第一目标分片,并用第二排序中第一个计算存储节点更新第一目标迁移计算存储节点,并返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤。
可以理解的是,在第一目标迁移计算存储节点为第二排序中的最后一个计算存储节点的情况下,如果在将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点之后,遍历到的计算存储节点的第一偏移量仍然是大于或等于第一阈值,说明此时该计算存储节点仍未实现索引的分片数量的维度的均衡。而最有可能的原因是第一目标分片的大小过小。在第一目标分片大小过小的情况下,即使遍历到的计算存储节点执行了多次分片的迁移操作,遍历到的计算存储节点的第一偏移量仍然是大于或等于第一阈值的。所以,此时需要在遍历到的计算存储节点上重新选择较大的第一目标分片,并对各个计算存储节点的第二排序进行更新,用更新后的第二排序中第一个计算存储节点来更新第一目标迁移计算存储节点。
根据本公开的实施例,在重新选择的第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点之后,如果遍历到的计算存储节点的第一偏移量仍然大于或等于第一阈值,则需要用更新后的第二排序中当前第一目标迁移计算存储节点的下一个计算存储节点来更新第一目标迁移计算存储节点。在这之后,循环执行分片迁移以及更新第一目标迁移计算存储节点和第一目标分片的操作,直到被遍历到的计算存储节点的第一偏移量小于第一阈值。
需要说明的是,只有在第一目标迁移计算存储节点为第二排序中的最后一个计算存储节点,且迁移后被遍历的计算存储节点的第一偏移量仍然大于或等于第一阈值的情况下,才需要重新选择第一目标分片。
示例性的,假设针对遍历到的索引A,遍历到的计算存储节点为计算存储节点D的第一偏移量达到第一阈值,且第二排序为计算存储节点A、计算存储节点B、计算存储节点C。在确定得到分片大小为10MB的第一目标分片之后,首先将第一目标分片从计算存储节点D迁移到计算存储节点A。当迁移后计算存储节点D的第一偏移量仍然大于或等于第一阈值,用计算存储节点B更新第一目标迁移计算存储节点。然后,更新第一目标分片,将第一目标分片从计算存储节点D迁移到计算存储节点B。当迁移后计算存储节点D的第一偏移量仍然大于或等于第一阈值,用计算存储节点C更新第一目标迁移计算存储节点。在这之后,更新第一目标分片,将第一目标分片从计算存储节点D迁移到计算存储节点C。当迁移后计算存储节点D的第一偏移量仍然大于或等于第一阈值,在计算存储节点D上未选择过的分片中,重新选择第一目标分片,重新选择的第一目标分片为20MB。假设更新后的第二排序仍为计算存储节点A、计算存储节点B、计算存储节点C。此时可以将重新选择的第一目标分片从计算存储节点D迁移到计算存储节点A,并循环执行分片迁移以及更新第一目标迁移计算存储节点和第一目标分片的操作,直到计算存储节点D的第一偏移量小于第一阈值。
上述步骤1810至步骤1840的实施例通过初始化第一目标迁移计算存储节点为第二排序中的第一个计算存储节点,可以使第一目标迁移计算存储节点接收尽可能多的分片。进一步的,通过选择合适的第一目标分片,并将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点,精确地实现了计算存储节点存储的单个索引的分片数量的维度的均衡。而且,当迁移后第一偏移量仍然大于或等于第一阈值,本公开的实施例还将用第二排序中继更新前的第一目标迁移计算存储节点的下一个计算存储节点作为更新后的第一目标迁移计算存储节点,来执行分片迁移,并循环执行这一步骤,直到迁移后第一偏移量小于第一阈值。通过对第一目标迁移计算存储节点进行更新,避免了同一个计算存储节点因为持续地接收分片而导致第一偏移量达到第一阈值的情况发生,精确地均衡了多个计算存储节点的单个索引的分片数量。
在一实施例中,参照图21,步骤1830包括:
步骤2110、获取遍历到的计算存储节点上各个分片的分片大小;
步骤2120、基于各个分片的分片大小,确定平均分片大小;
步骤2130、确定各个分片的分片大小相对于平均分片大小的第一倍数;
步骤2140、选择第一倍数与第一偏移量匹配的分片,作为第一目标分片。
下面对步骤2110至步骤2140进行详细描述。
在步骤2110中,获取遍历到的计算存储节点上各个分片的分片大小。
根据本公开的实施例,针对遍历到的第一偏移量达到第一阈值的计算存储节点,需要将该计算存储节点中的分片迁移出去。针对每个索引,计算存储节点存储有多个大小不同的分片,为了精确地确定要迁移出去的分片,首先需要获取遍历到的计算存储节点上各个分片的分片大小。
可以理解的是,针对分布式系统中的每个索引,每个计算存储节点均存储有该索引的分片的分片大小。在需要获取遍历到的计算存储节点上各个分片的分片大小时,调度节点可以直接向遍历到的计算存储节点发送获取请求,来获取该计算存储节点上各个分片的分片大小。
该获取请求可以采用特定的消息格式,例如包含请求类型(标识为获取分片大小请求)、请求的索引标识、调度节点的标识等信息。计算存储节点在接收到该请求后,会解析请求内容,根据请求中的索引标识查找该索引下各个分片的分片大小,并将这些分片大小信息按照预先约定的响应格式进行封装,例如采用JSON格式,包含索引标识和各个分片的分片大小列表,然后将响应消息返回给调度节点。
示例性的,参照图22,假设针对索引A,遍历到计算存储节点D,那么可以得到计算存储节点D上的索引A的各个分片的分片大小分别为50MB、53MB、60MB和55MB。
在步骤2120中,基于各个分片的分片大小,确定平均分片大小。
根据本公开的实施例,平均分片大小为各个分片的大小的平均值。
示例性的,假设针对索引A,遍历到计算存储节点D,可以得到计算存储节点D上的索引A的各个分片的分片大小分别为50MB、53MB、60MB和55MB。通过计算,可以确定得到计算存储节点D所存储的索引A的平均分片大小为54.5MB。
在步骤2130中,确定各个分片的分片大小相对于平均分片大小的第一倍数。
根据本公开的实施例,第一倍数是指分片的分片大小除以平均分片的商。
示例性的,假设针对索引A,遍历到计算存储节点D,可以得到计算存储节点D上的索引A的各个分片的分片大小分别为50MB、53MB、60MB和55MB,计算存储节点D所存储的索引A的平均分片大小为54.5MB。基于此,可以确定得到各个分片的分片大小相对于平均分片大小的第一倍数分别为100/109、106/109、120/109、110/109。
在步骤2140中,选择第一倍数与第一偏移量匹配的分片,作为第一目标分片。
根据本公开的实施例,如果选择第一倍数与第一偏移量不匹配的分片来作为第一目标分片,可能会出现这种情况:第一偏移量过大(即需要迁移出大量的分片),但第一倍数过小(即分片太小),在执行多次分片迁移步骤后,遍历到的计算存储节点的第一偏移量仍然大于或等于第一阈值,降低了分片迁移的效率。基于此,可以选择第一倍数与第一偏移量匹配的分片来作为第一目标分片,以减少执行分片迁移的次数,提高分片迁移的效率,也提高了实现计算存储节点的单个索引的分片数量的维度的均衡的速度。
可以理解的是,第一偏移量和第一倍数是成正比的关系。基于此,可以提前根据计算存储节点的运行实况来构造第一偏移量和第一倍数的函数,并在确定得到第一偏移量之后,将第一偏移量代入第一偏移量和第一倍数的函数,以确定得到与第一偏移量匹配的第一倍数。然后,可以基于与第一偏移量匹配的第一倍数来选择对应的分片,将该分片作为第一目标分片。
根据本公开的实施例,还可以通过查表的方式来选择与第一偏移量匹配的第一倍数,并基于与第一偏移量匹配的第一倍数来选择对应的分片,根据第一偏移量查找预先设置的第一偏移量和第一倍数对应关系表,便可以得到第一偏移量对应的第一倍数。基于此,可以快速查找到第一倍数与第一偏移量匹配的分片。
举例说明,第一偏移量和第一倍数可以是成正比的关系。基于此,可以提前根据计算存储节点的运行实况来构造第一偏移量和第一倍数的函数,例如构造函数f(x)=0.4x+0.8(其中x为第一偏移量)。在确定得到第一偏移量之后,将第一偏移量代入该函数,以确定得到与第一偏移量匹配的第一倍数。然后,可以基于与第一偏移量匹配的第一倍数来选择对应的分片,将该分片作为第一目标分片。也可以通过查表的方式来选择与第一偏移量匹配的第一倍数,例如第一偏移量和第一倍数的对应关系表如下:当第一偏移量在[0,0.5)时,对应的第一倍数范围为[0.8,1);当第一偏移量在[0.5,1)时,对应的第一倍数范围为[1,1.2);当第一偏移量在[1,1.5)时,对应的第一倍数范围为[1.2,1.4)等,根据第一偏移量查找该对应关系表,便可以得到第一偏移量对应的第一倍数,进而选择对应的分片作为第一目标分片。
上述步骤2110至步骤2140的实施例可以基于遍历到的计算存储节点上各个分片的分片大小来确定平均分片大小,并确定各个分片的分片大小相对于平均分片大小的第一倍数。通过选择第一倍数与第一偏移量匹配的分片作为第一目标分片,可以减少执行分片迁移的次数,提高分片迁移的效率,同时也提高了实现计算存储节点的单个索引的分片数量的维度的均衡的速度。
在一实施例中,此处对“当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点”的第一种实施方式进行说明,参照图23,步骤1320包括:
步骤2310、获取遍历到的计算存储节点上各个分片的分片大小;
步骤2320、基于各个分片的分片大小,确定平均分片大小;
步骤2330、确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数;
步骤2340、计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,待考察计算存储节点为第一偏移量小于遍历到的计算存储节点的计算存储节点;
步骤2350、将第二倍数与第一差匹配的其它计算存储节点,作为目标计算存储节点,并将目标分片迁移到目标计算存储节点。
下面对步骤2310至步骤2350进行详细描述:
在步骤2310中,获取遍历到的计算存储节点上各个分片的分片大小。
可以理解的是,上文中步骤2110的实施例已解释了“获取遍历到的计算存储节点上各个分片的分片大小”的方法,此处不对其进行赘述。
在步骤2320中,基于各个分片的分片大小,确定平均分片大小。
可以理解的是,上文中步骤2120的实施例已经解释了“基于各个分片的分片大小,确定平均分片大小”的方法,此处不对其进行赘述。
在步骤2330中,确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数。
根据本公开的实施例,目标分片是指遍历到的第一偏移量达到第一阈值的计算存储节点需要迁移出去的分片,其中,针对某个索引,目标分片可以通过在遍历到的计算存储节点所存储的该索引的分片中随机选取得到,也可以由预设的确定方法得到。“确定目标分片”的具体方法将在下文详细描述。第二倍数是指目标分片的分片大小除以平均分片的商。
可以理解的是,上文中步骤2130的实施方法和“确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数”的实施方法相似,此处不对其进行赘述。
在步骤2340中,计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,待考察计算存储节点为第一偏移量小于遍历到的计算存储节点的计算存储节点。
根据本公开的实施例,对遍历到的索引,如果存在某个计算存储节点的第一偏移量小于遍历到的计算存储节点的第一偏移量,那么可以将这个计算存储节点定义为待考察计算存储节点。待考察计算存储节点是指在第一偏移量大于或等于第一阈值时,第一偏移量小于遍历到的计算存储节点的计算存储节点,用于计算与遍历到的计算存储节点的第一偏移量的差值,以确定合适的目标计算存储节点接收迁移的分片。第一差是指遍历到的计算存储节点的第一偏移量减去待考察计算存储节点的第一偏移量的差。之所以要计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,是因为需要确定遍历到的计算存储节点所存储的分片数量和每个待考察计算存储节点所存储的分片数量的差距,通过这个差距,可以更明确地知道待考察计算存储节点可以接收的遍历到的计算存储节点迁移出的分片的大小。
示例性的,参照图24,假设分布式系统包括计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点D、计算存储节点E和计算存储节点F。如果当前遍历到计算存储节点D,可以确定得到计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点E和计算存储节点F均为待考察计算存储节点。而且,计算存储节点D的第一偏移量与计算存储节点A的第一偏移量的第一差为-0.9,计算存储节点D的第一偏移量与计算存储节点B的第一偏移量的第一差为-0.3,计算存储节点D的第一偏移量与计算存储节点C的第一偏移量的第一差为0.3,计算存储节点D的第一偏移量与计算存储节点E的第一偏移量的第一差为-0.9,计算存储节点D的第一偏移量与计算存储节点F的第一偏移量的第一差为0.3。
在步骤2350中,将第二倍数与第一差匹配的其它计算存储节点,作为目标计算存储节点,并将目标分片迁移到目标计算存储节点。
目标计算存储节点是指在第一偏移量大于或等于第一阈值时,将第二倍数与第一差匹配的其它计算存储节点,作为接收目标分片迁移的节点,以提高分片迁移的效率。
根据本公开的实施例,第二倍数越大,说明目标分片的分片大小越大。第一差越大,说明待考察计算存储节点可以接收的遍历到的计算存储节点迁移出的分片的大小越大。通过将第二倍数与第一差匹配的计算存储节点作为目标计算存储节点,可以减少执行分片迁移的次数,提高分片迁移的效率。
示例性的,将第二倍数与第一差匹配的计算存储节点作为目标计算存储节点之前,首先需要确定和第二倍数匹配的第一差,进而根据这个第一差来查找合适的计算存储节点。基于第二倍数确定匹配的第一差,可以采用查表的方式。根据第二倍数查找预先设置的第二倍数和第一差对应关系表,得到第二倍数对应的第一差。而且,基于第二倍数确定第一差,还可以采用代入公式的方法。将第二倍数代入预先设置的第二倍数和第一差的公式,得到第二倍数对应的第一差。
举例说明:基于第二倍数确定匹配的第一差,可以采用查表的方式。例如预先设置的第二倍数和第一差对应关系表如下:当第二倍数在[1,1.2)时,对应的第一差范围为[1,2);当第二倍数在[1.2,1.4)时,对应的第一差范围为[2,3);当第二倍数在[1.4,1.6)时,对应的第一差范围为[3,4)等,根据第二倍数查找该对应关系表,得到第二倍数对应的第一差。而且,基于第二倍数确定第一差,还可以采用代入公式的方法,例如预先设置的第二倍数m和第一差n的公式为n=2m-1,将第二倍数代入该公式,得到第二倍数对应的第一差。
示例性的,参照图24,假设当前遍历到计算存储节点D,已知计算存储节点D的第一偏移量与计算存储节点A的第一偏移量的第一差为-0.9,计算存储节点D的第一偏移量与计算存储节点B的第一偏移量的第一差为-0.3,计算存储节点D的第一偏移量与计算存储节点C的第一偏移量的第一差为0.3,计算存储节点D的第一偏移量与计算存储节点E的第一偏移量的第一差为-0.9,计算存储节点D的第一偏移量与计算存储节点F的第一偏移量的第一差为0.3。假设针对索引A来说,计算存储节点D的目标分片的分片大小相对于平均分片大小的第二倍数为1.2,那么根据第二倍数和第一差对应关系表可以得到,和第二倍数匹配的第一差的范围为(2,3]。基于此,可以将计算存储节点A或计算存储节点E来作为目标计算存储节点。
上述步骤2310至步骤2350的实施例基于各个分片的分片大小计算得到平均分片大小,并确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数,确定了遍历到的计算存储节点所存储的分片数量和每个待考察计算存储节点所存储的分片数量的差距。进一步的,通过计算遍历到的计算存储节点的第一偏移量和每个待考察计算存储节点的第一偏移量的第一差,将第二倍数与第一差匹配的计算存储节点作为目标计算存储节点,以把目标分片迁移到目标计算存储节点,可以减少执行分片迁移的次数,提高分片迁移的效率。
在一实施例中,参照图25,步骤2330的目标分片通过以下方式从各个分片确定:
步骤2510、确定遍历到的计算存储节点的存储容量;
步骤2520、确定遍历到的计算存储节点的存储空间利用率;
步骤2530、获取各个分片的分片大小;
步骤2540、基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。
下面对步骤2510至步骤2540进行详细描述:
在步骤2510中,确定遍历到的计算存储节点的存储容量。
根据本公开的实施例,遍历到的计算存储节点需要迁移出去的目标分片可以通过多个因素来确定,而其中一个因素便是遍历到的计算存储节点的存储容量。可以想到的是,遍历到的计算存储节点的存储容量越大,这个计算存储节点便有可能存储有多个负载较大的分片。
在步骤2520中,确定遍历到的计算存储节点的存储空间利用率。
根据本公开的实施例,在确定得到遍历到的计算存储节点的存储容量和存储空间利用率后,可以确定得到遍历到的计算存储节点的剩余存储空间。计算存储节点的剩余存储空间则决定着该计算存储节点在后续的运行工况中接收分片的能力,以及该计算存储节点需要迁移出去的目标分片的分片大小。
示例性的,假设确定得到遍历到的计算存储节点1的存储容量是900MB。如果当前计算存储节点1的存储空间利用率是80%,说明计算存储节点1的剩余存储空间为180MB。如果当前计算存储节点1的存储空间利用率是50%,说明计算存储节点1的剩余存储空间为450MB。
在步骤2530中,获取各个分片的分片大小。
根据本公开的实施例,因为计算存储节点的剩余存储空间的大小决定着迁移出去的目标分片的大小,为准确地从遍历到的计算存储节点存储的各个分片中选取出目标分片,还需要获取各个分片的分片大小。
示例性的,假设针对索引A,遍历到的计算存储节点1存储有分片1、分片2和分片3。通过获取可以得到分片1的分片大小为10MB,分片2的分片大小为15MB,分片3的分片大小为13MB。
在步骤2540中,基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。
基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。具体方法为,先根据存储容量和存储空间利用率计算出计算存储节点的剩余存储空间,即剩余存储空间=存储容量×(1-存储空间利用率)。若剩余存储空间较小,优先选择分片大小较大的分片作为目标分片,可将各个分片按分片大小降序排列,选取排在前面的分片;若剩余存储空间较大,优先选择分片大小较小的分片作为目标分片,可将各个分片按分片大小升序排列,选取排在前面的分片。例如,若计算得到剩余存储空间为100MB,而各个分片大小分别为20MB、30MB、50MB、80MB,由于剩余存储空间较小,可选择80MB的分片作为目标分片;若剩余存储空间为500MB,可选择20MB的分片作为目标分片。
根据本公开的实施例,基于遍历到的计算存储节点的存储容量和存储空间利用率可以确定得到遍历到的计算存储节点的剩余存储空间。可以想到的是,如果该计算存储节点的剩余存储空间较小,可以选择一个分片大小较大的目标分片从遍历到的计算存储节点中迁移出去。如果该计算存储节点的剩余存储空间较大,可以选择一个分片大小较小的目标分片从遍历到的计算存储节点中迁移出去。
示例性的,假设确定得到遍历到的计算存储节点1的存储容量是500MB,针对索引A,计算存储节点1存储有分片1、分片2、分片3和分片4。通过获取可以得到分片1的分片大小为50MB,分片2的分片大小为45MB,分片3的分片大小为47MB,分片4的分片大小为52MB。如果当前计算存储节点1的存储空间利用率是80%,说明计算存储节点1的剩余存储空间为100MB,可以将分片4作为目标分片,以从计算存储节点1中迁移出去。如果当前计算存储节点1的存储空间利用率是50%,说明计算存储节点1的剩余存储空间为250MB,可以将分片2作为目标分片,以从计算存储节点1中迁移出去。
上述步骤2510至步骤2540的实施例可以基于遍历到的计算存储节点的存储容量和存储空间利用率,确定得到遍历到的计算存储节点的剩余存储空间。并且,通过计算存储节点的剩余存储空间的大小进一步确定迁移出去的目标分片的大小。如果该计算存储节点的剩余存储空间较小,可以选择一个分片大小较大的目标分片从遍历到的计算存储节点中迁移出去。如果该计算存储节点的剩余存储空间较大,可以选择一个分片大小较小的目标分片从遍历到的计算存储节点中迁移出去。通过这种方法,可以准确地确定需要从遍历到的计算存储节点中迁移出去的目标分片,减少了执行分片迁移的次数,提高了分片迁移的效率。
在一实施例中,参照图26,步骤1330包括:
步骤2610、将计算存储节点按照第二偏移量由小到大的顺序,进行第三排序;
步骤2620、初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点;
步骤2630、在遍历到的计算存储节点上,选择第二目标分片;
步骤2640、将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点,当迁移后第二偏移量仍然大于或等于第二阈值,用第三排序中的下一个迁移计算存储节点更新第二目标迁移计算存储节点,返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤,直到迁移后第二偏移量小于第二阈值。
其中,第二目标迁移计算存储节点是指在第二偏移量大于或等于第二阈值时,接收遍历到的计算存储节点迁移出的分片的计算存储节点,初始化为第三排序中第一个计算存储节点,在迁移过程中根据情况进行更新。
第二目标分片是指遍历到的第二偏移量达到第二阈值的计算存储节点需要迁移出去的分片,通过选择合适大小的分片,以减小该计算存储节点的第二偏移量,实现分片总数量的维度的均衡。
下面对步骤2610至步骤2640进行详细描述:
在步骤2610中,将计算存储节点按照第二偏移量由小到大的顺序,进行第三排序。
根据本公开的实施例,第三排序是指基于第二偏移量对计算存储节点进行升序排序,以使第二偏移量最小的排在第一,第二偏移量最大的排在最后。第三排序具体是指基于第二偏移量对计算存储节点进行升序排序后得到的顺序,使第二偏移量最小的计算存储节点排在第一,第二偏移量最大的排在最后,用于在第二偏移量大于或等于第二阈值时,确定接收遍历到的计算存储节点迁移出分片的目标计算存储节点的选择顺序。其中,第二偏移量最小意味着该计算存储节点当前所存储的分片数量最小,可以容纳其它计算存储节点迁移出的最多分片。第二偏移量最大意味着该计算存储节点当前所存储的分片数量最多,需要迁移出最多分片。
示例性的,参照图27,假设分布式系统包括计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点D、计算存储节点E、计算存储节点F。计算存储节点A的第二偏移量为-31,计算存储节点B的第二偏移量为-2,计算存储节点C的第二偏移量为33,计算存储节点D的第二偏移量为60,计算存储节点E的第二偏移量为-28,计算存储节点F的第二偏移量为25。将计算存储节点按照第二偏移量从小到大的顺序进行第三排序,可以得到第三排序为:计算存储节点A、计算存储节点E、计算存储节点B、计算存储节点F、计算存储节点C、计算存储节点D。
在步骤2620中,初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点。
根据本公开的实施例,第二目标迁移计算存储节点是指接收遍历到的第二偏移量达到第二阈值的计算存储节点迁移出的分片的计算存储节点。可以理解的是,第三排序中的第一个计算存储节点可以容纳其它计算存储节点迁移出的最多分片,将第二目标迁移计算存储节点确定为第三排序中第一个计算存储节点,可以使第二目标迁移计算存储节点接收尽可能多的分片。进一步的,在确定第二目标迁移计算存储节点时,需要保证第二目标迁移计算存储节点在接收到遍历到的计算存储节点迁移出的分片之后,第一偏移量小于第一阈值,且第二偏移量仍小于第二阈值。
示例性的,参照图27,假设第三排序为:计算存储节点A、计算存储节点E、计算存储节点B、计算存储节点F、计算存储节点C、计算存储节点D,由此可以初始化第二目标迁移计算存储节点为计算存储节点A。
在步骤2630中,在遍历到的计算存储节点上,选择第二目标分片。
根据本公开的实施例,第二目标分片是指遍历到的第二偏移量达到第二阈值的计算存储节点需要迁移出去的分片。参照图28A可知,计算存储节点存储有不同索引的多个分片,不同索引的多个分片的大小并不相同,且同一个索引的多个分片的大小也不完全相同。存储有负载较大的分片会可能使得该计算存储节点的第二偏移量偏大,自然的,将该分片迁移出计算存储节点也将使得该计算存储节点的第二偏移量明显减小。基于此,在对遍历到的第二偏移量达到第二阈值的计算存储节点中的分片进行迁移时,首先需要选择合适大小的第二目标分片。
在步骤2640中,将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点,当迁移后第二偏移量仍然大于或等于第二阈值,用第三排序中的下一个迁移计算存储节点更新第二目标迁移计算存储节点,返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤,直到迁移后第二偏移量小于第二阈值。
根据本公开的实施例,在确定了第二目标分片和第二目标迁移计算存储节点之后,便可以将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点上。示例性的,参照图28A,假设针对遍历到的索引B,遍历到的计算存储节点D的第二偏移量达到第二阈值,且已经确定得到第二目标迁移计算存储节点为计算存储节点A,第二目标分片为索引B的一个分片。那么,可以将索引B的一个分片从计算存储节点D迁移到计算存储节点A中。图28B为完成分片迁移后的计算存储节点A和计算存储节点D的示意图,可以得到的是,此时计算存储节点D中不再存储有第二目标分片,计算存储节点A中存储有第二目标分片。
进一步的,在将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点之后,遍历到的计算存储节点和第二目标迁移计算存储节点的第二偏移量都将发生变化。此时,当迁移后遍历到的计算存储节点的第二偏移量仍然大于或等于第二阈值,则说明遍历到的计算存储节点还需要再选择第二目标分片来迁移出去。同时,为避免同一个计算存储节点因为持续地接收分片而导致第二偏移量达到第二阈值的情况发生,第二目标迁移计算存储节点也需要随着第二目标分片一同更新。因此,在每次将第二目标分片从遍历到的计算存储节点中迁移出去之后,都需要对第二目标分片和第二目标迁移计算存储节点进行更新和选择。其中,当对第二目标迁移计算存储节点进行更新时,一般会将第三排序中继更新前的第二目标迁移计算存储节点的下一个计算存储节点确定为更新后的第二目标迁移计算存储节点。当对第二目标分片进行更新时,更新后得到的第二目标分片的大小和更新前的第二目标分片的大小是相近的。
根据本公开的实施例,在更新完第二目标分片和第二目标迁移计算存储节点之后,还需返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤,以将更新后的第二目标分片从遍历到的计算存储节点迁移到更新后的第二目标迁移计算存储节点,直到遍历到的计算存储节点迁移后的第二偏移量小于第二阈值。其中,当遍历到的计算存储节点的第二偏移量小于第二阈值,则说明该计算存储节点已经实现了分片总数量的维度的均衡。
在一实施例中,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点,还包括:如果第二目标迁移计算存储节点为第三排序中的最后一个迁移计算存储节点,且迁移后第二偏移量仍然大于或等于第二阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第二目标分片,并用第三排序中第一个计算存储节点更新第二目标迁移计算存储节点,并返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤。
可以理解的是,在第二目标迁移计算存储节点为第三排序中的最后一个计算存储节点的情况下,如果在将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点之后,遍历到的计算存储节点的第二偏移量仍然是大于或等于第二阈值,说明此时该计算存储节点仍未实现分片总数量的维度的均衡。而最有可能的原因是第二目标分片的大小过小。在第二目标分片大小过小的情况下,即使遍历到的计算存储节点执行了多次分片的迁移操作,遍历到的计算存储节点的第二偏移量仍然是大于或等于第二阈值的。所以,此时需要在遍历到的计算存储节点上重新选择较大的第二目标分片,并对各个计算存储节点的第三排序进行更新,用更新后的第三排序中第一个计算存储节点来更新第二目标迁移计算存储节点。
根据本公开的实施例,在重新选择的第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点之后,如果遍历到的计算存储节点的第二偏移量仍然大于或等于第二阈值,则需要用更新后的第三排序中当前第二目标迁移计算存储节点的下一个计算存储节点来更新第二目标迁移计算存储节点。在这之后,循环执行分片迁移以及更新第二目标迁移计算存储节点和第二目标分片的操作,直到被遍历到的计算存储节点的第二偏移量小于第二阈值。
需要说明的是,只有在第二目标迁移计算存储节点为第三排序中的最后一个计算存储节点,且迁移后被遍历的计算存储节点的第二偏移量仍然大于或等于第二阈值的情况下,才需要重新选择第二目标分片。
示例性的,假设针对遍历到的索引B,遍历到的计算存储节点为计算存储节点D的第二偏移量达到第二阈值,且第三排序为计算存储节点A、计算存储节点B、计算存储节点C。在确定得到分片大小为15MB的第二目标分片之后,首先将第二目标分片从计算存储节点D迁移到计算存储节点A。当迁移后计算存储节点D的第二偏移量仍然大于或等于第二阈值,用计算存储节点B更新第二目标迁移计算存储节点。然后,更新第二目标分片,将第二目标分片从计算存储节点D迁移到计算存储节点B。当迁移后计算存储节点D的第二偏移量仍然大于或等于第二阈值,用计算存储节点C更新第二目标迁移计算存储节点。在这之后,更新第二目标分片,将第二目标分片从计算存储节点D迁移到计算存储节点C。当迁移后计算存储节点D的第二偏移量仍然大于或等于第二阈值,在计算存储节点D上未选择过的分片中,重新选择第二目标分片,重新选择的第二目标分片为20MB。假设更新后的第三排序仍为计算存储节点A、计算存储节点B、计算存储节点C。此时可以将重新选择的第二目标分片从计算存储节点D迁移到计算存储节点A,并循环执行分片迁移以及更新第二目标迁移计算存储节点和第二目标分片的操作,直到计算存储节点D的第二偏移量小于第二阈值。
上述步骤2610至步骤2640的实施例通过初始化第二目标迁移计算存储节点为第三排序中的第一个计算存储节点,可以使第二目标迁移计算存储节点接收尽可能多的分片。进一步的,通过选择合适的第二目标分片,并将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点,精确地实现了计算存储节点的分片总数量的维度的均衡。而且,当迁移后第二偏移量仍然大于或等于第二阈值,本公开的实施例还将用第三排序中继更新前的第二目标迁移计算存储节点的下一个计算存储节点作为更新后的第二目标迁移计算存储节点,来执行分片迁移,并循环执行这一步骤,直到迁移后第二偏移量小于第二阈值。通过对第二目标迁移计算存储节点进行更新,避免了同一个计算存储节点因为持续地接收分片而导致第二偏移量达到第二阈值的情况发生,精确地均衡了多个计算存储节点的分片总数量。
在一实施例中,参照图29,步骤1340包括:
步骤2910、将计算存储节点按照第三偏移量由小到大的顺序,进行第四排序;
步骤2920、初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点;
步骤2930、在遍历到的计算存储节点上,选择第三目标分片;
步骤2940、将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点,当迁移后第三偏移量仍然大于或等于第三阈值,用第四排序中的下一个迁移计算存储节点更新第三目标迁移计算存储节点,返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤,直到迁移后第三偏移量小于第三阈值。
其中,第三目标迁移计算存储节点是指在第三偏移量大于或等于第三阈值时,接收遍历到的计算存储节点迁移出的分片的计算存储节点,初始化为第四排序中第一个计算存储节点,在迁移过程中根据情况进行更新。
第三目标分片是指遍历到的第三偏移量达到第三阈值的计算存储节点需要迁移出去的分片,通过选择合适大小的分片,以减小该计算存储节点的第三偏移量,实现存储空间利用率的维度的均衡。
下面对步骤2910至步骤2940进行详细描述。
在步骤2910中,将计算存储节点按照第三偏移量由小到大的顺序,进行第四排序。
根据本公开的实施例,第四排序是指基于第三偏移量对计算存储节点进行升序排序,以使第三偏移量最小的排在第一,第三偏移量最大的排在最后。第四排序具体是指基于第三偏移量对计算存储节点进行升序排序后得到的顺序,使第三偏移量最小的计算存储节点排在第一,第三偏移量最大的排在最后,用于在第三偏移量大于或等于第三阈值时,确定接收遍历到的计算存储节点迁移出分片的目标计算存储节点的选择顺序。其中,第三偏移量最小意味着该计算存储节点当前的存储空间利用率最小,可以容纳其它计算存储节点迁移出的最多分片。第三偏移量最大意味着该计算存储节点当前的存储空间利用率最大,需要迁移出最多分片。
示例性的,参照图30,假设分布式系统包括计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点D、计算存储节点E、计算存储节点F。计算存储节点A的第三偏移量为-2%,计算存储节点B的第三偏移量为0,计算存储节点C的第三偏移量为1%,计算存储节点D的第三偏移量为3%,计算存储节点E的第三偏移量为-4%,计算存储节点F的第三偏移量为2%。将计算存储节点按照第三偏移量从小到大的顺序进行第四排序,可以得到第四排序为:计算存储节点E、计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点F、计算存储节点D。
在步骤2920中,初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点。
根据本公开的实施例,第三目标迁移计算存储节点是指接收遍历到的第三偏移量达到第三阈值的计算存储节点迁移出的分片的计算存储节点。可以理解的是,第四排序中的第一个计算存储节点可以容纳其它计算存储节点迁移出的最多分片,将第三目标迁移计算存储节点确定为第四排序中第一个计算存储节点,可以使第三目标迁移计算存储节点接收尽可能多的分片。进一步的,在确定第三目标迁移计算存储节点时,需要保证第三目标迁移计算存储节点在接收到遍历到的计算存储节点迁移出的分片之后,第一偏移量小于第一阈值,且第二偏移量小于第二阈值,且第三偏移量仍小于第三阈值。
示例性的,参照图30,假设第四排序为:计算存储节点E、计算存储节点A、计算存储节点B、计算存储节点C、计算存储节点F、计算存储节点D,由此可以初始化第三目标迁移计算存储节点为计算存储节点E。
在步骤2930中,在遍历到的计算存储节点上,选择第三目标分片。
根据本公开的实施例,第三目标分片是指遍历到的第三偏移量达到第三阈值的计算存储节点需要迁移出去的分片。参照图31A可知,计算存储节点存储有不同索引的多个分片,不同索引的多个分片的大小并不相同,且同一个索引的多个分片的大小也不完全相同。存储有负载较大的分片会可能使得该计算存储节点的第三偏移量偏大,自然的,将该分片迁移出计算存储节点也将使得该计算存储节点的第三偏移量明显减小。基于此,在对遍历到的第三偏移量达到第三阈值的计算存储节点中的分片进行迁移时,首先需要选择合适大小的第三目标分片。
在步骤2940中,将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点,当迁移后第三偏移量仍然大于或等于第三阈值,用第四排序中的下一个迁移计算存储节点更新第三目标迁移计算存储节点,返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤,直到迁移后第三偏移量小于第三阈值。
根据本公开的实施例,在确定了第三目标分片和第三目标迁移计算存储节点之后,便可以将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点上。示例性的,参照图31A,假设针对遍历到的索引C,遍历到的计算存储节点D的第三偏移量达到第三阈值,且已经确定得到第三目标迁移计算存储节点为计算存储节点E,第三目标分片为索引C的一个分片。那么,可以将索引C的一个分片从计算存储节点D迁移到计算存储节点E中。图31B为完成分片迁移后的计算存储节点E和计算存储节点D的示意图,可以得到的是,此时计算存储节点D中不再存储有第三目标分片,计算存储节点E中存储有第三目标分片。
进一步的,在将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点之后,遍历到的计算存储节点和第三目标迁移计算存储节点的第三偏移量都将发生变化。此时,当迁移后遍历到的计算存储节点的第三偏移量仍然大于或等于第三阈值,则说明遍历到的计算存储节点还需要再选择第三目标分片来迁移出去。同时,为避免同一个计算存储节点因为持续地接收分片而导致第三偏移量达到第三阈值的情况发生,第三目标迁移计算存储节点也需要随着第三目标分片一同更新。因此,在每次将第三目标分片从遍历到的计算存储节点中迁移出去之后,都需要对第三目标分片和第三目标迁移计算存储节点进行更新和选择。其中,当对第三目标迁移计算存储节点进行更新时,一般会将第四排序中继更新前的第三目标迁移计算存储节点的下一个计算存储节点确定为更新后的第三目标迁移计算存储节点。当对第三目标分片进行更新时,更新后得到的第三目标分片的大小和更新前的第三目标分片的大小是相近的。
根据本公开的实施例,在更新完第三目标分片和第三目标迁移计算存储节点之后,还需返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤,以将更新后的第三目标分片从遍历到的计算存储节点迁移到更新后的第三目标迁移计算存储节点,直到遍历到的计算存储节点迁移后的第三偏移量小于第三阈值。其中,当遍历到的计算存储节点的第三偏移量小于第三阈值,则说明该计算存储节点已经实现了存储空间利用率的维度的均衡。
在一实施例中,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点,还包括:如果第三目标迁移计算存储节点为第四排序中的最后一个迁移计算存储节点,且迁移后第三偏移量仍然大于或等于第三阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第三目标分片,并用第四排序中第一个计算存储节点更新第三目标迁移计算存储节点,并返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤。
可以理解的是,在第三目标迁移计算存储节点为第四排序中的最后一个计算存储节点的情况下,如果在将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点之后,遍历到的计算存储节点的第三偏移量仍然是大于或等于第三阈值,说明此时该计算存储节点仍未实现存储空间利用率的维度的均衡。而最有可能的原因是第三目标分片的大小过小。在第三目标分片大小过小的情况下,即使遍历到的计算存储节点执行了多次分片的迁移操作,遍历到的计算存储节点的第三偏移量仍然是大于或等于第三阈值的。所以,此时需要在遍历到的计算存储节点上重新选择较大的第三目标分片,并对各个计算存储节点的第四排序进行更新,用更新后的第四排序中第一个计算存储节点来更新第三目标迁移计算存储节点。
根据本公开的实施例,在重新选择的第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点之后,如果遍历到的计算存储节点的第三偏移量仍然大于或等于第三阈值,则需要用更新后的第四排序中当前第三目标迁移计算存储节点的下一个计算存储节点来更新第三目标迁移计算存储节点。在这之后,循环执行分片迁移以及更新第三目标迁移计算存储节点和第三目标分片的操作,直到被遍历到的计算存储节点的第三偏移量小于第三阈值。
需要说明的是,只有在第三目标迁移计算存储节点为第四排序中的最后一个计算存储节点,且迁移后被遍历的计算存储节点的第三偏移量仍然大于或等于第三阈值的情况下,才需要重新选择第三目标分片。
示例性的,假设针对遍历到的索引C,遍历到的计算存储节点为计算存储节点D的第三偏移量达到第三阈值,且第四排序为计算存储节点A、计算存储节点B、计算存储节点C。在确定得到分片大小为50MB的第三目标分片之后,首先将第三目标分片从计算存储节点D迁移到计算存储节点A。当迁移后计算存储节点D的第三偏移量仍然大于或等于第三阈值,用计算存储节点B更新第三目标迁移计算存储节点。然后,更新第三目标分片,将第三目标分片从计算存储节点D迁移到计算存储节点B。当迁移后计算存储节点D的第三偏移量仍然大于或等于第三阈值,用计算存储节点C更新第三目标迁移计算存储节点。在这之后,更新第三目标分片,将第三目标分片从计算存储节点D迁移到计算存储节点C。当迁移后计算存储节点D的第三偏移量仍然大于或等于第三阈值,在计算存储节点D上未选择过的分片中,重新选择第三目标分片,重新选择的第三目标分片为60MB。假设更新后的第四排序仍为计算存储节点A、计算存储节点B、计算存储节点C。此时可以将重新选择的第三目标分片从计算存储节点D迁移到计算存储节点A,并循环执行分片迁移以及更新第三目标迁移计算存储节点和第三目标分片的操作,直到计算存储节点D的第三偏移量小于第三阈值。
上述步骤2910至步骤2940的实施例通过初始化第三目标迁移计算存储节点为第四排序中的第一个计算存储节点,可以使第三目标迁移计算存储节点接收尽可能多的分片。进一步的,通过选择合适的第三目标分片,并将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点,精确地实现了计算存储节点的存储空间利用率的维度的均衡。而且,当迁移后第三偏移量仍然大于或等于第三阈值,本公开的实施例还将用第四排序中继更新前的第三目标迁移计算存储节点的下一个计算存储节点作为更新后的第三目标迁移计算存储节点,来执行分片迁移,并循环执行这一步骤,直到迁移后第三偏移量小于第三阈值。通过对第三目标迁移计算存储节点进行更新,避免了同一个计算存储节点因为持续地接收分片而导致第三偏移量达到第三阈值的情况发生,精确地均衡了多个计算存储节点的存储空间利用率。
本公开实施例的分布式系统负载均衡方法的实施细节
下面参照图32,详细示例性说明本公开实施例的分布式系统负载均衡方法的实施细节。
在步骤3210中,响应于索引变更请求,在多个计算存储节点上执行索引变更。
在一实施例中,索引变更请求是对目标索引的索引分配请求;响应于索引变更请求,在多个计算存储节点上执行索引变更,包括:将目标索引分成多个分片;将多个分片在多个计算存储节点上进行分配,使得针对目标索引,每个计算存储节点上的第一偏移量小于第一阈值。
在一实施例中,将目标索引分成多个分片,包括:获取每个计算存储节点的存储容量和存储空间利用率;基于每个计算存储节点的存储容量,计算各个计算存储节点的平均存储容量;基于每个计算存储节点的存储空间利用率,计算各个计算存储节点的平均存储空间利用率;基于目标索引的索引大小、平均存储容量和平均存储空间利用率,确定分片数目;基于分片数目,将目标索引进行切分,获得多个分片。
在步骤3220中,确定索引变更后每个计算存储节点的第一偏移量,第一偏移量为计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量;确定索引变更后每个计算存储节点的第二偏移量,第二偏移量为计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定索引变更后每个计算存储节点的第三偏移量,第三偏移量为计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量。
在步骤3231中,对分布式系统中的索引进行遍历。
在步骤3232中,针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序。
在一实施例中,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序,包括:基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序;对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序;对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,以形成第一排序。
在一实施例中,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序,包括:获取第一偏移量的第一权重、第二偏移量的第二权重和第三偏移量的第三权重;基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,其中,第一权重大于第二权重,第二权重大于第三权重;基于加权和由大到小的顺序,对多个计算存储节点进行第一排序。
在步骤3233中,按照第一排序对多个计算存储节点进行遍历,以针对遍历到的计算存储节点,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点。
在一实施例中,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点,包括:将计算存储节点按照第一偏移量由小到大的顺序,进行第二排序;初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点;在遍历到的计算存储节点上,选择第一目标分片;将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点,当迁移后第一偏移量仍然大于或等于第一阈值,用第二排序中的下一个计算存储节点更新第一目标迁移计算存储节点,返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤,直到迁移后第一偏移量小于第一阈值。
在一实施例中,在遍历到的计算存储节点上,选择第一目标分片,包括:获取遍历到的计算存储节点上各个分片的分片大小;基于各个分片的分片大小,确定平均分片大小;确定各个分片的分片大小相对于平均分片大小的第一倍数;选择第一倍数与第一偏移量匹配的分片,作为第一目标分片。
在一实施例中,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点,还包括:当第一目标迁移计算存储节点为第二排序中的最后一个迁移计算存储节点,且迁移后第一偏移量仍然大于或等于第一阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第一目标分片,并用第二排序中第一个计算存储节点更新第一目标迁移计算存储节点,并返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤。
在一实施例中,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点,包括:获取遍历到的计算存储节点上各个分片的分片大小;基于各个分片的分片大小,确定平均分片大小;确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数;计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,待考察计算存储节点为第一偏移量小于遍历到的计算存储节点的计算存储节点;将第二倍数与第一差匹配的其它计算存储节点,作为目标计算存储节点,并将目标分片迁移到目标计算存储节点。
在一实施例中,目标分片通过以下方式从各个分片中确定:确定遍历到的计算存储节点的存储容量;确定遍历到的计算存储节点的存储空间利用率;获取各个分片的分片大小;基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。
在步骤3234中,针对遍历到的计算存储节点,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点。
在一实施例中,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点,包括:将计算存储节点按照第二偏移量由小到大的顺序,进行第三排序;初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点;在遍历到的计算存储节点上,选择第二目标分片;将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点,当迁移后第二偏移量仍然大于或等于第二阈值,用第三排序中的下一个迁移计算存储节点更新第二目标迁移计算存储节点,返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤,直到迁移后第二偏移量小于第二阈值。
在一实施例中,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点,还包括:如果第二目标迁移计算存储节点为第三排序中的最后一个迁移计算存储节点,且迁移后第二偏移量仍然大于或等于第二阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第二目标分片,并用第三排序中第一个计算存储节点更新第二目标迁移计算存储节点,并返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤。
在步骤3235中,针对遍历到的计算存储节点,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点。
在一实施例中,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点,包括:将计算存储节点按照第三偏移量由小到大的顺序,进行第四排序;初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点;在遍历到的计算存储节点上,选择第三目标分片;将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点,当迁移后第三偏移量仍然大于或等于第三阈值,用第四排序中的下一个迁移计算存储节点更新第三目标迁移计算存储节点,返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤,直到迁移后第三偏移量小于第三阈值。
在一实施例中,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点,还包括:如果第三目标迁移计算存储节点为第四排序中的最后一个迁移计算存储节点,且迁移后第三偏移量仍然大于或等于第三阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第三目标分片,并用第四排序中第一个计算存储节点更新第三目标迁移计算存储节点,并返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤。
可以理解的是,虽然上述各个流程图中的各个步骤按照箭头的表征依次显示,但是这些步骤并不是必然按照箭头表征的顺序依次执行。除非本实施例中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,上述流程图中的至少一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时间执行完成,而是可以在不同的时间执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
需要说明的是,在本申请的各个具体实施方式中,当涉及到需要根据对象属性信息或属性信息集合等与对象特性相关的数据进行相关处理时,都会先获得对象的许可或者同意,而且,对这些数据的收集、使用和处理等,都会遵守相关法律法规和标准。此外,当本申请实施例需要获取对象属性信息时,会通过弹窗或者跳转到确认页面等方式获得对象的单独许可或者单独同意,在明确获得对象的单独许可或者单独同意之后,再获取用于使本申请实施例能够正常运行的必要的对象相关数据。
本公开实施例的装置和设备描述
图33为本公开实施例提供的分布式系统负载均衡装置3300的结构示意图。该分布式系统负载均衡装置3300应用于分布式系统中的调度节点,分布式系统包括调度节点和多个计算存储节点,分布式系统中存储的索引分解成分片,存储在多个计算存储节点中,索引是待进行计算和存储的多字段数据体,分布式系统负载均衡装置3300包括:
响应单元3310,用于响应于索引变更请求,在多个计算存储节点上执行索引变更;
第一确定单元3320,用于确定索引变更后每个计算存储节点的第一偏移量,第一偏移量为计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的单个索引的平均分片数量的偏移量;确定索引变更后每个计算存储节点的第二偏移量,第二偏移量为计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定索引变更后每个计算存储节点的第三偏移量,第三偏移量为计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量;
迁移单元3330,用于基于第一偏移量、第二偏移量和第三偏移量,进行多个计算存储节点上分片的迁移,使得迁移后每个计算存储节点上的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。
可选地,索引变更请求是对目标索引的索引分配请求;
响应单元3310具体用于:
将目标索引分成多个分片;
将多个分片在多个计算存储节点上进行分配,使得针对目标索引,每个计算存储节点上的第一偏移量小于第一阈值。
可选地,响应单元3310具体用于:
获取每个计算存储节点的存储容量和存储空间利用率;
基于每个计算存储节点的存储容量,计算各个计算存储节点的平均存储容量;
基于每个计算存储节点的存储空间利用率,计算各个计算存储节点的平均存储空间利用率;
基于目标索引的索引大小、平均存储容量和平均存储空间利用率,确定分片数目;
基于分片数目,将目标索引进行切分,获得多个分片。
可选地,迁移单元3330具体用于:
对分布式系统中的索引进行遍历;
针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点上的分片进行迁移。
可选地,迁移单元3330具体用于:
针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量,对多个计算存储节点进行第一排序;
按照第一排序对多个计算存储节点进行遍历,以针对遍历到的计算存储节点,当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于遍历到的计算存储节点的计算存储节点;
针对遍历到的计算存储节点,当第二偏移量大于或等于第二阈值,将遍历到的计算存储节点上的分片迁移到第二偏移量小于遍历到的计算存储节点的计算存储节点;
针对遍历到的计算存储节点,当第三偏移量大于或等于第三阈值,将遍历到的计算存储节点上的分片迁移到第三偏移量小于遍历到的计算存储节点的计算存储节点。
可选地,迁移单元3330具体用于:
基于第一偏移量由大到小的顺序,对多个计算存储节点进行排序;
对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序;
对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,以形成第一排序。
可选地,迁移单元3330具体用于:
获取第一偏移量的第一权重、第二偏移量的第二权重和第三偏移量的第三权重;
基于第一权重、第二权重和第三权重,对第一偏移量、第二偏移量和第三偏移量进行加权和,其中,第一权重大于第二权重,第二权重大于第三权重;
基于加权和由大到小的顺序,对多个计算存储节点进行第一排序。
可选地,迁移单元3330具体用于:
将计算存储节点按照第一偏移量由小到大的顺序,进行第二排序;
初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点;
在遍历到的计算存储节点上,选择第一目标分片;
将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点,当迁移后第一偏移量仍然大于或等于第一阈值,用第二排序中的下一个计算存储节点更新第一目标迁移计算存储节点,返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤,直到迁移后第一偏移量小于第一阈值。
可选地,迁移单元3330具体用于:
获取遍历到的计算存储节点上各个分片的分片大小;
基于各个分片的分片大小,确定平均分片大小;
确定各个分片的分片大小相对于平均分片大小的第一倍数;
选择第一倍数与第一偏移量匹配的分片,作为第一目标分片。
可选地,迁移单元3330具体用于:
当第一目标迁移计算存储节点为第二排序中的最后一个计算存储节点,且迁移后第一偏移量仍然大于或等于第一阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第一目标分片,并用第二排序中第一个计算存储节点更新第一目标迁移计算存储节点,并返回将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点的步骤。
可选地,迁移单元3330具体用于:
获取遍历到的计算存储节点上各个分片的分片大小;
基于各个分片的分片大小,确定平均分片大小;
确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数;
计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,待考察计算存储节点为第一偏移量小于遍历到的计算存储节点的计算存储节点;
将第二倍数与第一差匹配的其它计算存储节点,作为目标计算存储节点,并将目标分片迁移到目标计算存储节点。
可选地,迁移单元3330具体用于:
确定遍历到的计算存储节点的存储容量;
确定遍历到的计算存储节点的存储空间利用率;
获取各个分片的分片大小;
基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。
可选地,迁移单元3330具体用于:
将计算存储节点按照第二偏移量由小到大的顺序,进行第三排序;
初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点;
在遍历到的计算存储节点上,选择第二目标分片;
将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点,当迁移后第二偏移量仍然大于或等于第二阈值,用第三排序中的下一个迁移计算存储节点更新第二目标迁移计算存储节点,返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤,直到迁移后第二偏移量小于第二阈值。
可选地,迁移单元3330具体用于:
如果第二目标迁移计算存储节点为第三排序中的最后一个计算存储节点,且迁移后第二偏移量仍然大于或等于第二阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第二目标分片,并用第三排序中第一个计算存储节点更新第二目标迁移计算存储节点,并返回将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点的步骤。
可选地,迁移单元3330具体用于:
将计算存储节点按照第三偏移量由小到大的顺序,进行第四排序;
初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点;
在遍历到的计算存储节点上,选择第三目标分片;
将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点,当迁移后第三偏移量仍然大于或等于第三阈值,用第四排序中的下一个迁移计算存储节点更新第三目标迁移计算存储节点,返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤,直到迁移后第三偏移量小于第三阈值。
可选地,迁移单元3330具体用于:
如果第三目标迁移计算存储节点为第四排序中的最后一个计算存储节点,且迁移后第三偏移量仍然大于或等于第三阈值,在遍历到的计算存储节点上未选择过的分片中,重新选择第三目标分片,并用第四排序中第一个计算存储节点更新第三目标迁移计算存储节点,并返回将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点的步骤。
参照图34,图34为实现本公开实施例的分布式系统负载均衡方法的终端的部分的结构框图,该终端包括:射频(Radio Frequency,简称RF)电路3410、存储器3415、输入单元3430、显示单元3440、传感器3450、音频电路3460、无线保真(wireless fidelity,简称WiFi)模块3470、处理器3480、以及电源3490等部件。本领域技术人员可以理解,图34示出的终端结构并不构成对手机或电脑的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
RF电路3410可用于收发信息或通话过程中,信号的接收和发送,特别地,将基站的下行信息接收后,给处理器3480处理;另外,将设计上行的数据发送给基站。
存储器3415可用于存储软件程序以及模块,处理器3480通过运行存储在存储器3415的软件程序以及模块,从而执行内容终端的各种功能应用以及数据处理。
输入单元3430可用于接收输入的数字或字符信息,以及产生与内容终端的设置以及功能控制有关的键信号输入。具体地,输入单元3430可包括触控面板3431以及其他输入装置3432。
显示单元3440可用于显示输入的信息或提供的信息以及内容终端的各种菜单。显示单元3440可包括显示面板3441。
音频电路3460、扬声器3461,传声器3462可提供音频接口。
在本实施例中,该终端所包括的处理器3480可以执行前面实施例的分布式系统负载均衡方法。
本公开实施例的终端包括但不限于手机、电脑、智能语音交互设备、智能家电、车载终端、飞行器等。本发明实施例可应用于各种场景,包括但不限于分布式应用程序、分布式检索分析引擎等。
图35为实施本公开实施例的分布式系统负载均衡方法的服务器的部分的结构框图。服务器可因配置或性能不同而产生比较大的差异,可以包括一个或一个以上中央处理器(Central Processing Units,简称CPU)3522(例如,一个或一个以上处理器)和存储器3532,一个或一个以上存储应用程序3542或数据3544的存储介质3530(例如一个或一个以上海量存储装置)。其中,存储器3532和存储介质3530可以是短暂存储或持久存储。存储在存储介质3530的程序可以包括一个或一个以上模块(图示没标出),每个模块可以包括对服务器中的一系列指令操作。更进一步地,中央处理器3522可以设置为与存储介质3530通信,在服务器上执行存储介质3530中的一系列指令操作。
服务器还可以包括一个或一个以上电源3526,一个或一个以上有线或无线网络接口3550,一个或一个以上输入输出接口3558,和/或,一个或一个以上操作系统3541,例如Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM等等。
服务器中的中央处理器3522可以用于执行本公开实施例的分布式系统负载均衡方法。
本公开实施例还提供一种计算机可读存储介质,计算机可读存储介质用于存储程序代码,程序代码用于执行前述各个实施例的分布式系统负载均衡方法。
本公开实施例还提供了一种计算机程序产品,该计算机程序产品包括计算机程序。计算机设备的处理器读取该计算机程序并执行,使得该计算机设备执行实现上述的分布式系统负载均衡方法。
综上所述,本申请提供一种分布式系统负载均衡方法、装置、电子设备、计算机可读存储介质和计算机程序产品。在分布式系统中,调度节点起着关键的协调作用。当调度节点接收到索引变更请求时,这意味着系统中的索引数据需要进行调整,可能是新增索引、删除索引或者迁移分片等操作。此时,调度节点会在多个计算存储节点上执行索引变更。索引变更后,各计算存储节点存储的分片数目会发生改变,这种改变可能会打破原有的负载均衡状态。
为了降低因集中存储大分片而降低处理请求效率的情况发生,调度节点会确定每个计算存储节点的第一偏移量、第二偏移量和第三偏移量。第一偏移量反映了计算存储节点存储的单个索引的分片数量与各个计算存储节点存储的该单个索引的平均分片数量的差异;第二偏移量体现了计算存储节点存储的分片总数量与各个计算存储节点存储的平均分片总数量的偏差;第三偏移量则表示计算存储节点的存储空间利用率与各个计算存储节点的平均存储空间利用率的偏离程度。通过这三个偏移量,调度节点能够全面、精准地了解每个计算存储节点与系统整体状态的差异。
基于这些偏移量,调度节点进行分片迁移,使迁移后每个计算存储节点的第一偏移量小于第一阈值,第二偏移量小于第二阈值,第三偏移量小于第三阈值。从技术层面来看,在单个索引的分片数量维度上,当第一偏移量小于第一阈值时,避免了某些计算存储节点集中存储某个索引的大量分片,使得各个计算存储节点在处理该索引相关请求时的负载更加均衡,减少了因某个节点处理压力过大而导致的处理延迟,提高了处理请求的效率。在分片总数量维度上,第二偏移量小于第二阈值保证了各个计算存储节点的整体负载均衡,避免了部分节点因存储过多分片而成为系统瓶颈,提高了系统的整体处理能力。在存储空间利用率维度上,第三偏移量小于第三阈值使得各个计算存储节点的存储空间得到合理利用,避免了某些节点存储空间过度使用而其他节点存在大量空闲空间的情况,提高了系统的资源利用率。综合来看,这种负载均衡方式避免了大分片集中存储,提高了各计算存储节点处理请求的效率,也避免了部分节点因存储过多分片而效率降低的情况,提高了分布式系统的运行效率。
进一步地,当索引变更请求是对目标索引的索引分配请求时,调度节点会将目标索引分成多个分片,并在多个计算存储节点上进行分配,使针对目标索引,每个计算存储节点上的第一偏移量小于第一阈值。在分布式系统中,目标索引的分配是一个关键操作,如果分配不合理,可能会导致某些计算存储节点承担过多的负载。通过这种分配方式,调度节点能够确保每个计算存储节点在处理目标索引时的负载相对均衡。从技术角度分析,这使得各个计算存储节点在处理目标索引相关的计算和存储任务时,能够以相近的速度和效率完成,避免了因某个节点负载过重而影响整个系统对目标索引的处理性能,保障了计算存储节点存储的单个索引的分片数量的维度的均衡,实现了分布式系统的负载均衡,提高了系统的响应速度和处理能力。
在将目标索引分成多个分片时,调度节点会获取每个计算存储节点的存储容量和存储空间利用率,计算出各个计算存储节点的平均存储容量和平均存储空间利用率,再基于目标索引的索引大小、平均存储容量和平均存储空间利用率确定分片数目,最后基于分片数目将目标索引进行切分,获得多个分片。获取每个计算存储节点的存储容量和存储空间利用率,能够让调度节点了解每个节点的实际存储能力和使用情况。计算平均存储容量和平均存储空间利用率则为确定合理的分片数目提供了参考依据。基于这些信息确定的分片数目,能够更好地适应各个计算存储节点的实际情况,使得目标索引的分片能够更加均匀地分布在各个节点上。从技术层面来说,这种方式能更好地了解当前分布式系统的状态,在给计算存储节点分配分片时,实现了计算存储节点存储的单个索引的分片数量的维度的均衡,提高了系统的资源利用率和数据处理效率,避免了因分片分配不合理而导致的部分节点存储压力过大或部分节点资源闲置的问题。
基于第一偏移量、第二偏移量和第三偏移量进行多个计算存储节点上分片的迁移时,调度节点会对分布式系统中的索引进行遍历,针对遍历到的索引,基于这三个偏移量对多个计算存储节点上的分片进行迁移。在分布式系统中,索引数量众多且分布在不同的计算存储节点上,每个索引的分片情况也各不相同。通过对索引进行遍历,调度节点能够全面地检查每个索引在各个计算存储节点上的分布情况。基于三个偏移量进行分片迁移,能够根据每个计算存储节点的实际负载情况进行精准调整。从技术角度来看,这种方式精确地实现了计算存储节点在单个索引的分片数量、分片总数量和存储空间利用率三个维度的均衡。在单个索引层面,确保了每个索引的分片在各个计算存储节点上的分布更加合理,提高了对单个索引的处理效率;在分片总数量层面,使得各个计算存储节点的整体负载更加均衡,避免了部分节点因承担过多分片而导致的性能下降;在存储空间利用率层面,优化了各个计算存储节点的存储空间使用情况,提高了系统的资源利用率。
针对遍历到的索引,基于第一偏移量、第二偏移量和第三偏移量对多个计算存储节点进行第一排序后,调度节点会按照第一排序对多个计算存储节点进行遍历。当某个计算存储节点的第一偏移量大于或等于第一阈值时,会将该节点上的分片迁移到第一偏移量小于该节点的计算存储节点。第一排序的目的是为了确定分片迁移的优先级和顺序,使得负载不均衡的计算存储节点能够优先进行调整。当第一偏移量大于或等于第一阈值时,说明该节点在单个索引的分片数量上存在较大偏差,可能会影响该索引的处理效率。将该节点上的分片迁移到第一偏移量小于该节点的计算存储节点,能够快速地调整该索引在各个计算存储节点上的分布,实现了计算存储节点存储的单个索引的分片数量的维度的均衡。从技术层面分析,这种方式能够及时发现并解决单个索引分片分布不均衡的问题,提高了系统对单个索引的处理性能,减少了因单个索引处理延迟而对整个系统造成的影响。
在对多个计算存储节点进行第一排序时,一种方式是基于第一偏移量由大到小的顺序对多个计算存储节点进行排序,对于第一偏移量相等的计算存储节点,基于第二偏移量由大到小的顺序进行排序,对于第二偏移量相等的计算存储节点,基于第三偏移量由大到小的顺序进行排序,以形成第一排序。这种排序方式充分考虑了三个偏移量的重要性和优先级。首先按照第一偏移量排序,能够优先处理单个索引分片数量偏差较大的节点,因为这对单个索引的处理效率影响最为直接。对于第一偏移量相等的节点,再按照第二偏移量排序,进一步调整分片总数量的均衡性。对于第二偏移量也相等的节点,按照第三偏移量排序,优化存储空间利用率。从技术角度来看,这种排序方式方便了后续在不同偏移量达到阈值时,将分片迁移到合适的计算存储节点,提高了分片迁移的效率。它能够让调度节点更加有针对性地进行负载均衡调整,使得系统在多个维度上的负载更加均衡,提高了系统的整体性能和稳定性。
另一种对多个计算存储节点进行第一排序的方式是,获取第一偏移量的第一权重、第二偏移量的第二权重和第三偏移量的第三权重,基于这些权重对三个偏移量进行加权和,其中第一权重大于第二权重,第二权重大于第三权重,再基于加权和由大到小的顺序对多个计算存储节点进行第一排序。这种方式通过设置不同的权重,更加灵活地考虑了三个偏移量对系统负载均衡的影响程度。第一权重大于第二权重和第三权重,突出了单个索引的分片数量在负载均衡中的重要性,因为它直接影响到单个索引的处理效率。通过加权和的方式,能够综合考虑三个偏移量的信息,得到一个更能反映计算存储节点整体负载情况的指标。基于加权和进行排序,使得需要迁移出最多分片的计算存储节点排在最前面,使可以容纳其它计算存储节点迁移出的最多分片的计算存储节点排在最后面。从技术层面来说,这种排序方式便于后续将需要迁移分片的计算存储节点所存储的分片迁移到合适的节点,提高了分片迁移的准确性和效率,能够更有效地实现系统的负载均衡。
当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于该节点的计算存储节点时,调度节点会将计算存储节点按照第一偏移量由小到大的顺序进行第二排序,初始化第一目标迁移计算存储节点为第二排序中第一个计算存储节点,在遍历到的计算存储节点上选择第一目标分片,将第一目标分片从遍历到的计算存储节点迁移到第一目标迁移计算存储节点。当迁移后第一偏移量仍然大于或等于第一阈值,会用第二排序中的下一个计算存储节点更新第一目标迁移计算存储节点,直到迁移后第一偏移量小于第一阈值。第二排序的目的是为了确定接收分片的计算存储节点的优先级,将第一偏移量小的节点排在前面,使得这些节点能够优先接收分片,从而更好地平衡各个计算存储节点的单个索引分片数量。选择合适的第一目标分片进行迁移,能够更有效地降低遍历到的计算存储节点的第一偏移量。通过不断更新第一目标迁移计算存储节点,避免了同一个计算存储节点因为持续地接收分片而导致第一偏移量达到第一阈值的情况发生。从技术角度来看,这种方式精确地实现了计算存储节点存储的单个索引的分片数量的维度的均衡,提高了系统对单个索引的处理效率,减少了因单个索引分片分布不均衡而导致的处理延迟。
在遍历到的计算存储节点上选择第一目标分片时,调度节点会获取该节点上各个分片的分片大小,基于这些分片大小确定平均分片大小,确定各个分片的分片大小相对于平均分片大小的第一倍数,选择第一倍数与第一偏移量匹配的分片作为第一目标分片。获取各个分片的分片大小能够让调度节点了解每个分片的实际负载情况。确定平均分片大小和第一倍数,为选择合适的分片提供了量化的依据。选择第一倍数与第一偏移量匹配的分片作为第一目标分片,能够确保迁移的分片大小与需要调整的偏移量相适应。从技术层面分析,这种选择方式可以减少执行分片迁移的次数,提高分片迁移的效率,同时也提高了实现计算存储节点的单个索引的分片数量的维度的均衡的速度。因为选择合适大小的分片进行迁移,能够更快速地达到降低第一偏移量的目的,减少了不必要的迁移操作,提高了系统的处理效率。
当第一目标迁移计算存储节点为第二排序中的最后一个计算存储节点,且迁移后第一偏移量仍然大于或等于第一阈值时,调度节点会在遍历到的计算存储节点上未选择过的分片中,重新选择第一目标分片,并用第二排序中第一个计算存储节点更新第一目标迁移计算存储节点,继续进行分片迁移,直到迁移后第一偏移量小于第一阈值。这种情况说明之前选择的第一目标分片可能过小,无法有效降低遍历到的计算存储节点的第一偏移量。通过重新选择第一目标分片和更新第一目标迁移计算存储节点,能够继续尝试调整该节点的单个索引分片数量。从技术角度来看,这种处理方式确保了在各种情况下都能实现计算存储节点存储的单个索引的分片数量的维度的均衡,提高了系统的容错能力和稳定性,避免了因分片选择不当而导致的负载均衡调整失败的问题。
当第一偏移量大于或等于第一阈值,将遍历到的计算存储节点上的分片迁移到第一偏移量小于该节点的计算存储节点的另一种方式是,调度节点会获取遍历到的计算存储节点上各个分片的分片大小,基于这些分片大小确定平均分片大小,确定各个分片中的目标分片的分片大小相对于平均分片大小的第二倍数,计算遍历到的计算存储节点的第一偏移量与每个待考察计算存储节点的第一偏移量的第一差,将第二倍数与第一差匹配的其它计算存储节点作为目标计算存储节点,并将目标分片迁移到目标计算存储节点。获取分片大小、确定平均分片大小和第二倍数,能够对各个分片的负载情况进行量化分析。计算第一差能够了解遍历到的计算存储节点与其他节点在单个索引分片数量上的差异。将第二倍数与第一差匹配,能够选择出最适合接收目标分片的计算存储节点。从技术层面来说,这种方式可以减少执行分片迁移的次数,提高分片迁移的效率。因为通过精准匹配,能够一次性将目标分片迁移到最合适的节点,避免了多次迁移和调整,提高了系统的处理效率和负载均衡效果。
目标分片的确定方式是,调度节点会确定遍历到的计算存储节点的存储容量和存储空间利用率,获取各个分片的分片大小,基于存储容量、存储空间利用率和各个分片的分片大小,在各个分片中确定目标分片。如果该计算存储节点的剩余存储空间较小,会选择一个分片大小较大的目标分片从遍历到的计算存储节点中迁移出去;如果该计算存储节点的剩余存储空间较大,会选择一个分片大小较小的目标分片从遍历到的计算存储节点中迁移出去。确定存储容量和存储空间利用率能够了解计算存储节点的实际存储情况。根据剩余存储空间的大小选择合适的目标分片,能够更好地平衡各个计算存储节点的存储空间使用情况。从技术角度分析,这种确定方式可以准确地确定需要从遍历到的计算存储节点中迁移出去的目标分片,减少了执行分片迁移的次数,提高了分片迁移的效率。它能够避免因迁移的分片大小不合适而导致的存储空间浪费或节点负载不均衡的问题,提高了系统的资源利用率和稳定性。
当第二偏移量大于或等于第二阈值时,调度节点会将计算存储节点按照第二偏移量由小到大的顺序进行第三排序,初始化第二目标迁移计算存储节点为第三排序中第一个计算存储节点,在遍历到的计算存储节点上选择第二目标分片,将第二目标分片从遍历到的计算存储节点迁移到第二目标迁移计算存储节点。当迁移后第二偏移量仍然大于或等于第二阈值,会用第三排序中的下一个迁移计算存储节点更新第二目标迁移计算存储节点,直到迁移后第二偏移量小于第二阈值。第三排序的目的是为了确定接收分片的计算存储节点的优先级,将第二偏移量小的节点排在前面,使得这些节点能够优先接收分片,从而更好地平衡各个计算存储节点的分片总数量。选择合适的第二目标分片进行迁移,能够更有效地降低遍历到的计算存储节点的第二偏移量。通过不断更新第二目标迁移计算存储节点,避免了同一个计算存储节点因为持续地接收分片而导致第二偏移量达到第二阈值的情况发生。从技术角度来看,这种方式精确地实现了计算存储节点的分片总数量的维度的均衡,提高了系统的整体处理能力,避免了部分节点因承担过多分片而成为系统瓶颈的问题。
如果第二目标迁移计算存储节点为第三排序中的最后一个迁移计算存储节点,且迁移后第二偏移量仍然大于或等于第二阈值,调度节点会在遍历到的计算存储节点上未选择过的分片中,重新选择第二目标分片,并用第三排序中第一个计算存储节点更新第二目标迁移计算存储节点,继续进行分片迁移,直到迁移后第二偏移量小于第二阈值。这种情况说明之前选择的第二目标分片可能过小,无法有效降低遍历到的计算存储节点的第二偏移量。通过重新选择第二目标分片和更新第二目标迁移计算存储节点,能够继续尝试调整该节点的分片总数量。从技术角度来看,这种处理方式确保了在各种情况下都能实现计算存储节点的分片总数量的维度的均衡,提高了系统的容错能力和稳定性,避免了因分片选择不当而导致的负载均衡调整失败的问题。
当第三偏移量大于或等于第三阈值时,调度节点会将计算存储节点按照第三偏移量由小到大的顺序进行第四排序,初始化第三目标迁移计算存储节点为第四排序中第一个计算存储节点,在遍历到的计算存储节点上选择第三目标分片,将第三目标分片从遍历到的计算存储节点迁移到第三目标迁移计算存储节点。当迁移后第三偏移量仍然大于或等于第三阈值,会用第四排序中的下一个迁移计算存储节点更新第三目标迁移计算存储节点,直到迁移后第三偏移量小于第三阈值。第四排序的目的是为了确定接收分片的计算存储节点的优先级,将第三偏移量小的节点排在前面,使得这些节点能够优先接收分片,从而更好地平衡各个计算存储节点的存储空间利用率。选择合适的第三目标分片进行迁移,能够更有效地降低遍历到的计算存储节点的第三偏移量。通过不断更新第三目标迁移计算存储节点,避免了同一个计算存储节点因为持续地接收分片而导致第三偏移量达到第三阈值的情况发生。从技术角度来看,这种方式精确地实现了计算存储节点的存储空间利用率的维度的均衡,提高了系统的资源利用率,避免了部分节点因存储空间过度使用而影响系统性能的问题。
如果第三目标迁移计算存储节点为第四排序中的最后一个迁移计算存储节点,且迁移后第三偏移量仍然大于或等于第三阈值,调度节点会在遍历到的计算存储节点上未选择过的分片中,重新选择第三目标分片,并用第四排序中第一个计算存储节点更新第三目标迁移计算存储节点,继续进行分片迁移,直到迁移后第三偏移量小于第三阈值。这种情况说明之前选择的第三目标分片可能过小,无法有效降低遍历到的计算存储节点的第三偏移量。通过重新选择第三目标分片和更新第三目标迁移计算存储节点,能够继续尝试调整该节点的存储空间利用率。从技术角度来看,这种处理方式确保了在各种情况下都能实现计算存储节点的存储空间利用率的维度的均衡,提高了系统的容错能力和稳定性,避免了因分片选择不当而导致的存储空间使用不均衡的问题。
本公开的说明书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的内容,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本公开的实施例例如能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“包含”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或装置不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或装置固有的其它步骤或单元。
应当理解,在本公开中,“至少一个(项)”是指一个或者多个,“多个”是指两个或两个以上。“和/或”,用于描述关联内容的关联关系,表示可以存在三种关系,例如,“A和/或B”可以表示:只存在A,只存在B以及同时存在A和B三种情况,其中A,B可以是单数或者复数。字符“/”一般表示前后关联内容是一种“或”的关系。“以下至少一项(个)”或其类似表达,是指这些项中的任意组合,包括单项(个)或复数项(个)的任意组合。例如,a,b或c中的至少一项(个),可以表示:a,b,c,“a和b”,“a和c”,“b和c”,或“a和b和c”,其中a,b,c可以是单个,也可以是多个。
应了解,在本公开实施例的描述中,多个(或多项)的含义是两个以上,大于、小于、超过等理解为不包括本数,以上、以下、以内等理解为包括本数。
在本公开所提供的几个实施例中,应该理解到,所揭露的系统,装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本公开各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机装置(可以是个人计算机,服务器130,或者网络装置等)执行本公开各个实施例方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,简称ROM)、随机存取存储器(Random Access Memory,简称RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对发明专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。
Claims (18)
- 一种分布式系统负载均衡方法,应用于分布式系统中的调度节点,所述分布式系统包括所述调度节点和多个计算存储节点,所述分布式系统中存储的索引分解成分片,存储在所述多个计算存储节点中,所述索引是待进行计算和存储的多字段数据体,所述分布式系统负载均衡方法包括:响应于索引变更请求,在所述多个计算存储节点上执行索引变更;确定所述索引变更后每个所述计算存储节点的第一偏移量,所述第一偏移量为所述计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的所述单个索引的平均分片数量的偏移量;确定所述索引变更后每个所述计算存储节点的第二偏移量,所述第二偏移量为所述计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定所述索引变更后每个所述计算存储节点的第三偏移量,所述第三偏移量为所述计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量;及基于所述第一偏移量、所述第二偏移量和所述第三偏移量,进行所述多个计算存储节点上所述分片的迁移,使得迁移后每个所述计算存储节点上的所述第一偏移量小于第一阈值,所述第二偏移量小于第二阈值,所述第三偏移量小于第三阈值。
- 根据权利要求1所述的分布式系统负载均衡方法,所述索引变更请求是对目标索引的索引分配请求;所述响应于索引变更请求,在所述多个计算存储节点上执行索引变更,包括:将所述目标索引分成多个所述分片;将多个所述分片在所述多个计算存储节点上进行分配,使得针对所述目标索引,每个所述计算存储节点上的所述第一偏移量小于所述第一阈值。
- 根据权利要求2所述的分布式系统负载均衡方法,所述将所述目标索引分成多个所述分片,包括:获取每个所述计算存储节点的所述存储容量和所述存储空间利用率;基于每个所述计算存储节点的所述存储容量,计算各个所述计算存储节点的平均存储容量;基于每个所述计算存储节点的所述存储空间利用率,计算各个所述计算存储节点的平均存储空间利用率;基于所述目标索引的索引大小、所述平均存储容量和所述平均存储空间利用率,确定分片数目;基于所述分片数目,将所述目标索引进行切分,获得多个所述分片。
- 根据权利要求1至3任一项所述的分布式系统负载均衡方法,所述基于所述第一偏移量、所述第二偏移量和所述第三偏移量,进行所述多个计算存储节点上所述分片的迁移,包括:对所述分布式系统中的索引进行遍历;针对遍历到的索引,基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点上的所述分片进行迁移。
- 根据权利要求4所述的分布式系统负载均衡方法,所述针对遍历到的索引,基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点上的所述分片进行迁移,包括:针对遍历到的索引,基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点进行第一排序;按照所述第一排序对所述多个计算存储节点进行遍历,以针对遍历到的所述计算存储节点,当所述第一偏移量大于或等于所述第一阈值,将遍历到的所述计算存储节点上的分片迁移到所述第一偏移量小于所述遍历到的计算存储节点的所述计算存储节点。
- 根据权利要求5所述的分布式系统负载均衡方法,所述针对遍历到的索引,基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点上的所述分片进行迁移还包括:针对遍历到的所述计算存储节点,当所述第二偏移量大于或等于所述第二阈值,将遍历到的所述计算存储节点上的分片迁移到所述第二偏移量小于所述遍历到的计算存储节点的所述计算存储节点。
- 根据权利要求5或6所述的分布式系统负载均衡方法,所述针对遍历到的索引,基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点上的所述分片进行迁移还包括:针对遍历到的所述计算存储节点,当所述第三偏移量大于或等于所述第三阈值,将遍历到的所述计算存储节点上的分片迁移到所述第三偏移量小于所述遍历到的计算存储节点的所述计算存储节点。
- 根据权利要求5至7任一项所述的分布式系统负载均衡方法,所述基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点进行第一排序,包括:基于所述第一偏移量由大到小的顺序,对所述多个计算存储节点进行排序;对于所述第一偏移量相等的所述计算存储节点,基于所述第二偏移量由大到小的顺序进行排序;对于所述第二偏移量相等的所述计算存储节点,基于所述第三偏移量由大到小的顺序进行排序,以形成所述第一排序。
- 根据权利要求5至7任一项所述的分布式系统负载均衡方法,所述基于所述第一偏移量、所述第二偏移量和所述第三偏移量,对所述多个计算存储节点进行第一排序,包括:获取所述第一偏移量的第一权重、所述第二偏移量的第二权重和所述第三偏移量的第三权重;基于所述第一权重、所述第二权重和所述第三权重,对所述第一偏移量、所述第二偏移量和所述第三偏移量进行加权和,其中,所述第一权重大于所述第二权重,所述第二权重大于所述第三权重;基于所述加权和由大到小的顺序,对所述多个计算存储节点进行第一排序。
- 根据权利要求5至9任一项所述的分布式系统负载均衡方法,所述当所述第一偏移量大于或等于所述第一阈值,将遍历到的所述计算存储节点上的分片迁移到所述第一偏移量小于所述遍历到的计算存储节点的所述计算存储节点,包括:将所述计算存储节点按照所述第一偏移量由小到大的顺序,进行第二排序;初始化第一目标迁移计算存储节点为所述第二排序中第一个计算存储节点;在遍历到的所述计算存储节点上,选择第一目标分片;将所述第一目标分片从遍历到的所述计算存储节点迁移到所述第一目标迁移计算存储节点,当迁移后所述第一偏移量仍然大于或等于所述第一阈值,用所述第二排序中的下一个计算存储节点更新所述第一目标迁移计算存储节点,返回所述将所述第一目标分片从遍历到的所述计算存储节点迁移到所述第一目标迁移计算存储节点的步骤,直到所述迁移后所述第一偏移量小于所述第一阈值。
- 根据权利要求10所述的分布式系统负载均衡方法,所述在遍历到的所述计算存储节点上,选择第一目标分片,包括:获取遍历到的所述计算存储节点上各个分片的分片大小;基于所述各个分片的所述分片大小,确定平均分片大小;确定各个分片的所述分片大小相对于所述平均分片大小的第一倍数;选择所述第一倍数与所述第一偏移量匹配的所述分片,作为所述第一目标分片。
- 根据权利要求10或11所述的分布式系统负载均衡方法,所述当所述第一偏移量大于或等于所述第一阈值,将遍历到的所述计算存储节点上的分片迁移到所述第一偏移量小于所述遍历到的计算存储节点的所述计算存储节点,还包括:当所述第一目标迁移计算存储节点为所述第二排序中的最后一个计算存储节点,且迁移后所述第一偏移量仍然大于或等于所述第一阈值,在遍历到的所述计算存储节点上未选择过的分片中,重新选择所述第一目标分片,并用所述第二排序中第一个计算存储节点更新所述第一目标迁移计算存储节点,并返回所述将所述第一目标分片从遍历到的所述计算存储节点迁移到所述第一目标迁移计算存储节点的步骤。
- 根据权利要求5至9任一项所述的分布式系统负载均衡方法,所述当所述第一偏移量大于或等于所述第一阈值,将遍历到的所述计算存储节点上的分片迁移到所述第一偏移量小于所述遍历到的计算存储节点的所述计算存储节点,包括:获取遍历到的所述计算存储节点上各个分片的分片大小;基于所述各个分片的所述分片大小,确定平均分片大小;确定各个分片中的目标分片的所述分片大小相对于所述平均分片大小的第二倍数;计算遍历到的所述计算存储节点的所述第一偏移量与每个待考察计算存储节点的所述第一偏移量的第一差,所述待考察计算存储节点为所述第一偏移量小于所述遍历到的计算存储节点的所述计算存储节点;将所述第二倍数与所述第一差匹配的其它计算存储节点,作为目标计算存储节点,并将所述目标分片迁移到所述目标计算存储节点。
- 根据权利要求13所述的分布式系统负载均衡方法,所述目标分片通过以下方式从所述各个分片中确定:确定遍历到的所述计算存储节点的存储容量;确定遍历到的所述计算存储节点的存储空间利用率;获取所述各个分片的分片大小;基于所述存储容量、所述存储空间利用率、和所述各个分片的所述分片大小,在所述各个分片中确定所述目标分片。
- 一种分布式系统负载均衡装置,应用于分布式系统中的调度节点,所述分布式系统包括所述调度节点和多个计算存储节点,所述分布式系统中存储的索引分解成分片,存储在所述多个计算存储节点中,所述索引是待进行计算和存储的多字段数据体,所述分布式系统负载均衡装置包括:响应单元,用于响应于索引变更请求,在所述多个计算存储节点上执行索引变更;第一确定单元,用于确定所述索引变更后每个所述计算存储节点的第一偏移量,所述第一偏移量为所述计算存储节点存储的单个索引的分片数量相对于各个计算存储节点存储的所述单个索引的平均分片数量的偏移量;确定所述索引变更后每个所述计算存储节点的第二偏移量,所述第二偏移量为所述计算存储节点存储的分片总数量相对于各个计算存储节点存储的平均分片总数量的偏移量;确定所述索引变更后每个所述计算存储节点的第三偏移量,所述第三偏移量为所述计算存储节点的存储空间利用率相对于各个计算存储节点的平均存储空间利用率的偏移量;及迁移单元,用于基于所述第一偏移量、所述第二偏移量和所述第三偏移量,进行所述多个计算存储节点上所述分片的迁移,使得迁移后每个所述计算存储节点上的所述第一偏移量小于第一阈值,所述第二偏移量小于第二阈值,所述第三偏移量小于第三阈值。
- 一种电子设备,包括存储器和处理器,所述存储器存储有计算机程序,所述处理器执行所述计算机程序时实现权利要求1至14任意一项所述的分布式系统负载均衡方法。
- 一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序被处理器执行时实现权利要求1至14任意一项所述的分布式系统负载均衡方法。
- 一种计算机程序产品,该计算机程序产品包括计算机程序,所述计算机程序被计算机设备的处理器读取并执行,使得该计算机设备执行权利要求1至14任意一项所述的分布式系统负载均衡方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410997429.6 | 2024-07-24 | ||
| CN202410997429.6A CN121455651A (zh) | 2024-07-24 | 2024-07-24 | 分布式系统负载均衡方法、装置、电子设备及介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2026020992A1 true WO2026020992A1 (zh) | 2026-01-29 |
Family
ID=98564888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2025/098749 Pending WO2026020992A1 (zh) | 2024-07-24 | 2025-06-03 | 分布式系统负载均衡方法、装置、电子设备及介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN121455651A (zh) |
| WO (1) | WO2026020992A1 (zh) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106844399A (zh) * | 2015-12-07 | 2017-06-13 | 中兴通讯股份有限公司 | 分布式数据库系统及其自适应方法 |
| US20190268405A1 (en) * | 2018-02-28 | 2019-08-29 | International Business Machines Corporation | Load balancing with power of random choices |
| CN112181605A (zh) * | 2020-10-27 | 2021-01-05 | 北京城市网邻信息技术有限公司 | 负载均衡方法、装置、电子设备和计算机可读介质 |
| CN113177050A (zh) * | 2021-05-18 | 2021-07-27 | 浙江大华技术股份有限公司 | 一种数据均衡的方法、装置、查询系统及存储介质 |
| CN114647701A (zh) * | 2022-03-22 | 2022-06-21 | 北京百度网讯科技有限公司 | 分布式数据库的负载均衡方法、装置、电子设备及介质 |
| CN116955087A (zh) * | 2023-08-04 | 2023-10-27 | 北京中科闻歌科技股份有限公司 | 数据分片调整方法、装置、设备及可读存储介质 |
-
2024
- 2024-07-24 CN CN202410997429.6A patent/CN121455651A/zh active Pending
-
2025
- 2025-06-03 WO PCT/CN2025/098749 patent/WO2026020992A1/zh active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106844399A (zh) * | 2015-12-07 | 2017-06-13 | 中兴通讯股份有限公司 | 分布式数据库系统及其自适应方法 |
| US20190268405A1 (en) * | 2018-02-28 | 2019-08-29 | International Business Machines Corporation | Load balancing with power of random choices |
| CN112181605A (zh) * | 2020-10-27 | 2021-01-05 | 北京城市网邻信息技术有限公司 | 负载均衡方法、装置、电子设备和计算机可读介质 |
| CN113177050A (zh) * | 2021-05-18 | 2021-07-27 | 浙江大华技术股份有限公司 | 一种数据均衡的方法、装置、查询系统及存储介质 |
| CN114647701A (zh) * | 2022-03-22 | 2022-06-21 | 北京百度网讯科技有限公司 | 分布式数据库的负载均衡方法、装置、电子设备及介质 |
| CN116955087A (zh) * | 2023-08-04 | 2023-10-27 | 北京中科闻歌科技股份有限公司 | 数据分片调整方法、装置、设备及可读存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN121455651A (zh) | 2026-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11023535B1 (en) | Placement of data volumes in a data center | |
| US20220329651A1 (en) | Apparatus for container orchestration in geographically distributed multi-cloud environment and method using the same | |
| US10715460B2 (en) | Opportunistic resource migration to optimize resource placement | |
| US10489343B2 (en) | Cluster file system comprising data mover modules having associated quota manager for managing back-end user quotas | |
| US10261819B2 (en) | Determining an optimal computing environment for running an image based on performance of similar images | |
| US9329888B2 (en) | Computing optimized virtual machine allocations using equivalence combinations | |
| US20180144025A1 (en) | Map-reduce job virtualization | |
| US20130318525A1 (en) | Locality-aware resource allocation for cloud computing | |
| US20150052528A1 (en) | Management of prioritizing virtual machines in an operating environment | |
| US10616134B1 (en) | Prioritizing resource hosts for resource placement | |
| WO2018000991A1 (zh) | 一种数据均衡方法和装置 | |
| US11144500B2 (en) | Assignment of data within file systems | |
| CN111290712B (zh) | 块设备创建方法、装置、云计算管理系统及存储介质 | |
| CN110321225B (zh) | 负载均衡方法、元数据服务器及计算机可读存储介质 | |
| WO2020019315A1 (zh) | 一种基于图数据的计算运行调度方法、系统、计算机可读介质及设备 | |
| US11336519B1 (en) | Evaluating placement configurations for distributed resource placement | |
| Ma et al. | Dependency-aware data locality for MapReduce | |
| Ibrahim et al. | Improvement of job completion time in data-intensive cloud computing applications | |
| US10594620B1 (en) | Bit vector analysis for resource placement in a distributed system | |
| CN117880288A (zh) | 数据均衡方法及相关设备 | |
| Zhang et al. | Speeding up VM startup by cooperative VM image caching | |
| CN112416517A (zh) | 一种虚拟化计算组织控制管理系统及方法 | |
| CN112540844A (zh) | 集群内容器调度方法、装置、存储介质和电子设备 | |
| CN117348980A (zh) | 分布式事务处理方法、装置、设备、介质及程序产品 | |
| US11126623B1 (en) | Index-based replica scale-out |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 25843380 Country of ref document: EP Kind code of ref document: A1 |