CN108400999B - Load balancing method and device for mirror image nodes of database cluster - Google Patents
Load balancing method and device for mirror image nodes of database cluster Download PDFInfo
- Publication number
- CN108400999B CN108400999B CN201710067354.1A CN201710067354A CN108400999B CN 108400999 B CN108400999 B CN 108400999B CN 201710067354 A CN201710067354 A CN 201710067354A CN 108400999 B CN108400999 B CN 108400999B
- Authority
- CN
- China
- Prior art keywords
- cabinet
- mirror image
- current cabinet
- cabinets
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0659—Management of faults, events, alarms or notifications using network fault recovery by isolating or reconfiguring faulty entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Abstract
The invention relates to a load balancing method and a device for a mirror image node of a database cluster, wherein the method comprises the following steps: acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server; performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes. According to the technical scheme provided by the embodiment of the invention, the mirror image node examples corresponding to the main node examples of the current cabinet are dispersed in other different cabinets, and even if the current cabinet fails, the network pressure can be dispersed in different cabinets, so that the whole cluster can be in a load balancing state.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a load balancing method and device for mirror image nodes of a database cluster.
Background
Greenplus is a relational database cluster, which is actually a logical database composed of several independent database services. Unlike the share-disks architecture of Oracle RAC, greenplus adopts a Shared-Nothing architecture, and the whole cluster is composed of a control node (Master Host) and a plurality of data nodes (Segment Host) (which can be understood as a plurality of machines storing data), wherein each data node can run a plurality of databases.
The Greenplus database cluster is mostly used on a PC server with a cheaper X86 framework, so that the hardware failure rate is inevitably improved in the process of reducing the use cost, in order to better avoid the interruption of database service caused by software and hardware failures, the Greenplus also provides a Mirror protection mechanism of a database layer, namely, data of each database node is synchronously mirrored in other nodes, namely, when a master node (Primary) normally works, a manager of the cluster synchronizes the data of the master node to a Mirror node (Mirror), when the data node where the master node is located has hardware failures and loses service capability, the Mirror node immediately takes over all services of the master node, and the failures of the single node do not affect the use of the whole system.
Take an example of having one hundred server clusters: planning of placement of the cabinet and the servers is shown in fig. 1, because the number of the servers is large, the number of database instances in a greenplus database cluster (GP cluster for short) is also large, 98 data nodes are provided in the above environment, and each data node has 4 master node instances, that is, 392 master node instances and mirror node instances. Therefore, the distribution of the database cluster mirror nodes also needs to take full account of the physical location of the servers. In order to ensure high availability of the database cluster, after one cabinet has a problem, the availability of the GP is not affected, and the Primary instance of the data node and the corresponding Mirror instance are also deployed from the cabinet. And the Primary instance and the corresponding Mirror instance form a group by two cabinets to form a mutual Mirror image relationship.
Each server deploys 4 Primary instances and 4 Mirror instances. Primary instances are distributed with Mirror instances on servers in different cabinets. For example, the Mirror instances corresponding to the Primary instances of all the servers in the cabinet 1 are uniformly distributed on all the servers in the cabinet 2, and the deployment diagram is shown in fig. 2. Similarly, Mirror instances corresponding to the Primary instances of all servers on the cabinet 2 are uniformly distributed on all servers on the cabinet 1. A schematic deployment of cabinets 1 and 2 is shown in figure 3.
As shown in fig. 4, if a server of the cabinet 1 fails, the 4 Primary instances of the server are respectively taken over by four servers of the cabinet 2. Even if cabinet 1 fails as a whole, all Primary instances thereon can be taken over by the servers of cabinet 2. However, when a failure occurs in the entire enclosure, such as a failure of an enclosure switch, the mirror enclosure will be subjected to twice the network pressure of the original design, and a server crash may occur. In addition, all mirror image nodes need to be manually configured, and the efficiency is low. In the process of deploying a greenplus cluster, especially a large cluster, because the number of machines is large, the machines are stored between different racks, and network environments of the different racks are different, how to more effectively design the distribution of mirror image nodes to ensure that the whole cluster is still in a balanced state when a fault occurs becomes a difficult problem faced by a DBA (database manager).
Disclosure of Invention
The technical problem to be solved by the invention is to provide a load balancing method for a database cluster mirror image node, so that the defect that a mirror image cabinet is about to bear twice of the originally designed network pressure after the whole cabinet fails in the prior art is solved.
To this end, the invention provides a load balancing method for a database cluster mirror node, comprising the following steps:
acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
Optionally, the following operations are performed for each cabinet in the database cluster: according to the configuration parameters, uniformly distributing mirror node instances corresponding to all the main node instances of the current cabinet to all other cabinets except the current cabinet, including:
performing the following for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
performing the following for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
Optionally, the following operations are performed for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the mirror image node instances corresponding to the main node instances of the current cabinet to all other cabinets except the current cabinet, wherein the steps comprise:
numbering mirror image node examples corresponding to the main node examples of all the cabinets;
performing the following for each cabinet: and according to the numbers of the mirror image node instances corresponding to the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the numbers of the mirror image node instances corresponding to all the main node instances of the current cabinet to other cabinets except the current cabinet, and finally obtaining the number data of the mirror image node instances distributed by each cabinet.
Optionally, the numbers of the mirror image node instances corresponding to all the master node instances of the current cabinet are uniformly distributed to other cabinets except the current cabinet according to the numbers of the mirror image node instances corresponding to the master node instance of the current cabinet and the number of the cabinets, and specifically, the following hash functions are adopted for uniform distribution:
HashFunc(X)＝M％(N-1)
wherein, X is a hash address and represents the cabinet to which the number M is allocated; m is the serial number of mirror image node examples corresponding to all the main node examples of the current cabinet, and N is the number of the cabinets.
Optionally, the following operations are performed for each cabinet: according to the number of mirror image node instances distributed by the current cabinet and the number of servers of the current cabinet, the mirror image node instances distributed by the current cabinet are uniformly distributed to different servers of the current cabinet, and the method comprises the following steps:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
Optionally, the numbers of the mirror image node instances allocated to the current cabinet are uniformly allocated to different servers of the current cabinet according to the number data of the mirror image node instances allocated to the current cabinet and the number of the servers of the current cabinet, and specifically, the following hash functions are used for uniform allocation:
HashFunc(Y)＝K％I
wherein, Y is a hash address and represents the server to which the number K is allocated; k is the number of the mirror image node instance distributed by the current cabinet, and Y is the number of the servers of the current cabinet.
On the other hand, an embodiment of the present invention further provides a load balancing apparatus for a mirror node of a database cluster, including:
the parameter acquisition module is used for acquiring configuration parameters of the database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
a uniform distribution module to perform the following for each cabinet in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
Optionally, the uniform distribution module includes:
a cabinet allocation unit for performing the following operations for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
a server allocation unit for performing the following operations for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
Optionally, the cabinet distribution unit includes:
the numbering subunit is used for numbering the mirror image node examples corresponding to the main node examples of all the cabinets;
an enclosure number assignment subunit configured to perform the following for each enclosure: according to the numbers of the mirror image node examples corresponding to the main node examples of the current cabinet and the number of the cabinets, the numbers of the mirror image node examples corresponding to all the main node examples of the current cabinet are uniformly distributed to other cabinets except the current cabinet, and finally the number data of the mirror image node examples distributed by each cabinet are obtained;
optionally, the server allocation unit is specifically configured to:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
According to the load balancing method and device for the database cluster mirror image nodes, mirror image node examples corresponding to all main node examples of a current cabinet are uniformly distributed to all other cabinets except the current cabinet aiming at each cabinet according to configuration parameters of a database cluster, so that the mirror image node examples corresponding to the main node examples of the current cabinet are dispersed in other different cabinets, network pressure can be dispersed in different cabinets even if the current cabinet fails, the network pressure cannot be singly concentrated in the mirror image cabinet, and the whole cluster can be in a load balancing state when the current cabinet fails; in addition, the mirror image node instances are distributed independently of experience of maintenance personnel, reasonable distribution of the mirror image node instances is carried out according to configuration parameters, the situations of human errors and the like can be avoided, reasonable distribution processing of the mirror image node instances is guaranteed, and problems are avoided.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the invention in any way, and in which:
FIG. 1 is a schematic diagram of a prior art cabinet and server arrangement;
fig. 2 is a schematic diagram illustrating a mirror node instance corresponding to a master node instance of a cabinet 1 in the prior art distributed in the cabinet 2;
fig. 3 is a schematic diagram illustrating a mirror node instance corresponding to a primary node instance of a cabinet 2 in the prior art distributed in a cabinet 1;
fig. 4 is a schematic diagram illustrating the principle of the cabinet 2 taking over after a server of the cabinet 1 fails in the prior art;
fig. 5 is a schematic flowchart of a load balancing method for a mirror node of a database cluster according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a load balancing apparatus for a mirror node of a database cluster according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating a load balancing method for mirror nodes of a database cluster according to another embodiment of the present invention
Fig. 8 is a number diagram of mirror node examples of three cabinets according to an embodiment of the present invention;
FIG. 9 is a diagram illustrating a result after mirror image node instances are evenly distributed based on a cabinet according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating a result of mirror node instance uniform distribution based on a server according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a load balancing apparatus for mirror nodes of a database cluster according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 5, an embodiment of the present invention provides a load balancing method for a mirror node of a database cluster, where the method includes the following steps:
s1: acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
the load balancing method of the database cluster mirror image node provided by the embodiment of the invention can take the load balancing device of the database cluster mirror image node as an execution main body. As shown in fig. 6, the apparatus may be divided into three layers, a task scheduling layer, a model layer, and an analysis layer. The task scheduling layer is mainly responsible for initiating tasks and judging and processing input configuration parameters, namely judging whether the number of cabinets, the number of servers and the like are in a reasonable range or not, and if not, stopping the tasks to report errors. The model layer is mainly responsible for carrying out digital processing on the configuration parameters and converting machines, cabinets and examples in the production link into digital numbers capable of being processed quantitatively. And after the digitization processing is finished, the data analysis layer is responsible for uniformly distributing the mirror image node instances to all the cabinets according to the number of cabinets included by the configuration parameters. And uniformly distributing the distributed mirror node instances to the servers in the cabinet.
S2: performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
It should be noted that, in the prior art, when the entire cabinet 1 fails, all instances of the master node thereon are taken over by the server of the cabinet 2 (mirror cabinet), and therefore, the cabinet 2 will bear twice the network pressure of the original design, which may cause the server in the cabinet 2 to crash. According to the embodiment of the invention, the mirror image node examples corresponding to all the main node examples of the current cabinet are uniformly distributed to all other cabinets except the current cabinet according to the number of cabinets designed by database management, the number of servers of each cabinet and the number of main node examples of each server, so that when the whole cabinet 1 fails, network pressure is not concentrated in the cabinet 2 but dispersed in all other cabinets except the cabinet 1. Even if the cabinet 1 is wholly failed, the phenomenon of sudden increase of network pressure cannot occur in other fault-free cabinets, and the whole database cluster is still in a load balancing state when the fault occurs. For example, when the number of the cabinets is 3, since the mirror image node instances corresponding to the master node instance of the cabinet 1 are uniformly distributed to the cabinets 2 and 3, if the cabinet 1 fails, the servers of the cabinets 2 and 3 take over the failure, and the network pressure is not concentrated in the cabinet 2, thereby realizing load balancing of the mirror image node of the entire database cluster.
According to the load balancing method for the database cluster mirror image nodes, mirror image node examples corresponding to all main node examples of a current cabinet are uniformly distributed to all other cabinets except the current cabinet according to configuration parameters of a database cluster, so that the mirror image node examples corresponding to the main node examples of the current cabinet are dispersed in other different cabinets, network pressure can be dispersed in different cabinets even if the current cabinet fails, the mirror image node examples cannot be singly concentrated in the mirror image cabinet, and the whole cluster can be in a load balancing state when the current cabinet fails; in addition, the mirror image node instances are distributed independently of experience of maintenance personnel, reasonable distribution of the mirror image node instances is carried out according to configuration parameters, the situations of human errors and the like can be avoided, reasonable distribution processing of the mirror image node instances is guaranteed, and problems are avoided.
On the basis of the foregoing embodiment, step S2 specifically includes:
s201: performing the following for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
s202: performing the following for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
Specifically, referring to FIG. 7, the database administrator defines the tasks and enters configuration parameters (number of enclosures, number of servers, and number of instances each server is designed for). And the task allocation layer allocates tasks and judges the rationality of the input configuration parameters. It is unreasonable to terminate the task and make an error report. If the configuration parameters are reasonable, the model layer carries out digital processing on the input configuration parameters, namely, the cabinets, the servers and the instances can be converted into digital numbers which can be processed in a quantification mode, and the digital numbers are imported into the database. The data analysis layer may then perform the allocation operation in a loop based on the generated number numbers. Firstly, uniform distribution can be performed on the basis of the cabinets, and the mirror image node instances corresponding to the current cabinet are uniformly distributed to other cabinets for each cabinet in sequence. And generating a distribution list of all the image node instances in each cabinet according to the requirement. And then, aiming at each cabinet in sequence, uniformly distributing the mirror image node instances distributed by the current cabinet to different servers in the current cabinet, finally completing uniform distribution of all image node instances, generating a distribution list of all image node instances corresponding to each cabinet and each server according to needs, and realizing load balance even when a certain cabinet is wholly failed.
On the basis of the above embodiment, step S201 specifically includes:
s201-1: numbering mirror image node examples corresponding to the main node examples of all the cabinets;
s201-2: performing the following for each cabinet: and according to the numbers of the mirror image node instances corresponding to the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the numbers of the mirror image node instances corresponding to all the main node instances of the current cabinet to other cabinets except the current cabinet, and finally obtaining the number data of the mirror image node instances distributed by each cabinet.
For example, assuming that cabinet 1 has 9 servers, one server has 4 instances of the primary node, and the corresponding mirror node instances may be numbered 001, 002, 003, 004, cabinet 1 has numbers 001, 002 … … 036, and so on. And according to the generated number, circularly executing distribution, and uniformly distributing the mirror image node instances corresponding to the local cabinet to all other cabinets which are not in the local cabinet. Thus, each cabinet has a number that is evenly distributed to the mirror node instances of the local cabinet. The numbering of the cabinet can be reordered to generate a corresponding numbering list.
Taking three cabinets and three servers per cabinet as an example, sequentially numbering mirror image node instances corresponding to the main node instances of all the cabinets according to the obtained configuration parameters, as shown in fig. 8, where the main node instances are numbered 001, 002, 003 … … 036, and the corresponding mirror image node instances are numbered 001, 002, 003 … … 036. The numbers of all mirror node instances of cabinet 1 may be distributed to cabinets 2 and 3 according to the number of cabinets. Namely, the numbers (001-. By analogy, the numbers (013-.
Optionally, step S201-2 may specifically adopt the following HASH (HASH) function to implement uniform distribution:
HashFunc(X)＝M％(N-1)
wherein, X is a hash address and represents the cabinet to which the number M is allocated; m is the serial number of mirror image node examples corresponding to all the main node examples of the current cabinet, and N is the number of the cabinets.
Specifically, a Hash function is adopted for uniform distribution, the primary key M is a serial number corresponding to the mirror image node instance, the N is the number of the cabinets, and the primary key is used as the fragmentation attribute. And taking the remainder obtained by dividing the key by a number which is not more than the length N of the hash table as the hash address. For example: here, the number N of the cabinets is 3, the hash address is 1 if M is 001, and HashFunc (1) is 001% (3-1). Taking three servers of the cabinet 1 as an example, the mirror node instances are numbered (001002003004) (005006007008) (009010011012), when the cabinet 1 is down, the cabinets 2 and 3 are numbered 1 and 0 respectively, and the instance 001 obtains the Hash address 1 through a Hash function, so that the instance 001 is allocated to the number 1 (cabinet 2). By analogy, after the mirror image node instances of the cabinet 1 are uniformly distributed, the mirror image node instances of each cabinet are circularly executed until the mirror image node instances of each cabinet are uniformly distributed in all other cabinets, and the result is shown in fig. 9. As can be seen from fig. 9, the mirror node instances corresponding to the master node instance of the cabinet 1 are uniformly distributed to other cabinets (the cabinet 2 and the cabinet 3), so that when the cabinet 1 loses service capability, the cabinet 2 and the cabinet 3 take over to jointly bear network pressure, thereby achieving load balancing.
After the mirror image node instances of all the cabinets are distributed, the mirror image node instances stored in the cabinets can be reordered to form a corresponding number list, and the cabinet 1 is taken as an example to form a corresponding ordered list as shown in the following table 1.
TABLE 1 numbering ordering of mirror node instances assigned by Cabinet 1
On the basis of the foregoing embodiment, step S202 may specifically include:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
It should be noted that after step S201, each cabinet has numbers of mirror image node instances uniformly allocated to the local cabinet, and in each cabinet, because there are multiple servers, taking cabinet 1 as an example, the numbers of mirror image node instances in the cabinet may be sequentially allocated to different servers in the local cabinet according to the sequence in table 1. And analogizing until the serial numbers of the mirror image node instances in each cabinet are uniformly distributed among different servers in the cabinet.
Optionally, according to the number data of the mirror node instance allocated to the current cabinet and the number of servers of the current cabinet, the numbers of the mirror node instances allocated to the current cabinet are uniformly allocated to different servers of the current cabinet, and the uniform allocation may be specifically realized by using the following hash function:
HashFunc(Y)＝K％I
wherein, Y is a hash address and represents the server to which the number K is allocated; k is the number of the mirror image node instance distributed by the current cabinet, and Y is the number of the servers of the current cabinet.
Specifically, a Hash function is adopted for uniform distribution, a primary key K is a number corresponding to a mirror image node instance, I is the number of servers in the current cabinet, and the primary key is used as a fragmentation attribute. After the cabinet-based mirror node instance assignment is completed as shown in fig. 9, taking the cabinet 1 as an example, the numbers 0, 1, and 2 of the three servers of the cabinet 1 are numbered, and referring to the sorting of table 1, the numbers 013, 015, 017 …. If HashFunc (1) ═ 013% 3, that is, assigning the mirror node instance numbered 013 to the server numbered 1, and if HashFunc (2) ═ 017% 3, that is, assigning the mirror node instance numbered 017 to the server numbered 2. By analogy, the mirror node instances in each cabinet are uniformly distributed in the servers in the local cabinet, and the result is shown in fig. 10.
After the uniform distribution of all mirror image node instances in each cabinet and each server is completed, a new master node instance and a new mirror image node instance corresponding to each server may be generated, as shown in table 2 below, taking cabinet 1 as an example, the master node instance numbers corresponding to 3 servers are sequentially: 001. 002, 003 … ….012, mirror node instance numbering is in order: 013. 019, 025, 031, 015, 021, 027, 033, 017, 023, 029, 035.
Table 2 list of mirror node instance numbers assigned by cabinet 1 and different servers
According to the obtained number N of the cabinets and the number I of the servers installed on each cabinet, all mirror image node instances can be configured in a load balancing mode, the generated corresponding configuration list is provided for the GP database, initial distribution can be carried out according to the configuration list, and therefore the effect of load balancing after cabinet failure is achieved.
According to the load balancing method for the database cluster mirror image nodes, provided by the embodiment of the invention, under the condition that the current cabinet fails, network pressure can be dispersed in different cabinets and cannot be singly concentrated in the mirror image cabinet, and the whole cluster can be in a load balancing state when the current cabinet fails; according to different cabinets and the number of servers in the cabinets, a new mirror image node instance distribution list is automatically determined according to multiple Hash operation results, manual experience is not needed for determining distribution of mirror image node instances, experience of maintenance personnel is not needed, reasonable distribution of the mirror image node instances is carried out according to configuration parameters, artificial errors and the like are avoided, reasonable distribution processing of the mirror image node instances is guaranteed, and problems are avoided.
On the other hand, as shown in fig. 11, an embodiment of the present invention further provides a load balancing apparatus for a database cluster mirror node, where the apparatus may adopt the load balancing method for a database cluster mirror node described in the foregoing embodiment, and the method includes: a parameter acquisition module 111 and an even distribution module 112 connected in sequence;
the parameter obtaining module 111 is configured to obtain configuration parameters of the database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
the uniform distribution module 112 is configured to uniformly distribute, according to the configuration parameters, the mirror image node instances corresponding to all host node instances of the current enclosure to all other enclosures except the current enclosure, so as to implement load balancing of the database cluster mirror image nodes.
Specifically, the parameter obtaining module 111 obtains configuration parameters of the database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server; the uniform distribution module 112 performs the following for each cabinet in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
According to the load balancing device for the database cluster mirror image nodes, provided by the embodiment of the invention, mirror image node instances corresponding to all main node instances of a current cabinet are uniformly distributed to all other cabinets except the current cabinet aiming at each cabinet according to configuration parameters of a database cluster, so that the mirror image node instances corresponding to the main node instances of the current cabinet are dispersed in other different cabinets, even if the current cabinet fails, network pressure can be dispersed in different cabinets and cannot be singly concentrated in the mirror image cabinet, and when the current cabinet fails, the whole cluster can be in a load balancing state; in addition, the mirror image node instances are distributed independently of experience of maintenance personnel, reasonable distribution of the mirror image node instances is carried out according to configuration parameters, the situations of human errors and the like can be avoided, reasonable distribution processing of the mirror image node instances is guaranteed, and problems are avoided.
On the basis of the above embodiment, the uniform distribution module 112 includes:
a cabinet allocation unit for performing the following operations for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
a server allocation unit for performing the following operations for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
On the basis of the above embodiment, the cabinet distribution unit includes:
the numbering subunit is used for numbering the mirror image node examples corresponding to the main node examples of all the cabinets;
an enclosure number assignment subunit configured to perform the following for each enclosure: according to the numbers of the mirror image node examples corresponding to the main node examples of the current cabinet and the number of the cabinets, the numbers of the mirror image node examples corresponding to all the main node examples of the current cabinet are uniformly distributed to other cabinets except the current cabinet, and finally the number data of the mirror image node examples distributed by each cabinet are obtained;
on the basis of the foregoing embodiment, the server allocation unit is specifically configured to:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
For the embodiment of the load balancing device of the database cluster mirror node corresponding to the method, since the embodiment is basically similar to the embodiment of the method, and the achieved technical effect is the same as the effect of the embodiment of the method, the description is relatively simple, and relevant points can be referred to the partial description of the embodiment of the method.
In another aspect, as shown in fig. 12, an embodiment of the present invention further provides an electronic device, which may be the apparatus in the foregoing embodiment, where the electronic device includes at least one processor (processor)121, a communication Interface (Communications Interface)122, at least one memory (memory)123, and a bus 124, where the processor 121, the communication Interface 122, and the memory 123 complete mutual communication through the bus 124. The communication interface 122 may be used for information transfer between the electronic device and a server. Processor 121 may call logic instructions in memory 123 to perform the methods described in the above embodiments, including, for example: acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server; performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
In addition, the logic instructions in the memory 123 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which when executed by a computer, enable the computer to perform the methods provided by the above-mentioned method embodiments, for example, including: acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server; performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
Embodiments of the present invention provide a non-transitory computer-readable storage medium, which stores computer instructions, where the computer instructions cause the computer to perform the methods provided by the above method embodiments, for example, the methods include: acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server; performing the following for each enclosure in the database cluster: and according to the configuration parameters, uniformly distributing mirror image node examples corresponding to all main node examples of the current cabinet to all other cabinets except the current cabinet so as to realize load balance of the database cluster mirror image nodes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. A load balancing method for mirror nodes of a database cluster is characterized by comprising the following steps:
acquiring configuration parameters of a database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
performing the following for each enclosure in the database cluster: according to the configuration parameters, mirror image node examples corresponding to all main node examples of the current cabinet are uniformly distributed to all other cabinets except the current cabinet, so that load balance of the mirror image nodes of the database cluster is realized;
the method specifically comprises the following steps:
performing the following for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
performing the following for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
2. The method of claim 1, wherein the following is performed for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the mirror image node instances corresponding to the main node instances of the current cabinet to all other cabinets except the current cabinet, wherein the steps comprise:
numbering mirror image node examples corresponding to the main node examples of all the cabinets;
performing the following for each cabinet: and according to the numbers of the mirror image node instances corresponding to the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the numbers of the mirror image node instances corresponding to all the main node instances of the current cabinet to other cabinets except the current cabinet, and finally obtaining the number data of the mirror image node instances distributed by each cabinet.
3. The method according to claim 2, wherein the numbers of the mirror node instances corresponding to all the master node instances of the current cabinet are uniformly distributed to other cabinets except the current cabinet according to the numbers of the mirror node instances corresponding to the master node instances of the current cabinet and the number of the cabinets, and specifically, the following hash function is used for uniform distribution:
HashFunc(X)＝M％(N-1)
wherein, X is a hash address and represents the cabinet to which the number M is allocated; m is the serial number of mirror image node examples corresponding to all the main node examples of the current cabinet, and N is the number of the cabinets.
4. The method of claim 2, wherein the following is performed for each cabinet: according to the number of mirror image node instances distributed by the current cabinet and the number of servers of the current cabinet, the mirror image node instances distributed by the current cabinet are uniformly distributed to different servers of the current cabinet, and the method comprises the following steps:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
5. The method according to claim 4, wherein the numbers of the mirror node instances allocated to the current cabinet are uniformly allocated to the servers of different current cabinets according to the number data of the mirror node instances allocated to the current cabinet and the number of the servers of the current cabinet, and specifically, the following hash function is used for uniform allocation:
HashFunc(Y)＝K％I
wherein, Y is a hash address and represents the server to which the number K is allocated; k is the serial number of the mirror image node instance distributed by the current cabinet, and I is the number of the servers of the current cabinet.
6. A load balancing apparatus for a mirror node of a database cluster, comprising:
the parameter acquisition module is used for acquiring configuration parameters of the database cluster; the configuration parameters comprise the number of cabinets, the number of servers of each cabinet and the number of main node instances of each server;
a uniform distribution module to perform the following for each cabinet in the database cluster: according to the configuration parameters, mirror image node examples corresponding to all main node examples of the current cabinet are uniformly distributed to all other cabinets except the current cabinet, so that load balance of the mirror image nodes of the database cluster is realized;
wherein the uniform distribution module comprises:
a cabinet allocation unit for performing the following operations for each cabinet: according to the number of the main node instances of the current cabinet and the number of the cabinets, the mirror image node instances corresponding to the main node instances of the current cabinet are uniformly distributed to all other cabinets except the current cabinet;
a server allocation unit for performing the following operations for each cabinet: and uniformly distributing the mirror image node examples distributed by the current cabinet to different servers of the current cabinet according to the number of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet.
7. The apparatus of claim 6, wherein the cabinet allocation unit comprises:
the numbering subunit is used for numbering the mirror image node examples corresponding to the main node examples of all the cabinets;
an enclosure number assignment subunit configured to perform the following for each enclosure: and according to the numbers of the mirror image node instances corresponding to the main node instances of the current cabinet and the number of the cabinets, uniformly distributing the numbers of the mirror image node instances corresponding to all the main node instances of the current cabinet to other cabinets except the current cabinet, and finally obtaining the number data of the mirror image node instances distributed by each cabinet.
8. The apparatus according to claim 7, wherein the server allocation unit is specifically configured to:
performing the following for each cabinet: and according to the number data of the mirror image node examples distributed by the current cabinet and the number of the servers of the current cabinet, uniformly distributing the numbers of the mirror image node examples distributed by the current cabinet to different servers of the current cabinet, and finally determining the number data of the mirror image node examples distributed by each server of the current cabinet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710067354.1A CN108400999B (en) | 2017-02-06 | 2017-02-06 | Load balancing method and device for mirror image nodes of database cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710067354.1A CN108400999B (en) | 2017-02-06 | 2017-02-06 | Load balancing method and device for mirror image nodes of database cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108400999A CN108400999A (en) | 2018-08-14 |
| CN108400999B true CN108400999B (en) | 2020-10-30 |
Family
ID=63094181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710067354.1A Active CN108400999B (en) | 2017-02-06 | 2017-02-06 | Load balancing method and device for mirror image nodes of database cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108400999B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111459654B (en) * | 2019-01-22 | 2024-04-16 | 顺丰科技有限公司 | Method, device, equipment and storage medium for deploying server cluster |
| CN114124700A (en) * | 2021-10-26 | 2022-03-01 | 马上消费金融股份有限公司 | Cluster parameter configuration method and device, electronic equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916209A (en) * | 2010-08-06 | 2010-12-15 | 华东交通大学 | Cluster task resource allocation method for multi-core processor |
| CN103403666A (en) * | 2012-12-21 | 2013-11-20 | 华为技术有限公司 | Distributed storage control method, device and system |
| US8745264B1 (en) * | 2011-03-31 | 2014-06-03 | Amazon Technologies, Inc. | Random next iteration for data update management |
| US9330151B1 (en) * | 2013-03-08 | 2016-05-03 | Emc Corporation | Database change data capture within a storage array |
| CN106372161A (en) * | 2016-08-31 | 2017-02-01 | 天津南大通用数据技术股份有限公司 | Distributive database data backup piece deployment method and device |
-
2017
- 2017-02-06 CN CN201710067354.1A patent/CN108400999B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916209A (en) * | 2010-08-06 | 2010-12-15 | 华东交通大学 | Cluster task resource allocation method for multi-core processor |
| US8745264B1 (en) * | 2011-03-31 | 2014-06-03 | Amazon Technologies, Inc. | Random next iteration for data update management |
| CN103403666A (en) * | 2012-12-21 | 2013-11-20 | 华为技术有限公司 | Distributed storage control method, device and system |
| US9330151B1 (en) * | 2013-03-08 | 2016-05-03 | Emc Corporation | Database change data capture within a storage array |
| CN106372161A (en) * | 2016-08-31 | 2017-02-01 | 天津南大通用数据技术股份有限公司 | Distributive database data backup piece deployment method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108400999A (en) | 2018-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107004033B (en) | Massively parallel processor database system and method | |
| CN107666525B (en) | Method and device for IP allocation of cluster container | |
| WO2012144647A1 (en) | Virtual machine administration device, virtual machine administration method, and program | |
| CN109739919B (en) | Front-end processor and acquisition system for power system | |
| CN107430603A (en) | The system and method for MPP database | |
| CN106506201A (en) | VNF moving methods, MANO and system | |
| CN106557366B (en) | Task distribution method, device and system | |
| CN105701159A (en) | Data synchronization device and method | |
| CN108400999B (en) | Load balancing method and device for mirror image nodes of database cluster | |
| CN107682411A (en) | A kind of extensive SDN controllers cluster and network system | |
| CN111045811A (en) | Task allocation method and device, electronic equipment and storage medium | |
| CN113918281A (en) | Method for improving cloud resource expansion efficiency of container | |
| CN106372160A (en) | Distributive database and management method | |
| CN104394016A (en) | ISSU (In-Service Software Upgrade) method and system | |
| CN110868435B (en) | Bare metal server scheduling method and device and storage medium | |
| CN111352726B (en) | Stream data processing method and device based on containerized micro-service | |
| CN105302817B (en) | Distributed file system management method and device | |
| CN103336721B (en) | Method, device and system for allocating database operation request | |
| CN104378239A (en) | Rapid reliability index statistical system and method based on cluster frame | |
| CN112632193A (en) | Data distribution method for distributed database cluster system | |
| CN110019229B (en) | Database configuration system | |
| CN107547622B (en) | Resource adjusting method and device | |
| US20160014198A1 (en) | Information processing apparatus | |
| CN104462344B (en) | In the processing method and processing device of destination server structure ETL system | |
| CN109299191A (en) | A kind of data distribution method, device, server and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |