JP2015513333A - Database workload balancing through migration - Google Patents

Abstract

Database workload balancing is achieved. A method for database workload balancing between a plurality of database servers, when a new server is available, a set of master and slave databases to be migrated to the new server is selected to reduce migration costs. Migrating a database that is selected to balance the workload among all servers while minimizing it, and detecting the workload imbalance in real time due to workload fluctuations during operation, the workload balance Select the databases to be migrated to other servers one by one until you get them. [Selection] Figure 1

Description

  This application is based on US Provisional Application No. 6162151, filed April 4, 2012, the entire disclosure of which is incorporated herein.

  The present invention relates to database workload balancing.

  Enterprises are increasingly centralizing back-end computer systems for the purpose of building data centers for various economic and business reasons. Data centers typically house computer systems at a high concentration and density, and provide a database that supports customer needs. Data center operators must secure server boxes and prepare resources for the ever-changing workload. Furthermore, it is necessary to consider physical constraints such as physical resource capacity constraints when sharing resources on the same physical box with various workloads and provisioning for workloads. In recent years, there has been a shift to cloud computing for data intensive computing that presents great opportunities and challenges to data center operators.

One key challenge faced by data center operators is the provisioning of resources in the data center for customer workloads. For example, when a new server is added, each existing server
・ There is no master / slave combination co-located on a new server.
・ Master to slave ratio is almost the same for all servers.
-Memory capacity and CPU capacity are balanced as much as possible in all servers.
・ Minimum movement of data volume
Provide a master and slave set to migrate (migrate) to the new server.

  Another problem is the elimination of hot spots. In a given server configuration, if there is an overloaded server (the server is overloaded when it is a memory hotspot, a CPU hotspot, or both), migrate one master from each overloaded server This eliminates the server overload condition.

  Federated databases have been used by manually dividing relationships between individual servers and exposing them to clients in a unified manner. However, such a solution causes problems where manual device configuration, special merging wrapper algorithms, and dynamic rebalancing are not supported.

  Another solution is to add or delete slave databases based on workload. However, such a solution is not useful when the workload includes many write queries, since the slave database (replica) only supports read queries and all write queries must be passed to the master. Furthermore, this solution cannot handle fast workload fluctuations because adding slaves takes a long time.

  As another approach, it is conceivable to make the workload different between the master DB and its slave. In that case, the balance between workloads is maintained by exchanging the roles of the master and slave or the master and slave. However, such a method is effective only when the workloads of the master and the slave are greatly different and the DB system supports swapping. As yet another method, load balancing is achieved by (a) changing the allocation of resources between different virtual machines on the same server, and (b) adding and deleting servers. However, such a method is because each DB instance is wrapped in one virtual machine and each server must contain all DBs (either all masters of all DBs or one slave of each DB) On the other hand, the range in which the method can be applied is limited.

  One aspect of the present invention is a method for database workload balancing between a plurality of database servers, when a new server is available, selecting a set of master and slave databases to migrate to the new server; When migrating a selected database to balance the workload among all servers while minimizing the migration cost and detecting workload imbalance in real time due to workload fluctuations during operation, the workload Select databases to be migrated to other servers one by one until the balance is achieved.

  The advantages of the preferred embodiment are as follows. This system achieves database workload balancing by using real-time database migration. Such a solution has a fast response time, minimizes manual intervention, has various characteristics (memory and CPU footprint, etc.) and metrics (workload distribution, workload fluctuation L1 norm, server) Can be used. In this system, the overall system performance of all databases and all servers is optimized by achieving database workload balancing. High speed operation is achieved. In this system, the live migration technology of the memory resident database is utilized, so the operation becomes faster and the response to the fluctuation of the work load becomes faster. With this system, you can quickly find a cost-effective database layout that minimizes the running cost of the workload. The system supports data center operators preparing for individual workloads while minimizing the total operating cost (TOC) of the workload during operation. Workload balancing can be achieved in real time by migration, and the ACID characteristics (Atomicity, Consistency, Isolation, Durability) required for many database systems Is guaranteed.

It is a figure which shows an example of the workload balancing achieved by the migration of the database master or slave between servers. It is a figure explaining the low dispersion | distribution which means the state where the capacity | capacitance balance was taken in N = 2. It is a figure explaining the low dispersion | distribution which means the state where the capacity | capacitance balance was taken in N = 3. It is a figure which shows a mode that the optimal tenant does not exist. It is a figure which shows a hot spot elimination problem. It is a figure which shows the memory hot spot provided with several resources, ie, memory, and CPU. It is a figure which shows an example of the computer which performs database balancing.

  FIG. 1 is a diagram illustrating an example of workload balancing achieved by database master or slave migration between servers.

  SV1 to SV4 indicate four servers, M1 to M3 indicate three database masters, and each database master includes two database slaves (replicas) corresponding thereto. Therefore, S11 and S12 are M1 slaves. First, when a new server begins handling a workload, the system selects a set of master and slave databases to migrate to the new server. The selected database to be migrated will result in the best work load balancing of all servers (including new servers) while minimizing migration costs. The new server will have a master to slave ratio similar to that of the existing server (ie, the new server will be similar to other existing servers after migration is complete). Second, during operation, for example, if the workload of an arbitrary database fluctuates, and a workload imbalance is detected in real time, the database that migrates to other servers to balance the workload Select one by one repeatedly.

  In one embodiment, the M databases are hosted on N servers. Each database has a primary DB (master) and one or more secondary DBs (slave) hosted by other servers. That is, the master DB is not arranged on the same server as its own slave, and two slaves belonging to the same master are not arranged on the same server.

  Each database has a footprint of system resources such as CPU usage and memory usage that change from moment to moment. Each server has a maximum resource capacity that exceeds the state in which the server is considered overloaded. Each resource is considered additive, that is, the server resource usage is the sum of each resource usage of all databases hosted on the server. Such a premise is generally accepted in the field of CPU usage and memory usage. The system preferably executes DB live migration, for example, a real-time migration function provided by a system such as NEC TAM. Hereinafter, various terms will be described in detail.

This system is composed of _N servers SV ₁ ,..., SV _N.

The server SV _i accommodates K _i (t _i ¹ ,..., T _i ^ki ) tenants (Note: each server may have a different number of tenants).

· Each tenant t _i ^k becomes either the master or slave. The master tenant is not placed on the same TAM server as the slave.

-Each tenant t _i ^k has a memory usage t _i ^k . mem footprint and CPU usage t _i ^k . with cpu footprint.

-Memory usage and CPU usage are additive in server SV _i , and total memory usage is

And the total CPU usage is

It is.

  The memory unit and the CPU unit are compatible with each other (for example, the memory unit is gigabyte and the CPU unit is gigacycle / second).

Each server SV _i has a memory threshold value SV _i . memTh and CPU threshold SV _i . There is cpuTh. In this specification, when either one of the memory threshold and the CPU threshold is exceeded, the server is referred to as a hot spot (memory hot spot or CPU hot spot).

Each server SV _i has a memory capacity SV _i . memCap (free memory capacity before SV _i becomes a memory hot spot) and CPU capacity SV _i . There is cpuCap. SV _i . memCap <0 or SV _i . If cpuCap <0, server SV _i becomes a hot spot.

For each server SV _i
・ SV _i . memCap: the memory capacity of the server SV _i .

・ SV _i . cpuCap: CPU capacity of the server SV _i .

・ SV _i . memTh: memory threshold of server SV _i above which SV _i becomes a memory hot spot.

・ SV _i . cpuTh: CPU threshold of server SV _i above which SV _i becomes a CPU hot spot.

・ SV _i . tenants: Tenants {t _i ¹ ,..., t _i ^Ki } belonging to the server SV _i .

For each tenant _t ^{i k} is,
· _T ⁱ k. _{tid: t} ^{i k} of the tenant ID.

· _T ⁱ k. isMaster: A Boolean flag indicating that t _i ^k is a master (or slave).

· _T ⁱ k. mem: memory usage of t _i ^k .

· _T ⁱ k. _cpu: CPU utilization of _t ^{i k.}

· _T ⁱ k. _dataSize: is the data size of the _t ^{i k.}

An average value μ _x and variance σ _x ² of _N numerical values {x ₁ ,..., X _N } are defined as follows.

In the above formula, a small variance means that each server is balanced. _{{X 1, ..., x N} } When the the respective server resource capacity (memory or _{CPU), {x 1, ...} , x N} time is the low dispersion, a state of balance between the server is taken means. That is,

Σ _x ² is minimum when x ₁ =... = X _N. σ _x ² is a useful metric for evaluating the balance of the resource capacity of each server.

FIG. 2A is a diagram for explaining low dispersion, which means a state where the capacity is balanced when N = 2. When x ₁ + x ₂ = c and c = 2μ _x is a constant, the system simply redistributes the values to x ₁ and x ₂ . Because it is _{x 1 + x 2 = 2μ x} , pairs of all _{x 1, x _2} is located on the line shown in FIG. FIG. 1 shows isolines where σ _x ² has the same value (they are circular). As shown in the figure, when x ₁ = x ₂ = μ _x , σ _x ² has a minimum value of 0, and otherwise, {x ₁ , x ₂ } is a highly unbalanced high dispersion state. This is also true when N> 2. As an example, FIG. 2B is a diagram for explaining low dispersion, which means a state where capacity is balanced when N = 3.

SV _i is a hotspot server with resource (for example, memory) capacity x _i , SV _j is a light load server with resource capacity x _j , and this system migrates tenants with resource usage Δ from SV _i to SV _j To reduce the distribution of capacity between servers,

It is. Here, when this dispersion reduction is treated as a function of Δ,

And the best tenant to migrate is resource usage

As a result of migration, the change in distribution is

It becomes.

The solution is to select the maximum resource capacity. Resource usage of tenants are discrete values, as shown by examples to reach the optimum delta ^* in FIG. 3, the resource usage of the nearest tenants a delta ₁ and delta _2, resource usage delta ^* of It is not always possible to find a tenant. FIG. 3 shows a state where an optimal tenant does not exist.

Next, a new server migration process will be described. With the addition of new servers, each existing server
・ There is no master / slave co-located on the new server.
・ The master-to-slave ratio is almost the same on all servers.
-The memory capacity and CPU capacity must be balanced as much as possible on all servers.
-The amount of data to be moved is minimal,
A master and slave set that migrates to the new server is provided.

  The pseudo code for Method Example 1 of migrating a DB to a new server is as follows:

When the resource usage of the existing N servers {SV ₁ ,..., SV _N } is X = {x ₁ ,..., X _N }, one resource X is extracted for explanation. Here, a new server SV _{N + 1} is incorporated into the system. Let Y = {y ₁ ,..., Y _N , y _{N + 1} } be the resource usage of the N + 1 servers after providing the master and slave from each existing server to the new server. Average value and variance of resource usage after migration is

It becomes. Here, depending on the tenant configuration

It will not be.

Method 1 can be used and activated many times. The problem is that SV _{N + 1} is initially empty. In other words, the vacant SV _{N + 1} is quickly filled, which means that the greedy nature of Method 1 encourages the migration of large tenants on the existing server, so that only large tenants are installed on SV _{N + 1} and at the same time the existing server capacity is increased. A large hole is left (for example, some servers have large migrated tenants and some servers do not have migrated tenants).

Based on the above considerations, Method 2 does not consider the distribution of new servers. That is, in each iteration, the method searches for a tenant t that migrates to a new server SV _{N + 1} and maximizes the reduction in distribution of existing N servers (with respect to μ _Y instead of μ _X ).

As described above, Method 2 can be used for migration of a new server. In method 2, {SV ₁ ,..., SV _N } includes | M | master and | S | slave. The method then selects the | M | / (N + 1) master and | S | / (N + 1) slave to migrate to the new server SV _{N + 1} . In method 2, it is a parameter to select | M | / (N + 1) master.

It is a parameter to select | S | / (N + 1) slave

Called. Q is a priority queue that sorts objects by reducing the variance.

In Method 2,
Line 2-5 calculates the average resource usage μ _mem and μ _CPU (including the new server SV _{N + 1} ).
Line 6-9 extracts the best tenant candidate for migration at each server. These candidates can be determined locally at each server without consulting other servers.
Line 11-14 moves the best candidates from N servers to SV _{N + 1} .
Line 15-24 updates the best tenant candidate for each server. When migrating a slave, a slave that is newly moved to SV _{N + 1} is excluded from the best candidates in other servers, and thus recalculation is required.

Next, the hot spot elimination problem will be described. Predetermined overload server database configuration _(SV i _.memCap <memory hot spots is 0 or _SV i _.cpuCap <0 CPU hotspots or both,,) if there is, one master from the overloaded server The overload state of the server is resolved by migrating. This is illustrated in FIG.

By the way, the resource X is not actually one, but there are two resources (memCap, cpuCap). This affects two points: (1) how to define hot spots, and (2) how to evaluate the balance between servers. For (1), the new server SV _i is SV _i . memCap <0 or SV _i . If cpuCap <0, it becomes a hot spot. In this case, the hot spot must be eliminated. This is illustrated in FIG. 5, which illustrates a memory hot spot defined by a plurality of resources, ie, memory and CPU.

  Regarding (2), as a simple solution, a method of combining memory capacity distribution and CPU capacity distribution can be used.

Here, c _mem and c _cpu are parameters for calibrating different units (GB vs. G-cycle-per-sec) of the memory and the CPU.

Also, c _mem and c _cpu can be used to control the emphasis of the balance between the memory and the CPU. For example, since the CPU usage rate is bursty and the memory usage is relatively stable, the balance of the CPU usage is more important than the balance of the memory usage. In special cases

as well as

Set to

With this setting, the system normalizes both μ _memCap and μ _ccupap to 1 and treats the balance of memory and CPU with equal importance. This idea is reflected in lines 7, 8 and 11 of method 3.

  Since each tenant has a different data size, the cost for migrating them is also different. One solution is to review the objective function, such as the ratio of the variance reduction due to t migration and the cost ratio due to t migration. For example, if the migration cost of t is proportional to the data size of t, a new metric

And Here, σ ² (t) is a dispersion reduction when t is migrated.

  A master cannot be placed on the same server as its slaves. As a result, the hotspot server master cannot be migrated to a randomly chosen server. This is checked by the algorithm. That is, in this system, only the target server that does not have t slaves is considered for the tenant t that is a migration candidate.

  This system considers tenant candidates and target servers that eliminate existing hot spots and do not generate new hot spots. If there is no such solution, the algorithm returns to the null position.

The input to this algorithm is {SV ₁ ,..., SV _N }. If there is more than one hot spot on these servers, the algorithm solution should deal with all hot spots. This causes a more difficult problem. In the present invention, a method using a local greedy heuristic that eliminates hot spots one by one is selected. That is, (1) The hot spot of each server is treated as if it is only the hot spot, (2) The system configuration is provisionally updated every time the hot spot is dealt with, (3) and Address the next hotspot with the provisionally updated system configuration. (4) If the final configuration that eliminates all hotspots cannot be reached, the algorithm declares failure.

  This is because heuristically (1) monitors the order of the hot spots addressed and (2) finds a solution with a less greedy method that makes it easier to eliminate the next hot spot for each hot spot. Keep in mind that there are.

  FIG. 4 shows an example database workload balancing process at the database server using database migration (201). The method balances the workload of each server with the step (202) of migrating the database set from the set of existing servers to the newly available server so that the workload of each server is balanced. Thus, the step of migrating the database of the existing database server is included (209). In this process, a metric that evaluates the goodness of a predetermined database configuration is used (203). In this process, an array of database masters and slaves to be migrated to a new server is selected (204). In 205, various system resources such as a CPU usage rate and a memory usage amount are considered in the metric. At 206, the metric combines different factors in an integrated method with a weighting strategy and a vector norm (Euclidean norm or L1 norm).

  Considering now 204, the method includes selecting a set of master and slave databases to migrate to a newly available server. In 207, the selection process includes various factors including (a) expected reduction cost after migration, (b) master-to-slave ratio of each server after migration, and (c) migration cost related to the size of each database. Consider. At 208, the method selects the optimal migration order so that the negative performance affecting the system is minimized during the migration process.

  The method includes the step of migrating a database of an existing database server so that the workload of each server is balanced (209). Metrics are used to evaluate the goodness of a given database configuration (210). At 212, the metrics take into account various system resources such as CPU usage and memory usage. At 213, the metric combines different factors in an integrated way with a weighting strategy and a vector norm (Euclidean norm or L1 norm).

  Alternatively, at 211, the method includes repeatedly selecting the next database to migrate and the corresponding target database server to eliminate all hot spots. At 214, the method efficiently selects the next database to migrate and the corresponding target server. At 215, the method repeatedly migrates the database one by one until all hot spots are resolved.

  FIG. 6 shows an example of a computer that performs database balancing. This system is realized by hardware, firmware, software, or a combination thereof. Preferably, the present invention executes a computer program by a programmable computer comprising a processor, a data storage system, volatile and non-volatile memory and / or storage elements, at least one input device, and at least one output device. It is realized with.

  Next, as an example, a computer that supports this system will be described with reference to the block diagram of FIG. The computer preferably includes a processor, random access memory (RAM), program memory (preferably a rewritable read only memory (ROM) such as a flash ROM) and input / output (connected to a CPU bus). INPUT / OUTPUT: I / O) controller. The computer may include a hard drive controller connected to the hard disk via a CPU bus. The hard disk is used for storing application programs and data for realizing the present invention. The application program may be stored in RAM or ROM. The I / O controller is connected to the I / O interface by an I / O bus. The I / O interface transmits and receives data in an analog or digital format through a communication link such as a serial link, a local area network, a wireless link, and a parallel link. A display, a keyboard, and a pointing device (mouse) may be connected to the I / O bus. A separate connection (separate bus) may be used for the I / O interface, display, keyboard, and pointing device. A programmable processing system may be pre-programmed or programmed (or reprogrammed) by downloading the program from another source (eg, floppy disk, CD-ROM, or other computer). Good.

  The computer program is stored on a machine-readable storage medium or device (eg, program memory or magnetic disk) that can be read by a normal or special purpose programmable computer. The computer program is for configuration and control operations of the computer that are read from the storage medium or device by a computer that performs the procedures described herein. The system of the present invention also includes a computer readable storage medium including a computer program configured to cause a computer to operate in a specific and predefined manner that performs the functions described herein. Be considered.

  The system described above is described in detail herein to comply with patent law, provide those skilled in the art with the information necessary to apply the new principles, and configure the dedicated components required as required. Has been. However, it is to be understood that the present invention can be practiced with other equipment and devices, and that various details of the equipment and operation of the procedure can be achieved with various modifications without departing from the spirit of the invention.

Claims (20)

A method for database workload balancing between multiple database servers, comprising:
When a new server is available, select the set of master and slave databases that will be migrated to the new server and migrate the selected database to balance the workload among all servers while minimizing migration costs. Great and
A method of repeatedly selecting databases to be migrated to other servers one by one until the workload is balanced when workload imbalance is detected in real time due to workload fluctuations during operation.   The method of claim 1, wherein a metric for evaluating the goodness of a given database configuration is applied.   The method of claim 2, wherein the metrics are system resources including processor (CPU) usage and memory usage.   The method of claim 2, wherein the metric is a plurality of different factors in an integrated method with a weighting strategy and a vector norm.   The process of selecting a set of master database and slave database for migration includes (a) expected reduction cost after migration, (b) master to slave ratio of each server after migration, and (c) size of each database. The method of claim 1, further comprising the step of considering factors including associated migration costs.   The method of claim 1, further comprising selecting an optimal order of migration that has a minimal impact on the system during migration.   The method of claim 1, comprising repeatedly selecting a next database to migrate and a corresponding target database server to eliminate hot spots.   The method according to claim 1, wherein the database is migrated repeatedly one by one until the hot spot is eliminated. When c _mem and c _cpu are parameters for calibrating different units of memory and CPU,

The method of claim 1, further comprising the step of setting a metric for a predetermined database configuration including a variance and an average value determined in c _mem and c _cpu are

as well as

The method of claim 9, set to: A system for balancing database workload among multiple database servers,
A processor;
A database selected to allow the processor to select a master and slave set to migrate to the new server when a new server is available, and to balance the workload among all servers while minimizing migration costs Code executable on the processor, including instructions for migrating
Including instructions for causing the processor to detect in real time workload imbalance due to workload fluctuations during operation and repeatedly select databases to be migrated to other servers one by one until the workload is balanced Code executable on the processor;
Having a system.   The system of claim 11, comprising code for causing the processor to use a metric for evaluating the goodness of a given database configuration.   13. The system of claim 12, wherein the metrics are system resources including processor (CPU) usage and memory usage.   13. The system of claim 12, wherein the metric combines different factors in an integrated manner with a weighting strategy and a vector norm. The code to select the master database and slave database set for migration is
To allow the processor to consider factors including (a) the expected reduced cost after migration, (b) the master to slave ratio of each server after migration, and (c) the migration cost associated with the size of each database. The system of claim 11 further comprising a code.   The system of claim 11, comprising code for causing the processor to select an optimal order of migration that has a minimal impact on the system during migration.   12. The system of claim 11, comprising code for repeatedly causing the processor to select the next database to migrate and the corresponding target database server to eliminate hot spots.   12. The system of claim 11, comprising code for causing the processor to repeatedly migrate the database one at a time until the hot spot is resolved. When c _mem and c _cpu are parameters for calibrating different units of memory and CPU,

12. The system of claim 11, comprising code for setting metrics for a given database configuration including variance and average determined in c _mem and c _cpu

as well as

The system of claim 19, comprising a code for setting to:

Images