CN103593347A - Load balancing method and distributed database system - Google Patents
Load balancing method and distributed database system Download PDFInfo
- Publication number
- CN103593347A CN103593347A CN201210288434.7A CN201210288434A CN103593347A CN 103593347 A CN103593347 A CN 103593347A CN 201210288434 A CN201210288434 A CN 201210288434A CN 103593347 A CN103593347 A CN 103593347A
- Authority
- CN
- China
- Prior art keywords
- node
- file
- moving
- server node
- migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a load balancing method. The load balancing method comprises the steps: server nodes add currently-stored data to migration unit files in real time and establish migration unit file statistical tables according to the migration unit files; a master control server node periodically obtains the migration unit file statistical tables of the server nodes; the master control server node selects emigration nodes, emigration files of the emigration nodes and migration nodes corresponding to the emigration files as migration decisions according to the migration unit file statistical tables and migration strategies; the master control server node adjusts loads of the emigration nodes and the migration nodes according to the migration decisions when confirming the migration execution. The invention further discloses a distributed database system. By adopting the load balancing method, the load of the distributed database system can be balanced according to the load conditions of the server nodes.
Description
Technical field
The present invention relates to the database management technology of computer realm, relate in particular to a kind of method and distributed data base system of equally loaded.
Background technology
Distributed data base can by dynamic increase computing node and memory node, promote linearly overall performance and memory capacity in the situation that of non-interrupting service, and can be by the security of many copies technique guarantee data between the node of system.Described distributed data base is comprised of multiple servers node, and data are distributed on multiple servers node; Central processing unit in each server node (CPU, Central Processing Unit) and the synchronous deal with data of hard disk.The method of distributed data library storage data is: client, by consistency algorithm, after a selected station server node, sends the data to this server node and preserves.
But, because the selected mode of the server node of above-mentioned save data is that client is passed through consistency algorithm, such as Hash (Hash) algorithm carrys out selected server node, yet client is when being used consistency algorithm to select server, cannot judge the situation of canned data in each server, so may there is the problem of distributed data base system load imbalance, and then affect the storage service of system.Such as, the data of storing in server node A approach 100% of this server disk capacity, and the data of preserving in server node B take the disk size less than 10%, but client is stored data according to consistency algorithm selected server node A, so just may occur that server node A cannot store the situation of data.
Visible, in prior art, because storage is during data, do not consider the loading condition of each server node in current system, cause distributed data base to occur load imbalance problem.
Summary of the invention
In view of this, the object of the present invention is to provide a kind of method and distributed data base system of equally loaded, can be according to the loading condition of each server node, the load of equiblibrium mass distribution formula database.
For achieving the above object, technical scheme of the present invention is achieved in that
A kind of method that the invention provides equally loaded, the method comprises:
Server node adds the data of current storage to migration units file in real time, and according to migration units file set up migration units file statistical form;
Main control server node periodically obtains the migration units file statistical form of each server node;
Main control server node is according to migration units file statistical form and migration strategy, and move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making;
During the migration of main control server node determination, according to migration decision-making, adjust the load of move out node and the node of moving into.
In such scheme, described migration units file statistical form, comprise: the sign of server node, disk total volume and disk have been used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file.
In such scheme, described migration strategy, comprising: following one or more strategy:
Select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected migration units file is as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
According to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected migration units file is as the file of moving out.
In such scheme, before described main control server node periodically obtains the migration units file statistical form of each server node, the method also comprises: main control server node detects the state of Servers-all node in real time, if there is the state of server node to exit for preparing, using this server node as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into.
In such scheme, described according to migration decision-making, the move out load of node and the node of moving into of adjustment, comprise: main control server, according to the node of moving out of selecting, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, sends the notice that sends the file of moving out to the node of moving into respectively the corresponding node of moving out;
Move out node according to the move out notice of file of transmission, extract the file of moving out and send to the node of moving into;
When main control server node, receive the node of moving out and return while moving out successfully notice, the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive;
When main control server node, receive the node of moving into and returned after the notice of moving into, upgrade the migration units file statistical form of self.
The present invention also provides a kind of distributed data base system, and this system comprises: main control server node and server node; Wherein,
Server node, for adding the data of current storage to migration units file in real time, according to migration units file set up migration units file statistical form, and for main control server node provides migration units file statistical form, and receive the adjustment of main control server node to load as move into node or the node of moving out;
Main control server node, for periodically obtaining the migration units file statistical form of each server node; According to migration units file statistical form and migration strategy, move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making; And determine while moving, according to migration decision-making, adjust the load of move out node and the node of moving into.
In such scheme, described server node, specifically for utilizing sign, disk total volume and the disk of server node, used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file are set up migration units file statistical form.
In such scheme, described main control server node, specifically for preserving migration strategy, described migration strategy comprises following one or more strategy:
Select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected migration units file is as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
According to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected migration units file is as the file of moving out
In such scheme, described main control server node, also for detecting in real time the state of Servers-all node, if there is the state of server node to exit for preparing, using this server node as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
Accordingly, described server node, is also used to main control server node to provide the state of self to exit or normally operation for preparing.
In such scheme, described main control server node, specifically for according to the node of moving out selected, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, send to conduct the move out notice of file of the server node transmission of node of moving into respectively the corresponding node of moving out; When receiving as the server node of node of moving into, return to while moving out successfully notice the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive; When receiving that the server node as the node of moving into has returned after the notice of moving into, and upgrades the migration units file statistical form of self;
Accordingly, described server node, while moving out node specifically for conduct, sends the notice of the file of moving out according to main control server node, extract the file of moving out and send to the node of moving into, and to main control server node, returns to the successfully notice of moving out; And when moving into node, after completing and moving into, to main control server node, returned to the notice of moving into.
The method of equally loaded provided by the present invention and distributed data base system, migration units file statistical form and the migration strategy that can periodically according to Servers-all node, be set up by main control server node, move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making; Again according to the move out load of node and the node of moving into of migration decision-making adjustment.So, can adjust the load of each server node according to the situation of the storage data of each server node in distributed data base system, and then realize the load balancing of distributed data base.
Accompanying drawing explanation
Fig. 1 is the method flow schematic diagram of equally loaded of the present invention;
Fig. 2 is that distributed data base system of the present invention forms structural representation.
Embodiment
Basic thought of the present invention is: each server node adds the data of current storage to migration units file in real time, and according to migration units file set up migration units file statistical form; Main control server node periodically obtains the migration units file statistical form of each server node; Main control server node is according to migration units file statistical form and migration strategy, and move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making; During the migration of main control server node determination, according to migration decision-making, adjust the load of move out node and the node of moving into.
Below in conjunction with drawings and the specific embodiments, the present invention is further described in more detail.
The method of the equally loaded that the present invention proposes, as shown in Figure 1, comprises the following steps:
Step 101: each server node adds the data of current storage to migration units file in real time, and according to migration units file set up migration units file statistical form.
Here, the described data by current storage are added migration units file to: whether the real-time detection of server node receives the data that client is sent, if receive, whether server node detects current migration units file over dominant record threshold value, if surpass, stop upgrading current migration units file, newly-built migration units file, as current migration units file, stores data in current migration units file into; If do not surpass, server node directly stores data in current migration units file.Wherein, described dominant record threshold value is the numerical value preset according to actual conditions, can be the maximum quantity of the data that record in migration units file, maximum disk capacity that also can be shared for the data of preserving in migration units file.
Step 102: main control server node periodically obtains the migration units file statistical form of each server node.
Here, described migration units file statistical form comprises: the sign of server node, disk total volume and disk have been used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file;
Wherein, the sign of described server node can be the address of server node in system;
Described migration units fileinfo comprises: migration units quantity of documents, filename, and described filename can be comprised of the sign of server node and/or the order of setting up of this migration units file; Described periodicity is the duration arranging according to actual conditions, such as, can be set to one day; Described obtaining can, for each server node periodically reports migration units file statistical form to main control server, also can periodically be extracted migration units file statistical information for main control server from each server node of self-management.
Step 103: main control server node is according to all migration units file statistical forms and migration strategy, move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making.
Here, described migration strategy is for to be preset at following one or more strategy in main control server according to actual conditions:
One, select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Wherein, described from the migration units file statistical form of selecting, selected migration units file of moving out in node can arrange selected rule according to actual conditions as the file of moving out, such as, selected rule can be set is: calculate disk and used the difference between capacity and threshold value, the disk size sum that migration units file is taken, one or more migration units files consistent with difference are as the file of moving out; Described threshold values can be according to actual conditions setting, such as being set to 90% of disk total volume;
Described from server node corresponding to residue migration units file statistical form, selecting the node of moving into that each file of moving out is corresponding can be: according to current Servers-all node, use disk size, calculate and on average used disk size, from server node corresponding to residue migration units file statistical form, select with disk size lower than average with the server node of disk size as the node of moving into, calculating move into node by disk size and on average used the difference between disk size, from all files of moving out, select one or more files of moving out that the disk size sum that takies is identical with described difference corresponding with this node of moving into.
Two, calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected one or more migration units files are as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
Wherein, described prevalue can be ratio value, and difference and disk are with the ratio of disk total volume corresponding to the maximum server node of capacity.
Three, according to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected one or more migration units files are as the file of moving out;
Wherein, described from server node corresponding to residue migration units file statistical form the selected node of moving out, and selected one or more migration units files can be according to actual conditions setting as the file of moving out, such as, can be: according to current Servers-all node, use disk size, calculate and on average used disk size, from residue server node, select with disk size higher than average with the server node of disk size as the node of moving out, according to the node of moving out by disk size and on average used the difference between disk size, select one or more files of moving out.
Step 104: during the migration of main control server node determination, according to migration decision-making, adjust the load of move out node and the node of moving into.
Here, described definite migration can be by artificial execution migration order, or whether main control server Node configuration selected for first detecting move out node and the node of moving into, and if so, determine and move, if not, end process flow process;
Described according to migration decision-making, the load of adjusting move out node and the node of moving into comprises:
Step a: the current all migration units file statistical forms self preserved of main control server locking, and the migration units file statistical form of notice all move out node and the node lockings self of moving into selected; Main control server, according to the node of moving out of selecting, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, sends the notice that sends the file of moving out to the node of moving into respectively the corresponding node of moving out.
Wherein, described in, be locked as and stop any operation to migration units file statistical form.
Step b: the node of moving out, according to the move out notice of file of transmission, extracts the file of moving out and sends to the node of moving into, and upgrade the migration units file statistical form of self.
Wherein, described send mode can adopt file transfer protocol (FTP) (FTP, File Transfer Protocol) to transmit; The described extraction file of moving out sends to the node of moving into for extracting move out file compression, the file of moving out after compression to be sent to the node of moving into; Described boil down to prior art, does not repeat here.
Further, described extraction is moved out after file sends to the node of moving into, main control server node also can Real-Time Monitoring compressed file whether failure and Internet Transmission whether abnormal, if compressed file failure or Internet Transmission are abnormal, judge whether to recover, if can recover, judge whether the current number of times that repeats surpasses maximal value, if, end process flow process, if not, extract the file of moving out and send to the node of moving into, and will repeat number of times and add one; If cannot recover, migration operation failure, charges to system exception daily record, end process flow process; If compressed file success or Internet Transmission are normal, perform step c.
Step c: receive any one node of moving out when main control server node and return while moving out successfully notice, notify all nodes of moving into corresponding to the file of moving out of this node of moving out, process the file of moving out of receiving.
Wherein, described in move out before node returns to the notice of moving out successfully, also comprise: described in move out after the notice that node receives that Data Migration starts, by the identifier notice main control server node of the session instance of executing data migration; Described main control server is received after the message that comprises session identifier, periodically to the node of moving out, sends heartbeat message; The described node of moving out is received after heartbeat message, whether detects the session of specified session identifier in operation, if, continue to respond heartbeat message to main control server node, if surpass preset duration, can't detect heartbeat, decision data migration operation failure, notice main control node is moved out unsuccessfully.
Described session identifier can be current migration operation serial number; The transmission of described heartbeat message is the method adopting when system equipment is managed in prior art with responding, and does not repeat here.
Described main control server node receives that any one node of moving out returns to the successfully notice of moving out and comprises: whether the real-time detection of main control server node receives that the node of moving out returns to the successfully notice of moving out, when moving out successfully during notice of receiving that the node of moving out returns, the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive; When receiving the notice of the failure of moving out that the node of moving out returns, finish the migration process that this node of moving out is corresponding.
The file of moving out that described processing is received is: the node of moving into is preserved the file of moving out that the node of moving out is sent, and upgrades the migration units file statistical form of self, and to main control server node, has returned to the notice of moving into.Wherein, the described preservation file of moving out can comprise: after the file of moving out after the compression receiving is decompressed, preserve; Described decompress(ion) is condensed to the method corresponding with compression method, is prior art, does not repeat here.
Steps d: receive any one node of moving into when main control server node and returned after the notice of moving into, upgrade the migration units file statistical form of self.
Wherein, described in, be updated to record corresponding to the file of moving out changed.
In addition, before carrying out above-mentioned steps 101, the method can also comprise: main control server node detects the state of Servers-all node in real time, if there is the state of any one server node to exit for preparing, using this server node as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into.
Wherein, described from server node corresponding to residue migration units file statistical form, for each file of moving out, selecting the node of moving into can be: according to current all residue server nodes, use disk size, calculate and on average used disk size, select with disk size lower than average with the server node of disk size as the node of moving into, according to the node of moving into by disk size and on average used the difference between disk size, the node of moving into corresponding to one or more files of moving out of selecting one by one.
As shown in Figure 2, the invention provides a kind of distributed data base system, this system comprises: main control server node 21 and server node 22; Wherein,
Main control server node 21, for periodically obtaining the migration units file statistical form of each server node 22; According to migration units file statistical form and migration strategy, move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making; And determine while moving, according to migration decision-making, adjust the load of move out node and the node of moving into.
Described server node 22, specifically for detection in real time, whether receive the data that client is sent, if receive, detect current migration units file and whether surpass dominant record threshold value, if surpass, stop upgrading current migration units file, newly-built migration units file, as current migration units file, stores data in current migration units file into; If do not surpass, data are directly stored in current migration units file.
Described server node 22, specifically for preserving migration units file statistical form, comprise: the sign of server node, disk total volume and disk have been used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file.
Described main control server node 21, specifically for preserving migration strategy; Described migration strategy is following one or more strategy preset according to actual conditions:
One, select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Wherein, described from the migration units file statistical form of selecting, selected migration units file of moving out in node can arrange selected rule according to actual conditions as the file of moving out, such as, selected rule can be set is: calculate disk and used the difference between capacity and threshold value, the disk size sum that migration units file is taken, one or more migration units files consistent with difference are as the file of moving out; Described threshold values can be according to actual conditions setting, such as being set to 90% of disk total volume;
Described from server node corresponding to residue migration units file statistical form, selecting the node of moving into that each file of moving out is corresponding can be: according to current Servers-all node, use disk size, calculate and on average used disk size, from server node corresponding to residue migration units file statistical form, select with disk size lower than average with the server node of disk size as the node of moving into, calculating move into node by disk size and on average used the difference between disk size, from all files of moving out, select one or more files of moving out that the disk size sum that takies is identical with described difference corresponding with this node of moving into.
Two, calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected one or more migration units files are as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
Wherein, described prevalue can be ratio value, and difference and disk are with the ratio of disk total volume corresponding to the maximum server node of capacity.
Three, according to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected one or more migration units files are as the file of moving out;
Wherein, described from server node corresponding to residue migration units file statistical form the selected node of moving out, and selected one or more migration units files can be according to actual conditions setting as the file of moving out, such as, can be: according to current Servers-all node, use disk size, calculate and on average used disk size, from residue server node, select with disk size higher than average with the server node of disk size as the node of moving out, according to the node of moving out by disk size and on average used the difference between disk size, select one or more files of moving out.
Described main control server node 21, specifically for being determined and started migration by artificial execution migration order, or is set to first detect whether selected move out node and the node of moving into, and if so, determine and move, if not, end process flow process.
Described main control server node 21, specifically for locking current all migration units file statistical forms of self preserving, and the migration units file statistical form of notice all move out node and the node lockings self of moving into selected, main control server, according to the node of moving out of selecting, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, sends the notice that sends the file of moving out to the node of moving into respectively the corresponding node of moving out.
Described server node 22, specifically for when moving out node, according to the move out notice of file of transmission, extracts the file of moving out and sends to the node of moving into, and upgrade the migration units file statistical form of self.
Described server node 22, specifically for extracting move out file compression, sends to by the file of moving out after compression the node of moving into.
Described main control server node 21, specifically for Real-Time Monitoring as server node 22 compressed files of the node of moving out whether failure and Internet Transmission whether abnormal, if compressed file failure or Internet Transmission are abnormal, judge whether to recover, if can recover, judge whether the current number of times that repeats surpasses maximal value, if so, end process flow process, if not, extract the file of moving out and send to the node of moving into, and will repeat number of times and add one; If cannot recover, migration operation failure, charges to system exception daily record, end process flow process.
Described server node 22, while also moving out node for conduct, receive after the notice that Data Migration starts, by the identifier notice main control module of the session instance of executing data migration, and receive after the heartbeat message that main control server node 21 sends, whether detect the session of the session identifier that heartbeat message is corresponding in operation, if, continue to receive the response to main control server node 21, if surpass preset duration, can't detect heartbeat, decision data migration operation failure, notice main control server node 21 is moved out unsuccessfully; Accordingly, described main control server node 21, also for receiving after the message that comprises session identifier that server node 22 sends, periodically to the move out server node 22 transmission heartbeat message of node of conduct.
Described main control server node 21, specifically for detection in real time, whether receive that the node of moving out returns to the successfully notice of moving out, when receiving while moving out successfully notice, the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive, when receiving the notice of the failure of moving out that the node of moving out returns, finish the migration process that this node of moving out is corresponding.
Described server node 22, while moving into node specifically for conduct, preserves the file of moving out that the node of moving out is sent, and upgrades the migration units file statistical form of self, and to main control server node 21, has returned to the notice of moving into.
Described server node 22, while moving into node specifically for conduct, preserves after the file of moving out after the compression receiving is decompressed; Described decompress(ion) is condensed to the method corresponding with compression method, is prior art, does not repeat here.
Described main control server node 21, also for detecting in real time the state of server node 22, if there is the state of server node 22 to exit for preparing, using this server node 22 as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node 22 corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into; Accordingly, described server node 22, is also used to main control server node 21 to provide the state of self to exit or normally operation for preparing.
Described main control server node 21, specifically for using disk size according to current all residue server nodes 22, calculate and on average used disk size, select by disk size lower than average with server node 22 conducts of the disk size node of moving into, according to the node of moving into by disk size and on average used the difference between disk size, select one by one the node of moving into that one or more files of moving out are corresponding.
Main control server node 21 in said system and server node 22, except possessing function provided by the invention, all have all functions that the node device in the distributed data base of stipulating in prior art should possess, and do not repeat here.
The above, be only preferred embodiment of the present invention, is not intended to limit protection scope of the present invention.
Claims (10)
1. a method for equally loaded, is characterized in that, the method comprises:
Server node adds the data of current storage to migration units file in real time, and according to migration units file set up migration units file statistical form;
Main control server node periodically obtains the migration units file statistical form of each server node;
Main control server node is according to migration units file statistical form and migration strategy, and move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making;
During the migration of main control server node determination, according to migration decision-making, adjust the load of move out node and the node of moving into.
2. method according to claim 1, it is characterized in that, described migration units file statistical form, comprise: the sign of server node, disk total volume and disk have been used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file.
3. method according to claim 1, is characterized in that, described migration strategy, comprising: following one or more strategy:
Select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected migration units file is as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
According to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected migration units file is as the file of moving out.
4. method according to claim 1, it is characterized in that, before described main control server node periodically obtains the migration units file statistical form of each server node, the method also comprises: main control server node detects the state of Servers-all node in real time, if there is the state of server node to exit for preparing, using this server node as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into.
5. method according to claim 1, it is characterized in that, described according to migration decision-making, the move out load of node and the node of moving into of adjustment, comprise: main control server, according to the node of moving out of selecting, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, sends the notice that sends the file of moving out to the node of moving into respectively the corresponding node of moving out;
Move out node according to the move out notice of file of transmission, extract the file of moving out and send to the node of moving into;
When main control server node, receive the node of moving out and return while moving out successfully notice, the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive;
When main control server node, receive the node of moving into and returned after the notice of moving into, upgrade the migration units file statistical form of self.
6. a distributed data base system, is characterized in that, this system comprises: main control server node and server node; Wherein,
Server node, for adding the data of current storage to migration units file in real time, according to migration units file set up migration units file statistical form, and for main control server node provides migration units file statistical form, and receive the adjustment of main control server node to load as move into node or the node of moving out;
Main control server node, for periodically obtaining the migration units file statistical form of each server node; According to migration units file statistical form and migration strategy, move out file and the node of moving into corresponding to the file of moving out of selected move out node, the node of moving out are as migration decision-making; And determine while moving, according to migration decision-making, adjust the load of move out node and the node of moving into.
7. system according to claim 6, it is characterized in that, described server node, specifically for utilizing sign, disk total volume and the disk of server node, used capacity, all migration units fileinfos of preserving in server node, the disk size that each migration units file takies and the visiting frequency of each migration units file are set up migration units file statistical form.
8. system according to claim 6, is characterized in that, described main control server node, and specifically for preserving migration strategy, described migration strategy comprises following one or more strategy:
Select disk and with capacity, surpassed the migration units file statistical form of threshold values, using server node corresponding to the migration units file statistical form of selecting as the node of moving out; From the migration units file statistical form of selecting, selected migration units file of moving out in node is as the file of moving out; From server node corresponding to residue migration units file statistical form, select the node of moving into that each file of moving out is corresponding;
Calculate disk and with the maximum server node of capacity and disk, used the difference between the minimum server node of capacity, when this difference is greater than prevalue, using disk with the maximum server node of capacity as the node of moving out, from corresponding migration units file statistical form, selected migration units file is as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
According to migration units file statistical form, select disk with capacity lower than server node corresponding to the migration units file statistical form of threshold values as the node of moving into, the selected node of moving out from server node corresponding to residue migration units file statistical form, and selected migration units file is as the file of moving out.
9. system according to claim 8, is characterized in that,
Described main control server node, also for detecting in real time the state of Servers-all node, if there is the state of server node to exit for preparing, using this server node as the node of moving out, all migration units files that this is moved out in node are all as the file of moving out, from server node corresponding to residue migration units file statistical form, for each file of moving out is selected the node of moving into;
Accordingly, described server node, is also used to main control server node to provide the state of self to exit or normally operation for preparing.
10. system according to claim 6, is characterized in that,
Described main control server node, specifically for according to the node of moving out selected, move out file and the node of moving into corresponding to the file of moving out of the node of moving out, send to conduct the move out notice of file of the server node transmission of node of moving into respectively the corresponding node of moving out; When receiving as the server node of node of moving into, return to while moving out successfully notice the file of moving out of notifying all node processing of moving into corresponding to the file of moving out of this node of moving out to receive; When receiving that the server node as the node of moving into has returned after the notice of moving into, and upgrades the migration units file statistical form of self;
Accordingly, described server node, while moving out node specifically for conduct, sends the notice of the file of moving out according to main control server node, extract the file of moving out and send to the node of moving into, and to main control server node, returns to the successfully notice of moving out; And when moving into node, after completing and moving into, to main control server node, returned to the notice of moving into.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210288434.7A CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210288434.7A CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103593347A true CN103593347A (en) | 2014-02-19 |
| CN103593347B CN103593347B (en) | 2017-06-13 |
Family
ID=50083496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210288434.7A Active CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103593347B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049508A (en) * | 2015-07-21 | 2015-11-11 | 齐鲁工业大学 | Cloud data migration method |
| CN105227645A (en) * | 2015-09-15 | 2016-01-06 | 齐鲁工业大学 | A kind of cloud data migration method |
| CN105245381A (en) * | 2015-10-22 | 2016-01-13 | 上海斐讯数据通信技术有限公司 | Cloud server downtime monitoring migration system and method |
| CN105488079A (en) * | 2014-10-11 | 2016-04-13 | 苏州精易会信息技术有限公司 | Method for processing multi-tenant data |
| WO2016165441A1 (en) * | 2015-09-06 | 2016-10-20 | 中兴通讯股份有限公司 | Migration policy adjustment method, capacity-change suggestion method and device |
| WO2018014613A1 (en) * | 2016-07-19 | 2018-01-25 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for migrating data |
| CN108710925A (en) * | 2018-05-15 | 2018-10-26 | 南京博内特信息科技有限公司 | A method of the clothes commodity shelf system based on Internet of Things |
| CN108781189A (en) * | 2016-09-26 | 2018-11-09 | 华为技术有限公司 | A kind of method and relevant device of load balancing |
| WO2019061132A1 (en) * | 2017-09-28 | 2019-04-04 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
| CN110377430A (en) * | 2019-07-24 | 2019-10-25 | 中南民族大学 | Data migration method, equipment, storage medium and device |
| CN111143324A (en) * | 2019-12-20 | 2020-05-12 | 浪潮软件股份有限公司 | Kudu database data balancing system based on size and implementation method |
| EP3703342A1 (en) * | 2019-03-01 | 2020-09-02 | ABB Schweiz AG | Dynamic load balancing in network centric process control systems |
| CN113268203A (en) * | 2021-05-18 | 2021-08-17 | 天津中科曙光存储科技有限公司 | Capacity balancing method and device of storage system, computer equipment and storage medium |
| CN113821176A (en) * | 2021-09-29 | 2021-12-21 | 重庆紫光华山智安科技有限公司 | Data migration processing method and device and storage medium |
| CN116389442A (en) * | 2023-02-10 | 2023-07-04 | 上海弘积信息科技有限公司 | Web page request system and method based on load balancing |
| CN116991580A (en) * | 2023-07-27 | 2023-11-03 | 上海沄熹科技有限公司 | Distributed database system load balancing method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582850B (en) * | 2009-06-19 | 2012-05-02 | 优万科技（北京）有限公司 | Method and system for realizing load balance |
| CN102457428B (en) * | 2010-10-27 | 2015-10-21 | 中兴通讯股份有限公司 | The implementation of load balancing of distributed hashtable network and device |
-
2012
- 2012-08-14 CN CN201210288434.7A patent/CN103593347B/en active Active
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105488079A (en) * | 2014-10-11 | 2016-04-13 | 苏州精易会信息技术有限公司 | Method for processing multi-tenant data |
| CN105049508A (en) * | 2015-07-21 | 2015-11-11 | 齐鲁工业大学 | Cloud data migration method |
| WO2016165441A1 (en) * | 2015-09-06 | 2016-10-20 | 中兴通讯股份有限公司 | Migration policy adjustment method, capacity-change suggestion method and device |
| CN105227645A (en) * | 2015-09-15 | 2016-01-06 | 齐鲁工业大学 | A kind of cloud data migration method |
| CN105245381B (en) * | 2015-10-22 | 2019-08-16 | 上海斐讯数据通信技术有限公司 | Cloud Server delay machine monitors migratory system and method |
| CN105245381A (en) * | 2015-10-22 | 2016-01-13 | 上海斐讯数据通信技术有限公司 | Cloud server downtime monitoring migration system and method |
| US11620067B2 (en) | 2016-07-19 | 2023-04-04 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for migrating data |
| US10922004B2 (en) | 2016-07-19 | 2021-02-16 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for migrating data records |
| WO2018014613A1 (en) * | 2016-07-19 | 2018-01-25 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for migrating data |
| CN108781189A (en) * | 2016-09-26 | 2018-11-09 | 华为技术有限公司 | A kind of method and relevant device of load balancing |
| CN108781189B (en) * | 2016-09-26 | 2020-12-22 | 华为技术有限公司 | Load balancing method and related equipment |
| WO2019061132A1 (en) * | 2017-09-28 | 2019-04-04 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
| CN111095233B (en) * | 2017-09-28 | 2023-09-26 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, live migration and applications thereof |
| CN111095233A (en) * | 2017-09-28 | 2020-05-01 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration and applications thereof |
| US10810169B1 (en) | 2017-09-28 | 2020-10-20 | Research Institute Of Tsinghua University In Shenzhen | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
| CN108710925A (en) * | 2018-05-15 | 2018-10-26 | 南京博内特信息科技有限公司 | A method of the clothes commodity shelf system based on Internet of Things |
| CN111641669A (en) * | 2019-03-01 | 2020-09-08 | Abb瑞士股份有限公司 | Dynamic load balancing in a network-centric process control system |
| EP3703342A1 (en) * | 2019-03-01 | 2020-09-02 | ABB Schweiz AG | Dynamic load balancing in network centric process control systems |
| US11500690B2 (en) | 2019-03-01 | 2022-11-15 | Abb Schweiz Ag | Dynamic load balancing in network centric process control systems |
| CN111641669B (en) * | 2019-03-01 | 2024-05-17 | Abb瑞士股份有限公司 | Dynamic load balancing in a network-centric process control system |
| CN110377430B (en) * | 2019-07-24 | 2021-08-31 | 中南民族大学 | Data migration method, device, storage medium and device |
| CN110377430A (en) * | 2019-07-24 | 2019-10-25 | 中南民族大学 | Data migration method, equipment, storage medium and device |
| CN111143324A (en) * | 2019-12-20 | 2020-05-12 | 浪潮软件股份有限公司 | Kudu database data balancing system based on size and implementation method |
| CN111143324B (en) * | 2019-12-20 | 2023-05-02 | 浪潮软件股份有限公司 | Kudu database data equalization system based on size and implementation method |
| CN113268203A (en) * | 2021-05-18 | 2021-08-17 | 天津中科曙光存储科技有限公司 | Capacity balancing method and device of storage system, computer equipment and storage medium |
| CN113268203B (en) * | 2021-05-18 | 2022-11-04 | 天津中科曙光存储科技有限公司 | Capacity balancing method and device of storage system, computer equipment and storage medium |
| CN113821176A (en) * | 2021-09-29 | 2021-12-21 | 重庆紫光华山智安科技有限公司 | Data migration processing method and device and storage medium |
| CN116389442A (en) * | 2023-02-10 | 2023-07-04 | 上海弘积信息科技有限公司 | Web page request system and method based on load balancing |
| CN116991580A (en) * | 2023-07-27 | 2023-11-03 | 上海沄熹科技有限公司 | Distributed database system load balancing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103593347B (en) | 2017-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103593347A (en) | Load balancing method and distributed database system | |
| US8533731B2 (en) | Apparatus and method for distrubuting complex events based on correlations therebetween | |
| US8418003B2 (en) | Log collecting apparatus, program, method, and system | |
| CN104836819A (en) | Dynamic load balancing method and system, and monitoring and dispatching device | |
| CN103095824B (en) | Files passe control method and system | |
| CN103873507A (en) | Data block uploading and storing system and method | |
| CN107872517B (en) | Data processing method and device | |
| CN113655969B (en) | Data balanced storage method based on streaming distributed storage system | |
| CN106470123A (en) | Log collecting method, client, server and electronic equipment | |
| CN104202385A (en) | Data backup and updating method of distributed file system | |
| CN105242983A (en) | Data storage method and data storage management server | |
| CN109584105B (en) | Service response method and system | |
| CN106959820A (en) | A kind of data extraction method and system | |
| EP2432193A2 (en) | Method of data replication in a distributed data storage system and corresponding device | |
| CN104202386A (en) | High-concurrency distributed file system and secondary load balancing method thereof | |
| WO2022068392A1 (en) | Database cluster capacity expansion and reduction method, service system and storage medium | |
| CN109117292A (en) | A kind of method and apparatus and cluster storage system of cluster-based storage | |
| CN112436962B (en) | Block chain consensus network dynamic expansion method, electronic device, system and medium | |
| CN104915376A (en) | Cloud storage file archiving and compressing method | |
| CN115686368A (en) | Method, system, apparatus and medium for storage capacity expansion of nodes of block chain network | |
| CN107846429B (en) | File backup method, device and system | |
| CN109254880A (en) | A kind of method and device handling database delay machine | |
| US10089149B2 (en) | Method for scheduling multiple periodic requests and scheduling device | |
| CN114039836A (en) | Fault processing method and device for Exporter collector | |
| US20120072160A1 (en) | Measure presentation device, measure presentation method, and non-transitory computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |