CN103593347B - The method and distributed data base system of a kind of equally loaded - Google Patents
The method and distributed data base system of a kind of equally loaded Download PDFInfo
- Publication number
- CN103593347B CN103593347B CN201210288434.7A CN201210288434A CN103593347B CN 103593347 B CN103593347 B CN 103593347B CN 201210288434 A CN201210288434 A CN 201210288434A CN 103593347 B CN103593347 B CN 103593347B
- Authority
- CN
- China
- Prior art keywords
- node
- file
- moving
- migration
- server node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method of equally loaded, including：Currently stored data are added to migration units file in real time, and set up migration units file statistical form according to migration units file by server node；Main control server node periodically obtains the migration units file statistical form of each server node；According to migration units file statistical form and migration strategy, selected node of moving out, move out file and the corresponding node of moving into of file of moving out of node of moving out are used as migration decision-making for main control server node；When main control server node determines migration, according to migration decision-making, the load of move out node and node of moving into is adjusted.The present invention further simultaneously discloses a kind of distributed data base system, can be according to the loading condition of each server node, the load of equiblibrium mass distribution formula database using the present invention.
Description
Technical field
The present invention relates to the database management technology of computer realm, more particularly to a kind of equally loaded method and distribution
Formula Database Systems.
Background technology
Distributed data base can increase calculate node and memory node in the case of non-interrupting service by dynamic,
Overall performance and memory capacity are linearly lifted, and the safety of data can be ensured by multi-duplicate technology between the node of system
Property.The distributed data base is made up of multiple servers node, and data are distributed on multiple servers node；Each service
Central processing unit (CPU, Central Processing Unit) and hard disk synchronization process data in device node.Distributed number
Method according to library storage data is：Client after selecting a server node, sends the data to this by consistency algorithm
Server node is preserved.
But, because the selected mode of the server node of above-mentioned preservation data is that client passes through consistency algorithm, than
As Hash (Hash) algorithm carrys out selected server node, but client using consistency algorithm when server is selected, it is impossible to
Judge the situation of canned data in each server, be so likely to occur the problem of distributed data base system load imbalance,
And then influence the storage service of system.Such as, the data for being stored in server node A are close to the server disk capacity
100%, and the data preserved in server node B take the disk size less than 10%, but client is according to consistency algorithm
Selected server node A carrys out data storage, thus it is possible that server node A cannot data storage situation.
It can be seen that, when in the prior art due to data storage, do not account for the load of each server node in current system
Situation, causes distributed data base load imbalance problem occur.
The content of the invention
In view of this, it is an object of the invention to provide the method and distributed data base system of a kind of equally loaded, energy
According to the loading condition of each server node, the load of equiblibrium mass distribution formula database.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
The invention provides a kind of method of equally loaded, the method includes：
Currently stored data are added to migration units file in real time, and build according to migration units file by server node
Vertical migration units file statistical form；
Main control server node periodically obtains the migration units file statistical form of each server node；
Main control server node selectes move out node, node of moving out according to migration units file statistical form and migration strategy
Move out file and file of moving out it is corresponding move into node as migration decision-making；
When main control server node determines migration, according to migration decision-making, the load of move out node and node of moving into is adjusted.
In such scheme, the migration units file statistical form, including：The mark of server node, disk total capacity and
Disk capacity, all migration units fileinfos preserved in server node, the magnetic that each migration units file takes
The visiting frequency of disk capacity and each migration units file.
In such scheme, the migration strategy, including：One or more is tactful below：
The migration units file statistical form that disk has exceeded threshold values with capacity is selected, the migration units file statistics that will be selected
The corresponding server node of table is used as node of moving out；From the migration units file statistical form selected, in selected node of moving out
Migration units file is used as file of moving out；From the corresponding server node of remaining migration units file statistical form, each is selected
The corresponding node of moving into of file of moving out；
Disk is calculated with the most server node of capacity and disk between the minimum server node of capacity
Difference, when the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from corresponding
In migration units file statistical form, migration units file is selected as file of moving out, from remaining migration units file statistical form pair
It is that each file of moving out selects node of moving into the server node answered；
According to migration units file statistical form, migration units file statistical form pair of the disk with capacity less than threshold values is selected
The server node answered is selected from the corresponding server node of remaining migration units file statistical form and moved out as node of moving into
Node, and selected migration units file is used as file of moving out.
In such scheme, the main control server node periodically obtains the migration units file system of each server node
Before meter table, the method also includes：The state of main control server node real-time detection Servers-all node, if there is server section
The state of point is exited for preparation, then using the server node as node of moving out, by all migration units in the node of moving out
File is each file of moving out from the corresponding server node of remaining migration units file statistical form as file of moving out
Select node of moving into.
It is described according to migration decision-making in such scheme, the load of move out node and node of moving into is adjusted, including：Master control takes
Business device according to move out file and the corresponding node of moving into of file of moving out of the node of moving out, node of moving out selected, respectively to right
The node of moving out answered sends the notice that file of moving out is sent to node of moving into；
Node move out according to the notice for sending file of moving out, extracts file of moving out and is sent to node of moving into；
When main control server node receive move out node return move out successfully notify when, notify moving out for the node of moving out
The file of moving out that the corresponding all node processings of moving into of file are received；
After main control server node receives node of moving into returns to the notice for completing to move into, the migration units text of itself is updated
Part statistical form.
Present invention also offers a kind of distributed data base system, the system includes：Main control server node and server
Node；Wherein,
Server node, for currently stored data to be added into migration units file in real time, according to migration units text
Part sets up migration units file statistical form, and for master control server node provides migration units file statistical form, and conduct is moved
Ingress or node of moving out receive adjustment of the main control server node to loading；
Main control server node, the migration units file statistical form for periodically obtaining each server node；According to
Migration units file statistical form and migration strategy, selected node of moving out, move out file and file of the moving out correspondence of node of moving out
Move into node as migration decision-making；And when determining migration, according to migration decision-making, adjustment is moved out the negative of node and node of moving into
Carry.
In such scheme, the server node, specifically for the mark using server node, disk total capacity and magnetic
Disk capacity, all migration units fileinfos preserved in server node, the disk that each migration units file takes
The visiting frequency of capacity and each migration units file sets up migration units file statistical form.
In such scheme, the main control server node, specifically for preserve migration strategy, the migration strategy include with
Lower one or more strategy：
The migration units file statistical form that disk has exceeded threshold values with capacity is selected, the migration units file statistics that will be selected
The corresponding server node of table is used as node of moving out；From the migration units file statistical form selected, in selected node of moving out
Migration units file is used as file of moving out；From the corresponding server node of remaining migration units file statistical form, each is selected
The corresponding node of moving into of file of moving out；
Disk is calculated with the most server node of capacity and disk between the minimum server node of capacity
Difference, when the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from corresponding
In migration units file statistical form, migration units file is selected as file of moving out, from remaining migration units file statistical form pair
It is that each file of moving out selects node of moving into the server node answered；
According to migration units file statistical form, migration units file statistical form pair of the disk with capacity less than threshold values is selected
The server node answered is selected from the corresponding server node of remaining migration units file statistical form and moved out as node of moving into
Node, and selected migration units file is used as file of moving out
In such scheme, the main control server node is additionally operable to the state of real-time detection Servers-all node, if having
The state of server node is exited for preparation, then using the server node as node of moving out, will be all in the node of moving out
Migration units file is each from the corresponding server node of remaining migration units file statistical form as file of moving out
File of moving out selects node of moving into；
Accordingly, the server node, is additionally operable to provide the state of itself for preparation is exited for master control server node
Also it is up.
In such scheme, the main control server node, specifically for being moved according to the node of moving out selected, node of moving out
Go out file and the corresponding node of moving into of file of moving out, sent to the clothes as node of moving into corresponding node of moving out respectively
Business device node sends the notice of file of moving out；Successfully notice of moving out is returned when the server node as node of moving into is received
When, the file of moving out for notifying the corresponding all node processings of moving into of file of moving out of the node of moving out to receive；When receiving as moving
After the server node of ingress returns to the notice for completing to move into, the migration units file statistical form of itself is updated；
Accordingly, the server node, specifically for as move out node when, according to main control server node send move
Go out the notice of file, extract file of moving out and be sent to node of moving into, return to move out to main control server node and successfully notify；With
And as move into node when, after completion is moved into, returned to main control server node and complete the notice moved into.
The method and distributed data base system of equally loaded provided by the present invention, can be by the main control server node cycle
Property the migration units file statistical form and migration strategy set up according to Servers-all node, selected node, the section of moving out of moving out
Point move out file and file of moving out it is corresponding move into node as migration decision-making；Section of moving out is adjusted further according to migration decision-making
The load of point and node of moving into.In this way, can be according to the data storage of each server node in distributed data base system
Situation, adjusts the load of each server node, and then realizes the load balancing of distributed data base.
Brief description of the drawings
Fig. 1 is the method flow schematic diagram of equally loaded of the invention；
Fig. 2 is that distributed data base system of the invention constitutes structural representation.
Specific embodiment
Basic thought of the invention is：Currently stored data are added to migration units text by each server node in real time
Part, and migration units file statistical form is set up according to migration units file；Main control server node periodically obtains each service
The migration units file statistical form of device node；Main control server node is according to migration units file statistical form and migration strategy, choosing
Surely move out node, node of moving out move out file and file of moving out it is corresponding move into node as migration decision-making；Master control is serviced
When device node determines migration, according to migration decision-making, the load of move out node and node of moving into is adjusted.
Below in conjunction with the accompanying drawings and specific embodiment the present invention is further described in more detail.
The method of equally loaded proposed by the present invention, as shown in figure 1, comprising the following steps：
Step 101：Currently stored data are added to migration units file by each server node in real time, and according to moving
Move unit file and set up migration units file statistical form.
Here, it is described currently stored data are added to migration units file to be：Whether server node real-time detection
The data that client is sent are received, if receiving, server node detects whether current migration units file exceedes maximum note
Record threshold value, if exceeding, stops updating current migration units file, and newly-built migration units file is used as current migration units text
Part, in storing data into current migration units file；If not above be directly stored in for data and currently move by server node
Move in unit file.Wherein, it, according to the preset numerical value of actual conditions, can be migration units file that the dominant record threshold value is
The maximum quantity of the data of middle record, or the maximum disk capacity shared by data preserved in migration units file.
Step 102：Main control server node periodically obtains the migration units file statistical form of each server node.
Here, the migration units file statistical form includes：The mark of server node, disk total capacity and disk have been used
Capacity, in server node preserve all migration units fileinfos, each migration units file take disk size and
The visiting frequency of each migration units file；
Wherein, the mark of the server node can be server node address in systems；
The migration units fileinfo includes：Migration units quantity of documents, filename, the filename can be by servicing
The order of setting up of the mark of device node and/or the migration units file is constituted；The periodicity is to be set according to actual conditions
Duration, such as, could be arranged to one day；The acquisition can be moved for each server node is periodically reported to main control server
Move unit file statistical form, or main control server periodically extracts migration from each server node of self-management
Unit file statistical information.
Step 103：Main control server node selectes section of moving out according to all migration units file statistical forms and migration strategy
Point, node of moving out move out file and file of moving out it is corresponding move into node as migration decision-making.
Here, the migration strategy is following one or more plan being preset at according to actual conditions in main control server
Slightly：
First, the migration units file statistical form that disk has exceeded threshold values with capacity, the migration units file that will be selected are selected
The corresponding server node of statistical form is used as node of moving out；From the migration units file statistical form selected, selected node of moving out
In migration units file as file of moving out；From the corresponding server node of remaining migration units file statistical form, select
The corresponding node of moving into of each file of moving out；
Wherein, described from the migration units file statistical form selected, the migration units file in selected node of moving out is made
To move out, file can set selected rule according to actual conditions, such as, can set selected rule is：Calculate disk appearance
Difference between amount and threshold value, by the disk size sum of migration units file occupancy, one or more consistent with difference are moved
Unit file is moved as file of moving out；The threshold values can be set according to actual conditions, such as could be arranged to disk total capacity
90%；
In the corresponding server node from remaining migration units file statistical form, each file of moving out is selected corresponding
Node of moving into can be：According to the disk size of having used of current Servers-all node, calculating averagely has used disk size, from remaining
In the corresponding server node of remaining migration units file statistical form, select with disk size less than averagely having used disk size
Used as node of moving into, calculate node of moving into has used disk size and the average difference used between disk size to server node
Value, selected from all files of moving out occupancy disk size sum and the difference identical one or more move out file with
The node correspondence of moving into.
2nd, disk is calculated with the most server node of capacity and disk between the minimum server node of capacity
Difference, when the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from correspondence
Migration units file statistical form in, select one or more migration units files as file of moving out, from remaining migration units
It is that each file of moving out selects node of moving into the corresponding server node of file statistical form；
Wherein, the preset value can be ratio value, and difference is corresponding with the most server node of capacity with disk
The ratio of disk total capacity.
3rd, according to migration units file statistical form, select disk and counted less than the migration units file of threshold values with capacity
The corresponding server node of table is selected as node of moving into from the corresponding server node of remaining migration units file statistical form
Move out node, and select one or more migration units files as file of moving out；
Wherein, node of moving out is selected in the corresponding server node from remaining migration units file statistical form, and is selected
Fixed one or more migration units files can be set as file of moving out according to actual conditions, such as, and Ke Yiwei：According to current
Servers-all node has used disk size, calculates and has averagely used disk size, from remaining server node, selects
With disk size disk has been used higher than average according to node of moving out with the server node of disk size as node of moving out
Capacity and the average difference used between disk size, select one or more files of moving out.
Step 104：When main control server node determines migration, according to migration decision-making, move out node and node of moving into are adjusted
Load.
Here, the determination migration can be ordered by manually performing migration, or main control server node is set to examine first
Whether survey has selected move out node and node of moving into, if, it is determined that migration, if it is not, then terminating handling process；
It is described according to migration decision-making, the move out load of node and node of moving into of adjustment includes：
Step a：The current all migration units file statistical forms for itself preserving of master control server locking, and notify to select
All move out node and nodes of moving into lock the migration units file statistical form of itself；Main control server is according to moving out for selecting
Move out file and the corresponding node of moving into of file of moving out of node, node of moving out, respectively to corresponding node of moving out send to
Node of moving into sends the notice of file of moving out.
Wherein, it is described to be locked as stopping any operation to migration units file statistical form.
Step b：Node move out according to the notice for sending file of moving out, extracts file of moving out and is sent to node of moving into, and more
The new migration units file statistical form of itself.
Wherein, described sender formula can be carried out using FTP (FTP, File Transfer Protocol)
Transmission；Extraction file of moving out is sent to node of moving into and file and can be compressed for extraction is moved out, by compression after text of moving out
Part is sent to node of moving into；The boil down to prior art, does not repeat here.
Further, the file of moving out that extracts is sent to after node of moving into, and main control server node can also be supervised in real time
Whether measured compressed file fails and whether network transmission is abnormal, if compressed file failure or network transmission exception, judgement is
It is no to recover, if can recover, whether the current number of times that repeats is judged more than maximum, if so, then terminating treatment
Flow, if it is not, then extracting file of moving out is sent to node of moving into, and will repeat number of times and plus one；If cannot recover,
Then migration operation failure, charges to system exception daily record, terminates handling process；If compressed file success or network transmission are normal,
Perform step c.
Step c：When main control server node receive any one move out node return move out successfully notify when, notify should
Move out the corresponding all nodes of moving into of file of moving out of node, the file of moving out that treatment is received.
Wherein, before the node of moving out returns to successfully notice of moving out, also include：The node of moving out receives data and moves
After removing the notice of beginning, the identifier that will perform the session instance of Data Migration notifies main control server node；The master control clothes
After business device receives the message comprising session identifier, periodically heartbeat message is sent to node of moving out；The node receipts of moving out
To after heartbeat message, whether detection specifies the session of session identifier in operation, if so, then continuing to respond heartbeat message to master control
Server node, if it exceeds preset time period can't detect heartbeat, then judges data migration operation failure, notifies main control node
Move out unsuccessfully.
The session identifier can be current migration operation serial number；The transmission of the heartbeat message is equal with response
The method used during to be managed to system equipment in the prior art, does not repeat here.
The main control server node receives any one node of moving out and returns to move out successfully notify to be included：Master control is serviced
Device node real-time detection whether receive move out node return move out successfully notify, when receive move out node return move out into
During the notice of work(, the file of moving out for notifying the corresponding all node processings of moving into of file of moving out of the node of moving out to receive；When connecing
During the notice of the failure of moving out for receiving node return of moving out, terminate the corresponding migration process of node of moving out.
The file of moving out that receives of processing is：Node of moving into preserves the file of moving out that node of moving out is sent, and updates itself
Migration units file statistical form, and returned to main control server node and complete the notice moved into.Wherein, it is described to preserve text of moving out
Part can include：Preserved after file decompression of moving out after the compression that will be received；The decompression is condensed to and compression method
Corresponding method, is prior art, is not repeated here.
Step d：After main control server node receives any one node of moving into returns to the notice for completing to move into, update certainly
The migration units file statistical form of body.
Wherein, the corresponding record of the file that is updated to move out is modified.
In addition, before performing above-mentioned steps 101, the method can also include：Main control server node real-time detection owns
The state of server node, if having the state of any one server node for preparation is exited, using the server node as
Move out node, using all migration units files in the node of moving out as file of moving out, united from remaining migration units file
It is that each file of moving out selects node of moving into the corresponding server node of meter table.
Wherein, it is each file choosing of moving out in the corresponding server node from remaining migration units file statistical form
The node that goes out to move into can be：Disk size is used according to current all remaining server nodes, has been calculated and has averagely been used disk
Capacity, selects with disk size less than average with the server node of disk size as node of moving into, according to section of moving into
Used disk size and the average difference used between disk size, one or more selected one by one put move out file correspondingly
Node of moving into.
As shown in Fig. 2 the invention provides a kind of distributed data base system, the system includes：Main control server node
21 and server node 22；Wherein,
Server node 22, for currently stored data to be added into migration units file in real time, according to migration units
File sets up migration units file statistical form, and for master control server node 21 provides migration units file statistical form, Yi Jizuo
It is node or the 21 pairs of adjustment of load of node of node of moving out reception main control server of moving into；
Main control server node 21, the migration units file statistical form for periodically obtaining each server node 22；
According to migration units file statistical form and migration strategy, move out file and the file of moving out of selected move out node, node of moving out
Corresponding node of moving into is used as migration decision-making；And when determining migration, according to migration decision-making, adjust move out node and node of moving into
Load.
Whether the server node 22, the data that client is sent are received specifically for real-time detection, if receiving,
Detect whether current migration units file exceedes dominant record threshold value, if exceeding, stop updating current migration units file, newly
Migration units file is built as current migration units file, in storing data into current migration units file；If not above,
Data are directly stored in current migration units file.
The server node 22, includes specifically for preserving migration units file statistical form：The mark of server node,
Disk total capacity and disk capacity, all migration units fileinfos preserved in server node, each migration units
Disk size and the visiting frequency of each migration units file that file takes.
The main control server node 21, specifically for preserving migration strategy；The migration strategy is according to actual conditions
Preset following one or more strategy：
First, the migration units file statistical form that disk has exceeded threshold values with capacity, the migration units file that will be selected are selected
The corresponding server node of statistical form is used as node of moving out；From the migration units file statistical form selected, selected node of moving out
In migration units file as file of moving out；From the corresponding server node of remaining migration units file statistical form, select
The corresponding node of moving into of each file of moving out；
Wherein, described from the migration units file statistical form selected, the migration units file in selected node of moving out is made
To move out, file can set selected rule according to actual conditions, such as, can set selected rule is：Calculate disk appearance
Difference between amount and threshold value, by the disk size sum of migration units file occupancy, one or more consistent with difference are moved
Unit file is moved as file of moving out；The threshold values can be set according to actual conditions, such as could be arranged to disk total capacity
90%；
In the corresponding server node from remaining migration units file statistical form, each file of moving out is selected corresponding
Node of moving into can be：According to the disk size of having used of current Servers-all node, calculating averagely has used disk size, from remaining
In the corresponding server node of remaining migration units file statistical form, select with disk size less than averagely having used disk size
Used as node of moving into, calculate node of moving into has used disk size and the average difference used between disk size to server node
Value, selected from all files of moving out occupancy disk size sum and the difference identical one or more move out file with
The node correspondence of moving into.
2nd, disk is calculated with the most server node of capacity and disk between the minimum server node of capacity
Difference, when the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from correspondence
Migration units file statistical form in, select one or more migration units files as file of moving out, from remaining migration units
It is that each file of moving out selects node of moving into the corresponding server node of file statistical form；
Wherein, the preset value can be ratio value, and difference is corresponding with the most server node of capacity with disk
The ratio of disk total capacity.
3rd, according to migration units file statistical form, select disk and counted less than the migration units file of threshold values with capacity
The corresponding server node of table is selected as node of moving into from the corresponding server node of remaining migration units file statistical form
Move out node, and select one or more migration units files as file of moving out；
Wherein, node of moving out is selected in the corresponding server node from remaining migration units file statistical form, and is selected
Fixed one or more migration units files can be set as file of moving out according to actual conditions, such as, and Ke Yiwei：According to current
Servers-all node has used disk size, calculates and has averagely used disk size, from remaining server node, selects
With disk size disk has been used higher than average according to node of moving out with the server node of disk size as node of moving out
Capacity and the average difference used between disk size, select one or more files of moving out.
The main control server node 21, specifically for being migrated determining by manually performing migration order, or is set
To first detect whether to have selected move out node and node of moving into, if, it is determined that migration, if it is not, then terminating handling process.
The main control server node 21, specifically for the current all migration units files statistics for itself preserving of locking
Table, and notify that all node and the nodes of moving into of moving out selected lock the migration units file statistical form of itself, main control server
According to move out file and the corresponding node of moving into of file of moving out of the node of moving out, node of moving out selected, respectively to corresponding
Node of moving out sends the notice that file of moving out is sent to node of moving into.
The server node 22, specifically for as move out node when, according to the notice of file of moving out is sent, extraction is moved
Go out file and be sent to node of moving into, and update the migration units file statistical form of itself.
The server node 22, file and compresses specifically for extracting to move out, by compression after file of moving out be sent to
Move into node.
The main control server node 21, text is compressed specifically for real-time monitoring as the server node 22 of node of moving out
Whether part fails and whether network transmission is abnormal, if compressed file failure or network transmission exception, judging whether can be with extensive
It is multiple, if can recover, whether the current number of times that repeats is judged more than maximum, if so, then terminate handling process, if not
It is then to extract file of moving out and be sent to node of moving into, and number of times will be repeated plus one；If cannot recover, migration behaviour
Work fails, and charges to system exception daily record, terminates handling process.
The server node 22, be additionally operable to as move out node when, after receiving the notice that Data Migration starts, will perform
The identifier of the session instance of Data Migration notifies main control module, and receives the heartbeat message that main control server node 21 is sent
Afterwards, whether the session of the corresponding session identifier of detection heartbeat message is running, if so, then continue to receive the response giving master control service
Device node 21, if it exceeds preset time period can't detect heartbeat, then judges data migration operation failure, notifies main control server section
Point 21 is moved out unsuccessfully；Accordingly, the main control server node 21, is additionally operable to receive that server node 22 sends comprising session
After the message of identifier, periodically heartbeat message is sent to the server node 22 as node of moving out.
Whether the main control server node 21, node of moving out is received specifically for real-time detection and returns to successful leading to of moving out
Know, when receive move out successfully notify when, notify the node of moving out file of moving out it is corresponding it is all move into node processings receive
To file of moving out, when receive move out node return failure of moving out notice when, terminate the corresponding migration of node of moving out
Treatment.
The server node 22, specifically for as move into node when, preservation is moved out the file of moving out that node sends, more
The new migration units file statistical form of itself, and the notice for completing to move into is returned to main control server node 21.
The server node 22, specifically for as move into node when, the file solution of moving out after the compression that will be received
Preserved after compression；The decompression is condensed to method corresponding with compression method, is prior art, does not repeat here.
The main control server node 21, is additionally operable to the state of real-time detection server node 22, if there is server node
22 state is exited for preparation, then using the server node 22 as node of moving out, by all migration lists in the node of moving out
Meta file is that each is moved out from the corresponding server node 22 of remaining migration units file statistical form as file of moving out
File selects node of moving into；Accordingly, the server node 22, is additionally operable to provide the shape of itself for master control server node 21
State is exited for preparation and is also up.
The main control server node 21, specifically for being held with disk according to current all remaining server nodes 22
Amount, calculates and has averagely used disk size, selects with disk size less than the server node 22 for averagely having used disk size
As node of moving into, disk size and the average difference used between disk size are used according to node of moving into, selected one by one
The corresponding node of moving into of one or more files of moving out.
Main control server node 21 and server node 22 in said system in addition to the function that present invention offer is provided,
The institute that should possess with the node device in distributed data base specified in prior art is functional, does not repeat here.
The above, only presently preferred embodiments of the present invention is not intended to limit the scope of the present invention.
Claims (8)
1. a kind of method of equally loaded, it is characterised in that the method includes：
Currently stored data are added to migration units file by server node in real time, and are moved according to the foundation of migration units file
Move unit file statistical form；
Main control server node periodically obtains the migration units file statistical form of each server node；
According to migration units file statistical form and migration strategy, selected node of moving out, node of moving out are moved main control server node
Go out file and the corresponding node of moving into of file of moving out is used as migration decision-making；
When main control server node determines migration, according to migration decision-making, the load of move out node and node of moving into is adjusted；
Wherein, the migration strategy, including：One or more is tactful below：
Select the migration units file statistical form that disk has exceeded threshold value with capacity, the migration units file statistical form pair that will be selected
The server node answered is used as node of moving out；From the migration units file statistical form selected, the migration in selected node of moving out
Unit file is used as file of moving out；From the corresponding server node of remaining migration units file statistical form, select each and move out
The corresponding node of moving into of file；
Disk is calculated with the most server node of capacity and disk with the difference between the minimum server node of capacity,
When the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from corresponding migration
In unit file statistical form, migration units file is selected as file of moving out, it is corresponding from remaining migration units file statistical form
It is that each file of moving out selects node of moving into server node；
According to migration units file statistical form, disk is selected corresponding less than the migration units file statistical form of threshold value with capacity
Server node selectes section of moving out as node of moving into from the corresponding server node of remaining migration units file statistical form
Point, and selected migration units file is used as file of moving out.
2. method according to claim 1, it is characterised in that the migration units file statistical form, including：Server section
Mark, disk total capacity and the disk capacity put, all migration units fileinfos preserved in server node, each
Disk size and the visiting frequency of each migration units file that migration units file takes.
3. method according to claim 1, it is characterised in that the main control server node periodically obtains each service
Before the migration units file statistical form of device node, the method also includes：Main control server node real-time detection Servers-all
The state of node, if having the state of server node for preparation is exited, the server node as node of moving out moves this
All migration units files in egress are equal as file of moving out, from the corresponding server of remaining migration units file statistical form
It is that each file of moving out selects node of moving into node.
4. method according to claim 1, it is characterised in that described according to migration decision-making, adjustment is moved out and node and is moved into
The load of node, including：Move out file and move out file of the main control server according to the node of moving out, node of moving out selected
Corresponding node of moving into, sends the notice that file of moving out is sent to node of moving into corresponding node of moving out respectively；
Node move out according to the notice for sending file of moving out, extracts file of moving out and is sent to node of moving into；
When main control server node receive move out node return move out successfully notify when, notify the file of moving out of the node of moving out
The file of moving out that corresponding all node processings of moving into are received；
After main control server node receives node of moving into returns to the notice for completing to move into, the migration units file system of itself is updated
Meter table.
5. a kind of distributed data base system, it is characterised in that the system includes：Main control server node and server node；
Wherein,
Server node, for currently stored data to be added into migration units file in real time, builds according to migration units file
Vertical migration units file statistical form, and for master control server node provides migration units file statistical form, and as section of moving into
Point or node of moving out receive adjustment of the main control server node to loading；
Main control server node, the migration units file statistical form for periodically obtaining each server node；According to migration
Unit file statistical form and migration strategy, selected node of moving out, the file and file of moving out is corresponding moves of moving out of node of moving out
Ingress is used as migration decision-making；And when determining migration, according to migration decision-making, adjust the load of move out node and node of moving into；
Wherein, the main control server node, specifically for preserving migration strategy, the migration strategy is included with next or many
Bar strategy：
Select the migration units file statistical form that disk has exceeded threshold value with capacity, the migration units file statistical form pair that will be selected
The server node answered is used as node of moving out；From the migration units file statistical form selected, the migration in selected node of moving out
Unit file is used as file of moving out；From the corresponding server node of remaining migration units file statistical form, select each and move out
The corresponding node of moving into of file；
Disk is calculated with the most server node of capacity and disk with the difference between the minimum server node of capacity,
When the difference is more than preset value, using disk with the most server node of capacity as node of moving out, from corresponding migration
In unit file statistical form, migration units file is selected as file of moving out, it is corresponding from remaining migration units file statistical form
It is that each file of moving out selects node of moving into server node；
According to migration units file statistical form, disk is selected corresponding less than the migration units file statistical form of threshold value with capacity
Server node selectes section of moving out as node of moving into from the corresponding server node of remaining migration units file statistical form
Point, and selected migration units file is used as file of moving out.
6. system according to claim 5, it is characterised in that the server node, specifically for utilizing server section
Mark, disk total capacity and the disk capacity put, all migration units fileinfos preserved in server node, each
The disk size and the visiting frequency of each migration units file that migration units file takes set up migration units file statistical form.
7. system according to claim 5, it is characterised in that
The main control server node, is additionally operable to the state of real-time detection Servers-all node, if there is the shape of server node
State is exited for preparation, then using the server node as node of moving out, all migration units files in the node of moving out are equal
It is that each file of moving out is selected and moved from the corresponding server node of remaining migration units file statistical form as file of moving out
Ingress；
Accordingly, the server node, is additionally operable to provide the state of itself for preparation is exited still for master control server node
Normal operation.
8. system according to claim 5, it is characterised in that
The main control server node, specifically for file and being moved out according to the node of moving out selected, moving out for node of moving out
The corresponding node of moving into of file, sends to be sent to the server node as node of moving into and moves out to corresponding node of moving out respectively
The notice of file；When receive as node of moving into server node return move out successfully notify when, notify the node of moving out
The file of moving out that receives of the corresponding all node processings of moving into of file of moving out；When receiving the server node as node of moving into
After returning to the notice for completing to move into, the migration units file statistical form of itself is updated；
Accordingly, the server node, specifically for as move out node when, sent according to main control server node and moved out text
The notice of part, extracts file of moving out and is sent to node of moving into, and returns to move out to main control server node and successfully notify；And make
For move into node when, after completion is moved into, returned to main control server node and complete the notice moved into.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210288434.7A CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210288434.7A CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103593347A CN103593347A (en) | 2014-02-19 |
| CN103593347B true CN103593347B (en) | 2017-06-13 |
Family
ID=50083496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210288434.7A Active CN103593347B (en) | 2012-08-14 | 2012-08-14 | The method and distributed data base system of a kind of equally loaded |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103593347B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105488079A (en) * | 2014-10-11 | 2016-04-13 | 苏州精易会信息技术有限公司 | Method for processing multi-tenant data |
| CN105049508A (en) * | 2015-07-21 | 2015-11-11 | 齐鲁工业大学 | Cloud data migration method |
| CN106502576B (en) * | 2015-09-06 | 2020-06-23 | 中兴通讯股份有限公司 | Migration strategy adjusting method and device |
| CN105227645A (en) * | 2015-09-15 | 2016-01-06 | 齐鲁工业大学 | A kind of cloud data migration method |
| CN105245381B (en) * | 2015-10-22 | 2019-08-16 | 上海斐讯数据通信技术有限公司 | Cloud Server delay machine monitors migratory system and method |
| CN106294538B (en) * | 2016-07-19 | 2019-07-16 | 浙江大华技术股份有限公司 | A kind of moving method and device from the data record in node |
| WO2018053838A1 (en) * | 2016-09-26 | 2018-03-29 | 华为技术有限公司 | Load balancing method and related device |
| CN111095233B (en) * | 2017-09-28 | 2023-09-26 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, live migration and applications thereof |
| CN108710925A (en) * | 2018-05-15 | 2018-10-26 | 南京博内特信息科技有限公司 | A method of the clothes commodity shelf system based on Internet of Things |
| EP3703342B1 (en) * | 2019-03-01 | 2023-07-26 | ABB Schweiz AG | Dynamic load balancing in network centric process control systems |
| CN110377430B (en) * | 2019-07-24 | 2021-08-31 | 中南民族大学 | Data migration method, device, storage medium and device |
| CN111143324B (en) * | 2019-12-20 | 2023-05-02 | 浪潮软件股份有限公司 | Kudu database data equalization system based on size and implementation method |
| CN113268203B (en) * | 2021-05-18 | 2022-11-04 | 天津中科曙光存储科技有限公司 | Capacity balancing method and device of storage system, computer equipment and storage medium |
| CN113821176B (en) * | 2021-09-29 | 2023-07-21 | 重庆紫光华山智安科技有限公司 | Data migration processing method, device and storage medium |
| CN116389442A (en) * | 2023-02-10 | 2023-07-04 | 上海弘积信息科技有限公司 | Web page request system and method based on load balancing |
| CN116991580A (en) * | 2023-07-27 | 2023-11-03 | 上海沄熹科技有限公司 | Distributed database system load balancing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582850A (en) * | 2009-06-19 | 2009-11-18 | 优万科技（北京）有限公司 | Method and system for realizing load balance |
| CN102457428A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Load balancing realization method and device for DHT (distributed Hash table) network |
-
2012
- 2012-08-14 CN CN201210288434.7A patent/CN103593347B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582850A (en) * | 2009-06-19 | 2009-11-18 | 优万科技（北京）有限公司 | Method and system for realizing load balance |
| CN102457428A (en) * | 2010-10-27 | 2012-05-16 | 中兴通讯股份有限公司 | Load balancing realization method and device for DHT (distributed Hash table) network |
Non-Patent Citations (1)
| Title |
|---|
| 分布式文件系统的动态负载均衡算法;张聪萍等;《小型微型计算机系统》;20110731;第32卷(第7期);第1424-1426页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103593347A (en) | 2014-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103593347B (en) | The method and distributed data base system of a kind of equally loaded | |
| CN107145393A (en) | A kind of load regulation method, equipment and computer-readable recording medium | |
| CN104852857B (en) | Distributed data transport method and system based on load balancing | |
| CN102577241B (en) | Method, device and system for scheduling distributed buffer resources | |
| CN109951537B (en) | Load balancing distribution method facing block chain | |
| CN109246229A (en) | A kind of method and apparatus of distribution resource acquisition request | |
| CN101741635B (en) | Method, system and device for synchronous altering message | |
| CN110381560B (en) | Wireless sensor network communication method suitable for power field | |
| CN111131082A (en) | Charging facility data transmission dynamic control method and system | |
| CN106953926A (en) | A kind of method for routing and device | |
| CN107846457A (en) | Load-balancing method, device, storage medium and the system of server cluster | |
| WO2019015856A1 (en) | Monitoring a blockchain | |
| CN108712469A (en) | A kind of sensing network connection distribution in Internet of Things and dispatching method | |
| CN104702647B (en) | Information requesting method and system | |
| CN106790409A (en) | Load-balancing method and its system based on the treatment of electric business platform user historical data | |
| CN103888310B (en) | Monitor processing method and system | |
| CN106603631A (en) | Distributed message system and expansion method thereof | |
| CN107547643A (en) | A kind of load sharing method and device | |
| CN112436962A (en) | Block chain consensus network dynamic expansion method, electronic device, system and medium | |
| EP2988476B1 (en) | Method and apparatus for processing operation on endpoint peripheral | |
| CN114116237B (en) | Hierarchical autonomous remote management method for large-scale network equipment | |
| CN106953884A (en) | Middleware message treatment method, device and middleware platform | |
| CN104978344B (en) | A kind of data operating method and device | |
| CN106550342A (en) | The overload controlling method and device of charging request message | |
| CN110233858A (en) | The methods of risk assessment and system of smart machine based on cloud prestige library |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |