CN104765748B - A kind of method and device that duplication table is converted to fragment table - Google Patents
A kind of method and device that duplication table is converted to fragment table Download PDFInfo
- Publication number
- CN104765748B CN104765748B CN201410006477.0A CN201410006477A CN104765748B CN 104765748 B CN104765748 B CN 104765748B CN 201410006477 A CN201410006477 A CN 201410006477A CN 104765748 B CN104765748 B CN 104765748B
- Authority
- CN
- China
- Prior art keywords
- fragment
- back end
- clustered node
- deletion
- duplication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of method and device that duplication table is converted to fragment table, the method includes：To the data to be loaded setting fragment rule, the clustered node replicates table according to the fragment rule creation, and the duplication table is sent to the back end that the clustered node managed, wherein described more than one fragment for replicating the number that table is included according to the fragment rule creation；When the duplication table is converted to fragment table, the clustered node is according to the fragment rule and preset fragment distribution rules, determine the corresponding deletion information of each back end, and by the back end for deleting information and being respectively correspondingly sent to the clustered node and being managed, wherein the deletion information includes the number of fragment；Fragment in each back end duplication table according to the deletion information deletion respectively received.
Description
Technical field
The present invention relates to big data treatment technology more particularly to a kind of methods and dress that duplication table is converted to fragment table
It puts.
Background technology
With the arrival in big data epoch, with the traditional Relational DataBase that minicomputer plus storage array are built due to
The limitation of ability extending transversely has increasingly been not suitable with demand of the mass data to computing capability.In recent years, distributed system
System becomes a kind of trend.Each database manufacturer and user are also in the actively framework of research distributed relational database and application.It is uncommon
The data-base cluster that High Availabitity and high extended capability are provided using cheap X86 servers stacking is hoped, so as to break away to large size
The dependence of equipment.
Most manufacturers and user select X86 servers and the mysql databases increased income to build Distributed Relational data
Library cluster.In distributed relational database, a server and mysql databases are a node, and all nodes are common
Form distributed type assemblies.Distributed system, and can be by service request the biggest advantage is to good ability extending transversely
Each node is distributed to complete.And in distributed relation database system, in order to promote the concurrently access energy of cluster as possible
Power, usually there are two types of common tables：Replicate table and fragment table.So-called duplication table is exactly in the identical number of each node
According to table.It is frequent to be usually applied to inquiry, updates less scene.Benefit is search efficiency height, can give full play to distributed type assemblies
Calculating and input and output（I/O, Input/Output）Ability, shortcoming are redundancy height, and memory space occupies big.Fragment table, just
It is according to certain rule（The modes such as hash, rang, list）Data fragmentation is stored in different nodes, each node is only protected
Deposit a portion data.Benefit is to save memory space, and be conducive to the concurrently access of multinode.Shortcoming is exactly possible
There are certain cross-node access.Also to avoid the data correlation inquiry of cross-node may as possible when fragment designs simultaneously.
Due to life cycle of information characteristic, over time, the high table of certain temperatures can gradually turn cold, in order to save
Memory space also just has the demand that duplication table is converted into fragment table.The method of common duplication table conversion fragment table is weight
Build method.Assuming that the table for needing to convert is T, reconstruction method includes the following steps：
Step A1 creates fragment table T_sharding by fragment rule.
Step A2, duplication table T are renamed as T_back.
Step A3 extracts T_back data, re-writes T_sharding tables.
Step A4, T_sharding are renamed as fragment rules of the table T. according to fragment table, re-create fragment table.
Step A5 abandons T_back tables.
Conversion of the duplication table to fragment table is finally completed by above-mentioned A1 to A5 steps, but reconstruction method is asked there are following
Topic：1）When needing to delete and reloading data, efficiency is low, the time is long and can increase grid and the expense of I/O；2）
In the transition period of table, data are unavailable, Operational Visit interrupts, and table is bigger, and the time of service disconnection is longer；This is in telecommunications
It is difficult to receive in grade operation system；3）In data conversion process, after there are abnormal conditions and is restarted in node, it is difficult to
Ensure the consistency and availability of data.
Invention content
In view of this, the embodiment of the present invention is solves the problems, such as that of the existing technology provide a kind of is converted to duplication table
The method and device of fragment table can promote transfer efficiency, reduce business interruption time.
What the technical solution of the embodiment of the present invention was realized in：
A kind of method that duplication table is converted to fragment table, the method are used for distributed relational database, the data
Library includes more than one clustered node, and the clustered node manages more than one back end；To the data to be loaded
Fragment rule is set, the method includes：
The clustered node replicates table according to the fragment rule creation, and the duplication table is sent to the cluster section
The managed back end of point, wherein described more than one point for replicating the number that table is included according to the fragment rule creation
Piece；
When the duplication table is converted to fragment table, the clustered node is according to the fragment rule and preset fragment
Distribution rules determine the corresponding deletion information of each back end, and the deletion information are respectively correspondingly sent to
The back end that the clustered node is managed, wherein the number deleted information and include fragment；
Fragment in each back end duplication table according to the deletion information deletion respectively received.
Preferably, the method further includes：
When the duplication table is converted to fragment table, the deletion information was written for the first local day by the clustered node
Will.
Preferably, the deletion information further includes the storage location with the corresponding fragment of number of the fragment；The side
Method further includes：
Each back end the storage location of the number of the respective fragment and its corresponding fragment is written respective
Corresponding second local daily record.
Preferably, the fragment in each back end duplication table according to the deletion information deletion respectively received
Before, the method further includes：
The current state of the fragment of required deletion is configured to delete shape according to the deletion information by each back end
State, and the corresponding second local daily record is written into the deletion state；And by the metadata attributes for replicating table from again
Tabulation is revised as fragment table, then sends modification metadata request message to the clustered node.
Preferably, the method further includes：
The clustered node receives the modification metadata request that the back end that the clustered node is managed is sent and disappears
After breath, by the described first local daily record of modification metadata request message write-in, and the metadata information of cluster is changed.
A kind of method that duplication table is converted to fragment table, this method are used for distributed relational database, the database
Including more than one clustered node, the clustered node manages more than one back end；The data to be loaded are set
Fragment rule is put, this method includes：
The clustered node replicates table according to the fragment rule creation, and the duplication table is sent to the cluster section
The managed back end of point, wherein described more than one point for replicating the number that table is included according to the fragment rule creation
Piece；
When the duplication table is converted to fragment table, the clustered node is according to the fragment rule and preset fragment
Distribution rules determine the corresponding deletion information of each back end, and the deletion information are respectively correspondingly sent to
The back end that the clustered node is managed, wherein the number deleted information and include fragment.
A kind of method that duplication table is converted to fragment table, the method are used for distributed relational database, the data
Library includes more than one clustered node, and the clustered node manages more than one back end；The method includes：
Point in each back end duplication table according to the deletion information deletion that the clustered node belonged to is sent
Piece.
A kind of device that duplication table is converted to fragment table, described device include setting unit, creating unit, determine list
Member, transmitting element and deletion unit, wherein：
The setting unit, for the data to be loaded setting fragment rule；
The creating unit for replicating table according to the fragment rule creation, and the duplication table is sent to described
The back end that clustered node is managed, wherein it is described replicate table include according to one of number of the fragment rule creation with
Upper fragment；
The determination unit, for when the duplication table is converted to fragment table, according to the fragment rule and presetting
Fragment distribution rules, determine the corresponding deletion information of each back end, the number deleted information and include fragment；
The transmitting element, for the deletion information to be respectively correspondingly sent to the number that the clustered node managed
According to node；
The deletion unit, for each back end according to the deletion information deletion respectively received duplication table
In fragment.
A kind of device that duplication table is converted to fragment table, described device include setting unit, creating unit, determination unit
And transmitting element；Wherein：
The setting unit, for the data to be loaded setting fragment rule；
The creating unit for replicating table according to the fragment rule creation, and the duplication table is sent to described
The back end that clustered node is managed, wherein it is described replicate table include according to one of number of the fragment rule creation with
Upper fragment；
The determination unit, for when the duplication table is converted to fragment table, according to the fragment rule and presetting
Fragment distribution rules, determine the corresponding deletion information of each back end, the number deleted information and include fragment；
The transmitting element, for the deletion information to be respectively correspondingly sent to the number that the clustered node managed
According to node.
A kind of device that duplication table is converted to fragment table, described device include deleting unit；
The deletion unit, in duplication table described in the deletion information deletion according to the clustered node transmission belonged to
Fragment.
In the embodiment of the present invention, to the data to be loaded setting fragment rule, the clustered node is according to the fragment
Rule creation replicates table, and the duplication table is sent to the back end that the clustered node managed, wherein the duplication
Table includes more than one fragment of the number according to the fragment rule creation；When the duplication table is converted to fragment table,
The clustered node determines that each back end is corresponding and deletes according to the fragment rule and preset fragment distribution rules
The back end that the clustered node managed respectively correspondingly is sent to except information, and by the deletion information, wherein described
Delete the number that information includes fragment；In each back end duplication table according to the deletion information deletion respectively received
Fragment；It so, it is possible to promote transfer efficiency, reduce business interruption time.
Description of the drawings
Fig. 1 is the realization flow diagram for the method that duplication table is converted to fragment table by the embodiment of the present invention one；
Fig. 2 is that back end of the embodiment of the present invention creates the structure diagram for replicating table；
Fig. 3-1 and Fig. 3-2 is the structure diagram that back end of the embodiment of the present invention deletes fragment from duplication table；
Fig. 4 is the realization flow diagram for the method that duplication table is converted to fragment table by the embodiment of the present invention two；
Fig. 5 is the composition structure diagram for the device that duplication table is converted to fragment table by the embodiment of the present invention three.It is specific real
Apply mode
In the various embodiments of the invention, when creating duplication table, clustered node replicates table simultaneously according to fragment rule creation
Its back end managed is sent to, which is made of more than one fragment；When table conversion fragment table will be replicated,
Each back end deletes the fragment table for not needing to storage thereon according to fragment rule and preset fragment distribution rules, you can completes
Replicate conversion of the table to fragment table；In this way, technical solution provided in an embodiment of the present invention can avoid dividing by the conversion of duplication table
The process reloaded of data during piece table, so as to promote transfer efficiency.
The technical solution of the present invention is further elaborated in the following with reference to the drawings and specific embodiments.
There are two types of nodes for distributed relational database usually presence：Clustered node and back end, wherein clustered node are used
In entire more than one back end of cluster of management etc., storage cluster metadata information；Wherein metadata information include table name,
Data distribution information etc. in table type, fragment rule, fragment distribution rules, the structural information of table, table, wherein table type include multiple
Tabulation or fragment table；Back end is used for storage table data.In clustered node and back end, it is no illustrate when, table one
As include replicating table or fragment table.
Embodiment one
Fig. 1 is the realization flow diagram for the method that duplication table is converted to fragment table by the embodiment of the present invention one, such as Fig. 1 institutes
Show, this method is used for distributed relational database, and the database includes more than one clustered node, the clustered node pipe
Manage more than one back end；This method includes：
Step 101, to the data to be loaded setting fragment rule.
Here, described that the data to be loaded setting fragment rule can be completed by clustered node, clustered node can
To obtain fragment rule from the rule base of user setting, the fragment that user sets the data to be loaded can also be directly acquired
Rule.
Step 102, the clustered node replicates table according to the fragment rule creation, and the duplication table is sent to institute
The back end that clustered node is managed is stated, wherein replicate the number that table is included according to the fragment rule creation one
More than fragment.
Fig. 2 is that back end of the embodiment of the present invention creates the structure diagram for replicating table, as shown in Fig. 2, the clustered node
20 three back end 21,22 and 23 of management, wherein each back end create duplication table 27, and the duplication table 27 is by 3
Fragment 24,25 and 26 is opened to form.
Step 103, when the duplication table is converted to fragment table, the clustered node is according to the fragment rule and in advance
If fragment distribution rules, determine the corresponding deletion information of each back end.
Step 104, the deletion information is respectively correspondingly sent to the clustered node and is managed by the clustered node
Back end, wherein the information of deleting includes the number of fragment.
Step 105, the fragment in each back end duplication table according to the deletion information deletion respectively received.
In the embodiment of the present invention, what the step 101 to 104 can be completed in clustered node, step 105 can be
It is completed in the back end that clustered node is managed.
Example shown in Fig. 2, Fig. 3-1 and Fig. 3-2 is accepted to delete from duplication table for back end of the embodiment of the present invention
The structure diagram of fragment, as shown in figure 3-1, in step 103, clustered node 20 is according to the fragment rule and preset point
Piece distribution rules determine the fragment 26 stored needed for back end 21, required deletion fragment 24 and 25, determine 22 institute of back end
Fragment 25, the required deletion fragment 24 and 26 that need to be stored, determine the fragment 24 stored needed for back end 23, required deletion fragment
25 and 26；Then by step 104 and step 105, back end 21 remains the fragment 26 in former duplication table 27, back end 22
The fragment 25 in former duplication table 27 is remained, back end 23 remains the fragment 24 in former duplication table 27, in this way, in data section
By deleting unwanted fragment in point 21,22 and 23, original duplication table fragment table has been rapidly transformed into just.
As shown in figure 3-2, in step 103, clustered node 20 is according to the fragment rule and preset fragment distribution rule
Then, the fragment 24 and 25 stored needed for back end 21, required deletion fragment 26 are determined, determines to delete institute needed for back end 22
Some fragments determine the fragment 26 stored needed for back end 23, required deletion fragment 25 and 24；Then by step 104 and step
Rapid 105, back end 21 remains the fragment 24 and 25 in former duplication table 27, and back end 23 is remained in former duplication table 27
Fragment 26, in this way, by deleting unwanted fragment in back end 21,22 and 23, just by original duplication table rapidly
Conversion is for fragment table.
Embodiment two
Fig. 4 is the realization flow diagram for the method that duplication table is converted to fragment table by the embodiment of the present invention two, such as Fig. 4 institutes
Show, this method is used for distributed relational database, and the database includes more than one clustered node, the clustered node pipe
Manage more than one back end；This method includes：
Step 401, to the data to be loaded setting fragment rule；
Step 402, the clustered node replicates table according to the fragment rule creation, and the duplication table is sent to institute
The back end that clustered node is managed is stated, wherein replicate the number that table is included according to the fragment rule creation one
More than fragment.
Step 403, when the duplication table is converted to fragment table, the clustered node is by the deletion information write-in the
One local daily record；And according to the fragment rule and preset fragment distribution rules, determine that each back end is corresponding
Information is deleted, then by the back end for deleting information and being respectively correspondingly sent to the clustered node and being managed.
Here, the number deleted information and include at least fragment.
Step 404, the deletion information respectively received was written for the corresponding second local day by each back end
Will；
Step 405, each back end according to the deletion information respectively received by the fragment table of required deletion work as
Preceding state is configured to deletion state, and the deletion state is written the corresponding second local daily record；
Here, its status set is first deletion state by the fragment that each back end deletes needs, is then performed
Delete operation, it is therefore an objective to it is most fast to restore externally to provide service, it can substantially realize no service disconnection；
Step 406, each back end replicates the fragment in table according to the deletion information deletion；And by described in
The metadata attributes of duplication table are revised as fragment table from duplication table, and then sending modification metadata request to the clustered node disappears
Breath；
Step 407, the clustered node receives the modification member number that the back end that the clustered node is managed is sent
After request message, by the described first local daily record of modification metadata request message write-in, and the metadata of cluster is changed
Information.
In embodiments of the present invention, before any operation is carried out, daily record is first write；This daily record of the embodiment of the present invention is first
Capable mechanism is the consistency and availability of data in order to ensure under abnormal conditions.When clustered node or back end are in different
When in the case of often, can accordingly it be restored according to the different conditions of respective daily record after restarting；Such as：
1）If step 403 is abnormal, the first local daily record of information write-in is not deleted, then explanation is converted to duplication table point
The operation of piece table carries out not yet, therefore, does not influence the inconsistent of data, does not need to do any measure.
2）If step 404 is abnormal, each back end is not by the second local day of the deletion information respectively received write-in
Will, then clustered node is needed to retransmit the instruction for deleting data to all back end, has been successfully written second
The back end of ground daily record does not need to repetitive operation；It is not successfully written the second local daily record of back end write-in of daily record.
3）If step 405 is abnormal, state of deleting is not successfully written the second local daily record, then back end is restarted
Afterwards, current state is written by the second local daily record according to the deletion state of configuration.
4）If clustered node is abnormal in step 407, i.e., not by the modification metadata request message write-in first
Ground daily record, according to the first local daily record of modification metadata request message write-in after restarting, and according to the first local daily record modification collection
The metadata information of group.
From the above it can be seen that either exception occurs in which step, may continue to after restarting according to set step
Suddenly it continues to execute down, ensures the consistency and availability of system.
Embodiment three
The embodiment of the present invention additionally provides a kind of device that duplication table is converted to fragment table, and Fig. 5 is the embodiment of the present invention
Three are converted to duplication table the composition structure diagram of the device of fragment table, as shown in figure 5, the device include setting unit 51,
Creating unit 52, determination unit 53, transmitting element 54 and deletion unit 55, wherein：
The setting unit 51, for the data to be loaded setting fragment rule；
The duplication table for replicating table according to the fragment rule creation, and is sent to institute by the creating unit 52
The back end that clustered node is managed is stated, wherein replicate the number that table is included according to the fragment rule creation one
More than fragment；
Fig. 2 is that back end of the embodiment of the present invention creates the structure diagram for replicating table, as shown in Fig. 2, the clustered node
20 three back end 21,22 and 23 of management, wherein each back end create duplication table 27, and the duplication table 27 is by 3
Fragment 24,25 and 26 is opened to form.
The determination unit 53, for when the duplication table is converted to fragment table, according to the fragment rule and pre-
If fragment distribution rules, determine the corresponding deletion information of each back end, the volume deleted information and include fragment
Number；
The transmitting element 54, for the deletion information to be respectively correspondingly sent to what the clustered node was managed
Back end；
The deletion unit 55, is replicated for each back end according to the deletion information deletion respectively received
Fragment in table.
Example shown in Fig. 2 is accepted, Fig. 3-1 and Fig. 3-2 are the knot that back end of the embodiment of the present invention deletes fragment table
Structure schematic diagram, as shown in figure 3-1, in step 103, clustered node 20 is according to the fragment rule and preset fragment distribution rule
Then, the fragment 26 stored needed for back end 21, required deletion fragment 24 and 25 are determined, determines what is stored needed for back end 22
Fragment 25, required deletion fragment 24 and 26 determine the fragment 24 stored needed for back end 23, required deletion fragment 25 and 26；
Then by step 104 and step 105, back end 21 remains the fragment 26 in former duplication table 27, and back end 22 remains
Original replicates the fragment 25 in table 27, and back end 23 remains the fragment 24 in former duplication table 27, in this way, back end 21,
By deleting unwanted fragment in 22 and 23, original duplication table fragment table has been rapidly transformed into just.
As shown in figure 3-2, in step 103, clustered node 20 is according to the fragment rule and preset fragment distribution rule
Then, the fragment 24 and 25 stored needed for back end 21, required deletion fragment 26 are determined, determines to delete institute needed for back end 22
Some fragments determine the fragment 26 stored needed for back end 23, required deletion fragment 25 and 24；Then by step 104 and step
Rapid 105, back end 21 remains the fragment 24 and 25 in former duplication table 27, and back end 23 is remained in former duplication table 27
Fragment 26, in this way, by deleting unwanted fragment in back end 21,22 and 23, just by original duplication table rapidly
Conversion is for fragment table.
In the embodiment of the present invention, setting unit 51, creating unit 52, determination unit 53 and transmitting element 54 can be set
In clustered node, the deletion unit 55 is arranged in the back end that clustered node is managed.
In the embodiment of the present invention, described device further includes the first writing unit, for being converted to point by the duplication table
During piece table, the first local daily record is written in the deletion information by the clustered node.
In the embodiment of the present invention, the deletion information further includes the storage position with the corresponding fragment of number of the fragment
It puts；Described device further includes the second writing unit, for each back end by the number of the respective fragment and its right
The the corresponding second local daily record of storage location write-in for the fragment answered.
In the embodiment of the present invention, described device further includes dispensing unit and the first modification unit, wherein：
The dispensing unit, for being configured to delete by the current state of the fragment of required deletion according to the deletion information
State, and the corresponding second local daily record is written into the deletion state；
The first modification unit, for the metadata attributes of the duplication table to be revised as fragment table from duplication table, so
Afterwards modification metadata request message is sent to the clustered node.
In the embodiment of the present invention, described device further includes the second modification unit；
First writing unit is additionally operable to receive the modification member that the back end that the clustered node is managed is sent
After data request information, by the described first local daily record of modification metadata request message write-in；
Accordingly, the second modification unit, for receiving all back end hairs that the clustered node is managed
After the modification metadata request message sent, the metadata information of cluster is changed.
It will be understood by those of skill in the art that duplication table is converted to each of the device of fragment table in the embodiment of the present invention
The realization function of unit can refer to the associated description of the aforementioned method that duplication table is converted to fragment table and understand.The present invention is implemented
The function that duplication table is converted to creating unit in the device of fragment table, determination unit and transmitting element by example can pass through collection
The processor of group node is realized, and the function of deleting unit can be by the processor for the back end that clustered node is managed
To realize；Certainly it can also be realized by specific logic circuit；For example, in practical applications, processor can be central processing
Device（Central Processing Unit, CPU）, microprocessor（Micro Processor Unit, MPU）, at digital signal
Manage device（Digital Signal Processor, DSP）Or field programmable gate array（Field Programmable Gate
Array, FPGA）Deng.
If the above-mentioned integrated unit of the present invention is realized in the form of software function module and is independent product sale
Or it in use, can also be stored in a computer read/write memory medium.Based on such understanding, the embodiment of the present invention
Technical solution substantially in other words can be embodied the part that the prior art contributes in the form of software product, the meter
Calculation machine software product is stored in a storage medium, is used including some instructions so that a computer equipment（Can be
People's computer, server or network equipment etc.）Perform all or part of each embodiment the method for the present invention.It is and preceding
The storage medium stated includes：Movable storage device, read-only memory（ROM, Read-Only Memory）, magnetic disc or CD etc.
The various media that can store program code.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (10)
- A kind of 1. method that duplication table is converted to fragment table, which is characterized in that the method is used for distributed relational database, The database includes more than one clustered node, and the clustered node manages more than one back end；To be added The data setting fragment rule of load, the method includes：The clustered node replicates table according to the fragment rule creation, and the duplication table is sent to the clustered node institute The back end of management, wherein described more than one fragment for replicating the number that table is included according to the fragment rule creation；When the duplication table is converted to fragment table, the clustered node is distributed according to the fragment rule and preset fragment Rule, determines the corresponding deletion information of each back end, and the deletion information is respectively correspondingly sent to described The back end that clustered node is managed, wherein the number deleted information and include fragment；Fragment in each back end duplication table according to the deletion information deletion respectively received.
- 2. according to the method described in claim 1, it is characterized in that, the method further includes：When the duplication table is converted to fragment table, the first local daily record is written in the deletion information by the clustered node.
- 3. according to the method described in claim 1, it is characterized in that, the deletion information further includes the number pair with the fragment The storage location for the fragment answered；The method further includes：Each back end is respectively corresponding by the storage location write-in of the number of the respective fragment and its corresponding fragment The second local daily record.
- 4. method according to any one of claims 1 to 3, which is characterized in that in each back end according to respective institute Before the fragment deleted in duplication table described in information deletion of reception, the method further includes：The current state of the fragment of required deletion is configured to deletion state by each back end according to the deletion information, and By the corresponding second local daily record of deletion state write-in；And the metadata attributes of the duplication table are repaiied from duplication table Fragment table is changed to, then sends modification metadata request message to the clustered node.
- 5. according to the method described in claim 4, it is characterized in that, the method further includes：After the clustered node receives the modification metadata request message that the back end that the clustered node is managed is sent, By the first local daily record of modification metadata request message write-in, and change the metadata information of cluster.
- A kind of 6. method that duplication table is converted to fragment table, which is characterized in that this method is used for distributed relational database, institute It states database and includes more than one clustered node, the clustered node manages more than one back end；To be loaded Data setting fragment rule, this method includes：The clustered node replicates table according to the fragment rule creation, and the duplication table is sent to the clustered node institute The back end of management, wherein described more than one fragment for replicating the number that table is included according to the fragment rule creation；When the duplication table is converted to fragment table, the clustered node is distributed according to the fragment rule and preset fragment Rule, determines the corresponding deletion information of each back end, and the deletion information is respectively correspondingly sent to described The back end that clustered node is managed, wherein the number deleted information and include fragment.
- 7. a kind of method that duplication table is converted to fragment table, the method is used for distributed relational database, the database Including more than one clustered node, the clustered node manages more than one back end；It is characterized in that, the method Including：Fragment in each back end duplication table according to the deletion information deletion that the clustered node belonged to is sent.
- 8. a kind of device that duplication table is converted to fragment table, described device is used for distributed relational database, the database Including more than one clustered node, the clustered node manages more than one back end；It is characterized in that, described device Including setting unit, creating unit, determination unit, transmitting element and unit is deleted, wherein：The setting unit, for the data to be loaded setting fragment rule；The duplication table for replicating table according to the fragment rule creation, and is sent to the cluster by the creating unit The back end that node is managed, wherein described more than one point for replicating the number that table is included according to the fragment rule creation Piece；The determination unit, for when the duplication table is converted to fragment table, according to the fragment rule and preset point Piece distribution rules determine the corresponding deletion information of each back end, the number deleted information and include fragment；The transmitting element, for the deletion information to be respectively correspondingly sent to the data section that the clustered node managed Point；The deletion unit, in each back end duplication table according to the deletion information deletion respectively received Fragment.
- 9. a kind of device that duplication table is converted to fragment table, described device is used for distributed relational database, the database Including more than one clustered node, the clustered node manages more than one back end；It is characterized in that, described device Including setting unit, creating unit, determination unit and transmitting element；Wherein：The setting unit, for the data to be loaded setting fragment rule；The duplication table for replicating table according to the fragment rule creation, and is sent to the cluster by the creating unit The back end that node is managed, wherein described more than one point for replicating the number that table is included according to the fragment rule creation Piece；The determination unit, for when the duplication table is converted to fragment table, according to the fragment rule and preset point Piece distribution rules determine the corresponding deletion information of each back end, the number deleted information and include fragment；The transmitting element, for the deletion information to be respectively correspondingly sent to the data section that the clustered node managed Point.
- 10. a kind of device that duplication table is converted to fragment table, which is characterized in that described device includes deleting unit；The deletion unit is arranged in the back end that clustered node is managed, for being sent out according to the clustered node belonged to That send deletes the fragment in duplication table described in information deletion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006477.0A CN104765748B (en) | 2014-01-07 | 2014-01-07 | A kind of method and device that duplication table is converted to fragment table |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006477.0A CN104765748B (en) | 2014-01-07 | 2014-01-07 | A kind of method and device that duplication table is converted to fragment table |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104765748A CN104765748A (en) | 2015-07-08 |
| CN104765748B true CN104765748B (en) | 2018-06-26 |
Family
ID=53647581
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410006477.0A Active CN104765748B (en) | 2014-01-07 | 2014-01-07 | A kind of method and device that duplication table is converted to fragment table |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104765748B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897281B (en) | 2015-12-17 | 2020-08-14 | 阿里巴巴集团控股有限公司 | Log fragmentation method and device |
| CN107391508B (en) * | 2016-05-16 | 2020-07-17 | 顺丰科技有限公司 | Data loading method and system |
-
2014
- 2014-01-07 CN CN201410006477.0A patent/CN104765748B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN104765748A (en) | 2015-07-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11520670B2 (en) | Method and apparatus for restoring data from snapshots | |
| CN111723160B (en) | Multi-source heterogeneous incremental data synchronization method and system | |
| CN103116661B (en) | A kind of data processing method of database | |
| Makris et al. | A classification of NoSQL data stores based on key design characteristics | |
| US8103628B2 (en) | Directed placement of data in a redundant data storage system | |
| CN108287835B (en) | Data cleaning method and device | |
| US9589041B2 (en) | Client and server integration for replicating data | |
| CN108073656A (en) | A kind of method of data synchronization and relevant device | |
| CN107209704A (en) | Detect the write-in lost | |
| CN104735110B (en) | Metadata management method and system | |
| CN106105161A (en) | To cloud data storage device Backup Data while maintaining storage efficiency | |
| CN107908503A (en) | Recover database from standby system streaming | |
| JP6086463B2 (en) | Method, device and system for peer-to-peer data replication and method, device and system for master node switching | |
| WO2012045245A1 (en) | Method and system for maintaining data consistency | |
| US10650024B2 (en) | System and method of replicating data in a distributed system | |
| Manoj | Comparative study of nosql document, column store databases and evaluation of cassandra | |
| CN105069151A (en) | HBase secondary index construction apparatus and method | |
| US11860892B2 (en) | Offline index builds for database tables | |
| US11615068B2 (en) | Methods and systems of managing deletes in a database node of a NoSQL database | |
| KR101527634B1 (en) | Method and apparatus for providing sharding service | |
| Tang et al. | Achieving convergent causal consistency and high availability for cloud storage | |
| CN104765748B (en) | A kind of method and device that duplication table is converted to fragment table | |
| CN109947730A (en) | Metadata restoration methods, device, distributed file system and readable storage medium storing program for executing | |
| Qi | Digital forensics and NoSQL databases | |
| CN104951475A (en) | Distributed file system and implementation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |