CN108038153A - The online data moving method and device of Hbase - Google Patents
The online data moving method and device of Hbase Download PDFInfo
- Publication number
- CN108038153A CN108038153A CN201711262754.4A CN201711262754A CN108038153A CN 108038153 A CN108038153 A CN 108038153A CN 201711262754 A CN201711262754 A CN 201711262754A CN 108038153 A CN108038153 A CN 108038153A
- Authority
- CN
- China
- Prior art keywords
- cluster
- data
- moment
- clusters
- snapshot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Abstract
This application provides the online data moving method and device of a kind of Hbase, this method includes：Since the first moment, by the data image of the first cluster to the second cluster；At the second moment after mirror image, open the bi-directional synchronization of the first cluster and the second cluster, and the data that the first cluster after the second moment and the second cluster update are synchronized between two clusters；By the data copy updated from the first moment to the first cluster in the period at the second moment to the second cluster.In the case where not stopping to the write access of former cluster, the online data migration of non-interrupting service is realized.
Description
Technical field
This application involves internet data service technology, and in particular to the online data moving method and dress of a kind of Hbase
Put.
Background technology
Hadoop databases (Hadoop Database, HBase) be a high reliability, high-performance, towards row, can stretch
The distributed memory system of contracting, can be erected greatly using HBase technologies on cheap PC server (PC Server)
Structure of size storage cluster.
When the data of HBase are migrated offline, data are copied to target cluster from former cluster；And data are carried out
When line migrates, it usually needs stop the write access to former cluster, front-end business can be interrupted.
The content of the invention
In view of this, the embodiment of the present application provides the online data moving method and device of a kind of Hbase, to solve
The online data migration of Hbase can interrupt the technical problem of front-end business.
According to the one side of the embodiment of the present application, there is provided the online data moving method of Hbase a kind of, the method
Including：Since the first moment, by the data image of the first cluster to the second cluster；At the second moment after mirror image, open
The bi-directional synchronization of the first cluster and the second cluster is opened, by the first cluster after the second moment and the data of the second cluster renewal two
Synchronized between a cluster；By from the first moment to the data copy that the first cluster in the period at the second moment updates to the
Two clusters.
In one embodiment, since the first moment, the data image of the first cluster is included to the second cluster：Will be with
First cluster needs the identical table Structure Creating of the table being migrated to the second cluster；Controlling the first cluster to obtain needs what is be migrated
The snapshot of table and by the snap copy to the second cluster；The second cluster is controlled to be resetted under the table structure according to the snapshot
Corresponding data.
In one embodiment, the second moment after mirror image, opens the two-way same of the first cluster and the second cluster
Step, by the data that the first cluster after the second moment and the second cluster update synchronized between two clusters including：Establish
Synchronous replication relation between first cluster and the second cluster；The number updated for the first cluster after the second moment and the second cluster
Attribute is replicated according to the table addition at place；The table for being added duplication attribute is subjected to data synchronization updating between two clusters.
In one embodiment, the data copy that will be updated from the first moment to the first cluster in the period at the second moment
Include to the second cluster：The synchronous replication relation between the first cluster and the second cluster is kept, when will be from the first moment to second
The data copy that the first cluster updates in the period at quarter is to the corresponding table of the second cluster.
In one embodiment, the online data moving method of Hbase further includes：Data will be migrated for the first cluster
Read access and write access be switched to the second cluster.
According to the other side of the embodiment of the present application, there is provided the online data moving apparatus of Hbase a kind of, the dress
Put including：Data image module, for since the first moment, by the data image of the first cluster to the second cluster；It is two-way same
Module is walked, for the second moment after mirror image, the bi-directional synchronization of the first cluster of unlatching and the second cluster, by the second moment
The first cluster and the data of the second cluster renewal synchronize between two clusters afterwards；Module is copied, for will be from first
The data copy that the first cluster updates in moment to the period at the second moment is to the second cluster.
In one embodiment, data image module includes：Table Structure Creating unit, for will be needed with the first cluster by
The identical table Structure Creating of the table of migration is to the second cluster；Snapshot processing unit, needs to be moved for controlling the first cluster to obtain
The snapshot of the table of shifting and by the snap copy to the second cluster；Data reset unit, for controlling the second cluster in the table
Corresponding data are resetted according to the snapshot under structure.
In one embodiment, bi-directional synchronization module includes：Synchronized relation establishes unit, for establishing the first cluster and
Synchronous replication relation between two clusters；Attribute adding device, for for the first cluster after the second moment and the second cluster more
Table addition where new data replicates attribute；Synchronized update unit, for the table for replicating attribute will to be added in two clusters
Between carry out data synchronization updating.
In one embodiment, copy module is further used for：Keep between the first cluster and the second cluster synchronously answer
Relation processed, will be corresponding to the second cluster from the first moment to the data copy of the first cluster renewal in the period at the second moment
Table.
In one embodiment, the online data moving apparatus of the Hbase further includes：Transferring module is read and write, for inciting somebody to action
The read access of data is migrated for the first cluster and write access is switched to the second cluster.
According to the other side of the embodiment of the present application, there is provided a kind of electronic equipment, including memory and processor；Deposit
Reservoir is used to store one or more computer instruction, wherein, one or more computer is performed by the processor to realize
The online data moving method of above-mentioned Hbase.
According to the other side of the embodiment of the present application, there is provided a kind of computer-readable recording medium, is stored thereon with
The online data moving method of above-mentioned Hbase is realized in computer instruction, the instruction when being executed by processor.
The beneficial effect of the embodiment of the present application includes：In the case where not stopping to the write access of former cluster, realize not
The online data migration of interrupting service, it is ensured that need the tables of data that migrates completely the same in two clusters, to original during migration
The write access of group service only has the influence of second rank.
Brief description of the drawings
By the description to the embodiment of the present application referring to the drawings, the above-mentioned and other purpose of the application, feature and
Advantage will be apparent from, in the accompanying drawings：
Fig. 1 is the schematic diagram of the online data migration of the embodiment of the present application Hbase；
Fig. 2 is the flow chart of the online data moving method of Hbase provided by the embodiments of the present application；
Fig. 3 is the block diagram of the online data moving apparatus of Hbase provided by the embodiments of the present application；
Fig. 4 is the structure diagram of electronic equipment provided by the embodiments of the present application.
Embodiment
The application is described below based on embodiment, but the application is not restricted to these embodiments.Under
Text is detailed to describe some specific detail sections in the detailed description of the application.Do not have for a person skilled in the art
The description of these detail sections can also understand the application completely.In order to avoid obscuring the essence of the application, known method, mistake
The not narration in detail of journey, flow, element and circuit.
In addition, it should be understood by one skilled in the art that provided herein attached drawing be provided to explanation purpose, and
What attached drawing was not necessarily drawn to scale.
Unless the context clearly requires otherwise, otherwise entire disclosure is similar with the " comprising " in claims, "comprising" etc.
Word should be construed to the implication included rather than exclusive or exhaustive implication；That is, it is containing for " including but not limited to "
Justice.
In the description of the present application, it is to be understood that term " first ", " second " etc. are only used for description purpose, without
It is understood that to indicate or implying relative importance.In addition, in the description of the present application, unless otherwise indicated, the implication of " multiple "
It is two or more.
The embodiment of the present application does not stop the write access to former cluster, and the tables of data that former cluster needs to migrate first is mirrored to mesh
Cluster is marked, the bi-directional synchronization between former cluster and target cluster is opened after mirror image and keeps the real-time of two company-datas renewals
Synchronously, the data filling of former cluster renewal copies target cluster to before starting after then mirror image is started to bi-directional synchronization, it is ensured that needs
The tables of data to be migrated is completely the same in two clusters, realizes the online data migration of non-interrupting service.
Fig. 1 is the online data migration schematic diagram between the embodiment of the present application Central Plains cluster 10 and target cluster 11, altogether bag
Three key steps are included, former cluster 10 and target cluster 11 are Hbase clusters.
1st step, target cluster, mirror image mistake are mirrored in the first moment t1 by the tables of data for needing to be migrated in former cluster 10
Journey can be completed in a manner of tables of data snapshot.The data that former cluster 10 needs to be migrated can be the full dose data of former cluster 10,
It can also be partial data.
Table structure is mirrored to target cluster 11 first.Created in target cluster 11 has phase with the tables of data being migrated
With the tables of data of table structure, and the state for the tables of data that target cluster 11 is created is set to unavailable (disable) state.
The former cluster 10 of control obtains the snapshot for the table for needing to be migrated and by snap copy to target cluster 11.With transport number
Exemplified by table " TestTable ", former cluster 10 is controlled to perform：
snapshot'TestTable','TestTable_snapshot'；It is current to obtain tables of data " TestTable "
Snapshot.
Then the snap copy of table that is migrated will be needed to target cluster 11.Former cluster uses parallel computational model
(MapReduce) mode is performed to issue orders：
Hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot-snapshot
TestTable_snapshot-copy-to hdfs://IP:PORT/xxxx；" TestTable_snapshot " represents snapshot name
Claim, " hdfs://IP:PORT/xxxx " represents catalogue of the target cluster 11 in Hadoop distributed file systems (HDFS).
Control targe cluster 11 resets corresponding data under the table structure of establishment according to snapshot.By snapshot
The data reset of " TestTable_snapshot " is into the table structure created and the tables of data that creates target cluster 11
The state of " TestTable " is set to available (enable) state.
2nd step, the second moment t2 after above-mentioned image process, opens the two-way of former cluster 10 and target cluster 11
It is synchronous, the data that former cluster 10 and target cluster 11 update after the second moment t2 are synchronized between two clusters.The
At the time of two moment t2 can be at the end of mirror image or after any time.
Initially set up synchronous duplication (Replication) relation between former cluster 10 and target cluster 11.The former collection of control
Group 10 is performed to issue orders：
Add_peer'1', " catalogue of the target cluster 11 in zookeeper ".
In former cluster 10, add for former cluster 10 after the second moment t2 and the table where the data of the renewal of target cluster 11
Add " replicating (Replication) " attribute.The data of former cluster 10 and target cluster 11 renewal refer to, in Yuan Ji after mirror image
The data updated in the data that group 10 needs to migrate, and after mirror image in the data that target cluster 11 replicates generation
The data updated.Tables of data to need to migrate is performed to issue orders when adding " duplication " attribute：
Alter'TestTable', { NAME=>' info', REPLICATION_SCOPE=>'1'}；
This step can change the attribute of all region of the tables of data, and the of short duration inaccessible of the tables of data, whole during change
A operation is completed in rank time second.Then, the table for being added duplication attribute is subjected to data synchronously more between two clusters
Newly.
Data and needs of the data for needing to migrate before the first moment t1 are migrated by above-mentioned two step
The data that data produce change after the second moment t2 are migrated between two clusters, since former cluster 11 does not stop
The only write access of front-end business, so between the first moment t1 and the second moment t2 in this period, needs in former cluster 10
The data of migration are there may be renewal, therefore following 3rd step needs to produce the Data Migration of renewal in this period to mesh
Mark cluster 11.
3rd step, by from the first moment t1 to the data copy that former cluster 10 updates in the period of the second moment t2 to mesh
Mark cluster 11.
After the second moment t2, continue to keep the synchronous replication relation between former cluster 10 and target cluster 11, will be from
The data copy that former cluster 10 updates in the period of first moment t1 to the second moment t2 is to 11 corresponding table of target cluster.This
When, using Copytable instruments to the data that former cluster 10 needs to migrate between the supplement of target cluster 11 t1 and t2 due to writing behaviour
Make and produce the data of renewal, it is as follows to perform order：
Hbase org.apache.hadoop.hbase.mapreduce.CopyTable--starttime=are (when first
Carve time points of the t1 either before t1) -- endtime=xxxxxxx (time point after the second moment t2 or t2) --
Peer.adr=IP:PORT/xxxx (catalogue of the target cluster 11 in zookeeper) TestTable (table name).
It will be produced more due to write access in former cluster 10 needs to migrate in this period of t1 to t2 data by the 3rd step
New data filling copies to target cluster 11, it is ensured that needs the data that migrate complete in the table of former cluster 10 and target cluster 11
It is complete consistent.
It is above-mentioned after line transition process is all finished, can be decided whether for transport number according to practical business demand
According to read access and write access be also fully transferred to target cluster 11；Object set is transferred to if necessary to read access and write access
Group 11, then continue to keep the synchronous replication relation of former cluster 10 and target cluster 11, keep the bi-directional synchronization between two clusters.
Based on described above, the embodiment of the present application provides the online data moving method of Hbase a kind of, is suitable for
The control node or terminal of Hbase clusters, as shown in Fig. 2, this method comprises the following steps.
S20, since the first moment, by the data image of the first cluster to the second cluster；
S21, at the second moment after mirror image, open the bi-directional synchronization of the first cluster and the second cluster, during by second
The first cluster and the data of the second cluster renewal synchronize between two clusters after quarter；
S22, by the data copy updated from the first moment to the first cluster in the period at the second moment to the second cluster.
The embodiment of the present application realizes the online data migration of non-interrupting service, it is ensured that the tables of data for needing to migrate is at two
Completely the same in cluster, business need not stop write access during migration, and only second rank influences.
Preferably, S20 is configured to：
S201, will need the identical table Structure Creating of the table that is migrated to the second cluster with the first cluster；
S202, the first cluster of control obtain the snapshot of table that needs to be migrated and by snap copies to the second cluster；
S203, the second cluster of control reset corresponding data under the table structure according to snapshot.
Preferably, S21 is configured to：
S211, establishes the synchronous replication relation between the first cluster and the second cluster；
S212, the table addition for the first cluster after the second moment and where the data of the second cluster renewal replicate attribute；
S213, data synchronization updating is carried out by the table for being added duplication attribute between two clusters.
Preferably, S22 is configured to：
The synchronous replication relation between the first cluster and the second cluster is kept, by from the first moment to the time at the second moment
The data copy of the first cluster renewal is to the corresponding table of the second cluster in section.
In one embodiment, the online data moving method of Hbase further includes：
S23, the read access of data will be migrated for the first cluster and write access is switched to the second cluster.
The online data moving method of corresponding above-mentioned Hbase, the embodiment of the present application provide a kind of online data of Hbase
Moving apparatus, as shown in figure 3, the device includes：
Data image module 30, for since the first moment, by the data image of the first cluster to the second cluster；
Bi-directional synchronization module 31, for the second moment after mirror image, pair of the first cluster of unlatching and the second cluster
To synchronization, the data that the first cluster after the second moment and the second cluster update are synchronized between two clusters；
Module 32 is copied, for the data copy that will be updated from the first moment to the first cluster in the period at the second moment
To the second cluster.
Preferably, data image module 30 further comprises：
Table Structure Creating unit, for the identical table Structure Creating of the table that is migrated will to be needed with the first cluster to the second collection
Group；
Snapshot processing unit, for the snapshot of table that controls the first cluster to obtain to need to be migrated and by the snap copy
To the second cluster；
Data reset unit, for controlling the second cluster to reset corresponding data according to snapshot under table structure.
Preferably, bi-directional synchronization module 31 further comprises：
Synchronized relation establishes unit, for establishing the synchronous replication relation between the first cluster and the second cluster；
Attribute adding device, for adding for the first cluster after the second moment and the table where the data of the second cluster renewal
Add duplication attribute；
Synchronized update unit, data synchronization updating is carried out for will be added the table for replicating attribute between two clusters.
Preferably, copy module 32 is further used for：The synchronous replication relation between the first cluster and the second cluster is kept,
By the data copy updated from the first moment to the first cluster in the period at the second moment to the corresponding table of the second cluster.
In one embodiment, the online data moving apparatus of Hbase further includes：
Transferring module is read and write, for the read access of data will to be migrated for the first cluster and write access is switched to the second collection
Group.
In addition, the above-mentioned online data moving apparatus based on Hbase can also pass through hardware processor (hardware
Processor) above-mentioned each program step is realized.Below with reference to Fig. 4, it illustrates suitable for for realizing the embodiment of the present application
Electronic equipment computer system 600 structure diagram.Electronic equipment shown in Fig. 4 is only an example, should not be right
The function and use scope of the embodiment of the present application bring any restrictions.
As shown in figure 4, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into program in random access storage device (RAM) 603 from storage part 608 and
Perform various appropriate actions and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interfaces 605 are connected to lower component：Importation 606 including keyboard, mouse etc.；Penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage part 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net performs communication process.Driver 610 is also according to needing to be connected to I/O interfaces 606.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc., are installed on driver 610, in order to read from it as needed
Computer program be mounted into as needed storage part 608.
Especially, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product, it includes being carried on computer-readable medium
On computer program, the computer program include be used for execution flow chart shown in method program code.In such reality
Apply in example, which can be downloaded and installed by communications portion 609 from network, and/or from detachable media
611 are mounted.When the computer program is performed by central processing unit (CPU) 601, perform what is limited in the present processes
Above-mentioned function.It should be noted that computer-readable medium described herein can be computer-readable signal media or
Computer-readable recording medium either the two any combination.Computer-readable recording medium for example can be --- but
Be not limited to --- electricity, magnetic, optical, electromagnetic, system, device or the device of infrared ray or semiconductor, or it is any more than combination.
The more specifically example of computer-readable recording medium can include but is not limited to：Electrical connection with one or more conducting wires,
Portable computer diskette, hard disk, random access storage device (RAM), read-only storage (ROM), erasable type may be programmed read-only deposit
Reservoir (EPROM or flash memory), optical fiber, portable compact disc read-only storage (CD-ROM), light storage device, magnetic memory
Part or above-mentioned any appropriate combination.In this application, computer-readable recording medium can any be included or store
The tangible medium of program, the program can be commanded the either device use or in connection of execution system, device.And
In the application, computer-readable signal media can include believing in a base band or as the data that a carrier wave part is propagated
Number, wherein carrying computer-readable program code.The data-signal of this propagation can take various forms, including but not
It is limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer
Any computer-readable medium beyond readable storage medium storing program for executing, the computer-readable medium can send, propagate or transmit use
In by instruction execution system, device either device use or program in connection.Included on computer-readable medium
Program code any appropriate medium can be used to transmit, include but not limited to：Wirelessly, electric wire, optical cable, RF etc., Huo Zheshang
Any appropriate combination stated.
Flow chart and block diagram in attached drawing, it is illustrated that according to the system of the various embodiments of the application, method and computer journey
Architectural framework in the cards, function and the operation of sequence product.At this point, each square frame in flow chart or block diagram can generation
The part of one module of table, program segment or code, a part for the module, program segment or code include one or more
The executable instruction of logic function as defined in being used for realization.It should also be noted that some as replace realization in, institute in square frame
The function of mark can also be with different from the order marked in attached drawing generation.For example, two square frames succeedingly represented are actual
On can perform substantially in parallel, they can also be performed in the opposite order sometimes, this is depending on involved function.Also
It is noted that the combination of each square frame and block diagram in block diagram and/or flow chart and/or the square frame in flow chart, Ke Yiyong
The dedicated hardware based systems of functions or operations as defined in execution is realized, or can be referred to specialized hardware and computer
The combination of order is realized.
Being described in module involved in the embodiment of the present application can be realized by way of software, can also be by hard
The mode of part is realized.Described module can also be set within a processor, for example, can be described as：A kind of processor bag
Include data image module, bi-directional synchronization module and copy module.
As on the other hand, present invention also provides a kind of nonvolatile computer storage media, the non-volatile calculating
Machine storage medium can be nonvolatile computer storage media included in device described in above-described embodiment；Can also be
Individualism, without the nonvolatile computer storage media in supplying electronic equipment.Above-mentioned non-volatile computer storage is situated between
Matter is stored with one or more program, when one or more program stored is performed by an equipment so that described
Equipment：Since the first moment, by the data image of the first cluster to the second cluster；At the second moment after mirror image, open
The bi-directional synchronization of the first cluster and the second cluster is opened, by the first cluster after the second moment and the data of the second cluster renewal two
Synchronized between a cluster；By from the first moment to the data copy that the first cluster in the period at the second moment updates to the
Two clusters.
The foregoing is merely the preferred embodiment of the application, the application is not limited to, for those skilled in the art
For, the application can have various modifications and changes.All any modifications made within spirit herein and principle, be equal
Replace, improve etc., it should be included within the protection domain of the application.
Claims (10)
- A kind of 1. online data moving method of Hbase, it is characterised in that the described method includes：Since the first moment, by the data image of the first cluster to the second cluster；At the second moment after mirror image, open the bi-directional synchronization of the first cluster and the second cluster, by after the second moment the One cluster and the data of the second cluster renewal synchronize between two clusters；By the data copy updated from the first moment to the first cluster in the period at the second moment to the second cluster.
- 2. according to the method described in claim 1, it is characterized in that, since the first moment, by the data image of the first cluster Include to the second cluster：The identical table Structure Creating of the table that is migrated will be needed to the second cluster with the first cluster；The first cluster is controlled to obtain the snapshot of table that needs to be migrated and by the snap copy to the second cluster；The second cluster is controlled to reset corresponding data according to the snapshot under the table structure.
- 3. according to the method described in claim 1, it is characterized in that, the second moment after mirror image, opens the first cluster With the bi-directional synchronization of the second cluster, the data that the first cluster after the second moment and the second cluster are updated are between two clusters Synchronize including：Establish the synchronous replication relation between the first cluster and the second cluster；Table addition where the data updated for the first cluster after the second moment and the second cluster replicates attribute；The table for being added duplication attribute is subjected to data synchronization updating between two clusters.
- 4. according to the method described in claim 1, it is characterized in that, by from the first moment in the period at the second moment first The data copy of cluster renewal includes to the second cluster：The synchronous replication relation between the first cluster and the second cluster is kept, within from the first moment to the period at the second moment The data copy of first cluster renewal is to the corresponding table of the second cluster.
- 5. according to the method described in claim 1, it is characterized in that, the method further includes：The read access of data will be migrated for the first cluster and write access is switched to the second cluster.
- 6. the online data moving apparatus of a kind of Hbase, it is characterised in that described device includes：Data image module, for since the first moment, by the data image of the first cluster to the second cluster；Bi-directional synchronization module, for the second moment after mirror image, the bi-directional synchronization of the first cluster of unlatching and the second cluster, The data that first cluster after second moment and the second cluster update are synchronized between two clusters；Module is copied, for the data copy that will be updated from the first moment to the first cluster in the period at the second moment to second Cluster.
- 7. device according to claim 6, it is characterised in that data image module includes：Table Structure Creating unit, for the identical table Structure Creating of the table that is migrated will to be needed with the first cluster to the second cluster；Snapshot processing unit, for the snapshot of table that controls the first cluster to obtain to need to be migrated and by the snap copy to the Two clusters；Data reset unit, for controlling the second cluster to reset corresponding data according to the snapshot under the table structure.
- 8. device according to claim 6, it is characterised in that bi-directional synchronization module includes：Synchronized relation establishes unit, for establishing the synchronous replication relation between the first cluster and the second cluster；Attribute adding device, it is multiple for the table addition for the first cluster after the second moment and where the data of the second cluster renewal Attribute processed；Synchronized update unit, data synchronization updating is carried out for will be added the table for replicating attribute between two clusters.
- 9. a kind of electronic equipment, it is characterised in that including memory and processor；The memory is used to store one or more Computer instruction, wherein, one or more computer is performed by the processor to realize that claim 1 to 5 is any one The step of item the method.
- 10. a kind of computer-readable recording medium, is stored thereon with computer instruction, it is characterised in that the instruction is by processor The step of claim 1 to 5 any one the method is realized during execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711262754.4A CN108038153A (en) | 2017-12-04 | 2017-12-04 | The online data moving method and device of Hbase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711262754.4A CN108038153A (en) | 2017-12-04 | 2017-12-04 | The online data moving method and device of Hbase |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108038153A true CN108038153A (en) | 2018-05-15 |
Family
ID=62095196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711262754.4A Pending CN108038153A (en) | 2017-12-04 | 2017-12-04 | The online data moving method and device of Hbase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038153A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376010A (en) * | 2018-09-28 | 2019-02-22 | 上海思询信息科技有限公司 | A method of across cluster resource migration is realized based on Openstack |
| CN110120979A (en) * | 2019-05-20 | 2019-08-13 | 华为技术有限公司 | A kind of dispatching method, device and relevant device |
| CN111241060A (en) * | 2020-01-08 | 2020-06-05 | 苏州科达科技股份有限公司 | Data migration method, system, device and storage medium |
| CN111367892A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Data migration method and device |
| CN113010515A (en) * | 2021-03-03 | 2021-06-22 | 中国工商银行股份有限公司 | Database updating method, device, equipment and medium |
| CN114780293A (en) * | 2022-04-26 | 2022-07-22 | 北京科杰科技有限公司 | Remote double-activity disaster recovery method, device and equipment based on hadoop and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004077216A2 (en) * | 2003-01-30 | 2004-09-10 | Vaman Technologies (R & D) Limited | System and method for heterogeneous data migration in real-time |
| CN105549904A (en) * | 2015-12-08 | 2016-05-04 | 华为技术有限公司 | Data migration method applied in storage system and storage devices |
| CN106326041A (en) * | 2016-08-31 | 2017-01-11 | 杭州沃趣科技股份有限公司 | Second-level recovery method for database |
| CN107015972A (en) * | 2016-01-27 | 2017-08-04 | 五八同城信息技术有限公司 | A kind of computer room business migration methods, devices and systems |
| CN107391758A (en) * | 2017-08-24 | 2017-11-24 | 阿里巴巴集团控股有限公司 | Database switching method, device and equipment |
-
2017
- 2017-12-04 CN CN201711262754.4A patent/CN108038153A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004077216A2 (en) * | 2003-01-30 | 2004-09-10 | Vaman Technologies (R & D) Limited | System and method for heterogeneous data migration in real-time |
| CN105549904A (en) * | 2015-12-08 | 2016-05-04 | 华为技术有限公司 | Data migration method applied in storage system and storage devices |
| CN107015972A (en) * | 2016-01-27 | 2017-08-04 | 五八同城信息技术有限公司 | A kind of computer room business migration methods, devices and systems |
| CN106326041A (en) * | 2016-08-31 | 2017-01-11 | 杭州沃趣科技股份有限公司 | Second-level recovery method for database |
| CN107391758A (en) * | 2017-08-24 | 2017-11-24 | 阿里巴巴集团控股有限公司 | Database switching method, device and equipment |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376010A (en) * | 2018-09-28 | 2019-02-22 | 上海思询信息科技有限公司 | A method of across cluster resource migration is realized based on Openstack |
| CN110120979A (en) * | 2019-05-20 | 2019-08-13 | 华为技术有限公司 | A kind of dispatching method, device and relevant device |
| CN111241060A (en) * | 2020-01-08 | 2020-06-05 | 苏州科达科技股份有限公司 | Data migration method, system, device and storage medium |
| CN111367892A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Data migration method and device |
| CN111367892B (en) * | 2020-03-31 | 2024-01-19 | 中国建设银行股份有限公司 | Data migration method and device |
| CN113010515A (en) * | 2021-03-03 | 2021-06-22 | 中国工商银行股份有限公司 | Database updating method, device, equipment and medium |
| CN114780293A (en) * | 2022-04-26 | 2022-07-22 | 北京科杰科技有限公司 | Remote double-activity disaster recovery method, device and equipment based on hadoop and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038153A (en) | The online data moving method and device of Hbase | |
| US10360536B2 (en) | Implementing a consistent ordering of operations in collaborative editing of shared content items | |
| US20230090977A1 (en) | Synchronized content library | |
| US9325571B2 (en) | Access permissions for shared content | |
| CN108628874B (en) | Method and device for migrating data, electronic equipment and readable storage medium | |
| CN109117425A (en) | Management is stored as the digital asset of component and packaging file | |
| EP2976871A1 (en) | Local server for synced online content management system | |
| CN108052664A (en) | The data migration method and device of database purchase cluster | |
| US9547707B2 (en) | Copy of replication status for synchronization | |
| CN106605217B (en) | For the method and system for being moved to another website from a website will to be applied | |
| CN109144785A (en) | Method and apparatus for Backup Data | |
| CN107203327A (en) | A kind of method and apparatus for storage system | |
| CN111104069A (en) | Multi-region data processing method and device of distributed storage system and electronic equipment | |
| CN103475721A (en) | System for updating digital assets and method thereof | |
| US9037539B2 (en) | Data synchronization | |
| CN109558191A (en) | A kind of method, apparatus, storage medium and computer equipment configuring the page | |
| CN104462342B (en) | database snapshot synchronization processing method and device | |
| US10127270B1 (en) | Transaction processing using a key-value store | |
| CN109614383A (en) | Data copy method, device, electronic equipment and storage medium | |
| CN113361236A (en) | Method and device for editing document | |
| US11593098B2 (en) | Synchronization of source code under development in multiple concurrent instances of an integrated development environment | |
| CN109614440A (en) | Method of data synchronization and relevant device based on big data | |
| CN108733460B (en) | Method and apparatus for maintaining sessions for network storage devices | |
| US20170091253A1 (en) | Interrupted synchronization detection and recovery | |
| US11526534B2 (en) | Replicating data changes through distributed invalidation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180515 |
|
| RJ01 | Rejection of invention patent application after publication |