CN110362582A - A kind of method and apparatus for realizing zero shutdown upgrading - Google Patents
A kind of method and apparatus for realizing zero shutdown upgrading Download PDFInfo
- Publication number
- CN110362582A CN110362582A CN201810303780.5A CN201810303780A CN110362582A CN 110362582 A CN110362582 A CN 110362582A CN 201810303780 A CN201810303780 A CN 201810303780A CN 110362582 A CN110362582 A CN 110362582A
- Authority
- CN
- China
- Prior art keywords
- hbase cluster
- version
- data
- hbase
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000001360 synchronised effect Effects 0.000 claims abstract description 55
- 230000010076 replication Effects 0.000 claims description 39
- 238000003860 storage Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 6
- 230000005012 migration Effects 0.000 abstract description 7
- 238000013508 migration Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 34
- 238000012360 testing method Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 230000006854 communication Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
Abstract
The invention discloses the method and apparatus that upgrading is shut down in realization zero, are related to field of computer technology.One specific embodiment of this method includes: in lowest version HBase cluster default namespace；The data of new write-in lowest version HBase cluster are specified into corresponding NameSpace value, the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster.The embodiment, which is able to solve, cannot achieve from lowest version HBase cluster to highest version HBase cluster the problem of not shutting down migration.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of method and apparatus for realizing zero shutdown upgrading.
Background technique
In the epoch of data high-speed development, distributed system becomes critically important role.In the back of the development of big data
Under scape, distributed type assemblies, distributed data base, distributed file system all becomes by focus concerned by people.It is dynamic at this
Under the trend of power, Hadoop is clustered into for more mature distributed system.Distributed data base HBase is in Hadoop ecosystem
Continuous perfect in system, HBase is different from traditional database, it is one and is suitable for opening for unstructured data storage
Source database, has high scalability, high-performance, and column store, is scalable, can read and write the characteristics of big data in real time.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery:
In the use process of distributed data base HBase, with new technology, new function is continuously added, to use
These new functions, will inevitably upgrade database, generally in the case where version span is little, upgrade database, original place upgrade just
Can be with, but in actual production environment, the version span of upgrading may be very big, the no longer compatible lowest version of highest version.
Table on HBase1.1.X is under namespace, if not specifying namespace, table can have default
Under namespace.HBase0.94 does not have namespace, while the agreement that HBase0.94 and HBase1.1.X are used is different
Sample, and HBase1.1.X changes some API, causes not in compatible HBase0.98 version below, if migrating data,
It then needs to build a cluster, then by the cluster migrating data built, therefore needs to shut down a period of time in migrating data,
The duration of shutdown increases and decreases according to the data volume size of business.The data of 1T must be shut down 1 hour or more under normal conditions.Shut down for
For some important business, do not allow.Even shut down one minute, caused by lose and also should not be underestimated.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of method and apparatus for realizing zero shutdown upgrading, being able to solve can not
It realizes from lowest version HBase cluster to highest version HBase cluster and does not shut down the problem of migration.
To achieve the above object, according to an aspect of an embodiment of the present invention, provide it is a kind of realize zero shutdown upgrading
Method is included in lowest version HBase cluster default namespace；The data of new write-in lowest version HBase cluster are specified corresponding
NameSpace value, the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster.
Optionally, the data of new write-in lowest version HBase cluster are specified into corresponding NameSpace value, comprising:
In the server by NameSpace preset in lowest version HBase cluster storage, when lowest version HBase cluster has
When the data being newly written, the NameSpace value in server is obtained, specifies phase with the data to new write-in lowest version HBase cluster
The NameSpace value answered.
Optionally, the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster, comprising:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock,
Then the overstocked data are copied into highest version HBase cluster in HDFS level；
After the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster, then in lowest version
Turn-on data is synchronous in HBase cluster.
Optionally, the data are copied into highest version HBase cluster in HDFS level, comprising:
In lowest version HBase cluster and it is separately turned on Replication function in highest version HBase cluster, and is specified
The port of Thrift service；Then the data are replicated to highest version using Thrift agreement by Replication function
In HBase cluster.
Optionally, it after the data of new write-in lowest version HBase cluster being synchronized in highest version HBase cluster, also wraps
It includes:
The data being copied in highest version HBase cluster are repaired, to confirm that the data are available.
Optionally, add_peer order default namespace is used in lowest version HBase cluster shell.
Optionally, further includes:
Replication option pack based on Thrift, by the api interface and highest version of lowest version HBase cluster
The api interface of HBase cluster is corresponding；
Default namespace is added in the Replication option pack based on Thrift of lowest version HBase cluster
Value, the data of lowest version HBase cluster are synchronized in highest version HBase cluster；
Highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version
HBase cluster works normally.
In addition, according to an aspect of an embodiment of the present invention, providing a kind of device for realizing zero shutdown upgrading, including set
Module is set, in lowest version HBase cluster default namespace；Synchronization module, for lowest version HBase cluster will to be newly written
Data specify corresponding NameSpace value, the data of new write-in lowest version HBase cluster are synchronized to highest version HBase collection
In group.
Optionally, the data of new write-in lowest version HBase cluster are specified corresponding NameSpace value by the synchronization module,
Include:
In the server by NameSpace preset in lowest version HBase cluster storage, when lowest version HBase cluster has
When the data being newly written, the NameSpace value in server is obtained, specifies phase with the data to new write-in lowest version HBase cluster
The NameSpace value answered.
Optionally, the data of new write-in lowest version HBase cluster are synchronized to highest version HBase cluster by the synchronization module
In, comprising:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock,
Then the overstocked data are copied into highest version HBase cluster in HDFS level；
After the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster, then in lowest version
Turn-on data is synchronous in HBase cluster.
Optionally, the synchronization module copies the data into highest version HBase cluster in HDFS level, comprising:
In lowest version HBase cluster and it is separately turned on Replication function in highest version HBase cluster, and is specified
The port of Thrift service；Then the data are replicated to highest version using Thrift agreement by Replication function
In HBase cluster.
Optionally, the synchronization module, is also used to:
The data being copied in highest version HBase cluster are repaired, to confirm that the data are available.
Optionally, the setup module is empty using the default name of add_peer order in lowest version HBase cluster shell
Between.
Optionally, the setup module, is also used to:
Replication option pack based on Thrift, by the api interface and highest version of lowest version HBase cluster
The api interface of HBase cluster is corresponding；
Default namespace is added in the Replication option pack based on Thrift of lowest version HBase cluster
Value, the data of lowest version HBase cluster are synchronized in highest version HBase cluster；
Highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version
HBase cluster works normally.
Other side according to an embodiment of the present invention, additionally provides a kind of electronic equipment, comprising:
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processing
Device realizes method described in any of the above-described embodiment for realizing zero shutdown upgrading.
Other side according to an embodiment of the present invention additionally provides a kind of computer-readable medium, is stored thereon with meter
Calculation machine program realizes method described in any of the above-described embodiment for realizing zero shutdown upgrading when described program is executed by processor.
One embodiment in foregoing invention has the following advantages that or the utility model has the advantages that the present invention is directed to realize lowest version
HBase cluster shuts down upgrade function to highest version HBase cluster zero, meets important service and does not shut down upgrading HBase database
Demand so that these business no longer suffer shut down bring massive losses.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the schematic diagram of the main flow of the method according to an embodiment of the present invention for realizing zero shutdown upgrading；
Fig. 2 is the schematic diagram that can refer to the main flow of the method for the shutdown upgrading of realization zero of embodiment according to the present invention；
Fig. 3 is the schematic diagram of the main modular of the device according to an embodiment of the present invention for realizing zero shutdown upgrading；
Fig. 4 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 5 is adapted for the structural representation of the computer system for the terminal device or server of realizing the embodiment of the present invention
Figure.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
Fig. 1 is the method according to an embodiment of the present invention for realizing zero shutdown upgrading, as shown in Figure 1, described realize that zero shuts down
The method of upgrading includes:
Step S101 presets namespace in lowest version HBase cluster.
Preferably, presetting namespace using add_peer order in lowest version HBase cluster shell.Wherein,
Namespace is " NameSpace ", also referred to as " name space ", " name space ".One kind of various language uses in VS.NET
The form of code organization is classified by name space, is distinguished different code functions while being also the complete of all classes in VS.NET
A part of full title.
The data of new write-in lowest version HBase cluster are specified corresponding namespace value, will newly write by step S102
The data for entering lowest version HBase cluster are synchronized in highest version HBase cluster.
Preferably, preset namespace is stored in the server, as lowest version HBase in lowest version HBase cluster
When cluster has the data being newly written, the namespace value in server is obtained, to new write-in lowest version HBase cluster
Data specify corresponding namespace value.Preferably, zookeeper server can be used.Wherein, zookeeper refers to one
A distributed, open source code distribution application system coordination service, is the significant components of Hadoop and HBase, for distribution
Formula application provides Consistency service.
In one embodiment, the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster,
Specifically implementation process includes:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock,
Then the data are copied into highest version HBase cluster in HDFS level.Then, by new write-in lowest version HBase cluster
After data are synchronized in highest version HBase cluster, then turn-on data is synchronous in lowest version HBase cluster.
It further, can be in lowest version when HDFS level copies the data into highest version HBase cluster
Replication function, and the port of specified Thrift service are separately turned in HBase cluster and in highest version HBase cluster；
Then the data are replicated into highest version HBase cluster using Thrift agreement by Replication function.
As another embodiment of the invention, the data of new write-in lowest version HBase cluster are synchronized to highest version
After in HBase cluster, it is also necessary to the data being copied in highest version HBase cluster are repaired, to confirm that the data are available.
Also it is worth noting that, the present invention by step S101 and S102 can complete that lowest version HBase cluster is newly written
Data be synchronized to highest version HBase cluster, and for largely being counted present in lowest version HBase cluster before data are newly written
It is synchronized in highest version HBase cluster according to can then be realized by following process:
Firstly, the Replication option pack based on Thrift, by the api interface of lowest version HBase cluster and high version
The api interface of this HBase cluster is corresponding.Then, in the Replication function based on Thrift of lowest version HBase cluster
Default namespace value is added in patch, the data of lowest version HBase cluster are synchronized in highest version HBase cluster.Most
Afterwards, highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version HBase
Cluster works normally.
According to various embodiments above, it can be seen that the method that upgrading is shut down in the realization zero, the present invention solve
It is existing to can not achieve lowest version HBase cluster and the problem of not shutting down migration, realize zero shutdown liter to highest version HBase cluster
Grade, HBase cluster do not have to shut down, and user, which no longer suffers, shuts down bring huge economic losses, while can be to highest version HBase
The table synchrodata of the specified namespace of cluster.Moreover, it is achieved that lowest version HBase cluster does not stop to highest version HBase cluster
The function of machine migration is very useful, and there is an urgent need to.
Fig. 2 is the schematic diagram that can refer to the main flow of the method for the shutdown upgrading of realization zero of embodiment according to the present invention,
The realization zero shuts down the method upgraded and may include:
Step S201, the Replication option pack based on Thrift, by the api interface of lowest version HBase cluster with
The api interface of highest version HBase cluster is corresponding.
Wherein, the lowest version is HBase0.98 version below, and the highest version is HBase1.1.X version
And the above version.
In embodiment, since the frame of lowest version and highest version is not identical, the API of especially high and low two version is connect
Mouthful setting it is not identical, therefore the patch patch of lowest version HBase cluster not can be used directly on highest version HBase cluster,
The api interface information of lowest version carried out with highest version api interface information for this problem embodiment of the present invention corresponding.Its
In, it is highest version HBase cluster that the patch patch of the lowest version HBase cluster, which is exactly by lowest version HBase cluster upgrade,
Patch patch.
Step S202 is added default in the Replication option pack based on Thrift of lowest version HBase cluster
The data of lowest version HBase cluster are synchronized in highest version HBase cluster by namespace value.
It should be noted that the step may be implemented the data in current lowest version HBase cluster are fully synchronized supreme
In version HBase cluster.
Step S203 restarts the regionserver of highest version HBase cluster and lowest version HBase cluster, so that
Highest version HBase cluster and lowest version HBase cluster work normally.
In embodiment, can restarting regionserver, (it is each that regionserver is that HBase cluster operates in
Service on working node) so that code can work normally.
Preferably, Replication synchronizing function is work on regionserver, HBase is mainly by java
What language was write, (it is highest version that the patch patch is by lowest version HBase cluster upgrade in the present embodiment to patch patch
HBase cluster relevant program) be to beat in java code, be then compiled into jar file, update and arrive regionserver
In corresponding server, restarting regionserver is to allow the code of change to work normally to reload jar file.
Step S204 presets namespace using add_peer order in lowest version HBase cluster shell.
Wherein, shell is computer shell, is commonly called as shell, refers to and provides user's software (command analysis for using interface
Device).And namespace refers to name space, the logic groups of database table, the table in same group has similar purposes.
In embodiment, we are one default using add_peer order in the shell of lowest version HBase cluster
namespace。
Preferably, when using add_peer order in the shell of lowest version, it is possible to specify HBaseReplication function
Used protocol type.Wherein, for Replication function for replicating between cluster, it is that (master is more by master/slaves
From) duplication of structural formula.It is pushed by master, multiple slave are received.This duplication operation be it is asynchronous, the state of duplication and
The information preservations such as progress are into zookeeper server.It assists to complete replication work by zookeeper server.
Preferably, HBase Replication function uses Thrift agreement.And Thrift is a software frame, is used
In can expand and the exploitation of the service across language.Such as:
add_peer‘1’,”hbase-1.1:2181:/hbase”,”THRIFT”,”namespace_test”
The order is that add one from cluster be peer to HBase0.94, and specified Replication function is assisted using Thrift
View, toward namespace_test under synchronization.
Step S205, when lowest version HBase cluster has the data being newly written, to new write-in lowest version HBase cluster
Data specify corresponding namespace value.
Preferably, preset namespace can store in zookeeper server in lowest version HBase cluster,
When lowest version HBase cluster has the data being newly written, the namespace value in zookeeper server is obtained, to new
The data of write-in lowest version HBase cluster specify corresponding namespace value.
It is possible to further which the value of namespace is stored in namespace_test table, and will be described
Namespace_test table has been stored in zookeeper server.When the synchronous incremental data of Replication, then from
Namespace_test is read in zookeeper, each data to be newly written adds upper corresponding namespace value.
To which the data being newly written are passed through Replication function in highest version HBase cluster, Thrift agreement is used to carry out data
When duplication, data are written in the table under the namespace that can be specified in previous highest version HBase cluster.
That is, in lowest version HBase cluster preset namespace value be can be with highest version HBase cluster
In namespace value carry out corresponding, and then the data in lowest version HBase cluster could be synchronized to highest version HBase
In cluster.
Step S206 suspends data by disable peer order and synchronizes, so as to which lowest version HBase cluster is newly written
Data are overstock, then in HDFS level copy data into high version HBase cluster.
Wherein, HDFS is Hadoop distributed file system, is configured to be suitble to operate in the distribution on common hardware
Formula file system has Error Tolerance, is suitble to be deployed on cheap machine.The data that HDFS can provide high-throughput are visited
It asks, is suitble to apply on large-scale dataset.
Also it is worth noting that, the present embodiment, which uses, during data are synchronous first carries out disable peer life
Pause data synchronizing function is enabled so that data are overstock, data to be synchronized are completed to copy and be executed again after can be used normally
Enable peer order, turn-on data synchronizing function.That is, synchronous data can be played with a kind of effect of caching.
It preferably, can be in lowest version HBase when HDFS level copy data are into high version HBase cluster
Replication function, and the port of specified Thrift service are separately turned in cluster and in highest version HBase cluster.Then
The data are replicated into highest version HBase cluster using Thrift agreement by Replication function.
Further, following configuration is added in the configuration file of hbase-site.xml:
<property>
<name>hbase.replication</name>
<value>true</value>
<description>open replication function</description>
</property>
<property>
<name>hbase.replication.sink.enable.thrift</name>
<value>true</value>
</property>
<property><name>hbase.replication.thrift.server.port</name>
<value>9090</value>
</property>
Thrift service use is then turned on such as to issue orders:
$HBASE_HOME/bin/hbase-daemon.sh start thrift
Step S207 repairs the data being copied in highest version HBase cluster, to confirm that the data are available, then exists
It is synchronous by enable peer order turn-on data in the shell of lowest version HBase cluster.
Preferably, being that can be used as ordered repairing the data being copied in highest version HBase cluster:
FixTableOrphans, fixMeta and fix.
Such as: on the HDFS that the data copy of table tb_test to HBase1.1.X stores, when repair data, process is as follows:
hdfs dfs-rm–r hdfs://Hadoop_master_ip_address:9000/hbase/data/
namespace_test/tb_test/.tableinfo.0000000001
hbase hbck-fixTableOrphans"tb_test"
hbase hbck-fixMeta"tb_test"
hbase hbck-fix"tb_test"
In one embodiment, (data being newly written can pass through step after completing the synchronization of all data
S205 to S207 is synchronized), highest version HBase cluster can be switched to.It is possible to further be arranged one in lowest version
The migration table of HBase cluster can when the data volume in migration table is consistent with the data volume in highest version HBase cluster
Switch to highest version HBase cluster.
Fig. 3 is the device according to an embodiment of the present invention for realizing zero shutdown upgrading, as shown in figure 3, described realize that zero shuts down
The device 300 of upgrading includes setup module 301 and synchronization module 302.Wherein, setup module 301 is pre- in lowest version HBase cluster
If namespace.Later, the data that lowest version HBase cluster is newly written in synchronization module 302 specify corresponding namespace value,
The data of new write-in lowest version HBase cluster to be synchronized in highest version HBase cluster.
Preferably, setup module 301 is default using add_peer order in lowest version HBase cluster shell
namespace。
As embodiment, the preset namespace in lowest version HBase cluster of synchronization module 302 is stored in server
In, when lowest version HBase cluster has the data being newly written, the namespace value in server is obtained, with low to being newly written
The data of version HBase cluster specify corresponding namespace value.Preferably, zookeeper server can be used.Wherein,
Zookeeper refers to distributed, open source code a distribution application system coordination service, is Hadoop and Hbase
Significant components provide Consistency service for Distributed Application.
In addition, the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster by synchronization module 302,
Specifically implementation process includes:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock,
Then the data are copied into highest version HBase cluster in HDFS level.Then, by new write-in lowest version HBase cluster
After data are synchronized in highest version HBase cluster, then turn-on data is synchronous in lowest version HBase cluster.
It further, can be in lowest version when HDFS level copies the data into highest version HBase cluster
Replication function is opened in HBase cluster, specifies the port of Thrift service；Then pass through Replication function,
Using Thrift agreement, the data are replicated into highest version HBase cluster.
As another embodiment of the invention, synchronization module 302 is synchronous by the data of new write-in lowest version HBase cluster
After into highest version HBase cluster, it is also necessary to the data being copied in highest version HBase cluster are repaired, to confirm the number
According to available.
Also it is worth noting that, the data that the present invention can complete newly to be written lowest version HBase cluster are synchronized to highest version
HBase cluster, and then can be by the way that mould is arranged for mass data present in lowest version HBase cluster before data is newly written
Block 301 is synchronized in highest version HBase cluster using the realization of following process:
Firstly, the Replication option pack based on Thrift, by the api interface of lowest version HBase cluster and high version
The api interface of this HBase cluster is corresponding.Then, in the Replication function based on Thrift of lowest version HBase cluster
Default namespace value is added in patch, the data of lowest version HBase cluster are synchronized in highest version HBase cluster.Most
Afterwards, highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version HBase
Cluster works normally.
It should be noted that the zero specific implementation content for shutting down the device upgraded is realized of the present invention, in institute above
It states and has been described in detail in the method for realizing zero shutdown upgrading, therefore no longer illustrate in this duplicate contents.
Fig. 4, which is shown, can shut down the method for upgrading using the realization zero of the embodiment of the present invention or realize that zero shuts down upgrading
The exemplary system architecture 400 of device.Or Fig. 4 shows the side that upgrading can be shut down using the realization zero of the embodiment of the present invention
Method or the exemplary system architecture 400 for realizing the device that zero shuts down upgrading.
As shown in figure 4, system architecture 400 may include terminal device 401,402,403, network 404 and server 405.
Network 404 between terminal device 401,402,403 and server 405 to provide the medium of communication link.Network 404 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 401,402,403 and be interacted by network 404 with server 405, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 401,402,403
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform softwares.
Terminal device 401,402,403 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 405 can be to provide the server of various services, such as utilize terminal device 401,402,403 to user
The shopping class website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to reception
To the data such as information query request analyze etc. processing, and by processing result (such as target push information, product letter
Breath -- merely illustrative) feed back to terminal device.
It should be noted that realizing that zero shuts down the method for upgrading generally by server 405 provided by the embodiment of the present invention
It executes, correspondingly, the device for realizing that zero shuts down upgrading is generally positioned in server 405.
It should be understood that the number of terminal device, network and server in Fig. 4 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Below with reference to Fig. 5, it illustrates the computer systems 500 for the terminal device for being suitable for being used to realize the embodiment of the present invention
Structural schematic diagram.Terminal device shown in Fig. 5 is only an example, function to the embodiment of the present invention and should not use model
Shroud carrys out any restrictions.
As shown in figure 5, computer system 500 includes central processing unit (CPU) 501, it can be read-only according to being stored in
Program in memory (ROM) 502 or be loaded into the program in random access storage device (RAM) 503 from storage section 508 and
Execute various movements appropriate and processing.In RAM503, also it is stored with system 500 and operates required various programs and data.
CPU501, ROM 502 and RAM503 is connected with each other by bus 504.Input/output (I/O) interface 505 is also connected to bus
504。
I/O interface 505 is connected to lower component: the importation 506 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 507 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 508 including hard disk etc.；
And the communications portion 509 of the network interface card including LAN card, modem etc..Communications portion 509 via such as because
The network of spy's net executes communication process.Driver 510 is also connected to I/O interface 505 as needed.Detachable media 511, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 510, in order to read from thereon
Computer program be mounted into storage section 508 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer
Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.?
In such embodiment, which can be downloaded and installed from network by communications portion 509, and/or from can
Medium 511 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 501, system of the invention is executed
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
Include setup module and synchronization module.Wherein, the title of these modules does not constitute the limit to the module itself under certain conditions
It is fixed.
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
Obtaining the equipment includes: to preset namespace in lowest version HBase cluster；The data of new write-in lowest version HBase cluster are specified
The data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster by corresponding namespace value.
Technical solution according to an embodiment of the present invention, being able to solve cannot achieve from lowest version HBase cluster to highest version
HBase cluster does not shut down the problem of migration.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (16)
1. a kind of method for realizing zero shutdown upgrading characterized by comprising
In lowest version HBase cluster default namespace；
The data of new write-in lowest version HBase cluster are specified into corresponding NameSpace value, lowest version HBase collection will newly be written
The data of group are synchronized in highest version HBase cluster.
2. the method according to claim 1, wherein the data of new write-in lowest version HBase cluster are specified phase
The NameSpace value answered, comprising:
In the server by NameSpace preset in lowest version HBase cluster storage, it is newly write when lowest version HBase cluster has
When the data entered, the NameSpace value in server is obtained, it is specified corresponding with the data to new write-in lowest version HBase cluster
NameSpace value.
3. the method according to claim 1, wherein the data of new write-in lowest version HBase cluster are synchronized to
In highest version HBase cluster, comprising:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock, then
The overstocked data are copied into highest version HBase cluster in HDFS level；
After the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster, in lowest version HBase collection
Turn-on data is synchronous in group.
4. according to the method described in claim 3, it is characterized in that, copying the data to highest version HBase in HDFS level
In cluster, comprising:
In lowest version HBase cluster and it is separately turned on Replication function in highest version HBase cluster, and is specified
The port of Thrift service；Then the data are replicated to highest version using Thrift agreement by Replication function
In HBase cluster.
5. the method according to claim 1, wherein the data of new write-in lowest version HBase cluster are synchronized to
After in highest version HBase cluster, further includes:
The data being copied in highest version HBase cluster are repaired, to confirm that the data are available.
6. the method according to claim 1, wherein using add_peer in lowest version HBase cluster shell
Order default namespace.
7. -6 any method according to claim 1, which is characterized in that further include:
Replication option pack based on Thrift, by the api interface of lowest version HBase cluster and highest version HBase collection
The api interface of group is corresponding；
Default namespace value is added in the Replication option pack based on Thrift of lowest version HBase cluster, with
The data of lowest version HBase cluster are synchronized in highest version HBase cluster；
Highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version HBase
Cluster works normally.
8. a kind of device for realizing zero shutdown upgrading characterized by comprising
Setup module, in lowest version HBase cluster default namespace；
Synchronization module will newly be written for the data of new write-in lowest version HBase cluster to be specified corresponding NameSpace value
The data of lowest version HBase cluster are synchronized in highest version HBase cluster.
9. device according to claim 8, which is characterized in that lowest version HBase cluster will be newly written in the synchronization module
Data specify corresponding NameSpace value, comprising:
In the server by NameSpace preset in lowest version HBase cluster storage, it is newly write when lowest version HBase cluster has
When the data entered, the NameSpace value in server is obtained, it is specified corresponding with the data to new write-in lowest version HBase cluster
NameSpace value.
10. device according to claim 8, which is characterized in that lowest version HBase cluster will be newly written in the synchronization module
Data be synchronized in highest version HBase cluster, comprising:
It is synchronous to suspend data in lowest version HBase cluster, so that the new data that lowest version HBase cluster is written are overstock, then
The overstocked data are copied into highest version HBase cluster in HDFS level；
After the data of new write-in lowest version HBase cluster are synchronized in highest version HBase cluster, then in lowest version HBase
Turn-on data is synchronous in cluster.
11. device according to claim 10, which is characterized in that the synchronization module copies the data in HDFS level
Into highest version HBase cluster, comprising:
In lowest version HBase cluster and it is separately turned on Replication function in highest version HBase cluster, and is specified
The port of Thrift service；Then the data are replicated to highest version using Thrift agreement by Replication function
In HBase cluster.
12. device according to claim 8, which is characterized in that the synchronization module is also used to:
The data being copied in highest version HBase cluster are repaired, to confirm that the data are available.
13. device according to claim 8, which is characterized in that the setup module is in lowest version HBase cluster shell
It is middle to use add_peer order default namespace.
14. according to any device of claim 8-13, which is characterized in that the setup module is also used to:
Replication option pack based on Thrift, by the api interface of lowest version HBase cluster and highest version HBase collection
The api interface of group is corresponding；
Default namespace value is added in the Replication option pack based on Thrift of lowest version HBase cluster, with
The data of lowest version HBase cluster are synchronized in highest version HBase cluster；
Highest version HBase cluster and lowest version HBase cluster are restarted, so that highest version HBase cluster and lowest version HBase
Cluster works normally.
15. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 1-7.
16. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
The method as described in any in claim 1-7 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810303780.5A CN110362582A (en) | 2018-04-03 | 2018-04-03 | A kind of method and apparatus for realizing zero shutdown upgrading |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810303780.5A CN110362582A (en) | 2018-04-03 | 2018-04-03 | A kind of method and apparatus for realizing zero shutdown upgrading |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110362582A true CN110362582A (en) | 2019-10-22 |
Family
ID=68213437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810303780.5A Pending CN110362582A (en) | 2018-04-03 | 2018-04-03 | A kind of method and apparatus for realizing zero shutdown upgrading |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110362582A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112632032A (en) * | 2020-12-18 | 2021-04-09 | 华人运通(上海)云计算科技有限公司 | Data migration method and device, storage medium and terminal equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631907A (en) * | 2013-11-26 | 2014-03-12 | 中国科学院信息工程研究所 | Method and system for migrating relational data to HBbase |
| CN105243067A (en) * | 2014-07-07 | 2016-01-13 | 北京明略软件系统有限公司 | Method and apparatus for realizing real-time increment synchronization of data |
| CN106855871A (en) * | 2015-12-09 | 2017-06-16 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of Data Migration |
| CN107122361A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
-
2018
- 2018-04-03 CN CN201810303780.5A patent/CN110362582A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631907A (en) * | 2013-11-26 | 2014-03-12 | 中国科学院信息工程研究所 | Method and system for migrating relational data to HBbase |
| CN105243067A (en) * | 2014-07-07 | 2016-01-13 | 北京明略软件系统有限公司 | Method and apparatus for realizing real-time increment synchronization of data |
| CN106855871A (en) * | 2015-12-09 | 2017-06-16 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of Data Migration |
| CN107122361A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112632032A (en) * | 2020-12-18 | 2021-04-09 | 华人运通(上海)云计算科技有限公司 | Data migration method and device, storage medium and terminal equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086409B (en) | Microservice data processing method and device, electronic equipment and computer readable medium | |
| RU2500023C2 (en) | Document synchronisation on protocol not using status information | |
| CN108804306B (en) | Method and system for automatic test system | |
| CN109542445A (en) | A kind of method and apparatus that Android plug-in unit melts hair | |
| US20140195514A1 (en) | Unified interface for querying data in legacy databases and current databases | |
| CN110083660A (en) | A kind of method, apparatus of synchrodata, medium and electronic equipment | |
| CN110019211A (en) | The methods, devices and systems of association index | |
| CN110472207A (en) | List generation method and device | |
| CN108958744A (en) | Dispositions method, device, medium and the electronic equipment of big data distributed type assemblies | |
| CN107506218A (en) | The management method and management system of a kind of configuration file | |
| CN110019062A (en) | Method of data synchronization and system | |
| US20160140141A1 (en) | File lookup in a distributed file system | |
| CN110019539A (en) | A kind of method and apparatus that the data of data warehouse are synchronous | |
| CN109241033A (en) | The method and apparatus for creating real-time data warehouse | |
| CN110781197A (en) | Hive offline synchronous verification method and device and electronic equipment | |
| US20230259358A1 (en) | Documentation enforcement during compilation | |
| US8527940B2 (en) | Integrated development environment with automatic working sets | |
| CN113128197A (en) | Method and device for managing application production versions | |
| CN109828964A (en) | A kind of data lead-in method, device and electronic equipment | |
| CN109725913A (en) | The method and apparatus that data update | |
| CN110362582A (en) | A kind of method and apparatus for realizing zero shutdown upgrading | |
| CN110580216B (en) | Application lifting method and device | |
| CN110347654A (en) | A kind of method and apparatus of online cluster features | |
| US10146791B2 (en) | Open file rebalance | |
| CN110858199A (en) | Document data distributed computing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |