CN103491192B - The Namenode changing method of a kind of distributed system and system - Google Patents
The Namenode changing method of a kind of distributed system and system Download PDFInfo
- Publication number
- CN103491192B CN103491192B CN201310462918.3A CN201310462918A CN103491192B CN 103491192 B CN103491192 B CN 103491192B CN 201310462918 A CN201310462918 A CN 201310462918A CN 103491192 B CN103491192 B CN 103491192B
- Authority
- CN
- China
- Prior art keywords
- namenode
- main
- data
- memory
- memory block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
nullThe embodiment of the invention discloses the Namenode switching method and apparatus of a kind of distributed system，Configuration at least two Namenode，Described Namenode is connected in a transient node and obtains Connecting quantity，Determine a main Namenode and at least one is from Namenode，Distributed system only uses described main Namenode to provide service，When main Namenode disconnects with transient node，Trigger determining from the Namenode described call back function of execution and using new main Namenode to provide service in described transient node，Visible，When the main Namenode of distributed system does not provides service，The main Namenode that makes new advances can be determined from other Namenode arranged to come for distributed system continue to provide service，Therefore distributed system will not cannot normally be worked，Substantially increase the stability of distributed system.
Description
Technical field
The present invention relates to computer realm, particularly relate to the Namenode switching side of a kind of distributed system
Method and system.
Background technology
Along with needing data volume to be processed to get more and more on network, for the distributed system processing mass data
The range of application of system is more and more extensive.Metadata node Namenode in a distributed system is whole
The core of the document storage system in distributed system, all of operation of distributed file storage system is both needed to
Participated in by Namenode.Further, Namenode is responsible for safeguarding all literary compositions in whole distributed file system
The metamessage of part and directory information, if Namenode occurs in that failure, then distributed document stores
In system, All Files information will all be lost.Although in general distributed system is for every number evidence all
Many parts of data backups can be carried out according to configuration, i.e. will not occur in that failure because of Namenode and cause be
Data in system are all lost, but whole distributed system still can be because of an only Namenode
Cannot work and cause providing normal service.
Summary of the invention
Nothing is caused because an only Namenode cannot work in order to solve above-mentioned distributed system
Method provides the technical problem of normal service, the invention provides the Namenode of a kind of distributed system
Changing method and system.
The embodiment of the invention discloses following technical scheme:
The Namenode changing method of a kind of distributed system, configures at least two Namenode, by institute
Stating Namenode and be connected in a transient node and obtain Connecting quantity, described Namenode is with default
Call back function determines a main Namenode and at least according to described Connecting quantity from described Namenode
One from Namenode, distributed system only uses described main Namenode to provide service, including:
When main Namenode disconnects with transient node, then judge that main Namenode cannot provide clothes
Business, trigger in described transient node from Namenode perform described call back function described from
Namenode determines new main Namenode；
Described new main Namenode is used to provide service.
Preferably, also include:
When determining main Namenode, the network parameter of described main Namenode is updated default
In memory element；
Triggering user side and server side read the network of described main Namenode from described memory element
Parameter so that described master is passed through according to the described network parameter read in described user side and server side
Namenode carries out data interaction.
Preferably, also include that distributed sha red memory system, described distributed sha red memory system are divided into many
Individual memory block carries out block-by-block storage to the data of write:
Described main Namenode receives data write request；
Data content in the data write request that described main Namenode will be successfully processed writes described distribution
In the memory block of formula shared memory systems；
When distributed sha red memory system detect currently stored piece write full data after, switch another memory block
For storing the data of described main Namenode write, and send hand off signaling in described transient node
From Namenode, described hand off signaling includes the positional information of the memory block writing full data being just switched；
Accordingly, just it has been switched from described according to positional information described hand off signaling from Namenode
Write in the memory block of full data reading data content and being stored to the internal memory of corresponding described Namenode in advance.
Preferably, before the described new main Namenode of described use provides service, also include:
Whether the described new main Namenode described distributed sha red memory system of detection comprise not written
Full memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
Preferably, also include:
Described Connecting quantity specially Namenode is connected to the Connection Time of transient node.
A kind of Namenode switching device of distributed system, including:
Dispensing unit, is used for configuring at least two Namenode, described Namenode is connected to one and faces
In Shi Jiedian and obtain Connecting quantity, described Namenode connects ginseng with default call back function according to described
Number determines a main Namenode from described Namenode and at least one is from Namenode, distribution
Formula system only uses described main Namenode to provide service；
Main Namenode judging unit again, is used for when main Namenode disconnects with transient node,
Then judge that main Namenode cannot provide service, trigger performing from Namenode in described transient node
Described call back function determines new main Namenode described from Namenode；
There is provided service unit, for using described new main Namenode to provide service.
Preferably, also include:
Network parameter storage unit, for when determining main Namenode, by described main Namenode
Network parameter update in default memory element；
Trigger and read unit, be used for triggering user side and server side and read described from described memory element
The network parameter of main Namenode so that described user side and server side are according to the described network ginseng read
Number carries out data interaction by described main Namenode.
Preferably, also include that distributed sha red memory system, described distributed sha red memory system are divided into many
Individual memory block carries out block-by-block storage to the data of write:
Receive write request unit, for receiving data write request for described main Namenode；
Data write unit, the number in the data write request being successfully processed for described main Namenode
Write according to content in the memory block of described distributed sha red memory system；
When distributed sha red memory system, memory block switch unit, for for detecting that currently stored piece is write full
After data, switch another memory block for storing the data of described main Namenode write, and send switching
Signaling in described transient node from Namenode, described hand off signaling includes that be just switched writes full number
According to the positional information of memory block；
Synchrodata content element, is used for being according to the positional information described hand off signaling from Namenode
From the described memory block writing full data being just switched, read data content and be stored in advance described in correspondence
In the internal memory of Namenode.
Preferably, also include:
Memory block detector unit, before providing service at the described new main Namenode of described use, for
Whether the described new main Namenode described distributed sha red memory system of detection comprise not written full
Memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
Preferably, also include:
Described Connecting quantity specially Namenode is connected to the Connection Time of transient node.
By technique scheme it can be seen that arrange at least two Namenode for distributed system, as
It is main Namenode when not providing service based on some situation, can be immediately from other Namenode arranged
Middle determine the main Namenode made new advances come for distributed system continue provide service so that distributed system is not
Cannot can normally work because of the problem of Namenode, substantially increase the stability of distributed system.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to reality
Execute the required accompanying drawing used in example or description of the prior art to be briefly described, it should be apparent that below,
Accompanying drawing in description is only some embodiments of the present invention, for those of ordinary skill in the art,
On the premise of not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the method flow schematic diagram of the Namenode changing method of a kind of distributed system of the present invention；
Fig. 2 is that another method flow of the Namenode changing method of a kind of distributed system of the present invention shows
It is intended to；
Fig. 3 is the apparatus structure schematic diagram of the Namenode switching device of a kind of distributed system of the present invention；
Fig. 4 is that another apparatus structure of the Namenode switching device of a kind of distributed system of the present invention shows
It is intended to；
Fig. 5 is that another apparatus structure of the Namenode switching device of a kind of distributed system of the present invention shows
It is intended to；
Fig. 6 is that another apparatus structure of the Namenode switching device of a kind of distributed system of the present invention shows
It is intended to.
Detailed description of the invention
Embodiments provide Namenode changing method and the system of a kind of distributed system.One side
Face, arranges at least two Namenode for distributed system and is connected in a transient node, one of them
Namenode provides service as main Namenode for described distributed system, when as main Namenode
When disconnecting based on some situation and transient node, in this transient node, other are described distributed system
Described, arrange can determine that from Namenode new main Namenode comes for described from Namenode
Distributed system continues to provide service so that described distributed system will not be because of the problem of Namenode
Cannot normally work, thus substantially increase the stability of distributed system.
On the other hand, while determine new main Namenode for distributed system, update and be used for storing
The memory element of main Namenode network parameter, and trigger the user side kimonos using above-mentioned distributed system
Business device side reads the net-work parameter information of newly determined main Namenode from described memory element, makes with this
Obtain user and server can use new main Namenode to carry out data interaction timely, thus will not
User or server is occurred still to be continuing with the most not providing the Namenode of service and cause cannot be complete
The problem becoming normal data interaction, further makes distributed system can complete to automatically switch
The function of Namenode.
On the other hand, main Namenode is by the number in the write request of the user side being successfully processed and server side
Writing distributed sha red memory system according to content, distributed sha red memory system will be write with the form of memory block
The data entered carry out the preservation of block-by-block, say, that write a full memory block and just switch another memory block and continue
Renew storage write data, when switch memory block when, can in transient node other from Namenode
Send and comprise the hand off signaling writing full memory block positional information being just switched, can be by from Namenode
The data content write in full memory block being just switched is stored to corresponding from the internal memory of Namenode by advance
In, so make the data content preserved from Namenode internal memory and main Namenode internal memory preserve
Data content basic synchronization, at most the data content in one memory block of difference, is so leading
Namenode loses connection, cannot work in the case of, be confirmed as new main Namenode former from
The data content that Namenode has only to read in the memory block being currently used in storage data can complete
Full data syn-chronization, the handover operation completing Namenode with this, thus substantially reduce distributed system
Time required for switching Namenode, improve user experience.
Understandable, below in conjunction with the accompanying drawings for enabling the above-mentioned purpose of the present invention, feature and advantage to become apparent from
The embodiment of the present invention is described in detail.
Embodiment one
Referring to Fig. 1, it is the method stream of Namenode changing method of a kind of distributed system of the present invention
Journey schematic diagram, the method comprises the following steps:
Configuration step S100: configuration at least two Namenode, is connected to one by described Namenode and faces
In Shi Jiedian and obtain Connecting quantity, described Namenode connects ginseng with default call back function according to described
Number determines a main Namenode from described Namenode and at least one is from Namenode, distribution
Formula system only uses described main Namenode to provide service.
It is to say, be that a distributed system configures multiple Namenode, but synchronization only uses one
Individual providing service as main Namenode, other Namenode is as from Namenode, namely
Say and the most also to configure two Namenode.Only cannot use or in the case of other at main Namenode,
From Namenode, determine that is used as a new main Namenode from least one so that distributed
As long as system can have one can also provide the service normal work of just energy in the Namenode configured for it
Make.The preferred arrangement of the present embodiment is by least two for a distributed system configuration
Namenode is all connected on same transient node, and by call back function according to being connected to transient node
On Connecting quantity from the Namenode configured for this distributed system, determine one as main
Namenode be distributed system provide service, remaining as from Namenode as this distributed system
The alternative Namenode of system, and it should be noted that same transient node connects Namenode
It is the Namenode that same distributed system is configured, and preferably, is to use for large-scale point
Reliable system zookeeper of coordinating of cloth system sets up the transient node connection for Namenode.
A kind of preferred Connecting quantity is the time that Namenode is connected to transient node, like this, returns
Letter of transfer number is using the Connection Time as variable, every time it needs to be determined that during main Namenode, all can trigger each from
Namenode performs call back function, and determines whether oneself is current according to the size order of Connection Time
Connection Time minimum, if it is, determining oneself is new main Namenode, and carry out follow-up phase
Close the operation of master and slave Namenode conversion.
S101: when main Namenode disconnects with transient node, then judge that main Namenode cannot
There is provided service, trigger in described transient node from Namenode perform described call back function described from
Namenode determines new main Namenode；
Needing exist for explanation, main Namenode is mainly possibly due to the connection disconnection of transient node
Main Namenode broken string, damage or roll off the production line etc. and cannot normally provide the situation of service to cause.This feelings
Under condition in addition to judging that former main Namenode cannot provide service, in addition it is also necessary to determine the master made new advances
Namenode replaces former main Namenode and continues as the distributed system due service of offer, and determines
The scheme of the main Namenode made new advances is exactly when transient node finds that the Namenode quantity connected becomes
After change, trigger other Namenode being connected on transient node and perform call back function, pass through Connecting quantity
Determine the main Namenode made new advances.
Preferably, also include:
When determining main Namenode, the network parameter of described main Namenode is updated default
In memory element；
Triggering user side and server side read the network of described main Namenode from described memory element
Parameter so that described master is passed through according to the described network parameter read in described user side and server side
Namenode carries out data interaction.
It is to say, after determining a main Namenode, all can be by the network of this Namenode
Parameter such as IP address etc. update and are saved in a memory element pre-set, and, often
Secondary when need to be determined main Namenode, except triggering other that connect in transient node
Namenode performs beyond call back function, also triggers each use of the user side using this distributed system
Data message in each server described memory element of reading of family and server side, like this,
Every time during more change owner Namenode, main Namenode is used to carry out user side and the server of data interaction
Side can obtain the network parameter of up-to-date main Namenode, and root within the very first time from memory element
Set up data interaction according to this network parameter with new main Namenode to be connected, like this would not occur old
Main Namenode stopped service or occurred being replaced service by new main Namenode after flash
Time, user side and/or server side have used other Namenode to complete owing to not upgrading in time
Data interaction services, and still uses old main Namenode to carry out data interaction, to such an extent as to cannot be smooth
The problem completing operation.And this triggering while switching master/slave Namenode uses distributed system
The user side of system and server side update the network parameter of the main Namenode of distributed system, so that
Master/slave Namenode for distributed system switches not only by manual switching, it is also possible to directly
By system automatic decision, complete to switch automatically.
S102: use described new main Namenode to provide service.
After the associative operation completing master and slave Namenode conversion, distributed system can use new master
Namenode carries out servicing, and the associative operation of concrete master and slave Namenode conversion will implemented
Example two is described.
By the present embodiment it can be seen that arrange at least two Namenode for distributed system and be connected to one
In transient node, one of them Namenode provides clothes as main Namenode for described distributed system
Business, when disconnecting with transient node based on some situation as main Namenode, in this transient node
Other for described distributed system arrange from Namenode can described determine from Namenode new
Main Namenode to continue to provide service for described distributed system so that described distributed system will not be because of
Cannot normally work for the problem of Namenode, thus substantially increase the stability of distributed system.
On the other hand, while determine new main Namenode for distributed system, update and be used for storing
The memory element of main Namenode network parameter, and trigger the user side kimonos using above-mentioned distributed system
Business device side reads the net-work parameter information of newly determined main Namenode from described memory element, makes with this
Obtain user and server can use new main Namenode to carry out data interaction timely, thus will not
User or server is occurred still to be continuing with the most not providing the Namenode of service and cause cannot be complete
The problem becoming normal data interaction, further makes distributed system can complete to automatically switch
The function of Namenode.
Embodiment two
On the basis of embodiment one, the present embodiment will mainly for determine one from Namenode as newly
Main Namenode after, how this associative operation being converted into main Namenode from Namenode is entered
Line description, refers to Fig. 2, and it is Namenode changing method another of a kind of distributed system of the present invention
One method flow schematic diagram, also includes that distributed sha red memory system, the storage of described Distributed sharing are
System is divided into multiple memory block that the data of write are carried out block-by-block storage, including:
Needing exist for explanation, distributed sha red memory system is the data storage of a kind of piecemeal memory-type
System, such as the distributed sha red memory system of log stream is provided for distributed system
Bookkeeper, its Main Function is to a kind of backup of data interaction in distributed system, works as master
When Namenode services, such as receive data write request (usually to the data on server
Content carries out data deletion, increase and amendment etc., all can be considered a kind of write operation to data) time, main
Data content in the data write request that Namenode not only can will be successfully processed is saved in the internal memory of correspondence
In, also can be written in distributed sha red memory system, so can ensure that when main Namenode cannot be just
When often working, new main Namenode can read data to correspondence from distributed sha red memory system
Internal memory continues to provide the service of former main Namenode.
S201: described main Namenode receives data write request；
S202: the data content write in the data write request that described main Namenode will be successfully processed is described
In the memory block of distributed sha red memory system；
S203: when distributed sha red memory system detect currently stored piece write full data after, switch another
Memory block is for storing the data of described main Namenode write, and sends hand off signaling to described interim joint
In point from Namenode, described hand off signaling includes the position of the memory block writing full data being just switched
Information；
S204: corresponding, from Namenode according to the positional information described hand off signaling from described just quilt
The memory block writing full data of switching reads data content and is stored to the interior of corresponding described Namenode in advance
In depositing.
This data in data backup region that read in advance are primarily to when master/slave Namenode switches
The time is spent to synchronize on the data content that old main Namenode is preserved at new main Namenode less.
Certainly, even if doing so, due to the data content in the most stored full memory block that simply prestores, the most not
Synchronization of data content completely, if so current main Namenode cannot normally work and need by
The when of switching, newly determined main Namenode also needs to write currently storing former main Namenode
Enter in the memory block of data data content be saved in the internal memory of corresponding newly determined main Namenode
Can be only achieved the complete of the data content that newly determined main Namenode and former main Namenode preserve and
Cause property.Therefore, it is preferred that before the described new main Namenode of described use provides service, also include:
Whether the described new main Namenode described distributed sha red memory system of detection comprise not written
Full memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
Do so so that newly determined main Namenode is when providing service, the data content in internal memory
Identical with the content in old Namenode internal memory before, the feelings of any loss of data do not occur
Condition, has reached to synchronize completely with this.
As can be seen from the above-described embodiment, main Namenode is by the user side being successfully processed and server side
Data content write distributed sha red memory system in write request, distributed sha red memory system is with storage
The data of write are carried out the preservation of block-by-block by the form of block, say, that write the just switching of a full memory block another
One memory block continues the data of storage write, when switching memory block when, can in transient node its
He sends from Namenode and comprises the hand off signaling writing full memory block positional information being just switched, from
Namenode can thus the data content write in full memory block being just switched be stored in advance corresponding from
In the internal memory of Namenode, so make data content and the master preserved from Namenode internal memory
The data content basic synchronization preserved in Namenode internal memory, the most just data in one memory block of difference
Content, so in the case of main Namenode loses connection, cannot work, is confirmed as new master
The former of Namenode has only to read the data the memory block being currently used in storage data from Namenode
Content can complete data syn-chronization completely, the handover operation completing Namenode with this, the most significantly subtracts
Lack the time required for distributed system switching Namenode, improve user experience.
Embodiment three
Corresponding with the Namenode changing method of above-mentioned a kind of distributed system, the embodiment of the present invention also carries
Supply the Namenode switching device of a kind of distributed system.Referring to Fig. 3, it divides for one of the present invention
The apparatus structure schematic diagram of the Namenode switching device of cloth system, this device include dispensing unit 300,
Main Namenode judging unit 301 again and offer service unit 302.Work below in conjunction with this device is former
Reason is further described its internal structure and annexation.
Dispensing unit 300, is used for configuring at least two Namenode, is connected to by described Namenode
In one transient node and obtain Connecting quantity, described Namenode with default call back function according to described company
Connect parameter determine from described Namenode a main Namenode and at least one from Namenode,
Distributed system only uses described main Namenode to provide service；
Main Namenode judging unit 301 again, for disconnecting with transient node as main Namenode
Time, then judge that main Namenode cannot provide service, trigger in described transient node from Namenode
Perform described call back function and from Namenode, determine new main Namenode described；
There is provided service unit 302, for using described new main Namenode to provide service.
Preferably, the device shown in earlier figures 3 also includes network parameter storage unit 401 and triggers reading
Unit 402, as shown in Figure 4:
Network parameter storage unit 401, for when determining main Namenode, by described master
The network parameter of Namenode updates in default memory element；
Trigger and read unit 402, be used for triggering user side and server side from described memory element, read institute
State the network parameter of main Namenode so that described user side and server side are according to the described network read
Parameter carries out data interaction by described main Namenode.
Preferably, also include that distributed sha red memory system, described distributed sha red memory system are divided into many
Individual memory block carries out block-by-block storage to the data of write, and the device shown in earlier figures 3 also includes that reception is write asks
Ask unit 501, data write unit 502, memory block switch unit 503 and synchrodata content element 504,
As shown in Figure 5:
Receive write request unit 501, for receiving data write request for described main Namenode；
Data write unit 502, in the data write request being successfully processed for described main Namenode
Data content write described distributed sha red memory system memory block in；
When distributed sha red memory system, memory block switch unit 503, for for detecting that currently stored piece is write
After full data, switch another memory block for storing the data of described main Namenode write, and send and cut
Change signaling in described transient node from Namenode, writing that described hand off signaling includes just being switched is full
The positional information of the memory block of data；
Synchrodata content element 504, is used for being according to the position described hand off signaling from Namenode
Information reads data content from the described memory block writing full data being just switched and is stored in advance described in correspondence
In the internal memory of Namenode.
Preferably, the device shown in earlier figures 3 also includes memory block detector unit 601, as shown in Figure 6:
Memory block detector unit 601, before providing service at the described new main Namenode of described use,
It is not written full for whether the described new main Namenode described distributed sha red memory system of detection comprises
Memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
Preferably, also include:
Described Connecting quantity specially Namenode is connected to the Connection Time of transient node.
As can be seen from the above-described embodiment, on the one hand, for distributed system, at least two Namenode is set
And be connected in a transient node, one of them Namenode is described distributed as main Namenode
System provides service, when disconnecting with transient node based on some situation as main Namenode, and should
In transient node other be described distributed system arrange can be described from Namenode from Namenode
Middle determine new main Namenode come for described distributed system continue provide service so that described distributed
System cannot normally will not work because of the problem of Namenode, thus substantially increases distributed system
Stability.
On the other hand, while determine new main Namenode for distributed system, update and be used for storing
The memory element of main Namenode network parameter, and trigger the user side kimonos using above-mentioned distributed system
Business device side reads the net-work parameter information of newly determined main Namenode from described memory element, makes with this
Obtain user and server can use new main Namenode to carry out data interaction timely, thus will not
User or server is occurred still to be continuing with the most not providing the Namenode of service and cause cannot be complete
The problem becoming normal data interaction, further makes distributed system can complete to automatically switch
The function of Namenode.
On the other hand, main Namenode is by the number in the write request of the user side being successfully processed and server side
Writing distributed sha red memory system according to content, distributed sha red memory system will be write with the form of memory block
The data entered carry out the preservation of block-by-block, say, that write a full memory block and just switch another memory block and continue
Renew storage write data, when switch memory block when, can in transient node other from Namenode
Send and comprise the hand off signaling writing full memory block positional information being just switched, can be by from Namenode
The data content write in full memory block being just switched is stored to corresponding from the internal memory of Namenode by advance
In, so make the data content preserved from Namenode internal memory and main Namenode internal memory preserve
Data content basic synchronization, at most the data content in one memory block of difference, is so leading
Namenode loses connection, cannot work in the case of, be confirmed as new main Namenode former from
The data content that Namenode has only to read in the memory block being currently used in storage data can complete
Full data syn-chronization, the handover operation completing Namenode with this, thus substantially reduce distributed system
Time required for switching Namenode, improve user experience.
It should be noted that one of ordinary skill in the art will appreciate that and realize in above-described embodiment method
All or part of flow process, can be by computer program and completes to instruct relevant hardware, described
Program can be stored in a computer read/write memory medium, and this program is upon execution, it may include as above-mentioned
The flow process of the embodiment of each method.Wherein, described storage medium can be magnetic disc, CD, read-only storage
Memory body (Read-Only Memory, ROM) or random store-memory body (Random Access
Memory, RAM) etc..
Above the Namenode switching method and apparatus of a kind of distributed system provided by the present invention is entered
Having gone and be discussed in detail, principle and the embodiment of the present invention are carried out by specific embodiment used herein
Illustrating, the explanation of above example is only intended to help to understand method and the core concept thereof of the present invention；With
Time, for one of ordinary skill in the art, according to the thought of the present invention, in detailed description of the invention and should
All will change with in scope, in sum, this specification content should not be construed as the present invention's
Limit.
Claims (8)
1. the Namenode changing method of a distributed system, it is characterised in that configuration at least two
Namenode, is connected to described Namenode in a transient node and obtains Connecting quantity, described
Namenode determines one with default call back function according to described Connecting quantity from described Namenode
Main Namenode is with at least one from Namenode, and distributed system only uses described main Namenode
Service is provided, including:
When main Namenode disconnects with transient node, then judge that main Namenode cannot provide clothes
Business, trigger in described transient node from Namenode perform described call back function described from
Namenode determines new main Namenode；
Described new main Namenode is used to provide service；
Also include:
Described main Namenode receives data write request；
Data content write in the data write request that described main Namenode will be successfully processed is distributed common
Enjoying in the memory block of storage system, described distributed sha red memory system is divided into multiple memory block to write
Data carry out block-by-block storage；
When distributed sha red memory system detect currently stored piece write full data after, switch another memory block
For storing the data of described main Namenode write, and send hand off signaling in described transient node
From Namenode, described hand off signaling includes the positional information of the memory block writing full data being just switched；
Accordingly, just it has been switched from described according to positional information described hand off signaling from Namenode
Write in the memory block of full data reading data content and being stored to the internal memory of corresponding described Namenode in advance.
Method the most according to claim 1, it is characterised in that also include:
When determining main Namenode, the network parameter of described main Namenode is updated default
In memory element；
Triggering user side and server side read the network of described main Namenode from described memory element
Parameter so that described master is passed through according to the described network parameter read in described user side and server side
Namenode carries out data interaction.
Method the most according to claim 1, it is characterised in that the described new master of described use
Before Namenode provides service, also include:
Whether the described new main Namenode described distributed sha red memory system of detection comprise not written
Full memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
4. according to the method described in claims 1 to 3 any one, it is characterised in that also include:
Described Connecting quantity specially Namenode is connected to the Connection Time of transient node.
5. the Namenode switching device of a distributed system, it is characterised in that including:
Dispensing unit, is used for configuring at least two Namenode, described Namenode is connected to one and faces
In Shi Jiedian and obtain Connecting quantity, described Namenode connects ginseng with default call back function according to described
Number determines a main Namenode from described Namenode and at least one is from Namenode, distribution
Formula system only uses described main Namenode to provide service；
Main Namenode judging unit again, is used for when main Namenode disconnects with transient node,
Then judge that main Namenode cannot provide service, trigger performing from Namenode in described transient node
Described call back function determines new main Namenode described from Namenode；
There is provided service unit, for using described new main Namenode to provide service；
Also include:
Receive write request unit, for receiving data write request for described main Namenode；
Data write unit, the number in the data write request being successfully processed for described main Namenode
In memory block according to content write distributed sha red memory system, described distributed sha red memory system is divided into
Multiple memory blocks carry out block-by-block storage to the data of write；
When distributed sha red memory system, memory block switch unit, for for detecting that currently stored piece is write full
After data, switch another memory block for storing the data of described main Namenode write, and send switching
Signaling in described transient node from Namenode, described hand off signaling includes that be just switched writes full number
According to the positional information of memory block；
Synchrodata content element, is used for being according to the positional information described hand off signaling from Namenode
From the described memory block writing full data being just switched, read data content and be stored in advance described in correspondence
In the internal memory of Namenode.
Device the most according to claim 5, it is characterised in that also include:
Network parameter storage unit, for when determining main Namenode, by described main Namenode
Network parameter update in default memory element；
Trigger and read unit, be used for triggering user side and server side and read described from described memory element
The network parameter of main Namenode so that described user side and server side are according to the described network ginseng read
Number carries out data interaction by described main Namenode.
Device the most according to claim 5, it is characterised in that also include:
Memory block detector unit, before providing service at the described new main Namenode of described use, for
Whether the described new main Namenode described distributed sha red memory system of detection comprise not written full
Memory block；
If it has, then data content in described not written full memory block is read in the internal memory of correspondence.
8. according to the device described in claim 5 to 7 any one, it is characterised in that also include:
Described Connecting quantity specially Namenode is connected to the Connection Time of transient node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310462918.3A CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310462918.3A CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103491192A CN103491192A (en) | 2014-01-01 |
| CN103491192B true CN103491192B (en) | 2016-08-17 |
Family
ID=49831145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310462918.3A Active CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103491192B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105007172A (en) * | 2015-05-28 | 2015-10-28 | 杭州健港信息科技有限公司 | Method for realizing HDFS high-availability scheme |
| CN107168970A (en) * | 2016-03-07 | 2017-09-15 | 中兴通讯股份有限公司 | A kind of distributed file system HDFS management method, apparatus and system |
| CN107528710B (en) * | 2016-06-22 | 2021-08-20 | 华为技术有限公司 | Method, equipment and system for switching leader nodes of raft distributed system |
| CN106331132A (en) * | 2016-08-31 | 2017-01-11 | 天津南大通用数据技术股份有限公司 | Node replacement method and device in distributed database |
| CN108270718A (en) * | 2016-12-30 | 2018-07-10 | 北京观数科技有限公司 | A kind of control method and system based on Hadoop clusters |
| CN107612720B (en) * | 2017-08-29 | 2021-06-22 | 深圳市盛路物联通讯技术有限公司 | Access point backup method and gateway |
| CN108234602B (en) * | 2017-12-11 | 2021-02-09 | 武汉市烽视威科技有限公司 | MySQL multi-layer data synchronization method |
| CN109889561A (en) * | 2017-12-25 | 2019-06-14 | 新华三大数据技术有限公司 | A kind of data processing method and device |
| CN109302497A (en) * | 2018-11-29 | 2019-02-01 | 北京京东尚科信息技术有限公司 | Data processing method, access agent device and system based on HADOOP |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719831A (en) * | 2005-07-15 | 2006-01-11 | 清华大学 | High-available distributed boundary gateway protocol system based on cluster router structure |
| CN102752093A (en) * | 2012-06-29 | 2012-10-24 | 中国联合网络通信集团有限公司 | Distributed file system-based data processing method, device and system |
-
2013
- 2013-09-30 CN CN201310462918.3A patent/CN103491192B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719831A (en) * | 2005-07-15 | 2006-01-11 | 清华大学 | High-available distributed boundary gateway protocol system based on cluster router structure |
| CN102752093A (en) * | 2012-06-29 | 2012-10-24 | 中国联合网络通信集团有限公司 | Distributed file system-based data processing method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| Hadoop平台高可用性方案的设计与实现;杨帆;《中国优秀硕士学位论文全文数据库》;20120815;第18-20页、第36-37页、第43页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103491192A (en) | 2014-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103491192B (en) | The Namenode changing method of a kind of distributed system and system | |
| CN106331098B (en) | Server cluster system | |
| CN102148850B (en) | Cluster system and service processing method thereof | |
| CN103019875B (en) | The method of the two main transformation of a kind of fulfillment database and device | |
| CN104794028B (en) | A kind of disaster tolerance processing method, device, primary data center and preliminary data center | |
| CN110362381A (en) | HDFS cluster High Availabitity dispositions method, system, equipment and storage medium | |
| CN105933391A (en) | Node capacity expansion method, device and system | |
| CN105407117B (en) | The methods, devices and systems of distributed backup data | |
| CN104320401A (en) | Big data storage and access system and method based on distributed file system | |
| CN103647849A (en) | Method and device for migrating businesses and disaster recovery system | |
| CN110377459A (en) | A kind of disaster tolerance system, disaster tolerance processing method, monitoring node and backup cluster | |
| US20090187668A1 (en) | Protocol Independent Server Replacement and Replication in a Storage Area Network | |
| CN102932409B (en) | The method and system that a kind of virtual machine moves online | |
| CN105095317A (en) | Distributive database service management system | |
| CN104158693A (en) | A method and a system for disaster recovery backup of data service | |
| CN104301142A (en) | Backup method and device for configuration files | |
| CN105554074A (en) | NAS resource monitoring system and monitoring method based on RPC communication | |
| CN104917630A (en) | Method for recovering virtual network and system for recovering virtual network | |
| CN106874142B (en) | Real-time data fault-tolerant processing method and system | |
| CN105635311A (en) | Method for synchronizing resource pool information in cloud management platform | |
| CN104993940A (en) | Method and device for reducing data loss in main/standby node fault switching process | |
| CN105337780A (en) | Server node configuration method and physical nodes | |
| CN104468641B (en) | A kind of business datum moving method, device and cloud storage system | |
| CN104517067A (en) | Method, device and system for data access | |
| CN110535947A (en) | A kind of memory device set group configuration node switching method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 100086 12, 1201, 3 building, 2 South Road, Haidian District Academy of Sciences, Beijing. Patentee after: Beijing Sohu New Media Information Technology Co., Ltd. Address before: 100084 Beijing Haidian District Zhongguancun East Road 1 hospital 9 building Sohu cyber building 15 Floor Patentee before: Beijing Sohu New Media Information Technology Co., Ltd. |