CN103491192A - Namenode switching method and system of distributed system - Google Patents
Namenode switching method and system of distributed system Download PDFInfo
- Publication number
- CN103491192A CN103491192A CN201310462918.3A CN201310462918A CN103491192A CN 103491192 A CN103491192 A CN 103491192A CN 201310462918 A CN201310462918 A CN 201310462918A CN 103491192 A CN103491192 A CN 103491192A
- Authority
- CN
- China
- Prior art keywords
- namenode
- main
- data
- memory block
- main namenode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The embodiment of the invention discloses a Namenode switching method and device of a distributed system. According to the Namenode switching method and device of the distributed system, at least two Namenodes are configured, the Namenodes are connected to a temporary node and connecting parameters are obtained, a master Namenode and at least one slave Namenode are determined, only the master Namenode provides services for the distributed system, and the slave Namenode in the temporary node is triggered to execute a callback function when the master Namenode is disconnected with the temporary node and then a new master Namenode is used for providing services. According to the Namenode switching method and device of the distributed system, a new master Namenode can be determined from other set Namenodes to continue to provide services for the distributed system when the master Namenode of the distributed system does not provide services, then normal work of the distributed system is guaranteed, and stability of the distributed system is greatly improved.
Description
Technical field
The present invention relates to computer realm, particularly relate to a kind of Namenode changing method and system of distributed system.
Background technology
Along with needing data volume to be processed more and more on network, also more and more extensive for the range of application of the distributed system of processing mass data.Metadata node Namenode in distributed system is the core of the document storage system in whole distributed system, and all operations of distributed file storage system all need be participated in by Namenode.And Namenode is responsible for safeguarding metamessage and the directory information of All Files in whole distributed file system, if failure has appearred in Namenode, in distributed file storage system, All Files information will all be lost so.Although in general distributed system can be carried out the backup of many piece of data according to configuration for every piece of data, can not cause the data in system all to be lost because Namenode failure occurred, but whole distributed system still can can't be worked and cause providing normal service because of an only Namenode.
Summary of the invention
Cause providing the technical problem of normal service because an only Namenode can't work in order to solve above-mentioned distributed system, the invention provides a kind of Namenode changing method and system of distributed system.
The embodiment of the invention discloses following technical scheme:
A kind of Namenode changing method of distributed system, configure at least two Namenode, be connected in an interim node by described Namenode and obtain the connection parameter, described Namenode determines a main Namenode according to described connection parameter with the call back function of presetting from described Namenode and at least one is from Namenode, distributed system is only used described main Namenode that service is provided, and comprising:
When main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Use described new main Namenode that service is provided.
Preferably, also comprise:
When determining main Namenode, the network parameter of described main Namenode is updated in default memory cell;
Trigger user's side and server side and read the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
Preferably, also comprise the Distributed sharing storage system, described Distributed sharing storage system is divided into a plurality of memory blocks the data that write is carried out to the block-by-block storage:
Described main Namenode receives the data write request;
Data content in the data write request that described main Namenode processes success writes in the memory block of described Distributed sharing storage system;
After the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
Accordingly, from Namenode in the internal memory according to reading out data content the described Namenode of correspondence that prestores from the described memory block of writing full data just be switched of the positional information described hand off signaling.
Preferably, before the described new main Namenode of described use provides service, also comprise:
Described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
Preferably, also comprise:
Described connection parameter is specially the connect hours that Namenode is connected to interim node.
A kind of Namenode switching device shifter of distributed system comprises:
Dispensing unit, for at least two Namenode of configuration, be connected in an interim node by described Namenode and obtain the connection parameter, described Namenode determines from described Namenode according to described connection parameter that with the call back function preset a main Namenode and at least one are from Namenode, and distributed system is only used described main Namenode that service is provided;
Main Namenode is judging unit again, for when main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Service unit is provided, for using described new main Namenode, provides service.
Preferably, also comprise:
The network parameter storage unit, for when determining main Namenode, be updated to the network parameter of described main Namenode in default memory cell;
Trigger reading unit, for triggering user's side and server side reads the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
Preferably, also comprise the Distributed sharing storage system, described Distributed sharing storage system is divided into a plurality of memory blocks the data that write is carried out to the block-by-block storage:
Receive the write request unit, be used to described main Namenode to receive the data write request;
Data write unit, the data content in the data write request that is used to described main Namenode that success is processed writes in the memory block of described Distributed sharing storage system;
The memory block switch unit, be used to after the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
The synchrodata content element, be used to from Namenode in the internal memory according to reading out data content the described Namenode of correspondence that prestores from the described memory block of writing full data just be switched of the positional information described hand off signaling.
Preferably, also comprise:
The memory block detecting unit, for before the described new main Namenode of described use provides service, for described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
Preferably, also comprise:
Described connection parameter is specially the connect hours that Namenode is connected to interim node.
By technique scheme, can be found out, for at least two Namenode of distributed system setting, when as main Namenode, based on some situation, not providing service, can determine from other Namenode that arrange that immediately the main Namenode made new advances continues to provide service for distributed system, make distributed system can can't not work because of the problem of Namenode, greatly improved the stability of distributed system.
The accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, below will the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
The method flow schematic diagram of the Namenode changing method that Fig. 1 is a kind of distributed system of the present invention;
Another method flow schematic diagram of the Namenode changing method that Fig. 2 is a kind of distributed system of the present invention;
The apparatus structure schematic diagram of the Namenode switching device shifter that Fig. 3 is a kind of distributed system of the present invention;
Another apparatus structure schematic diagram of the Namenode switching device shifter that Fig. 4 is a kind of distributed system of the present invention;
Another apparatus structure schematic diagram of the Namenode switching device shifter that Fig. 5 is a kind of distributed system of the present invention;
Another apparatus structure schematic diagram of the Namenode switching device shifter that Fig. 6 is a kind of distributed system of the present invention.
Embodiment
The embodiment of the present invention provides a kind of Namenode changing method and system of distributed system.On the one hand, for at least two Namenode of distributed system setting and be connected in an interim node, one of them Namenode provides service as main Namenode for described distributed system, when disconnecting with interim node based on some situation as main Namenode, what in this interim node, other were described distributed system setting can determine that from Namenode new main Namenode continues to provide service for described distributed system described from Namenode, make described distributed system can can't not work because of the problem of Namenode, greatly improved thus the stability of distributed system.
On the other hand, determining new main Namenode for distributed system when, upgrade the memory cell for storing main Namenode network parameter, and user's side of above-mentioned distributed system and the network parameter information that server side reads the new main Namenode determined from described memory cell are used in triggering, with this, make user and server can use timely new main Namenode to carry out data interaction, do not provide the Namenode of service and cause the problem that can't complete normal data interaction with regard to not there will be user or server still to continue use like this, further make distributed system can complete the function of automatic switchover Namenode.
On the other hand, data content in user's side that main Namenode processes success and the write request of server side writes the Distributed sharing storage system, the Distributed sharing storage system is carried out the data that write the preservation of block-by-block with the form of memory block, that is to say that writing a full memory block just switches the data that another memory block continuation storage writes, in the time of the switching memory block, can be in interim node, other send and comprise the hand off signaling of writing full memory block positional information just be switched from Namenode, can thus the data content of writing full memory block just be switched be prestored to correspondence from the internal memory of Namenode from Namenode, make like this data content basic synchronization of preserving in the data content preserved and main Namenode internal memory from the Namenode internal memory, just differ at most a data content in memory block, lose connection at main Namenode like this, in situation about can't work, what be confirmed as new main Namenode formerly only need to read the current data content for the memory block of storing data from Namenode and can complete that data are synchronous completely, complete the handover operation of Namenode with this, greatly reduced thus the distributed system switching needed time of Namenode, improved user experience.
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, below in conjunction with accompanying drawing, the embodiment of the present invention is described in detail.
Embodiment mono-
Refer to Fig. 1, the method flow schematic diagram of its Namenode changing method that is a kind of distributed system of the present invention, the method comprises the following steps:
Configuration step S100: configure at least two Namenode, be connected in an interim node by described Namenode and obtain the connection parameter, described Namenode determines from described Namenode according to described connection parameter that with the call back function preset a main Namenode and at least one are from Namenode, and distributed system is only used described main Namenode that service is provided.
That is to say, be that a distributed system configures a plurality of Namenode, but synchronization is only used one as main Namenode, to provide service, and other Namenode, as from Namenode, that is to say and at least also will configure two Namenode.Only main Namenode can't use or other situations under, determine one from least one and be used as new main Namenode from Namenode, making distributed system can in the Namenode needed only as its configuration, have one can also provide service just can work.The preferred arrangement of the present embodiment is that at least two Namenode that are a distributed system configuration all are connected on same interim node, and determine one provide service as main Namenode for distributed system from the Namenode for this distributed system configuration according to being connected to connection parameter on interim node by call back function, remaining is as the alternative Namenode as this distributed system from Namenode, and it should be noted that, connect Namenode in same interim node and be the Namenode that same distributed system configures, and preferred, to use the reliable coherent system zookeeper for large-scale distributed system to set up the connection of interim node for Namenode.
A kind of preferred connection parameter is the time that Namenode is connected to interim node, like this, call back function is usingd the connect hours as variable, while needing to determine main Namenode at every turn, all can trigger each and carry out call back function from Namenode, and whether determine according to the size order of connect hours oneself be current connect hours minimum, if, determine oneself to be new main Namenode, and carry out the operation of follow-up relevant master and slave Namenode conversion.
S101: when main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Here it should be noted that, main Namenode may be mainly because main Namenode broken string, damage or roll off the production line etc. and can't normally provide the situation of service to cause with disconnecting being connected of interim node.In this case except judging that former main Namenode can't provide service, also need to determine that the former main Namenode of main Namenode replacement made new advances continues as distributed system due service is provided, and the scheme of determining the main Namenode made new advances is exactly after interim node finds that the Namenode quantity connected changes, other Namenode that triggering is connected on interim node carry out call back function, by connecting parameter, determine the main Namenode made new advances.
Preferably, also comprise:
When determining main Namenode, the network parameter of described main Namenode is updated in default memory cell;
Trigger user's side and server side and read the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
That is to say, after determining a main Namenode, capital is by the network parameter of this Namenode such as the renewal such as IP address being saved in a memory cell set in advance, and, when need to determining main Namenode at every turn, other Namenode that connect in triggering interim node carry out call back function, also can trigger each user of user's side of using this distributed system and each server of server side and read the data message in described memory cell, like this, at every turn more during change owner Namenode, the user's side and the server side that use main Namenode to carry out data interaction can obtain the network parameter of up-to-date main Namenode within the very first time from memory cell, and set up data interaction according to this network parameter with new main Namenode and be connected, while after like this just not there will be old main Namenode to stop service or flash occurred, by new main Namenode, replacing service, user's side and/or server side complete data interaction with other Namenode owing to not upgrading in time and have served, still use old main Namenode to carry out data interaction, to such an extent as to the problem of complete operation smoothly.And the network parameter of the main Namenode of this user's side of trigger using distributed system in the master/slave Namenode of switching and server side renewal distribution formula system, making thus not only can be by manual switchover for the master/slave Namenode switching of distributed system, also can directly pass through the system automatic decision, complete automatic switching.
S102: use described new main Namenode that service is provided.
After the associative operation that completes master and slave Namenode conversion, distributed system can be used new main Namenode to be served, and the associative operation of concrete master and slave Namenode conversion will be described in embodiment bis-.
By the present embodiment, can be found out, for at least two Namenode of distributed system setting and be connected in an interim node, one of them Namenode provides service as main Namenode for described distributed system, when disconnecting with interim node based on some situation as main Namenode, what in this interim node, other were described distributed system setting can determine that from Namenode new main Namenode continues to provide service for described distributed system described from Namenode, make described distributed system can can't not work because of the problem of Namenode, greatly improved thus the stability of distributed system.
On the other hand, determining new main Namenode for distributed system when, upgrade the memory cell for storing main Namenode network parameter, and user's side of above-mentioned distributed system and the network parameter information that server side reads the new main Namenode determined from described memory cell are used in triggering, with this, make user and server can use timely new main Namenode to carry out data interaction, do not provide the Namenode of service and cause the problem that can't complete normal data interaction with regard to not there will be user or server still to continue use like this, further make distributed system can complete the function of automatic switchover Namenode.
Embodiment bis-
On the basis of embodiment mono-, the present embodiment will mainly for determine one from Namenode as new main Namenode, how will be somebody's turn to do the associative operation that converts main Namenode to from Namenode is described, refer to Fig. 2, another method flow schematic diagram of its Namenode changing method that is a kind of distributed system of the present invention, also comprise the Distributed sharing storage system, described Distributed sharing storage system is divided into a plurality of memory blocks the data that write is carried out to the block-by-block storage, comprising:
Here it should be noted that, the Distributed sharing storage system is a kind of data-storage system of piecemeal memory-type, such as provide the Distributed sharing storage system bookkeeper of log stream for distributed system, its Main Function is a kind of backup to data interaction in distributed system, when main Namenode is served, such as receive the data write request, (be generally that the data content on server is carried out to the data deletion, increase and modification etc., all can be considered a kind of write operation to data) time, data content in the data write request that main Namenode not only can process success is saved in corresponding internal memory, also can be written in the Distributed sharing storage system, main Namenode can guarantee like this when can't work, new main Namenode can be from the Distributed sharing storage system reading out data to the service that continues to provide former main Namenode in corresponding internal memory.
S201: described main Namenode receives the data write request;
S202: the data content in the data write request that described main Namenode processes success writes in the memory block of described Distributed sharing storage system;
S203: after the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
S204: corresponding, in the internal memory from Namenode according to reading out data content the described Namenode of correspondence that prestores from the described memory block of writing full data just be switched of the positional information described hand off signaling.
It is mainly on the data content that spended time is preserved at the synchronous old main Namenode of new main Namenode less when master/slave Namenode switches that this reading out data in advance backs up regional data.Certainly, even so do, store the data content in full memory block owing to just prestoring, it not also synchronization of data content completely, when need to be switched so if current main Namenode can't work, the new main Namenode determined also need by current memory block of storing former main Namenode data writing data content be saved in the internal memory of the corresponding new main Namenode determined the complete and consistency that just can reach the new main Namenode determined and the data content of former main Namenode preservation.Therefore, preferred, before the described new main Namenode of described use provides service, also comprise:
Described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
Doing like this can be so that the new main Namenode determined be when providing service, and the content in the data content in internal memory and old Namenode internal memory before is identical, the situation of any loss of data do not occur, with this, has reached synchronous completely.
As can be seen from the above-described embodiment, data content in user's side that main Namenode processes success and the write request of server side writes the Distributed sharing storage system, the Distributed sharing storage system is carried out the data that write the preservation of block-by-block with the form of memory block, that is to say that writing a full memory block just switches the data that another memory block continuation storage writes, in the time of the switching memory block, can be in interim node, other send and comprise the hand off signaling of writing full memory block positional information just be switched from Namenode, can thus the data content of writing full memory block just be switched be prestored to correspondence from the internal memory of Namenode from Namenode, make like this data content basic synchronization of preserving in the data content preserved and main Namenode internal memory from the Namenode internal memory, just differ at most a data content in memory block, lose connection at main Namenode like this, in situation about can't work, what be confirmed as new main Namenode formerly only need to read the current data content for the memory block of storing data from Namenode and can complete that data are synchronous completely, complete the handover operation of Namenode with this, greatly reduced thus the distributed system switching needed time of Namenode, improved user experience.
Embodiment tri-
Corresponding with the Namenode changing method of above-mentioned a kind of distributed system, the embodiment of the present invention also provides a kind of Namenode switching device shifter of distributed system.Refer to Fig. 3, its apparatus structure schematic diagram that is the Namenode switching device shifter of a kind of distributed system of the present invention, this device comprises dispensing unit 300, main Namenode judging unit 301 and service unit 302 is provided again.Operation principle below in conjunction with this device is further introduced its internal structure and annexation.
Main Namenode is judging unit 301 again, for when main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Preferably, the device shown in earlier figures 3 also comprises network parameter storage unit 401 and triggering reading unit 402, as shown in Figure 4:
Network parameter storage unit 401, for when determining main Namenode, be updated to the network parameter of described main Namenode in default memory cell;
Trigger reading unit 402, for triggering user's side and server side reads the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
Preferably, also comprise the Distributed sharing storage system, described Distributed sharing storage system is divided into a plurality of memory blocks the data that write is carried out to the block-by-block storage, device shown in earlier figures 3 also comprises reception write request unit 501, data write unit 502, memory block switch unit 503 and synchrodata content element 504, as shown in Figure 5:
Receive write request unit 501, be used to described main Namenode to receive the data write request;
Data write unit 502, the data content in the data write request that is used to described main Namenode that success is processed writes in the memory block of described Distributed sharing storage system;
Memory block switch unit 503, be used to after the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
Preferably, the device shown in earlier figures 3 also comprises memory block detecting unit 601, as shown in Figure 6:
Memory block detecting unit 601, for before the described new main Namenode of described use provides service, for described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
Preferably, also comprise:
Described connection parameter is specially the connect hours that Namenode is connected to interim node.
As can be seen from the above-described embodiment, on the one hand, for at least two Namenode of distributed system setting and be connected in an interim node, one of them Namenode provides service as main Namenode for described distributed system, when disconnecting with interim node based on some situation as main Namenode, what in this interim node, other were described distributed system setting can determine that from Namenode new main Namenode continues to provide service for described distributed system described from Namenode, make described distributed system can can't not work because of the problem of Namenode, greatly improved thus the stability of distributed system.
On the other hand, determining new main Namenode for distributed system when, upgrade the memory cell for storing main Namenode network parameter, and user's side of above-mentioned distributed system and the network parameter information that server side reads the new main Namenode determined from described memory cell are used in triggering, with this, make user and server can use timely new main Namenode to carry out data interaction, do not provide the Namenode of service and cause the problem that can't complete normal data interaction with regard to not there will be user or server still to continue use like this, further make distributed system can complete the function of automatic switchover Namenode.
On the other hand, data content in user's side that main Namenode processes success and the write request of server side writes the Distributed sharing storage system, the Distributed sharing storage system is carried out the data that write the preservation of block-by-block with the form of memory block, that is to say that writing a full memory block just switches the data that another memory block continuation storage writes, in the time of the switching memory block, can be in interim node, other send and comprise the hand off signaling of writing full memory block positional information just be switched from Namenode, can thus the data content of writing full memory block just be switched be prestored to correspondence from the internal memory of Namenode from Namenode, make like this data content basic synchronization of preserving in the data content preserved and main Namenode internal memory from the Namenode internal memory, just differ at most a data content in memory block, lose connection at main Namenode like this, in situation about can't work, what be confirmed as new main Namenode formerly only need to read the current data content for the memory block of storing data from Namenode and can complete that data are synchronous completely, complete the handover operation of Namenode with this, greatly reduced thus the distributed system switching needed time of Namenode, improved user experience.
It should be noted that, one of ordinary skill in the art will appreciate that all or part of flow process realized in above-described embodiment method, to come the hardware that instruction is relevant to complete by computer program, described program can be stored in a computer read/write memory medium, this program, when carrying out, can comprise the flow process as the embodiment of above-mentioned each side method.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-Only Memory, ROM) or random store-memory body (Random Access Memory, RAM) etc.
The above switching method and apparatus of the Namenode to a kind of distributed system provided by the present invention is described in detail, applied specific embodiment herein principle of the present invention and execution mode are set forth, the explanation of above embodiment is just for helping to understand method of the present invention and core concept thereof; , for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention simultaneously.
Claims (10)
1. the Namenode changing method of a distributed system, it is characterized in that, configure at least two Namenode, be connected in an interim node by described Namenode and obtain the connection parameter, described Namenode determines a main Namenode according to described connection parameter with the call back function of presetting from described Namenode and at least one is from Namenode, distributed system is only used described main Namenode that service is provided, and comprising:
When main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Use described new main Namenode that service is provided.
2. method according to claim 1, is characterized in that, also comprises:
When determining main Namenode, the network parameter of described main Namenode is updated in default memory cell;
Trigger user's side and server side and read the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
3. method according to claim 1, is characterized in that, also comprises the Distributed sharing storage system, and described Distributed sharing storage system is divided into a plurality of memory blocks the data that write are carried out to the block-by-block storage:
Described main Namenode receives the data write request;
Data content in the data write request that described main Namenode processes success writes in the memory block of described Distributed sharing storage system;
After the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
Accordingly, from Namenode in the internal memory according to reading out data content the described Namenode of correspondence that prestores from the described memory block of writing full data just be switched of the positional information described hand off signaling.
4. method according to claim 3, is characterized in that, before the described new main Namenode of described use provides service, also comprises:
Described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
5. according to the described method of claim 1 to 4 any one, it is characterized in that, also comprise:
Described connection parameter is specially the connect hours that Namenode is connected to interim node.
6. the Namenode switching device shifter of a distributed system, is characterized in that, comprising:
Dispensing unit, for at least two Namenode of configuration, be connected in an interim node by described Namenode and obtain the connection parameter, described Namenode determines from described Namenode according to described connection parameter that with the call back function preset a main Namenode and at least one are from Namenode, and distributed system is only used described main Namenode that service is provided;
Main Namenode is judging unit again, for when main Namenode disconnects with interim node, judge that main Namenode can't provide service, trigger carrying out described call back function from Namenode and determine new main Namenode described from Namenode in described interim node;
Service unit is provided, for using described new main Namenode, provides service.
7. device according to claim 6, is characterized in that, also comprises:
The network parameter storage unit, for when determining main Namenode, be updated to the network parameter of described main Namenode in default memory cell;
Trigger reading unit, for triggering user's side and server side reads the network parameter of described main Namenode from described memory cell, make described user's side and server lateral root carry out data interaction according to the described network parameter read by described main Namenode.
8. device according to claim 6, is characterized in that, also comprises the Distributed sharing storage system, and described Distributed sharing storage system is divided into a plurality of memory blocks the data that write are carried out to the block-by-block storage:
Receive the write request unit, be used to described main Namenode to receive the data write request;
Data write unit, the data content in the data write request that is used to described main Namenode that success is processed writes in the memory block of described Distributed sharing storage system;
The memory block switch unit, be used to after the Distributed sharing storage system detects current memory block and writes full data, switch the data that another memory block writes for storing described main Namenode, and send hand off signaling in described interim node from Namenode, described hand off signaling comprises the positional information of the memory block of writing full data just be switched;
The synchrodata content element, be used to from Namenode in the internal memory according to reading out data content the described Namenode of correspondence that prestores from the described memory block of writing full data just be switched of the positional information described hand off signaling.
9. device according to claim 8, is characterized in that, also comprises:
The memory block detecting unit, for before the described new main Namenode of described use provides service, for described new main Namenode detects in described Distributed sharing storage system whether comprise the memory block be not fully written;
If have, data content in the described memory block be not fully written is read in corresponding internal memory.
10. according to the described device of claim 6 to 9 any one, it is characterized in that, also comprise:
Described connection parameter is specially the connect hours that Namenode is connected to interim node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310462918.3A CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310462918.3A CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103491192A true CN103491192A (en) | 2014-01-01 |
| CN103491192B CN103491192B (en) | 2016-08-17 |
Family
ID=49831145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310462918.3A Active CN103491192B (en) | 2013-09-30 | 2013-09-30 | The Namenode changing method of a kind of distributed system and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103491192B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105007172A (en) * | 2015-05-28 | 2015-10-28 | 杭州健港信息科技有限公司 | Method for realizing HDFS high-availability scheme |
| CN106331132A (en) * | 2016-08-31 | 2017-01-11 | 天津南大通用数据技术股份有限公司 | Node replacement method and device in distributed database |
| WO2017152736A1 (en) * | 2016-03-07 | 2017-09-14 | 中兴通讯股份有限公司 | Hadoop distributed file system hdfs management method, device and system |
| CN107528710A (en) * | 2016-06-22 | 2017-12-29 | 华为技术有限公司 | Switching method, equipment and the system of raft distributed system leader nodes |
| CN107612720A (en) * | 2017-08-29 | 2018-01-19 | 深圳市盛路物联通讯技术有限公司 | Access point backup method and gateway |
| CN108234602A (en) * | 2017-12-11 | 2018-06-29 | 武汉市烽视威科技有限公司 | The multi-layer data synchronous method of MySQL |
| CN108270718A (en) * | 2016-12-30 | 2018-07-10 | 北京观数科技有限公司 | A kind of control method and system based on Hadoop clusters |
| CN109302497A (en) * | 2018-11-29 | 2019-02-01 | 北京京东尚科信息技术有限公司 | Data processing method, access agent device and system based on HADOOP |
| CN109889561A (en) * | 2017-12-25 | 2019-06-14 | 新华三大数据技术有限公司 | A kind of data processing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719831A (en) * | 2005-07-15 | 2006-01-11 | 清华大学 | High-available distributed boundary gateway protocol system based on cluster router structure |
| CN102752093A (en) * | 2012-06-29 | 2012-10-24 | 中国联合网络通信集团有限公司 | Distributed file system-based data processing method, device and system |
-
2013
- 2013-09-30 CN CN201310462918.3A patent/CN103491192B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719831A (en) * | 2005-07-15 | 2006-01-11 | 清华大学 | High-available distributed boundary gateway protocol system based on cluster router structure |
| CN102752093A (en) * | 2012-06-29 | 2012-10-24 | 中国联合网络通信集团有限公司 | Distributed file system-based data processing method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 杨帆: "Hadoop平台高可用性方案的设计与实现", 《中国优秀硕士学位论文全文数据库》, 15 August 2012 (2012-08-15) * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105007172A (en) * | 2015-05-28 | 2015-10-28 | 杭州健港信息科技有限公司 | Method for realizing HDFS high-availability scheme |
| WO2017152736A1 (en) * | 2016-03-07 | 2017-09-14 | 中兴通讯股份有限公司 | Hadoop distributed file system hdfs management method, device and system |
| CN107168970A (en) * | 2016-03-07 | 2017-09-15 | 中兴通讯股份有限公司 | A kind of distributed file system HDFS management method, apparatus and system |
| CN107528710A (en) * | 2016-06-22 | 2017-12-29 | 华为技术有限公司 | Switching method, equipment and the system of raft distributed system leader nodes |
| CN107528710B (en) * | 2016-06-22 | 2021-08-20 | 华为技术有限公司 | Method, equipment and system for switching leader nodes of raft distributed system |
| CN106331132A (en) * | 2016-08-31 | 2017-01-11 | 天津南大通用数据技术股份有限公司 | Node replacement method and device in distributed database |
| CN108270718A (en) * | 2016-12-30 | 2018-07-10 | 北京观数科技有限公司 | A kind of control method and system based on Hadoop clusters |
| CN107612720A (en) * | 2017-08-29 | 2018-01-19 | 深圳市盛路物联通讯技术有限公司 | Access point backup method and gateway |
| CN107612720B (en) * | 2017-08-29 | 2021-06-22 | 深圳市盛路物联通讯技术有限公司 | Access point backup method and gateway |
| CN108234602A (en) * | 2017-12-11 | 2018-06-29 | 武汉市烽视威科技有限公司 | The multi-layer data synchronous method of MySQL |
| CN109889561A (en) * | 2017-12-25 | 2019-06-14 | 新华三大数据技术有限公司 | A kind of data processing method and device |
| CN109302497A (en) * | 2018-11-29 | 2019-02-01 | 北京京东尚科信息技术有限公司 | Data processing method, access agent device and system based on HADOOP |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103491192B (en) | 2016-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103491192A (en) | Namenode switching method and system of distributed system | |
| CN202798798U (en) | High availability system based on cloud computing technology | |
| CN101557315B (en) | Method, device and system for active-standby switch | |
| CN102981931B (en) | Backup method and device for virtual machine | |
| CN102346779B (en) | Distributed file system and master control node backup method | |
| CN102752093B (en) | Based on the data processing method of distributed file system, equipment and system | |
| CN102411639B (en) | Multi-copy storage management method and system of metadata | |
| CN101729595A (en) | Network backup device and backup method thereof | |
| CN104320401A (en) | Big data storage and access system and method based on distributed file system | |
| CN104714811A (en) | Method and device for manufacturing difference upgrade package and system difference upgrade method and device | |
| CN104158693A (en) | A method and a system for disaster recovery backup of data service | |
| CN102681917A (en) | Operating system (OS) and recovery method thereof | |
| CN111045708B (en) | Software upgrading method, electronic device and computer readable storage medium | |
| CN103167026A (en) | Processing method, system and device for cloud storage environmental data | |
| CN109634638B (en) | Cluster software upgrading method, device, equipment and medium | |
| CN101639785A (en) | Method and device for software updating | |
| CN106874142B (en) | Real-time data fault-tolerant processing method and system | |
| CN102497288A (en) | Dual-server backup method and dual system implementation device | |
| WO2015135370A1 (en) | Data update method and system | |
| CN103369660A (en) | Network-element data synchronization method and network-element device | |
| CN104040512A (en) | Method and device for processing storage space and non-volatile computer readable storage medium | |
| CN104468641B (en) | A kind of business datum moving method, device and cloud storage system | |
| CN110535947A (en) | A kind of memory device set group configuration node switching method, device and equipment | |
| CN102520956A (en) | Novel data storage method and system | |
| CN108984635A (en) | A kind of HDFS storage system and date storage method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 100086 12, 1201, 3 building, 2 South Road, Haidian District Academy of Sciences, Beijing. Patentee after: Beijing Sohu New Media Information Technology Co., Ltd. Address before: 100084 Beijing Haidian District Zhongguancun East Road 1 hospital 9 building Sohu cyber building 15 Floor Patentee before: Beijing Sohu New Media Information Technology Co., Ltd. |