CN103019626A - Storage system, method and device for controlling cluster metadata - Google Patents
Storage system, method and device for controlling cluster metadata Download PDFInfo
- Publication number
- CN103019626A CN103019626A CN2012105478769A CN201210547876A CN103019626A CN 103019626 A CN103019626 A CN 103019626A CN 2012105478769 A CN2012105478769 A CN 2012105478769A CN 201210547876 A CN201210547876 A CN 201210547876A CN 103019626 A CN103019626 A CN 103019626A
- Authority
- CN
- China
- Prior art keywords
- metadata
- cluster
- memory node
- cluster metadata
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a storage system, a method and a device for controlling cluster metadata, wherein the method comprises the following steps: dividing the cluster metadata according to the relevance of the cluster metadata and each storage node to obtain the relevant metadata of each storage node; each storage node is a storage node in a cluster to which the cluster metadata belongs; sending the related metadata to related storage nodes in the cluster; when the cluster metadata can not be obtained, obtaining the issued related metadata from each storage node in the cluster; and constructing the acquired related metadata into cluster metadata. The cluster metadata are divided and sent to each storage node for storage, so that the main metadata server and the standby metadata server have faults, and the cluster metadata can be constructed and obtained by acquiring the metadata from each storage node under the condition that the cluster metadata are lost, so that the cluster metadata are recovered, and the problem that the cluster metadata are lost and cannot be recovered due to the faults of the main metadata server and the standby metadata server in the prior art is solved.
Description
Technical field
The present invention relates to memory technology, relate in particular to method and the device of a kind of storage system, control cluster metadata.
Background technology
In the storage system, the data that recorded the reflection relation of mathematical logic and physical location are the cluster metadata.Wherein, cluster refers to the memory device cluster in the storage system.Comprised the positional information of storage of subscriber data etc. in the cluster metadata, if lost the cluster metadata, can't find the memory location of user data, thereby will finally cause losing of user data.
Usually, the cluster metadata store is in meta data server, and in order to improve the reliability of cluster metadata store, meta data server generally has two, and one is the pivot data server, and one is standby meta data server.Wherein, the pivot data server is when storage system is worked, for client provides the cluster metadata, so that the cluster metadata that client provides according to the pivot data server reads the user data of storage.Standby meta data server is used for the cluster metadata is backed up, and when pivot data server cisco unity malfunction, the standby alternative pivot data server of meta data server provides the cluster metadata for client.
But, when the pivot data server with when being the standby equal cisco unity malfunction of meta data server, will cause whole storage system to work.
Summary of the invention
In view of this, the embodiment of the invention provides method and the device of a kind of storage system, control cluster metadata, after losing for the existing storage system cluster metadata of solution, and the problem that can't automatically recover.
First aspect, the embodiment of the invention provide a kind of method of controlling the cluster metadata, comprising:
The cluster metadata according to dividing with the correlativity of each memory node, is obtained the associated metadata of described each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata;
Described associated metadata is issued to memory node relevant in the described cluster;
When described cluster metadata can't be obtained, each memory node in described cluster obtained the associated metadata that issues;
The associated metadata that obtains is built into described cluster metadata.
Second aspect, the embodiment of the invention provide a kind of method for control cluster metadata, comprising:
The associated metadata that reception and storing metadata controller issue, described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node;
Described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
The 3rd aspect, the embodiment of the invention provide a kind of method for control cluster metadata, comprising:
Receive the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller;
Preserve the described cluster metadata that receives.
The 4th aspect, the embodiment of the invention provides a kind of device of controlling the cluster metadata, comprising:
The first division unit is used for the cluster metadata obtaining the associated metadata of described each memory node according to dividing with the correlativity of each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata;
First issues the unit, is used for described associated metadata is issued to memory node relevant in the described cluster;
The first acquiring unit is used for when described cluster metadata can't be obtained, and each memory node in described cluster obtains the associated metadata that issues;
The first construction unit is used for the associated metadata that obtains is built into described cluster metadata.
The 5th aspect, the embodiment of the invention provides a kind of memory node equipment, comprising:
The metadata receiving element is used for the associated metadata that reception and storing metadata controller issue, and described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node;
The metadata transmitting element is used for described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
The 6th aspect, the embodiment of the invention provides a kind of meta data server, comprising:
The first receiving element, be used for receiving the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller;
Storage unit is used for preserving the described cluster metadata that described cluster metadata receiving element receives.
The 7th aspect, the embodiment of the invention provides a kind of storage system, comprises the device of above-mentioned control cluster metadata, above-mentioned memory node equipment and above-mentioned meta data server.
The storage system that the embodiment of the invention provides, method and the device of control cluster metadata, by being divided and be issued to each memory node, the cluster metadata preserves, so that pivot data server and the standby equal fault of meta data server, lost in the situation of cluster metadata, still can be by obtaining metadata from each memory node, structure obtains the cluster metadata, thereby recover the cluster metadata, solve pivot data server and standby meta data server equal fault in the prior art and caused the cluster metadata to be lost and the problem that can't recover, improved the reliability of storage system.
Description of drawings
A kind of process flow diagram of controlling the method for cluster metadata that Fig. 1 provides for the embodiment of the invention;
The process flow diagram of a kind of method for control cluster metadata that Fig. 2 provides for the embodiment of the invention;
The another kind that Fig. 3 provides for the embodiment of the invention is used for the process flow diagram of the method for control cluster metadata;
Fig. 4 uses schematic diagram for the method for the control cluster metadata that the embodiment of the invention provides;
MDC rebuilds the process flow diagram of the cluster metadata of whole storage system in the method for the control cluster metadata that Fig. 5 provides for the embodiment of the invention;
A kind of structural representation of controlling the device of cluster metadata that Fig. 6 provides for the embodiment of the invention;
The structural representation of a kind of memory node equipment that Fig. 7 provides for the embodiment of the invention;
The structural representation of a kind of meta data server that Fig. 8 provides for the embodiment of the invention;
The structural representation of a kind of storage system that Fig. 9 provides for the embodiment of the invention.
Embodiment
A kind of process flow diagram of controlling the method for cluster metadata that Fig. 1 provides for the embodiment of the invention.The method that the present embodiment provides can be carried out by the master control borad of storage system under the cluster metadata, for example can increase a control module at master control borad, the method that provides to carry out the present embodiment.As shown in Figure 1, the method comprises:
Step 11, with the cluster metadata according to dividing with the correlativity of each memory node, obtain the associated metadata of this each memory node; This each memory node is the memory node in the cluster under the described cluster metadata.
For example, the device clusters of some storage systems comprises memory node A, B, C, D and E, and so, the cluster metadata just can be divided into 5 associated metadata parts: associated metadata a, b, c, d and e.Wherein, associated metadata a is the metadata relevant with memory node A, associated metadata b is the metadata relevant with memory node B, associated metadata c is the metadata relevant with memory node C, associated metadata d is the metadata relevant with memory node D, associated metadata d is the metadata relevant with memory node D, and associated metadata e is the metadata relevant with memory node E.
Step 12, above-mentioned associated metadata is issued in the above-mentioned cluster relevant memory node.
For example, associated metadata a is issued to memory node A, associated metadata b is issued to memory node B, associated metadata c is issued to memory node C, associated metadata d is issued to memory node D, associated metadata d is issued to memory node D, associated metadata e is issued to memory node E.
Step 13, when above-mentioned cluster metadata can't be obtained, each memory node in above-mentioned cluster obtains the associated metadata that issues.
For example, when existing meta data server fault, the user just can't obtain the cluster metadata.In this case, can obtain associated metadata from each memory node, to recover the cluster metadata.Still with above-mentioned memory node A, B, C, D and E, and associated metadata a, b, c, d and e are example, when the cluster metadata can't be obtained, can obtain associated metadata a from memory node A, obtain associated metadata b from memory node B, obtain associated metadata c from memory node C, obtain associated metadata d from memory node D, obtain associated metadata e from memory node E.
Step 14, the associated metadata that obtains is built into above-mentioned cluster metadata.
For example, associated metadata a, b, c, d and the e that obtains merged reconstruct, obtain the cluster metadata in the above-mentioned steps 11.
Alternatively, the method for the control cluster metadata that provides of the embodiment of the invention also comprises:
When each the memory node change in the above-mentioned cluster, upgrade above-mentioned cluster metadata according to the change of this each memory node, obtain new cluster metadata.
Still take above-mentioned memory node A, B, C, D and E as example, when memory node D fault or when being cancelled, correspondingly, the cluster metadata is upgraded, delete relevant with memory node D, and the metadata that has nothing to do with other memory nodes.If increased memory node F in the storage system, correspondingly, in the cluster metadata, increase the metadata relevant with memory node F.
The method that the present embodiment provides is preserved by the cluster metadata is divided and is issued to each memory node, so that pivot data server and the standby equal fault of meta data server, lost in the situation of cluster metadata, still can be by obtaining metadata from each memory node, structure obtains the cluster metadata, thereby recover the cluster metadata, solve pivot data server and standby meta data server equal fault in the prior art and caused the cluster metadata to be lost and the problem that can't recover, improved the reliability of storage system.
When the cluster metadata is updated, also can carry out to new cluster metadata the operation of above-mentioned steps 11 ~ step 14, for example the method for the control cluster metadata that provides of the embodiment of the invention also can comprise:
Above-mentioned new cluster metadata according to dividing with the correlativity of residue memory node, is obtained the new associated metadata of this residue memory node;
The associated metadata that this is new is issued to this residue memory node;
When this new cluster metadata can't be obtained, obtain the new associated metadata that issues from this residue memory node;
The new associated metadata that obtains is built into above-mentioned new cluster metadata.
Wherein, the residue memory node is the residue memory node of broad sense, and quantity that namely should the residue memory node can be less than the quantity of original memory node, also can be greater than the quantity of original storing received.Still take above-mentioned memory node A, B, C, D and E as example, when memory node D fault or when being cancelled, the residue memory node is memory node A, B, C and E; If increased memory node F in the storage system, then remaining memory node is memory node A, B, C, D, E and F.
Alternatively, the new associated metadata that obtains is built into after the above-mentioned new cluster metadata, also comprises:
Above-mentioned new cluster metadata is sent to meta data server.Wherein, meta data server can be the pivot data server, also can be standby meta data server.
When said method when being arranged on control module on the master control borad and carrying out, control module issues before the associated metadata, at first will obtain the cluster metadata.For example, the cluster metadata according to before dividing with the correlativity of each memory node, is also comprised:
The configuration file that generates when starting according to the storage system of above-mentioned cluster obtains above-mentioned cluster metadata, perhaps obtains above-mentioned cluster metadata from meta data server.
Correspondingly, because the cluster metadata can't obtain, may be because the meta data server fault causes, therefore, the associated metadata that obtains is built into after the above-mentioned cluster metadata, also comprise:
The cluster metadata that is built into is sent to meta data server.Like this, after meta data server recovers, also can obtain the cluster metadata, solve in the prior art in the situation of the equal fault of active and standby meta data server, can't recover the problem of cluster metadata.
Correspondingly, the process flow diagram of a kind of method for control cluster metadata of providing for the embodiment of the invention of Fig. 2.The method that the present embodiment provides is the operating process of the memory node in the cluster in other words in the storage system.As shown in Figure 2, this flow process comprises:
The associated metadata that step 21, reception and storing metadata controller issue, this associated metadata is relevant with this memory node, by this metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node.
Wherein, metadata controller is namely realized the device of method shown in Figure 1, can be the control module that newly increases, also master control borad.The division of associated metadata and issuing sees the explanation among above-mentioned Fig. 1 for details.
Wherein, the cluster metadata sees the explanation among above-mentioned Fig. 1 for details.When the device of storage cluster metadata broke down, metadata controller need to obtain the associated metadata that is handed down to memory node from memory node, to make up the cluster metadata, realized the recovery of cluster metadata.
The associated metadata that the method that above-described embodiment provides issues by receiving metadata controller, and in the situation that the cluster metadata can't obtain, send the associated metadata that it issues to metadata controller, so that metadata controller can be enough the associated metadata that sends of memory node make up and obtain the cluster metadata, solved pivot data server and the standby equal fault of meta data server in the prior art and caused the cluster metadata to be lost and the problem that can't recover.
Correspondingly, the another kind that provides for the embodiment of the invention of Fig. 3 is used for the process flow diagram of the method for control cluster metadata.The method that the present embodiment provides is the operating process of meta data server, and meta data server can be the pivot data server, also standby meta data server.As shown in Figure 3, this flow process comprises:
The cluster metadata that step 31, reception metadata controller send, this cluster metadata is made up the associated metadata that obtains by this metadata controller and obtains, when this cluster metadata can't be obtained, each memory node in this cluster acquired this associated metadata by this metadata controller.
Wherein, metadata controller is namely realized the device of method shown in Figure 1, can be the control module that newly increases, also master control borad.The division of associated metadata and issuing, and obtain the method that associated metadata makes up the cluster metadata, see the explanation among above-mentioned Fig. 1 for details.
After the meta data server fault, restart, can receive the cluster metadata of recovering from metadata controller.
The above-mentioned cluster metadata that step 32, preservation receive.
Meta data server is preserved the cluster metadata that receives from metadata controller, thereby recovers the storage of the cluster metadata of meta data server.
The method that above-described embodiment provides utilizes associated metadata to make up the cluster metadata that obtains by receiving metadata controller, recovered the cluster metadata in the meta data server, solved pivot data server and the standby equal fault of meta data server in the prior art and caused the cluster metadata to be lost and the problem that can't recover.
When the memory node change caused the cluster metadata updates in the cluster, the method that is used for control cluster metadata that the embodiment of the invention provides also can comprise:
Receive and preserve the new cluster metadata that above-mentioned metadata controller sends, this new cluster metadata, is upgraded above-mentioned cluster metadata according to the change of this each memory node and is obtained when each memory node in this cluster changes by above-mentioned metadata controller.
Like this, meta data server can be in the situation that the cluster metadata evolution, by the new cluster metadata that receives and the Preservation Metadata controller sends, and the cluster metadata of corresponding updated stored.
The below is described in further detail the method for controlling the cluster metadata take Fig. 4, Fig. 5 as example.
Leave the integrity problem that the cluster metadata is brought concentratedly in order to solve in the prior art meta data server, introduced role's metadata controller (Metadata Controller, MDC) of a cluster metadata Control Server in the present embodiment.
The cluster metadata store is on meta data server, and meta data server can be set up active and standby mechanism.MDC is in charge of and controls the cluster metadata, with whole cluster metadata according to dividing with the correlativity (corresponding relation of memory node and user data) of each memory node, and the metadata after will dividing is handed down to respectively each relevant memory node, and each memory node meta-data preservation that self is relevant is on local hard drive.
When active and standby meta data server all broke down, MDC was responsible for rebuilding the cluster metadata of whole storage system.Specifically referring to Fig. 5, comprise the steps:
Step 53, MDC with memory node 1, memory node 2 ..., the metadata returned of memory node N merges reconstruct, obtains the cluster metadata.
Step 54, MDC send to meta data server with the cluster metadata that obtains in the step 53, the cluster metadata data of losing to recover meta data server.
The present embodiment has increased MDC on the basis of existing storage system, in the situation that the meta data server fault, automatic collected metadata information from the memory node, and rebuild recovery has improved the reliabilty and availability of metadata by MDC.And MDC can restart new meta data server, and the cluster metadata after will recovering writes meta data server, has guaranteed professional automatic recovery.By increasing MDC the cluster metadata is controlled, so that meta data server is independent from the business processing of complexity, only need carry out the storage of metadata, namely by with the control of cluster metadata and the separate processing of storage of cluster metadata, guaranteed further reliability and the extensibility of meta data server.
One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each embodiment of the method can be finished by the relevant hardware of programmed instruction.Aforesaid program can be stored in the computer read/write memory medium.This program is carried out the step that comprises above-mentioned each embodiment of the method when carrying out; And aforesaid storage medium comprises: the various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
A kind of structural representation of controlling the device of cluster metadata that Fig. 6 provides for the embodiment of the invention.The device that the present embodiment provides is used for realizing method shown in Figure 1, can be the metadata controller among the said method embodiment.As shown in Figure 6, this device comprises: the first division unit 61, first issues unit 62, the first acquiring unit 63 and the first construction unit 64.
The first division unit 61 is used for the cluster metadata obtaining the associated metadata of described each memory node according to dividing with the correlativity of each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata
The first acquiring unit 63 is used for when described cluster metadata can't be obtained, and each memory node in described cluster obtains the associated metadata that issues.
The first construction unit 64 is used for the associated metadata that obtains is built into described cluster metadata.
Alternatively, the device of the control cluster metadata that provides of the embodiment of the invention also comprises:
Updating block is used for upgrading described cluster metadata according to the change of described each memory node when each the memory node change in the described cluster, obtains new cluster metadata.
Alternatively, the device of the control cluster metadata that provides of the embodiment of the invention also comprises:
The second division unit, be used for will described new cluster metadata according to dividing with the correlativity that remains memory node, obtain the new associated metadata of described residue memory node;
Second issues the unit, is used for described new associated metadata is issued to described residue memory node;
Second acquisition unit is used for obtaining the new associated metadata that issues from described residue memory node when described new cluster metadata can't be obtained;
The second construction unit, the new associated metadata that is used for obtaining is built into described new cluster metadata.
Alternatively, the device of the control cluster metadata that the embodiment of the invention provides also comprises: the first transmitting element, be used for after the new associated metadata that described the second construction unit will obtain is built into described new cluster metadata, described new cluster metadata being sent to meta data server.
Alternatively, the device of the control cluster metadata that the embodiment of the invention provides also comprises: the cluster metadata obtains the unit, be used in described the first division unit the cluster metadata according to before dividing with the correlativity of each memory node, the configuration file that generates when starting according to the storage system of described cluster obtains described cluster metadata, perhaps obtains described cluster metadata from meta data server.
Alternatively, the device of the control cluster metadata that the embodiment of the invention provides also comprises: the second transmitting element, be used for after described the first construction unit is built into described cluster metadata with the associated metadata that obtains, the cluster metadata that is built into being sent to meta data server.
The structural representation of a kind of memory node equipment that Fig. 7 provides for the embodiment of the invention.The device that the present embodiment provides is used for realizing method shown in Figure 2, can be the memory node among the said method embodiment.As shown in Figure 7, this equipment comprises: metadata receiving element 71 and metadata transmitting element 72.
Metadata receiving element 71 is for the associated metadata that receives and the storing metadata controller issues, and described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node.
Metadata transmitting element 72 is used for described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
The structural representation of a kind of meta data server that Fig. 8 provides for the embodiment of the invention.The device that the present embodiment provides is used for realizing method shown in Figure 3, can be the metadata controller among the said method embodiment.As shown in Figure 8, this server comprises: the first receiving element 81 and storage unit 82.
The first receiving element 81 is used for receiving the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller.
Alternatively, the meta data server that provides of the embodiment of the invention also comprises:
The second receiving element, be used for receiving and preserving the new cluster metadata that described metadata controller sends, described new cluster metadata, is upgraded described cluster metadata according to the change of described each memory node and is obtained when each memory node in the described cluster changes by described metadata controller.
The structural representation of a kind of storage system that Fig. 9 provides for the embodiment of the invention.The device that the present embodiment provides can be used for realizing Fig. 1-method shown in Figure 5.As shown in Figure 9, this storage system comprises metadata controller 91, memory node 92 and meta data server 93.
Metadata controller 91 can be the device of above-mentioned any control cluster metadata, be used for dividing and issue metadata and arrive memory node, and in the situation that the meta data server fault is lost the cluster metadata, obtain metadata from memory node, recover the cluster metadata.Memory node 92 can be any memory node equipment that above-described embodiment provides, be used for the storage metadata relevant with oneself, and in the situation that the meta data server fault is lost the cluster metadata, each autocorrelative metadata is sent to metadata controller 91, be reconstructed, recover the cluster metadata.Meta data server 93 can be any meta data server that above-described embodiment provides, lose the cluster metadata after, can obtain the cluster metadata from metadata controller 91, realize the recovery of cluster metadata.
In addition, the embodiment of the invention gives a kind of computer program, and this computer program comprises computer-readable medium, and this computer-readable recording medium comprises second group of program code, is used for carrying out the step of above-mentioned method shown in Figure 1:
The cluster metadata according to dividing with the correlativity of each memory node, is obtained the associated metadata of described each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata;
Described associated metadata is issued to memory node relevant in the described cluster;
When described cluster metadata can't be obtained, each memory node in described cluster obtained the associated metadata that issues;
The associated metadata that obtains is built into described cluster metadata.
Alternatively, also comprise:
When each the memory node change in the described cluster, upgrade described cluster metadata according to the change of described each memory node, obtain new cluster metadata.
Alternatively, also comprise:
Described new cluster metadata according to dividing with the correlativity of residue memory node, is obtained the new associated metadata of described residue memory node;
Described new associated metadata is issued to described residue memory node;
When described new cluster metadata can't be obtained, obtain the new associated metadata that issues from described residue memory node;
The new associated metadata that obtains is built into described new cluster metadata.
Alternatively, the new associated metadata that obtains is built into after the described new cluster metadata, also comprises:
Described new cluster metadata is sent to meta data server.
Alternatively, the cluster metadata according to before dividing with the correlativity of each memory node, is also comprised:
The configuration file that generates when starting according to the storage system of described cluster obtains described cluster metadata, perhaps obtains described cluster metadata from meta data server.
Alternatively, the associated metadata that obtains is built into after the described cluster metadata, also comprises:
The cluster metadata that is built into is sent to meta data server.
In addition, the embodiment of the invention gives another kind of computer program, and this computer program comprises computer-readable medium, and this computer-readable recording medium comprises second group of program code, is used for carrying out the step of above-mentioned method shown in Figure 2:
The associated metadata that reception and storing metadata controller issue, described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node;
Described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
At last, the embodiment of the invention gives another computer program, and this computer program comprises computer-readable medium, and this computer-readable recording medium comprises second group of program code, is used for carrying out the step of above-mentioned method shown in Figure 3:
Receive the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller;
Preserve the described cluster metadata that receives.
Alternatively, also comprise the execution following steps:
Receive and preserve the new cluster metadata that described metadata controller sends, described new cluster metadata, is upgraded described cluster metadata according to the change of described each memory node and is obtained when each memory node in the described cluster changes by described metadata controller.
It should be noted that at last: above each embodiment is not intended to limit only in order to technical scheme of the present invention to be described; Although with reference to aforementioned each embodiment the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be made amendment to the technical scheme that aforementioned each embodiment puts down in writing, and perhaps some or all of technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the scope of various embodiments of the present invention technical scheme.
Claims (19)
1. a method of controlling the cluster metadata is characterized in that, comprising:
The cluster metadata according to dividing with the correlativity of each memory node, is obtained the associated metadata of described each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata;
Described associated metadata is issued to memory node relevant in the described cluster;
When described cluster metadata can't be obtained, each memory node in described cluster obtained the associated metadata that issues;
The associated metadata that obtains is built into described cluster metadata.
2. described method according to claim 1 is characterized in that, also comprises:
When each the memory node change in the described cluster, upgrade described cluster metadata according to the change of described each memory node, obtain new cluster metadata.
3. described method according to claim 2 is characterized in that, also comprises:
Described new cluster metadata according to dividing with the correlativity of residue memory node, is obtained the new associated metadata of described residue memory node;
Described new associated metadata is issued to described residue memory node;
When described new cluster metadata can't be obtained, obtain the new associated metadata that issues from described residue memory node;
The new associated metadata that obtains is built into described new cluster metadata.
4. described method according to claim 3 is characterized in that, the new associated metadata that obtains is built into after the described new cluster metadata, also comprises:
Described new cluster metadata is sent to meta data server.
5. each described method is characterized in that according to claim 1-4, and the cluster metadata according to before dividing with the correlativity of each memory node, is also comprised:
The configuration file that generates when starting according to the storage system of described cluster obtains described cluster metadata, perhaps obtains described cluster metadata from meta data server.
6. each described method is characterized in that according to claim 1-4, and the associated metadata that obtains is built into after the described cluster metadata, also comprises:
The cluster metadata that is built into is sent to meta data server.
7. a method that is used for control cluster metadata is characterized in that, comprising:
The associated metadata that reception and storing metadata controller issue, described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node;
Described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
8. a method that is used for control cluster metadata is characterized in that, comprising:
Receive the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller;
Preserve the described cluster metadata that receives.
9. described method according to claim 8 is characterized in that, also comprises:
Receive and preserve the new cluster metadata that described metadata controller sends, described new cluster metadata, is upgraded described cluster metadata according to the change of described each memory node and is obtained when each memory node in the described cluster changes by described metadata controller.
10. a device of controlling the cluster metadata is characterized in that, comprising:
The first division unit is used for the cluster metadata obtaining the associated metadata of described each memory node according to dividing with the correlativity of each memory node; Described each memory node is the memory node in the cluster under the described cluster metadata;
First issues the unit, is used for described associated metadata is issued to memory node relevant in the described cluster;
The first acquiring unit is used for when described cluster metadata can't be obtained, and each memory node in described cluster obtains the associated metadata that issues;
The first construction unit is used for the associated metadata that obtains is built into described cluster metadata.
11. described device is characterized in that according to claim 10, also comprises:
Updating block is used for upgrading described cluster metadata according to the change of described each memory node when each the memory node change in the described cluster, obtains new cluster metadata.
12. described device is characterized in that according to claim 11, also comprises:
The second division unit, be used for will described new cluster metadata according to dividing with the correlativity that remains memory node, obtain the new associated metadata of described residue memory node;
Second issues the unit, is used for described new associated metadata is issued to described residue memory node;
Second acquisition unit is used for obtaining the new associated metadata that issues from described residue memory node when described new cluster metadata can't be obtained;
The second construction unit, the new associated metadata that is used for obtaining is built into described new cluster metadata.
13. described device according to claim 12, it is characterized in that, also comprise: the first transmitting element is used for after the new associated metadata that described the second construction unit will obtain is built into described new cluster metadata described new cluster metadata being sent to meta data server.
14. each described device according to claim 10-13, it is characterized in that, also comprise: the cluster metadata obtains the unit, be used in described the first division unit the cluster metadata according to before dividing with the correlativity of each memory node, the configuration file that generates when starting according to the storage system of described cluster obtains described cluster metadata, perhaps obtains described cluster metadata from meta data server.
15. each described device according to claim 10-13, it is characterized in that, also comprise: the second transmitting element is used for after described the first construction unit is built into described cluster metadata with the associated metadata that obtains the cluster metadata that is built into being sent to meta data server.
16. a memory node equipment is characterized in that, comprising:
The metadata receiving element is used for the associated metadata that reception and storing metadata controller issue, and described associated metadata is relevant with this memory node, by described metadata controller the cluster metadata is obtained according to dividing with the correlativity of each memory node;
The metadata transmitting element is used for described associated metadata is sent to described metadata controller, so that when described cluster metadata can't be obtained, described metadata controller was built into described cluster metadata with the associated metadata that receives.
17. a meta data server is characterized in that, comprising:
The first receiving element, be used for receiving the cluster metadata that metadata controller sends, described cluster metadata is made up the associated metadata that obtains by described metadata controller and obtains, when described cluster metadata can't be obtained, each memory node in described cluster acquired described associated metadata by described metadata controller;
Storage unit is used for preserving the described cluster metadata that described cluster metadata receiving element receives.
18. described server is characterized in that according to claim 17, also comprises:
The second receiving element, be used for receiving and preserving the new cluster metadata that described metadata controller sends, described new cluster metadata, is upgraded described cluster metadata according to the change of described each memory node and is obtained when each memory node in the described cluster changes by described metadata controller.
19. a storage system is characterized in that, comprises device, the claims 16 described memory node equipment and the claims 17 or the 18 described meta data servers of each described control cluster metadata of the claims 10-15.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210547876.9A CN103019626B (en) | 2012-12-17 | 2012-12-17 | Storage system, method and device for controlling cluster metadata |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210547876.9A CN103019626B (en) | 2012-12-17 | 2012-12-17 | Storage system, method and device for controlling cluster metadata |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103019626A true CN103019626A (en) | 2013-04-03 |
| CN103019626B CN103019626B (en) | 2016-04-13 |
Family
ID=47968278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210547876.9A Active CN103019626B (en) | 2012-12-17 | 2012-12-17 | Storage system, method and device for controlling cluster metadata |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103019626B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017143843A1 (en) * | 2016-02-26 | 2017-08-31 | 华为技术有限公司 | Metadata recovery method and device |
| CN108271420A (en) * | 2016-11-02 | 2018-07-10 | 华为技术有限公司 | Manage method, file system and the server system of file |
| CN109426587A (en) * | 2017-08-25 | 2019-03-05 | 杭州海康威视数字技术股份有限公司 | A kind of data reconstruction method and device |
| CN111338647A (en) * | 2018-12-18 | 2020-06-26 | 杭州海康威视数字技术股份有限公司 | Big data cluster management method and device |
| CN114428760A (en) * | 2021-12-30 | 2022-05-03 | 北京云宽志业网络技术有限公司 | Cluster storage system and metadata recovery method |
| CN117390078A (en) * | 2023-12-13 | 2024-01-12 | 中国对外经济贸易信托有限公司 | Data processing method, device, storage medium and computer equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080215732A1 (en) * | 2007-02-14 | 2008-09-04 | Thomas Haug | Multi-site scenarios in the storage and archiving of medical data objects |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101741731A (en) * | 2009-12-03 | 2010-06-16 | 中兴通讯股份有限公司 | Content metadata storing, inquiring method and managing system in content delivery network (CDN) |
| US7949569B2 (en) * | 2007-01-30 | 2011-05-24 | Oracle International Corporation | Distributed device information management system as a distributed information repository system |
| CN102088389A (en) * | 2009-12-02 | 2011-06-08 | 中兴通讯股份有限公司 | Distributed content access scheduling device and content reading method |
-
2012
- 2012-12-17 CN CN201210547876.9A patent/CN103019626B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7949569B2 (en) * | 2007-01-30 | 2011-05-24 | Oracle International Corporation | Distributed device information management system as a distributed information repository system |
| US20080215732A1 (en) * | 2007-02-14 | 2008-09-04 | Thomas Haug | Multi-site scenarios in the storage and archiving of medical data objects |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102088389A (en) * | 2009-12-02 | 2011-06-08 | 中兴通讯股份有限公司 | Distributed content access scheduling device and content reading method |
| CN101741731A (en) * | 2009-12-03 | 2010-06-16 | 中兴通讯股份有限公司 | Content metadata storing, inquiring method and managing system in content delivery network (CDN) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017143843A1 (en) * | 2016-02-26 | 2017-08-31 | 华为技术有限公司 | Metadata recovery method and device |
| US10572335B2 (en) | 2016-02-26 | 2020-02-25 | Huawei Technologies Co., Ltd. | Metadata recovery method and apparatus |
| CN108271420A (en) * | 2016-11-02 | 2018-07-10 | 华为技术有限公司 | Manage method, file system and the server system of file |
| CN108271420B (en) * | 2016-11-02 | 2020-11-27 | 华为技术有限公司 | Method for managing files, file system and server system |
| CN109426587A (en) * | 2017-08-25 | 2019-03-05 | 杭州海康威视数字技术股份有限公司 | A kind of data reconstruction method and device |
| CN111338647A (en) * | 2018-12-18 | 2020-06-26 | 杭州海康威视数字技术股份有限公司 | Big data cluster management method and device |
| CN111338647B (en) * | 2018-12-18 | 2023-09-12 | 杭州海康威视数字技术股份有限公司 | Big data cluster management method and device |
| CN114428760A (en) * | 2021-12-30 | 2022-05-03 | 北京云宽志业网络技术有限公司 | Cluster storage system and metadata recovery method |
| CN117390078A (en) * | 2023-12-13 | 2024-01-12 | 中国对外经济贸易信托有限公司 | Data processing method, device, storage medium and computer equipment |
| CN117390078B (en) * | 2023-12-13 | 2024-02-09 | 中国对外经济贸易信托有限公司 | Data processing method, device, storage medium and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103019626B (en) | 2016-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10503616B2 (en) | Periodic data replication | |
| CN103019626B (en) | Storage system, method and device for controlling cluster metadata | |
| CN102981931B (en) | Backup method and device for virtual machine | |
| CN103136074B (en) | Data storage method and data storage system of multiple disk array systems | |
| CN103780638B (en) | Method of data synchronization and system | |
| CN104679772B (en) | Method, apparatus, equipment and the system of file are deleted in Distributed Data Warehouse | |
| CN102521269B (en) | Index-based computer continuous data protection method | |
| US8818954B1 (en) | Change tracking | |
| CN100464334C (en) | Data incremental backup and recovery method applied to data bank | |
| CN106407040A (en) | Remote data copy method and system | |
| CN106502823A (en) | data cloud backup method and system | |
| CN102308273A (en) | Storage system | |
| WO2018098972A1 (en) | Log recovery method, storage device and storage node | |
| CN101809558A (en) | System and method for remote asynchronous data replication | |
| CN102508733B (en) | A kind of data processing method based on disk array and disk array manager | |
| CN102955720A (en) | Method for improving stability of EXT (extended) file system | |
| US9525733B2 (en) | Method and system for data synchronization | |
| CN105278882A (en) | Disk management method of distributed file system | |
| CN102347866A (en) | Data configuration system and method in network management | |
| US8527454B2 (en) | Data replication using a shared resource | |
| CN106569751A (en) | Multi-controller cache mirroring method and system | |
| CN102902561A (en) | Data storage method and data storage system | |
| CN102999399A (en) | Method and device of automatically restoring storage of JBOD (just bundle of disks) array | |
| CN104583966A (en) | Backup and restore system for a deduplicated file system and corresponding server and method | |
| US9612921B2 (en) | Method and system for load balancing a distributed database providing object-level management and recovery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220526 Address after: No. 1899 Xiyuan Avenue, high tech Zone (West District), Chengdu, Sichuan 610041 Patentee after: Chengdu Huawei Technologies Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |