CN103106207A - Method and equipment for metadata distribution in object storing system - Google Patents
Method and equipment for metadata distribution in object storing system Download PDFInfo
- Publication number
- CN103106207A CN103106207A CN2011103556788A CN201110355678A CN103106207A CN 103106207 A CN103106207 A CN 103106207A CN 2011103556788 A CN2011103556788 A CN 2011103556788A CN 201110355678 A CN201110355678 A CN 201110355678A CN 103106207 A CN103106207 A CN 103106207A
- Authority
- CN
- China
- Prior art keywords
- meta data
- data server
- range
- independent sub
- mds
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and equipment for metadata distribution in an object storing system. The method mainly comprises that according to storage capacity of each MDS in a metadata server cluster, an independent subinterval is distributed for each MDS, according to the mapping relationship of the independent subinterval and an operation result after metadata is subjected to overall-situation unique identification operation in an object storage system, the independent subinterval where the operation result is determined, the metadata is distributed to the MDS corresponding to the independent subinterval, a metadata server is used for management of the metadata, the storage capacity of the MDS is fully considered for the distribution of the metadata in the metadata server cluster, so that the stronger the storage capacity of the MDS is, the larger the managed metadata is, and distribution balance of the metadata in the metadata server cluster is guaranteed.
Description
Technical field
The present invention program relates to field of storage, relates in particular to the method and apparatus of meta-data distribution in a kind of object storage system.
Background technology
Object storage system generally comprises client, meta data server (MetaData Server, MDS) and object storage device (Object-based Storage Device, OSD), wherein: MDS mainly is responsible for the management of object metadata information and relevant security certificate etc. as the bridge of client and OSD; OSD mainly offers as the actual storage carrier of data the interface that client is carried out data access; Client mainly is responsible for as the node of application program operation the access interface that provides general.
In the pattern of this Three Party Communication, stored a large amount of metadata informations in MDS, comprised the path address of the corresponding data information memory of each metadata in OSD in described metadata information; When client in the time need to reading and writing the data in OSD, need first to carry out information interaction with MDS, obtain the metadata information in object (comprising metadata information and data) from MDS, determine that data in object are stored in the address in OSD, and then and corresponding OSD between carry out information interaction, read or data writing.
In distributed memory system, in order to support the level and smooth expansion of jumbo storage and system, the mode that usually adopts the meta data server cluster manages the distribution of metadata, generally can adopt following two kinds of methods to realize:
First method is hash method.
Described hash method is: after the file identifier of metadata is carried out Hash operation, the quantitative value of MDS in the operation result that obtains and meta data server cluster is carried out complementation, select MDS corresponding to this residual value to manage this metadata according to the residual value that obtains.
when utilizing hash method that metadata is managed, when if in the meta data server cluster, the quantity of MDS changes (increasing the quantity of MDS or the quantity of minimizing MDS), will again carry out complementation to the operation result after the metadata Hash operation according to new MDS quantity, because changing, MDS quantity will cause the remainder result to change, therefore, for arbitrary MDS, after MDS quantity changes, its metadata of managing will change, when this can cause each MDS quantity to change, the migration of a large amount of metadata between MDS can occur, increase the expense of network operation.
Second method is the subtree split plot design.
Described subtree split plot design is the globally unique identifier in object storage system with metadata, according to the mode of space TOC level, is divided into independently subtree, and each MDS in the meta data server cluster is in charge of one of them or a plurality of subtree.
When utilizing the subtree split plot design that metadata is managed, each one or more subtree of MDS random administration so just makes in the meta data server cluster load of certain or some MDS overweight, becomes the performance bottleneck of whole object storage system.
Summary of the invention
The invention provides the method and apparatus of meta-data distribution in a kind of object storage system, be used for the problem of the MDS load imbalance of solution meta data server cluster.
A kind of method of meta-data distribution in object storage system, the method comprises:
According to the size of each meta data server storage capacity in the meta data server cluster, for each meta data server distributes independent sub-range, wherein, the independent sub-range that the meta data server that storage capacity is larger distributes is larger;
Determine the operation result after the globally unique identifier of metadata in object storage system carries out computing, and according to default operation result and the mapping relations between described independent sub-range, determine the independent sub-range that this operation result is affiliated;
Meta data server corresponding to described independent sub-range that described meta-data distribution is extremely definite.
The equipment of meta-data distribution in a kind of object storage system, this equipment comprises:
Distribution module is used for the size according to each meta data server storage capacity of meta data server cluster, and for each meta data server distributes independent sub-range, wherein, the independent sub-range that the meta data server that storage capacity is larger distributes is larger;
Determination module is used for determining the operation result of metadata after the globally unique identifier of object storage system carries out computing, and the independent sub-range under this operation result;
Distribution module is used for described meta-data distribution to meta data server corresponding to described independent sub-range of determining.
Beneficial effect of the present invention is as follows:
in the embodiment of the present invention, size according to each MDS storage capacity in the meta data server cluster, for each MDS distributes independent sub-range, and according to metadata the independent sub-range under the operation result after the globally unique identifier's computing in object storage system, with this meta-data distribution to the corresponding MDS in this independent sub-range, utilize this meta data server that this metadata is managed, fully taken into account the storage capacity of MDS due to the distribution of metadata in the meta data server cluster, the metadata that the MDS that storage capacity is stronger manages is larger, guaranteed the harmony that metadata distributes in the meta data server cluster.
Description of drawings
Fig. 1 is the process flow diagram of the method for meta-data distribution in a kind of object storage system of the present embodiment one;
Fig. 2 is for distributing independent sub-range schematic diagram take the form of closed loop as meta data server;
Fig. 3 is when adding new meta data server, dwindles the schematic diagram in the independent sub-range that original meta data server distributes;
Fig. 4 is for distributing the schematic diagram in independent sub-range for the meta data server that newly adds;
Fig. 5 shifts out the schematic diagram that distributes independent sub-range after meta data server for remaining meta data server;
Fig. 6 is the structural representation of the equipment of meta-data distribution in a kind of object storage system of the present embodiment two.
Embodiment
in order to realize purpose of the present invention, the invention provides the method and apparatus of meta-data distribution in a kind of object storage system, size according to each MDS storage capacity in the meta data server cluster, for each MDS distributes independent sub-range, and according to metadata the independent sub-range under the operation result after the globally unique identifier's computing in object storage system, with this meta-data distribution to the corresponding MDS in this independent sub-range, utilize this meta data server that this metadata is managed, fully taken into account the storage capacity of MDS due to the distribution of metadata in the meta data server cluster, the metadata that the MDS that storage capacity is stronger manages is larger, guaranteed the harmony that metadata distributes in the meta data server cluster.
Be described in detail below in conjunction with the scheme of Figure of description to the embodiment of the present invention.
Embodiment one:
As shown in Figure 1, be the process flow diagram of the method for meta-data distribution in a kind of object storage system of the present embodiment one, particular content comprises the following steps:
Step 101: according to the size of each MDS storage capacity in the meta data server cluster, for each MDS distributes independent sub-range, wherein, the independent sub-range that the MDS that storage capacity is larger distributes is larger.
In this step, add the difference in period due to the large-scale storage systems equipment as object storage system, in the meta data server cluster, the hardware configuration of MDS is also different, just means that also the size of MDS storage capacity is also different.In the solution of the present invention, can be according to the size of the memory size of MDS for the measurement of MDS storage capacity, also can be according to the size of the hard-disk capacity of MDS, so long as can be used for determining the MDS storage capacity parameter can, here do not limit.
Particularly, take the memory power of MDS as example, when the MDS internal memory was larger, its storage capacity was just larger.That that is to say, in the meta data server cluster, the manageable independent sub-range that the MDS that internal memory is larger distributes is just larger.
For MDS distributes the concrete grammar in independent sub-range as follows:
The first step: determine total storage capacity that the meta data server cluster is current.
Take the memory power of MDS as example, due in object storage system, when system scale reaches PB (10
15) when level, suppose that the mean size of object is 1MB, what comprise object in object storage system adds up to 10
9Individual, on average the metadata size of each object is 400 bytes, if MDS in save as 4GB, the MDS cluster scale is estimated as ((400*10 so
9Storage demand can be satisfied in)/4GB)=100.
Easy in order to describe, supposing in this step has 4 MDS in the meta data server cluster, and the sign of these 4 MDS is respectively MDS_0, MDS_1, MDS_2 and MDS_3, and its internal memory is all 4GB, and total storage capacity of this meta data server cluster is 16GB.
Second step: the ratio of determining storage capacity with the total storage capacity of each MDS.
Still take above-mentioned MDS_0, MDS_1, MDS_2 and MDS_3 as example, because the memory size of these 4 MDS is identical, therefore, the storage capacity of these 4 MDS is 1/4 with the ratio of total storage capacity.
The 3rd goes on foot: be that each MDS distributes independent sub-range, wherein, the ratio in the independent sub-range that each MDS distributes and total interval equals the ratio of storage capacity with the total storage capacity of this MDS.
The size in described total interval is fixed, and does not change because of the change of MDS quantity, and the resource pool that it can arrange for the scale according to the meta data server cluster can be also that a hop count value is interval.If it is interval that total interval is a hop count value, an independent sub-range is that total interval interior numerical value is interval.
When distributing independent sub-range for MDS, non-intersect for the independent sub-range that any two MDS distribute, and equal described total interval for the independent sub-range sum of all meta data servers distribution.
Suppose total interval for [0,2
128-1), because the storage capacity of MDS_0, MDS_1, MDS_2 and MDS_3 is identical, be the big or small identical of this 4 MDS independent sub-range of distributing, if total intervally represent with closed loop shown in Figure 2, the independent sub-range for the MDS_0 distribution is [0,2
126-1) the independent sub-range of, distributing for MDS_1 is [2
126, 2
127-1) the independent sub-range of, distributing for MDS_2 is [2
127, 3*2
126-1) the independent sub-range of, distributing for MDS_3 is [3*2
126, 2
128-1).
Step 102: determine the operation result after the globally unique identifier of metadata in object storage system carries out computing, and according to default operation result and the mapping relations between described independent sub-range, determine the independent sub-range that this operation result is affiliated.
The globally unique identifier of metadata in object storage system can be the memory address of object under metadata, can be also the information that other can a metadata of unique expression, it does not limited herein.
That the memory address of object under metadata is as example take globally unique identifier, in object storage system, bucket is considered to be similar to the catalogue in traditional file systems, be used for storage object, because bucket can not be nested, that is to say in bucket and can not create again bucket, and object storage system is not supported the rename operation to bucket, this has just determined the bucket name in case create immutable, therefore, the bucket name has global uniqueness as the memory address of object, can be used as the globally unique identifier of metadata.
For the metadata of a MDS to be distributed to, after having determined the globally unique identifier of this metadata in object storage system, can utilize set algorithm, this globally unique identifier is carried out computing and obtains operation result.
Described set algorithm can be the MD5 algorithm, in the time of can also being other with globally unique identifier as input, can export the algorithm of the operation result with uniqueness.
For example: two buckets of establishment are respectively: bucket_01 and bucket_02, store object object_01 (being that object identifier is bucket_01/object_01) under bucket_01, store object object_02 (being that object identifier is bucket_02/object_02) under bucket_02.
For the metadata of bucket_01/object_01, determine that the globally unique identifier of this metadata is bucket_01, utilize the MD5 algorithm to carry out computing to bucket_01, the operation result that obtains MD5 (bucket_01) is 2
100, 4 independent sub-ranges of this operation result and MDS_0～MDS_3 distribution are shone upon, determine that this operation result should map to [0,2
126-1) in.
If it is interval that independent sub-range is numerical value, globally unique identifier carries out computing and the operation result that obtains is a numerical value.
After obtaining operation result, according to default operation result and the mapping relations between independent sub-range, determine the independent sub-range that this operation result is affiliated.
Step 103: meta data server corresponding to described independent sub-range that described meta-data distribution is extremely definite.
Still MDS_0, MDS_1, MDS_2 and the MDS_3 in the step 101 be as example, and the operation result that the globally unique identifier of supposing a metadata carries out after computing is 2
100, this operation result belongs to independent sub-range [0,2
126-1), due to this independent sub-range [0,2
126-1) be the independent sub-range of distributing to MDS_0, therefore, this metadata can be distributed to MDS_0, is managed by MDS_0.
By the scheme of above-mentioned steps 101 to step 103, come to distribute independent sub-range for MDS according to the storage capacity of each MDS in the meta data server cluster, because the independent sub-range that the stronger meta data server of storage capacity distributes is larger, therefore, the metadata that can map to this independent sub-range is also just more, make the stronger more metadata of MDS load of storage capacity, the less metadata of MDS load that storage capacity is more weak has realized the load balancing of MDS.
Further, the scale of meta data server cluster can change according to the actual demand of object storage system, for example, and when the popularization of object storage system, can enlarge simultaneously the scale of meta data server cluster, newly add one or more MDS in the meta data server cluster; When the scale down of object storage system or MDS break down, can shift out one or more MDS, the scale of dwindling the meta data server cluster from the meta data server cluster.
When the scale of meta data server cluster changed, total storage capacity of meta data server cluster also can change, and at this moment, the independent sub-range that the MDS in the meta data server cluster distributes also needs to adjust, and concrete adjustment mode is as follows:
1, the situation that newly adds MDS in the meta data server cluster.
Owing to newly having added MDS in the meta data server cluster, total storage capacity of meta data server cluster will increase, therefore, the storage capacity of MDS originally will diminish with the ratio that newly adds the total storage capacity after MDS, need to dwindle the scope in the independent sub-range that original MDS distributes, at this moment, the independent sub-range of MDS will appear also being not yet assigned to, the MDS that newly adds is distributed in these independent sub-ranges that also are not yet assigned to MDS, after having realized newly adding MDS, each MDS is distributed the adjustment in independent sub-range.
Need to prove, if newly add the storage capacity of MDS larger, the scope that need to dwindle of original MDS is also just larger, therefore, can distribute to the independent sub-range that newly adds MDS just larger, if add many MDS with stylish, the independent sub-range that the MDS that storage capacity is larger distributes is larger.
The meta data server cluster that still consists of take above-mentioned 4 MDS is as example, if newly adding 2 storage capacities is all meta data server MDS_4 and the MDS_5 of 8GB, at this moment, the storage capacity of meta data server cluster has been increased to 32GB by original 16GB, the storage capacity of original MDS_0 to MDS_3 becomes 1/8 with the ratio of total storage capacity by 1/4, institute namely, the independent sub-range that MDS_0 to MDS_3 distributes all will dwindle half:
The independent sub-range that MDS_0 distributes is by [0,2
126-1) become [0,2
125-1), remaining independent sub-range [2
125, 2
126-1) become unappropriated independent sub-range;
The independent sub-range that MDS_1 distributes is by [2
126, 2
127-1) become [2
126, 3*2
125-1), remaining independent sub-range [3*2
125, 2
127-1) become unappropriated independent sub-range;
The independent sub-range that MDS_2 distributes is by [2
127, 3*2
126-1) become [2
127, 5*2
125-1), remaining independent sub-range [5*2
125, 3*2
126-1) become unappropriated independent sub-range;
The independent sub-range that MDS_3 distributes is by [3*2
126, 2
128-1) become [3*2
126, 7*2
125-1), remaining independent sub-range [7*2
125, 2
128-1) become unappropriated independent sub-range.
At this moment, the independent sub-range that MDS_0 to MDS_3 distributes and total interval represent as shown in Figure 3 have 4 sections independent sub-ranges to distribute to MDS_0 to MDS_3 in total interval with closed loop, remain 4 sections independent sub-ranges also unallocated.
Because the storage capacity of 2 MDS that newly add is identical, the storage capacity of MDS_4 and MDS_5 is identical with the ratio of total storage capacity, therefore, remain 4 sections also unappropriated independent sub-range be averagely allocated to MDS_4 and MDS_5, distribute schematic diagram as shown in Figure 4, complete the adjustment in the independent sub-range that meta data server is distributed.Need to prove, a MDS can only be distributed in an independent sub-range, but a MDS can distribute a plurality of independent sub-ranges.
2, shift out the situation of meta data server in the meta data server cluster.
Owing to having shifted out meta data server in the meta data server cluster, total storage capacity of meta data server cluster will reduce, therefore, the storage capacity of meta data server originally will become large with the ratio that shifts out the total storage capacity after meta data server, originally distribute to the independent sub-range of shifting out meta data server and will become unappropriated independent sub-range, according to the size of residue meta data server storage capacity, remaining meta data server is distributed in described unappropriated independent sub-range.
The meta data server cluster that still consists of take above-mentioned 4 MDS is as example, if shift out MDS_3, at this moment, the storage capacity of meta data server cluster is reduced to 12GB by original 16GB, because the storage capacity of MDS_0 to MDS_2 is identical, therefore, originally distributed to the independent sub-range of MDS_3 by [3*2
126, 2
128-1) will on average be divided into 3 parts, distribute to MDS_0 to MDS_2, at this moment, distribute schematic diagram as shown in Figure 5, complete the adjustment in the independent sub-range that meta data server is distributed.
No matter newly add MDS in the meta data server cluster or shift out MDS, after the independent sub-range adjustment that each MDS in the meta data server cluster distributes is completed, may exist some independent sub-range to distribute to the situation of different MDS before and after adjusting, in the case, operation result belongs to the MDS that the metadata in this class independence sub-range distributes and also needs to change, and the MDS that metadata is distributed before from this independent sub-range migrates to the MDS of this current distribution in independent sub-range.
When the quantity of MDS changes in the meta data server cluster, the independent sub-range that MDS originally distributes not is all to change, but variation has occured in a part of independent sub-range, therefore, operation result belongs to the metadata in the independent sub-range that does not change and need to not move between MDS, with respect to existing hash method, can effectively reduce the migration amount of metadata between MDS, and then reduce the expense of network operation.
Embodiment two:
As shown in Figure 6, be the structural representation of the equipment of meta-data distribution in a kind of object storage system of the present embodiment one, this equipment comprises: distribution module 11, determination module 12 and distribution module 13, wherein:
Because the scale of object storage system is constantly to change, the scale of Metadata Service cluster also can change, and therefore, described distribution module 11 also is used for:
When newly adding meta data server in the meta data server cluster, dwindle the scope in the independent sub-range of original meta data server distribution, obtain being not yet assigned to the independent sub-range of meta data server, and the meta data server that newly adds is distributed in unappropriated independent sub-range, wherein: newly add the storage capacity of meta data server larger, unappropriated independent sub-range is larger; Perhaps,
When shifting out meta data server in the meta data server cluster, according to the size of residue meta data server storage capacity, remaining meta data server is distributed in the independent sub-range that the meta data server that shifts out distributes.
no matter newly add MDS in the meta data server cluster or shift out MDS, after the independent sub-range adjustment that each MDS in the meta data server cluster distributes is completed, exist some independent sub-range to distribute to the situation of different MDS before and after adjusting, therefore, this equipment, also comprise transferring module 14, wherein, described transferring module, be used for when the meta data server of the independent sub-range distribution under the operation result of metadata after the globally unique identifier of object storage system carries out computing changes, the meta data server that this metadata is distributed before from this independent sub-range migrates to the meta data server of this current distribution in independent sub-range.
Obviously, those skilled in the art can carry out various changes and modification and not break away from the spirit and scope of the present invention the present invention.Like this, if within of the present invention these are revised and modification belongs to the scope of claim of the present invention and equivalent technologies thereof, the present invention also is intended to comprise these changes and modification interior.
Claims (10)
1. the method for meta-data distribution in an object storage system, is characterized in that, the method comprises:
According to the size of each meta data server storage capacity in the meta data server cluster, for each meta data server distributes independent sub-range, wherein, the independent sub-range that the meta data server that storage capacity is larger distributes is larger;
Determine the operation result after the globally unique identifier of metadata in object storage system carries out computing, and according to default operation result and the mapping relations between described independent sub-range, determine the independent sub-range that this operation result is affiliated;
Meta data server corresponding to described independent sub-range that described meta-data distribution is extremely definite.
2. the method for claim 1, is characterized in that, described globally unique identifier is the memory address of object under metadata.
3. the method for claim 1, is characterized in that, the method also comprises:
When newly adding meta data server in the meta data server cluster, dwindle the scope in the independent sub-range of original meta data server distribution, obtain being not yet assigned to the independent sub-range of meta data server, and the meta data server that newly adds is distributed in unappropriated independent sub-range, wherein: newly add the storage capacity of meta data server larger, unappropriated independent sub-range is larger;
When shifting out meta data server in the meta data server cluster, according to the size of residue meta data server storage capacity, remaining meta data server is distributed in the independent sub-range that the meta data server that shifts out distributes.
4. method as claimed in claim 3, is characterized in that, the method also comprises:
Globally unique identifier when metadata in object storage system carries out meta data server corresponding to independent sub-range under operation result after computing when changing, with this metadata from this independent sub-range before corresponding meta data server migrate to the meta data server of this current correspondence in independent sub-range.
5. described method as arbitrary in claim 1～4, is characterized in that, non-intersect for the independent sub-range that any two meta data servers distribute, and the independent sub-range sum that all meta data servers distribute is default total interval.
6. the equipment of meta-data distribution in an object storage system, is characterized in that, this equipment comprises:
Distribution module is used for the size according to each meta data server storage capacity of meta data server cluster, and for each meta data server distributes independent sub-range, wherein, the independent sub-range that the meta data server that storage capacity is larger distributes is larger;
Determination module is used for determining the operation result of metadata after the globally unique identifier of object storage system carries out computing, and the independent sub-range under this operation result;
Distribution module is used for described meta-data distribution to meta data server corresponding to described independent sub-range of determining.
7. equipment as claimed in claim 6, is characterized in that, described distribution module also is used for:
When newly adding meta data server in the meta data server cluster, dwindle the scope in the independent sub-range of original meta data server distribution, obtain being not yet assigned to the independent sub-range of meta data server, and the meta data server that newly adds is distributed in unappropriated independent sub-range, wherein: newly add the storage capacity of meta data server larger, unappropriated independent sub-range is larger.
8. equipment as claimed in claim 6, is characterized in that, described distribution module also is used for:
When shifting out meta data server in the meta data server cluster, according to the size of residue meta data server storage capacity, remaining meta data server is distributed in the independent sub-range that the meta data server that shifts out distributes.
9. equipment as claimed in claim 7 or 8, is characterized in that, this equipment also comprises:
Transferring module, be used for when the meta data server of the independent sub-range distribution under the operation result of metadata after the globally unique identifier of object storage system carries out computing changes, the meta data server that this metadata is distributed before from this independent sub-range migrates to the meta data server of this current distribution in independent sub-range.
10. as claim 6,7 or 8 described equipment, it is characterized in that, described distribution module is that the independent sub-range of any two meta data servers distribution is non-intersect, and the independent sub-range sum that all meta data servers distribute is default total interval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110355678.8A CN103106207B (en) | 2011-11-10 | The method and apparatus of meta-data distribution in a kind of object storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110355678.8A CN103106207B (en) | 2011-11-10 | The method and apparatus of meta-data distribution in a kind of object storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103106207A true CN103106207A (en) | 2013-05-15 |
| CN103106207B CN103106207B (en) | 2016-12-14 |
Family
ID=
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103647797A (en) * | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
| CN104461403A (en) * | 2014-12-29 | 2015-03-25 | 北京网秦天下科技有限公司 | Method and device for operating target storage system by means of command line of Linux |
| CN106446197A (en) * | 2016-09-30 | 2017-02-22 | 华为数字技术（成都）有限公司 | Data storage method, device and system |
| WO2018000993A1 (en) * | 2016-06-29 | 2018-01-04 | 华为技术有限公司 | Distributed storage method and system |
| CN107920101A (en) * | 2016-10-10 | 2018-04-17 | 阿里巴巴集团控股有限公司 | A kind of file access method, device, system and electronic equipment |
| CN109445694A (en) * | 2018-10-19 | 2019-03-08 | 郑州云海信息技术有限公司 | A kind of distributed memory system separated from meta-data method and apparatus |
| WO2019148841A1 (en) * | 2018-01-31 | 2019-08-08 | 华为技术有限公司 | Distributed storage system, data processing method and storage node |
| CN114780252A (en) * | 2022-06-15 | 2022-07-22 | 阿里云计算有限公司 | Resource management method and device of data warehouse system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
| CN102055650A (en) * | 2009-10-29 | 2011-05-11 | 华为技术有限公司 | Load balance method and system and management server |
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102055650A (en) * | 2009-10-29 | 2011-05-11 | 华为技术有限公司 | Load balance method and system and management server |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103647797A (en) * | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
| CN104461403A (en) * | 2014-12-29 | 2015-03-25 | 北京网秦天下科技有限公司 | Method and device for operating target storage system by means of command line of Linux |
| WO2018000993A1 (en) * | 2016-06-29 | 2018-01-04 | 华为技术有限公司 | Distributed storage method and system |
| CN106446197A (en) * | 2016-09-30 | 2017-02-22 | 华为数字技术（成都）有限公司 | Data storage method, device and system |
| CN106446197B (en) * | 2016-09-30 | 2019-11-19 | 华为数字技术（成都）有限公司 | A kind of date storage method, apparatus and system |
| CN107920101A (en) * | 2016-10-10 | 2018-04-17 | 阿里巴巴集团控股有限公司 | A kind of file access method, device, system and electronic equipment |
| US11262916B2 (en) | 2018-01-31 | 2022-03-01 | Huawei Technologies Co., Ltd. | Distributed storage system, data processing method, and storage node |
| WO2019148841A1 (en) * | 2018-01-31 | 2019-08-08 | 华为技术有限公司 | Distributed storage system, data processing method and storage node |
| CN109445694A (en) * | 2018-10-19 | 2019-03-08 | 郑州云海信息技术有限公司 | A kind of distributed memory system separated from meta-data method and apparatus |
| CN109445694B (en) * | 2018-10-19 | 2022-02-18 | 郑州云海信息技术有限公司 | Metadata separation method and device for distributed storage system |
| CN114780252A (en) * | 2022-06-15 | 2022-07-22 | 阿里云计算有限公司 | Resource management method and device of data warehouse system |
| CN114780252B (en) * | 2022-06-15 | 2022-11-18 | 阿里云计算有限公司 | Resource management method and device of data warehouse system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102567495B (en) | Mass information storage system and implementation method | |
| CN103793534B (en) | Distributed file system and balanced metadata storage and the implementation method for accessing load | |
| CN102819599A (en) | Method for constructing hierarchical catalogue based on consistent hashing data distribution | |
| CN102033938B (en) | Secondary mapping-based cluster dynamic expansion method | |
| CN103310000B (en) | Metadata management method | |
| CN102013991B (en) | Method, management equipment and system for automatically expanding capacity | |
| CN107667363A (en) | Object-based storage cluster with plurality of optional data processing policy | |
| CN102546823A (en) | File storage management system of cloud storage system | |
| CN104317742A (en) | Thin provisioning method for optimizing space management | |
| CN101944124A (en) | Distributed file system management method, device and corresponding file system | |
| CN105389367B (en) | The multilevel distributed storage method of grid graph multi-time Scales based on Mongo databases | |
| CN103294785B (en) | A kind of packet-based metadata server cluster management method | |
| CN102999571B (en) | The implementation method of unit multinode in a kind of cluster | |
| CN102955845A (en) | Data access method and device as well as distributed database system | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN106095589A (en) | Partition allocation method, device and system | |
| CN106953910A (en) | A kind of Hadoop calculates storage separation method | |
| CN105354250A (en) | Data storage method and device for cloud storage | |
| CN106713250A (en) | Data access method and device based on distributed system | |
| CN106354890A (en) | Implementation method of random access file system based on N-ary tree structure | |
| CN101419600A (en) | Data copy mapping method and device based on object-oriented LANGUAGE | |
| CN106991190A (en) | A kind of database automatically creates subdata base system | |
| CN103778120A (en) | Global file identification generation method, generation device and corresponding distributed file system | |
| CN105468541A (en) | Cache management method for transparent-computing-oriented intelligent terminal | |
| CN105159925A (en) | Database cluster data distribution method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |