CN102411637A - Metadata management method of distributed file system - Google Patents
Metadata management method of distributed file system Download PDFInfo
- Publication number
- CN102411637A CN102411637A CN2011104542621A CN201110454262A CN102411637A CN 102411637 A CN102411637 A CN 102411637A CN 2011104542621 A CN2011104542621 A CN 2011104542621A CN 201110454262 A CN201110454262 A CN 201110454262A CN 102411637 A CN102411637 A CN 102411637A
- Authority
- CN
- China
- Prior art keywords
- metadata
- meta data
- data server
- collection
- copy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a metadata management method of a distributed file system, which is used for managing metadata on the basis of a metadata server cluster and can ensure consistency of the metadata under concurrent access and load balance of the cluster. By adopting the invention, demands of concurrent access of a plurality of client ends can be effectively met.
Description
Technical field
The present invention relates to memory technology, particularly relate to a kind of metadata management method of distributed file system.
Background technology
In the existing distributed file system, adopt single meta data server that metadata is managed usually.But along with the growth of quantity of documents, single meta data server has been difficult to satisfy the demand of the concurrent visit of a large amount of clients.To this situation, proposed at present in distributed file system, to introduce the meta data server cluster concept, promptly utilize a plurality of meta data servers that metadata is managed.
At present still unexposedly a kind ofly in distributed file system, realize technical scheme to metadata management based on the meta data server cluster.
Summary of the invention
In view of this, fundamental purpose of the present invention is to provide a kind of metadata management method of distributed file system, and this method can satisfy the demand of the concurrent visit of a large amount of clients.
In order to achieve the above object, the technical scheme of the present invention's proposition is:
A kind of metadata management method of distributed file system, this method may further comprise the steps:
A, distributed file system are set up M collection of metadata in advance, according to the first preset hash function, adopt the mode of Hash hash that said collection of metadata is disperseed to be arranged in each meta data server in the metadata service cluster.Said meta data server is preserved a global set mapping table separately, and mapping relations and all that said global set mapping table is used for the metadata that each collection of metadata that the minute book server is provided with comprises with it are not arranged on each collection of metadata and the mapping relations of meta data server under it in the book server;
B, when file is preserved in the meta data server S_1 request of client in the metadata service cluster; Said meta data server S_1 is global index of said file allocation number; And according to this global index number; Adopt the mode of Hash hash, confirm the affiliated collection of metadata W of metadata of said file, and this metadata leaves the index node position D among the said collection of metadata W in; The global set mapping table that said meta data server S_1 preserves according to self is confirmed the meta data server S_i at said collection of metadata W place, triggers said meta data server S_i the metadata of said file is kept among the said meta data server S_i.
In sum; The metadata management method of the distributed file system that the present invention proposes; Through metadata being managed based on the meta data server cluster; And guarantee the consistance of metadata under the concurrent visit, the load balancing of cluster, therefore, can effectively satisfy the demand of the concurrent visit of a large amount of clients.
Description of drawings
Fig. 1 is the schematic flow sheet of the embodiment of the invention one.
Embodiment
For making the object of the invention, technical scheme and advantage clearer, will combine accompanying drawing and specific embodiment that the present invention is done to describe in detail further below.
Core concept of the present invention is: based on the meta data server cluster metadata is managed, when storing metadata, utilize hash function to confirm the address of storing metadata file, guarantee the load balancing of cluster; When carrying out metadata store, simultaneously the corresponding wave file of storage in other servers is also made amendment to its copy when metadata is revised, when a server breaks down, carries out the Coordination Treatment of fault through the server that is used to coordinate preset.So, can guarantee the load balancing of consistance, integrality and the cluster server of metadata under the concurrent visit.
Fig. 1 is the schematic flow sheet of the embodiment of the invention one, and is as shown in Figure 1, and this embodiment one mainly may further comprise the steps:
Said metadata (metadata); Be meant the metadata information of file or catalogue; Comprise the information such as name, size, modification time, access limit, the owner of file, for concrete file or catalogue, (inode) preserves these metadata informations by index node.Here; Said meta data server is preserved a global set mapping table separately; Said global set mapping table is used for the mapping relations of the metadata that each collection of metadata that the minute book server is provided with comprises with it, and all are not arranged on each collection of metadata and the mapping relations of meta data server under it in the book server.Particularly; The global set mapping table will be preserved the mapping relations of the global index number of file to affiliated collection of metadata; Particularly; As Hash table,, search the positional information that this global set mapping table can obtain collection of metadata under this file metadata with this global set mapping table with the high H position of file global index number parameter as hash function.
Said collection of metadata (set); By the aggregation units that the metadata of many files is formed, will preserve state and the metadata mapping table of this set of set in the set, this metadata mapping table is used for writing down the global index number of each file of collection of metadata and the corresponding relation of its index node position; Particularly; With this metadata mapping table is Hash table, with the low L position of the file global index number parameter as hash function, searches this metadata mapping table; Can obtain the position of index node at the metadata place of this document, on the position of this index node with the metadata of in store file.
In this step,, can realize effectively that metadata set is combined in the equiblibrium mass distribution on each meta data server, help realizing the load balancing of cluster under the concurrent visit through the mode of Hash hash.
Here, said global index number is the unique identification of the metadata of each file in the distributed file system, is used for locating the set at this document metadata place and in the concrete preservation position of set.
Preferably, the collection of metadata W under the said metadata of confirming said file, and this metadata leave among the said collection of metadata W index node position D specifically can for:
Said global index number is divided into high H position and low L position two parts, wherein, L+H=Z, said Z is the number of bits of preset said global index number;
For the high H position of said global index number, carry out hash calculation through the second preset hash function, utilize resulting cryptographic hash to search said global set mapping table, obtain the positional information of said collection of metadata W; According to the positional information of said collection of metadata W, find said collection of metadata W;
For the low L position of said global index number, carry out hash calculation through the 3rd preset hash function, with resultant cryptographic hash as said index node position D.
Here, the setting of said H and L can be provided with by those skilled in the art according to actual needs, preferably, can be: H=16, L=16.
Preferably, with the metadata of said file be kept among the said meta data server S_i specifically can for:
Meta data server S_i is the allocation index node on the D of index node position, and the metadata of said file is kept in the index node of the said index node position D in the book server;
The mapping relations of the global index of meta data server S_i said metadata of record in the said global set mapping table that self preserves number and the position of said collection of metadata W;
The mapping relations of the global index of meta data server S_i said metadata of record in the metadata mapping table of this collection of metadata W that self preserves number and the index node of said metadata.
Further, can also realize the backup of data,, particularly, can further comprise in the said step 101 with safety and the integrality of guaranteeing data through set up the mode of copy for each collection of metadata:
Said distributed file system is that a said M collection of metadata is set up the copy set respectively; And according to the principle of load balancing; The copy set of a said M collection of metadata is arranged in each meta data server in the said metadata service cluster; And it is inequality to guarantee that said collection of metadata and its copy are gathered the place meta data server; Preserve the former copy mapping table of a set in said each meta data server respectively; The former copy mapping table of said set is used for preserving each collection of metadata that is arranged on book server and belongs to the mapping relations of meta data server with its copy set, and each copy that is arranged in the book server is gathered the mapping relations that each collection of metadata corresponding with it belongs to meta data server originally.
After setting up that metadata set is fashionable and setting up copy set for it, operations such as the follow-up storage of carrying out file, visit the time, also need carry out synchronous operation, to guarantee the consistance of data to the copy set.Particularly, when carrying out the preservation of file in the step 102, also need carry out following steps:
Said meta data server S_i is according to the former copy mapping table of self preserving of set; Confirm the meta data server W_copy at the copy set place of said collection of metadata W, trigger said meta data server W_copy preserves said file in the copy set of said collection of metadata W metadata.
Here; The concrete grammar of in the copy set, preserving metadata is identical with aforementioned metadata store method to its former set; The meta data server of promptly depositing copy set also need be safeguarded the mapping table of mapping table, a copy that a copy set and its institute comprise the global index number of metadata index node address and file global index in gathering number; When carrying out the preservation of copy set, then need on these mapping tables, write down corresponding information.
Further, client can conduct interviews to file after being kept at file in the distributed file system as required; At this moment; Also need carry out corresponding process operations to the metadata of this document, particularly, following method capable of using realizes the visit to metadata:
When client need ask to visit the file F of appointment, described request is sent to the arbitrary meta data server A in the said metadata service cluster, carry the ino of global index of said file F and the type of request visit in the described request;
Said meta data server A is according to the said ino of global index; Confirm the affiliated collection of metadata J of metadata of said file F; According to the said global set mapping table of self preserving; Confirm the meta data server S_J at said collection of metadata J place, if this meta data server S_J is a book server, the metadata mapping table of the said collection of metadata J of the type of then visiting, the said ino of global index and self preservation according to described request; Visit the metadata of said file F and carry out corresponding process operations; And, confirm the meta data server at the copy set place of said collection of metadata J according to the former copy mapping table of self preserving of said set, the copy set that said processing operation is synchronized to the said collection of metadata J on this meta data server closes; Otherwise; The meta data server S_J at said collection of metadata J place notifies to said client; Trigger said client and send described request to said meta data server S_J; Said meta data server S_J is according to the metadata mapping table of the said collection of metadata J of the type of described request visit, the said ino of global index and self preservation; Visit the metadata of said file F and carry out corresponding process operations; And, confirm the meta data server at the copy set place of said collection of metadata J according to the former copy mapping table of self preserving of said set, the copy set that said processing operation is synchronized to the said collection of metadata J on this meta data server closes.
Here,, confirm the concrete grammar of the collection of metadata J under the metadata of said file F according to the said ino of global index, ditto said.
According to the metadata mapping table of the type of described request visit, the said ino of global index and the said collection of metadata J that self preserves, the metadata of visiting said file F specifically can for:
Metadata mapping table with collection of metadata J is a Hash table; Low L position with the said ino of global index is the parameter of said the 3rd hash function; Carry out hash calculation, utilize resulting cryptographic hash to search said metadata mapping table, obtain the position of the corresponding index node of the metadata of said file F; Can find the metadata of said file F based on this position, and can operate accordingly according to the type of described request visit.Concrete access type and the same existing system of corresponding process operations repeat no more at this.
Preferably; Can preset a coordination server in the metadata service cluster of said distributed file system; This coordination server can be monitored each meta data server in the said metadata service cluster, so that know out of order meta data server immediately through sending heartbeat message termly; And then can in time carry out fault handling, guarantee the integrality that distributed file system is preserved metadata.Concrete grammar is following:
When said coordination server monitors arbitrary meta data server B when breaking down, said failure message is notified to other each meta data servers except that said meta data server B in the said metadata service cluster;
Receive each meta data server of said notice; According to said notice and the former copy mapping table of self preserving of said set; Judge the copy set of self whether having preserved the collection of metadata on the said meta data server B; If, from other meta data servers that other do not break down, confirm to be used to preserve the new meta data server of said copy set then according to the principle of load balance; Trigger said new meta data server said copy set is preserved, and said copy set cooperation is arranged in the book server for its corresponding set originally.
Preferably, a said M collection of metadata with and copy set close also and can set up by above-mentioned coordination server.
Preferably, can be through a meta data server that adopts electoral machinery to elect in the meta data server in the metadata service cluster, as said coordination server.Said coordination server is a service routine, and itself does not preserve metadata information, and it accepts the meta data server registration, and passes through the status information of the dynamic generator data server of the log-on message cluster of meta data server.More excellent ground is seen from the angle of disposing, this coordination server can with the shared same hardware server of any meta data server.
From technique scheme, can find out; Adopt the present invention can guarantee the load balancing of cluster under the consistance of metadata under the concurrent visit, the concurrent visit, when single metadata node breaks down; Information synchronization problem when metadata integrity and metadata access; Thereby can satisfy the demand of the concurrent visit of a large amount of clients, realize based on effective management of server cluster to metadata in the distributed file system.
In sum, more than being merely preferred embodiment of the present invention, is not to be used to limit protection scope of the present invention.All within spirit of the present invention and principle, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. the metadata management method of a distributed file system is characterized in that, this method may further comprise the steps:
A, distributed file system are set up M collection of metadata in advance, according to the first preset hash function, adopt the mode of Hash hash that said collection of metadata is disperseed to be arranged in each meta data server in the metadata service cluster.Said meta data server is preserved a global set mapping table separately, and mapping relations and all that said global set mapping table is used for the metadata that each collection of metadata that the minute book server is provided with comprises with it are not arranged on each collection of metadata and the mapping relations of meta data server under it in the book server;
B, when file is preserved in the meta data server S_1 request of client in the metadata service cluster; Said meta data server S_1 is global index of said file allocation number; And according to this global index number; Adopt the mode of Hash hash, confirm the affiliated collection of metadata W of metadata of said file, and this metadata leaves the index node position D among the said collection of metadata W in; The global set mapping table that said meta data server S_1 preserves according to self is confirmed the meta data server S_i at said collection of metadata W place, triggers said meta data server S_i the metadata of said file is kept among the said meta data server S_i.
2. method according to claim 1 is characterized in that, confirm the affiliated collection of metadata W of metadata of said file described in the step b, and the index node position D that this metadata leaves among the said collection of metadata W is:
Said global index number is divided into high H position and low L position two parts, wherein, L+H=Z, said Z is the number of bits of preset said global index number;
For the high H position of said global index number, carry out hash calculation through the second preset hash function, utilize resulting cryptographic hash to search said global set mapping table, obtain the positional information of said collection of metadata W; According to the positional information of said collection of metadata W, find said collection of metadata W;
For the low L position of said global index number, carry out hash calculation through the 3rd preset hash function, with resultant cryptographic hash as said index node position D.
3. method according to claim 1 is characterized in that, among the step b metadata of said file is kept among the said meta data server S_i to comprise:
Meta data server S_i is the allocation index node on the D of index node position, and the metadata of said file is kept in the index node of the said index node position D in the book server;
The mapping relations of the global index of meta data server S_i said metadata of record in the said global set mapping table that self preserves number and the position of said collection of metadata W;
The mapping relations of the global index of meta data server S_i said metadata of record in the metadata mapping table of this collection of metadata W that self preserves number and the index node of said metadata.
4. method according to claim 1 is characterized in that, further comprises among the said step a:
Said distributed file system is that a said M collection of metadata is set up the copy set respectively; And according to the principle of load balancing; The copy set of a said M collection of metadata is arranged in each meta data server in the said metadata service cluster; And it is inequality to guarantee that said collection of metadata and its copy are gathered the place meta data server; Preserve the former copy mapping table of a set in said each meta data server respectively, the former copy mapping table of said set is used for preserving each collection of metadata that is arranged on book server and gathers the mapping relations that each collection of metadata corresponding with it belongs to meta data server originally with the mapping relations of its copy set place meta data server and each copy that is arranged in the book server.
5. method according to claim 4 is characterized in that, further comprises among the said step b:
Said meta data server S_i is according to the former copy mapping table of self preserving of set; Confirm the meta data server W_copy at the copy set place of said collection of metadata W, trigger said meta data server W_copy preserves said file in the copy set of said collection of metadata W metadata.
6. method according to claim 5 is characterized in that, said method further comprises:
When client need ask to visit the file F of appointment, described request is sent to the arbitrary meta data server A in the said metadata service cluster, carry the ino of global index of said file F and the type of request visit in the described request;
Said meta data server A is according to the said ino of global index; Confirm the affiliated collection of metadata J of metadata of said file F; According to the said global set mapping table of self preserving; Confirm the meta data server S_J at said collection of metadata J place, if this meta data server S_J is a book server, the metadata mapping table of the said collection of metadata J of the type of then visiting, the said ino of global index and self preservation according to described request; Visit the metadata of said file F and carry out corresponding process operations; And, confirm the meta data server at the copy set place of said collection of metadata J according to the former copy mapping table of self preserving of said set, the copy set that said processing operation is synchronized to the said collection of metadata J on this meta data server closes; Otherwise; The meta data server S_J at said collection of metadata J place notifies to said client; Trigger said client and send described request to said meta data server S_J; Said meta data server S_J is according to the metadata mapping table of the said collection of metadata J of the type of described request visit, the said ino of global index and self preservation; Visit the metadata of said file F and carry out corresponding process operations; And, confirm the meta data server at the copy set place of said collection of metadata J according to the former copy mapping table of self preserving of said set, the copy set that said processing operation is synchronized to the said collection of metadata J on this meta data server closes.
7. method according to claim 5 is characterized in that, said method further comprises:
A preset coordination server in the said metadata service cluster, said coordination server is monitored each meta data server in the said metadata service cluster through sending heartbeat message;
When said coordination server monitors arbitrary meta data server B when breaking down, said failure message is notified to other each meta data servers except that said meta data server B in the said metadata service cluster;
Receive each meta data server of said notice; According to said notice and the former copy mapping table of self preserving of said set; Judge the copy set of self whether having preserved the collection of metadata on the said meta data server B; If, from other meta data servers that other do not break down, confirm to be used to preserve the new meta data server of said copy set then according to the principle of load balance; Trigger said new meta data server said copy set is preserved, and said copy set cooperation is arranged in the book server for its corresponding set originally.
8. method according to claim 1 is characterized in that, a said M collection of metadata is set up by a coordination server preset in the said distributed file system.
9. method according to claim 2 is characterized in that, the copy set of a said M collection of metadata is set up by a coordination server preset in the said distributed file system.
10. according to claim 7,8 or 9 described methods, it is characterized in that the meta data server of said coordination server for electing through electoral machinery in the meta data server in the metadata service cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110454262 CN102411637B (en) | 2011-12-30 | 2011-12-30 | Metadata management method of distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110454262 CN102411637B (en) | 2011-12-30 | 2011-12-30 | Metadata management method of distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102411637A true CN102411637A (en) | 2012-04-11 |
| CN102411637B CN102411637B (en) | 2013-07-24 |
Family
ID=45913708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110454262 Active CN102411637B (en) | 2011-12-30 | 2011-12-30 | Metadata management method of distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102411637B (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102694863A (en) * | 2012-05-30 | 2012-09-26 | 电子科技大学 | Realization method of distributed storage system on basis of load adjustment and system fault tolerance |
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN103095687A (en) * | 2012-12-19 | 2013-05-08 | 华为技术有限公司 | Method and device of metadata processing |
| CN103106286A (en) * | 2013-03-04 | 2013-05-15 | 曙光信息产业(北京)有限公司 | Method and device for managing metadata |
| CN103218404A (en) * | 2013-03-20 | 2013-07-24 | 华中科技大学 | Multi-dimensional metadata management method and system based on association characteristics |
| CN103324466A (en) * | 2013-05-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Data dependency serialization IO parallel processing method |
| CN103544156A (en) * | 2012-07-10 | 2014-01-29 | 腾讯科技（深圳）有限公司 | Method and device for storing files |
| CN103631820A (en) * | 2012-08-27 | 2014-03-12 | 阿里巴巴集团控股有限公司 | Metadata management method and device of distributed file system |
| CN103778120A (en) * | 2012-10-17 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Global file identification generation method, generation device and corresponding distributed file system |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| WO2015055035A1 (en) * | 2013-10-18 | 2015-04-23 | 华为技术有限公司 | Method and device for hashing metadata object |
| CN104657115A (en) * | 2015-03-12 | 2015-05-27 | 浪潮集团有限公司 | Cluster file system client-side multi-core concurrence and load implementation method |
| CN104866497A (en) * | 2014-02-24 | 2015-08-26 | 华为技术有限公司 | Metadata updating method and device based on column storage of distributed file system as well as host |
| CN105005588A (en) * | 2015-06-26 | 2015-10-28 | 深圳市腾讯计算机系统有限公司 | Training data processing method and apparatus |
| WO2016202199A1 (en) * | 2015-06-18 | 2016-12-22 | 阿里巴巴集团控股有限公司 | Distributed file system and file meta-information management method thereof |
| CN107547257A (en) * | 2017-07-14 | 2018-01-05 | 新华三技术有限公司 | A kind of server cluster implementation method and device |
| CN107817951A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of method and device for realizing the fusion of Ceph clusters |
| CN108667836A (en) * | 2018-05-08 | 2018-10-16 | 众安信息技术服务有限公司 | Block chain common recognition method |
| WO2019000949A1 (en) * | 2017-06-28 | 2019-01-03 | 华为技术有限公司 | Metadata storage method and system in distributed storage system, and storage medium |
| CN109802986A (en) * | 2017-11-17 | 2019-05-24 | 华为技术有限公司 | Device management method, system, device and server |
| CN112395281A (en) * | 2020-12-10 | 2021-02-23 | 太极计算机股份有限公司 | Heterogeneous multi-source data fusion system |
| CN113239013A (en) * | 2021-05-17 | 2021-08-10 | 北京青云科技股份有限公司 | Distributed system and storage medium |
| CN114945023A (en) * | 2022-05-20 | 2022-08-26 | 济南浪潮数据技术有限公司 | Network connection multiplexing method, device, equipment and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
-
2011
- 2011-12-30 CN CN 201110454262 patent/CN102411637B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101957861A (en) * | 2010-10-18 | 2011-01-26 | 江苏大学 | Novel metadata server cluster and metadata management method based on reconciliation statement |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102694863A (en) * | 2012-05-30 | 2012-09-26 | 电子科技大学 | Realization method of distributed storage system on basis of load adjustment and system fault tolerance |
| CN102694863B (en) * | 2012-05-30 | 2015-08-26 | 电子科技大学 | Based on the implementation method of the distributed memory system of adjustment of load and System Fault Tolerance |
| CN103544156A (en) * | 2012-07-10 | 2014-01-29 | 腾讯科技（深圳）有限公司 | Method and device for storing files |
| CN103544156B (en) * | 2012-07-10 | 2019-04-09 | 腾讯科技（深圳）有限公司 | File memory method and device |
| CN102855284B (en) * | 2012-08-03 | 2016-08-10 | 北京联创信安科技股份有限公司 | The data managing method of a kind of cluster storage system and system |
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN103631820A (en) * | 2012-08-27 | 2014-03-12 | 阿里巴巴集团控股有限公司 | Metadata management method and device of distributed file system |
| CN103631820B (en) * | 2012-08-27 | 2017-09-12 | 阿里巴巴集团控股有限公司 | The metadata management method and equipment of distributed file system |
| CN103778120A (en) * | 2012-10-17 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Global file identification generation method, generation device and corresponding distributed file system |
| CN103778120B (en) * | 2012-10-17 | 2018-01-16 | 腾讯科技（深圳）有限公司 | Global profile mark generating method, generating means and corresponding distributed file system |
| CN103095687B (en) * | 2012-12-19 | 2015-08-26 | 华为技术有限公司 | metadata processing method and device |
| CN103095687A (en) * | 2012-12-19 | 2013-05-08 | 华为技术有限公司 | Method and device of metadata processing |
| CN103106286A (en) * | 2013-03-04 | 2013-05-15 | 曙光信息产业(北京)有限公司 | Method and device for managing metadata |
| CN103106286B (en) * | 2013-03-04 | 2017-02-01 | 曙光信息产业(北京)有限公司 | Method and device for managing metadata |
| CN103218404A (en) * | 2013-03-20 | 2013-07-24 | 华中科技大学 | Multi-dimensional metadata management method and system based on association characteristics |
| CN103218404B (en) * | 2013-03-20 | 2015-11-18 | 华中科技大学 | A kind of multi-dimensional metadata management method based on associate feature and system |
| CN103324466B (en) * | 2013-05-24 | 2017-05-03 | 浪潮电子信息产业股份有限公司 | Data dependency serialization IO parallel processing method |
| CN103324466A (en) * | 2013-05-24 | 2013-09-25 | 浪潮电子信息产业股份有限公司 | Data dependency serialization IO parallel processing method |
| WO2015055035A1 (en) * | 2013-10-18 | 2015-04-23 | 华为技术有限公司 | Method and device for hashing metadata object |
| CN104866497B (en) * | 2014-02-24 | 2018-06-15 | 华为技术有限公司 | The metadata updates method, apparatus of distributed file system column storage, host |
| WO2015124042A1 (en) * | 2014-02-24 | 2015-08-27 | 华为技术有限公司 | Method, device and host for updating metadata stored in columns in distributed file system |
| CN104866497A (en) * | 2014-02-24 | 2015-08-26 | 华为技术有限公司 | Metadata updating method and device based on column storage of distributed file system as well as host |
| US9678969B2 (en) | 2014-02-24 | 2017-06-13 | Huawei Technologies Co., Ltd. | Metadata updating method and apparatus based on columnar storage in distributed file system, and host |
| CN103793534B (en) * | 2014-02-28 | 2017-09-08 | 苏州博纳讯动软件有限公司 | Distributed file system and balanced metadata storage and the implementation method for accessing load |
| CN103793534A (en) * | 2014-02-28 | 2014-05-14 | 苏州博纳讯动软件有限公司 | Distributed file system and implementation method for balancing storage loads and access loads of metadata |
| CN104657115B (en) * | 2015-03-12 | 2017-04-19 | 浪潮集团有限公司 | Cluster file system client-side multi-core concurrence and load implementation method |
| CN104657115A (en) * | 2015-03-12 | 2015-05-27 | 浪潮集团有限公司 | Cluster file system client-side multi-core concurrence and load implementation method |
| CN106326239A (en) * | 2015-06-18 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Distributed file system and file meta-information management method thereof |
| WO2016202199A1 (en) * | 2015-06-18 | 2016-12-22 | 阿里巴巴集团控股有限公司 | Distributed file system and file meta-information management method thereof |
| CN106326239B (en) * | 2015-06-18 | 2020-01-31 | 阿里巴巴集团控股有限公司 | Distributed file system and file meta-information management method thereof |
| CN105005588A (en) * | 2015-06-26 | 2015-10-28 | 深圳市腾讯计算机系统有限公司 | Training data processing method and apparatus |
| CN105005588B (en) * | 2015-06-26 | 2018-04-20 | 深圳市腾讯计算机系统有限公司 | A kind of processing method and processing device of training data |
| WO2019000949A1 (en) * | 2017-06-28 | 2019-01-03 | 华为技术有限公司 | Metadata storage method and system in distributed storage system, and storage medium |
| CN107547257A (en) * | 2017-07-14 | 2018-01-05 | 新华三技术有限公司 | A kind of server cluster implementation method and device |
| CN107817951A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of method and device for realizing the fusion of Ceph clusters |
| CN109802986A (en) * | 2017-11-17 | 2019-05-24 | 华为技术有限公司 | Device management method, system, device and server |
| CN108667836A (en) * | 2018-05-08 | 2018-10-16 | 众安信息技术服务有限公司 | Block chain common recognition method |
| CN112395281A (en) * | 2020-12-10 | 2021-02-23 | 太极计算机股份有限公司 | Heterogeneous multi-source data fusion system |
| CN112395281B (en) * | 2020-12-10 | 2021-05-11 | 太极计算机股份有限公司 | Heterogeneous multi-source data fusion system |
| CN113239013A (en) * | 2021-05-17 | 2021-08-10 | 北京青云科技股份有限公司 | Distributed system and storage medium |
| CN113239013B (en) * | 2021-05-17 | 2024-04-09 | 北京青云科技股份有限公司 | Distributed system and storage medium |
| CN114945023A (en) * | 2022-05-20 | 2022-08-26 | 济南浪潮数据技术有限公司 | Network connection multiplexing method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102411637B (en) | 2013-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102411637B (en) | Metadata management method of distributed file system | |
| TWI759789B (en) | Computer-implemented method, non-transitory computer readable media and storage system | |
| TWI737395B (en) | Log-structured storage systems and method | |
| TWI766309B (en) | Log-structured storage systems | |
| TWI733514B (en) | A storage system, a network node of a blockchain network, and a blockchain-based log-structured storage system | |
| US11422728B2 (en) | Log-structured storage systems | |
| WO2019228572A2 (en) | Log-structured storage systems | |
| US20210081373A1 (en) | Log-structured storage systems | |
| EP3695303B1 (en) | Log-structured storage systems | |
| CN103095806B (en) | A kind of load balancing management system of the real-time dataBase system towards bulk power grid | |
| CN102411639B (en) | Multi-copy storage management method and system of metadata | |
| US10903981B1 (en) | Log-structured storage systems | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN102662992A (en) | Method and device for storing and accessing massive small files | |
| CN103067525A (en) | Cloud storage data backup method based on characteristic codes | |
| US10942852B1 (en) | Log-structured storage systems | |
| CN103795801A (en) | Metadata group design method based on real-time application group | |
| CN101771548A (en) | File synchronizing method and system | |
| CN102737131A (en) | Method and system for processing redo log in database | |
| JP6055197B2 (en) | Database system | |
| CN104104731A (en) | Method and device for maintaining data consistency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 518057 Room 501A, 9 Buildings, Shenzhen Software Park (Phase 2), No. 1 Science and Technology Middle Road, Nanshan High-tech Zone, Shenzhen, Guangdong Province Co-patentee after: Innovation Technology Co., Ltd. Patentee after: Shenzhen Innovation Software Technology Co., Ltd. Address before: 518057 Section D West, 2nd Floor, 1st # Building, Industrial Building, Great Wall Computer Building, No. 3 Kefa Road, Nanshan District, Shenzhen City, Guangdong Province Co-patentee before: Innovation and Technology Storage Technology Co., Ltd. Patentee before: Chuangxinke Software Technology (Shenzhen) Co., Ltd. |