CN102904949A - Replica-based dynamic metadata cluster system - Google Patents
Replica-based dynamic metadata cluster system Download PDFInfo
- Publication number
- CN102904949A CN102904949A CN201210378035XA CN201210378035A CN102904949A CN 102904949 A CN102904949 A CN 102904949A CN 201210378035X A CN201210378035X A CN 201210378035XA CN 201210378035 A CN201210378035 A CN 201210378035A CN 102904949 A CN102904949 A CN 102904949A
- Authority
- CN
- China
- Prior art keywords
- metadata
- module
- data server
- meta data
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a replica-based dynamic metadata cluster system which comprises a client-side module and a server module, wherein the client-side module runs at a client side; the server module runs on each metadata server node in a metadata server cluster; the client-side module is used for sending a metadata request to the corresponding metadata server node, and adjusting sending of a read request of the client side according to load information, so that read load is distributed on the metadata server nodes; the server module is used for maintaining MLTs (metadata lookup tables) buffered in all the metadata servers in the metadata server cluster, migrating metadata during addition of a new metadata server node into the cluster and during recovery period of metadata, and counting the load information on each metadata server. According to a method disclosed by the invention, load is balanced via replicas; and expandability of the cluster, and service performance and availability of the metadata cluster are improved via a replica-based dynamic metadata cluster scheme.
Description
Technical field
The invention belongs to computer memory technical field, be specifically related to for the metadata cluster system in the distributed file system.
Background technology
In large-scale distributed parallel file system, improve the performance of system, extensibility and availability are more and more important.File system is the basic platform of all application systems, and for the upper strata provides the file related service, its performance, stability, extensibility and availability directly have influence on the stable and efficient operation of all application.Within the time of in the past quite looking, the NFS file system is industrial quarters NFS commonly used.But along with the fast development of computer technology and Internet technology, the particularly development of web2.0 technology, the increment of information is volatile trend, and mass data can not be efficiently managed in the rapid growth of traditional not competent data of NFS.In order to improve the performance of NFS, and improve to a certain extent the extensibility of system, the distributed file system that tripartite framework occurred, the Metadata Service of file system is separated with data, services, provide specially Metadata Service by meta data server, by the parallel data, services that provides of a plurality of data servers.The GFS of Google, the TFS of Taobao, lustre etc. adopt this framework.But only have a meta data server that service is provided online in the distributed file system of most of tripartite framework, be easy to become the bottleneck of system, and have the scalability problem of single point failure and Metadata Service.
For performance bottleneck and the extensibility that solves meta data server, the solution of meta data server clustering has appearred.The metadata cluster is exactly to form cluster by many meta data servers, and Metadata Service is provided together.The meta data server cluster must be managed according to certain dividing method the NameSpace of whole file system by different meta data servers.Existing metadata cluster solution has two class methods usually, one class is based on the scheme that hash divides, comprising pure hash scheme, namely by the method for file identifier (as: file path name) Hash is located and storing metadata, so only need generally speaking can find metadata once the step.The Hash method can be between meta data server even distribution metadata, but the hash method has been destroyed the local property of the metadata under the same catalogue, the expense of the operation as " ls " is just larger like this, efficient is lower, and folder rename operation causes a large amount of metadata migration owing to the hash value that can cause file under its catalogue and catalogue changes in addition.The LH scheme is another kind of scheme wherein, and it cuts apart advantage with pure hash algorithm in conjunction with subtree, and uses inertia metadata migration strategy and inertia ACL update strategy, and more efficiently metadata management is provided.But, system uses inert method to carry out metadata reorientation (namely when metadata is actual accessed it being moved) in this scheme, although this way can reduce instantaneous migration systematic function is impacted, but this is so that removing of meta data server becomes relatively difficult, after contingency table is made modification, need one uncertain period to finish migration to metadata.
The another kind of splitting scheme that is based on the catalogue subtree, wherein most typical splitting scheme is that static subtree is cut apart.This scheme is divided into different subtrees with the name space of whole file system, and subtree is assigned to different meta data servers according to certain distribution method.This static subtree splitting scheme is comparatively ripe, realize comparatively simple, use also comparatively general, but the variation that it can not the answering system load can not realize load balancing.
Summary of the invention
The object of the present invention is to provide a kind of dynamic metadata group system based on copy, by creating the metadata copy, utilize I/O to dispatch to alleviate the load imbalance that causes because of the metadata division; And pass through the meta data server inefficacy that the metadata copy allows some, improve the reliability of system; And can dynamically newly add the Metadata Service ability that meta data server comes expanding system.
A kind of distributed document metadata cluster system based on copy of the present invention comprises:
(1) run on the software module of client, comprising: meta data server is selected module 100, and meta data server connects maintenance module 101, and MLT shows maintenance module 102, load balancing module 103.
(1.1) meta data server selection module 100 major functions are the meta data server that client sends to the metadata request of correspondence according to the information of the metadata cluster of its buffer memory correspondence.
(1.2) the connection maintenance module 101 of meta data server is mainly used in being connected of meta data server in maintain customer end and the cluster.When the metadata set Swarms changes, such as newly added node, because client does not connect with the meta data server of new adding, before sending request, the meta data server to correspondence connecting with it.
(1.3) MLT table maintenance module 102 is mainly used in upgrading the MLT table of client-cache, so that the variation that client can real-time tracking MDS cluster, thereby so that client can request correctly navigate to corresponding meta data server.
(1.4) load balancing module 103 is responsible for the load information by collection MDS cluster, and regulates the transmission of client read request according to load information, shares on the metadata copy so that read load, thereby realizes load balancing.
(2) run on the software module of meta data server, comprising: MLT shows maintenance module 201, metadata migration module 202, and failure detection module 203, main MDS election module 204, new node adds module 205, load statistics module 206.
(2.1) MLT table maintenance module, the MLT table maintenance module of MDS end is used for safeguarding the MLT table of the MDS institute buffer memory that the MDS cluster is all, so that the current state that the MLT of all MDS buffer memorys table can the correct response cluster in the system, this is also so that client can in time be followed the tracks of the variation of meta data server cluster in reciprocal process simultaneously.
(2.2) the metadata migration module is responsible for when new MDS node adding and metadata recovery part metadata in the cluster being moved on the another one MDS node from a MDS node, and safeguards the MLT table of each MDS buffer memory by calling 201 modules;
(2.3) the failure detection module is responsible for detecting metadata cluster system, looks into and sees if there is node failure, and in current MLT table this meta data server is labeled as inefficacy, stops outwards to provide service.
(2.4) main MDS election module: the unavoidable processing center node failure problem of wanting of centralization design.The election that must carry out new main MDS so main MDS delays in the machine situation could continue to provide to the upper strata reliable service.Because all MDS nodes all are cached with MDS information all in the cluster in the system, adopt main MDS election strategy, electing the minimum and MDS that be in the alive state of MDS ID in the current cluster is the new main MDS of cluster.Then every other non-main MDS and new MDS connect, and recover cluster state.
(2.5) new node adding module mainly is responsible for newly adding in the situations such as or the inefficacy of associated metadata server saturated in current cluster ability to work meta data server with the service ability of further raising metadata cluster.
(2.6) the load statistics module 206 main load informations of being responsible on each meta data server of statistics carry out the master data of load balancing so that client modules 103 to be provided.
The division of the whole file system name space of MLT table (metadata lookup table) record described in the MLT table maintenance module among the present invention.MLT has recorded the division of metadata, is responsible for the MDS of corresponding element data, the information such as state that the catalogue subtree is current.The dividing condition of the NameSpace of catalogue subtree path whole file system by name, the id of the server of metadata under the corresponding subtree catalogue is deposited and managed to metadata server IDs record, wherein first id is meta data server id corresponding to primary copy, and second and the 3rd is respectively two slave copies.The subtree state recording the residing state of current subtree, such as information such as whether current subtree are moving.Client and meta data servers all in the whole file system are all safeguarded systematic MLT table.Client uses the comspec of file and the MLT table of its buffer memory to carry out the complete trails coupling in the access file system, finds the affiliated MDS of corresponding element data, to the MDS transmission service request of correspondence.
MLT table maintenance module 201 is mainly used in the consistency of the MLT table of client and meta data server in the maintenance system among the present invention.Because the MLT table has been safeguarded the division information of file system name space, can correctly locate metadata in order to ensure client, must carry out consistency maintenance to the MLT table of buffer memory in the system.Mainly comprise client MLT consistency maintenance and the conforming maintenance of meta data server MLT.
When the MDS cluster changed, the MLT of buffer memory table all needed to upgrade in all meta data servers.Such as after new MDS joins in the system, MLT table information changes, edition upgrading, and main MDS shows one by one active push all MDS in the cluster to up-to-date MLT.
The form that the renewal of client MLT table does not use MDS to push is safeguarded, but use inertia strategy upgrades in client and when MDS has mutual request.Each sends the version number of the MLT table of its current institute buffer memory of having encoded in the request client, the MLT table version number that the client of at first decoding when MDS receives client-requested sends, and compare with the version number of its buffer memory, do not mate if find, then return not matching error code of MLT version number, client receives that error code obtains up-to-date MLT table from current main MDS again, then navigates to metadata request correct meta data server.
Relate to the metadata copy among the present invention, the maintenance of copy is by client executing, when client is revised or is deleted metadata, obtain the corresponding element data the primary copy parent directory write lock, then latest copy more successively discharges at last and writes lock, because read operation in the time of most of for the operation of metadata, the probability that upgrades is very little, so the copy consistency maintenance costs is little, so the performance of overall evaluation system can be highly improved.
Load balancing module mainly comprises load collection, these three modules of load calculation and load reorientation among the present invention.The load collection module comprises two parts, and a part is positioned at MDS, and another part is positioned at client, and the load collection module on the MDS is mainly used in collecting the load information of self, comprises cpu busy percentage, memory usage, disk I/O, network I/O etc.Load collection module on the Client is collected load information on each MDS by network.The load calculation module is positioned at client, and to reflect the loading level of each MDS, the calculating of weights is according to traditional method of proportional computation according to the weights of its load information computational load of collecting for client.Load reorientation module also is positioned at client, and client carries out load regulation according to the fixed cycle.Because for all comprising three parts of copies in every a metadata system, so can realize load regulation to the read request reorientation, the current adjusting granularity of using with subtree as unit, starting load is regulated program when the unbalanced degree of system load surpasses threshold value, choosing the highest MDS of load is the migration source, the minimum MDS of load is move target, and searches the subtree that meets the demands in current client, and the read access of corresponding subtree is moved to purpose MDS according to certain probability.
The present invention uses the metadata copy in conjunction with the catalogue spatial division, jointly provides Metadata Service with a plurality of meta data servers, so that meta data server can also continue to provide service under part of nodes is delayed the situation of machine; Under meta data server is delayed machine and meta data server cluster performance saturated conditions, by carrying out subtree migration and new node adding method, improve performance and the extensibility of system; By accessing the equilibrium of the metadata copy realization load on other metadata server nodes.
Description of drawings
Fig. 1 is the structural representation of system of the present invention;
Fig. 2 is client carry metadata cluster step schematic diagram;
Fig. 3 is that client passes through MLT table access metadata step schematic diagram;
Fig. 4 is that the client end upgrades MLT table step schematic diagram;
Fig. 5 is that new node adds the step schematic diagram;
Fig. 6 and 7 is MDS cluster MLT table step of updating schematic diagram.
Embodiment
Below in conjunction with the drawings and specific embodiments the present invention is described in further detail.Following embodiment only is exemplary, and not consisting of is limitation of the invention.
Fig. 1 is system configuration schematic diagram of the present invention.The system of present embodiment comprises the client modules that runs on distributed type file system client side (Client), and it comprises meta data server selection module 100, and meta data server connects maintenance module 101, and MLT shows maintenance module 102, load balancing module 103.
And the server module that runs on meta data server (MDS) end, it comprises MLT table maintenance module 201, metadata migration module 202, and failure detection module 203, main MDS election module 204, new node adds module 205 and load statistics module 206.The rpc communication service module of all calling all module 201-206 realizes the communicating by letter each other with meta data server MDS.
The meta data server of client modules connects maintenance module 101 and calls distributed file system and safeguard and being connected of MDS cluster.Load balancing module 103 is called the load that the rpc communication module is obtained the metadata cluster, and MLT table maintenance module 102 is used for realizing the renewal of MLT table.
MLT table maintenance module 201 in the server module is used for safeguarding that the MLT of the meta data server institute buffer memory that the meta data server cluster is all shows;
The meta data server that pivot data server election module 204 in the server module is used for electing current cluster ID minimum and be in active state under the pivot data server being delayed the machine situation is pivot data server new in the cluster.
In the system initialization process, at first according to current demand the division information of the NameSpace of file system is written in the configuration file.Then main MDS starts the content that demons read configuration file, and all DS of mount, waits for that then other non-main MDS adds cluster.The ip of the current cluster master MDS of input in the configuration file of non-main MDS, then start demons, connect with main MDS, obtain the MLT table of main MDS, foundation is connected with the heartbeat of main MDS, all DS of the same mount of non-main MDS, and then main MDS revises MLT up-to-date MLT table is pushed to all MDS, after initialization was complete, cluster outwards provided service.
Be illustrated in figure 2 as the step schematic diagram of client carry metadata cluster, at first client is according to MDS corresponding to ip address mount that formulates in the mount order, then obtain the MLT table of current MDS cluster from the MDS of correspondence, then according to all MDS object and RPC sessions in the MDS table content initialization MLT table.
Fig. 3 is that client passes through MLT table access metadata step schematic diagram, client is in the position that needs accesses meta-data at every turn, add the complete trails match code, corresponding information according to metadata mates, finally obtain the ID of the metadata that needs access, carry out metadata access to corresponding meta data server.
Because system supports newly added node and the machine of delaying, the MLT that client must its buffer memory of correct maintenance shows and could correctly locate request.Fig. 4 is that the client end upgrades MLT table step schematic diagram, when metadata request occurring, client is at first carried out the longest path coupling according to original MLT table, in the MLT table, find the MDS that accesses, and check the version number that MLT shows to this MDS, if the table of the MLT among the client is consistent with the MLT table version among this MDS, then carrying out follow-up metadata processes, if discovery is inconsistent with the version number of this MDS, then to the up-to-date MLT table of main MDS request, and re-start the longest path coupling.Then judge whether set up client and being connected of request purpose MDS, if set up, follow-up metadata processing procedure, otherwise connect first.
MLT table (metadata lookup table) in the MLT table maintenance module among the present invention is for the division of recording whole file system name space.MLT has recorded the division of metadata, is responsible for the MDS of corresponding element data, the information such as state that catalogue is current.Be the basic format of MLT such as following table 1.The dividing condition of the NameSpace of the directory path that wherein comprises whole file system by name, the server id of metadata under the corresponding subtree catalogue is deposited and managed to metadata server IDs record, wherein first id is meta data server id corresponding to primary copy, and second and the 3rd is respectively two slave copies.The subtree state recording the residing state of current subtree, such as information such as whether current subtree are moving.Client and meta data servers all in the whole file system are all safeguarded systematic MLT table.Client uses the comspec of file and the MLT table of its buffer memory to carry out the complete trails coupling in the access file system, finds the affiliated MDS of corresponding element data, to the MDS transmission service request of correspondence.
| The subtree pathname | Metadata server IDs | The subtree state |
| / | 1,2,3 | 0 |
| /a | 2,3,1 | 0 |
| /a/c | 3,1,2 | 0 |
| /b | 1,2,3 | 0 |
Table 1MLT table
Fig. 5 is that new node adds the step schematic diagram, when new MDS joined in the cluster, the keeper at first needed to check to write in the node in the existing cluster/and the main MDSIP of proc file system, and this IP is written in the configuration file of new MDS node.New node reads the ip of the main MDS in the configuration file and connects with it in start-up course, reciprocal process will trigger a subtree migration, namely main MDS selects suitable subtree to move to the node of new adding according to the load information of all mds in the current cluster and the load information of subtree, and upgrades the MLT information of MDS buffer memory in the cluster.
Fig. 6 and 7 is MDS cluster MLT table step of updating schematic diagram, and delay machine or to have new node to add fashionable of MDS occurs in cluster, and main MDS can upgrade the MLT table, then the MLT table is pushed on all MDS of current alive.
Claims (9)
1. dynamic metadata group system based on copy, it comprises the client modules that runs on client and runs on the server module on each metadata server node in the meta data server cluster, wherein,
Described client modules is used for metadata request is sent to corresponding metadata server node, and regulates the transmission of client read request according to load information, shares on each metadata server node so that read load;
Described server module is used for safeguarding the MLT table of the meta data server institute buffer memory that the meta data server cluster is all, and add in the cluster and metadata is carried out metadata migration between convalescence at new metadata server node, add up simultaneously the load information on each meta data server.
2. a kind of dynamic metadata group system based on copy according to claim 1 is characterized in that described client modules comprises:
Meta data server is selected module (100), is used for metadata request is sent to corresponding meta data server;
Meta data server connects maintenance module (101), and it is used for setting up being connected of meta data server of client and new adding cluster, and being connected of each meta data server in maintain customer end and the meta data server cluster;
MLT shows maintenance module (102), is used for upgrading the MLT table of client-cache, so that the variation of client real-time tracking meta data server cluster, will ask correctly to navigate to corresponding meta data server;
Load balancing module (103) is used for the load information of collected metadata server cluster, and regulates the transmission of client read request according to described load information, shares on each meta data server so that read load.
3. a kind of dynamic metadata group system based on copy according to claim 1 and 2 is characterized in that described server module comprises:
MLT shows maintenance module (201), is used for safeguarding the MLT table of the meta data server institute buffer memory that the meta data server cluster is all;
Metadata migration module (202) is used in new metadata server node adding and when metadata is recovered a part of metadata in the cluster being moved on the another one metadata server node from a metadata server node;
Load statistics module (206) is used for the load information on each meta data server of statistics.
4. a kind of dynamic metadata group system based on copy according to claim 3, it is characterized in that, also have pivot data server election module (204) on the described server module, its meta data server that is used for electing current cluster ID minimum and be in active state under the pivot data server being delayed the machine situation is pivot data server new in the cluster.
5. according to claim 3 or 4 described a kind of dynamic metadata group systems based on copy, it is characterized in that also having failure detection module (203) on the described server module, it is for detection of whether there being failure node in the meta data server cluster.
6. described a kind of dynamic metadata group system based on copy one of according to claim 1-5, it is characterized in that, described MLT table is used for the division in log file systematic naming method space, comprise the division of metadata, the meta data server of corresponding element data, and the current state of catalogue subtree of file system.
7. one of according to claim 1-6 described a kind of dynamic metadata group system based on copy is characterized in that, the renewal of the MLT table of described client is specially:
Coding has the version number of the MLT table of its current institute buffer memory in the transmission request of client, the MLT table version number that the client of at first decoding when meta data server receives client-requested sends, and compare with the version number of its buffer memory, do not mate if find, then return not matching error code of MLT version number, client receives that error code obtains up-to-date MLT table from current pivot data server again, then navigates to metadata request correct meta data server.
8. described a kind of dynamic metadata group system based on copy one of according to claim 2-7, it is characterized in that, described load balancing module (102) comprises the load collection module, load calculation module and load reorientation module, wherein said load collection module is used for the load information on the collected metadata server, the weights of the load information computational load that its basis of described load calculation module is collected are to reflect the loading level of each meta data server, described load reorientation module is carried out reorientation according to described loading level to read request, to realize load regulation.
9. described a kind of dynamic metadata group system based on copy one of according to claim 1-8, it is characterized in that, comprise also on the described server module that new node adds module, it is used for adding under or the associated metadata server failure conditions saturated in current cluster ability to work new meta data server with the service ability of further raising cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210378035.XA CN102904949B (en) | 2012-10-08 | 2012-10-08 | Replica-based dynamic metadata cluster system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210378035.XA CN102904949B (en) | 2012-10-08 | 2012-10-08 | Replica-based dynamic metadata cluster system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102904949A true CN102904949A (en) | 2013-01-30 |
| CN102904949B CN102904949B (en) | 2015-07-01 |
Family
ID=47576984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210378035.XA Active CN102904949B (en) | 2012-10-08 | 2012-10-08 | Replica-based dynamic metadata cluster system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102904949B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103763365A (en) * | 2014-01-16 | 2014-04-30 | 浪潮(北京)电子信息产业有限公司 | Method and system for load balancing of metadata service under cloud storage |
| CN103795801A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | Metadata group design method based on real-time application group |
| CN105389392A (en) * | 2015-12-18 | 2016-03-09 | 浪潮(北京)电子信息产业有限公司 | Metadata load statistical method and system |
| CN105991752A (en) * | 2015-03-06 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Data block update method and apparatus |
| CN106445411A (en) * | 2016-09-13 | 2017-02-22 | 乐视控股（北京）有限公司 | Data reading method and device and distributed storage system |
| CN106572181A (en) * | 2016-11-08 | 2017-04-19 | 深圳市中博睿存科技有限公司 | Object storage interface load balancing method and system based on cluster file system |
| CN103220165B (en) * | 2013-03-20 | 2017-04-19 | 杭州华三通信技术有限公司 | Processing method and device for server active downtime |
| CN109783226A (en) * | 2018-12-13 | 2019-05-21 | 交控科技股份有限公司 | The distributed ATS system of dispersible data pressure |
| CN109885552A (en) * | 2019-02-18 | 2019-06-14 | 天固信息安全系统(深圳)有限责任公司 | The metadata dynamic management approach and distributed file system of distributed file system |
| CN110830539A (en) * | 2018-08-14 | 2020-02-21 | 贵州白山云科技股份有限公司 | Network storage system and method |
| CN111104057A (en) * | 2018-10-25 | 2020-05-05 | 华为技术有限公司 | Node capacity expansion method in storage system and storage system |
| CN112491986A (en) * | 2016-02-29 | 2021-03-12 | 华为技术有限公司 | Method, device and system for distributing commands in distributed system |
| CN113672436A (en) * | 2021-07-30 | 2021-11-19 | 济南浪潮数据技术有限公司 | Disaster recovery backup method, device, equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
-
2012
- 2012-10-08 CN CN201210378035.XA patent/CN102904949B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
Non-Patent Citations (1)
| Title |
|---|
| TAN ZHIPENG等: ""An Improvement of Static Subtree Partitioning in Metadata Server Cluster"", 《HINDAWI》, 11 July 2012 (2012-07-11), pages 1 - 5 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220165B (en) * | 2013-03-20 | 2017-04-19 | 杭州华三通信技术有限公司 | Processing method and device for server active downtime |
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103763365A (en) * | 2014-01-16 | 2014-04-30 | 浪潮(北京)电子信息产业有限公司 | Method and system for load balancing of metadata service under cloud storage |
| CN103795801A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | Metadata group design method based on real-time application group |
| CN105991752B (en) * | 2015-03-06 | 2019-06-21 | 阿里巴巴集团控股有限公司 | A kind of data block update method and equipment |
| CN105991752A (en) * | 2015-03-06 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Data block update method and apparatus |
| CN105389392A (en) * | 2015-12-18 | 2016-03-09 | 浪潮(北京)电子信息产业有限公司 | Metadata load statistical method and system |
| CN112491986B (en) * | 2016-02-29 | 2022-07-29 | 华为技术有限公司 | Method, device and system for distributing commands in distributed system |
| CN112491986A (en) * | 2016-02-29 | 2021-03-12 | 华为技术有限公司 | Method, device and system for distributing commands in distributed system |
| CN106445411A (en) * | 2016-09-13 | 2017-02-22 | 乐视控股（北京）有限公司 | Data reading method and device and distributed storage system |
| CN106572181A (en) * | 2016-11-08 | 2017-04-19 | 深圳市中博睿存科技有限公司 | Object storage interface load balancing method and system based on cluster file system |
| CN110830539A (en) * | 2018-08-14 | 2020-02-21 | 贵州白山云科技股份有限公司 | Network storage system and method |
| CN110830539B (en) * | 2018-08-14 | 2022-09-06 | 贵州白山云科技股份有限公司 | Network storage system and method |
| CN111104057A (en) * | 2018-10-25 | 2020-05-05 | 华为技术有限公司 | Node capacity expansion method in storage system and storage system |
| CN111104057B (en) * | 2018-10-25 | 2022-03-29 | 华为技术有限公司 | Node capacity expansion method in storage system and storage system |
| CN109783226B (en) * | 2018-12-13 | 2020-09-25 | 交控科技股份有限公司 | Distributed ATS system capable of dispersing data pressure |
| CN109783226A (en) * | 2018-12-13 | 2019-05-21 | 交控科技股份有限公司 | The distributed ATS system of dispersible data pressure |
| CN109885552A (en) * | 2019-02-18 | 2019-06-14 | 天固信息安全系统(深圳)有限责任公司 | The metadata dynamic management approach and distributed file system of distributed file system |
| CN109885552B (en) * | 2019-02-18 | 2023-08-18 | 天固信息安全系统(深圳)有限责任公司 | Metadata dynamic management method of distributed file system and distributed file system |
| CN113672436A (en) * | 2021-07-30 | 2021-11-19 | 济南浪潮数据技术有限公司 | Disaster recovery backup method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102904949B (en) | 2015-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102904949B (en) | Replica-based dynamic metadata cluster system | |
| JP7309005B2 (en) | Database tenant migration system and method | |
| JP6510112B2 (en) | Datastream Capture and Persistence Policy | |
| US9460185B2 (en) | Storage device selection for database partition replicas | |
| US11068501B2 (en) | Single phase transaction commits for distributed database transactions | |
| US10922303B1 (en) | Early detection of corrupt data partition exports | |
| CA2930101C (en) | Partition-based data stream processing framework | |
| CA2929777C (en) | Managed service for acquisition, storage and consumption of large-scale data streams | |
| CA2929776C (en) | Client-configurable security options for data streams | |
| US10853242B2 (en) | Deduplication and garbage collection across logical databases | |
| CN102420854A (en) | Distributed file system facing to cloud storage | |
| US10860604B1 (en) | Scalable tracking for database udpates according to a secondary index | |
| US7069270B1 (en) | Automated method and mechanism for converting a single instance application to a multiple instance application | |
| Won et al. | Moving metadata from ad hoc files to database tables for robust, highly available, and scalable HDFS | |
| US11079960B2 (en) | Object storage system with priority meta object replication | |
| US11093465B2 (en) | Object storage system with versioned meta objects | |
| US11947555B1 (en) | Intelligent query routing across shards of scalable database tables | |
| Junping | Analysis of key technologies of distributed file system based on big data [J] | |
| WO2023097270A1 (en) | Detecting idle periods at network endpoints for management actions at processing clusters for managed databases | |
| Mukherjee | Benchmarking Hadoop performance on different distributed storage systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |