CN102904949B - Replica-based dynamic metadata cluster system - Google Patents

Replica-based dynamic metadata cluster system Download PDF

Info

Publication number
CN102904949B
CN102904949B CN201210378035.XA CN201210378035A CN102904949B CN 102904949 B CN102904949 B CN 102904949B CN 201210378035 A CN201210378035 A CN 201210378035A CN 102904949 B CN102904949 B CN 102904949B
Authority
CN
China
Prior art keywords
metadata
server
module
cluster
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201210378035.XA
Other languages
Chinese (zh)
Other versions
CN102904949A (en
Inventor
谭支鹏
冯丹
孙建良
詹天
桂权力
张文华
赵恒�
周炜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201210378035.XA priority Critical patent/CN102904949B/en
Publication of CN102904949A publication Critical patent/CN102904949A/en
Application granted granted Critical
Publication of CN102904949B publication Critical patent/CN102904949B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a replica-based dynamic metadata cluster system which comprises a client-side module and a server module, wherein the client-side module runs at a client side; the server module runs on each metadata server node in a metadata server cluster; the client-side module is used for sending a metadata request to the corresponding metadata server node, and adjusting sending of a read request of the client side according to load information, so that read load is distributed on the metadata server nodes; the server module is used for maintaining MLTs (metadata lookup tables) buffered in all the metadata servers in the metadata server cluster, migrating metadata during addition of a new metadata server node into the cluster and during recovery period of metadata, and counting the load information on each metadata server. According to a method disclosed by the invention, load is balanced via replicas; and expandability of the cluster, and service performance and availability of the metadata cluster are improved via a replica-based dynamic metadata cluster scheme.

Description

A kind of dynamic metadata group system based on copy
Technical field
The invention belongs to computer memory technical field, be specifically related to for the metadata group system in distributed file system.
Background technology
In large-scale distributed parallel file system, improve the performance of system, extensibility and availability more and more important.File system is the basic platform of all application systems, and for upper strata provides file related service, its performance, stability, extensibility and availability directly have influence on the stable of all application and Effec-tive Function.Quite looking in the time in the past, NFS file system was the conventional NFS of industrial quarters.But along with the fast development, the particularly development of web2.0 technology of computer technology and Internet technology, the increment of information is volatile trend, and the rapid growth of the not competent data of traditional NFS, efficiently can not manage mass data.In order to improve the performance of NFS, and improve the extensibility of system to a certain extent, there is the distributed file system of tripartite's framework, the Metadata Service of file system is separated with data, services, there is provided Metadata Service specially by meta data server, provide data, services by multiple data server is parallel.The GFS of Google, the TFS of Taobao, lustre etc. adopt this framework.But only have a meta data server to provide service online in the distributed file system of most of tripartite's framework, be easy to the bottleneck becoming system, and there is the scalability problem of single point failure and Metadata Service.
In order to solve performance bottleneck and the extensibility of meta data server, there is the solution of metadata server cluster.Metadata cluster is exactly form cluster by multiple stage meta data server, provides Metadata Service together.Metadata server cluster must be managed according to certain dividing method the NameSpace of whole file system by different meta data servers.Existing metadata Cluster Solution has two class methods usually, one class is the scheme divided based on hash, comprising pure hash scheme, namely by locating and storing metadata the method for file identifier (as: file path name) Hash, so generally only need a step just can find metadata.Hash method can be uniformly distributed metadata between meta data server, but hash method destroys the local property of the metadata under same catalogue, the expense of such operation as " ls " is just larger, efficiency comparison is low, and folder rename operation causes a large amount of metadata migration due to the hash value of the file under its catalogue and catalogue can be caused to change in addition.LH scheme is wherein another kind of scheme, and it in conjunction with the advantage of subtree segmentation and pure hash algorithm, and uses inertia metadata migration strategy and inertia ACL update strategy, provides more efficient metadata management.But, in the program, system uses inert method to carry out metadata reorientation (namely moving it when metadata is actual accessed time), although this way can reduce instantaneous migration, systematic function is impacted, but this makes removing of meta data server become relative difficulty, after making amendment to contingency table, one uncertain period is needed to complete migration to metadata.
Another kind of is splitting scheme based on catalogue subtree, and wherein most typical splitting scheme is the segmentation of static subtree.The name space of whole file system is divided into different subtrees by the program, and subtree is assigned to different meta data servers according to certain distribution method.This static subtree splitting scheme is comparatively ripe, realizes comparatively simple, uses also comparatively general, but it can not the change of answering system load, can not realize load balancing.
Summary of the invention
The object of the present invention is to provide a kind of dynamic metadata group system based on copy, by creating metadata copy, utilizing I/O to dispatch and alleviating because metadata divides the load imbalance caused; And allow the meta data server of some to lose efficacy by metadata copy, improve the reliability of system; And dynamically newly can add the Metadata Service ability that meta data server carrys out expanding system.
A kind of distributed document metadata group system based on copy of the present invention, comprising:
(1) run on the software module of client, comprising: meta data server selects module 100, meta data server connects maintenance module 101, MLT and shows maintenance module 102, load balancing module 103.
(1.1) meta data server selects module 100 major function to be that the metadata request of correspondence is sent to corresponding meta data server according to the information of the metadata cluster of its buffer memory by client.
(1.2) the connection maintenance module 101 of meta data server is mainly used in the connection of the meta data server in maintain customer end and cluster.When metadata set group changes, such as newly added node, because client does not connect with the meta data server newly added, connected with it before sending request to the meta data server of correspondence.
(1.3) MLT shows the MLT table that maintenance module 102 is mainly used in upgrading client-cache, makes client can the change of real-time tracking MDS cluster, thus makes client request can correctly navigate to corresponding meta data server.
(1.4) load balancing module 103 is responsible for the load information by collecting MDS cluster, and regulates the transmission of client read request according to load information, makes to read load and shares on metadata copy, thus realize load balancing.
(2) run on the software module of meta data server, comprising: MLT shows maintenance module 201, metadata migration module 202, failure detection module 203, main MDS elects module 204, and new node adds module 205, load statistics module 206.
(2.1) MLT shows maintenance module, the MLT of MDS end shows maintenance module for safeguarding the MLT table of MDS institute buffer memorys all in MDS cluster, make the MLT table of MDS buffer memorys all in system can the current state of correct response cluster, this also makes client can the change of tracking in time metadata server cluster in reciprocal process simultaneously.
(2.2) metadata migration module in charge is when new MDS node adds and metadata recovers, and by part metadata in cluster from a MDS node migrates to another one MDS node, and safeguards the MLT table of each MDS buffer memory by calling 201 modules;
(2.3) failure detection module in charge Detection theory data cluster system, looks into and sees if there is node failure, and in current MLT table, this meta data server is labeled as inefficacy, stops outwards providing service.
(2.4) main MDS elects module: centralization design is unavoidable wants processing center node failure problem.The election must carrying out new main MDS so main MDS delays in machine situation could continue to provide reliable service to upper strata.Because MDS nodes all in system is all cached with MDS information all in cluster, adopt main MDS election strategy, in election current cluster, MDS ID MDS that is minimum and that be in alive state is the main MDS that cluster is new.Then every other non-master MDS and new MDS connects, and recovers cluster state.
(2.5) new node adds module primary responsibility and newly adds meta data server to improve the service ability of metadata cluster further in current cluster ability to work in the situation such as saturated or associated metadata server failure.
(2.6) load statistics module 206 primary responsibility adds up the load information on each meta data server, with the master data providing client modules 103 to carry out load balancing.
In the present invention, MLT shows MLT table (metadata lookup table) described in maintenance module and records the division of whole file system name space.MLT have recorded the division of metadata, is responsible for the MDS of corresponding element data, the information such as the state that catalogue subtree is current.Catalogue subtree path is called the dividing condition of the NameSpace of whole file system, the id of the server of metadata under corresponding subtree catalogue is deposited and managed to metadata server IDs record, wherein first id is the meta data server id that primary copy is corresponding, and second and the 3rd is two slave copies respectively.The state of subtree state recording residing for current subtree, as information such as whether current subtree are moving.Clients all in whole file system and meta data server all safeguard that systematic MLT shows.Client, when access file system, uses the comspec of file to show to carry out complete trails with the MLT of its buffer memory and mates, find MDS belonging to corresponding element data, and the MDS to correspondence sends service request.
In the present invention, MLT shows the consistency that maintenance module 201 is mainly used in the MLT table of client and meta data server in maintenance system.Because MLT table maintains the division information of file system name space, correctly can locate metadata in order to ensure client, consistency maintenance must be carried out to the MLT table of buffer memory in system.Mainly comprise client MLT consistency maintenance and the conforming maintenance of meta data server MLT.
When MDS cluster changes, in all meta data servers, the MLT table of buffer memory all needs to upgrade.Such as after new MDS joins in system, MLT shows information and changes, edition upgrading, and main MDS shows MDS all in active push to cluster one by one up-to-date MLT.
The form that the renewal of client MLT table does not use MDS to push is safeguarded, but uses lazy strategy, upgrades when client and MDS have interaction request.Client each send request in encode its current institute buffer memory MLT table version number, MDS first decode when receiving client-requested client send MLT show version number, and with version number's comparison of its buffer memory, do not mate if find, then return MLT version number not matching error code, client receives error code and obtains up-to-date MLT table from current main MDS again, then metadata request is navigated to correct meta data server.
Metadata copy is related in the present invention, the maintenance of copy is by client executing, in client amendment or when deleting metadata, what obtain the primary copy parent directory of corresponding element data writes lock, then more latest copy successively, and lock is write in finally release, due to read operation time most of for the operation of metadata, the probability upgraded is very little, and therefore replica consistency maintenance expense is little, so the performance of overall evaluation system can be highly improved.
In the present invention, load balancing module mainly comprises load collection, load calculating and these three modules of load reorientation.Load collection module comprises two parts, and a part is positioned at MDS, and another part is positioned at client, and the load collection module on MDS is mainly used in the load information collecting self, comprises cpu busy percentage, memory usage, disk I/O, network I/O etc.Load collection module on Client is by the load information on each MDS of network collection.Load computing module is positioned at client, the weights of the load information computational load that client collects according to it to reflect the loading level of each MDS, the calculating method of proportional computation traditionally of weights.Load reorientation module is also positioned at client, and client carries out load regulation according to the fixed cycle.Owing to all comprising three parts of copies in every a metadata system, so read request reorientation can be realized load regulation, the adjustment granularity of current use is in units of subtree, when the unbalanced degree of system load exceedes threshold value, starting load regulates program, choosing the highest MDS of load is migration source, the minimum MDS of load is move target, and in current client, search the subtree met the demands, and the read access of corresponding subtree is moved to object MDS according to certain probability.
The present invention uses metadata copy to divide in conjunction with directory space, uses multiple meta data server jointly to provide Metadata Service, makes meta data server can also continue when part of nodes delays machine to provide service; Under meta data server delays machine and metadata server cluster performance saturated conditions, by carrying out subtree migration and new node Adding Way, improve performance and the extensibility of system; The equilibrium of load is realized by the metadata copy of accessing on other metadata server nodes.
Accompanying drawing explanation
Fig. 1 is the structural representation of present system;
Fig. 2 is client carry metadata cluster step schematic diagram;
Fig. 3 is that client is by MLT table access metadata step schematic diagram;
Fig. 4 is that client end upgrades MLT table step schematic diagram;
Fig. 5 is that new node adds step schematic diagram;
Fig. 6 and 7 is that MDS cluster MLT shows step of updating schematic diagram.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in further detail.Following embodiment is only exemplary, and not forming is limitation of the invention.
Fig. 1 is system configuration schematic diagram of the present invention.The system of the present embodiment comprises the client modules running on distributed type file system client side (Client), and it comprises meta data server and selects module 100, and meta data server connects maintenance module 101, MLT and shows maintenance module 102, load balancing module 103.
And running on the server module that meta data server (MDS) holds, it comprises MLT and shows maintenance module 201, metadata migration module 202, failure detection module 203, and main MDS elects module 204, and new node adds module 205 and load statistics module 206.All module 201-206 call rpc communication service module and realize and meta data server MDS communication each other.
The meta data server of client modules connects maintenance module 101 and calls distributed file system to safeguard the connection with MDS cluster.Load balancing module 103 calls the load that rpc communication module obtains metadata cluster, and MLT shows maintenance module 102 for realizing the renewal of MLT table.
MLT in server module shows maintenance module 201, for safeguarding the MLT table of meta data server institute buffer memorys all in metadata server cluster;
Metadata migration module 202 in server module, for adding at new metadata server node and when metadata is recovered, moving to another one metadata server node by part metadata in cluster from a metadata server node;
Master metadata server election module 204 in server module is for electing the meta data server that is minimum and that be in active state of ID in current cluster to be master metadata server new in cluster in machine situation of delaying at master metadata server.
Whether the failure detection module 203 in server module is for existing failure node in Detection theory data server cluster.
Failure detection module 203 and main MDS elect module 204, and new node adds module 205 and all calls the MLT table that MLT table maintenance module 201 upgrades MDS buffer memory in MDS cluster.
Load statistics module 206 in server module, for adding up the load information on each meta data server.
In system initialization process, first according to current demand, the division information of the NameSpace of file system is written in configuration file.Then main MDS starts the content that demons read configuration file, and the DS that mount is all, then wait for that other non-master MDS adds cluster.The ip of the main MDS of current cluster is inputted in the configuration file of non-master MDS, then demons are started, connect with main MDS, obtain the MLT table of main MDS, set up and be connected with the heartbeat of main MDS, the DS that the same mount of non-master MDS is all, then main MDS revises MLT and up-to-date MLT table is pushed to all MDS, after initialization is complete, cluster outwards provides service.
Be illustrated in figure 2 the step schematic diagram of client carry metadata cluster, first the MDS that client is corresponding according to the ip address mount formulated in mount order, then obtain the MLT table of current MDS cluster from the MDS of correspondence, then show all MDS objects and RPC session in content initialization MLT table according to MDS.
Fig. 3 is that client is by MLT table access metadata step schematic diagram, client is needing the position of accesses meta-data at every turn, add complete trails match code, corresponding information according to metadata mates, the final ID obtaining the metadata needing access, carries out metadata access to corresponding meta data server.
Support newly added node due to system and to delay machine, client must the MLT of its buffer memory of correct maintenance show correctly to locate request.Fig. 4 is that client end upgrades MLT table step schematic diagram, when there is metadata request, first client carries out longest path coupling according to original MLT table, from MLT table, find accessed MDS, and check the version number of MLT table to this MDS, if to show version consistent for the MLT table in client and the MLT in this MDS, then carry out follow-up metadata process, if find with the version number of this MDS inconsistent, then ask up-to-date MLT table to main MDS, and re-start longest path coupling.Then judge whether client sets up with the connection of request object MDS, if set up, follow-up metadata processing procedure, otherwise first connect.
MLT in the present invention MLT shown in maintenance module shows (metadata lookup table) for recording the division of whole file system name space.MLT have recorded the division of metadata, is responsible for the MDS of corresponding element data, the information such as the state that catalogue is current.If following table 1 is the basic format of MLT.Wherein comprised directory path is called the dividing condition of the NameSpace of whole file system, the server id of metadata under corresponding subtree catalogue is deposited and managed to metadata server IDs record, wherein first id is the meta data server id that primary copy is corresponding, and second and the 3rd is two slave copies respectively.The state of subtree state recording residing for current subtree, as information such as whether current subtree are moving.Clients all in whole file system and meta data server all safeguard that systematic MLT shows.Client, when access file system, uses the comspec of file to show to carry out complete trails with the MLT of its buffer memory and mates, find MDS belonging to corresponding element data, and the MDS to correspondence sends service request.
Subtree pathname Metadata server IDs Subtree state
/ 1,2,3 0
/a 2,3,1 0
/a/c 3,1,2 0
/b 1,2,3 0
Table 1MLT table
Fig. 5 is that new node adds step schematic diagram, and when new MDS joins in cluster, first keeper needs in the node in existing cluster, checks the main MDSIP of write/proc file system, and is written to by this IP in the configuration file of new MDS node.New node reads the ip of the main MDS in configuration file and connects with it in start-up course, reciprocal process is by a triggering subtree migration, namely main MDS selects suitable subtree to move to the node newly added according to load informations of mds all in current cluster and the load information of subtree, and upgrades the MLT information of MDS buffer memory in cluster.
Fig. 6 and 7 is that MDS cluster MLT shows step of updating schematic diagram, and to delay machine or to have new node to add fashionable when there is MDS in cluster, main MDS can upgrade MLT table, then MLT table is pushed on all MDS of current alive.

Claims (6)

1., based on a dynamic metadata group system for copy, it comprises the client modules running on client and the server module run in metadata server cluster on each metadata server node, wherein,
Described client modules for metadata request being sent to corresponding metadata server node, and regulates the transmission of client read request according to load information, makes to read load and shares on each metadata server node;
Described server module is for safeguarding the MLT table of meta data server institute buffer memorys all in metadata server cluster, and add in cluster at new metadata server node and carry out metadata migration between metadata convalescence, add up the load information on each meta data server simultaneously;
Wherein, described client modules comprises:
Meta data server selects module (100), for metadata request being sent to corresponding meta data server, meta data server connects maintenance module (101), and it is for setting up client and the connection newly adding the meta data server in cluster, and the connection of each meta data server in maintain customer end and metadata server cluster, MLT shows maintenance module (102), for upgrading the MLT table of client-cache, makes the change of client real-time tracking metadata server cluster, so that request is correctly navigated to corresponding meta data server, load balancing module (103), for the load information of collected metadata server cluster, and the transmission of client read request is regulated according to described load information, making to read load shares on each meta data server, this load balancing module (103) comprises load collection module, load computing module and load reorientation module, the load information of wherein said load collection module on collected metadata server, the weights of the load information computational load that its basis of described load computing module is collected are to reflect the loading level of each meta data server, described load reorientation module carries out reorientation according to described loading level to read request, to realize load regulation,
Described server module comprises: MLT shows maintenance module (201), for safeguarding the MLT table of meta data server institute buffer memorys all in metadata server cluster; Metadata migration module (202), for adding at new metadata server node and when metadata is recovered, moving to another one metadata server node by metadata a part of in cluster from a metadata server node; Load statistics module (206), for adding up the load information on each meta data server.
2. a kind of dynamic metadata group system based on copy according to claim 1, it is characterized in that, described server module also has master metadata server election module (204), it is for electing the meta data server that is minimum and that be in active state of ID in current cluster to be master metadata server new in cluster in machine situation of delaying at master metadata server.
3. a kind of dynamic metadata group system based on copy according to claim 1 and 2, it is characterized in that, described server module also has failure detection module (203), whether it is for existing failure node in Detection theory data server cluster.
4. a kind of dynamic metadata group system based on copy according to claim 1 and 2, it is characterized in that, described MLT table is used for the division of log file system name space, comprise the division of metadata, the meta data server of corresponding element data, and the state that the catalogue subtree of file system is current.
5. a kind of dynamic metadata group system based on copy according to claim or 2, is characterized in that, the renewal of the MLT table of described client is specially:
Client send request the version number that middle coding has the MLT of its current institute buffer memory to show, first the MLT that client of decoding when meta data server receives client-requested sends shows version number, and with version number's comparison of its buffer memory, do not mate if find, then return MLT version number not matching error code, client receives error code and obtains up-to-date MLT table from current master metadata server again, then metadata request is navigated to correct meta data server.
6. a kind of dynamic metadata group system based on copy according to claim 1 and 2, it is characterized in that, described server module also comprises new node and add module, it is for adding new meta data server to improve the service ability of cluster further in or associated metadata server failure situation saturated in current cluster ability to work.
CN201210378035.XA 2012-10-08 2012-10-08 Replica-based dynamic metadata cluster system Active CN102904949B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210378035.XA CN102904949B (en) 2012-10-08 2012-10-08 Replica-based dynamic metadata cluster system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210378035.XA CN102904949B (en) 2012-10-08 2012-10-08 Replica-based dynamic metadata cluster system

Publications (2)

Publication Number Publication Date
CN102904949A CN102904949A (en) 2013-01-30
CN102904949B true CN102904949B (en) 2015-07-01

Family

ID=47576984

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210378035.XA Active CN102904949B (en) 2012-10-08 2012-10-08 Replica-based dynamic metadata cluster system

Country Status (1)

Country Link
CN (1) CN102904949B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103220165B (en) * 2013-03-20 2017-04-19 杭州华三通信技术有限公司 Processing method and device for server active downtime
CN103544322A (en) * 2013-11-08 2014-01-29 北京邮电大学 Hotspot metadata management method based on server cluster
CN103763365A (en) * 2014-01-16 2014-04-30 浪潮(北京)电子信息产业有限公司 Method and system for load balancing of metadata service under cloud storage
CN103795801A (en) * 2014-02-12 2014-05-14 浪潮电子信息产业股份有限公司 Metadata group design method based on real-time application group
CN105991752B (en) * 2015-03-06 2019-06-21 阿里巴巴集团控股有限公司 A kind of data block update method and equipment
CN105389392A (en) * 2015-12-18 2016-03-09 浪潮(北京)电子信息产业有限公司 Metadata load statistical method and system
CN107135240B (en) * 2016-02-29 2020-12-15 华为技术有限公司 Method, device and system for distributing commands in distributed system
CN106445411A (en) * 2016-09-13 2017-02-22 乐视控股(北京)有限公司 Data reading method and device and distributed storage system
CN106572181A (en) * 2016-11-08 2017-04-19 深圳市中博睿存科技有限公司 Object storage interface load balancing method and system based on cluster file system
CN110830539B (en) * 2018-08-14 2022-09-06 贵州白山云科技股份有限公司 Network storage system and method
CN111104057B (en) * 2018-10-25 2022-03-29 华为技术有限公司 Node capacity expansion method in storage system and storage system
CN109783226B (en) * 2018-12-13 2020-09-25 交控科技股份有限公司 Distributed ATS system capable of dispersing data pressure
CN109885552B (en) * 2019-02-18 2023-08-18 天固信息安全系统(深圳)有限责任公司 Metadata dynamic management method of distributed file system and distributed file system
CN113672436A (en) * 2021-07-30 2021-11-19 济南浪潮数据技术有限公司 Disaster recovery backup method, device, equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102143215A (en) * 2011-01-20 2011-08-03 中国人民解放军理工大学 Network-based PB level cloud storage system and processing method thereof

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102143215A (en) * 2011-01-20 2011-08-03 中国人民解放军理工大学 Network-based PB level cloud storage system and processing method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"An Improvement of Static Subtree Partitioning in Metadata Server Cluster";Tan Zhipeng等;《Hindawi》;20120711;第1-5页 *

Also Published As

Publication number Publication date
CN102904949A (en) 2013-01-30

Similar Documents

Publication Publication Date Title
CN102904949B (en) Replica-based dynamic metadata cluster system
US11755415B2 (en) Variable data replication for storage implementing data backup
US11068501B2 (en) Single phase transaction commits for distributed database transactions
US20230012697A1 (en) Methods, devices and systems for maintaining consistency of metadata and data across data centers
US10534768B2 (en) Optimized log storage for asynchronous log updates
US20190188406A1 (en) Dynamic quorum membership changes
CA2930101C (en) Partition-based data stream processing framework
CA2929777C (en) Managed service for acquisition, storage and consumption of large-scale data streams
JP6346937B2 (en) Database system with database engine and separate distributed storage service
JP6404907B2 (en) Efficient read replica
US9460008B1 (en) Efficient garbage collection for a log-structured data store
Auradkar et al. Data infrastructure at LinkedIn
US20200341855A1 (en) Object store specialized backup and point-in-time recovery architecture
Deka A survey of cloud database systems
CN102420854A (en) Distributed file system facing to cloud storage
AU2014381598A1 (en) Client-configurable security options for data streams
WO2016148670A1 (en) Deduplication and garbage collection across logical databases
WO2015070232A1 (en) Data stream ingestion and persistence techniques
US10803012B1 (en) Variable data replication for storage systems implementing quorum-based durability schemes
US11818012B2 (en) Online restore to different topologies with custom data distribution
CN111935320B (en) Data synchronization method, related device, equipment and storage medium
Chandra et al. A study on cloud database
CN114925075B (en) Real-time dynamic fusion method for multi-source time-space monitoring information
Brinkmann et al. Towards archiving-as-a-service: a distributed index for the cost-effective access to replicated multi-version data
US11947555B1 (en) Intelligent query routing across shards of scalable database tables

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant