CN114491111A - Distributed metadata system for picture storage - Google Patents
Distributed metadata system for picture storage Download PDFInfo
- Publication number
- CN114491111A CN114491111A CN202210140735.9A CN202210140735A CN114491111A CN 114491111 A CN114491111 A CN 114491111A CN 202210140735 A CN202210140735 A CN 202210140735A CN 114491111 A CN114491111 A CN 114491111A
- Authority
- CN
- China
- Prior art keywords
- metadata
- picture
- group
- storage
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a distributed metadata system for storing pictures, which comprises a metadata server cluster consisting of at least one group, wherein each group comprises at least one metadata server; the metadata stored by each metadata server in the same group are the same, and the metadata are synchronized in the group in a binlog file form; the metadata server stores metadata of a key-value structure using a BDB; the BDB is an embedded database with an open source and runs in the same process space with an application program; the BDB saves the metadata of the pictures in the BDB file. The advantages are that: the problems of capacity upper limit, performance bottleneck, high-availability configuration and the like in the central node mode are solved.
Description
Technical Field
The invention relates to the technical field of picture storage, in particular to a distributed metadata system for picture storage.
Background
The application programs such as e-commerce, chat and forum can use pictures on a large scale, the number of pictures stored in the website is huge, the pictures are multiplied every year, and meanwhile, the access amount of users is also large. Such picture files have the characteristics of small size and large quantity, metadata management is an important link when a picture storage system is designed, and the existing solutions mainly comprise two types:
1. the central node manages metadata, such as storage systems of TFS, GFS and the like, and records file metadata by using the central node, so that the problems of data statistics, load balancing and system capacity expansion can be conveniently solved; the disadvantage is that the central node can form a single point of failure and a performance bottleneck.
2. The method comprises the following steps that (1) no metadata design is adopted, for example, storage systems such as Ceph and GlusterFS are adopted, when files are stored, hash operation is firstly carried out on the files, the obtained hash value is then operated on node topology, path weight and the like of a cluster, and the mapping relation between the hash value and a storage position is obtained, so that the system has no performance restriction of a central node and is convenient to expand; the disadvantages are that: (1) when the cluster expands, because the cluster topology changes, data migration is needed, and the access performance of upper-layer services is affected; (2) since the user data is not saved, the statistical analysis performance is low, such as the "ls directory" is time-consuming.
In response to these problems, a need exists to develop an efficient and easy-to-use picture storage metadata management system.
Disclosure of Invention
It is an object of the present invention to provide a distributed metadata system for picture storage that solves the aforementioned problems in the prior art.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a distributed metadata system for picture storage comprises a metadata server cluster composed of at least one group, each group containing at least one metadata server; the metadata stored by each metadata server in the same group are the same, and the metadata are synchronized in the group in a binlog file form;
the metadata server stores metadata of a key-value structure using a BDB;
the BDB is an embedded database with an open source and runs in the same process space with an application program; the BDB saves the metadata of the picture in the BDB file.
Preferably, when a piece of metadata is added to the metadata server, a log is generated in the binlog file, the content is 'time, data content and newly added mark', the binlog file is read by the synchronization thread, the data with the newly added mark is pushed to other metadata servers in the same group, the metadata is inserted into the BDB file after the other metadata servers receive the data, a log is added to the binlog file, and the content is 'time, data content and synchronously added mark'; because of the sync added flag, the add operation is not resynchronized to other metadata servers.
Preferably, when deleting a piece of metadata, the metadata server generates a log in the binlog file, where the content is "time, data content, and deletion flag", reads the binlog file in the same thread, pushes the data with the deletion flag to other metadata servers in the same group, and after receiving the data, the other metadata servers delete the metadata in the BDB file and add a log in the binlog file, where the content is "time, data content, and synchronous deletion flag"; because of the sync added flag, the delete operation is not resynchronized to other metadata servers.
Preferably, when the metadata server cluster is expanded, the metadata server adopts a logical grouping mode to avoid the change of hash distribution; the logical grouping mode is specifically that one physical grouping corresponds to one group of metadata servers, and the group of metadata servers are divided into a plurality of logical groupings; one metadata server can simultaneously support a plurality of logic groups, and each logic group corresponds to one BDB file; estimating the number of logic groups needed in the future at the initial stage of metadata system construction, configuring the logic groups to physical groups, migrating one or more logic groups to newly-added physical groups during capacity expansion, and copying corresponding BDB files to complete metadata migration.
Preferably, the metadata system is combined with the Web server, the application program and the storage unit to construct a picture storage system; in the picture storage system, the Web server provides a picture access Restful or S3 format interface; the application program helps a user to send a picture access request; the storage unit consists of a management node and a plurality of storage nodes, and after the BDB file is uploaded to the storage unit, a storage ID associated with a storage position is generated; the metadata system stores the correspondence of storage IDs with user data.
Preferably, when the user uploads the picture by using the application program, the user has user parameters, the Web server calls the SDK to store the picture file in the storage unit and obtain the storage ID, calculates the metadata of the picture through the SDK and stores the metadata in which group and which metadata server of the metadata server cluster, then inserts the metadata of a key-value structure into the corresponding metadata server, the Web server returns the URL of the picture to the application program, and after the application program obtains the URL of the picture returned by the Web server, the URL is used for retrieving, downloading and deleting the picture;
the key comprises a picture directory and a file name, and the value comprises a storage ID and a creation time.
Preferably, the specific process of calculating which group of the metadata server cluster the metadata of the picture is stored in by the SDK is,
dividing the hash value range associated with each group according to the number of the groups; that is, assuming that there are N groups in the metadata server cluster, the hash value is averagely divided into N segments, and each group corresponds to a segment of hash value range;
calculating a Hash value corresponding to the picture file name by using a BKDR Hash algorithm;
according to the hash value range of the hash value corresponding to the picture file name, locating the group where the metadata is located;
calculating the ID of the metadata server according to the group where the metadata is located; the metadata server ID is the number of metadata servers within the group where the hash value% metadata is located.
Preferably, the Web server returns the URL of the picture to the application program, and after the application program obtains the URL of the picture returned by the Web server, the application program uses the URL to retrieve, download and delete the picture specifically,
and after receiving a picture access request sent by a user through an application program, the Web server calls the SDK to perform access operation, the SDK analyzes the picture directory and the file name from the URL to construct a key, the key is used for inquiring the storage ID of the picture from the storage unit, and the storage ID is used for retrieving, downloading and deleting the picture in the storage unit.
The invention has the beneficial effects that: 1. compared with a central node metadata scheme, the distributed architecture is adopted to store metadata, so that the problems of capacity upper limit, performance bottleneck, high-availability configuration and the like in a central node mode are solved. 2. Compared with a metadata-free architecture, because metadata is stored in an independent system, data migration does not need to be considered when the storage system expands the capacity; the statistical analysis task does not need to traverse all storage files, but adopts a BDB inquiry mode, and is more efficient and convenient. 3. By using the concept of logical grouping, one logical grouping corresponds to one BDB file, so that the capacity expansion of a metadata system is facilitated.
Drawings
FIG. 1 is a schematic diagram of the structure of a distributed metadata system in an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a picture storage system composed of an application program, a storage unit and a metadata system in an embodiment of the present invention;
fig. 3 is a schematic diagram of storage distribution of metadata in a distributed metadata system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, in this embodiment, a distributed metadata system for picture storage includes a metadata server cluster composed of at least one group, where each group includes at least one metadata server; the metadata stored by each metadata server in the same group are the same, and the metadata are synchronized in the group in a binlog file form;
the metadata server stores metadata of a key-value structure using a BDB;
the BDB is an open-source embedded database, runs in the same process space with an application program, can be used as a MySQL storage engine, and has higher performance and concurrency than a common relational database. The BDB saves the metadata of the picture in the BDB file.
In this embodiment, the metadata server (mdserver) stores data of a key-value structure by using a Berkeley DB (hereinafter, referred to as BDB) by a distributed hash program. The key consists of three parts: namespace, table, keyname, this design is similar to the hierarchical partitioning of the database: namespace corresponds to database, and tables correspond to tables and keyname corresponding fields, so that data can be simply organized to improve query performance.
The BDB stores data in BDB files, a single BDB file supports 256TB at most, if an mDServer uses BDB files working on a 2TB disk, the metadata (including the directory, file name, storage ID, creation time and the like) of a picture is about 200Byte, and the BDB file can store about 110 hundred million pieces of metadata.
A metadata server cluster consists of one or more groups, each group having one or more service nodes (see fig. 1). The data stored by the nodes (metadata servers) in the same group are the same, and the data are synchronized in the group (group) in a binlog (operation log for data addition, deletion and the like).
In this embodiment, when a piece of metadata is added to the metadata server, a log is generated in the binlog file, where the content is "time, data content, and new added flag a", the synchronization thread reads the binlog file, pushes the data with the new added flag a to other metadata servers in the same group, and after the other metadata servers receive the data, inserts the metadata into the BDB file, and adds a log in the binlog file, where the content is "time, data content, and synchronous added flag a"; due to the presence of the sync increase flag a, the increase operation is not resynchronized to other metadata servers.
In this embodiment, when deleting a piece of metadata, the metadata server generates a log in the binlog file, where the content is "time, data content, and deletion flag D", reads the binlog file in the same thread, and pushes the data with the deletion flag D to other metadata servers in the same group, and after receiving the data, the other metadata servers delete the metadata in the BDB file, and add a log in the binlog file, where the content is "time, data content, and synchronous deletion flag D"; due to the presence of the sync addition flag d, the delete operation is not resynchronized to other metadata servers.
In this embodiment, when the metadata server cluster performs capacity expansion, the metadata server avoids hash distribution from changing in a logical grouping manner; the logical grouping mode is specifically that one physical grouping corresponds to one group of metadata servers, and the group of metadata servers are divided into a plurality of logical groupings; one metadata server can simultaneously support a plurality of logic groups, and each logic group corresponds to one BDB file; estimating the number of logic groups needed in the future at the initial stage of metadata system construction, configuring the logic groups to physical groups, migrating one or more logic groups to newly-added physical groups during capacity expansion, and copying corresponding BDB files to complete metadata migration.
In the metadata system capacity expansion process, when an operation environment is built, the number of required logical groups is estimated, for example, 16 logical groups are planned, the groups can be configured in 2 physical groups, and each physical group has 8 logical groups. And (3) adding 2 physical groups during capacity expansion, wherein 4 physical groups exist at the moment, and each physical group averagely has 4 logical groups, and copying the logical group BDB files to be migrated to the newly added server to complete data migration.
As shown in fig. 2, in the present embodiment, the metadata system combines a Web server, an application program, and a storage unit to construct a picture storage system; in the picture storage system, the Web server provides a picture access Restful or S3 format interface; the application program helps a user to send a picture access request; the storage unit consists of a management node and a plurality of storage nodes, and after the BDB file is uploaded to the storage unit, a storage ID associated with a storage position is generated; the metadata system stores the correspondence of storage IDs with user data.
The picture storage system comprises four parts: the system comprises a Web server, a Client, a storage unit and a metadata system. The application program comprises a plurality of clients. The Web server provides a picture access Restful or S3 format interface; a Client (such as a browser) helps a user to send a picture access HTTP request; the storage unit is not particularly limited, may be TFS, GlusterFS, FastDFS, etc., and generally consists of a management node NodeManager and a plurality of storage nodes dataservers, and after the BDB file is uploaded to the storage unit, a storage ID associated with a storage location is generated; the metadata system stores the corresponding relation between the picture storage ID and the user data, and the BDB file used by the mdserver can be placed in an SSD disk to improve the reading and writing speed.
In the embodiment, when the Client uploads the picture, the Client takes user parameters such as a directory and a file name, the Web server calls the SDK to store the picture file in the storage unit and obtain a storage ID, calculates the metadata of the picture through the SDK and stores the metadata in which group and which metadata server of the metadata server cluster, then inserts a piece of metadata with a key-value structure into the corresponding metadata server, the Web server returns the URL (Retful or S3 format) of the picture to an application program, and after the Client obtains the URL of the picture returned by the Web server, the Client uses the URL to retrieve, download and delete the picture;
the key comprises a picture directory and a file name, and the value comprises a storage ID and a creation time.
In this embodiment, the specific process of calculating which group of the metadata server cluster the metadata of the picture is stored in by the SDK is that,
1. dividing the hash value range associated with each group according to the number of the groups; that is, assuming that there are N groups in the metadata server cluster, the hash value is averagely divided into N segments, and each group corresponds to a segment of hash value range; as shown in FIG. 3, assuming that there are 3 groups in the metadata server cluster, the hash value (0x 00000000-0 xffffffff) is divided into 3 segments on average, group1 corresponds to 0x 00000000-0 x 5555555554, group2 corresponds to 0x 55555555555555-0 xaaaaaaa9, and group3 corresponds to 0 xaaaaaaaaaaa-0 xfffffff;
2. calculating a Hash value corresponding to the picture file name by using a BKDR Hash algorithm;
3. according to the hash value range of the hash value corresponding to the picture file name, locating the group where the metadata is located; as shown in FIG. 3, assuming that the hash value of file1 is 0x12345678 and the hash value of file2 is 0xaaaabbbb, then the file1 metadata is located in group1 and the file2 metadata is located in group 3;
4. calculating the ID of the metadata server according to the group where the metadata is located; the metadata server ID is the number of metadata servers within the group where the hash value% metadata is located.
In this embodiment, the Web server returns the URL of the picture to the application program, and after the application program obtains the URL of the picture returned by the Web server, the application program uses the URL to perform the operations of retrieving, downloading and deleting the picture specifically,
and after receiving a picture access request sent by a user through an application program, the Web server calls the SDK to perform access operation, the SDK analyzes the picture directory and the file name from the URL to construct a key, the key is used for inquiring the storage ID of the picture from the storage unit, and the storage ID is used for retrieving, downloading and deleting the picture in the storage unit.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained:
the invention provides a distributed metadata system for storing pictures, which adopts a distributed architecture to store metadata relative to a central node metadata scheme, thereby avoiding the problems of capacity upper limit, performance bottleneck, high-availability configuration and the like in a central node mode. Compared with a metadata-free architecture, the system has the advantages that the metadata are stored in the independent system, so that data migration is not required to be considered when the storage system is expanded; the statistical analysis task is more efficient and convenient by adopting a BDB inquiry mode instead of traversing all storage files. The system uses the concept of logical grouping, one logical grouping corresponds to one BDB file, and the capacity expansion of a metadata system is facilitated.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements should also be considered within the scope of the present invention.
Claims (8)
1. A distributed metadata system for picture storage, characterized by: the method comprises the steps that a metadata server cluster consisting of at least one group is included, and each group comprises at least one metadata server; the metadata stored by each metadata server in the same group are the same, and the metadata are synchronized in the group in a binlog file form;
the metadata server stores metadata of a key-value structure using a BDB;
the BDB is an embedded database with an open source and runs in the same process space with an application program; the BDB saves the metadata of the picture in the BDB file.
2. The distributed metadata system for picture storage according to claim 1, wherein: when a metadata server adds a piece of metadata, a log is generated in a binlog file, the content is 'time, data content and newly added marks', a synchronization thread reads the binlog file and pushes the data with the newly added marks to other metadata servers in the same group, after the other metadata servers receive the data, the metadata is inserted into a BDB file, a log is added in the binlog file, and the content is 'time, data content and synchronously added marks'; because of the sync added flag, the add operation is not resynchronized to other metadata servers.
3. The distributed metadata system for picture storage according to claim 1, wherein: when deleting a piece of metadata, the metadata server generates a log in a binlog file, wherein the content is 'time, data content and deletion mark', the same thread reads the binlog file, and pushes the data with the deletion mark to other metadata servers in the same group, after receiving the data, the other metadata servers delete the metadata in the BDB file, and add a log in the binlog file, wherein the content is 'time, data content and synchronous deletion mark'; because of the sync added flag, the delete operation is not resynchronized to other metadata servers.
4. The distributed metadata system for picture storage according to claim 1, wherein: when the metadata server cluster expands the capacity, the metadata server adopts a logical grouping mode to avoid the change of hash distribution; the logical grouping mode is specifically that one physical grouping corresponds to one group of metadata servers, and the group of metadata servers are divided into a plurality of logical groupings; one metadata server can simultaneously support a plurality of logic groups, and each logic group corresponds to one BDB file; estimating the number of logic groups needed in the future at the initial stage of metadata system construction, configuring the logic groups to physical groups, migrating one or more logic groups to newly-added physical groups during capacity expansion, and copying corresponding BDB files to complete metadata migration.
5. The distributed metadata system for picture storage according to claim 1, wherein: the metadata system is combined with the Web server, the application program and the storage unit to construct a picture storage system; in the picture storage system, the Web server provides a picture access Restful or S3 format interface; the application program helps a user to send a picture access request; the storage unit consists of a management node and a plurality of storage nodes, and after the BDB file is uploaded to the storage unit, a storage ID associated with a storage position is generated; the metadata system stores the correspondence of storage IDs with user data.
6. The distributed metadata system for picture storage according to claim 1, wherein: when a user uploads a picture by using an application program, the user parameters are carried, a Web server calls an SDK (software development kit) to store a picture file in a storage unit and acquire a storage ID (identity), calculates metadata of the picture through the SDK and stores the metadata in which group and which metadata server of a metadata server cluster, then inserts metadata of a key-value structure into a corresponding metadata server, the Web server returns a URL (uniform resource locator) of the picture to the application program, and the application program uses the URL to retrieve, download and delete the picture after acquiring the URL of the picture returned by the Web server;
the key comprises a picture directory and a file name, and the value comprises a storage ID and a creation time.
7. The distributed metadata system for picture storage according to claim 6, wherein: the concrete process of calculating which group of the metadata server cluster the metadata of the picture is stored in through the SDK is that,
dividing the hash value range associated with each group according to the number of the groups; that is, assuming that there are N groups in the metadata server cluster, the hash value is averagely divided into N segments, and each group corresponds to a segment of hash value range;
calculating a Hash value corresponding to the picture file name by using a BKDR Hash algorithm;
according to the hash value range of the hash value corresponding to the picture file name, locating the group where the metadata is located;
calculating the ID of the metadata server according to the group where the metadata is located; the metadata server ID is the number of metadata servers within the group where the hash value% metadata is located.
8. The distributed metadata system for picture storage according to claim 6, wherein: the Web server returns the URL of the picture to the application program, and after the application program acquires the URL of the picture returned by the Web server, the application program uses the URL to retrieve, download and delete the picture,
and after receiving a picture access request sent by a user through an application program, the Web server calls the SDK to perform access operation, the SDK analyzes the picture directory and the file name from the URL to construct a key, the key is used for inquiring the storage ID of the picture from the storage unit, and the storage ID is used for retrieving, downloading and deleting the picture in the storage unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210140735.9A CN114491111B (en) | 2022-02-16 | 2022-02-16 | Distributed metadata system for picture storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210140735.9A CN114491111B (en) | 2022-02-16 | 2022-02-16 | Distributed metadata system for picture storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114491111A true CN114491111A (en) | 2022-05-13 |
| CN114491111B CN114491111B (en) | 2022-09-16 |
Family
ID=81479773
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210140735.9A Active CN114491111B (en) | 2022-02-16 | 2022-02-16 | Distributed metadata system for picture storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114491111B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102833331A (en) * | 2012-08-21 | 2012-12-19 | 北京邦诺存储科技有限公司 | Cloud storage system and metadata writing-in method and metadata reading method thereof |
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN103294785A (en) * | 2013-05-17 | 2013-09-11 | 华中科技大学 | Packet-based metadata server cluster management method |
| AU2014271232A1 (en) * | 2004-10-25 | 2014-12-18 | Security First Corp. | Method and apparatus for managing access to data |
| CN111309266A (en) * | 2020-02-23 | 2020-06-19 | 苏州浪潮智能科技有限公司 | Distributed storage metadata system log optimization system and method based on ceph |
| CN113553314A (en) * | 2021-07-16 | 2021-10-26 | 北京天融信网络安全技术有限公司 | Service processing method, device, equipment and medium of super-convergence system |
-
2022
- 2022-02-16 CN CN202210140735.9A patent/CN114491111B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2014271232A1 (en) * | 2004-10-25 | 2014-12-18 | Security First Corp. | Method and apparatus for managing access to data |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN102833331A (en) * | 2012-08-21 | 2012-12-19 | 北京邦诺存储科技有限公司 | Cloud storage system and metadata writing-in method and metadata reading method thereof |
| CN103294785A (en) * | 2013-05-17 | 2013-09-11 | 华中科技大学 | Packet-based metadata server cluster management method |
| CN111309266A (en) * | 2020-02-23 | 2020-06-19 | 苏州浪潮智能科技有限公司 | Distributed storage metadata system log optimization system and method based on ceph |
| CN113553314A (en) * | 2021-07-16 | 2021-10-26 | 北京天融信网络安全技术有限公司 | Service processing method, device, equipment and medium of super-convergence system |
Non-Patent Citations (2)
| Title |
|---|
| 余洪春: "《Linux集群之美》", 31 January 2021 * |
| 吴伟等: "海量存储系统中高可扩展性元数据服务器集群设计", 《计算机科学》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114491111B (en) | 2022-09-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11520670B2 (en) | Method and apparatus for restoring data from snapshots | |
| US10209893B2 (en) | Massively scalable object storage for storing object replicas | |
| US9967298B2 (en) | Appending to files via server-side chunking and manifest manipulation | |
| US9898521B2 (en) | Massively scalable object storage system | |
| CN109783438B (en) | Distributed NFS system based on librados and construction method thereof | |
| US9183213B2 (en) | Indirection objects in a cloud storage system | |
| EP2498476B1 (en) | Massively scalable object storage system | |
| US10740300B1 (en) | Synchronization of metadata in a distributed storage system | |
| US7702640B1 (en) | Stratified unbalanced trees for indexing of data items within a computer system | |
| US9547706B2 (en) | Using colocation hints to facilitate accessing a distributed data storage system | |
| CN107562757B (en) | Query and access method, device and system based on distributed file system | |
| WO2018058949A1 (en) | Data storage method, device and system | |
| CN106484820B (en) | Renaming method, access method and device | |
| CN114610680A (en) | Method, device and equipment for managing metadata of distributed file system and storage medium | |
| CN114491111B (en) | Distributed metadata system for picture storage | |
| Klein et al. | Dxram: A persistent in-memory storage for billions of small objects | |
| Duan et al. | A multi‐channel architecture for metadata management in cloud storage systems by binding CPU‐cores to disks | |
| CN117215477A (en) | Data object storage method, device, computer equipment and storage medium | |
| Chalabi et al. | BlobSeer Scalability: A Multi-version Managers Approach | |
| Junping | Analysis of key technologies of distributed file system based on big data [J] | |
| CN114817253A (en) | Method for building document model based on search analysis engine and application thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |