WO2014177080A1 - Method and device for processing resource object storage - Google Patents

Method and device for processing resource object storage Download PDF

Info

Publication number
WO2014177080A1
WO2014177080A1 PCT/CN2014/077603 CN2014077603W WO2014177080A1 WO 2014177080 A1 WO2014177080 A1 WO 2014177080A1 CN 2014077603 W CN2014077603 W CN 2014077603W WO 2014177080 A1 WO2014177080 A1 WO 2014177080A1
Authority
WO
WIPO (PCT)
Prior art keywords
bucket
resource object
resource
stored
determining
Prior art date
Application number
PCT/CN2014/077603
Other languages
French (fr)
Chinese (zh)
Inventor
薛晔
周国胜
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2014177080A1 publication Critical patent/WO2014177080A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present invention relates to the field of communications, and in particular to a resource object storage processing method and apparatus.
  • BACKGROUND OF THE INVENTION A Chinese telecom cloud storage system is different from a traditional file system storage in that an object storage method based on a consistent HASH key value pair is used, and storage contents such as management resources and object attributes are distributed in a storage system in a key-value manner.
  • Each management resource (or object attribute) has a key in the system, and the resource itself (or attribute) is stored in the system as a value.
  • This storage method is compared with the traditional storage method. Performance improvement has a certain advantage.
  • storing resources in this way also solves the problem of security and physical isolation between different users to some extent.
  • the value is large, for example, it is a list containing thousands or tens of thousands of data, which will undoubtedly increase the difficulty of finding the data, and need to find a better storage method for the value.
  • an object uploading or downloading process first needs to access, authenticate, acquire attributes, etc. of the tenant, sub-tenant, and user to which the object belongs. If these resources are frequently accessed, the object will be The uploading and downloading rate has a certain impact, which affects the performance of the entire system. Furthermore, if there are too many users, the reasonable use of the storage space needs to be considered.
  • the present invention provides a resource object storage processing method and apparatus, so as to at least solve the problem of how to perform large-capacity storage processing in the related art, and which consumes a large space and is inconvenient to query.
  • a resource object storage processing method is provided, which is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket unit, and includes: receiving a storage to be stored a resource object; determining, according to a dynamic hash algorithm, a location of the bucket stored by the resource object; storing the resource object according to the determined location of the bucket.
  • determining the location of the bucket that is stored by the resource object according to the dynamic hashing algorithm includes: determining a hash algorithm corresponding to each bucket, where the bucket is located by using a keyword Key, and is used for each bucket.
  • determining, according to the dynamic hash algorithm, the location of the bucket that is stored by the resource object includes: determining, according to a first hash algorithm, a location of an initial bucket stored by the resource object; determining whether the initial bucket is full If the result of the determination is yes, the initial bucket splitting is triggered; and the split bucket and the bucket before the split store the existing resource object according to the second hash algorithm, wherein the existing resource object includes The resource object and the resource object stored in the bucket that existed before the split.
  • triggering the bucket splitting includes: determining whether there is a split pointer that points to the split bucket in advance; if the determination result is yes, preferentially triggering splitting of the bucket pointed by the split pointer.
  • the method further includes: receiving a resource access request; uploading the requested resource object according to the resource access request.
  • the resource according to the resource access request upload request includes: determining whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request.
  • a resource object storage processing apparatus is provided, based on a keyword-value
  • the storage method of the key-value, wherein the resource object is stored in a bucket unit comprising: a first receiving module, configured to receive a resource object to be stored; a determining module, configured to determine according to a dynamic hash algorithm a location of the bucket stored by the resource object; a storage module configured to store the resource object according to the determined location of the bucket.
  • the determining module includes: a first determining unit, configured to determine a hash algorithm corresponding to each bucket, where the bucket is located by using a keyword Key, and each bucket is named by using a predetermined rule; And the unit is configured to perform hash processing on the resource object according to the hash algorithm corresponding to the bucket to determine a bucket that stores the resource object, where the stored resource object corresponds to a value Value.
  • the determining module includes: a third determining unit, configured to determine, according to the first hash algorithm, a location of an initial bucket stored by the resource object; and a first determining unit, configured to determine whether the initial bucket is full; a triggering unit, configured to trigger the initial bucket splitting if the determining result of the determining unit is yes; and an allocating unit configured to store the existing resource according to the second hashing algorithm for the split bucket and the bucket before splitting
  • the object is reassigned, wherein the existing resource object includes the resource object and a resource object stored in a bucket existing before the splitting.
  • the triggering unit includes: a determining subunit, configured to determine whether there is a split pointer that points to the split bucket in advance; and a triggering subunit, configured to trigger the pair preferentially if the judgment result of the determining subunit is yes The bucket pointed by the split pointer splits.
  • the apparatus further includes: a second receiving module, configured to receive a resource access request; and an uploading module, configured to upload the requested resource object according to the resource access request.
  • the uploading module includes: a second determining unit, configured to determine whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request.
  • a storage method based on a keyword-value Key-Value is adopted, wherein the resource object is stored in a bucket unit, including: receiving a resource object to be stored; determining the resource according to a dynamic hash algorithm The location of the bucket stored by the object; storing the resource object according to the determined location of the bucket, solving the problem of how to perform large-capacity storage processing in the related art, and requiring a large space and inconvenient query, thereby achieving an improvement Storage efficiency, saving storage space, and improving query efficiency.
  • FIG. 2 is a block diagram showing a structure of a resource object storage processing apparatus according to an embodiment of the present invention
  • a preferred block diagram of the determination module 24 in the resource object storage processing device is shown in FIG. 4
  • FIG. 4 is a block diagram showing a preferred structure of the determination module 24 in the resource object storage processing device according to an embodiment of the present invention.
  • FIG. 5 is a block diagram showing a preferred structure of the trigger unit 46 in the determining module 24 in the resource object storage processing apparatus according to the embodiment of the present invention
  • FIG. 6 is a block diagram showing the structure of the resource object storage processing apparatus according to the embodiment of the present invention
  • a preferred block diagram of the uploading module 64 in the resource object storage processing device of the embodiment of the invention is a schematic diagram of a China Telecom cloud storage system architecture according to a preferred embodiment of the present invention
  • FIG. 9 is a schematic flowchart of a storage process according to a preferred embodiment of the present invention
  • FIG. 10 is a cloud storage system object uploading process according to a preferred embodiment of the present invention. Figure.
  • FIG. 1 is a flowchart of a resource object storage processing method according to an embodiment of the present invention, where the method is based on a key-value Key-Value storage method, wherein the resource The object is stored in a bucket. As shown in FIG. 1
  • Step S102 Receive a resource object to be stored
  • Step S104 determine, according to a dynamic hash algorithm, a location of a bucket stored by the resource object
  • S106 Store the resource object according to the determined location of the bucket.
  • the dynamic hashing algorithm is used to implement dynamic storage of resource objects based on the Key-Value storage method. Compared with the related technology, only the Key-Value mode is used for storage, or only the hash algorithm is used for storage.
  • the effective combination of the two not only solves the problem of how to carry out large-capacity storage processing in the related technology, but also consumes a large space and inconvenient query, thereby achieving the effects of improving storage efficiency, saving storage space, and improving query efficiency.
  • the following describes the storage method based on the keyword-value Key-Value, and determines the location of the bucket in which the resource object is stored according to the dynamic hash algorithm. First, the hash algorithm corresponding to each bucket is determined, where the bucket uses the keyword Key. Positioning, each bucket is named by using a predetermined rule; then, the resource object is hashed according to the hash algorithm corresponding to the bucket to determine the bucket of the storage resource object, wherein the stored resource object corresponds to the value Value.
  • the bucket naming rules can be various, for example, the following can be named The rule "list name_0, list name_1, list name_2 ", of course, other naming rules can also be used.
  • each element in the bucket has its own KEY before it is stored. By hashing this unique KEY value, it is placed in its own bucket.
  • determining the location of the bucket stored by the resource object according to the dynamic hash algorithm may be performed by: determining, according to the first hash algorithm, a location of the initial bucket stored by the resource object; determining whether the initial bucket is full; If yes, the initial bucket split is triggered; according to the second hash algorithm, the split bucket and the bucket before the split store the existing resource object, wherein the existing resource object includes the resource object and exists before the split.
  • the resource object stored in the bucket that is, when a new resource object is stored, the corresponding calculation is performed according to the dynamic hash algorithm, and then stored in the bucket obtained after the calculation.
  • the trigger bucket splitting may be solved in the following manner: First, it is determined whether there is a split pointer that points to the split bucket in advance; If the result of the determination is yes, the priority triggers the splitting of the bucket pointed to by the split pointer. After storing the resource object according to the determined location of the bucket, the method may further: receive the resource access request; upload the requested resource object according to the resource access request.
  • the resource object that is requested to be uploaded according to the resource access request includes: determining whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request, that is, authenticating the resource access request,
  • the security of the resource object is advantageously guaranteed.
  • a resource object storage processing device is also provided, which is used to implement the foregoing embodiments and preferred embodiments, and has not been described again.
  • the term "module" may implement a combination of software and/or hardware of a predetermined function.
  • the apparatus is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket unit, as shown in FIG. 2,
  • the apparatus includes a first receiving module 22, a determining module 24, and a storage module 26, which are described below.
  • the first receiving module 22 is configured to receive the resource object to be stored;
  • the determining module 24 is connected to the first receiving module 22, and is configured to determine the location of the bucket stored by the resource object according to the dynamic hash algorithm;
  • the storage module 26, Connected to the above determining module 24, it is arranged to store the resource object according to the determined location of the bucket.
  • the determining module 24 includes a first determining unit 32 and a second determining unit 34.
  • the determining module 24 is described below.
  • the first determining unit 32 is configured to determine a hash algorithm corresponding to each bucket, wherein the bucket is located by using a keyword Key, and each bucket is named by using a predetermined rule; and the second determining unit 34 is connected to the first determining unit. 32.
  • Set, according to the hash algorithm corresponding to the bucket perform hash processing on the resource object to determine a bucket that stores the resource object, where the stored resource object corresponds to the value Value.
  • the determining module 24 includes a third determining unit 42, a first determining unit 44, a triggering unit 46, and an allocating unit 48.
  • the determination module 24 will be described.
  • the third determining unit 42 is configured to determine, according to the first hash algorithm, the location of the initial bucket stored by the resource object; the first determining unit 44 is connected to the determining unit 42 to determine whether the initial bucket is full; the triggering unit 46, Connected to the first determining unit 44, configured to trigger the initial bucket splitting if the determining result of the determining unit is yes; the assigning unit 48 is connected to the triggering unit 46, and configured to split according to the second hashing algorithm.
  • the bucket and the bucket before the split store the existing resource object for reallocation, wherein the existing resource object includes the resource object and the resource object stored in the bucket existing before the split.
  • FIG. 5 is a block diagram showing a preferred structure of the triggering unit 46 in the determining module 24 in the resource object storage processing apparatus according to the embodiment of the present invention.
  • the triggering unit 46 includes a determining subunit 52 and a triggering subunit 54, below.
  • the trigger unit 46 will be described.
  • the determining sub-unit 52 is configured to determine whether there is a split pointer that points to the split bucket in advance; the trigger sub-unit 54 is connected to the judging sub-unit 52, and is set to trigger the pair preferentially if the judgment result of the judging sub-unit 52 is YES.
  • FIG. 6 is a structural block diagram of a resource object storage processing apparatus according to an embodiment of the present invention.
  • the apparatus includes a second receiving module 62 and an uploading module 64, in addition to all the modules shown in FIG. The device will be described.
  • the second receiving module 62 is connected to the storage module 26 and configured to receive a resource access request.
  • the uploading module 64 is connected to the second receiving module 62, and is configured to upload the requested resource object according to the resource access request.
  • FIG. 7 is a block diagram showing a preferred structure of the uploading module 64 in the resource object storage processing apparatus according to the embodiment of the present invention.
  • the uploading module 64 includes a second determining unit 72, and the second determining unit 72 is performed next. Description.
  • the second determining unit 72 is configured to determine whether the resource access request is legal.
  • the requested resource object is uploaded according to the resource access request.
  • a key-value based storage method is proposed.
  • the key-value-based resource management method includes: storing management resources (tenants, sub-tenants, users, etc.) based on key-value key values; learning dynamic storage of management resources by using dynamic hash processing method, and cloud storage system Neutron tenant, user resource management (add, delete, insert, modify, etc.); where, based on key-value key-value pairs to store management resources (tenants, sub-tenants, users, etc.), including:
  • key-value Storage can support the storage of massive data. Its distributed architecture determines that as long as there are enough machines, it can guarantee to store more data. Secondly, the key-value architecture can easily support tens of thousands of concurrent requests, making the system highly concurrent. Sex.
  • the dynamic storage of management resources is based on the dynamic hash processing method.
  • the key-value based storage architecture when the value is large (for example, a list), in order to save space and improve search performance, it is necessary to learn from dynamics.
  • the algorithm idea is used to solve this problem.
  • the dynamic hash algorithm can avoid the conflict problem generated by the static hash algorithm to a certain extent and solve the problem better.
  • the methods for implementing list management of sub-tenants and users in the cloud storage system include: First, sub-tenants and users have their own KEY values when they are created, and then sub-tenants and users are respectively stored in their respective sub-tenant lists through hash function calculation.
  • the management resource storage interface API, management resource list creation, management resource list deletion, management resource list modification, management resource search, etc. to facilitate upper-layer applications.
  • the traditional list storage method is abandoned, and the idea of dynamic hash algorithm is used to realize the allocation of space on demand, saving storage space, directly through HASH function. Positioning, shortening the average lookup length and improving performance.
  • the dynamic hash algorithm can solve the problem of space expansion and contraction well. Its basic idea is the same as the traditional hash. First, design the mapping function from the key field U to the storage space (hash file), through two HASH mappings. Function, used to recalculate the HASH value when the bucket is split.
  • a hash file is a set of records determined by a subfield of the key field U.
  • a record in the hash file is determined by a keyword in the key field U, and the file contains a plurality of buckets for storing records.
  • the basic unit for file operations is buckets. Each bucket can store b records. When more than b records are mapped to the same bucket through a hash function, the bucket overflows. At this point, the bucket is split (new bucket is added) Space), using the new HASH algorithm, recalculate the values in the current bucket and rearrange the data in the bucket.
  • the dynamic hashing idea is adopted to manage the management resources of sub-tenants and users, and to quickly query and locate sub-tenants and users.
  • FIG. 8 is a schematic diagram of a China Telecom cloud storage system architecture according to a preferred embodiment of the present invention.
  • the system mainly includes four parts, a physical storage layer, a storage interface layer, a service encapsulation layer, and a system management layer.
  • This structure illustrates a preferred embodiment of the present invention.
  • the physical storage layer provides physical storage space to the cloud storage system.
  • the management resources and objects in the system are stored therein.
  • the storage interface layer provides addition, deletion, and randomization, storage policy settings, and objects in the system according to the requirements of the upper-layer service system.
  • An interface for managing resource storage, the service encapsulation layer is an adaptation of related service interfaces, and provides a service interface that meets customer requirements.
  • the system management layer maintains, manages, and configures the system to implement system configuration and performance in the system. Statistics, operation and maintenance data collection and network management related functions.
  • the content involved is at the storage interface layer, which is an improvement and re-encapsulation of the management resource storage interface API in the original storage interface.
  • the encapsulated API includes the creation of a management resource list, the query, insertion, and deletion of management resources, and the unified application of the convenient service, and convenient storage for the next, saving physical space.
  • 9 is a schematic flowchart of a storage process according to a preferred embodiment of the present invention. As shown in FIG. 9, the storage of a sub-tenant is taken as an example.
  • Step S904 after the bucket STL_tenant_0 is split according to the new HASH algorithm, subt4 is stored in STL_tenant_2, subtl and subt6 remain in the bucket STL_tenant_0, and the subt6 storage location is moved forward, and the split pointer points to the bucket STL_tenant_l, also by the HI algorithm, STL_tenant_l splits, the newly generated buckets STL_tenant_3, subt5, sub7 are stored in STL_tenant_3 according to the new algorithm, sub2, sub8 are left in the original STL tenant l, and are discharged in the new order.
  • FIG. 10 is a flowchart of uploading an object of a cloud storage system according to a preferred embodiment of the present invention. As shown in FIG. 10, the process includes the following steps: an authentication step, a resource access processing, and an object upload processing. The following process is performed based on FIG.
  • Step S1002 Authentication processing: mainly implementing authentication between the application and the system;
  • Step S1004 The resource access processing is mainly used for accessing the management resource, and determining the legality of the request according to the attribute, and the following processing manner may be used during the operation: User status, whether the user status is available and whether the user has access, read the sub-tenant attribute of the user, determine whether the sub-tenant status is available, etc., and implement management by calling the encapsulated API (search resource search, etc.)
  • the access of the resource the attribute of the required resource (for example, the basic attribute of the state of the management resource);
  • Step S1006 which is mainly used to implement uploading of the object according to the request.
  • the search efficiency is greatly improved, thereby improving the rate of object uploading.
  • modules or steps of the present invention can be implemented by a general-purpose computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices.
  • they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein.
  • the steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • a resource object storage processing method and apparatus provided by an embodiment of the present invention have the following beneficial effects: Solving the problem of how to perform large-capacity storage processing in the related art, and requiring a large space and inconvenient query. In turn, it improves storage efficiency, saves storage space, and improves query efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Disclosed are a method and device for processing resource object storage. The method comprises: being based on a Key-Value storage manner, wherein the resource object is stored in units of bucket, comprising: receiving the resource object to be stored; determining the position of a bucket stored with the resource object according to a dynamic hash algorithm; and storing the resource object according to the determined position of the bucket. By means of the present invention, the problems in the related art of how to process large capacity storage, large space consumption and being inconvenient for query are solved, thus achieving the effects of improving the storage efficiency, saving the storage space and improving the query efficiency.

Description

资源对象存储处理方法及装置 技术领域 本发明涉及通信领域, 具体而言, 涉及一种资源对象存储处理方法及装置。 背景技术 中国电信云存储系统区别于传统的文件系统存储, 采用了基于一致性 HASH键值 对的对象存储方式,管理资源和对象属性等存储内容都以 key-value的方式分布于存储 系统中, 每个管理资源 (或对象属性) 在系统中都拥有一个 key, 而资源本身 (或属 性) 都以 value 的形式存储于系统中, 这种存储方式, 相较于传统的存储方式而言, 在性能提升上占有一定优势, 另一方面, 用这种方式存储资源, 也在一定程度上解决 了不同用户间安全性和物理隔离的问题。 对于 key-value的存储模式, 如果 value值较大, 例如, 是一个含有几千、 几万个 数据的列表, 这无疑会增加数据的查找难度, 需要为 value寻找一种较优的存储方法。 在目前的云存储系统中,一个对象的上传或下载流程, 首先要对该对象的所属的租户、 子租户、 用户进行访问, 鉴权, 获取属性等, 如果频繁地访问这些资源, 将对对象的 上传下载速率造成一定影响, 从而影响整个系统的性能, 再者, 如果用户过多, 对存 储空间的合理利用也是需要好好思量的, 因此, 在相关技术中存在如何进行大容量存 储处理, 以及耗费空间大, 查询不便的问题。 发明内容 本发明提供了一种资源对象存储处理方法及装置, 以至少解决相关技术中存在如 何进行大容量存储处理, 以及耗费空间大, 查询不便的问题。 根据本发明的一个方面, 提供了一种资源对象存储处理方法, 基于关键字-值 Key-Value的存储方式, 其中, 所述资源对象以桶为单位进行存储, 包括: 接收到要进 行存储的资源对象; 依据动态哈希算法确定所述资源对象存储的所述桶的位置; 依据 确定的所述桶的位置存储所述资源对象。 优选地, 依据所述动态哈希算法确定所述资源对象存储的所述桶的位置包括: 确 定各个桶对应的哈希算法, 其中, 所述桶通过关键字 Key来定位, 对每个桶采用预定 规则进行命名; 依据桶对应的所述哈希算法对所述资源对象进行哈希处理确定存储所 述资源对象的桶, 其中, 存储的所述资源对象对应于值 Value。 优选地, 依据所述动态哈希算法确定所述资源对象存储的所述桶的位置包括: 依 据第一哈希算法确定所述资源对象存储的初始桶的位置; 判断所述初始桶是否已满; 在判断结果为是的情况下, 触发所述初始桶分裂; 根据第二哈希算法对分裂的桶和分 裂之前的桶存放现有资源对象进行重新分配, 其中, 所述现有资源对象包括所述资源 对象和分裂之前所存在的桶中所存储的资源对象。 优选地, 在判断初始桶已满, 触发桶分裂包括: 判断是否存在预先指向分裂桶的 分裂指针; 在判断结果为是的情况下, 优先触发对所述分裂指针指向的桶进行分裂。 优选地, 在依据确定的所述桶的位置存储所述资源对象之后, 还包括: 接收资源 访问请求; 依据所述资源访问请求上传请求的所述资源对象。 优选地, 依据所述资源访问请求上传请求的所述资源包括: 判断所述资源访问请 求是否合法, 在判断结果为是的情况下, 依据所述资源访问请求上传请求的所述资源 对象。 根据本发明的另一方面, 提供了一种资源对象存储处理装置, 基于关键字-值TECHNICAL FIELD The present invention relates to the field of communications, and in particular to a resource object storage processing method and apparatus. BACKGROUND OF THE INVENTION A Chinese telecom cloud storage system is different from a traditional file system storage in that an object storage method based on a consistent HASH key value pair is used, and storage contents such as management resources and object attributes are distributed in a storage system in a key-value manner. Each management resource (or object attribute) has a key in the system, and the resource itself (or attribute) is stored in the system as a value. This storage method is compared with the traditional storage method. Performance improvement has a certain advantage. On the other hand, storing resources in this way also solves the problem of security and physical isolation between different users to some extent. For the key-value storage mode, if the value is large, for example, it is a list containing thousands or tens of thousands of data, which will undoubtedly increase the difficulty of finding the data, and need to find a better storage method for the value. In the current cloud storage system, an object uploading or downloading process first needs to access, authenticate, acquire attributes, etc. of the tenant, sub-tenant, and user to which the object belongs. If these resources are frequently accessed, the object will be The uploading and downloading rate has a certain impact, which affects the performance of the entire system. Furthermore, if there are too many users, the reasonable use of the storage space needs to be considered. Therefore, how to perform large-capacity storage processing in the related art, and It takes a lot of space and is inconvenient to query. SUMMARY OF THE INVENTION The present invention provides a resource object storage processing method and apparatus, so as to at least solve the problem of how to perform large-capacity storage processing in the related art, and which consumes a large space and is inconvenient to query. According to an aspect of the present invention, a resource object storage processing method is provided, which is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket unit, and includes: receiving a storage to be stored a resource object; determining, according to a dynamic hash algorithm, a location of the bucket stored by the resource object; storing the resource object according to the determined location of the bucket. Preferably, determining the location of the bucket that is stored by the resource object according to the dynamic hashing algorithm includes: determining a hash algorithm corresponding to each bucket, where the bucket is located by using a keyword Key, and is used for each bucket. Book The naming of the rule is performed by performing hash processing on the resource object according to the hash algorithm corresponding to the bucket to determine a bucket storing the resource object, where the stored resource object corresponds to the value Value. Preferably, determining, according to the dynamic hash algorithm, the location of the bucket that is stored by the resource object includes: determining, according to a first hash algorithm, a location of an initial bucket stored by the resource object; determining whether the initial bucket is full If the result of the determination is yes, the initial bucket splitting is triggered; and the split bucket and the bucket before the split store the existing resource object according to the second hash algorithm, wherein the existing resource object includes The resource object and the resource object stored in the bucket that existed before the split. Preferably, when it is determined that the initial bucket is full, triggering the bucket splitting includes: determining whether there is a split pointer that points to the split bucket in advance; if the determination result is yes, preferentially triggering splitting of the bucket pointed by the split pointer. Preferably, after storing the resource object according to the determined location of the bucket, the method further includes: receiving a resource access request; uploading the requested resource object according to the resource access request. Preferably, the resource according to the resource access request upload request includes: determining whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request. According to another aspect of the present invention, a resource object storage processing apparatus is provided, based on a keyword-value
Key-Value的存储方式, 其中, 所述资源对象以桶为单位进行存储, 包括: 第一接收模 块, 设置为接收到要进行存储的资源对象; 确定模块, 设置为依据动态哈希算法确定 所述资源对象存储的所述桶的位置; 存储模块, 设置为依据确定的所述桶的位置存储 所述资源对象。 优选地, 所述确定模块包括: 第一确定单元, 设置为确定各个桶对应的哈希算法, 其中, 所述桶通过关键字 Key来定位, 对每个桶采用预定规则进行命名; 第二确定单 元, 设置为依据桶对应的所述哈希算法对所述资源对象进行哈希处理确定存储所述资 源对象的桶, 其中, 存储的所述资源对象对应于值 Value。 优选地, 所述确定模块包括: 第三确定单元, 设置为依据第一哈希算法确定所述 资源对象存储的初始桶的位置; 第一判断单元, 设置为判断所述初始桶是否已满; 触 发单元, 设置为在所述判断单元的判断结果为是的情况下, 触发所述初始桶分裂; 分 配单元, 设置为根据第二哈希算法对分裂的桶和分裂之前的桶存放现有资源对象进行 重新分配, 其中, 所述现有资源对象包括所述资源对象和分裂之前所存在的桶中所存 储的资源对象。 优选地, 所述触发单元包括: 判断子单元, 设置为判断是否存在预先指向分裂桶 的分裂指针; 触发子单元, 设置为在所述判断子单元的判断结果为是的情况下, 优先 触发对所述分裂指针指向的桶进行分裂。 优选地, 该装置还包括: 第二接收模块, 设置为接收资源访问请求; 上传模块, 设置为依据所述资源访问请求上传请求的所述资源对象。 优选地, 所述上传模块包括: 第二判断单元, 设置为判断所述资源访问请求是否 合法, 在判断结果为是的情况下, 依据所述资源访问请求上传请求的所述资源对象。 通过本发明, 采用基于关键字-值 Key-Value的存储方式, 其中, 所述资源对象以 桶为单位进行存储, 包括: 接收到要进行存储的资源对象; 依据动态哈希算法确定所 述资源对象存储的所述桶的位置; 依据确定的所述桶的位置存储所述资源对象, 解决 了相关技术中存在如何进行大容量存储处理, 以及耗费空间大, 查询不便的问题, 进 而达到了提高存储效率, 节省存储空间, 以及提高查询效率的效果。 附图说明 此处所说明的附图用来提供对本发明的进一步理解, 构成本申请的一部分, 本发 明的示意性实施例及其说明用于解释本发明, 并不构成对本发明的不当限定。 在附图 中: 图 1是根据本发明实施例的资源对象存储处理方法的流程图; 图 2是根据本发明实施例的资源对象存储处理装置的结构框图; 图 3是根据本发明实施例的资源对象存储处理装置中确定模块 24的优选结构框图 一; 图 4是根据本发明实施例的资源对象存储处理装置中确定模块 24的优选结构框图 The storage method of the key-value, wherein the resource object is stored in a bucket unit, comprising: a first receiving module, configured to receive a resource object to be stored; a determining module, configured to determine according to a dynamic hash algorithm a location of the bucket stored by the resource object; a storage module configured to store the resource object according to the determined location of the bucket. Preferably, the determining module includes: a first determining unit, configured to determine a hash algorithm corresponding to each bucket, where the bucket is located by using a keyword Key, and each bucket is named by using a predetermined rule; And the unit is configured to perform hash processing on the resource object according to the hash algorithm corresponding to the bucket to determine a bucket that stores the resource object, where the stored resource object corresponds to a value Value. Preferably, the determining module includes: a third determining unit, configured to determine, according to the first hash algorithm, a location of an initial bucket stored by the resource object; and a first determining unit, configured to determine whether the initial bucket is full; a triggering unit, configured to trigger the initial bucket splitting if the determining result of the determining unit is yes; and an allocating unit configured to store the existing resource according to the second hashing algorithm for the split bucket and the bucket before splitting The object is reassigned, wherein the existing resource object includes the resource object and a resource object stored in a bucket existing before the splitting. Preferably, the triggering unit includes: a determining subunit, configured to determine whether there is a split pointer that points to the split bucket in advance; and a triggering subunit, configured to trigger the pair preferentially if the judgment result of the determining subunit is yes The bucket pointed by the split pointer splits. Preferably, the apparatus further includes: a second receiving module, configured to receive a resource access request; and an uploading module, configured to upload the requested resource object according to the resource access request. Preferably, the uploading module includes: a second determining unit, configured to determine whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request. According to the present invention, a storage method based on a keyword-value Key-Value is adopted, wherein the resource object is stored in a bucket unit, including: receiving a resource object to be stored; determining the resource according to a dynamic hash algorithm The location of the bucket stored by the object; storing the resource object according to the determined location of the bucket, solving the problem of how to perform large-capacity storage processing in the related art, and requiring a large space and inconvenient query, thereby achieving an improvement Storage efficiency, saving storage space, and improving query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are set to illustrate,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 1 is a flowchart of a resource object storage processing method according to an embodiment of the present invention; FIG. 2 is a block diagram showing a structure of a resource object storage processing apparatus according to an embodiment of the present invention; A preferred block diagram of the determination module 24 in the resource object storage processing device is shown in FIG. 4; FIG. 4 is a block diagram showing a preferred structure of the determination module 24 in the resource object storage processing device according to an embodiment of the present invention.
图 5是根据本发明实施例的资源对象存储处理装置中确定模块 24中触发单元 46 的优选结构框图; 图 6是根据本发明实施例的资源对象存储处理装置的结构框图; 图 7是根据本发明实施例的资源对象存储处理装置中上传模块 64的优选结构框 图; 图 8是根据本发明优选实施方式的中国电信云存储系统架构示意图; 图 9是根据本发明优选实施方式的存储处理的流程示意图; 图 10是根据本发明优选实施方式的云存储系统对象上传流程图。 具体实施方式 下文中将参考附图并结合实施例来详细说明本发明。 需要说明的是, 在不冲突的 情况下, 本申请中的实施例及实施例中的特征可以相互组合。 在本实施例中提供了一种资源对象存储处理方法, 图 1是根据本发明实施例的资 源对象存储处理方法的流程图, 该方法基于关键字-值 Key-Value的存储方式, 其中, 资源对象以桶为单位进行存储, 如图 1所示, 该流程包括如下步骤: 步骤 S102, 接收到要进行存储的资源对象; 步骤 S104, 依据动态哈希算法确定资源对象存储的桶的位置; 步骤 S106, 依据确定的桶的位置存储资源对象。 通过上述步骤, 基于 Key-Value的存储方式, 采用动态哈希算法实现资源对象的 动态存储, 相对于相关技术中仅仅依据 Key-Value模式进行存储, 或是仅仅依据哈希 算法进行存储, 不能将两者进行有效地结合, 不仅解决了相关技术中存在如何进行大 容量存储处理, 以及耗费空间大, 查询不便的问题, 进而达到了提高存储效率, 节省 存储空间, 以及提高查询效率的效果。 下面对基于关键字-值 Key-Value的存储方式, 依据动态哈希算法确定资源对象存 储的桶的位置进行说明, 首先,确定各个桶对应的哈希算法,其中,桶通过关键字 Key 来定位, 对每个桶采用预定规则进行命名; 之后, 依据桶对应的哈希算法对资源对象 进行哈希处理确定存储资源对象的桶, 其中, 存储的资源对象对应于值 Value。 δΡ, 首 先, 动态哈希表, 包括 N个哈希桶 (N%2=0), 每个哈希桶用一个 key值来定位, 桶 的命名规则可以有多种, 例如, 可以采用以下命名规则"列表名 _0, 列表名 _1, 列表名 _2 ... ... ", 当然, 也可以采用其它的命名规则。 其次, 桶中的每个元素存入之前有自 己的 KEY, 通过对这个唯一的 KEY值进行哈希, 放入各自桶中。 优选地, 依据动态哈希算法确定资源对象存储的桶的位置可以采用以下处理: 依 据第一哈希算法确定资源对象存储的初始桶的位置; 判断初始桶是否已满; 在判断结 果为是的情况下, 触发初始桶分裂; 根据第二哈希算法对分裂的桶和分裂之前的桶存 放现有资源对象进行重新分配, 其中, 现有资源对象包括资源对象和分裂之前所存在 的桶中所存储的资源对象, 即在对一个新的资源对象进行存储时, 依据动态哈希算法 分别进行相应的计算, 之后存储到计算后获得的桶中。 其中, 需要说明的是, 在通过分裂指针对分裂桶进行相应的指示时, 判断初始桶 已满, 触发桶分裂时可以采用以下方式来解决: 先判断是否存在预先指向分裂桶的分 裂指针; 在判断结果为是的情况下, 优先触发对分裂指针指向的桶进行分裂。 在依据确定的桶的位置存储资源对象之后, 还可以: 接收资源访问请求; 依据资 源访问请求上传请求的资源对象。另外,依据资源访问请求上传请求的资源对象包括, 可以先判断资源访问请求是否合法, 在判断结果为是的情况下, 依据资源访问请求上 传请求的资源对象, 即对资源访问请求进行鉴权, 有利地保证了资源对象的安全。 在本实施例中还提供了一种资源对象存储处理装置, 该装置用于实现上述实施例 及优选实施方式, 已经进行过说明的不再赘述。 如以下所使用的, 术语 "模块"可以 实现预定功能的软件和 /或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来 实现, 但是硬件, 或者软件和硬件的组合的实现也是可能并被构想的。 图 2是根据本发明实施例的资源对象存储处理装置的结构框图, 该装置基于关键 字-值 Key-Value的存储方式, 其中, 资源对象以桶为单位进行存储, 如图 2所示, 该 装置包括第一接收模块 22、 确定模块 24和存储模块 26, 下面对该装置进行说明。 第一接收模块 22, 设置为接收到要进行存储的资源对象; 确定模块 24, 连接至上 述第一接收模块 22, 设置为依据动态哈希算法确定资源对象存储的桶的位置; 存储模 块 26, 连接至上述确定模块 24, 设置为依据确定的桶的位置存储资源对象。 图 3是根据本发明实施例的资源对象存储处理装置中确定模块 24的优选结构框图 一, 该确定模块 24包括第一确定单元 32和第二确定单元 34, 下面对该确定模块 24 进行说明。 第一确定单元 32, 设置为确定各个桶对应的哈希算法, 其中, 桶通过关键字 Key 来定位, 对每个桶采用预定规则进行命名; 第二确定单元 34, 连接至上述第一确定单 元 32, 设置为依据桶对应的哈希算法对资源对象进行哈希处理确定存储资源对象的 桶, 其中, 存储的该资源对象对应于值 Value。 图 4是根据本发明实施例的资源对象存储处理装置中确定模块 24的优选结构框图 二, 该确定模块 24包括第三确定单元 42、第一判断单元 44、触发单元 46和分配单元 48, 下面对该确定模块 24进行说明。 第三确定单元 42, 设置为依据第一哈希算法确定资源对象存储的初始桶的位置; 第一判断单元 44, 连接至上述确定单元 42, 设置为判断初始桶是否已满; 触发单元 46, 连接至上述第一判断单元 44, 设置为在判断单元的判断结果为是的情况下, 触发 初始桶分裂; 分配单元 48, 连接至上述触发单元 46, 设置为根据第二哈希算法对分裂 的桶和分裂之前的桶存放现有资源对象进行重新分配, 其中, 现有资源对象包括资源 对象和分裂之前所存在的桶中所存储的资源对象。 图 5是根据本发明实施例的资源对象存储处理装置中确定模块 24中触发单元 46 的优选结构框图, 如图 5所示, 该触发单元 46包括判断子单元 52和触发子单元 54, 下面对该触发单元 46进行说明。 判断子单元 52,设置为判断是否存在预先指向分裂桶的分裂指针;触发子单元 54, 连接至上述判断子单元 52, 设置为在判断子单元 52的判断结果为是的情况下, 优先 触发对分裂指针指向的桶进行分裂。 图 6是根据本发明实施例的资源对象存储处理装置的结构框图, 如图 6所示, 该 装置除包括图 2所示的所有模块外, 还包括第二接收模块 62和上传模块 64, 下面对 该装置进行说明。 第二接收模块 62, 连接至上述存储模块 26, 设置为接收资源访问请求; 上传模块 64, 连接至上述第二接收模块 62, 设置为依据资源访问请求上传请求的资源对象。 图 7是根据本发明实施例的资源对象存储处理装置中上传模块 64的优选结构框 图, 如图 7所示, 该上传模块 64包括第二判断单元 72, 下面对该第二判断单元 72进 行说明。 第二判断单元 72,设置为判断资源访问请求是否合法,在判断结果为是的情况下, 依据资源访问请求上传请求的资源对象。 基于相关技术中的上述问题, 为了更好地实现云存储系统中子租户、 用户等管理 资源的统一管理、提高管理效率, 在本实施例中, 提出了一种基于 key-value的存储方 法。 该基于 key-value的资源管理方法,包括:基于 key-value键值对存储管理资源(租 户、 子租户、 用户等); 借鉴动态哈希处理方法实现管理资源的动态存储, 以及, 云存 储系统中子租户、 用户的资源管理 (新增、 删除、 插入、 修改等操作); 其中, 基于 key-value键值对存储管理资源 (租户、 子租户、 用户等), 包括: 首先, key-value存 储可以支持海量数据的存储, 它的分布式架构决定了只要有足够的机器, 就能保证存 储更多的数据; 其次, key-value架构可以轻松支持上万的并发请求, 使系统具备高并 发性。 借鉴动态哈希的处理方法实现管理资源的动态存储包括: 首先, 基于 key-value 的存储架构, 当 value值较大 (比如是一个列表) 时, 为了节省空间, 提高查找性能, 需要借鉴动态哈希算法思想来解决这一问题; 其次, 动态哈希算法能够在一定程度上 很好地避免静态哈希算法所产生的冲突问题, 更好地解决问题。 实现云存储系统中子租户、 用户的列表管理的方式包括: 首先, 子租户、 用户创 建时有各自的 KEY值, 然后, 通过哈希函数计算将子租户、用户分别存入各自的子租 户列表、 用记列表; 其次, 编写管理资源存储接口 API, 管理资源列表创建、 管理资 源列表删除、 管理资源列表修改、 管理资源查找等, 方便上层应用。 基于中国电信云存储系统的这种 KEY- VALUE的存储模式, 摒弃了传统的列表存 储方式, 借鉴了动态哈希算法的思想来实现的, 对空间按需分配, 节省存储空间, 通 过 HASH函数直接定位, 缩短平均查找长度, 提高性能。 动态哈希算法能很好地解决空间的扩充和收缩问题, 它的基本思想同传统哈希一 样, 首先设计从关键字域 U到存储空间 (哈希文件) 的映射函数, 通过两个 HASH映 射函数, 用于在桶分裂时重新计算 HASH值。一个哈希文件是关键字域 U的一个子域 所确定的记录集合, 哈希文件中的一个记录是由关键字域 U中的一个关键字确定, 文 件含有若干个用于存放记录的桶, 对文件操作的基本单位是桶, 每个桶可存放 b个记 录, 当有超过 b个记录通过哈希函数映射到同一个桶时, 该桶溢出, 此时将桶进行分 裂 (新增一个桶空间), 采用新的 HASH算法, 对当前桶中数值重新进行计算, 重新 排布桶中数据。 下面结合附图对本发明优选实施方式进行说明。 基于 key-value的存储模式, 采用动态哈希的思想, 对子租户、用户等管理资源的 统一管理, 实现子租户、 用户等资源的快速查询、 定位。 图 8是根据本发明优选实施方式的中国电信云存储系统架构示意图,如图 8所示, 该系统主要包括四个部分, 物理存储层、 存储接口层、 业务封装层、 系统管理层, 下 面结合该结构对本发明优选实施方式进行说明。 物理存储层向云存储系统提供物理存储空间, 系统中的管理资源和对象存储在其 中, 存储接口层主要根据上层业务系统的需求, 提供系统中对象的增删改查、 随机更 新、 存储策略设置、 管理资源存储等接口, 业务封装层, 是对相关业务接口的适配, 对外提供符合客户要求的业务接口, 系统管理层, 是对系统的维护、 管理和配置, 实 现系统中的系统配置, 性能统计、 运维数据采集和网管相关功能。 在本实施例中, 所涉及的内容是在存储接口层, 是对原存储接口中关于管理资源 存储接口 API的一个改进和重新封装。 其中, 封装的 API包括管理资源列表的创建, 管理资源的查询、 插入, 删除等, 对上方便业务统一应用, 对下方便存储, 节省物理 空间。 图 9是根据本发明优选实施方式的存储处理的流程示意图, 如图 9所示, 以对子 租户的存储为例进行说明, 假设子租户资源列表对应的 key为 STL_租户名称, 初始 化列表为 2 X 4, 该流程包括如下步骤: 步骤 S902, 资源列表中已插入 subtl〜subt7这 7个子租户, 分裂指针 splitptr指向 桶 STL_tenant_0, 桶 STL_tenant_l已满, 两个 HASH算法分别为 H0=Hash(key)%2, Hl=Hash(key)%4, 此时要插入子租户 subt8, 用 HO计算 subt8应插入的桶的位置, 所得 postion为 STL_tenant_l, 由于 STL_tenant_l已满, 此时触发桶分裂, 但是, 因 为此时分裂指针 splitptr指向 STL_tenant_0, 所以先将 subt8放入溢出页, 然后对桶 STL_tenant_0进行分裂, 以新的算法 HI来重新计算 STL_tenant_0内的数据, 以决定 原 STL_tenant_0中的数据 subl, sub4, sub6是放入 STL_tenant_0还是 STL_tenant_2 中。 步骤 S904, 桶 STL_tenant_0按新的 HASH算法分裂后, subt4存入 STL_tenant_2 中, subtl和 subt6仍留在桶 STL_tenant_0中, 并且 subt6存储位置前移, 此时分裂指 针指向桶 STL_tenant_l, 同样按 HI 算法, 对 STL_tenant_l 进行分裂, 新生成桶 STL_tenant_3 , subt5, sub7 按新的算法存入 STL_tenant_3 中, sub2, sub8 留在原 STL tenant l中, 并按新的顺序排放。 步骤 S906, STL_tenant_l 分裂后, 此时桶的个数为 4, 分裂指针重新指向 STL_tenant_0桶, 并且 HO = Hash(key)%4, Hl=Hash(key)%8, 开始新的一轮的插入 和分裂, 当桶的个数为 8时, 分裂指针又重新指向 STL_tenant_0, HO = Hash(key)%8, HI = Hash(key)%16, 依次类推, 如此循环。 图 10是根据本发明优选实施方式的云存储系统对象上传流程图, 如图 10所示, 该流程包括如下步骤: 鉴权步骤, 资源访问处理, 以及对象上传处理, 下面基于图 10 对上述流程进行说明。 步骤 S1002, 鉴权处理: 主要实现应用与系统之间的鉴权; 步骤 S1004, 资源访问处理, 主要用于访问管理资源, 根据属性判断请求的合法 性, 操作时可以采用以下处理方式: 读取用户状态, 判断用户状态是否可用以及用户 是否有权访问, 读取用户所属子租户属性, 判断子租户状态是否可用等处理, 通过对 所封装的 API的调用 (管理资源的查找等) 来实现管理资源的访问, 获取所需资源的 属性 (例如, 管理资源的状态等基本属性); 步骤 S1006, 主要用于根据请求来实现对象的上传。 采用上述实施例及优选实施方式的存储方法, 大大提高了查找效率, 从而提升了 对象上传的速率。 显然, 本领域的技术人员应该明白, 上述的本发明的各模块或各步骤可以用通用 的计算装置来实现, 它们可以集中在单个的计算装置上, 或者分布在多个计算装置所 组成的网络上, 可选地, 它们可以用计算装置可执行的程序代码来实现, 从而, 可以 将它们存储在存储装置中由计算装置来执行, 并且在某些情况下, 可以以不同于此处 的顺序执行所示出或描述的步骤, 或者将它们分别制作成各个集成电路模块, 或者将 它们中的多个模块或步骤制作成单个集成电路模块来实现。 这样, 本发明不限制于任 何特定的硬件和软件结合。 以上所述仅为本发明的优选实施例而已, 并不用于限制本发明, 对于本领域的技 术人员来说, 本发明可以有各种更改和变化。 凡在本发明的精神和原则之内, 所作的 任何修改、 等同替换、 改进等, 均应包含在本发明的保护范围之内。 工业实用性 如上所述, 本发明实施例提供的一种资源对象存储处理方法及装置具有以下 有益效果: 解决了相关技术中存在如何进行大容量存储处理, 以及耗费空间大, 查 询不便的问题, 进而达到了提高存储效率, 节省存储空间, 以及提高查询效率的效
Figure imgf000011_0001
5 is a block diagram showing a preferred structure of the trigger unit 46 in the determining module 24 in the resource object storage processing apparatus according to the embodiment of the present invention; FIG. 6 is a block diagram showing the structure of the resource object storage processing apparatus according to the embodiment of the present invention; A preferred block diagram of the uploading module 64 in the resource object storage processing device of the embodiment of the invention; 8 is a schematic diagram of a China Telecom cloud storage system architecture according to a preferred embodiment of the present invention; FIG. 9 is a schematic flowchart of a storage process according to a preferred embodiment of the present invention; FIG. 10 is a cloud storage system object uploading process according to a preferred embodiment of the present invention. Figure. BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. In this embodiment, a resource object storage processing method is provided. FIG. 1 is a flowchart of a resource object storage processing method according to an embodiment of the present invention, where the method is based on a key-value Key-Value storage method, wherein the resource The object is stored in a bucket. As shown in FIG. 1 , the process includes the following steps: Step S102: Receive a resource object to be stored; Step S104, determine, according to a dynamic hash algorithm, a location of a bucket stored by the resource object; S106. Store the resource object according to the determined location of the bucket. Through the above steps, the dynamic hashing algorithm is used to implement dynamic storage of resource objects based on the Key-Value storage method. Compared with the related technology, only the Key-Value mode is used for storage, or only the hash algorithm is used for storage. The effective combination of the two not only solves the problem of how to carry out large-capacity storage processing in the related technology, but also consumes a large space and inconvenient query, thereby achieving the effects of improving storage efficiency, saving storage space, and improving query efficiency. The following describes the storage method based on the keyword-value Key-Value, and determines the location of the bucket in which the resource object is stored according to the dynamic hash algorithm. First, the hash algorithm corresponding to each bucket is determined, where the bucket uses the keyword Key. Positioning, each bucket is named by using a predetermined rule; then, the resource object is hashed according to the hash algorithm corresponding to the bucket to determine the bucket of the storage resource object, wherein the stored resource object corresponds to the value Value. Ρ, first, the dynamic hash table, including N hash buckets (N%2 = 0), each hash bucket is located with a key value, the bucket naming rules can be various, for example, the following can be named The rule "list name_0, list name_1, list name_2 ...", of course, other naming rules can also be used. Second, each element in the bucket has its own KEY before it is stored. By hashing this unique KEY value, it is placed in its own bucket. Preferably, determining the location of the bucket stored by the resource object according to the dynamic hash algorithm may be performed by: determining, according to the first hash algorithm, a location of the initial bucket stored by the resource object; determining whether the initial bucket is full; If yes, the initial bucket split is triggered; according to the second hash algorithm, the split bucket and the bucket before the split store the existing resource object, wherein the existing resource object includes the resource object and exists before the split The resource object stored in the bucket, that is, when a new resource object is stored, the corresponding calculation is performed according to the dynamic hash algorithm, and then stored in the bucket obtained after the calculation. It should be noted that, when the split bucket is correspondingly indicated by the split pointer, it is determined that the initial bucket is full, and the trigger bucket splitting may be solved in the following manner: First, it is determined whether there is a split pointer that points to the split bucket in advance; If the result of the determination is yes, the priority triggers the splitting of the bucket pointed to by the split pointer. After storing the resource object according to the determined location of the bucket, the method may further: receive the resource access request; upload the requested resource object according to the resource access request. In addition, the resource object that is requested to be uploaded according to the resource access request includes: determining whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request, that is, authenticating the resource access request, The security of the resource object is advantageously guaranteed. In this embodiment, a resource object storage processing device is also provided, which is used to implement the foregoing embodiments and preferred embodiments, and has not been described again. As used hereinafter, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and conceivable. 2 is a structural block diagram of a resource object storage processing apparatus according to an embodiment of the present invention. The apparatus is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket unit, as shown in FIG. 2, The apparatus includes a first receiving module 22, a determining module 24, and a storage module 26, which are described below. The first receiving module 22 is configured to receive the resource object to be stored; the determining module 24 is connected to the first receiving module 22, and is configured to determine the location of the bucket stored by the resource object according to the dynamic hash algorithm; the storage module 26, Connected to the above determining module 24, it is arranged to store the resource object according to the determined location of the bucket. 3 is a block diagram of a preferred structure of the determining module 24 in the resource object storage processing apparatus according to an embodiment of the present invention. The determining module 24 includes a first determining unit 32 and a second determining unit 34. The determining module 24 is described below. . The first determining unit 32 is configured to determine a hash algorithm corresponding to each bucket, wherein the bucket is located by using a keyword Key, and each bucket is named by using a predetermined rule; and the second determining unit 34 is connected to the first determining unit. 32. Set, according to the hash algorithm corresponding to the bucket, perform hash processing on the resource object to determine a bucket that stores the resource object, where the stored resource object corresponds to the value Value. 4 is a block diagram 2 of a preferred structure of the determining module 24 in the resource object storage processing apparatus according to an embodiment of the present invention. The determining module 24 includes a third determining unit 42, a first determining unit 44, a triggering unit 46, and an allocating unit 48. The determination module 24 will be described. The third determining unit 42 is configured to determine, according to the first hash algorithm, the location of the initial bucket stored by the resource object; the first determining unit 44 is connected to the determining unit 42 to determine whether the initial bucket is full; the triggering unit 46, Connected to the first determining unit 44, configured to trigger the initial bucket splitting if the determining result of the determining unit is yes; the assigning unit 48 is connected to the triggering unit 46, and configured to split according to the second hashing algorithm. The bucket and the bucket before the split store the existing resource object for reallocation, wherein the existing resource object includes the resource object and the resource object stored in the bucket existing before the split. FIG. 5 is a block diagram showing a preferred structure of the triggering unit 46 in the determining module 24 in the resource object storage processing apparatus according to the embodiment of the present invention. As shown in FIG. 5, the triggering unit 46 includes a determining subunit 52 and a triggering subunit 54, below. The trigger unit 46 will be described. The determining sub-unit 52 is configured to determine whether there is a split pointer that points to the split bucket in advance; the trigger sub-unit 54 is connected to the judging sub-unit 52, and is set to trigger the pair preferentially if the judgment result of the judging sub-unit 52 is YES. The bucket pointed to by the split pointer splits. FIG. 6 is a structural block diagram of a resource object storage processing apparatus according to an embodiment of the present invention. As shown in FIG. 6, the apparatus includes a second receiving module 62 and an uploading module 64, in addition to all the modules shown in FIG. The device will be described. The second receiving module 62 is connected to the storage module 26 and configured to receive a resource access request. The uploading module 64 is connected to the second receiving module 62, and is configured to upload the requested resource object according to the resource access request. FIG. 7 is a block diagram showing a preferred structure of the uploading module 64 in the resource object storage processing apparatus according to the embodiment of the present invention. As shown in FIG. 7, the uploading module 64 includes a second determining unit 72, and the second determining unit 72 is performed next. Description. The second determining unit 72 is configured to determine whether the resource access request is legal. If the determination result is yes, the requested resource object is uploaded according to the resource access request. Based on the above-mentioned problems in the related art, in order to better implement unified management of management resources such as sub-tenants and users in the cloud storage system and improve management efficiency, in this embodiment, a key-value based storage method is proposed. The key-value-based resource management method includes: storing management resources (tenants, sub-tenants, users, etc.) based on key-value key values; learning dynamic storage of management resources by using dynamic hash processing method, and cloud storage system Neutron tenant, user resource management (add, delete, insert, modify, etc.); where, based on key-value key-value pairs to store management resources (tenants, sub-tenants, users, etc.), including: First, key-value Storage can support the storage of massive data. Its distributed architecture determines that as long as there are enough machines, it can guarantee to store more data. Secondly, the key-value architecture can easily support tens of thousands of concurrent requests, making the system highly concurrent. Sex. The dynamic storage of management resources is based on the dynamic hash processing method. First, the key-value based storage architecture, when the value is large (for example, a list), in order to save space and improve search performance, it is necessary to learn from dynamics. The algorithm idea is used to solve this problem. Secondly, the dynamic hash algorithm can avoid the conflict problem generated by the static hash algorithm to a certain extent and solve the problem better. The methods for implementing list management of sub-tenants and users in the cloud storage system include: First, sub-tenants and users have their own KEY values when they are created, and then sub-tenants and users are respectively stored in their respective sub-tenant lists through hash function calculation. Second, the management resource storage interface API, management resource list creation, management resource list deletion, management resource list modification, management resource search, etc., to facilitate upper-layer applications. Based on the storage mode of KEY-VALUE of China Telecom cloud storage system, the traditional list storage method is abandoned, and the idea of dynamic hash algorithm is used to realize the allocation of space on demand, saving storage space, directly through HASH function. Positioning, shortening the average lookup length and improving performance. The dynamic hash algorithm can solve the problem of space expansion and contraction well. Its basic idea is the same as the traditional hash. First, design the mapping function from the key field U to the storage space (hash file), through two HASH mappings. Function, used to recalculate the HASH value when the bucket is split. A hash file is a set of records determined by a subfield of the key field U. A record in the hash file is determined by a keyword in the key field U, and the file contains a plurality of buckets for storing records. The basic unit for file operations is buckets. Each bucket can store b records. When more than b records are mapped to the same bucket through a hash function, the bucket overflows. At this point, the bucket is split (new bucket is added) Space), using the new HASH algorithm, recalculate the values in the current bucket and rearrange the data in the bucket. Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Based on the key-value storage mode, the dynamic hashing idea is adopted to manage the management resources of sub-tenants and users, and to quickly query and locate sub-tenants and users. 8 is a schematic diagram of a China Telecom cloud storage system architecture according to a preferred embodiment of the present invention. As shown in FIG. 8, the system mainly includes four parts, a physical storage layer, a storage interface layer, a service encapsulation layer, and a system management layer. This structure illustrates a preferred embodiment of the present invention. The physical storage layer provides physical storage space to the cloud storage system. The management resources and objects in the system are stored therein. The storage interface layer provides addition, deletion, and randomization, storage policy settings, and objects in the system according to the requirements of the upper-layer service system. An interface for managing resource storage, the service encapsulation layer, is an adaptation of related service interfaces, and provides a service interface that meets customer requirements. The system management layer maintains, manages, and configures the system to implement system configuration and performance in the system. Statistics, operation and maintenance data collection and network management related functions. In this embodiment, the content involved is at the storage interface layer, which is an improvement and re-encapsulation of the management resource storage interface API in the original storage interface. The encapsulated API includes the creation of a management resource list, the query, insertion, and deletion of management resources, and the unified application of the convenient service, and convenient storage for the next, saving physical space. 9 is a schematic flowchart of a storage process according to a preferred embodiment of the present invention. As shown in FIG. 9, the storage of a sub-tenant is taken as an example. The key corresponding to the sub-tenant resource list is the STL_tenant name, and the initialization list is 2 X 4, the process includes the following steps: Step S902, seven sub-tenants subtl~subt7 are inserted in the resource list, the split pointer splitptr points to the bucket STL_tenant_0, the bucket STL_tenant_l is full, and the two HASH algorithms are H0=Hash(key) %2, Hl=Hash(key)%4, at this time, insert the sub-tenant subt8, use HO to calculate the position of the bucket that subt8 should be inserted, and the resulting postion is STL_tenant_l. Since STL_tenant_l is full, the bucket split is triggered at this time, but because At this point, the split pointer splitptr points to STL_tenant_0, so first put subt8 into the overflow page, then split the bucket STL_tenant_0, and recalculate the data in STL_tenant_0 with the new algorithm HI to determine the data subl, sub4, sub6 in the original STL_tenant_0 Put it in STL_tenant_0 or STL_tenant_2. Step S904, after the bucket STL_tenant_0 is split according to the new HASH algorithm, subt4 is stored in STL_tenant_2, subtl and subt6 remain in the bucket STL_tenant_0, and the subt6 storage location is moved forward, and the split pointer points to the bucket STL_tenant_l, also by the HI algorithm, STL_tenant_l splits, the newly generated buckets STL_tenant_3, subt5, sub7 are stored in STL_tenant_3 according to the new algorithm, sub2, sub8 are left in the original STL tenant l, and are discharged in the new order. Step S906, after STL_tenant_l is split, the number of buckets is 4, the split pointer points back to the STL_tenant_0 bucket, and HO = Hash(key)%4, Hl=Hash(key)%8, starting a new round of insertion. And splitting, when the number of buckets is 8, the split pointer points back to STL_tenant_0, HO = Hash(key)%8, HI = Hash(key)%16, and so on. FIG. 10 is a flowchart of uploading an object of a cloud storage system according to a preferred embodiment of the present invention. As shown in FIG. 10, the process includes the following steps: an authentication step, a resource access processing, and an object upload processing. The following process is performed based on FIG. 10 Be explained. Step S1002: Authentication processing: mainly implementing authentication between the application and the system; Step S1004: The resource access processing is mainly used for accessing the management resource, and determining the legality of the request according to the attribute, and the following processing manner may be used during the operation: User status, whether the user status is available and whether the user has access, read the sub-tenant attribute of the user, determine whether the sub-tenant status is available, etc., and implement management by calling the encapsulated API (search resource search, etc.) The access of the resource, the attribute of the required resource (for example, the basic attribute of the state of the management resource); Step S1006, which is mainly used to implement uploading of the object according to the request. By adopting the storage method of the above embodiment and the preferred embodiment, the search efficiency is greatly improved, thereby improving the rate of object uploading. Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention. Industrial Applicability As described above, a resource object storage processing method and apparatus provided by an embodiment of the present invention have the following beneficial effects: Solving the problem of how to perform large-capacity storage processing in the related art, and requiring a large space and inconvenient query. In turn, it improves storage efficiency, saves storage space, and improves query efficiency.
Figure imgf000011_0001

Claims

权 利 要 求 书 Claims
1. 一种资源对象存储处理方法, 基于关键字-值 Key-Value的存储方式, 其中, 所 述资源对象以桶为单位进行存储, 包括: A resource object storage processing method, which is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket, including:
接收到要进行存储的资源对象;  Received a resource object to be stored;
依据动态哈希算法确定所述资源对象存储的所述桶的位置; 依据确定的所述桶的位置存储所述资源对象。  Determining, according to a dynamic hash algorithm, a location of the bucket stored by the resource object; storing the resource object according to the determined location of the bucket.
2. 根据权利要求 1所述的方法, 其中, 依据所述动态哈希算法确定所述资源对象 存储的所述桶的位置包括: The method according to claim 1, wherein determining the location of the bucket stored by the resource object according to the dynamic hash algorithm comprises:
确定各个桶对应的哈希算法, 其中, 所述桶通过关键字 Key来定位, 对每 个桶采用预定规则进行命名;  Determining a hash algorithm corresponding to each bucket, wherein the bucket is located by using a keyword Key, and each bucket is named by using a predetermined rule;
依据桶对应的所述哈希算法对所述资源对象进行哈希处理确定存储所述资 源对象的桶, 其中, 存储的所述资源对象对应于值 Value。  The resource object is hashed according to the hash algorithm corresponding to the bucket to determine a bucket storing the resource object, wherein the stored resource object corresponds to a value Value.
3. 根据权利要求 1所述的方法, 其中, 依据所述动态哈希算法确定所述资源对象 存储的所述桶的位置包括: The method according to claim 1, wherein determining the location of the bucket stored by the resource object according to the dynamic hash algorithm comprises:
依据第一哈希算法确定所述资源对象存储的初始桶的位置; 判断所述初始桶是否已满;  Determining, according to a first hash algorithm, a location of an initial bucket stored by the resource object; determining whether the initial bucket is full;
在判断结果为是的情况下, 触发所述初始桶分裂;  In the case that the determination result is yes, the initial bucket splitting is triggered;
根据第二哈希算法对分裂的桶和分裂之前的桶存放现有资源对象进行重新 分配, 其中, 所述现有资源对象包括所述资源对象和分裂之前所存在的桶中所 存储的资源对象。  And reallocating the existing resource object by the split bucket and the bucket before the split according to the second hash algorithm, where the existing resource object includes the resource object and the resource object stored in the bucket existing before the splitting .
4. 根据权利要求 3所述的方法, 其中, 在判断初始桶已满, 触发桶分裂包括: 判断是否存在预先指向分裂桶的分裂指针; The method according to claim 3, wherein, when it is determined that the initial bucket is full, triggering the bucket splitting comprises: determining whether there is a split pointer that points to the split bucket in advance;
在判断结果为是的情况下, 优先触发对所述分裂指针指向的桶进行分裂。  If the result of the determination is yes, the bucket that is pointed to by the split pointer is preferentially triggered to split.
5. 根据权利要求 1至 4中任一项所述的方法, 其中, 在依据确定的所述桶的位置 存储所述资源对象之后, 还包括: The method according to any one of claims 1 to 4, further comprising: after storing the resource object according to the determined location of the bucket, further comprising:
接收资源访问请求; 依据所述资源访问请求上传请求的所述资源对象。 根据权利要求 5所述的方法, 其中, 依据所述资源访问请求上传请求的所述资 源对象包括: Receiving a resource access request; The resource object of the request is uploaded according to the resource access request. The method according to claim 5, wherein the resource object according to the resource access request upload request comprises:
判断所述资源访问请求是否合法, 在判断结果为是的情况下, 依据所述资 源访问请求上传请求的所述资源对象。 一种资源对象存储处理装置, 基于关键字-值 Key-Value的存储方式, 其中, 所 述资源对象以桶为单位进行存储, 包括:  Determining whether the resource access request is legal, and if the determination result is yes, uploading the requested resource object according to the resource access request. A resource object storage processing device, which is based on a key-value Key-Value storage method, wherein the resource object is stored in a bucket unit, and includes:
第一接收模块, 设置为接收到要进行存储的资源对象;  a first receiving module, configured to receive a resource object to be stored;
确定模块, 设置为依据动态哈希算法确定所述资源对象存储的所述桶的位 置;  Determining a module, configured to determine a location of the bucket stored by the resource object according to a dynamic hash algorithm;
存储模块, 设置为依据确定的所述桶的位置存储所述资源对象。 根据权利要求 7所述的装置, 其中, 所述确定模块包括: 第一确定单元, 设置为确定各个桶对应的哈希算法, 其中, 所述桶通过关 键字 Key来定位, 对每个桶采用预定规则进行命名;  And a storage module, configured to store the resource object according to the determined location of the bucket. The device according to claim 7, wherein the determining module comprises: a first determining unit, configured to determine a hash algorithm corresponding to each bucket, wherein the bucket is located by using a keyword Key, and is used for each bucket The predetermined rules are named;
第二确定单元, 设置为依据桶对应的所述哈希算法对所述资源对象进行哈 希处理确定存储所述资源对象的桶,其中,存储的所述资源对象对应于值 Value。 根据权利要求 7所述的装置, 其中, 所述确定模块包括: 第三确定单元, 设置为依据第一哈希算法确定所述资源对象存储的初始桶 的位置;  And a second determining unit, configured to perform hash processing on the resource object according to the hash algorithm corresponding to the bucket to determine a bucket that stores the resource object, where the stored resource object corresponds to a value Value. The apparatus according to claim 7, wherein the determining module comprises: a third determining unit, configured to determine a location of an initial bucket stored by the resource object according to a first hash algorithm;
第一判断单元, 设置为判断所述初始桶是否已满;  a first determining unit, configured to determine whether the initial bucket is full;
触发单元, 设置为在所述判断单元的判断结果为是的情况下, 触发所述初 始桶分裂;  a trigger unit, configured to trigger the initial bucket splitting if the determination result of the determining unit is yes;
分配单元, 设置为根据第二哈希算法对分裂的桶和分裂之前的桶存放现有 资源对象进行重新分配, 其中, 所述现有资源对象包括所述资源对象和分裂之 前所存在的桶中所存储的资源对象。 根据权利要求 9所述的装置, 其中, 所述触发单元包括: 判断子单元, 设置为判断是否存在预先指向分裂桶的分裂指针; 触发子单元, 设置为于在所述判断子单元的判断结果为是的情况下, 优先 触发对所述分裂指针指向的桶进行分裂。 And an allocation unit, configured to perform reallocation on the split bucket and the bucket before the split to store the existing resource object according to the second hash algorithm, where the existing resource object includes the resource object and the bucket existing before splitting The stored resource object. The device according to claim 9, wherein the triggering unit comprises: a determining subunit, configured to determine whether there is a split pointer that points to a split bucket in advance; The triggering subunit is configured to preferentially trigger splitting of the bucket pointed by the split pointer when the judgment result of the determining subunit is YES.
11. 根据权利要求 7至 10中任一项所述的装置, 其中, 还包括: 第二接收模块, 设置为接收资源访问请求; The device according to any one of claims 7 to 10, further comprising: a second receiving module, configured to receive a resource access request;
上传模块, 设置为依据所述资源访问请求上传请求的所述资源对象。  The uploading module is configured to upload the resource object according to the resource access request.
12. 根据权利要求 11所述的装置, 其中, 所述上传模块包括: The device of claim 11, wherein the uploading module comprises:
第二判断单元, 设置为判断所述资源访问请求是否合法, 在判断结果为是 的情况下, 依据所述资源访问请求上传请求的所述资源对象。  The second determining unit is configured to determine whether the resource access request is legal, and if the determination result is yes, upload the requested resource object according to the resource access request.
PCT/CN2014/077603 2013-11-06 2014-05-15 Method and device for processing resource object storage WO2014177080A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310545390.6 2013-11-06
CN201310545390.6A CN104639570A (en) 2013-11-06 2013-11-06 Resource object storage processing method and device

Publications (1)

Publication Number Publication Date
WO2014177080A1 true WO2014177080A1 (en) 2014-11-06

Family

ID=51843158

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/077603 WO2014177080A1 (en) 2013-11-06 2014-05-15 Method and device for processing resource object storage

Country Status (2)

Country Link
CN (1) CN104639570A (en)
WO (1) WO2014177080A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107766258B (en) * 2017-09-27 2021-11-16 恩亿科(北京)数据科技有限公司 Memory storage method and device and memory query method and device
CN111338571B (en) * 2020-02-18 2021-08-17 腾讯科技(深圳)有限公司 Task processing method, device, equipment and storage medium
CN114489483A (en) * 2021-12-24 2022-05-13 深圳市捷顺科技实业股份有限公司 Disk management method based on object storage and object storage module

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1858734A (en) * 2005-12-28 2006-11-08 华为技术有限公司 Data storaging and searching method
EP1970821A1 (en) * 2007-03-12 2008-09-17 Broadcom Corporation Method and apparatus for dual-hashing tables
CN101782922A (en) * 2009-12-29 2010-07-21 山东山大鸥玛软件有限公司 Multi-level bucket hashing index method for searching mass data
CN102232219A (en) * 2010-01-26 2011-11-02 华为技术有限公司 Method and device for storing and searching keyword

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101464901B (en) * 2009-01-16 2012-03-21 华中科技大学 Object search method in object storage device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1858734A (en) * 2005-12-28 2006-11-08 华为技术有限公司 Data storaging and searching method
EP1970821A1 (en) * 2007-03-12 2008-09-17 Broadcom Corporation Method and apparatus for dual-hashing tables
CN101782922A (en) * 2009-12-29 2010-07-21 山东山大鸥玛软件有限公司 Multi-level bucket hashing index method for searching mass data
CN102232219A (en) * 2010-01-26 2011-11-02 华为技术有限公司 Method and device for storing and searching keyword

Also Published As

Publication number Publication date
CN104639570A (en) 2015-05-20

Similar Documents

Publication Publication Date Title
US10901796B2 (en) Hash-based partitioning system
US10394847B2 (en) Processing data in a distributed database across a plurality of clusters
US11392497B1 (en) Low latency access to data sets using shared data set portions
US10275489B1 (en) Binary encoding-based optimizations at datastore accelerators
US20160378846A1 (en) Object based storage cluster with multiple selectable data handling policies
US11550713B1 (en) Garbage collection in distributed systems using life cycled storage roots
WO2015143983A1 (en) Method and apparatus for implementing acceleration processing on vnf
US20130218934A1 (en) Method for directory entries split and merge in distributed file system
US11245774B2 (en) Cache storage for streaming data
WO2015078194A1 (en) Configuration method and device for hash database
CN103338242B (en) A kind of mixed cloud storage system based on multi-level buffer and method
US9898477B1 (en) Writing to a site cache in a distributed file system
WO2008119286A1 (en) Method and system of data management
US11593270B1 (en) Fast distributed caching using erasure coded object parts
US10579597B1 (en) Data-tiering service with multiple cold tier quality of service levels
WO2014089828A1 (en) Method for accessing storage device and storage device
JP2009295127A (en) Access method, access device and distributed data management system
CN106570113B (en) Mass vector slice data cloud storage method and system
CN111723073B (en) Data storage processing method, device, processing system and storage medium
CN108540510B (en) Cloud host creation method and device and cloud service system
CN113806300A (en) Data storage method, system, device, equipment and storage medium
WO2014177080A1 (en) Method and device for processing resource object storage
CN115438016A (en) Dynamic fragmentation method, system, medium and device in distributed object storage
US10146833B1 (en) Write-back techniques at datastore accelerators
CN102970349B (en) A kind of memory load equalization methods of DHT network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14791328

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14791328

Country of ref document: EP

Kind code of ref document: A1