CN104270458A - Method and device for achieving data distribution management of cloud storage system - Google Patents
Method and device for achieving data distribution management of cloud storage system Download PDFInfo
- Publication number
- CN104270458A CN104270458A CN201410554245.9A CN201410554245A CN104270458A CN 104270458 A CN104270458 A CN 104270458A CN 201410554245 A CN201410554245 A CN 201410554245A CN 104270458 A CN104270458 A CN 104270458A
- Authority
- CN
- China
- Prior art keywords
- memory node
- key assignments
- data object
- mapping
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a method and device for achieving data distribution management of a cloud storage system. The method comprises the steps of connecting the starting address and the ending address of a storage space to form a closed chain-shaped space; calculating the key values of hashing all data objects into the chain-shaped space according to the set characteristics of the data objects, and mapping the data objects into the chain-shaped space according to the key values; calculating the key values of hashing all storage nodes into the chain-shaped space according to the position characteristics of the storage nodes, and mapping the corresponding storage nodes into the chain-shaped space according to the key values; traversing the chain-shaped space in one fixed direction according to the key value of one data object, and if the data object comes across the key value of one storage node, mapping the data object and the storage node to each other; conducting the process in this way till mutual mapping between all the data objects and the storage nodes is completed. The method and device can guarantee that uniform distribution of the data objects and the storage nodes in the system and shorten the average response time of the system to the maximum degree.
Description
Technical field
The present invention relates to the technology of SRM in cloud storage system, particularly relate to the method and device that realize the management of cloud storage system Data distribution8.
Background technology
Current, cloud computing is approved by industry gradually, and various cloud storage system realizes gradually and is committed to practice.In cloud storage system, rationally, efficiently use memory device resource, play a part very crucial for the overall high availability of cloud storage system and the validity of management.
The strategy of most of existing its Data distribution8 of cloud storage system is dull, poor efficiency, there is the problem that physical memory resources scheduling is unreasonable and resource utilization is low.Such as, the data distribution strategy of the cloud storage system had on the one hand does not consider the distinctive technical characterstic of system and application demand, along with system node constantly increase, scale constantly expand when, data object can not be made to distribute equably between each node in system, thus meet the requirement of memory load equilibrium; On the other hand, some clouds store data distribution strategy and fail to solve the unreasonable problem causing resource utilization low of physical memory resources scheduling, namely system can not complete addressing data efficiently in the information processing of magnanimity, reduce average response time to greatest extent, thus improve throughput and the entire system performance of data, services.These are the problems in puzzlement cloud Development of storage technology always above.
In order to ensure that cloud memory technology robustly develops, the storage resources in cloud storage system efficiently, reasonably can be used, needing to provide the system and method thereof that realize the management of cloud storage Data distribution8.
Summary of the invention
Technical problem to be solved by this invention is to provide a kind of method and the device that realize the management of cloud storage system Data distribution8, and the storage resources in cloud storage system can be made efficiently, reasonably to be used.
In order to solve the problems of the technologies described above, the invention provides a kind of method realizing the management of cloud storage system Data distribution8, the flow process containing data object and memory node initial mapping, comprises the steps:
The beginning and end address of a memory space is connected, forms a closed chain space;
Calculate the key assignments hashed to by all data objects of system in this chain space respectively according to the set feature of data object, and map corresponding data object to this chain space according to each key assignments; Calculate the key assignments hashed to by memory nodes all in system in this chain space respectively according to the position feature of memory node, and map corresponding memory node to this chain space according to each key assignments;
Travel through this chain space from the key assignments of a data object towards a fixed-direction, until run into the key assignments of a memory node, then this data object and this memory node are mapped mutually by respective key assignments; Then the mutual mapping of next data object and memory node is carried out according to identical mode, until complete the mutual mapping of all data objects and memory node.
Further, the method also containing the flow process that data object and memory node remap, comprises the steps:
When having monitored new memory node and will being increased to cloud storage system, calculate the key assignments hashed to by this newly-increased memory node in this chain space according to the position feature of this memory node, and mapped this newly-increased memory node to this chain space according to this key assignments;
And/or when having monitored memory node and will having removed from cloud storage system, from this chain space, remove the mapping of this memory node;
Traverse the key assignments place of first former memory node from the key assignments edge of memory node that is newly-increased or that remove and the opposite direction in the direction of mapping (enum) data object in initial mapping flow process in this chain space, determine the key assignments of the data object between two key assignments places, and again the part or all of data object between these two key assignments places and newly-increased memory node or the former memory node that traverses are mapped mutually by respective key assignments.
Further, adopt mapping function F to calculate the key assignments hashed to by all data objects of system in this chain space according to the set feature of data object, and adopt this mapping function F to calculate the key assignments maybe hashed to by newly-increased memory node by memory nodes all in system in this chain space according to the position feature of memory node; The set feature of data object comprise in the mark of data block, data block size and data type one or more, the position feature of memory node comprise in the IP address of memory node, Media Access Control address and machine name one or more; The content of mapping function F is as follows:
In mapping function, * str is the pointer of the feature string pointing to input, and BestDivert is best hashed value, and m is the address size of memory space.
In order to solve the problems of the technologies described above, the invention provides a kind of device realizing the management of cloud storage system Data distribution8, comprise storage space management module, data object mapping block, data and the mutual mapping block of node and memory node mapping block that connect successively, wherein:
Storage space management module, for being connected by the beginning and end address of a memory space, forms a closed chain space;
Data object mapping block, for calculating the key assignments hashed to by all data objects of system in described chain space respectively according to the set feature of data object, and maps corresponding data object to this chain space according to each key assignments;
Memory node mapping block also with storage space management model calling, for calculating the key assignments hashed to by memory nodes all in system in described chain space respectively according to the position feature of memory node, and map corresponding memory node to this chain space according to each key assignments;
Data and the mutual mapping block of node, for traveling through this chain space from the key assignments of a data object towards a fixed-direction, until run into the key assignments of a memory node, then mapped this data object and this memory node by respective key assignments mutually; Then the mutual mapping of next data object and memory node is carried out according to identical mode, until complete the mutual mapping of all data objects and memory node.
Further,
Storage space management module is when having monitored new memory node and will being increased to cloud storage system, newly-increased node notice is sent to memory node mapping block, and/or when having monitored memory node and will having removed from cloud storage system, send to memory node mapping block and remove node notice;
After memory node mapping block receives newly-increased node notice, calculate the key assignments hashed to by this newly-increased memory node in chain space according to the position feature of newly-increased memory node, and map this newly-increased memory node to chain space according to this key assignments; And/or storage space management module receive remove node notice after, from this chain space, remove the mapping of this memory node; Then notification data and the mutual mapping block of node carry out remapping of data and node;
Data and the mutual mapping block of node are according to the notice of memory node mapping block, in chain space from the key assignments of memory node that is newly-increased or that remove along traversing the key assignments place of first former memory node with the opposite direction in the direction of mapping (enum) data object in initial mapping, determine the key assignments of the data object between two key assignments places, and again by the part or all of data object between these two key assignments places and newly-increased memory node or mutually mapped by respective key assignments with the former memory node traversed.
Further,
Data object mapping block adopts mapping function F to calculate the key assignments hashed to by all data objects of system in this chain space according to the set feature of data object, and wherein the set feature of data object comprises one or more in the mark of data block, data block size and data type;
Memory node mapping block adopts this mapping function F to calculate the key assignments maybe hashed to by newly-increased memory node by memory nodes all in system in this chain space according to the position feature of memory node; Wherein the position feature of memory node comprises one or more in the IP address of memory node, Media Access Control address and machine name;
The content of mapping function F is as follows:
In mapping function, * str is the pointer of the feature string pointing to input, and BestDivert is best hashed value, and m is the address size of memory space.
The present invention can ensure data object being uniformly distributed in cloud storage system and between each physical store node, also greatly reduces the impact that the Data Migration that causes when new storage devices node adds or existing storage devices node exits brings entire system performance; Addressing data can be completed efficiently simultaneously, reduce its average response time to greatest extent, thus improve throughput and the entire system performance of data, services.
Accompanying drawing explanation
Fig. 1 is the flow chart realizing data object and memory node being carried out in the method for cloud storage system Data distribution8 management the embodiment tentatively mapped of the present invention;
Fig. 2 is the schematic diagram forming closed chain space in the embodiment of the method for Fig. 1;
Fig. 3,4 is the schematic diagram respectively data object, memory node being mapped to same closed chain space in the embodiment of the method for Fig. 1;
Fig. 5 is the schematic diagram data object and memory node that are mapped to same closed chain space in Fig. 3,4 being carried out mutually mapping;
Fig. 6 is that the cloud that realizes of the present invention stores the flow chart carrying out the embodiment that data object and memory node remap in the method for Data distribution8 management when memory node increases and decreases;
Data object is again mapped to the schematic diagram newly adding memory node in the embodiment of the method for Fig. 6 by Fig. 7 after interpolation memory node;
Fig. 8 is the schematic diagram again data object being mapped to a former memory node in the embodiment of the method for Fig. 6 after removing memory node;
Fig. 9 is the structural representation realizing the device embodiment of cloud storage system Data distribution8 management of the present invention.
Embodiment
Below in conjunction with accompanying drawing and preferred embodiment, technical scheme of the present invention is set forth in detail.Should be appreciated that, the embodiment below enumerated only for instruction and explanation of the present invention, and does not form the restriction to technical solution of the present invention.
As shown in Figure 1, illustrate the flow process realizing data object and memory node being carried out in the method for cloud storage system Data distribution8 management the embodiment tentatively mapped provided by the invention, comprise the steps:
110: the beginning and end address of a memory space is connected, form a closed chain space;
As shown in Figure 2, such as m=32, namely memory space address is 32 in the signal in above-mentioned closed chain space.Multiple memory space with identical address position or different address bit may be there is in cloud storage system, all can form a closed chain space in the manner described above.
120: calculate the key assignments hashed to by all data objects of system in chain space respectively according to the set feature of data object, and map corresponding data object to chain space according to each key assignments;
At this, adopt mapping function according to the key assignments Kdi of all data objects in the set feature computing system of data object, and corresponding data object is mapped in closed chain space according to its key assignments.The set feature of data object comprise in the mark of data block, block size and data type one or more.
The content of mapping function F is as follows:
The effect of this mapping function F be the key assignments of data object or the key assignments energy uniform broadcasting of memory node that ensure to calculate in above-mentioned closed chain space, thus ensure that data object and poke node all can be mapped in closed chain space equably.The core of this mapping function is the execution of hash function hash.
The corresponding data object of key assignments Kdi of all data objects calculated is mapped to result in closed chain space as shown in Figure 3, wherein i is for example 1 ~ 4.
Certainly, except adopting above-mentioned hash algorithm to form except mapping function, other hashing algorithm also can be adopted to form mapping function, such as, except remaining method, digital analysis method and character string hashing algorithm etc.
130: calculate the key assignments hashed to by memory nodes all in system in chain space respectively according to the position feature of memory node, and map corresponding memory node to chain space according to each key assignments;
At this, still adopt above-mentioned mapping function according to the key assignments Knj of all memory nodes in the position feature of memory node respectively computing system, and corresponding memory node is mapped in same chain space according to its key assignments.The position feature of memory node comprise in the IP address of memory node, MAC (medium access control, Media Access Control) address and machine name one or more.
The corresponding memory node of key assignments Knj of all memory nodes calculated is mapped to result in closed chain space as shown in Figure 4, wherein j is for example 1 ~ 3.
140: the key assignments choosing a data object from chain space;
150: from the key assignments of the data object chosen, travel through this chain space towards a fixed-direction, until run into the key assignments of first memory node, then corresponding data object and corresponding memory node are mapped mutually by respective key assignments;
As shown in Figure 5, such as, from the key assignments Kd1 of the data object chosen, travel through chain space toward the clockwise direction, until run into the key assignments Kn1 of first memory node, then corresponding data object and corresponding memory node are mapped mutually by key assignments Kd1, key assignments Kn1; Then, from the data object Kd2 that the next one is chosen, travel through chain space toward the clockwise direction, until run into the key assignments Kn2 of first memory node, then corresponding data object and corresponding memory node are mapped mutually by key assignments Kd2, key assignments Kn2; Finally, from the data object Kd4 chosen, travel through chain space toward the clockwise direction, until run into the key assignments Kn3 of first memory node, then corresponding data object and corresponding memory node are mapped mutually by key assignments Kd4, key assignments Kn3.
160: the mapping having judged whether all data objects and memory node in chain space, be then process ends, otherwise perform step 170;
170: the key assignments choosing Next data object, then return step 150 and perform.
As shown in Figure 6, illustrate the cloud that realizes provided by the invention and store the flow process of carrying out the embodiment that data object and memory node remap in the method for Data distribution8 management when memory node increases and decreases, it starts when monitoring in cloud storage system the increase and decrease having memory node to perform, and comprises the steps:
210: having judged whether that new memory node is increased to system, is perform step 220, otherwise perform step 230;
220: calculate the key assignments hashed to by this memory node in chain space according to the position feature of the memory node that will increase, and map this memory node to chain space according to key assignments; Then jump to step 250 to perform;
Be still the key assignments adopting above-mentioned mapping function to calculate this memory node according to the position feature of memory node, and this memory node is mapped in above-mentioned chain space according to its key assignments.The position feature of memory node comprise in the IP address of memory node, mac address and machine name one or more.
Result in same chain space is mapped to as shown in Figure 7 according to the key assignments Kn4 calculated by calculating the memory node that newly increases.
230: judging whether will remove memory node in system, is perform step 240, otherwise process ends;
240: the mapping removing corresponding memory node from chain space;
The mapping of the memory node removed from chain space as shown in Figure 8, such as removes the mapping of memory node Kn3.
250: the opposite direction mapping direction in chain space from the memory node key assignments edge increased newly or remove and initial distribution traverses the key assignments place of a former memory node, determine the key assignments of the data object between two places, and again by wherein partly or entirely data object mutually mapped by respective key assignments with former memory node that is newly-increased or that traverse, then process ends.
As shown in Figure 7, by result that affected data object maps mutually with newly-increased memory node after the system that illustrates increases node newly.Traverse the key assignments Kn3 place of a former memory node in the counterclockwise direction from newly-increased memory node key assignments Kn4 in chain space, determine that the key assignments of the data object between two places is for example Kd3, and again the data object of its correspondence and newly-increased memory node are mapped mutually by respective key assignments Kd3, Kn4.
As shown in Figure 8, by result that affected data object and former memory node map mutually after the system that illustrates removes node.Traverse the key assignments Kn1 place of a former memory node in the counterclockwise direction from the memory node key assignments Kn3 removed in chain space, determine that the key assignments of the data object between two places is for example Kd4, again the former memory node that the data object of correspondence and key assignments Kn2 represent is mapped mutually by respective key assignments Kd4, Kn2 under ensureing to map the consistent condition in direction.
Why the present invention adopts when carrying out data object and memory node and remapping and being reversed of direction during initial mapping, and is to ensure the consistency mapped, namely guaranteeing data object and memory node mapping relations one to one.So the reliability of data access is guaranteed by the monotonicity of mapping relations, the each memory node of management can be made only to need to safeguard the information of a small amount of adjacent node simultaneously, and when node adds/logs off, participate in by means of only relevant a small amount of node in the maintenance of topology.
The present invention is directed to said method embodiment and correspondingly additionally provide a kind of device embodiment realizing the management of cloud storage system Data distribution8, its structure as shown in Figure 9, comprise storage space management module, data object mapping block, data and the mutual mapping block of node and memory node mapping block that connect successively, wherein:
Storage space management module, for being connected by the beginning and end address of a memory space, forms a closed chain space;
Data object mapping block, for calculating the key assignments hashed to by all data objects of system in this chain space respectively according to the set feature of data object, and maps corresponding data object to this chain space according to each key assignments;
Memory node mapping block also with storage space management model calling, for calculating the key assignments hashed to by memory nodes all in system in this chain space respectively according to the position feature of memory node, and map corresponding memory node to this chain space according to each key assignments;
Data and the mutual mapping block of node, for traveling through this chain space from the key assignments of a data object towards a fixed-direction, until run into the key assignments of a memory node, then mapped this data object and this memory node by respective key assignments mutually; Then the mutual mapping of next data object and memory node is carried out according to identical mode, until complete the mutual mapping of all data objects and memory node.
In said apparatus embodiment,
Storage space management module is when having monitored new memory node and will being increased to cloud storage system, newly-increased node notice is sent to memory node mapping block, and/or when having monitored memory node and will having removed from cloud storage system, send to memory node mapping block and remove node notice;
This newly-increased memory node is hashed to the key assignments in chain space by the position feature calculating of the newly-increased memory node that memory node mapping block notifies according to storage space management module, and maps this newly-increased memory node to chain space according to this key assignments; And/or according to storage space management module notice remove the mapping that memory node removes this memory node from chain space; Then notification data and the mutual mapping block of node carry out remapping of data and node;
Data and the mutual mapping block of node are according to the notice of memory node mapping block, in chain space from the key assignments of memory node that is newly-increased or that remove along traversing the key assignments place of first former memory node with the opposite direction in the direction of mapping (enum) data object in initial mapping, determine the key assignments of the data object between two key assignments places, and again by the part or all of data object between these two key assignments places and newly-increased memory node or mutually mapped by respective key assignments with the former memory node traversed.
In said apparatus embodiment,
Data object mapping block adopts mapping function F to calculate the key assignments hashed to by all data objects of system in described chain space according to the set feature of data object, and wherein the set feature of data object comprises one or more in the mark of data block, data block size and data type;
Memory node mapping block adopts same mapping function F to calculate according to the position feature of memory node and newly-increased memory node is maybe hashed to the key assignments in chain space by memory nodes all in system; The position feature of memory node comprise in the IP address of memory node, Media Access Control address and machine name one or more.
Claims (6)
1. realize a method for cloud storage system Data distribution8 management, the flow process containing data object and memory node initial mapping, comprises the steps:
The beginning and end address of a memory space is connected, forms a closed chain space;
Calculate the key assignments hashed to by all data objects of system in described chain space respectively according to the set feature of data object, and map corresponding data object to described chain space according to each key assignments; Calculate the key assignments hashed to by memory nodes all in system in described chain space respectively according to the position feature of memory node, and map corresponding memory node to described chain space according to each key assignments;
Travel through described chain space from the key assignments of a data object towards a fixed-direction, until run into the key assignments of a memory node, then this data object and this memory node are mapped mutually by respective key assignments; Then the mutual mapping of next data object and memory node is carried out according to identical mode, until complete the mutual mapping of all data objects and memory node.
2. in accordance with the method for claim 1, it is characterized in that, also containing the flow process that data object and memory node remap, comprise the steps:
When having monitored new memory node and will being increased to cloud storage system, calculate the key assignments hashed to by this newly-increased memory node in described chain space according to the position feature of this memory node, and mapped this newly-increased memory node to described chain space according to this key assignments;
And/or when having monitored memory node and will having removed from cloud storage system, from described chain space, remove the mapping of this memory node;
Traverse the key assignments place of first former memory node from the key assignments edge of memory node that is newly-increased or that remove and the opposite direction in the direction of mapping (enum) data object in described initial mapping flow process in described chain space, determine the key assignments of the data object between two key assignments places, and again the part or all of data object between described two key assignments places and newly-increased memory node or the former memory node that traverses are mapped mutually by respective key assignments.
3. in accordance with the method for claim 2, it is characterized in that,
Adopt mapping function F to calculate the key assignments hashed to by all data objects of system in described chain space according to the set feature of data object, and adopt described mapping function F to calculate the key assignments maybe hashed to by newly-increased memory node by memory nodes all in system in described chain space according to the position feature of memory node; The set feature of described data object comprise in the mark of data block, data block size and data type one or more, the position feature of described memory node comprise in the IP address of memory node, Media Access Control address and machine name one or more; The content of described mapping function F is as follows:
In described mapping function, described * str is the pointer of the feature string pointing to input, and described BestDivert is best hashed value, and described m is the address size of described memory space.
4. realize a device for cloud storage system Data distribution8 management, it is characterized in that, comprise storage space management module, data object mapping block, data and the mutual mapping block of node and memory node mapping block that connect successively, wherein:
Storage space management module, for being connected by the beginning and end address of a memory space, forms a closed chain space;
Data object mapping block, for calculating the key assignments hashed to by all data objects of system in described chain space respectively according to the set feature of data object, and maps corresponding data object to this chain space according to each key assignments;
Memory node mapping block also with storage space management model calling, for calculating the key assignments hashed to by memory nodes all in system in described chain space respectively according to the position feature of memory node, and map corresponding memory node to this chain space according to each key assignments;
Data and the mutual mapping block of node, for traveling through this chain space from the key assignments of a data object towards a fixed-direction, until run into the key assignments of a memory node, then mapped this data object and this memory node by respective key assignments mutually; Then the mutual mapping of next data object and memory node is carried out according to identical mode, until complete the mutual mapping of all data objects and memory node.
5., according to device according to claim 4, it is characterized in that,
Described storage space management module is when having monitored new memory node and will being increased to cloud storage system, newly-increased node notice is sent to described memory node mapping block, and/or when having monitored memory node and will having removed from cloud storage system, send to described memory node mapping block and remove node notice;
After described memory node mapping block receives described newly-increased node notice, calculate the key assignments hashed to by this newly-increased memory node in chain space according to the position feature of newly-increased memory node, and map this newly-increased memory node to chain space according to this key assignments; And/or storage space management module receive described in remove node notice after, from described chain space, remove the mapping of this memory node; Then notify that described data and the mutual mapping block of node carry out remapping of data and node;
Data and the mutual mapping block of node are according to the notice of described memory node mapping block, in described chain space from the key assignments of memory node that is newly-increased or that remove along traversing the key assignments place of first former memory node with the opposite direction in the direction of mapping (enum) data object in initial mapping, determine the key assignments of the data object between two key assignments places, and again by the part or all of data object between these two key assignments places and newly-increased memory node or mutually mapped by respective key assignments with the former memory node traversed.
6., according to device according to claim 5, it is characterized in that,
Described data object mapping block adopts mapping function F to calculate the key assignments hashed to by all data objects of system in described chain space according to the set feature of data object, the set feature of wherein said data object comprise in the mark of data block, data block size and data type one or more;
Described memory node mapping block adopts described mapping function F to calculate according to the position feature of memory node and newly-increased memory node is maybe hashed to the key assignments in chain space by memory nodes all in system; The position feature of wherein said memory node comprise in the IP address of memory node, Media Access Control address and machine name one or more;
The content of described mapping function F is as follows:
In described mapping function, described * str is the pointer of the feature string pointing to input, and described BestDivert is best hashed value, and described m is the address size of described memory space.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410554245.9A CN104270458A (en) | 2014-10-17 | 2014-10-17 | Method and device for achieving data distribution management of cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410554245.9A CN104270458A (en) | 2014-10-17 | 2014-10-17 | Method and device for achieving data distribution management of cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104270458A true CN104270458A (en) | 2015-01-07 |
Family
ID=52161947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410554245.9A Pending CN104270458A (en) | 2014-10-17 | 2014-10-17 | Method and device for achieving data distribution management of cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104270458A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298541A (en) * | 2014-10-22 | 2015-01-21 | 浪潮(北京)电子信息产业有限公司 | Data distribution algorithm and data distribution device for cloud storage system |
| CN106302383A (en) * | 2016-07-22 | 2017-01-04 | 北京奇虎科技有限公司 | The processing method of data access request and processing means |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609446A (en) * | 2012-01-05 | 2012-07-25 | 厦门市美亚柏科信息股份有限公司 | Distributed Bloom filter system and application method thereof |
| CN103078946A (en) * | 2013-01-10 | 2013-05-01 | 绍兴电力局 | Mass real-time data distributing method and mass real-time data access method |
| CN103281358A (en) * | 2013-05-07 | 2013-09-04 | 汉柏科技有限公司 | Method for storing resource based on consistency hash algorithm |
| CN103929500A (en) * | 2014-05-06 | 2014-07-16 | 刘跃 | Method for data fragmentation of distributed storage system |
| CN104050270A (en) * | 2014-06-23 | 2014-09-17 | 成都康赛信息技术有限公司 | Distributed storage method based on consistent Hash algorithm |
-
2014
- 2014-10-17 CN CN201410554245.9A patent/CN104270458A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609446A (en) * | 2012-01-05 | 2012-07-25 | 厦门市美亚柏科信息股份有限公司 | Distributed Bloom filter system and application method thereof |
| CN103078946A (en) * | 2013-01-10 | 2013-05-01 | 绍兴电力局 | Mass real-time data distributing method and mass real-time data access method |
| CN103281358A (en) * | 2013-05-07 | 2013-09-04 | 汉柏科技有限公司 | Method for storing resource based on consistency hash algorithm |
| CN103929500A (en) * | 2014-05-06 | 2014-07-16 | 刘跃 | Method for data fragmentation of distributed storage system |
| CN104050270A (en) * | 2014-06-23 | 2014-09-17 | 成都康赛信息技术有限公司 | Distributed storage method based on consistent Hash algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 周敬利，周正达: "改进的云存储系统数据分布策略", 《计算机应用》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298541A (en) * | 2014-10-22 | 2015-01-21 | 浪潮(北京)电子信息产业有限公司 | Data distribution algorithm and data distribution device for cloud storage system |
| CN106302383A (en) * | 2016-07-22 | 2017-01-04 | 北京奇虎科技有限公司 | The processing method of data access request and processing means |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Grandison et al. | Towards a formal definition of a computing cloud | |
| CN104407879B (en) | A kind of power network sequential big data loaded in parallel method | |
| CN101957863A (en) | Data parallel processing method, device and system | |
| CN103929454A (en) | Load balancing storage method and system in cloud computing platform | |
| CN105357322A (en) | Virtual machine distribution method based on topology partition | |
| CN103294799B (en) | A kind of data parallel batch imports the method and system of read-only inquiry system | |
| CN104951352A (en) | Method and equipment for migrating data flow | |
| CN102142032A (en) | Method and system for reading and writing data of distributed file system | |
| CN111966289A (en) | Partition optimization method and system based on Kafka cluster | |
| CN104036286A (en) | Multispectral remote-sensing image classification processing method based on Hadoop | |
| CN105007328A (en) | Network cache design method based on consistent hash | |
| CN102970349B (en) | A kind of memory load equalization methods of DHT network | |
| CN113835823A (en) | Resource scheduling method and device, electronic equipment and computer readable storage medium | |
| CN104270458A (en) | Method and device for achieving data distribution management of cloud storage system | |
| CN104951442A (en) | Method and device for determining result vector | |
| CN204906437U (en) | Big data storage application network framework | |
| CN107967172B (en) | Spark task dynamic migration method and system for heterogeneous storage | |
| WO2024021470A1 (en) | Cross-region data scheduling method and apparatus, device, and storage medium | |
| CN109388615A (en) | Task processing method and system based on Spark | |
| Song et al. | Ashman: A bandwidth fragmentation-based dynamic flow scheduling for data center networks | |
| CN101566933A (en) | Method and device for configurating cache and electronic equipment and data read-write equipment | |
| CN104850548A (en) | Method and system used for implementing input/output process of big data platform | |
| CN110554916B (en) | Distributed cluster-based risk index calculation method and device | |
| CN114691315A (en) | Memory pool data placement techniques | |
| Fan et al. | Smart-blocking file storage method in cloud computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150107 |