CN105760480A - Distribution algorithm for cluster file system - Google Patents
Distribution algorithm for cluster file system Download PDFInfo
- Publication number
- CN105760480A CN105760480A CN201610086344.8A CN201610086344A CN105760480A CN 105760480 A CN105760480 A CN 105760480A CN 201610086344 A CN201610086344 A CN 201610086344A CN 105760480 A CN105760480 A CN 105760480A
- Authority
- CN
- China
- Prior art keywords
- file system
- dummy node
- cluster file
- group document
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention discloses a distribution algorithm for a cluster file system. According to the algorithm, MD5 values are divided into at least one equivalent range according to the number of virtual nodes in the cluster file system, each MD5 value range corresponds to one virtual node in cluster files, a cluster file catalog is stored on each virtual node, and the MD5 values can be dynamically adjusted according to the number of the virtual nodes. By means of the distribution algorithm for the cluster file system, it can be effectively avoided that extension of the cluster file system is limited.
Description
Technical field
The present invention relates to technical field of data processing, particularly relate to the Distribution Algorithm of cluster file system.
Background technology
Along with the development of cluster-based storage technology, the distribution of the cluster file system extended capability particular importance to cluster.Most cluster file system all adopts Centroid, and the mode of meta data server realizes, and has a disadvantage in that the extension of cluster file system is subject to Centroid, the restriction of meta data server.
Summary of the invention
One of them purpose of the present invention is the Distribution Algorithm proposing a kind of cluster file system, solves prior art and there is the technical problem that the extension of cluster file system is restricted.Many technique effects elaboration as detailed below that optimal technical scheme in many technical schemes provided by the invention can be generated by.
For achieving the above object, the invention provides techniques below scheme:
A kind of Distribution Algorithm of cluster file system, described algorithm is by being divided at least one equivalency range according to the dummy node number in cluster file system by MD5 value, each MD5 value scope is corresponding to a dummy node in group document, on each dummy node, storage has group document catalogue, and described MD5 value dynamically can adjust according to the quantity of dummy node.
Preferably, by the MD5 value of the group document catalogue that MD5 algorithm calculates, value range is searched this MD5 value, it is determined that the dummy node in group document, and then determine document location.
Preferably, each dummy node includes at least one physical node, and described physical node is used for storage cluster file data, and between described physical node, the group document data of storage are identical.
Preferably, when increasing in cluster file system or reducing dummy node, MD5 value can be divided at least one equivalency range by new nodes by described Distribution Algorithm again, and updates the dummy node in the group document that MD5 value scope is corresponding.
Preferably, the described dummy node updated in the group document that MD5 value scope is corresponding refers to that existing file will make group document migrate according to new corresponding relation by balanced algorithm, and described balanced algorithm can first be set up file association link and group document is linked to new dummy node.
Preferably, when trunked idle, group document will need the file migration migrated to new dummy node.
Preferably, after group document moves to new dummy node, delete file association link and original group document.
Based on technique scheme, the embodiment of the present invention can be prevented effectively from the extension of cluster file system and be restricted.
Accompanying drawing explanation
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, and the schematic description and description of the present invention is used for explaining the present invention, is not intended that inappropriate limitation of the present invention.In the accompanying drawings:
The cluster file system distribution schematic diagram that Fig. 1 provides for the embodiment of the present invention；
The cluster file system distribution schematic diagram that Fig. 2 provides for embodiment of the present invention preferred implementation.
Detailed description of the invention
It is referred to accompanying drawing Fig. 1 below and word content understands the distinctive points between present disclosure and the present invention and prior art.Hereafter by the mode of accompanying drawing and some alternative embodiments enumerating the present invention, technical scheme (including optimal technical scheme) is described in further detail.It should be understood that any technical characteristic in the present embodiment, any technical scheme is all one or more in the technical characteristic of plurality of optional or optional technical scheme, cannot all alternative technical characteristic of the exhaustive present invention and alternative technical scheme in order to describe succinct need in presents, also the embodiment being not easy to each technical characteristic all emphasizes that it is one of optional numerous embodiments, so those skilled in the art should know: any technology means provided by the invention can be replaced or any two provided by the invention or more technological means or technical characteristic are combined mutually and obtain new technical scheme.Any technical characteristic and any technical scheme in the present embodiment are all not intended to protection scope of the present invention, protection scope of the present invention should include those skilled in the art do not pay creative work thinkable any alternate embodiments and those skilled in the art new technical scheme that any two provided by the invention or more technological means or technical characteristic are combined mutually and obtain.
Embodiments provide the Distribution Algorithm of a kind of cluster file system.
Below in conjunction with Fig. 1, technical scheme provided by the invention is carried out elaboration specifically.
The Distribution Algorithm of a kind of cluster file system that the embodiment of the present invention provides, described algorithm is by being divided at least one equivalency range according to the dummy node number in cluster file system by MD5 (message digest algorithm the 5th edition) value, each MD5 value scope is corresponding to a dummy node in group document, on each dummy node, storage has group document catalogue, and described MD5 value dynamically can adjust according to the quantity of dummy node.nullAs shown in Figure 1，MD5 value is divided into two scopes，Corresponding two dummy nodes respectively，Wherein the first corresponding for dummy node VirNode1 MD5 value scope is 00000000～7FFFFFFF，Second corresponding for dummy node VirNode2 MD5 value scope is 8FFFFFFF～FFFFFFFF，Under the first dummy node and the second dummy node, all storage has group document catalogue，When locating file，First absolute path and the group document catalogue of file are obtained，The MD5 value of group document catalogue is calculated again through MD5 algorithm，Then in value range, search this MD5 value，Determine the dummy node in group document，And then determine document location，The MD5 value of a such as file directory is 2FFFFFFF，Then this document is positioned at the first dummy node VirNode1，In the first dummy node VirNode1, file is found again through ff algorithm.
As in figure 2 it is shown, each dummy node includes at least one physical node, described physical node is used for storage cluster file data, and between described physical node, the group document data of storage are identical.Data can be stored on multiple physical node, and each physical node can have a complete file data.Data in so each dummy node will form redundancy, namely have physical node to break down in use dummy node and also can guarantee that the safety of file data can use.
When increasing in cluster file system or reducing dummy node, MD5 value range can adjust dynamically according to the quantity of node, namely adopts elastic MD5 scope, it is possible to the extension being prevented effectively from cluster file system is restricted.MD5 value can be divided at least one equivalency range by new nodes by described Distribution Algorithm again, and updates the dummy node in the group document that MD5 value scope is corresponding.It is to say, when increasing in cluster file system or reducing dummy node, MD5 value range will be calculated by new dummy node number again, and new MD5 value range will set up corresponding relation with dummy node again.
Data balancing operation is completed by other algorithm and balanced algorithm, the process of its operation can produce Data Migration, in order to cluster performance radiography not being rung, file is first associated by the mode adopting file link by new MD5 distribution, namely set up file association link and group document is linked on new dummy node, just carry out real migration operation when trunked idle.File association link and original group document are deleted in migration after completing.
Any technology scheme disclosed in the invention described above is unless otherwise stated, if it discloses numerical range, so disclosed numerical range is preferred numerical range, any it should be appreciated by those skilled in the art: preferred numerical range is only the numerical value that in many enforceable numerical value, technique effect is obvious or representative.Owing to numerical value is more, it is impossible to exhaustive, so the present invention just discloses component values to illustrate technical scheme, and, the above-mentioned numerical value enumerated should not constitute the restriction to the invention protection domain.
If being used herein the word such as " first ", " second " to limit parts, those skilled in the art should know: the use of " first ", " second " is intended merely to facilitate in description and parts carries out difference as not having outside Stated otherwise, and above-mentioned word does not have special implication.
It addition, term its implication unless otherwise stated for representing position relationship or shape applied in any technology scheme disclosed in the invention described above includes approximate with it, similar or close state or shape.Either component provided by the invention both can be assembled by multiple independent ingredients, it is also possible to the produced separate part of the technique that is one of the forming.
Finally should be noted that: above example is only in order to illustrate that technical scheme is not intended to limit；Although the present invention being described in detail with reference to preferred embodiment, those of ordinary skill in the field are it is understood that still can modify to the specific embodiment of the present invention or portion of techniques feature carries out equivalent replacement；Without deviating from the spirit of technical solution of the present invention, it all should be encompassed in the middle of the technical scheme scope that the present invention is claimed.
Claims (7)
1. the Distribution Algorithm of a cluster file system, described algorithm is by being divided at least one equivalency range according to the dummy node number in cluster file system by MD5 value, each MD5 value scope is corresponding to a dummy node in group document, on each dummy node, storage has group document catalogue, and described MD5 value dynamically can adjust according to the quantity of dummy node.
2. the Distribution Algorithm of cluster file system according to claim 1, it is characterised in that by the MD5 value of the group document catalogue that MD5 algorithm calculates, search this MD5 value in value range, it is determined that the dummy node in group document, and then determine document location.
3. the Distribution Algorithm of cluster file system according to claim 1, it is characterised in that each dummy node includes at least one physical node, described physical node is used for storage cluster file data, and between described physical node, the group document data of storage are identical.
4. the Distribution Algorithm of the cluster file system according to any one of claims 1 to 3, it is characterized in that, when cluster file system increasing or reducing dummy node, MD5 value can be divided at least one equivalency range by new nodes by described Distribution Algorithm again, and updates the dummy node in the group document that MD5 value scope is corresponding.
5. the Distribution Algorithm of cluster file system according to claim 4, it is characterized in that, the described dummy node updated in the group document that MD5 value scope is corresponding refers to that existing file will make group document migrate according to new corresponding relation by balanced algorithm, and described balanced algorithm can first be set up file association link and group document is linked to new dummy node.
6. the Distribution Algorithm of cluster file system according to claim 5, it is characterised in that when trunked idle, will need the file migration migrated to new dummy node in group document.
7. the Distribution Algorithm of cluster file system according to claim 6, it is characterised in that after group document moves to new dummy node, deletes file association link and original group document.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610086344.8A CN105760480A (en) | 2016-02-15 | 2016-02-15 | Distribution algorithm for cluster file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610086344.8A CN105760480A (en) | 2016-02-15 | 2016-02-15 | Distribution algorithm for cluster file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105760480A true CN105760480A (en) | 2016-07-13 |
Family
ID=56330710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610086344.8A Pending CN105760480A (en) | 2016-02-15 | 2016-02-15 | Distribution algorithm for cluster file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105760480A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106933508A (en) * | 2017-02-14 | 2017-07-07 | 深信服科技股份有限公司 | The moving method and device of application container |
| CN111897494A (en) * | 2020-07-27 | 2020-11-06 | 星辰天合（北京）数据科技有限公司 | Target file processing method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6058400A (en) * | 1998-04-28 | 2000-05-02 | Sun Microsystems, Inc. | Highly available cluster coherent filesystem |
| CN102395962A (en) * | 2009-03-11 | 2012-03-28 | 甲骨文国际公司 | Composite hash and list partitioning of database tables |
-
2016
- 2016-02-15 CN CN201610086344.8A patent/CN105760480A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6058400A (en) * | 1998-04-28 | 2000-05-02 | Sun Microsystems, Inc. | Highly available cluster coherent filesystem |
| CN102395962A (en) * | 2009-03-11 | 2012-03-28 | 甲骨文国际公司 | Composite hash and list partitioning of database tables |
Non-Patent Citations (1)
| Title |
|---|
| 余秦勇 等: "一种无中心的云存储架构分析", 《通信技术》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106933508A (en) * | 2017-02-14 | 2017-07-07 | 深信服科技股份有限公司 | The moving method and device of application container |
| CN106933508B (en) * | 2017-02-14 | 2020-05-12 | 深信服科技股份有限公司 | Application container migration method and device |
| CN111897494A (en) * | 2020-07-27 | 2020-11-06 | 星辰天合（北京）数据科技有限公司 | Target file processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9996264B2 (en) | Data block migration | |
| CN105045877B (en) | Database data fragment storage method and device, data query method and apparatus | |
| US9904599B2 (en) | Method, device, and system for data reconstruction | |
| US9092153B2 (en) | Method for creating clone file, and file system adopting the same | |
| JP5992511B2 (en) | Cloud service reconnection automation | |
| JP2020525906A (en) | Database tenant migration system and method | |
| US20110307456A1 (en) | Active file instant cloning | |
| CN109992206B (en) | Data distribution storage method and related device | |
| CN111078121A (en) | Data migration method, system and related components of distributed storage system | |
| US20170177652A1 (en) | Copying garbage collector for b+ trees under multi-version concurrency control | |
| US20150269214A1 (en) | Identifying files in change logs using file content location identifiers | |
| WO2018201832A1 (en) | Data migration method and system for distributed cache database | |
| US11256578B2 (en) | Determining a replication path for resources of different failure domains | |
| US10970256B2 (en) | Reconciler for a distributed storage system | |
| CN105760480A (en) | Distribution algorithm for cluster file system | |
| US7962453B2 (en) | Dynamic redistribution of a distributed memory index when individual nodes have different lookup indexes | |
| Gao et al. | An efficient ring-based metadata management policy for large-scale distributed file systems | |
| CN104168158B (en) | The method that multidimensional network carries out network analysis based on betweenness center degree | |
| JP2016099969A (en) | Information processing apparatus, data storage system, and data storage method | |
| WO2017145214A1 (en) | Computer system for transferring data from center node to edge node | |
| Zhou et al. | JPR: Exploring joint partitioning and replication for traffic minimization in online social networks | |
| Avilés-González et al. | Scalable metadata management through OSD+ devices | |
| CN115563073A (en) | Method and device for data processing of distributed metadata and electronic equipment | |
| Liu et al. | AngleCut: A ring-based hashing scheme for distributed metadata management | |
| US10684992B1 (en) | Causally ordering distributed file system events |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160713 |