CN104572842B - A kind of striping in distributed file system rolls up growth data moving method - Google Patents
A kind of striping in distributed file system rolls up growth data moving method Download PDFInfo
- Publication number
- CN104572842B CN104572842B CN201410768279.8A CN201410768279A CN104572842B CN 104572842 B CN104572842 B CN 104572842B CN 201410768279 A CN201410768279 A CN 201410768279A CN 104572842 B CN104572842 B CN 104572842B
- Authority
- CN
- China
- Prior art keywords
- volume
- son
- striping
- data
- file system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/119—Details of migration of file systems
Abstract
A kind of striping in distributed file system rolls up growth data moving method, extends volume RI including calculating and calculates the new volume position in RI；New storage location of original sub- volume strip data after volume extension can be iterated to calculate out by the calculation method, data are disposably directly migrated into target volume, significantly reduce Data Migration expense, and the strip data after migrating is uniformly distributed on all sub- volumes, to ensure that the processing capability in real time and simultaneous operation property of file system, increases IOPS and transmission gulps down bandwidth.
Description
Technical field
Striping the present invention relates to growth data moving method technical field, in especially a kind of distributed file system
Roll up growth data moving method.
Background technique
In distributed file system, for the IOPS and transmission bandwidth for improving parallel memorizing, it can be protected using striping volume
Deposit file and read-write data.One striping volume is rolled up comprising several band beggars.File is divided according to preset stripe size
For each data block, these strip datas are successively stored in band beggar volume by certain sequence rules.Therefore, in single son volume
The not complete file of storage, the part strip data of file.Hold when the increase of storage file scale has exceeded existing volume
It when amount, needs to be extended striping volume, increases son volume quantity, expand and roll up memory space.In order to improve the band after extension
The concurrent reading and writing performance for changing volume needs to carry out data migration operation from original sub- volume to new sub- volume.Data between band beggar volume
Migration target is: being evenly distributed on data strip on each sub- volume, each volume band number is of substantially equal；2) data are reduced to the greatest extent to move
Number is moved, migration overhead is reduced.This patent proposes a kind of new distributed file system striping volume growth data migrating technology,
No matter carrying out more large-scale son volume extensions, extended operation can be realized with the mobile number of the smallest band, significantly reduced
Data Migration expense, and when strip data can be uniformly distributed on all sub- volumes, to improve the real-time processing of file system
Ability and simultaneous operation property, effectively increase IOPS and throughput of transmissions.
Summary of the invention
In view of the above-mentioned problems, the present invention is intended to provide the striping volume growth data in a kind of distributed file system migrates
Method.
To realize the technical purpose, the invention adopts the following technical scheme: the striping in a kind of distributed file system
Growth data moving method is rolled up, including calculating extension volume RI and calculating the new volume position in RI；
Wherein in extension volume RI is calculated: calculating the RI of N number of extension volumei, wherein i=(2,3 ..., N), specific steps
It is as follows；
Step 1, when being extended to 2 son volumes from the 1st son volume, RI2=2；
Step 2, if be extended to N-1 volume, the N-1 volumes of RIN-1=k then extends to N volume are as follows:
That is: RINThe product of the power of=whole prime numbers less than or equal to N, wherein the power side of each prime number is less than or equal to N；Example
Such as: RI5=22× 3 × 5=60, RI6=22× 3 × 5=60；
In the new volume position calculated in RI: calculating the new volume position in RI, specific step is as follows；
Step 1, pre-establishing size in memory is RINMapping table, multiple tables can be established for different N；
Step 2, iterative calculation:
(1), to 1 to RINA strip data carries out n times and extends iteration, and each new sub- volume position is generated in table；
(2), iterative formula:
It is rolled up assuming that striping volume is extended to n son, n son volume can be regarded as n column, roll up interior each data block by rows,
Every height rolls up every one data block of behavior；For arbitrary data block a, if its position coordinates in striping volume is (xn,a,
yn,a),xn,aSon volume number, y where an,aThe son where a rolls up interior line number, then can iteratively solve (x by following equationn,a,
yn,a),
Or
Wherein, q yn-1,aDivided by the integer part of n；
Compared with prior art, the beneficial effects of the present invention are: can be iterated to calculate out by above-mentioned calculation method original
Data are disposably directly migrated to target volume, greatly reduced by new storage location of the son volume strip data after volume extension
Data Migration expense, and the strip data after migrating is uniformly distributed on all sub- volumes, to ensure that file system
Processing capability in real time and simultaneous operation property, increase IOPS and transmission gulp down bandwidth.
Specific embodiment
The technical scheme in the embodiments of the invention will be clearly and completely described below, it is clear that described implementation
Example is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, this field is common
Technical staff's all other embodiment obtained without making creative work belongs to the model that the present invention protects
It encloses.
In the embodiment of the present invention, the striping in a kind of distributed file system rolls up growth data moving method, including meter
It calculates extension volume RI and calculates the new volume position in RI；Due to following certain rule by the mode of volume iteration, current son volume item
It is also regular behavior with disposable redistribution to N number of extension volume.It is found by analysis, successively calculates each band in N number of son
When the new position of volume, by certain bin number interval, calculated new position, which will appear, to be repeated cyclically, this repeat number
The recurrence interval of referred to as N number of band beggar volume, is denoted as RI.
If can know that out the band distributing position in RI, position is newly rolled up after being known that the extension of whole bands.
Wherein in extension volume RI is calculated: calculating the RI of N number of extension volumei, wherein i=(2,3 ..., N), specific steps
It is as follows；
Step 1, when being extended to 2 son volumes from the 1st son volume, RI2=2；
Step 2, if be extended to N-1 volume, the N-1 volumes of RIN-1=k then extends to the recurrence interval of N volume
Are as follows:
That is: RINThe product of the power of=whole prime numbers less than or equal to N, wherein the power side of each prime number is less than or equal to N；Example
Such as: RI5=22× 3 × 5=60, RI6=22× 3 × 5=60；
In the new volume position calculated in RI: the new volume position in RI is calculated, such as: RI10=23×32× 5 × 7=
2520；
Step 1, pre-establishing size in memory is RINMapping table, multiple tables can be established for different N；
Step 2, iterative calculation:
(1), to 1 to RINA strip data carries out n times and extends iteration, and each new sub- volume position is generated in table；
(2), iterative formula:
It is rolled up assuming that striping volume is extended to n son, n son volume can be regarded as n column, roll up interior each data block by rows,
Every height rolls up every one data block of behavior；For arbitrary data block a, if its position coordinates in striping volume is (xn,a,
yn,a),xn,aSon volume number, y where an,a, line number in son volume, then can iteratively solve (x by following equation where a is an,a,
yn,a),
Or
Wherein q is yn-1,aDivided by the integer part of n；
The smallest proof (summary) of Data Migration number；
The proof (summary) of the uniform redistribution of data.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case where without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Benefit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent elements of the claims
Variation is included within the present invention.Any label in claim should not be construed as limiting the claims involved.
The above, only presently preferred embodiments of the present invention, are not intended to limit the invention, all skills according to the present invention
Art any trickle amendment, equivalent replacement and improvement substantially to the above embodiments, should be included in technical solution of the present invention
Protection scope within.
Claims (1)
1. the striping in a kind of distributed file system rolls up growth data moving method, including calculates extension volume RI and calculate RI
Interior new volume position；It is characterized by:
Wherein in extension volume RI is calculated: calculating the RI of N number of extension volumei, wherein i=(2,3 ..., N), the specific steps are as follows:
Step 1, when being extended to 2 son volumes from the 1st son volume, RI2=2；
Step 2, if be extended to N-1 volume, if the N-1 volumes of RIN-1=k, k are positive integer, then extend to N volume are as follows:
That is: RINThe product of the power of=whole prime numbers less than or equal to N, wherein the power side of each prime number is less than or equal to N；It calculates in RI
New volume position, the specific steps are as follows:
Step 1, pre-establishing size in memory is RINMapping table, multiple tables can be established for different N；
Step 2, iterative calculation:
(1), to 1 to RINA strip data carries out n times and extends iteration, and each new sub- volume position is generated in table；
(2), iterative formula:
It is rolled up assuming that striping volume is extended to n son, n son volume can be regarded as n column, roll up interior each data block by rows, each
Son rolls up every one data block of behavior；For arbitrary data block a, if its position coordinates in striping volume is (xN, a, yN, a),
xN, aSon volume number, y where aN, aThe son where a rolls up interior line number, then can iteratively solve (x by following equationN, a, yN, a),
Or
Wherein, q yN-1, aDivided by the integer part of n.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410768279.8A CN104572842B (en) | 2014-12-04 | 2014-12-04 | A kind of striping in distributed file system rolls up growth data moving method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410768279.8A CN104572842B (en) | 2014-12-04 | 2014-12-04 | A kind of striping in distributed file system rolls up growth data moving method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104572842A CN104572842A (en) | 2015-04-29 |
| CN104572842B true CN104572842B (en) | 2018-12-28 |
Family
ID=53088904
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410768279.8A Expired - Fee Related CN104572842B (en) | 2014-12-04 | 2014-12-04 | A kind of striping in distributed file system rolls up growth data moving method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104572842B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106909595B (en) * | 2016-06-20 | 2020-12-29 | 创新先进技术有限公司 | Data migration method and device |
| CN110058789B (en) * | 2018-01-18 | 2022-06-03 | 伊姆西Ip控股有限责任公司 | Method for managing storage system, storage system and computer program product |
| CN113590042B (en) * | 2021-07-29 | 2024-03-19 | 杭州宏杉科技股份有限公司 | Data protection storage method, device and equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6000010A (en) * | 1997-05-09 | 1999-12-07 | Unisys Corporation | Method of increasing the storage capacity of a level five RAID disk array by adding, in a single step, a new parity block and N--1 new data blocks which respectively reside in a new columns, where N is at least two |
| CN104182176A (en) * | 2013-05-24 | 2014-12-03 | 华中科技大学 | Rapid dilatation method for RAID 5 (redundant array of independent disks) |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080276041A1 (en) * | 2007-05-01 | 2008-11-06 | International Business Machines Corporation | Data storage array scaling method and system with minimal data movement |
-
2014
- 2014-12-04 CN CN201410768279.8A patent/CN104572842B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6000010A (en) * | 1997-05-09 | 1999-12-07 | Unisys Corporation | Method of increasing the storage capacity of a level five RAID disk array by adding, in a single step, a new parity block and N--1 new data blocks which respectively reside in a new columns, where N is at least two |
| CN104182176A (en) * | 2013-05-24 | 2014-12-03 | 华中科技大学 | Rapid dilatation method for RAID 5 (redundant array of independent disks) |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104572842A (en) | 2015-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104572842B (en) | A kind of striping in distributed file system rolls up growth data moving method | |
| EP2306293A3 (en) | Methods and apparatus for managing virtual ports and logical units on storage systems | |
| TW200603013A (en) | Method, computer program product and data structure for representing two or three-dimensional object | |
| GB2506828A (en) | Transferring de-duplicated data organized in virtual volumes to a target set of physical media | |
| CN103677674B (en) | A kind of data processing method and device | |
| CN105608084B (en) | The method of adjustment and device of web page layout | |
| CN105160442A (en) | Physical layout algorithm for circular pieces | |
| WO2012033650A3 (en) | Machine, computer program product and method to generate unstructured grids and carry out parallel reservoir simulation | |
| CN104504742A (en) | Three-dimensional military mark plotting method based on electronic sand table system | |
| CN104548597A (en) | Navigation grid automatic-generation method and navigation grid automatic-generation device | |
| CN103092886B (en) | A kind of implementation method of data query operation, Apparatus and system | |
| CN108874200B (en) | The control method and device of writing speed | |
| EP2921960A3 (en) | Method of, and apparatus for, accelerated data recovery in a storage system | |
| CN102289605A (en) | Contour line interpolation method | |
| CN104636038B (en) | A kind of system and method for painting slider bar certainly | |
| WO2015065423A3 (en) | Treemap optimization | |
| CN104200530A (en) | Painting method and system of contour line of spatial curved surface | |
| CN110490371A (en) | Packing method based on genetic algorithm | |
| CN101696997B (en) | Adaptive rapid determining method for multi-medium power frequency electric field distribution | |
| CN108875275A (en) | A kind of vector field real-time emulation method and system based on extensive streamline | |
| JP2015088141A5 (en) | ||
| CN203659267U (en) | Logistics from-to-chart algorithm training aid | |
| CN107168892A (en) | The wiring method and device of a kind of data | |
| CN204305088U (en) | A kind of cloud for education and scientific research unit stores integrative machine | |
| CN105321208B (en) | A kind of quadrilateral subdivision curved surface access method based on vertex encoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181228 Termination date: 20191204 |