CN102622284B - Data asynchronous replication method directing to mass storage system - Google Patents
Data asynchronous replication method directing to mass storage system Download PDFInfo
- Publication number
- CN102622284B CN102622284B CN201210039678.1A CN201210039678A CN102622284B CN 102622284 B CN102622284 B CN 102622284B CN 201210039678 A CN201210039678 A CN 201210039678A CN 102622284 B CN102622284 B CN 102622284B
- Authority
- CN
- China
- Prior art keywords
- data
- lease
- copy
- server
- data server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention relates to a data asynchronous replication method directing to a mass storage system. The method comprises the following steps: step 1, a client inquires data server information containing write lease of a data block from a management server; step 2, the management server carries out inquiry and allocation operation of the write lease; step 3, after receiving the data server information containing the write lease, the client sends a write request to the data server, and after receiving the write request, the data server writes data in a corresponding copy and adds one to the version number of the copy; step 4, the management server takes tasks from an asynchronous replication array periodically, and replication operation is performed to the copy by the data server; and step 5, the data server informs the management server that the task of copy replication is already finished, the management server removes the task from the asynchronous replication array, and the next task from the asynchronous replication array is executed. Compared with the prior art, the method has the advantages of high replication efficiency and the like.
Description
Technical field
The present invention relates to a kind of data copy method, especially relate to a kind of data asynchronous replication method towards mass storage system (MSS).
Background technology
Mass storage system (MSS) is a kind of network store system that can store Large Volume Data.Mass storage system (MSS), in order to obtain the performance of larger capacity and Geng Gao, often adopts distributed system architecture at present.Wherein typical architecture is a management server, a plurality of data servers, a plurality of client computer.File is divided into the data block of formed objects, and data block is kept on data server.On management server, storing the name space of system, the metadata of file, the information such as mapping relations of file and data block.It is mutual that client computer and management server carry out metadata, carries out data interaction with data server.
In mass storage system (MSS), have a large amount of memory devices, therefore can think that it is recurrent situation that memory device breaks down.For the high availability of system, for a data block, to preserve multiple copies, and leave on different data servers as far as possible.When carrying out write operation, owing to data will being write to multiple copies, the efficiency of write operation becomes a problem.
Through existing literature search is found, in < < The Google File System (Google file system) > >, a kind of method that data synchronously copy has been proposed.What Google file system (GFS) adopted is the typical architecture of mass storage system (MSS) mentioned above, and the data block (Chunk) that it is divided into 64MB by file is stored.Client computer is carried out write operation to a Chunk, after all copies that must wait until this Chunk have all completed write operation, just to client computer, returns to the successful information of write operation.Although this method is conceptive simple, the each write operation of the method all will write all copies immediately, has reduced the efficiency of write operation, particularly, when copy number is many, can have a strong impact on write operation execution speed.
Summary of the invention
Object of the present invention is exactly to provide in order to overcome the defect of above-mentioned prior art existence a kind of data asynchronous replication method towards mass storage system (MSS) that duplicating efficiency is high.
Object of the present invention can be achieved through the following technical solutions:
A data asynchronous replication method towards mass storage system (MSS), is characterized in that, comprises the following steps:
The first step, client computer has to management server queries the data server information that data block is write lease;
Second step, management server receives after inquiry request, carries out inquiry and the batch operation of writing lease;
The 3rd step, client computer receives and has after the data server information of writing lease, to it, send write request, data server receives after write request, write data into corresponding copy, and the version number of copy is added to one, and to client computer, return to write operation successful information, to the asynchronous replication queue of management server, add new task;
The 4th step, management server is periodically got task from asynchronous replication queue, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
The 5th step, data server notice management server has completed copy replication task, and management server is deleted this task from asynchronous replication queue, then carries out the next task in asynchronous replication queue.
The inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block, if find that there is the lease of writing that data server has data block, by having the data server information of writing lease, return to client computer, finish to write inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block, if find that there is the lease of reading that data server has data block, according to the alternative of read-write lease, notice client computer cannot obtain the lease of writing of data block, finish to write inquiry and the batch operation of lease, otherwise carry out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at the copy place that version number is the highest, and the lease of writing of data block is distributed to this data server, and returns to client computer by having the data server information of writing lease.
The replicate run of the copy in described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece, management server receives after request, check writing lease and whether being held of data block, if words replication task is joined to asynchronous replication queue afterbody, reply data server application and read lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and perform step 43);
43) data server sends read request to the data server at top version number copy place, and the data server at top version number copy place receives after request, and data and version number are passed to the data server sending request;
44) data server writes the data that receive in the data block copy of the machine, after data write, version number is upgraded to identical with top version number copy.
Compared with prior art, the present invention has the following advantages:
When having the data block of multiple copies and carry out write operation, only need to write successfully and just return and write successfully immediately one of them copy, the employing data asynchronous replication method that writes of other copies, thus improved the efficiency of write operation.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of writing lease inquiry and batch operation in second step of the present invention;
Fig. 2 is the process flow diagram of the copy replication operation in the present invention's the 4th step.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.
Embodiment 1
A data asynchronous replication method towards mass storage system (MSS), is characterized in that, comprises the following steps:
The first step, client computer has to management server queries the data server information that data block is write lease;
Second step, management server receives after inquiry request, carries out inquiry and the batch operation of writing lease;
The 3rd step, client computer receives and has after the data server information of writing lease, to it, send write request, data server receives after write request, write data into corresponding copy, and the version number of copy is added to one, and to client computer, return to write operation successful information, to the asynchronous replication queue of management server, add new task;
The 4th step, management server is periodically got task from asynchronous replication queue, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
The 5th step, data server notice management server has completed copy replication task, and management server is deleted this task from asynchronous replication queue, then carries out the next task in asynchronous replication queue.
As shown in Figure 1, the inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block, if find that there is the lease of writing that data server has data block, by having the data server information of writing lease, return to client computer, finish to write inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block, if find that there is the lease of reading that data server has data block, according to the alternative of read-write lease, notice client computer cannot obtain the lease of writing of data block, finish to write inquiry and the batch operation of lease, otherwise carry out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at the copy place that version number is the highest, and the lease of writing of data block is distributed to this data server, and returns to client computer by having the data server information of writing lease.
As shown in Figure 2, the replicate run of the copy in described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece, management server receives after request, check writing lease and whether being held of data block, if words replication task is joined to asynchronous replication queue afterbody, reply data server application and read lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and perform step 43);
43) data server sends read request to the data server at top version number copy place, and the data server at top version number copy place receives after request, and data and version number are passed to the data server sending request;
44) data server writes the data that receive in the data block copy of the machine, after data write, version number is upgraded to identical with top version number copy.
Embodiment 2
The present embodiment comprises: the management server of 1 storing metadata, and the data server of several storage data, several client computer, comprise the following steps:
The first step, client computer, before data block CHUNK-A is carried out to write operation, has to management server queries the data server information that CHUNK-A writes lease.
Second step, management server receives after the inquiry request that CHUNK-A writes lease, carries out inquiry and the batch operation of writing lease.
The described inquiry of writing lease and batch operation, concrete steps are:
1) management server is searched the lease of writing that whether has data server to have CHUNK-A, if find that there is the lease of writing that data server has CHUNK-A, this data server is DS-A in the present embodiment, the information of DS-A is returned to client computer, finish to write inquiry and the batch operation of lease, otherwise carry out 2);
2) management server is searched the lease of reading that whether has data server to have CHUNK-A, if find that there is the lease of reading that data server has CHUNK-A, this data server is DS-B in the present embodiment, notify client computer cannot obtain the lease of writing of CHUNK-A, finish to write inquiry and the batch operation of lease, otherwise carry out 3);
3) management server is checked copy information table, search the highest copy of CHUNK-A version number, CHUNK-A has 3 parts of copies in the present embodiment, is respectively CHUNK-A1, CHUNK-A2, CHUNK-A3, their version number is respectively 10,10,9, they are stored in respectively data server DS-C, and DS-D and DS-E are upper, so the highest copy of version number is CHUNK-A1;
4) management server is checked the data server at CHUNK-A1 place, and this data server is DS-C in the present embodiment, the lease of writing of CHUNK-A is distributed to DS-C, and the information of DS-C is returned to client computer.
The 3rd step, client computer receives after the information of DS-C, to it, sends the write request to CHUNK-A1, DS-C writes CHUNK-A1 by data, the version number of CHUNK-A1 is increased to 11, to client computer, returns to write operation successful information, to the asynchronous replication queue of management server, add new task.
The 4th step, management server is taking-up task from asynchronous replication queue, carries out the replicate run of copy, and in the present embodiment, this task is the data that copy the upper CHUNK-A1 of DS-C to the CHUNK-A2 on DS-D.
The replicate run of described copy, concrete steps are as follows:
1) DS-D checks the lease of reading that oneself whether has CHUNK-A, if words carry out 3), otherwise carry out 2);
2) DS-D is to the lease of reading of management server application CHUNK-A, management server receives after request, first check writing lease and whether being held of CHUNK-A, if words replication task is joined to asynchronous replication queue afterbody, reply DS-D application and read lease failure, finish the replicate run of copy, otherwise management server will be read lease and distribute to DS-D;
3) DS-D sends read request to DS-C, and DS-C receives after request, and the data of CHUNK-A1 and version number are passed to DS-D;
4) DS-D writes the data that receive in CHUNK-A2, after data write, the version number of CHUNK-A2 is upgraded to identically with CHUNK-A1, is 11 in the present embodiment.
The 5th step, DS-D notice management server has completed data Replica operation, and management server is deleted this task from asynchronous replication queue, then carries out the next task in asynchronous replication queue.
Claims (2)
1. towards a data asynchronous replication method for mass storage system (MSS), it is characterized in that, comprise the following steps:
The first step, client computer has to management server queries the data server information that data block is write lease;
Second step, management server receives after inquiry request, carries out inquiry and the batch operation of writing lease;
The 3rd step, client computer receives and has after the data server information of writing lease, to it, send write request, data server receives after write request, write data into corresponding copy, and the version number of copy is added to one, and to client computer, return to write operation successful information, to the asynchronous replication queue of management server, add new task;
The 4th step, management server is periodically got task from asynchronous replication queue, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
The 5th step, data server notice management server has completed copy replication task, and management server is deleted this task from asynchronous replication queue, then carries out the next task in asynchronous replication queue;
The replicate run of the copy in described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece, management server receives after request, check writing lease and whether being held of data block, if words replication task is joined to asynchronous replication queue afterbody, reply data server application and read lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and perform step 43);
43) data server sends read request to the data server at top version number copy place, and the data server at top version number copy place receives after request, and data and version number are passed to the data server sending request;
44) data server writes the data that receive in the data block copy of the machine, after data write, version number is upgraded to identical with top version number copy.
2. a kind of data asynchronous replication method towards mass storage system (MSS) according to claim 1, is characterized in that, the inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block, if find that there is the lease of writing that data server has data block, by having the data server information of writing lease, return to client computer, finish to write inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block, if find that there is the lease of reading that data server has data block, according to the alternative of read-write lease, notice client computer cannot obtain the lease of writing of data block, finish to write inquiry and the batch operation of lease, otherwise carry out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at the copy place that version number is the highest, and the lease of writing of data block is distributed to this data server, and returns to client computer by having the data server information of writing lease.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210039678.1A CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210039678.1A CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102622284A CN102622284A (en) | 2012-08-01 |
| CN102622284B true CN102622284B (en) | 2014-04-16 |
Family
ID=46562211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210039678.1A Expired - Fee Related CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102622284B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106484321A (en) * | 2016-09-08 | 2017-03-08 | 华为数字技术（成都）有限公司 | A kind of date storage method and data center |
| CN106406770A (en) * | 2016-09-28 | 2017-02-15 | 乐视控股（北京）有限公司 | Data write-in method and system |
| CN107329695B (en) * | 2017-06-22 | 2020-03-20 | 上海七牛信息技术有限公司 | Distributed storage memory management method, system and computer storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101911037A (en) * | 2007-12-27 | 2010-12-08 | 微软公司 | Asynchronous replication |
| CN102136003A (en) * | 2011-03-25 | 2011-07-27 | 上海交通大学 | Large-scale distributed storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7296125B2 (en) * | 2001-11-29 | 2007-11-13 | Emc Corporation | Preserving a snapshot of selected data of a mass storage system |
-
2012
- 2012-02-21 CN CN201210039678.1A patent/CN102622284B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101911037A (en) * | 2007-12-27 | 2010-12-08 | 微软公司 | Asynchronous replication |
| CN102136003A (en) * | 2011-03-25 | 2011-07-27 | 上海交通大学 | Large-scale distributed storage system |
Non-Patent Citations (2)
| Title |
|---|
| "分布式文件系统KFS的架构与性能分析";张轶彬;《微型电脑应用》;20111220;第27卷(第12期);第1～4页 * |
| 张轶彬."分布式文件系统KFS的架构与性能分析".《微型电脑应用》.2011,第27卷(第12期),第1～4页. |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102622284A (en) | 2012-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101259557B1 (en) | Cluster data management system and method for data recovery using parallel processing in cluster data management system | |
| JP6362685B2 (en) | Replication method, program, and apparatus for online hot standby database | |
| CN102136003A (en) | Large-scale distributed storage system | |
| CN109800185B (en) | Data caching method in data storage system | |
| US20180107601A1 (en) | Cache architecture and algorithms for hybrid object storage devices | |
| KR20140061444A (en) | Volatile memory representation of nonvolatile storage device set | |
| CN101997823A (en) | Distributed file system and data access method thereof | |
| WO2014180232A1 (en) | Method and device for responding to a request, and distributed file system | |
| CN103942112A (en) | Magnetic disk fault-tolerance method, device and system | |
| US20210165760A1 (en) | Managing Dependent Delete Operations among Data Stores | |
| CN113377868A (en) | Offline storage system based on distributed KV database | |
| CN111309245B (en) | Hierarchical storage writing method and device, reading method and device and system | |
| CN107295030B (en) | Data writing method and device, data processing method, device and system | |
| CN102622284B (en) | Data asynchronous replication method directing to mass storage system | |
| CN111930716A (en) | Database capacity expansion method, device and system | |
| US10719497B2 (en) | Utilization of optimized ordered metadata structure for container-based large-scale distributed storage | |
| CN111638853A (en) | Data storage method and device, storage cluster, gateway equipment and main equipment | |
| US8880794B2 (en) | Populating a sparsely allocated memory in a storage array | |
| CN102004773A (en) | Data read-write processing method and web application server | |
| US11625503B2 (en) | Data integrity procedure | |
| CN102867029B (en) | A kind of method managing distributive catalogue of document system and distributed file system | |
| CN104484136A (en) | Method capable of supporting high-concurrency memory data | |
| CN110134551B (en) | Continuous data protection method and device | |
| CN111221814B (en) | Method, device and equipment for constructing secondary index | |
| CN104850548B (en) | A kind of method and system for realizing big data platform input/output processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140416 Termination date: 20170221 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |