CN102622284A - Data asynchronous replication method directing to mass storage system - Google Patents
Data asynchronous replication method directing to mass storage system Download PDFInfo
- Publication number
- CN102622284A CN102622284A CN2012100396781A CN201210039678A CN102622284A CN 102622284 A CN102622284 A CN 102622284A CN 2012100396781 A CN2012100396781 A CN 2012100396781A CN 201210039678 A CN201210039678 A CN 201210039678A CN 102622284 A CN102622284 A CN 102622284A
- Authority
- CN
- China
- Prior art keywords
- data
- lease
- copy
- server
- data server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a data asynchronous replication method directing to a mass storage system. The method comprises the following steps: step 1, a client inquires data server information containing write lease of a data block from a management server; step 2, the management server carries out inquiry and allocation operation of the write lease; step 3, after receiving the data server information containing the write lease, the client sends a write request to the data server, and after receiving the write request, the data server writes data in a corresponding copy and adds one to the version number of the copy; step 4, the management server takes tasks from an asynchronous replication array periodically, and replication operation is performed to the copy by the data server; and step 5, the data server informs the management server that the task of copy replication is already finished, the management server removes the task from the asynchronous replication array, and the next task from the asynchronous replication array is executed. Compared with the prior art, the method has the advantages of high replication efficiency and the like.
Description
Technical field
The present invention relates to a kind of data copy method, especially relate to a kind of data asynchronous replication method towards mass storage system (MSS).
Background technology
Mass storage system (MSS) is a kind of network store system that can store Large Volume Data.Mass storage system (MSS) often adopts distributed system architecture in order to obtain the performance of bigger capacity and Geng Gao at present.Wherein a kind of typical architecture is a management server, a plurality of data servers, a plurality of client computer.File is divided into the data block of identical size, and data block is kept on the data server.Storing the name space of system on the management server, the metadata of file, the information such as mapping relations of file and data block.It is mutual that client computer and management server carry out metadata, carries out data interaction with data server.
Have a large amount of memory devices in the mass storage system (MSS), therefore can think that it is recurrent situation that memory device breaks down.For the high availability of system, to preserve multiple copies for a data block, and leave on the different data servers as far as possible.When carrying out write operation, owing to will data be write multiple copies, the efficient of write operation becomes a problem.
Through existing literature search is found that " The Google File System (Google file system) " lining has proposed the method that a kind of data sync is duplicated.What Google file system (GFS) adopted is the typical architecture of mass storage system (MSS) mentioned above, and it stores the data block (Chunk) that file is divided into 64MB.Client computer is carried out write operation to a Chunk, must wait until that all copies of this Chunk have all been accomplished after the write operation, just returns the write operation information of successful to client computer.Though this method is conceptive simple, the each write operation of this method all will write all copies immediately, has reduced the efficient of write operation, particularly when copy number is many, can have a strong impact on the write operation execution speed.
Summary of the invention
The object of the invention is exactly to provide a kind of duplicating efficiency the high data asynchronous replication method towards mass storage system (MSS) for the defective that overcomes above-mentioned prior art existence.
The object of the invention can be realized through following technical scheme:
A kind of data asynchronous replication method towards mass storage system (MSS) is characterized in that, may further comprise the steps:
The first step, client computer has the data server information that data block is write lease to management server queries;
In second step, after management server receives query requests, carry out inquiry and the batch operation of writing lease;
The 3rd step; Client computer is write request to its transmission, after data server receives the request of writing after receiving and having the data server information of writing lease; Write data into corresponding copy; And the version number of copy added one, and return the write operation successful information to client computer, add new task to the asynchronous replication formation of management server;
In the 4th step, management server is periodically got task in the asynchronous replication formation, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
In the 5th step, data server notice management server has been accomplished the copy replication task, and management server is deleted this task in the asynchronous replication formation, carry out the next task in the asynchronous replication formation then.
The inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block; There is data server to have the lease of writing of data block if find; To have the data server information of writing lease and return to client computer, and finish to write the inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block; There is data server to have the lease of reading of data block if find; Alternative according to the read-write lease; The notice client computer can't obtain the lease of writing of data block, finishes to write the inquiry and the batch operation of lease, otherwise carries out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, and is a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at version number the highest copy place, and the lease of writing of data block is distributed to this data server, and will have the data server information of writing lease and return to client computer.
The replicate run of the copy in the described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words then carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece; After management server receives request; Check writing lease and whether being held of data block; If words then replication task is joined asynchronous replication formation afterbody, reply the data server application and read the lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and execution in step 43);
43) data server sends read request to the data server at top version number copy place, after the data server at top version number copy place receives request, data and version number is passed to the data server that transmission is asked;
44) data server writes the data that receive in the data block copy of this machine, after data write and finish version number is upgraded to identical with the top version number copy.
Compared with prior art, the present invention has the following advantages:
When the data block that has multiple copies is carried out write operation, only need write success and write success with regard to returning immediately to one of them copy, data asynchronous replication method is adopted in writing of other copies, thereby has improved the efficient of write operation.
Description of drawings
Fig. 1 is the process flow diagram of writing lease inquiry and batch operation in second step of the present invention;
Fig. 2 is the process flow diagram of the copy replication operation in the 4th step of the present invention.
Embodiment
Below in conjunction with accompanying drawing and specific embodiment the present invention is elaborated.
Embodiment 1
A kind of data asynchronous replication method towards mass storage system (MSS) is characterized in that, may further comprise the steps:
The first step, client computer has the data server information that data block is write lease to management server queries;
In second step, after management server receives query requests, carry out inquiry and the batch operation of writing lease;
The 3rd step; Client computer is write request to its transmission, after data server receives the request of writing after receiving and having the data server information of writing lease; Write data into corresponding copy; And the version number of copy added one, and return the write operation successful information to client computer, add new task to the asynchronous replication formation of management server;
In the 4th step, management server is periodically got task in the asynchronous replication formation, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
In the 5th step, data server notice management server has been accomplished the copy replication task, and management server is deleted this task in the asynchronous replication formation, carry out the next task in the asynchronous replication formation then.
As shown in Figure 1, the inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block; There is data server to have the lease of writing of data block if find; To have the data server information of writing lease and return to client computer, and finish to write the inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block; There is data server to have the lease of reading of data block if find; Alternative according to the read-write lease; The notice client computer can't obtain the lease of writing of data block, finishes to write the inquiry and the batch operation of lease, otherwise carries out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, and is a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at version number the highest copy place, and the lease of writing of data block is distributed to this data server, and will have the data server information of writing lease and return to client computer.
As shown in Figure 2, the replicate run of the copy in the described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words then carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece; After management server receives request; Check writing lease and whether being held of data block; If words then replication task is joined asynchronous replication formation afterbody, reply the data server application and read the lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and execution in step 43);
43) data server sends read request to the data server at top version number copy place, after the data server at top version number copy place receives request, data and version number is passed to the data server that transmission is asked;
44) data server writes the data that receive in the data block copy of this machine, after data write and finish version number is upgraded to identical with the top version number copy.
Embodiment 2
Present embodiment comprises: the management server of 1 storing metadata, and the data server of several storage data, several client computer may further comprise the steps:
The first step, client computer had the data server information that CHUNK-A writes lease to management server queries before data block CHUNK-A is carried out write operation.
In second step, management server receives after CHUNK-A writes the query requests of lease, carries out inquiry and the batch operation of writing lease.
Described inquiry and the batch operation of writing lease, concrete steps are:
1) management server is searched the lease of writing that whether has data server to have CHUNK-A; There is data server to have the lease of writing of CHUNK-A if find; This data server is DS-A in the present embodiment; Then the information with DS-A returns to client computer, finishes to write the inquiry and the batch operation of lease, otherwise carries out 2);
2) management server is searched the lease of reading that whether has data server to have CHUNK-A; There is data server to have the lease of reading of CHUNK-A if find; This data server is DS-B in the present embodiment; The lease of writing of then notifying client computer can't obtain CHUNK-A finishes to write the inquiry and the batch operation of lease, otherwise carries out 3);
3) management server is checked the copy information table, searches the highest copy of CHUNK-A version number, and CHUNK-A has 3 parts of copies in the present embodiment; Be respectively CHUNK-A1, CHUNK-A2, CHUNK-A3; Their version number is respectively 10,10, and 9; They are stored in data server DS-C respectively, and on DS-D and the DS-E, so the highest copy of version number is CHUNK-A1;
4) management server is checked the data server at CHUNK-A1 place, and this data server is DS-C in the present embodiment, the lease of writing of CHUNK-A is distributed to DS-C, and the information of DS-C is returned to client computer.
In the 3rd step, after client computer receives the information of DS-C, send the request of writing to CHUNK-A1 to it; DS-C writes CHUNK-A1 with data; The version number of CHUNK-A1 is increased to 11, returns the write operation successful information, add new task to the asynchronous replication formation of management server to client computer.
In the 4th step, management server taking-up task in the asynchronous replication formation is carried out the replicate run of copy, and this task is to duplicate data that DS-C the goes up CHUNK-A1 CHUNK-A2 to the DS-D in the present embodiment.
The replicate run of described copy, concrete steps are following:
1) DS-D checks the lease of reading that oneself whether has CHUNK-A, if words then carry out 3), otherwise carry out 2);
2) DS-D is to the lease of reading of management server application CHUNK-A; After management server receives request; Check writing lease and whether being held of CHUNK-A earlier, if words then replication task is joined asynchronous replication formation afterbody, reply the DS-D application and read the lease failure; Finish the replicate run of copy, otherwise management server will be read lease and distribute to DS-D;
3) DS-D sends read request to DS-C, after DS-C receives request, data and the version number of CHUNK-A1 is passed to DS-D;
4) DS-D writes the data that receive in the CHUNK-A2, after data write and finish the version number of CHUNK-A2 is upgraded to identically with CHUNK-A1, is 11 in the present embodiment.
In the 5th step, DS-D notice management server has been accomplished the data replicate run, and management server is deleted this task in the asynchronous replication formation, carry out the next task in the asynchronous replication formation then.
Claims (3)
1. the data asynchronous replication method towards mass storage system (MSS) is characterized in that, may further comprise the steps:
The first step, client computer has the data server information that data block is write lease to management server queries;
In second step, after management server receives query requests, carry out inquiry and the batch operation of writing lease;
The 3rd step; Client computer is write request to its transmission, after data server receives the request of writing after receiving and having the data server information of writing lease; Write data into corresponding copy; And the version number of copy added one, and return the write operation successful information to client computer, add new task to the asynchronous replication formation of management server;
In the 4th step, management server is periodically got task in the asynchronous replication formation, assigns the task to the data server at the copy place of lowest version number, and data server carries out the replicate run of copy;
In the 5th step, data server notice management server has been accomplished the copy replication task, and management server is deleted this task in the asynchronous replication formation, carry out the next task in the asynchronous replication formation then.
2. a kind of data asynchronous replication method towards mass storage system (MSS) according to claim 1 is characterized in that the inquiry of writing lease and the batch operation of described step 2 are specially:
21) management server is searched the lease of writing that whether has data server to have data block; There is data server to have the lease of writing of data block if find; To have the data server information of writing lease and return to client computer, and finish to write the inquiry and the batch operation of lease, otherwise carry out 22);
22) management server is searched the lease of reading that whether has data server to have data block; There is data server to have the lease of reading of data block if find; Alternative according to the read-write lease; The notice client computer can't obtain the lease of writing of data block, finishes to write the inquiry and the batch operation of lease, otherwise carries out 23);
23) management server is checked the copy information table of data block, searches the highest copy of version number, and is a plurality of if the highest copy of version number has, and just selects wherein any one;
24) management server is checked the data server at version number the highest copy place, and the lease of writing of data block is distributed to this data server, and will have the data server information of writing lease and return to client computer.
3. a kind of data asynchronous replication method towards mass storage system (MSS) according to claim 1 is characterized in that the replicate run of the copy in the described step 4 is specific as follows:
41) data server is checked the lease of reading that oneself whether has data block, if words then carry out 43), otherwise carry out 42);
42) data server is to the lease of reading of management server request for data piece; After management server receives request; Check writing lease and whether being held of data block; If words then replication task is joined asynchronous replication formation afterbody, reply the data server application and read the lease failure, finish the replicate run of this copy; Otherwise management server will be read lease and distribute to data server, and execution in step 43);
43) data server sends read request to the data server at top version number copy place, after the data server at top version number copy place receives request, data and version number is passed to the data server that transmission is asked;
44) data server writes the data that receive in the data block copy of this machine, after data write and finish version number is upgraded to identical with the top version number copy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210039678.1A CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210039678.1A CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102622284A true CN102622284A (en) | 2012-08-01 |
| CN102622284B CN102622284B (en) | 2014-04-16 |
Family
ID=46562211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210039678.1A Expired - Fee Related CN102622284B (en) | 2012-02-21 | 2012-02-21 | Data asynchronous replication method directing to mass storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102622284B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106406770A (en) * | 2016-09-28 | 2017-02-15 | 乐视控股（北京）有限公司 | Data write-in method and system |
| CN106484321A (en) * | 2016-09-08 | 2017-03-08 | 华为数字技术（成都）有限公司 | A kind of date storage method and data center |
| CN107329695A (en) * | 2017-06-22 | 2017-11-07 | 上海七牛信息技术有限公司 | A kind of distributed storage EMS memory management process, system and computer-readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030101321A1 (en) * | 2001-11-29 | 2003-05-29 | Ohran Richard S. | Preserving a snapshot of selected data of a mass storage system |
| CN101911037A (en) * | 2007-12-27 | 2010-12-08 | 微软公司 | Asynchronous replication |
| CN102136003A (en) * | 2011-03-25 | 2011-07-27 | 上海交通大学 | Large-scale distributed storage system |
-
2012
- 2012-02-21 CN CN201210039678.1A patent/CN102622284B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030101321A1 (en) * | 2001-11-29 | 2003-05-29 | Ohran Richard S. | Preserving a snapshot of selected data of a mass storage system |
| CN101911037A (en) * | 2007-12-27 | 2010-12-08 | 微软公司 | Asynchronous replication |
| CN102136003A (en) * | 2011-03-25 | 2011-07-27 | 上海交通大学 | Large-scale distributed storage system |
Non-Patent Citations (2)
| Title |
|---|
| 《微型电脑应用》 20111220 张轶彬 "分布式文件系统KFS的架构与性能分析" 第1～4页 1-3 第27卷, 第12期 * |
| 张轶彬: ""分布式文件系统KFS的架构与性能分析"", 《微型电脑应用》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106484321A (en) * | 2016-09-08 | 2017-03-08 | 华为数字技术（成都）有限公司 | A kind of date storage method and data center |
| WO2018045758A1 (en) * | 2016-09-08 | 2018-03-15 | 华为技术有限公司 | Data storage method and data centre |
| CN106406770A (en) * | 2016-09-28 | 2017-02-15 | 乐视控股（北京）有限公司 | Data write-in method and system |
| CN107329695A (en) * | 2017-06-22 | 2017-11-07 | 上海七牛信息技术有限公司 | A kind of distributed storage EMS memory management process, system and computer-readable storage medium |
| WO2018233332A1 (en) * | 2017-06-22 | 2018-12-27 | 上海七牛信息技术有限公司 | Distributed storage internal storage management method and system, and computer storage medium |
| CN107329695B (en) * | 2017-06-22 | 2020-03-20 | 上海七牛信息技术有限公司 | Distributed storage memory management method, system and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102622284B (en) | 2014-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102136003A (en) | Large-scale distributed storage system | |
| CN107562757B (en) | Query and access method, device and system based on distributed file system | |
| CN105549905A (en) | Method for multiple virtual machines to access distributed object storage system | |
| CN105069048A (en) | Small file storage method, query method and device | |
| CN103647797A (en) | Distributed file system and data access method thereof | |
| KR20100070968A (en) | Cluster data management system and method for data recovery using parallel processing in cluster data management system | |
| EP3076307A1 (en) | Method and device for responding to a request, and distributed file system | |
| CN101997823A (en) | Distributed file system and data access method thereof | |
| CN102932415A (en) | Method and device for storing mirror image document | |
| CN102662992A (en) | Method and device for storing and accessing massive small files | |
| CN102200972A (en) | Database maintenance method and database maintenance equipment | |
| RU2654144C1 (en) | Redo logging for partitioned data set in memory | |
| CN108108476A (en) | The method of work of highly reliable distributed information log system | |
| US11960506B2 (en) | Data processing method and system for cloud platform, and electronic apparatus and storage medium | |
| CN102262512A (en) | System, device and method for realizing disk array cache partition management | |
| CN113220795A (en) | Data processing method, device, equipment and medium based on distributed storage | |
| US20210034462A1 (en) | Simple and efficient technique to support disk extents of different sizes for mapped raid | |
| CN111309245A (en) | Layered storage writing method and device, reading method and device and system | |
| CN102622284B (en) | Data asynchronous replication method directing to mass storage system | |
| CN111638853A (en) | Data storage method and device, storage cluster, gateway equipment and main equipment | |
| CN111913917A (en) | File processing method, device, equipment and medium | |
| CN107493309B (en) | File writing method and device in distributed system | |
| CN102724301A (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN107133334B (en) | Data synchronization method based on high-bandwidth storage system | |
| US20220004664A1 (en) | Data integrity procedure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140416 Termination date: 20170221 |