CN101945131A - Storage virtualization-based data migration method - Google Patents
Storage virtualization-based data migration method Download PDFInfo
- Publication number
- CN101945131A CN101945131A CN 201010282427 CN201010282427A CN101945131A CN 101945131 A CN101945131 A CN 101945131A CN 201010282427 CN201010282427 CN 201010282427 CN 201010282427 A CN201010282427 A CN 201010282427A CN 101945131 A CN101945131 A CN 101945131A
- Authority
- CN
- China
- Prior art keywords
- migration
- data
- throughput
- logical volume
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a storage virtualization-based data migration method, which is realized on the basis of a data migration system of which the storage is virtualized and is a process of migrating the data of a source logical volume to a target logical volume in a client proxy. In the method, the logical volume serves as a migration unit so as to satisfy the rule of performing full writing operation in the target logical volume and then improve the migration efficiency. The migration rate is controlled in the migration process so as to optimally satisfy user requirements preferentially and then effectively guarantee uninterrupted running of services. The method of the invention simplifies the complexity of the method of the prior art and ensures the data consistency of the source logical volume and the target logical volume.
Description
Technical field
The invention belongs to field of data migration, relate in particular to wherein online migration and rate controlled field.This method can be implemented in data migration time, the user is asked transparent, and can effectively control migration rate.
Background technology
The data migration is meant the data message on the source device, move on the target device by hardware structure and the software platform set up, and guarantee all data on the equipment of migration front and back, comprise that any change does not take place for common data, link etc., after migration is finished, just can come into operation so at once.At present, data migration method mainly contains following several method:
(1) logical volume data image method is promptly all done the logical volume data image to each volume of needs migration, as IBM LVM, and Veritas VxVM.This method is applicable to the system that adopts LVM, can utilize the management function of LVM to finish the migration of legacy data to new storage, and support online migration.LVM can be understood as logical storage management level that increase between physical storage device and operating system.The advantage of logical volume data image method mainly contains: support the migration between any storage system and be suitable for the non-recurrent migration of host stores.
(2) direct copy method, the data of promptly utilizing the direct copy of operating system command to move copy to the destination that will move to then, as IBMAIX tar, dd, savevg, mksysb, orders such as cpio.This method is applicable to the environment of off line migration.
(3) back-up restoring method promptly utilizes backup management software that data are backuped, and returns to the destination then, as IBM T SM, and EMC Legato NetWorker, Symantec Veritas NetBackup etc.This method is applicable to for online demanding environment, can return to the destination then in conjunction with the method for online backup.
(4) method of Storage Virtualization is promptly moved to destination with data from the source end by storage virtualization technology, as the SVC of Invista, the IBM of EMC and LSI StoreAge SVM etc.The method of Storage Virtualization is to increase necessary memory virtual equipment in source and destination, by network data is moved to the destination from the source end.The major advantage of this method has: compatible existing mainstream storage device, as: IBM, EMC, HDS, LSI Logic, and support the data between different vendor's different brands memory device to move and disaster tolerance, be suitable for the large enterprise of frequent mobile data.
Aspect the data migration, the business datum of magnanimity is that enterprise brings following challenge:
(1) the data migration causes system down time long, reaches a couple of days sometimes, and operation brings great inconvenience to business event;
(2) data migrations just means to use and rolls off the production line;
(3) there is the data consistency risk in the data migration process;
(4) data volume huge (TB rank) that need move;
This shows that data migrations is a very great engineering to IT department of enterprise, careless slightlyly will cause the loss that is difficult to gain.
Summary of the invention
The object of the present invention is to provide a kind of data migration method based on Storage Virtualization, this method can guarantee finishing smoothly of user's request in migration data, promptly guarantee professional continuity, and can control migration rate effectively, make the user can occupy bigger bandwidth.
A kind of data migration method provided by the invention based on Storage Virtualization, its step comprises:
(1) Client Agent receives the data migration signal that comes from the SRM device;
(2) Client Agent is provided with migration sign, and when there was the migration sign in system, user's write request involved in row to source logical volume and target logic simultaneously and writes, and when there was not the migration sign in system, user's write request was only write the source logical volume;
(3) open source logical volume root, and read first subdirectory item under the root;
(4) judge whether the subdirectory item is empty, if the migration of expression data is all finished, and then forwards step (11) to; Otherwise the migration of expression data is not all finished as yet, then forwards step (5) to;
(5) this migration of mark time started T1, the user throughput B1 of record data migration this migration during this period of time;
(6) data migration process begins, and the data sync of source logical volume in the subdirectory item of target logic volume, and is write down the migration throughput B2 of this migration;
(7) indicate this migration concluding time T2;
(8) obtain this transition process time T=T2-T1, and the value of B1 and B2 calculates overall system throughput B1+B2 when utilizing T2, make that P is the percentage of the shared overall system throughput of user throughput in the T time, then P=B1/ (B1+B2);
(9) carry out migration rate control according to the P value that is calculated:, sleep the W1 time when the P value is in the first estate; When the P value is in second grade, the sleep W2 time; When the P value is in the tertiary gradient, the sleep W3 time, wherein, the first estate is meant that the percentage of the shared overall system throughput of user throughput is more than or equal to 0 and less than 20%, the percentage that second grade is meant the shared overall system throughput of user throughput more than or equal to 20% and the percentage that is meant the shared overall system throughput of user throughput less than 80%, the second grade more than or equal to 80% and less than 100%, W1, W2, W3 represent the high, medium and low sleep threshold that sets in advance respectively;
(10) read next subdirectory item under the root, change step (4) over to;
(11) Client Agent scavenge system migration sign;
(12) data migration process finishes.
The inventive method has been simplified the complexity of art methods, has guaranteed the data consistency of source logical volume and target logic volume, and specifically, the present invention has the following advantages:
(1) the present invention is transfer unit with the logical volume, expires the principle of write operation to satisfy in the target logic volume, thereby has improved transport efficiency.
(2) the present invention carries out migration rate control in transition process, preferentially satisfies user's request with maximum possible, and the business that effectively guaranteed is moved incessantly.
Description of drawings
Fig. 1 is the structural representation of available data migratory system;
User's request process schematic diagram when Fig. 2 moves for data;
Fig. 3 is the migration rate control flow chart.
Embodiment
The inventive method is to realize in the data migratory system based on Storage Virtualization, and as shown in Figure 1, this data migratory system comprises:
(1) Client Agent, major function are to obtain the storing device information that satisfies condition, the various file operations that realize the Virtual File System layer to following SRM device, as open file, operation such as reading and writing of files and carry out the data migration operation;
(2) SRM device, major function are following storage resources reserve pond of storage resources agency tissue, for above-mentioned Client Agent provides memory device;
(3) the storage resources agency is made up of the physical volume of different performance, different capabilities, different availabilities, and major function is to be convenient to Client Agent storage data for Client Agent provides memory device.
In the data migratory system based on Storage Virtualization, Client Agent, SRM device and storage resources agency link together by Ethernet switch, the storage resources agency is divided into quick, medium and slow devices according to the difference of physical volume performance with it.The initiating terminal of data migration is called as the source logical volume, and the migration destination end is called as the target logic volume.
As shown in Figure 2, user's request process mainly is divided into the read and write request process during data migration, owing to have the data migration operation, so have two logical volumes in the Client Agent, and promptly source logical volume and target logic are rolled up.User's read operation is only to the operation of source logical volume, the data above the logical volume of promptly read-only source; User's write operation need carry out simultaneously to two logical volumes, and promptly write operation should be carried out on the logical volume of source, also will carry out on the target logic volume, has guaranteed the upward consistency of data of source logical volume and target logic volume like this.
A kind of method of moving based on the data of Storage Virtualization provided by the invention, this method realize that based on the data migratory system of Storage Virtualization this data migratory system comprises Client Agent, SRM device and storage resources agency.Data migration process is carried out in Client Agent, is the process of the data of source logical volume being moved to the target logic volume, as shown in Figure 3, and its specific implementation such as step (A1)-(A12):
(A1) Client Agent receives the data migration signal that comes from the SRM device;
(A2) Client Agent is provided with the migration sign, and when there was the migration sign in system, user's write request need be write simultaneously to source logical volume and target logic volume, and when there was not the migration sign in system, user's write request only need be write the source logical volume;
(A3) open source logical volume root, and read first subdirectory item under the root;
(A4) judge whether the subdirectory item is empty, if the migration of expression data is all finished, and then forwards step (A11) to; Otherwise the migration of expression data is not all finished as yet, then forwards step (A5) to;
(A5) this migration of mark time started T1 writes down the user throughput B1 of this migration;
(A6) data migration process begins, and the data sync of source logical volume in the subdirectory item of target logic volume, and is write down the migration throughput B2 of this migration;
(A7) indicate this migration concluding time T2;
(A8) obtain this transition process time T=T2-T1.Overall system throughput (being user throughput and migration throughput sum) is B1+B2, makes that P is the percentage of the shared overall system throughput of user throughput in the T time, then P=B1/ (B1+B2);
(A9) carry out migration rate control according to the P value that is calculated, concrete control strategy is: according to the P value percentage of the shared overall system throughput of user throughput is divided into Three Estate L1 (0≤P＜20%), L2 (20%≤P＜80%), L3 (80≤P≤100%), the expression user throughput is low, in, height, three sleep threshold W1 are set simultaneously, W2, W3 is made as 120s usually, 60s, 1s.In the P value is in the L1 grade, illustrate that then user throughput is very low, should control the migration throughput, then sleep the W1 time; When the P value is in the L2 grade, illustrate that user throughput is lower, should control the migration throughput, then sleep the W2 time; In the time of in the P value is in the L3 grade, illustrate that user throughput is higher, do not need control migration throughput, then sleep the W3 time.
(A10) read next subdirectory item under the root, change step (A4) over to;
(A11) Client Agent scavenge system migration sign;
(A12) data migration process finishes.
The above is preferred embodiment of the present invention, but the present invention should not be confined to the disclosed content of this embodiment and accompanying drawing.So everyly do not break away from the equivalence of finishing under the spirit disclosed in this invention or revise, all fall into the scope of protection of the invention.
Claims (1)
1. data migration method based on Storage Virtualization, its step comprises:
(1) Client Agent receives the data migration signal that comes from the SRM device;
(2) Client Agent is provided with migration sign, and when there was the migration sign in system, user's write request involved in row to source logical volume and target logic simultaneously and writes, and when there was not the migration sign in system, user's write request was only write the source logical volume;
(3) open source logical volume root, and read first subdirectory item under the root;
(4) judge whether the subdirectory item is empty, if the migration of expression data is all finished, and then forwards step (11) to; Otherwise the migration of expression data is not all finished as yet, then forwards step (5) to;
(5) this migration of mark time started T1 writes down the user throughput B1 of this migration;
(6) data migration process begins, and the data sync of source logical volume in the subdirectory item of target logic volume, and is write down the migration throughput B2 of this migration;
(7) indicate this migration concluding time T2;
(8) obtain this transition process time T=T2-T1, and the value of B1 and B2 calculates overall system throughput B1+B2 when utilizing T2, make that P is the percentage of the shared overall system throughput of user throughput in the T time, then P=B1/ (B1+B2);
(9) carry out migration rate control according to the P value that is calculated:, sleep the W1 time when the P value is in the first estate; When the P value is in second grade, the sleep W2 time; When the P value is in the tertiary gradient, the sleep W3 time, wherein, the first estate is meant that the percentage of the shared overall system throughput of user throughput is more than or equal to 0 and less than 20%, the percentage that second grade is meant the shared overall system throughput of user throughput more than or equal to 20% and the percentage that is meant the shared overall system throughput of user throughput less than 80%, the second grade more than or equal to 80% and less than 100%, W1, W2, W3 represent the high, medium and low sleep threshold that sets in advance respectively;
(10) read next subdirectory item under the root, change step (4) over to;
(11) Client Agent scavenge system migration sign;
(12) data migration process finishes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102824277A CN101945131B (en) | 2010-09-16 | 2010-09-16 | Storage virtualization-based data migration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102824277A CN101945131B (en) | 2010-09-16 | 2010-09-16 | Storage virtualization-based data migration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101945131A true CN101945131A (en) | 2011-01-12 |
| CN101945131B CN101945131B (en) | 2012-08-15 |
Family
ID=43436898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010102824277A Active CN101945131B (en) | 2010-09-16 | 2010-09-16 | Storage virtualization-based data migration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101945131B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103152407A (en) * | 2013-02-20 | 2013-06-12 | 浪潮电子信息产业股份有限公司 | Method for designing integral data disaster recovery backup system |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
| CN105549904A (en) * | 2015-12-08 | 2016-05-04 | 华为技术有限公司 | Data migration method applied in storage system and storage devices |
| CN106649335A (en) * | 2015-10-30 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN107193489A (en) * | 2017-05-15 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of storage level data migration method and device based on storage virtual gateway |
| CN110764699A (en) * | 2019-09-29 | 2020-02-07 | 苏州浪潮智能科技有限公司 | Online migration method and device for storage volume |
| CN111078121A (en) * | 2018-10-18 | 2020-04-28 | 深信服科技股份有限公司 | Data migration method, system and related components of distributed storage system |
| CN111913663A (en) * | 2020-07-29 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Storage volume online migration method and device and storage volume online migration system |
| CN111930707A (en) * | 2020-07-10 | 2020-11-13 | 江苏安超云软件有限公司 | Method and system for correcting drive letter of windows cloud migration |
| CN112685391A (en) * | 2021-03-22 | 2021-04-20 | 腾讯科技（深圳）有限公司 | Service data migration method and device, computer equipment and storage medium |
| CN114398005A (en) * | 2021-12-22 | 2022-04-26 | 上海金仕达软件科技有限公司 | Data secure migration method and device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1877540A (en) * | 2005-06-10 | 2006-12-13 | 北京艾德斯科技有限公司 | Snapshot system for network storage and method therefor |
| CN101464812A (en) * | 2009-01-06 | 2009-06-24 | 北京航空航天大学 | Virtual machine migration method |
-
2010
- 2010-09-16 CN CN2010102824277A patent/CN101945131B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1877540A (en) * | 2005-06-10 | 2006-12-13 | 北京艾德斯科技有限公司 | Snapshot system for network storage and method therefor |
| CN101464812A (en) * | 2009-01-06 | 2009-06-24 | 北京航空航天大学 | Virtual machine migration method |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103152407A (en) * | 2013-02-20 | 2013-06-12 | 浪潮电子信息产业股份有限公司 | Method for designing integral data disaster recovery backup system |
| CN105530273B (en) * | 2014-09-29 | 2018-08-03 | 国际商业机器公司 | method and apparatus for data migration |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
| CN106649335A (en) * | 2015-10-30 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN105549904B (en) * | 2015-12-08 | 2019-01-08 | 华为技术有限公司 | A kind of data migration method and storage equipment applied in storage system |
| CN105549904A (en) * | 2015-12-08 | 2016-05-04 | 华为技术有限公司 | Data migration method applied in storage system and storage devices |
| CN107193489A (en) * | 2017-05-15 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of storage level data migration method and device based on storage virtual gateway |
| CN111078121A (en) * | 2018-10-18 | 2020-04-28 | 深信服科技股份有限公司 | Data migration method, system and related components of distributed storage system |
| CN110764699A (en) * | 2019-09-29 | 2020-02-07 | 苏州浪潮智能科技有限公司 | Online migration method and device for storage volume |
| CN111930707A (en) * | 2020-07-10 | 2020-11-13 | 江苏安超云软件有限公司 | Method and system for correcting drive letter of windows cloud migration |
| CN111930707B (en) * | 2020-07-10 | 2022-08-02 | 江苏安超云软件有限公司 | Method and system for correcting drive letter of windows cloud migration |
| CN111913663A (en) * | 2020-07-29 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Storage volume online migration method and device and storage volume online migration system |
| CN112685391A (en) * | 2021-03-22 | 2021-04-20 | 腾讯科技（深圳）有限公司 | Service data migration method and device, computer equipment and storage medium |
| CN112685391B (en) * | 2021-03-22 | 2021-06-08 | 腾讯科技（深圳）有限公司 | Service data migration method and device, computer equipment and storage medium |
| CN114398005A (en) * | 2021-12-22 | 2022-04-26 | 上海金仕达软件科技有限公司 | Data secure migration method and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101945131B (en) | 2012-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101945131B (en) | Storage virtualization-based data migration method | |
| US9823877B2 (en) | Virtual machine backup from storage snapshot | |
| US11397648B2 (en) | Virtual machine recovery method and virtual machine management device | |
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| US9043637B2 (en) | Failure recovery method in information processing system and information processing system | |
| CN106062742B (en) | System and method for improving snapshot performance | |
| CN102713856B (en) | Method and system for recovering file system of computer system | |
| US20130024722A1 (en) | Virtual disk replication using log files | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| CN102402471A (en) | Method and system for backing up data in real time based on snapshot function of memory array | |
| CN104239166A (en) | Method for realizing file backup for running virtual machine | |
| CN104375877A (en) | Virtual machine online snapshot generating and deleting method | |
| CN103793258A (en) | Hypervisor-based server duplication system and method | |
| US20190012238A1 (en) | Integrating resources at a backup site | |
| CN103019804B (en) | The virtualized VPS quick migration method of OpenVZ | |
| WO2022021856A1 (en) | Method and apparatus for online migration of multi-disk virtual machine into different storage pools | |
| CN106104515A (en) | Utilize File system design and the fault recovery method of nonvolatile memory | |
| US9785641B2 (en) | Reducing a backup time of a backup of data files | |
| CN110058813A (en) | It is a kind of that management method is locally stored based on the storage of cloud platform block | |
| CN104583966A (en) | Backup and restore system for a deduplicated file system and corresponding server and method | |
| CN111367856A (en) | Data copying method and device, electronic equipment and machine-readable storage medium | |
| CN104268097A (en) | Metadata processing method and system | |
| CN109062516A (en) | Invalid data method for removing in a kind of windows virtual machine backup procedure | |
| CN103440179B (en) | The continuous store method of storage device configurations information | |
| CN103064762B (en) | Heavily delete restoration methods and the device of Backup Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |