CN102103629A - Online data migration method - Google Patents
Online data migration method Download PDFInfo
- Publication number
- CN102103629A CN102103629A CN 201010590698 CN201010590698A CN102103629A CN 102103629 A CN102103629 A CN 102103629A CN 201010590698 CN201010590698 CN 201010590698 CN 201010590698 A CN201010590698 A CN 201010590698A CN 102103629 A CN102103629 A CN 102103629A
- Authority
- CN
- China
- Prior art keywords
- data
- disk
- distributor
- migration
- new disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an online data migration method, which is used for solving the technical problems of low efficiency and low universality of the conventional data migration method. The technical scheme comprises the following steps of: copying data in a logic unit number (LUN) of a source logic unit into the LUN of a target logic unit, and continuously running a system and a service in a data migration process; simultaneously copying the data updated into the LUN of the source logic unit into the LUN of the target logic unit in the migration process; simultaneously reflecting updating and copying processes by using a single bitmap mapping table; and after the data migration process is finished, using and updating used equipment at any time, and deleting the used equipment. The system and the service are continuously run in the data migration process, so the data migration efficiency and the data migration universality are improved.
Description
Technical field
The present invention relates to a kind of data migration method, particularly a kind of online data moving method.
Background technology
With reference to Fig. 3.Data migtation is along with the company information amount increases, a job of carrying out in the time of need upgrading to the existing information system.Existing data migration method mainly contains: the off-line data migration, and based on the data migtation of XML.Document " research and the design of the conventional data Migration tools in the infosystem, computing machine and modernization, 2010 the 6th phases, total the 178th phase " discloses a kind of method of carrying out data-switching and migration based on XML between heterogeneous database.This method is moved to after the conversion in the new database based on the data layout that XML analyzes source database.
But there are the following problems at least in the prior art: 1. source database can not upgrade operation in data-switching and the transition process, can't not begin normal business before data have migration not finish, and needs to shut down the long period, and data migtation is big to service impact.2. only can be applicable to the data migtation of certain database, can't handle that versatility is relatively poor to the data that other non-databases are preserved.
Summary of the invention
Existing data migration method efficient is low in order to overcome, the deficiency of versatility difference, the invention provides a kind of online data moving method.This method is ignored data content and form, and the replicating original data block can be carried out data migtation to database and different file system.This method copies to the data in the source disk in the target disk, system and professional operation continuously in the data migration process; Be copied in the target disk simultaneously to the source disk data updated in the transition process; Use single bitmap mapping table to reflect the process of upgrading and duplicating simultaneously; After data migration process is finished, at any time used equipment is used and upgrade, and the deletion used equipment.Because system and business run without interruption in the data migration process, can improve efficiency of data migration, improve the versatility of data migtation.
The technical scheme that the present invention solves its technical matters employing is that a kind of online data moving method is characterized in may further comprise the steps:
(a) close service server, new disk array is connected to switch.
(b) access rights of new disk array are set, make professional function identification, use the logic magnetic disc of appointment on the new disk array.
(c) start professional machine, the multi-route management software starting system of laying equal stress on is installed thereon.
(d) multi-route management software is set, simultaneously with old disk O, new disk N is mapped as unity logic disk L and continues operation for service server.
(e) the migration management control program sends the migration enabled instruction to multi-route management driving and distributor.
Distributor is all clear 0 with all bit in the bitmap table, is set to not transition state of data.The state that data are duplicated in the bitmap table record logic magnetic disc, use 1bit represents 512 bytes in the logic magnetic disc.
Distributor foundation is connected with new disk.
(f) drive as multi-route management and receive and then from old disk, read the read operation scsi command.
Data migtation: drive as multi-route management and to receive the write operation scsi command, then write to old disk and new disk respectively, as the data that write surpass 512 bytes, then the corresponding bit in the updated space chart is 1.
Data migtation: distributor is according to the record in the bitmap table, is 0 as the value of corresponding bit in the bitmap table, then sends the old disk of read request, and the 512bit data block of reading in write corresponding bit is 1 in new disk and the updated space chart.
Repeating step (f), all bit are 1 in bitmap table, represent that promptly all data have migrated to new disk.
Distributor informing removal supervisor control program data migtation is finished.
(g) migration management control program prompting migration of subscriber data is finished, and after the user confirms, sends data migtation to distributor and multi-route management driver and finishes order.
Distributor withdraws from.
The multi-route management driver is directed to new disk with all read-write requests.
The invention has the beneficial effects as follows: the present invention copies to the data in the source disk in the target disk, system and professional operation continuously in the data migration process; Be copied in the target disk simultaneously to the source disk data updated in the transition process; Use single bitmap mapping table to reflect the process of upgrading and duplicating simultaneously; After data migration process is finished, at any time used equipment is used and upgrade, and the deletion used equipment.Because system and business run without interruption in the data migration process, have improved the versatility of efficiency of data migration and data migtation.
Below in conjunction with drawings and Examples the present invention is elaborated.
Description of drawings
Fig. 1 is an online data moving method rig-site utilization layout of the present invention.
Fig. 2 is an online data moving method module structure drafting of the present invention.
Fig. 3 is a background technology data migration method module structure drafting.
Embodiment
With reference to Fig. 1～2.
The inventive method writes down by multi-route management read-write requests and bitmap table and the control data transition process makes in the data migration process, and source disk still can normally be read and write, and reduces the shutdown maintenance time of system.
The data migtation management system that realizes the method for the invention is installed on service server; Service server connects new and old disk array simultaneously by optical fiber switch, visits the logic magnetic disc on it simultaneously.
The data migtation system that realizes this method comprises the multi-route management driver, distributor, 4 parts of bitmap table and migration management control program.
The multi-route management driver is mapped as a logic magnetic disc L with new disk N and old disk O, upwards offers operating system.When operating system is carried out write operation to logic magnetic disc L, this driver will carry out write operation to new disk N and old disk O respectively, and the updated space chart.
Bitmap table is a region of memory that can be had access to simultaneously by multi-route management program and distributor, uses 1bit represent 512 bytes in the logic magnetic disc, and data block contents corresponding in the new and old disk of 0 expression is inconsistent, and 1 represents data consistent.All bit are 0 under the bitmap table original state, drive or distributor when the block copy on the new and old disk is become consistent state as multi-route management, and the bit of correspondence is revised as 1.As bit all in the bitmap table is to represent that data migtation finished at 1 o'clock.Bitmap table has been represented the process of data migtation, and is the distributor reading of data and the foundation of moving.
The data consistent state in the bitmap table is read in distributor circulation, is 0 as the bit of correspondence, then reads data block corresponding on the source disk, writes in the new disk, and bit corresponding in the bitmap table is revised as 1.When detecting all bit and be revised as 1, end loop and informing removal supervisor control program.
The migration management control program is controlled the beginning and the end of whole data migration process.After data sync is finished, prompting user end data transition process.
Data migration process can be divided into system's connection and setting and two stages of data migtation, and step and method are as follows:
Connect and step be set:
1. close service server.
2. dispensing device and new disk array are connected to switch.
3. the access rights of new disk array are set, make professional machine and data delivery device can discern, use the logic magnetic disc of appointment on it.
4. start professional machine, the multi-route management software starting system of laying equal stress on is installed thereon.
5. multi-route management software is set, and simultaneously with old disk O, new disk N is mapped as unity logic disk L and continues operation for service server.
Connect and after setting finishes, can pass through migration management control program log-on data transition process in system, concrete steps and method are as follows:
Data migration method and step:
This method is divided into to migration starts, and data migtation and migration finish 3 stages, in different phase, use different method of data flow controls, and under application program and the impregnable prerequisite of operating system, data are carried out online migration.
1. migration starts: the migration management control program sends the migration enabled instruction to multi-route management driving and distributor.
2. migration starts: distributor is all clear 0 with all bit in the bitmap table, is set to not transition state of data.The state that data are duplicated in the bitmap table record logic magnetic disc, use 1bit represents 512 bytes in the logic magnetic disc.
3. migration starts: distributor foundation is connected with new disk.
4. data migtation: drive as multi-route management and to receive and then from old disk, read the read operation scsi command.
5. data migtation: drive as multi-route management and to receive the write operation scsi command, then write to old disk and new disk respectively, as the data that write surpass 512 bytes, then the corresponding bit in the updated space chart is 1.
6. data migtation: distributor is according to the record in the bitmap table, is 0 as the value of corresponding bit in the bitmap table, then sends the old disk of read request, and the 512bit data block of reading in write corresponding bit is 1 in new disk and the updated space chart.
7. repeating step 4～6, and all bit are 1 in bitmap table, represent that promptly all data have migrated to new disk.
8. data migtation: distributor informing removal supervisor control program data migtation is finished.
9. migration is finished: migration management control program prompting migration of subscriber data is finished, and after the user confirms, sends data migtation to distributor and multi-route management driver and finishes order.
10. migration is finished: distributor withdraws from.
11. migration is finished: the multi-route management driver is directed to new disk with all read-write requests.
Through the above description of the embodiments, the technology people of this area can be well understood to the present invention and can realize by the mode that software adds essential general hardware platform, can certainly realize by hardware.
Claims (1)
1. online data moving method is characterized in that may further comprise the steps:
(a) close service server, new disk array is connected to switch;
(b) access rights of new disk array are set, make professional function identification, use the logic magnetic disc of appointment on the new disk array;
(c) start professional machine, the multi-route management software starting system of laying equal stress on is installed thereon;
(d) multi-route management software is set, simultaneously with old disk O, new disk N is mapped as unity logic disk L and continues operation for service server;
(e) the migration management control program sends the migration enabled instruction to multi-route management driving and distributor;
Distributor is all clear 0 with all bit in the bitmap table, is set to not transition state of data; The state that data are duplicated in the bitmap table record logic magnetic disc, use 1bit represents 512 bytes in the logic magnetic disc;
Distributor foundation is connected with new disk;
(f) drive as multi-route management and receive and then from old disk, read the read operation scsi command;
Data migtation: drive as multi-route management and to receive the write operation scsi command, then write to old disk and new disk respectively, as the data that write surpass 512 bytes, then the corresponding bit in the updated space chart is 1;
Data migtation: distributor is according to the record in the bitmap table, is 0 as the value of corresponding bit in the bitmap table, then sends the old disk of read request, and the 512bit data block of reading in write corresponding bit is 1 in new disk and the updated space chart;
Repeating step (f), all bit are 1 in bitmap table, represent that promptly all data have migrated to new disk;
Distributor informing removal supervisor control program data migtation is finished;
(g) migration management control program prompting migration of subscriber data is finished, and after the user confirms, sends data migtation to distributor and multi-route management driver and finishes order;
Distributor withdraws from;
The multi-route management driver is directed to new disk with all read-write requests.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590698A CN102103629B (en) | 2010-12-14 | 2010-12-14 | Online data migration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590698A CN102103629B (en) | 2010-12-14 | 2010-12-14 | Online data migration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102103629A true CN102103629A (en) | 2011-06-22 |
| CN102103629B CN102103629B (en) | 2012-09-26 |
Family
ID=44156403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010590698A Expired - Fee Related CN102103629B (en) | 2010-12-14 | 2010-12-14 | Online data migration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102103629B (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917072A (en) * | 2012-10-31 | 2013-02-06 | 北京奇虎科技有限公司 | Device, system and method for carrying out data migration between data server clusters |
| CN103605617A (en) * | 2013-11-21 | 2014-02-26 | 华为技术有限公司 | Method and device for recording changes in stored data |
| CN103902233A (en) * | 2012-12-27 | 2014-07-02 | 中国电信股份有限公司 | Virtual machine system on-line storage migration method and device and server |
| WO2015089798A1 (en) * | 2013-12-19 | 2015-06-25 | 华为技术有限公司 | Data migration method and device |
| CN105159908A (en) * | 2015-07-02 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Database cloning method and apparatus |
| CN105607873A (en) * | 2015-12-23 | 2016-05-25 | 浪潮电子信息产业股份有限公司 | Method for carrying out LUN capacity expansion by using data migration |
| CN105760109A (en) * | 2014-12-16 | 2016-07-13 | 华为技术有限公司 | Data migration method and storage array |
| CN106294471A (en) * | 2015-06-03 | 2017-01-04 | 中兴通讯股份有限公司 | Data Migration processing method and processing device |
| CN106371950A (en) * | 2015-07-20 | 2017-02-01 | 中兴通讯股份有限公司 | Method and device for achieving RAID level conversion |
| CN106383667A (en) * | 2016-09-18 | 2017-02-08 | 华为数字技术（成都）有限公司 | Method and apparatus for LUN migration in business operation |
| CN106446111A (en) * | 2016-09-14 | 2017-02-22 | 广东欧珀移动通信有限公司 | Data migration method and terminal |
| CN106534226A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Processing method and apparatus for data migration |
| CN106598981A (en) * | 2015-10-15 | 2017-04-26 | 中国移动通信集团四川有限公司 | Data migration method and device |
| CN106777345A (en) * | 2017-01-16 | 2017-05-31 | 山东浪潮商用系统有限公司 | A kind of data pick-up loading method based on mass data migration |
| CN107391192A (en) * | 2017-07-21 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of database server upgrade method and device |
| CN107679200A (en) * | 2017-10-11 | 2018-02-09 | 郑州云海信息技术有限公司 | A kind of online data moving method and device |
| CN107704206A (en) * | 2017-10-09 | 2018-02-16 | 郑州云海信息技术有限公司 | Method, apparatus, equipment and the storage medium of online migration heterogeneous system data |
| US10210232B2 (en) | 2014-10-01 | 2019-02-19 | SkyKick, Inc. | Automated staged data migration |
| CN110568995A (en) * | 2019-08-30 | 2019-12-13 | 珠海格力电器股份有限公司 | Data migration method and system |
| CN110688056A (en) * | 2018-07-05 | 2020-01-14 | 北京忆恒创源科技有限公司 | Storage medium replacement for NVM groups |
| CN111078121A (en) * | 2018-10-18 | 2020-04-28 | 深信服科技股份有限公司 | Data migration method, system and related components of distributed storage system |
| US11593325B2 (en) | 2018-10-26 | 2023-02-28 | Tata Consultancy Services Limited | Systems and methods of data migration in multi-layer model-driven applications |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100076933A1 (en) * | 2008-09-11 | 2010-03-25 | Microsoft Corporation | Techniques for resource location and migration across data centers |
| US20100115131A1 (en) * | 2008-11-03 | 2010-05-06 | International Business Machines Corporation | Maintaining Storage Area Network ('SAN') Access Rights During Migration Of Operating Systems |
| CN101789021A (en) * | 2010-02-24 | 2010-07-28 | 浪潮通信信息系统有限公司 | Universal configurable database data migration method |
| CN101887456A (en) * | 2010-06-30 | 2010-11-17 | 百视通网络电视技术发展有限责任公司 | System and method for migrating data |
-
2010
- 2010-12-14 CN CN201010590698A patent/CN102103629B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100076933A1 (en) * | 2008-09-11 | 2010-03-25 | Microsoft Corporation | Techniques for resource location and migration across data centers |
| US20100115131A1 (en) * | 2008-11-03 | 2010-05-06 | International Business Machines Corporation | Maintaining Storage Area Network ('SAN') Access Rights During Migration Of Operating Systems |
| CN101789021A (en) * | 2010-02-24 | 2010-07-28 | 浪潮通信信息系统有限公司 | Universal configurable database data migration method |
| CN101887456A (en) * | 2010-06-30 | 2010-11-17 | 百视通网络电视技术发展有限责任公司 | System and method for migrating data |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917072B (en) * | 2012-10-31 | 2016-07-13 | 北京奇虎科技有限公司 | For carrying out the equipment of Data Migration, system and method between data server cluster |
| CN102917072A (en) * | 2012-10-31 | 2013-02-06 | 北京奇虎科技有限公司 | Device, system and method for carrying out data migration between data server clusters |
| CN103902233A (en) * | 2012-12-27 | 2014-07-02 | 中国电信股份有限公司 | Virtual machine system on-line storage migration method and device and server |
| CN103902233B (en) * | 2012-12-27 | 2018-09-11 | 中国电信股份有限公司 | On-line storage moving method, device and the server of virtual machine system |
| CN103605617B (en) * | 2013-11-21 | 2017-08-18 | 华为技术有限公司 | The method and device that a kind of records store data changes |
| CN103605617A (en) * | 2013-11-21 | 2014-02-26 | 华为技术有限公司 | Method and device for recording changes in stored data |
| WO2015089798A1 (en) * | 2013-12-19 | 2015-06-25 | 华为技术有限公司 | Data migration method and device |
| US10210232B2 (en) | 2014-10-01 | 2019-02-19 | SkyKick, Inc. | Automated staged data migration |
| US10733207B2 (en) | 2014-10-01 | 2020-08-04 | SkyKick, Inc. | Automated staged data migration |
| CN105760109A (en) * | 2014-12-16 | 2016-07-13 | 华为技术有限公司 | Data migration method and storage array |
| CN105760109B (en) * | 2014-12-16 | 2019-03-01 | 华为技术有限公司 | Data migration method and storage array |
| CN106294471A (en) * | 2015-06-03 | 2017-01-04 | 中兴通讯股份有限公司 | Data Migration processing method and processing device |
| CN105159908A (en) * | 2015-07-02 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Database cloning method and apparatus |
| CN106371950B (en) * | 2015-07-20 | 2020-11-03 | 中兴通讯股份有限公司 | Method and device for realizing RAID level conversion |
| CN106371950A (en) * | 2015-07-20 | 2017-02-01 | 中兴通讯股份有限公司 | Method and device for achieving RAID level conversion |
| CN106534226A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Processing method and apparatus for data migration |
| CN106598981A (en) * | 2015-10-15 | 2017-04-26 | 中国移动通信集团四川有限公司 | Data migration method and device |
| CN105607873A (en) * | 2015-12-23 | 2016-05-25 | 浪潮电子信息产业股份有限公司 | Method for carrying out LUN capacity expansion by using data migration |
| CN106446111B (en) * | 2016-09-14 | 2020-01-14 | Oppo广东移动通信有限公司 | Data migration method and terminal |
| CN106446111A (en) * | 2016-09-14 | 2017-02-22 | 广东欧珀移动通信有限公司 | Data migration method and terminal |
| CN106383667A (en) * | 2016-09-18 | 2017-02-08 | 华为数字技术（成都）有限公司 | Method and apparatus for LUN migration in business operation |
| CN106383667B (en) * | 2016-09-18 | 2019-06-21 | 华为数字技术（成都）有限公司 | The method and apparatus that LUN is migrated in service operation |
| CN106777345A (en) * | 2017-01-16 | 2017-05-31 | 山东浪潮商用系统有限公司 | A kind of data pick-up loading method based on mass data migration |
| CN106777345B (en) * | 2017-01-16 | 2020-07-28 | 浪潮软件科技有限公司 | Data extraction loading method based on mass data migration |
| CN107391192A (en) * | 2017-07-21 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of database server upgrade method and device |
| CN107704206A (en) * | 2017-10-09 | 2018-02-16 | 郑州云海信息技术有限公司 | Method, apparatus, equipment and the storage medium of online migration heterogeneous system data |
| CN107704206B (en) * | 2017-10-09 | 2020-09-18 | 苏州浪潮智能科技有限公司 | Method, device, equipment and storage medium for online migration of heterogeneous system data |
| CN107679200A (en) * | 2017-10-11 | 2018-02-09 | 郑州云海信息技术有限公司 | A kind of online data moving method and device |
| CN110688056A (en) * | 2018-07-05 | 2020-01-14 | 北京忆恒创源科技有限公司 | Storage medium replacement for NVM groups |
| CN111078121A (en) * | 2018-10-18 | 2020-04-28 | 深信服科技股份有限公司 | Data migration method, system and related components of distributed storage system |
| US11593325B2 (en) | 2018-10-26 | 2023-02-28 | Tata Consultancy Services Limited | Systems and methods of data migration in multi-layer model-driven applications |
| CN110568995A (en) * | 2019-08-30 | 2019-12-13 | 珠海格力电器股份有限公司 | Data migration method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102103629B (en) | 2012-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102103629B (en) | Online data migration method | |
| US7991758B2 (en) | System and method for performing a search operation within a sequential access data storage subsystem | |
| US8762672B2 (en) | Storage system and storage migration method | |
| CN102057355B (en) | Synchronizing virtual machine and application life cycles | |
| CN101452407B (en) | System and method for preventing user o.s. in vmm system from deenergizing device being used by service o.s. | |
| US8392670B2 (en) | Performance management of access to flash memory in a storage device | |
| KR100317691B1 (en) | Efficient volume copy using pre-configuration of log structured target storage | |
| CN104360892B (en) | Create the system and method for virtual machine | |
| JP2008203937A (en) | Computer system, storage management server, and data migration method | |
| CN101739307A (en) | Embedded integrated virtual tape library-based data archiving protection system and method | |
| CN101840362A (en) | Method and device for achieving copy-on-write snapshot | |
| CN101916172B (en) | Virtual disk image logic partitioning-based on-demand software distribution method | |
| KR20110097609A (en) | Capturing and naming dynamic storage tiering configurations to support data pre-staging | |
| CN103885719A (en) | Device, system and method for online storage migration of virtual machine system | |
| CN103902233A (en) | Virtual machine system on-line storage migration method and device and server | |
| JP2002132554A (en) | Access method for database | |
| JP5074676B2 (en) | Device controller | |
| CN101034363A (en) | Data backup device, data backup method, and recording medium storing data backup program | |
| US20100070706A1 (en) | Method of Allocating Physical Volume Area to Virtualized Volume, and Storage Device | |
| US7099995B2 (en) | Metadata access during error handling routines | |
| CN100585558C (en) | Method and device for providing and recovering data | |
| JPS58151660A (en) | Magnetic disk control device | |
| CN101814043B (en) | Method for performing on-line recovery by using snapshot technology | |
| CN101650679A (en) | Efficient snapshot technology based on disk IO read-write change | |
| CN100419681C (en) | Novel network duplication system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120926 Termination date: 20161214 |