WO2016037777A1 - Data migration tool with intermediate incremental copies - Google Patents

Data migration tool with intermediate incremental copies Download PDF

Info

Publication number
WO2016037777A1
WO2016037777A1 PCT/EP2015/068227 EP2015068227W WO2016037777A1 WO 2016037777 A1 WO2016037777 A1 WO 2016037777A1 EP 2015068227 W EP2015068227 W EP 2015068227W WO 2016037777 A1 WO2016037777 A1 WO 2016037777A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
copy
incremental
cutover
storage system
Prior art date
Application number
PCT/EP2015/068227
Other languages
French (fr)
Inventor
Ives Aerts
Kim Marivoet
Original Assignee
Datadobi Cvba
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Datadobi Cvba filed Critical Datadobi Cvba
Priority to CA2959647A priority Critical patent/CA2959647C/en
Priority to US15/510,154 priority patent/US10114579B2/en
Publication of WO2016037777A1 publication Critical patent/WO2016037777A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0617Improving the reliability of storage systems in relation to availability
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the invention relates to the field of data migration tools. Such tools aid in the automated migration of digital data from a source storage system to a destination storage system. [02] More particular, the invention relates to the migration of huge amounts of data where a single copy of all data on the source storage system to the destination storage system may last in the order of days, weeks or even months.
  • a storage system user may decide to migrate his data from its current storage system to a new one. This decision may be driven by several factors. A first factor may be financial considerations, where the new storage system provider offers the same or more capacity for a better price. Another factor may be that the capacity of the current storage system can no longer be increased and a migration to a new and larger storage system is inevitable. [05] In all of these cases, a data migration is to be performed, i.e., all data on the source system needs to be copied to the destination system and, at some point in time, users need to be switched to the new destination system. During the actual switch or cutover, the users are typically denied access to both storage systems in order to ensure data integrity. This way users cannot write to data that is being copied which could cause data corruption or users cannot write to a data location that has already passed the migration which could cause data loss.
  • WO12149220A discloses the concept of incremental copies to shorten the actual cutover time. First, an initial or baseline copy is made of all data to be migrated from the source to the destination system. Then, one or more incremental copies are made before the actual switchover. An incremental copy only considers the differences between the source and destination system. It thus applies all changes from the users that are still using the source storage system.
  • the users are still allowed access to their data on the source storage system. Then, at a certain planned point in time, the actual cutover is performed. During the cutover, the users are denied all access from the storage systems and a last or cutover incremental copy is made. When the cutover copy is done, the users are switched to the new destination storage system and can again access their data.
  • This object is achieved by a computer implemented method for migrating data from a source storage system to a destination storage system comprising the step of performing an initial copy, subsequently performing one or more incremental copies and subsequently performing a final cutover incremental copy.
  • This performing one or more incremental copies further comprises excluding from a respective one of the one or more incremental copies first data portions of the data that are likely to change before this performing a final cutover incremental copy.
  • one or more incremental copies are made. In an incremental copy, the differences in data between the source storage system and the destination storage system are applied to the destination storage system.
  • An incremental copy may comprise the copying of a data portion from the source to the destination, a deletion of a data portion in the destination or an update of a data portion in the destination.
  • the data comprises a plurality of data portions, which are defined as units of data that are copied from the source to the destination system.
  • a data portion is a file residing hierarchically structured in a file system and the data is thus copied on a file by file basis.
  • the actual cutover is performed and a last incremental copy referred to as the final cutover incremental copy is made.
  • users may start using the destination storage system. Typically, users are denied access to both storage systems during the cutover to ensure data integrity.
  • the performing an initial copy may further comprise excluding second data portions of the data from the initial copy that are likely to change before the performing the final cutover incremental copy.
  • the final cutover incremental copy is performed when a transfer size of the one or more incremental copies has reached a steady state.
  • the incremental copies will typically decrease in transfer size and transfer time till they reach a certain steady state in transfer size, i.e., up to a moment where the transfer size and/or time of subsequent incremental copies are substantially the same.
  • the transfer size of this first initial copy may thus be very large taking days to months to execute for large storage systems.
  • the transfer size of an incremental copy depends on the duration since the previous copy. As the transfer time of the initial copy is so large, the first incremental copy will still be considerably large. Every subsequent incremental copy will then decrease in size until a certain steady state in transfer size and time.
  • the excluding of first and/or second data comprises retrieving metadata associated with data portions of the data on the source storage system wherein the metadata is indicative for a likelihood that a respective data portion will change before the final cutover incremental copy.
  • the excluding further comprises selecting the first and/or second data portions based on this metadata.
  • Metadata is available information about the data portions that provide an indication if the data portion is likely to change before the cutover.
  • a first type of metadata may be available from the source storage system itself. One example of this first type is the type of a respective data portion or a file type if the data portion corresponds to a file. A predetermined list of file types may then be used to decide whether a data portion is likely to change. For example, .pst files which are typically used for storing an email database may be excluded from the incremental copies
  • a second example of the first type of metadata is a change history of a data portion or a file. Files that are changed often or still changed after a certain predetermined time may then be excluded from the incremental copies.
  • a third example is a directory path of the data portion. It may then be decided beforehand that files in a certain directory location should be excluded from the incremental copies.
  • a fourth example is the read and write privileges of a data portion. Read-only data portions may always be included in the incremental copies.
  • Another example is file ownership where files belonging to certain users are excluded or included in the incremental copies because certain users, be it actual persons or system processes, may be more active than others.
  • the method further comprises performing an intermediate incremental copy wherein no data portions are excluded and using a duration of this intermediate incremental copy as an estimate for a duration of the final cutover incremental copy.
  • the cutover itself is a crucial moment in the migration procedure. For large organisations it may be planned months in advance. Typically they are planned in the weekends to minimize the impact on the organization's productivity. To guarantee and plan the completion of the cutover it is important to know how long the final cutover incremental copy will take. As the partial incremental copies that are taken before exclude some of the data portions they are not a good indication for the cutover duration. Therefore, an intermediate incremental copy without excluding data portions is performed for which the duration provides a good estimate of the duration for the cutover copy. [28] Advantageously, the performing and intermediate incremental copy is executed on a same day of the week as when the performing a final cutover incremental copy is planned.
  • the performing an intermediate incremental copy is executed on a same hour of the day as when the performing a final cutover incremental copy is planned.
  • the duration of an incremental copy may depend on the time when it is performed. Therefore, by planning it on the same day and/or hour as the final cutover, a better estimate is obtained.
  • This performing an intermediate incremental copy may then be executed when a transfer size of the one or more incremental copies has reached a steady state.
  • the performing an initial copy and/or performing one or more incremental copies and/or performing a final cutover incremental copy and/or performing an intermediate incremental copy comprises: scanning all or part of the data to be migrated on the source storage system and/or scanning all or part of the data on said destination storage system which was already copied and creating a list of commands for executing the performing. Then, subsequently executing the list of commands.
  • a list of all the commands to execute the copies is created and then the commands are executed. As the list of commands is known before the actual execution, the progress of the actual copy is known when being executed as the progress of the copy may be derived from the current position in the list of commands.
  • the invention relates to a computer program product comprising computer-executable instructions for performing the method according to the first aspect when the program is run on a computer.
  • the invention relates to a computer readable storage medium comprising the computer program product according to the second aspect.
  • the invention relates to a data processing system programmed for carrying out the method according to the first aspect.
  • Fig. 1 illustrates an example of the transfer time and transfer size of data copies from a source storage system to a destination storage system according to an embodiment
  • Fig. 2 illustrates an example of a source and destination storage system
  • FIG. 3 illustrates an example of the transfer time and transfer size of data copies from a source storage system to a destination storage system according to an embodiment
  • FIG. 4 illustrates steps of a method for performing a data migration from a source to a destination storage system according to an embodiment
  • Fig. 5 illustrates steps of a method for performing a data migration from a source to a destination storage system according to an embodiment
  • Fig. 6 illustrates steps of a method for performing a copy of data from a source to a destination storage system according to an embodiment
  • Fig. 7 illustrates steps of a method for generating a list of commands for performing a copy of data from a source to a destination storage system according to an embodiment
  • FIG. 8 illustrates an exemplary embodiment of a device for performing a data migration.
  • the current disclosure relates to data migration between data storage systems and more particular the data migration from a source storage system to a destination storage system.
  • Fig. 2 illustrates an exemplary embodiment of such a source 200 and destination 220 storage systems.
  • the source storage system comprises a plurality of storage servers 203 each housing one or more digital storage means 202.
  • the destination system comprises a plurality of storage servers 223 each housing one or more digital storage means 222.
  • the storage servers 203 and 223 may be housed in a same or different data centre inside or outside a company's data network.
  • the storage systems 200 and 220 can offer data storage and access to users and services. Such access may be done over the network 230.
  • the data to be migrated from the system 200 to the system 220 typically comprises a set of data portions, which in the most common case will be files organized according to a file system. These files may be data files belonging to users or groups, system files used by an operating system or applications files used by and for applications. [48] In the embodiments below various steps are provided for performing a data migration from a source system 200 to a destination storage 220. When referring to data, it does not necessarily refer to all data on the storage system.
  • the data may first be split in several chunks of data and a data migration may then be performed for each chunk of data as disclosed by the embodiments below.
  • a chunk may for example comprise all data belonging to a certain department of an organization or to a specific subdirectory or mounting point of a file system.
  • Fig. 4 illustrates steps for performing a data migration according to an embodiment of the invention. The steps are further illustrated by Fig. 3 where the transfer size and transfer time of copies 301 -31 1 from the source storage system 200 to the destination storage system 220 are illustrated.
  • a data migration is started. Before and during the migration data storage is still provided from the source data storage system. During the migration the destination storage system is populated with copies of the data. At the end of the migration, during the cutover or switchover, all user access is denied to both source and destination storage systems and the last bits of data are copied to the destination storage system. Then, all users are given access to their data on the destination storage while the source storage system can be taken out of business. By the cutover where access is denied, data integrity is guaranteed.
  • a first step 431 an initial copy of the data is performed.
  • this initial copy is illustrated by the block 301 where its width represents the time it takes to perform the initial copy and its height represents the data size of the transfer. For typical large data migrations, such an initial copy can take several days, weeks or even months. Apart from the size of the data, the transfer time will also be restricted by the available bandwidth for transferring the data between the source 200 and destination 220.
  • the initial copy 301 comprises all data that is to be migrated. In the first step 431 all data portions making up the data are thus copied from the source storage system 200 to the destination storage system 220. [53] According to an alternative embodiment, data portions that are likely to change before the cutover are excluded from the initial copy 301 . As the data portions are still likely to change, a new copy will anyhow have to be made before or during the cutover. Therefore, by excluding such a data portions from the initial copy, the initial copy will take less time to perform and network bandwidth is saved.
  • one or more incremental copies 302 to 306 are made until the start of the actual cutover.
  • an incremental copy only differences between the source and destination system 200 and 220 are applied to or copied to the destination system 220.
  • the first incremental copy is illustrated by block 302. If a data portion on the source has already a copy on the destination that was copied there during the initial copy 301 , the data portion is thus not copied during the incremental copy. Therefore, the incremental copy 302 will be smaller than the initial copy 301 as it is unlikely that all files on the source storage system will have changed. Moreover, data portions that are likely to change before the cutover are excluded from the incremental copy 302.
  • an incremental copy that excludes certain data portions is also referred to as a partial incremental copy in the current disclosure.
  • the step 432 of performing the incremental copies may be repeated several times until the cutover. During a next incremental copy, data portions that were excluded before may now be copied or the other way around. Depending on the criteria used, it could be that a data portion that was previously classified as 'likely to change' is re-evaluated as 'not likely to change' during a subsequent iteration.
  • the step 432 is repeated at least until the transfer size of the incremental copies has reached a steady state. In Fig. 3 the incremental copies 304, 305 and 306 have reached a steady state with regards to their transfer size. This effect is caused by the dependency of the transfer size of an incremental copy on the transfer time of the previous copy.
  • Incremental copy 304 thus depends on the transfer time of the copy 303, 303 depends on 302 and 302 depends on its turn on the transfer time of the initial copy. As the transfer time of the initial copy was large, it takes a few iterations before the incremental copies 302-306 have reached a steady state.
  • step 437 the actual cutover copy is performed during the actual cutover 322, preferably after a steady state is reached according to the condition 433. During this cutover 322, all access to the data is denied and a final cutover incremental copy 31 1 is made. The final cutover incremental copy is similar to the previous partial incremental copies except that no files are excluded. After the cutover 322, users are again granted access to the data, but now on the destination storage system 220.
  • Fig. 5 together with Fig. 1 illustrates the use of an extra intermediate incremental copy 107 in order to estimate the duration 122 of the cutover copy 1 1 1 according to an embodiment.
  • the first step 531 is similar to step 431 where an initial full or partial copy 101 is made.
  • step 532 a first set of partial incremental copies 102 till 106 are made.
  • a dry run of the cutover copy is performed in step 534.
  • an incremental copy is made without excluding any data portions thereby mimicking the final cutover incremental copy.
  • the transfer size and time of this intermediate copy 107 is used to estimate 538 the duration of the final cutover copy 1 1 1 . This estimate can then be used to see if the cutover can be done as planned. If the estimated cutover period takes too long, more time can be allocated for the cutover or a smaller chunk of data can be defined for the migration.
  • the intermediate incremental copy 107 is performed on the same day and even more preferably on the same hour as the planned cutover copy 1 1 1 . This assures that time dependent factors such as the available bandwidth for the data transfers match as close as possible to the cutover.
  • only the data migration of a single data chunk is performed during both the intermediate copy 107 and the cutover copy 1 1 1 to ensure a short transfer time and a good estimate.
  • step 534 After the intermediate copy in step 534, one or more partial incremental copies 108-1 10 are again performed in step 535 until a steady state 121 in the transfer size of the incremental copies is reached again according to the condition 536. Then, the cutover copy 1 1 1 is performed in step 537 similar to step 437
  • Fig. 6 illustrates steps to perform a copy of data during a date migration from the source storage system 200 to the destination storage system 220 according to an embodiment. These steps may be executed to perform the initial copy or partial initial copy according to steps 431 and 531 , to perform the incremental copy according to steps 432, 532 or 535, to perform the intermediate dry-run incremental copy according to step 534 or to perform the final cutover incremental copy in steps 437 and 537.
  • step 641 the metadata of the source storage system is retrieved and scanned and in step 642 the metadata of the destination storage system is retrieved and scanned. Then, in step 643 a list of commands is generated by comparing the scanned metadata.
  • Such commands may comprise:
  • steps 641 and 642 It is not always necessary to fully perform the steps 641 and 642, i.e., it is not necessary to fully scan both source and destination storage systems. For example, a list with changes since the previous iteration may be obtained from the source system. Additionally, the state of the destination storage system may be derived from one of its previous states or index and the outcome of the previous commands in order to calculate the new system state. If this information is available, the list of commands can be derived in step 643.
  • Fig. 7 illustrates steps performed for generating 643 the list of commands according to an embodiment for the case where data portions are to be excluded to perform a partial incremental copy. The illustrated steps are performed for every data portion on the source storage system 200.
  • the first step 751 it is checked whether there is a difference between the data portion on the source and the destination based on the associated metadata that was scanned in the steps 641 and 642. This check may comprise:
  • the data portion is present on the source storage system but not on the destination storage system.
  • the data portion is present on the destination storage system but not on the source storage system.
  • the data portion is present on the destination storage system, but its metadata such as the file change history indicates that the data portion on the source storage system was changed.
  • the data portion is both present on the source and destination system and its content is unaltered, but some of its metadata has changed. For example, a data file may be unchanged, but user rights may be different.
  • the method proceeds to the next step 752, otherwise, the data portion nor its metadata has changed and no command needs to generated for the respective data portion.
  • this next step 752 it is checked whether the respective data portion is likely to change before the cutover. If it is likely to change and even though there is a difference for this data portion between the source and destination storage system, no further command is generated and this data portion is skipped from the copy.
  • the type of the data portion such as the file type is checked against a predetermined list of types that are excluded from the copies. This list then comprises file types that typically hold data that is likely to change a lot.
  • file types may for example be mailbox file types, database file types, system files of an operating system and cache files for storing temporary data.
  • the owner or group to which a file belongs is checked against a predetermined list of users and groups.
  • the change history of a file is checked. If the file has been recently changed, for example after a certain date, the file is excluded from the copy.
  • a location is within a set of predetermined locations.
  • a location may for example be a directory. This way locations in the directory system that are known to comprise files that change a lot can be excluded by default.
  • step 752 When it is determined under step 752 that the respective data portion is not likely to change, the method proceeds to step 753 where a command is generated depending on the detected difference under step 751 .
  • a command may for example be:
  • Fig. 8 shows a suitable computing system 800 for performing the steps according to the method of the above embodiments.
  • Computing system 800 may in general be formed as a suitable general purpose computer and comprise a bus 810, a processor 802, a local memory 804, one or more optional input interfaces 814, one or more optional output interfaces 816, a communication interface 812, a storage element interface 806 and one or more storage elements 808.
  • Bus 810 may comprise one or more conductors that permit communication among the components of the computing system 800.
  • Processor 802 may include any type of conventional processor or microprocessor that interprets and executes programming instructions.
  • Local memory 804 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 802 and/or a read only memory (ROM) or another type of static storage device that stores static information and instructions for use by processor 802.
  • Input interface 814 may comprise one or more conventional mechanisms that permit an operator to input information to the computing device 800, such as a keyboard 820, a mouse 830, a pen, voice recognition and/or biometric mechanisms, etc.
  • Output interface 816 may comprise one or more conventional mechanisms that output information to the operator, such as a display 840, a printer 850, a speaker, etc.
  • Communication interface 812 may comprise any transceiver-like mechanism such as for example one or more Ethernet interfaces that enables computing system 800 to communicate with other devices and/or systems, for example mechanisms for communicating with the source and destination storage systems 200 and 220 of Fig. 2.
  • the communication interface 812 of computing system 800 may be connected to such another computing system by means of a local area network (LAN) or a wide area network (WAN) such as for example the internet.
  • Storage element interface 806 may comprise a storage interface such as for example a Serial Advanced Technology Attachment (SATA) interface or a Small Computer System Interface (SCSI) for connecting bus 810 to one or more storage elements 808, such as one or more local disks, for example SATA disk drives, and control the reading and writing of data to and/or from these storage elements 808.
  • SATA Serial Advanced Technology Attachment
  • SCSI Small Computer System Interface
  • the storage elements 808 above is described as a local disk, in general any other suitable computer-readable media such as a removable magnetic disk, optical storage media such as a CD or DVD, -ROM disk, solid state drives, flash memory cards, ... could be used.
  • the system 800 described above can also run as a Virtual Machine above the physical hardware.
  • the steps illustrated by the above embodiments can be implemented as programming instructions stored in local memory 804 of the computing system 800 for execution by its processor 802. Alternatively the instruction can be stored on the storage element 808 or be accessible from another computing system through the communication interface 812.
  • the system 800 may be connected to the network 230 of Fig. 2 by its communication interface 812. This way the system 800 has access to both the source storage system 200 and destination storage system 220 for executing the steps according to the various embodiments.
  • the steps according to the above embodiments may also be performed as instructions on one of the servers 203, 223 where these servers have a similar architecture as the system 800 of Fig. 8.
  • top, bottom, over, under, and the like are introduced for descriptive purposes and not necessarily to denote relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances and embodiments of the invention are capable of operating according to the present invention in other sequences, or in orientations different from the one(s) described or illustrated above.

Abstract

According to an embodiment the invention relates to a method for migrating data from a source storage system to a destination storage system comprising the following steps. In a first step, an initial copy (301) is performed. In a second step, one or more incremental copies (302-306) are performed and then a final cutover incremental copy (311) is performed. The performing the one or more incremental copies (302-306) further comprise excluding from a respective one of the one or more incremental copies first data portions of the data that are likely to change before this performing a final cutover incremental copy (311).

Description

DATA MIGRATION TOOL WITH INTERMEDIATE INCREMENTAL COPIES
Field of the Invention
[01] In general, the invention relates to the field of data migration tools. Such tools aid in the automated migration of digital data from a source storage system to a destination storage system. [02] More particular, the invention relates to the migration of huge amounts of data where a single copy of all data on the source storage system to the destination storage system may last in the order of days, weeks or even months.
Background of the Invention
[03] The need for data storage capacity is increasing rapidly every year. Today, a company's storage system may be distributed over different locations and comprise multiple server racks in one or multiple data centres where each rack houses multiple storage servers. Some companies outsource their storage needs to external storage providers offering cloud based storage solutions.
[04] At some point in time, a storage system user may decide to migrate his data from its current storage system to a new one. This decision may be driven by several factors. A first factor may be financial considerations, where the new storage system provider offers the same or more capacity for a better price. Another factor may be that the capacity of the current storage system can no longer be increased and a migration to a new and larger storage system is inevitable. [05] In all of these cases, a data migration is to be performed, i.e., all data on the source system needs to be copied to the destination system and, at some point in time, users need to be switched to the new destination system. During the actual switch or cutover, the users are typically denied access to both storage systems in order to ensure data integrity. This way users cannot write to data that is being copied which could cause data corruption or users cannot write to a data location that has already passed the migration which could cause data loss.
[06] For large storage systems serving tens of Terabytes up to several Petabytes of data, a single copy of all data may take in the order of days, weeks or even months. Denying user access to the storage system for such a long time is simply unacceptable and thus solutions are needed to shorten the switchover or cutover time. [07] WO12149220A discloses the concept of incremental copies to shorten the actual cutover time. First, an initial or baseline copy is made of all data to be migrated from the source to the destination system. Then, one or more incremental copies are made before the actual switchover. An incremental copy only considers the differences between the source and destination system. It thus applies all changes from the users that are still using the source storage system. During the initial copy and incremental copies, the users are still allowed access to their data on the source storage system. Then, at a certain planned point in time, the actual cutover is performed. During the cutover, the users are denied all access from the storage systems and a last or cutover incremental copy is made. When the cutover copy is done, the users are switched to the new destination storage system and can again access their data.
[08] Although the above concept greatly reduces the actual cutover time, it is still an object to further shorten the cutover time.
Summary of the Invention
[09] This object is achieved by a computer implemented method for migrating data from a source storage system to a destination storage system comprising the step of performing an initial copy, subsequently performing one or more incremental copies and subsequently performing a final cutover incremental copy. This performing one or more incremental copies further comprises excluding from a respective one of the one or more incremental copies first data portions of the data that are likely to change before this performing a final cutover incremental copy. [10] After the initial copy or baseline copy of all data that is being migrated, one or more incremental copies are made. In an incremental copy, the differences in data between the source storage system and the destination storage system are applied to the destination storage system. An incremental copy may comprise the copying of a data portion from the source to the destination, a deletion of a data portion in the destination or an update of a data portion in the destination. The data comprises a plurality of data portions, which are defined as units of data that are copied from the source to the destination system. In typical data storage systems such a data portion is a file residing hierarchically structured in a file system and the data is thus copied on a file by file basis. At the end, the actual cutover is performed and a last incremental copy referred to as the final cutover incremental copy is made. After a successful cutover, users may start using the destination storage system. Typically, users are denied access to both storage systems during the cutover to ensure data integrity.
[11] When a certain incremental copy is made or prepared, it is checked whether a data portion is still likely to change before the cutover and is excluded from the incremental copy when it is. If a data portion is likely to change in the future before the cutover, it will anyhow end up in a future incremental copy or in the cutover copy and it is thus not necessary to already include it in the current incremental copy. When data is excluded from an incremental copy, the copy is also referred to as a partial incremental copy. [12] By excluding data portions from the incremental copies that will anyhow be included in the final cutover copy, the incremental copies are smaller in size and can thus be executed in a shorter time period. When an incremental copy is smaller, less changes will be made by the users up till the next incremental copy and also up till the final cutover copy. This way, the final cutover copy is smaller and thus takes less time to execute. It is thus an advantage that the cutover time where users might have no access to the storage system is reduced.
[13] It is a further advantage that less data is copied from the source to the destination system thereby saving in bandwidth. [14] The performing an initial copy may further comprise excluding second data portions of the data from the initial copy that are likely to change before the performing the final cutover incremental copy.
[15] The same principle of excluding data portions is thus applied to the full initial copy thereby also reducing the size and duration of this first copy.
[16] Advantageously, the final cutover incremental copy is performed when a transfer size of the one or more incremental copies has reached a steady state.
[17] The incremental copies will typically decrease in transfer size and transfer time till they reach a certain steady state in transfer size, i.e., up to a moment where the transfer size and/or time of subsequent incremental copies are substantially the same. At the time of the first initial copy, there is no data yet on the destination storage system. The transfer size of this first initial copy may thus be very large taking days to months to execute for large storage systems. The transfer size of an incremental copy depends on the duration since the previous copy. As the transfer time of the initial copy is so large, the first incremental copy will still be considerably large. Every subsequent incremental copy will then decrease in size until a certain steady state in transfer size and time.
[18] By performing the cutover after this steady state has been reached, the cutover time is further reduced.
[19] According to an embodiment, the excluding of first and/or second data comprises retrieving metadata associated with data portions of the data on the source storage system wherein the metadata is indicative for a likelihood that a respective data portion will change before the final cutover incremental copy. The excluding further comprises selecting the first and/or second data portions based on this metadata.
[20] Metadata is available information about the data portions that provide an indication if the data portion is likely to change before the cutover. [21] A first type of metadata may be available from the source storage system itself. One example of this first type is the type of a respective data portion or a file type if the data portion corresponds to a file. A predetermined list of file types may then be used to decide whether a data portion is likely to change. For example, .pst files which are typically used for storing an email database may be excluded from the incremental copies
[22] A second example of the first type of metadata is a change history of a data portion or a file. Files that are changed often or still changed after a certain predetermined time may then be excluded from the incremental copies.
[23] A third example is a directory path of the data portion. It may then be decided beforehand that files in a certain directory location should be excluded from the incremental copies.
[24] A fourth example is the read and write privileges of a data portion. Read-only data portions may always be included in the incremental copies. [25] Another example is file ownership where files belonging to certain users are excluded or included in the incremental copies because certain users, be it actual persons or system processes, may be more active than others.
[26] According to a further embodiment, the method further comprises performing an intermediate incremental copy wherein no data portions are excluded and using a duration of this intermediate incremental copy as an estimate for a duration of the final cutover incremental copy.
[27] The cutover itself is a crucial moment in the migration procedure. For large organisations it may be planned months in advance. Typically they are planned in the weekends to minimize the impact on the organization's productivity. To guarantee and plan the completion of the cutover it is important to know how long the final cutover incremental copy will take. As the partial incremental copies that are taken before exclude some of the data portions they are not a good indication for the cutover duration. Therefore, an intermediate incremental copy without excluding data portions is performed for which the duration provides a good estimate of the duration for the cutover copy. [28] Advantageously, the performing and intermediate incremental copy is executed on a same day of the week as when the performing a final cutover incremental copy is planned.
[29] More advantageously, the performing an intermediate incremental copy is executed on a same hour of the day as when the performing a final cutover incremental copy is planned.
[30] The duration of an incremental copy may depend on the time when it is performed. Therefore, by planning it on the same day and/or hour as the final cutover, a better estimate is obtained.
[31] This performing an intermediate incremental copy may then be executed when a transfer size of the one or more incremental copies has reached a steady state.
[32] When the final cutover copy is performed after a steady state of the incremental copies, a good estimate by the intermediate copy is ensured by also performing the intermediate incremental copy when a steady state in transfer size of the incremental copies that were taken before is achieved.
[33] According to a particular embodiment, the performing an initial copy and/or performing one or more incremental copies and/or performing a final cutover incremental copy and/or performing an intermediate incremental copy comprises: scanning all or part of the data to be migrated on the source storage system and/or scanning all or part of the data on said destination storage system which was already copied and creating a list of commands for executing the performing. Then, subsequently executing the list of commands. [34] First, a list of all the commands to execute the copies is created and then the commands are executed. As the list of commands is known before the actual execution, the progress of the actual copy is known when being executed as the progress of the copy may be derived from the current position in the list of commands.
[35] According to a second aspect, the invention relates to a computer program product comprising computer-executable instructions for performing the method according to the first aspect when the program is run on a computer.
[36] According to a third aspect, the invention relates to a computer readable storage medium comprising the computer program product according to the second aspect. [37] According to a fourth aspect, the invention relates to a data processing system programmed for carrying out the method according to the first aspect.
Brief Description of the Drawings
[38] Fig. 1 illustrates an example of the transfer time and transfer size of data copies from a source storage system to a destination storage system according to an embodiment; and [39] Fig. 2 illustrates an example of a source and destination storage system; and
[40] Fig. 3 illustrates an example of the transfer time and transfer size of data copies from a source storage system to a destination storage system according to an embodiment; and
[41] Fig. 4 illustrates steps of a method for performing a data migration from a source to a destination storage system according to an embodiment; and
[42] Fig. 5 illustrates steps of a method for performing a data migration from a source to a destination storage system according to an embodiment; and [43] Fig. 6 illustrates steps of a method for performing a copy of data from a source to a destination storage system according to an embodiment; and [44] Fig. 7 illustrates steps of a method for generating a list of commands for performing a copy of data from a source to a destination storage system according to an embodiment; and
[45] Fig. 8 illustrates an exemplary embodiment of a device for performing a data migration.
Detailed Description of Embodiment(s) [46] The current disclosure relates to data migration between data storage systems and more particular the data migration from a source storage system to a destination storage system. Fig. 2 illustrates an exemplary embodiment of such a source 200 and destination 220 storage systems. The source storage system comprises a plurality of storage servers 203 each housing one or more digital storage means 202. Similarly the destination system comprises a plurality of storage servers 223 each housing one or more digital storage means 222. The storage servers 203 and 223 may be housed in a same or different data centre inside or outside a company's data network. The storage systems 200 and 220 can offer data storage and access to users and services. Such access may be done over the network 230. Various protocols may be used for accessing the data such as for example CIFS, SMB, FTP or NFS. Companywide storage systems may offer a huge data storage capacity and are often deployed and maintained by external storage providers such as for example NetApp, EMC or Hitachi. [47] The data to be migrated from the system 200 to the system 220 typically comprises a set of data portions, which in the most common case will be files organized according to a file system. These files may be data files belonging to users or groups, system files used by an operating system or applications files used by and for applications. [48] In the embodiments below various steps are provided for performing a data migration from a source system 200 to a destination storage 220. When referring to data, it does not necessarily refer to all data on the storage system. The data may first be split in several chunks of data and a data migration may then be performed for each chunk of data as disclosed by the embodiments below. Such a chunk may for example comprise all data belonging to a certain department of an organization or to a specific subdirectory or mounting point of a file system.
[49] Fig. 4 illustrates steps for performing a data migration according to an embodiment of the invention. The steps are further illustrated by Fig. 3 where the transfer size and transfer time of copies 301 -31 1 from the source storage system 200 to the destination storage system 220 are illustrated.
[50] At some point in time, a data migration is started. Before and during the migration data storage is still provided from the source data storage system. During the migration the destination storage system is populated with copies of the data. At the end of the migration, during the cutover or switchover, all user access is denied to both source and destination storage systems and the last bits of data are copied to the destination storage system. Then, all users are given access to their data on the destination storage while the source storage system can be taken out of business. By the cutover where access is denied, data integrity is guaranteed.
[51] In a first step 431 an initial copy of the data is performed. In Fig. 3 this initial copy is illustrated by the block 301 where its width represents the time it takes to perform the initial copy and its height represents the data size of the transfer. For typical large data migrations, such an initial copy can take several days, weeks or even months. Apart from the size of the data, the transfer time will also be restricted by the available bandwidth for transferring the data between the source 200 and destination 220.
[52] According to an embodiment, the initial copy 301 comprises all data that is to be migrated. In the first step 431 all data portions making up the data are thus copied from the source storage system 200 to the destination storage system 220. [53] According to an alternative embodiment, data portions that are likely to change before the cutover are excluded from the initial copy 301 . As the data portions are still likely to change, a new copy will anyhow have to be made before or during the cutover. Therefore, by excluding such a data portions from the initial copy, the initial copy will take less time to perform and network bandwidth is saved.
[54] After performing the initial copy in step 431 , one or more incremental copies 302 to 306 are made until the start of the actual cutover. During an incremental copy only differences between the source and destination system 200 and 220 are applied to or copied to the destination system 220. In Fig. 3, the first incremental copy is illustrated by block 302. If a data portion on the source has already a copy on the destination that was copied there during the initial copy 301 , the data portion is thus not copied during the incremental copy. Therefore, the incremental copy 302 will be smaller than the initial copy 301 as it is unlikely that all files on the source storage system will have changed. Moreover, data portions that are likely to change before the cutover are excluded from the incremental copy 302. As the data portions are still likely to change, a new copy will anyhow have to be made before or during the cutover. Therefore, by excluding such a data portions from the incremental copy, the incremental copy will take less time to perform and network bandwidth is saved. An incremental copy that excludes certain data portions is also referred to as a partial incremental copy in the current disclosure.
[55] The step 432 of performing the incremental copies may be repeated several times until the cutover. During a next incremental copy, data portions that were excluded before may now be copied or the other way around. Depending on the criteria used, it could be that a data portion that was previously classified as 'likely to change' is re-evaluated as 'not likely to change' during a subsequent iteration. Preferably, the step 432 is repeated at least until the transfer size of the incremental copies has reached a steady state. In Fig. 3 the incremental copies 304, 305 and 306 have reached a steady state with regards to their transfer size. This effect is caused by the dependency of the transfer size of an incremental copy on the transfer time of the previous copy. Incremental copy 304 thus depends on the transfer time of the copy 303, 303 depends on 302 and 302 depends on its turn on the transfer time of the initial copy. As the transfer time of the initial copy was large, it takes a few iterations before the incremental copies 302-306 have reached a steady state.
[56] Then, in step 437, the actual cutover copy is performed during the actual cutover 322, preferably after a steady state is reached according to the condition 433. During this cutover 322, all access to the data is denied and a final cutover incremental copy 31 1 is made. The final cutover incremental copy is similar to the previous partial incremental copies except that no files are excluded. After the cutover 322, users are again granted access to the data, but now on the destination storage system 220.
[57] By excluding files from the incremental copies, the transfer size and thus also the transfer time of the incremental copies has been reduced. As the size of the final cutover incremental copy depends on the previous incremental copy, also the transfer time of the final copy will be reduced.
[58] In data migration, planning is of crucial importance. Typically, a cutover is performed during a weekend when less users are affected by denied access to their data compared with working days. In order to verify that the cutover can be finalized in a desired time window, a good estimate for the duration of the cutover copy is important.
[59] Fig. 5 together with Fig. 1 illustrates the use of an extra intermediate incremental copy 107 in order to estimate the duration 122 of the cutover copy 1 1 1 according to an embodiment. The first step 531 is similar to step 431 where an initial full or partial copy 101 is made. Then, in step 532 a first set of partial incremental copies 102 till 106 are made. When these copies have reached a steady state 120 according to the condition 533 where their transfer size is substantially the same as the transfer size of the previous partial incremental copy, a dry run of the cutover copy is performed in step 534. During this step, an incremental copy is made without excluding any data portions thereby mimicking the final cutover incremental copy. In other words, the transfer size and time of this intermediate copy 107 is used to estimate 538 the duration of the final cutover copy 1 1 1 . This estimate can then be used to see if the cutover can be done as planned. If the estimated cutover period takes too long, more time can be allocated for the cutover or a smaller chunk of data can be defined for the migration.
[60] Preferably, the intermediate incremental copy 107 is performed on the same day and even more preferably on the same hour as the planned cutover copy 1 1 1 . This assures that time dependent factors such as the available bandwidth for the data transfers match as close as possible to the cutover. Advantageously, only the data migration of a single data chunk is performed during both the intermediate copy 107 and the cutover copy 1 1 1 to ensure a short transfer time and a good estimate.
[61] After the intermediate copy in step 534, one or more partial incremental copies 108-1 10 are again performed in step 535 until a steady state 121 in the transfer size of the incremental copies is reached again according to the condition 536. Then, the cutover copy 1 1 1 is performed in step 537 similar to step 437
[62] Fig. 6 illustrates steps to perform a copy of data during a date migration from the source storage system 200 to the destination storage system 220 according to an embodiment. These steps may be executed to perform the initial copy or partial initial copy according to steps 431 and 531 , to perform the incremental copy according to steps 432, 532 or 535, to perform the intermediate dry-run incremental copy according to step 534 or to perform the final cutover incremental copy in steps 437 and 537.
[63] In step 641 , the metadata of the source storage system is retrieved and scanned and in step 642 the metadata of the destination storage system is retrieved and scanned. Then, in step 643 a list of commands is generated by comparing the scanned metadata. Such commands may comprise:
- An instruction to copy a data portion from the source storage system to the destination storage system.
- An instruction to delete a data portion from the destination storage system.
- An instruction to update the metadata of a certain data portion on the destination storage system such as for example user rights, ownership and author information. [64] Then, in the last step 644 the list of commands is executed thereby performing the actual copy of the data.
[65] It is not always necessary to fully perform the steps 641 and 642, i.e., it is not necessary to fully scan both source and destination storage systems. For example, a list with changes since the previous iteration may be obtained from the source system. Additionally, the state of the destination storage system may be derived from one of its previous states or index and the outcome of the previous commands in order to calculate the new system state. If this information is available, the list of commands can be derived in step 643.
[66] Fig. 7 illustrates steps performed for generating 643 the list of commands according to an embodiment for the case where data portions are to be excluded to perform a partial incremental copy. The illustrated steps are performed for every data portion on the source storage system 200. In the first step 751 , it is checked whether there is a difference between the data portion on the source and the destination based on the associated metadata that was scanned in the steps 641 and 642. This check may comprise:
- The data portion is present on the source storage system but not on the destination storage system.
- The data portion is present on the destination storage system but not on the source storage system.
- The data portion is present on the destination storage system, but its metadata such as the file change history indicates that the data portion on the source storage system was changed.
- The data portion is both present on the source and destination system and its content is unaltered, but some of its metadata has changed. For example, a data file may be unchanged, but user rights may be different. [67] If a difference is detected, the method proceeds to the next step 752, otherwise, the data portion nor its metadata has changed and no command needs to generated for the respective data portion. In this next step 752, it is checked whether the respective data portion is likely to change before the cutover. If it is likely to change and even though there is a difference for this data portion between the source and destination storage system, no further command is generated and this data portion is skipped from the copy.
[68] There are several possibilities for detecting this likelihood from the scanned metadata and these may further be combined. Some examples according to an embodiment are:
- The type of the data portion such as the file type is checked against a predetermined list of types that are excluded from the copies. This list then comprises file types that typically hold data that is likely to change a lot. Such file types may for example be mailbox file types, database file types, system files of an operating system and cache files for storing temporary data.
- The owner or group to which a file belongs is checked against a predetermined list of users and groups.
- The change history of a file is checked. If the file has been recently changed, for example after a certain date, the file is excluded from the copy.
- It is checked whether the location of a file is within a set of predetermined locations. Such a location may for example be a directory. This way locations in the directory system that are known to comprise files that change a lot can be excluded by default.
[69] When it is determined under step 752 that the respective data portion is not likely to change, the method proceeds to step 753 where a command is generated depending on the detected difference under step 751 . Such a command may for example be:
- Copy the respective data portion from the source to the destination storage system.
- Delete the respective data portion from the destination storage system.
- Update the metadata of the data portion on the destination with the metadata of the data portion on the source.
[70] The steps of Fig. 7 are performed for all scanned data portions, both on the source and destination storage system resulting in the list of commands of step 643 in Fig. 6. [71] Fig. 8 shows a suitable computing system 800 for performing the steps according to the method of the above embodiments. Computing system 800 may in general be formed as a suitable general purpose computer and comprise a bus 810, a processor 802, a local memory 804, one or more optional input interfaces 814, one or more optional output interfaces 816, a communication interface 812, a storage element interface 806 and one or more storage elements 808. Bus 810 may comprise one or more conductors that permit communication among the components of the computing system 800. Processor 802 may include any type of conventional processor or microprocessor that interprets and executes programming instructions. Local memory 804 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 802 and/or a read only memory (ROM) or another type of static storage device that stores static information and instructions for use by processor 802. Input interface 814 may comprise one or more conventional mechanisms that permit an operator to input information to the computing device 800, such as a keyboard 820, a mouse 830, a pen, voice recognition and/or biometric mechanisms, etc. Output interface 816 may comprise one or more conventional mechanisms that output information to the operator, such as a display 840, a printer 850, a speaker, etc. Communication interface 812 may comprise any transceiver-like mechanism such as for example one or more Ethernet interfaces that enables computing system 800 to communicate with other devices and/or systems, for example mechanisms for communicating with the source and destination storage systems 200 and 220 of Fig. 2. The communication interface 812 of computing system 800 may be connected to such another computing system by means of a local area network (LAN) or a wide area network (WAN) such as for example the internet. Storage element interface 806 may comprise a storage interface such as for example a Serial Advanced Technology Attachment (SATA) interface or a Small Computer System Interface (SCSI) for connecting bus 810 to one or more storage elements 808, such as one or more local disks, for example SATA disk drives, and control the reading and writing of data to and/or from these storage elements 808. Although the storage elements 808 above is described as a local disk, in general any other suitable computer-readable media such as a removable magnetic disk, optical storage media such as a CD or DVD, -ROM disk, solid state drives, flash memory cards, ... could be used. The system 800 described above can also run as a Virtual Machine above the physical hardware.
[72] The steps illustrated by the above embodiments can be implemented as programming instructions stored in local memory 804 of the computing system 800 for execution by its processor 802. Alternatively the instruction can be stored on the storage element 808 or be accessible from another computing system through the communication interface 812. [73] The system 800 may be connected to the network 230 of Fig. 2 by its communication interface 812. This way the system 800 has access to both the source storage system 200 and destination storage system 220 for executing the steps according to the various embodiments. The steps according to the above embodiments may also be performed as instructions on one of the servers 203, 223 where these servers have a similar architecture as the system 800 of Fig. 8.
[74] Although the present invention has been illustrated by reference to specific embodiments, it will be apparent to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied with various changes and modifications without departing from the scope thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. In other words, it is contemplated to cover any and all modifications, variations or equivalents that fall within the scope of the basic underlying principles and whose essential attributes are claimed in this patent application. It will furthermore be understood by the reader of this patent application that the words "comprising" or "comprise" do not exclude other elements or steps, that the words "a" or "an" do not exclude a plurality, and that a single element, such as a computer system, a processor, or another integrated unit may fulfil the functions of several means recited in the claims. Any reference signs in the claims shall not be construed as limiting the respective claims concerned. The terms "first", "second", third", "a", "b", "c", and the like, when used in the description or in the claims are introduced to distinguish between similar elements or steps and are not necessarily describing a sequential or chronological order. Similarly, the terms "top", "bottom", "over", "under", and the like are introduced for descriptive purposes and not necessarily to denote relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances and embodiments of the invention are capable of operating according to the present invention in other sequences, or in orientations different from the one(s) described or illustrated above.

Claims

1 . A computer implemented method for migrating data from a source storage system 200 to a destination storage system 220 comprising the following steps:
- performing (431 , 531 ) an initial copy (101 , 301 ); and
- subsequently performing (432, 532, 535) one or more incremental copies (102-106, 108-1 10, 302-306); and then
- subsequently performing (437, 537) a final cutover incremental copy (1 1 1 , 31 1 );
characterized in that said performing one or more incremental copies (102-1 10) further comprises excluding (752) from a respective one of said one or more incremental copies first data portions of said data that are likely to change before said performing a final cutover incremental copy (1 1 1 , 31 1 ).
2. A method according to claim 1 wherein said performing (431 , 531 ) an initial copy (101 , 301 ) further comprises excluding (752) second data portions of said data from said initial copy that are likely to change before said performing (437, 537) said final cutover incremental copy (1 1 1 , 31 1 ).
3. A method according to claim 1 or 2 wherein said final cutover incremental copy (1 1 1 , 31 1 ) is performed when (536) a transfer size of said one or more incremental copies has reached a steady state (121 , 321 ).
4. A method according to any one of the preceding claims wherein said excluding of first and/or second data comprises:
- retrieving (641 ) metadata associated with data portions of said data on said source storage system; wherein said metadata is indicative for a likelihood that a respective data portion will change before said final cutover incremental copy;
- selecting said first and/or second data portions based on said metadata.
5. A method according to claim 4 wherein said metadata comprises a file type of a respective data portion; and wherein said selecting of first and/or second data portions comprises selecting data portions of a predetermined file type.
6. A method according to claim 4 or 5 wherein said metadata comprises a change history of a respective data portion; and wherein said selecting of first and/or second data portions comprises selecting data portions that where changed after a predetermined time.
7. A method according to any one of the preceding claims further comprising:
- performing (534) an intermediate incremental copy (107) wherein no data portions are excluded; and
- using (538) a duration of said intermediate incremental copy as an estimate for a duration of said final cutover incremental copy.
8. A method according to claim 7 wherein said performing and intermediate incremental copy is executed on a same day of the week as when said performing a final cutover incremental copy is planned.
9. A method according to claim 7 or 8 wherein said performing an intermediate incremental copy is executed on a same hour of the day as when said performing a final cutover incremental copy is planned.
10. A method according to any one of claims 7 to 9 and claim 3 wherein said performing an intermediate incremental copy (107) is executed when (533) a transfer size of said one or more incremental copies (102-106) has reached a steady state (120).
1 1 . A method according to any one of the preceding claims wherein performing an initial copy and/or performing one or more incremental copies and/or performing a final cutover incremental copy and/or performing an intermediate incremental copy comprises:
- scanning (641 ) all or part of the data to be migrated on said source storage system and/or scanning (642) all or part of the data on said destination storage system which was already copied; and
- subsequently creating (643) a list of commands for executing said performing; and subsequently executing (644) said list of commands.
12. A computer program product comprising computer-executable instructions for performing the method according to any one of claims 1 to 1 1 when the program is run on a computer (800).
13. A computer readable storage medium (808) comprising the computer program product according to claim 12.
14. A data processing system programmed for carrying out the method according to any one of claims 1 to 1 1 .
PCT/EP2015/068227 2014-09-11 2015-08-07 Data migration tool with intermediate incremental copies WO2016037777A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CA2959647A CA2959647C (en) 2014-09-11 2015-08-07 Data migration tool with intermediate incremental copies
US15/510,154 US10114579B2 (en) 2014-09-11 2015-08-07 Data migration tool with intermediate incremental copies

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP14184344.1A EP2996025B1 (en) 2014-09-11 2014-09-11 Data migration tool with intermediate incremental copies
EP14184344.1 2014-09-11

Publications (1)

Publication Number Publication Date
WO2016037777A1 true WO2016037777A1 (en) 2016-03-17

Family

ID=51542173

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/068227 WO2016037777A1 (en) 2014-09-11 2015-08-07 Data migration tool with intermediate incremental copies

Country Status (6)

Country Link
US (1) US10114579B2 (en)
EP (1) EP2996025B1 (en)
BE (1) BE1022200B1 (en)
CA (1) CA2959647C (en)
DK (1) DK2996025T3 (en)
WO (1) WO2016037777A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10754073B2 (en) 2016-11-29 2020-08-25 3M Innovative Properties Company Optical stack having reflective polarizer with polymeric layers and specified reflectance and transmittance

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10909120B1 (en) * 2016-03-30 2021-02-02 Groupon, Inc. Configurable and incremental database migration framework for heterogeneous databases
US10970310B2 (en) 2018-08-02 2021-04-06 Netapp Inc. Synchronous replication based cutover engine
EP3889753A1 (en) * 2020-03-30 2021-10-06 Datadobi bv Data migration
EP4124938A1 (en) * 2021-07-27 2023-02-01 Datadobi bv Synchronizing file system items having a data and metadata portion between a source storage system and a destination storage system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7743028B1 (en) * 2005-07-13 2010-06-22 Symantec Corporation Incremental backup of partial volumes

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6996586B2 (en) * 2003-06-18 2006-02-07 International Business Machines Corporation Method, system, and article for incremental virtual copy of a data block
US7873601B1 (en) * 2006-06-29 2011-01-18 Emc Corporation Backup of incremental metadata in block based backup systems
US20120278553A1 (en) 2011-04-28 2012-11-01 Mudhiganti Devender R System and method for migration of data clones
US8990162B1 (en) * 2011-09-30 2015-03-24 Emc Corporation Metadata generation for incremental backup

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7743028B1 (en) * 2005-07-13 2010-06-22 Symantec Corporation Incremental backup of partial volumes

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10754073B2 (en) 2016-11-29 2020-08-25 3M Innovative Properties Company Optical stack having reflective polarizer with polymeric layers and specified reflectance and transmittance

Also Published As

Publication number Publication date
EP2996025A1 (en) 2016-03-16
US20170269869A1 (en) 2017-09-21
EP2996025B1 (en) 2018-04-25
CA2959647C (en) 2021-02-23
BE1022200B1 (en) 2016-02-29
US10114579B2 (en) 2018-10-30
CA2959647A1 (en) 2016-03-17
DK2996025T3 (en) 2018-06-18

Similar Documents

Publication Publication Date Title
US9122692B1 (en) Systems and methods for reducing file-system fragmentation when restoring block-level backups utilizing an identification module, an optimization module, and a restore module
CA2959647C (en) Data migration tool with intermediate incremental copies
US9250819B2 (en) Learning machine to optimize random access in a storage system
EP2989556B1 (en) Object storage using multiple dimensions of object information
US8108446B1 (en) Methods and systems for managing deduplicated data using unilateral referencing
US8984027B1 (en) Systems and methods for migrating files to tiered storage systems
US9158631B1 (en) Systems and methods for providing backup interfaces
US9612910B2 (en) Systems and methods for generating catalogs for snapshots
US10997127B2 (en) Preventing inefficient recalls in a hierarchical storage management (HSM) system
JP4267353B2 (en) Data migration support system and data migration support method
US8131671B2 (en) Uninterrupted data access during the migration of data between physical file systems
US8825653B1 (en) Characterizing and modeling virtual synthetic backup workloads
US10372547B1 (en) Recovery-chain based retention for multi-tier data storage auto migration system
US8983899B1 (en) Systems and methods for archiving files in distributed replication environments
US11175846B2 (en) Data co-location in a hierarchical storage management (HSM) system
US10620865B2 (en) Writing files to multiple tapes
US20210326271A1 (en) Stale data recovery using virtual storage metadata
US10884649B2 (en) Tape drive data reclamation
US10732843B2 (en) Tape drive data reclamation
US8495026B1 (en) Systems and methods for migrating archived files
US11188455B2 (en) Media import and export in storage management systems
CN109241011B (en) Virtual machine file processing method and device
US9128823B1 (en) Synthetic data generation for backups of block-based storage
US11762603B2 (en) Storing modified or unmodified portions of a file based on tape loading
US20230169038A1 (en) File Transfer Prioritization During Replication

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15753649

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2959647

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 15510154

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15753649

Country of ref document: EP

Kind code of ref document: A1