WO2017167100A1 - 一种数据迁移方法和装置 - Google Patents

一种数据迁移方法和装置 Download PDF

Info

Publication number
WO2017167100A1
WO2017167100A1 PCT/CN2017/077722 CN2017077722W WO2017167100A1 WO 2017167100 A1 WO2017167100 A1 WO 2017167100A1 CN 2017077722 W CN2017077722 W CN 2017077722W WO 2017167100 A1 WO2017167100 A1 WO 2017167100A1
Authority
WO
WIPO (PCT)
Prior art keywords
interface
data
upper layer
class
metadata
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/077722
Other languages
English (en)
French (fr)
Inventor
陈逸喆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of WO2017167100A1 publication Critical patent/WO2017167100A1/zh
Priority to US16/147,376 priority Critical patent/US11354283B2/en
Anticipated expiration legal-status Critical
Priority to US17/659,448 priority patent/US11940965B2/en
Priority to US18/587,138 priority patent/US20240193136A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/178—Techniques for file synchronisation in file systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/214—Database migration support
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present application relates to the field of computer technologies, and in particular, to a data migration method and a data migration device.
  • cloud server is the most basic and important product, followed by cloud storage.
  • All cloud service providers have their own cloud storage products, such as AWS (Amazon Web Services), S3, Facebook Cloud OSS (Object Storage Service), Microsoft Cloud Computing Platform Azure storage data service ( Blob).
  • AWS Amazon Web Services
  • S3 Amazon Cloud OSS
  • Azure Microsoft Cloud Computing Platform Azure storage data service
  • service providers like Seven Niu Cloud Storage that provide cloud storage services.
  • cloud storage products Although there are many types of cloud storage products, they all have some commonalities in terms of functions and usage methods, but they are specifically programmed into API (Application Programming Interface) and SDK (Software Development Kit). There are also some differences and their own characteristics. In actual use, sometimes due to the relocation of the overall business, it may be switched from one cloud service provider to another. In this case, the cloud storage data needs to be migrated synchronously; or sometimes for data security. Synchronize backup data not only on multiple cloud storage products provided by the same service provider, but also back up data to cloud storage products provided by another service provider to reduce risks; or migrate data in the client file system to the same time. Among the cloud storage products provided by a cloud service provider.
  • the prior data migration technology or tools can only transmit data between cloud storage products and users provided by a certain service provider, and do not support other cloud storage products; or data in other storage systems. Migrating to a cloud storage product provided by a service provider cannot perform bidirectional migration of data between any two storage systems.
  • the present application discloses a data migration method, which includes:
  • the upper layer interfaces corresponding to the migration task are invoked, and the upper layer operation interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second system.
  • the calling corresponds to each upper layer interface of the migration task, and each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to migrate data of the first system to the second system.
  • Steps include:
  • the lower layer operation interface of the first system is invoked through the abstract interface class to acquire data from the first system;
  • the lower layer operation interface of the second system is called by the abstract interface class to synchronize the data acquired from the first system to the second system.
  • the step of calling the bottom operating interface of the first system by using the abstract interface class includes:
  • the first abstract class interface corresponding to the first upper layer interface in the abstract interface class is directly invoked as the lower layer operation interface;
  • the first upper layer interface is universal for each system. Interface;
  • the first interface class for the first system is generated by the abstract interface class, and the lower layer operation interface corresponding to each second upper layer interface is called in the first interface class.
  • the second upper layer interface is not an interface common to each system.
  • the step of calling the bottom operation interface of the second system by using the abstract interface class includes:
  • the second abstract class interface corresponding to the third upper-layer interface in the abstract interface class is directly invoked as the lower-level operation interface;
  • the third upper-layer interface is universal for each system. Interface;
  • the second interface class for the second system is generated by the abstract interface class, and the lower layer operation interface corresponding to each fourth upper layer interface is called in the second interface class
  • the fourth upper layer interface is not an interface common to each system.
  • the step of invoking the lower layer operation interface of the first system by the abstract interface class to acquire data from the first system for each upper layer interface belonging to the acquired data includes:
  • the upper layer interface for obtaining the metadata is executed by the scanner of the management component, and the upper layer interface for acquiring the metadata invokes the lower layer operation interface of the first system for acquiring the metadata through the abstract class interface, and the corresponding task of the migration task is obtained from the first system.
  • Metadata
  • the upper interface that acquires the actual data is executed by the receiver of the working component according to the received synchronization task, and the upper layer interface that obtains the actual data invokes the bottom layer acquisition interface of the first system to obtain the actual data through the abstract class interface, and obtains the first layer system from the first system.
  • the actual data of the migration task is executed by the receiver of the working component according to the received synchronization task, and the upper layer interface that obtains the actual data invokes the bottom layer acquisition interface of the first system to obtain the actual data through the abstract class interface, and obtains the first layer system from the first system.
  • the step of invoking the low-level operation interface of the second system by the abstract interface class to synchronize the data acquired from the first system to the second system for each upper-layer interface belonging to the synchronization data includes:
  • Each upper layer interface of the synchronization data is executed by a synchronizer of the working component, and each upper layer interface of the synchronization data invokes an underlying operation interface of the second system through an abstract interface class to synchronize the actual data to the second system.
  • the management component is in a server of the cluster, and the working component is in a cluster In each server.
  • the step of generating, by the distributor of the management component, the synchronization task based on each metadata and distributing to the receiver of each working component comprises:
  • the amount of data recorded in the metadata is less than the first threshold, directly distributing the metadata as a synchronization task to a receiver of a working component in a server where the management component is located;
  • the metadata is distributed as a synchronization task to the receiver of the working component in the server with the least load;
  • the step of generating, by the distributor of the management component, the synchronization task based on each metadata and distributing to the receiver of each working component comprises:
  • the amount of data recorded in the metadata is less than the first threshold, directly distributing the metadata as a synchronization task to a receiver of a working component in a server where the management component is located;
  • the amount of data recorded in the metadata is not less than the first threshold but less than the second threshold, distributing the metadata generation synchronization task to the receiver of the working component in the server with the least load;
  • the upper interface for slicing the large file is invoked, and the upper interface of the large file slice calls the second system's underlying operation on the large file slice through the abstract interface class.
  • the slice information of the different slices of the metadata is generated and distributed to the receiver of the working component of the plurality of servers with low load.
  • data is transmitted between the servers of the cluster by using a JGroups communication protocol.
  • the upper layer interface of the synchronization data is executed by the synchronizer of the working component, and each upper layer interface of the synchronization data invokes an underlying operation interface of the second system by using an abstract interface class,
  • the step of synchronizing the actual data to the second system includes:
  • the synchronization device of the working component calls the thread corresponding to the speed limit instruction to execute each upper layer interface of the synchronization data according to the speed limit instruction, and each upper layer interface of the synchronization data calls the lower layer operation interface of the second system through the abstract interface class, The actual data is synchronized to the second system.
  • the step of invoking the upper layer interfaces corresponding to the migration task, and calling the first system and the bottom layer operation interface of the second system by the upper interface interface through the abstract interface class to migrate the data of the first system to the second system it also includes:
  • the upper layer interfaces corresponding to the migration task are repeatedly executed, and the upper layer interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second interface.
  • the steps of the system until the number of retries reaches the retry threshold or succeeds within the retry threshold.
  • calling the first system and the bottom layer operation interface of the second system through the abstract interface class to migrate the data of the first system to the second system further including:
  • the data of the first system is suspended from being migrated to the second system, and the migrated data location in the first system at the time of the pause is recorded;
  • the method further includes:
  • the upper layer interfaces corresponding to the migration task are recalled according to the migrated data position in the first system at the time of the pause, and the lower layer operation interfaces of the first system and the second system are invoked through the abstract interface class.
  • the number in the first system is received before the steps of migrating to the migration task of the second system, it also includes:
  • An application is installed in the client that emulates each of the underlying operational interfaces for the specified file.
  • the calling corresponds to each upper layer interface of the migration task, and each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to migrate data of the first system to the second system.
  • each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to migrate data of the first system to the second system.
  • each upper layer interface corresponding to the migration task is invoked, and each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to synchronize the change data of the client file system to the first Two systems.
  • the method further includes:
  • the method further includes:
  • the upper layer interface of the signature authentication is invoked, and the lower layer interface of the signature authentication of the second system is called by the upper layer interface of the signed authentication to perform signature verification on the data.
  • the upper layer interfaces corresponding to the migration task are invoked, and the upper layer operation interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second system.
  • the step it also includes:
  • the migration task is sent by a client, where the migration task includes identifier information of the first system, identity information of the user in the first system, file information of the data to be migrated in the first system, and identifier of the second system. Information, the identity information of the user in the second system.
  • the method further includes:
  • the upper layer interface of the first system is called, and the upper layer interface of the first layer of the first system is used to verify the identity information of the user in the first system.
  • the upper layer interface of the second system is called for the second system, and the upper layer interface of the authentication system calls the lower layer operation interface of the second system through the abstract interface class to verify the identity information of the user in the second system.
  • the present application also discloses a data migration device, which includes:
  • a task receiving module configured to receive a migration task of migrating data in the first system to the second system
  • a migration module configured to invoke each upper-layer interface corresponding to the migration task, and invoke, by each upper-layer interface, an underlying operation interface of the first system and the second system through an abstract interface class, to migrate data of the first system to the second system .
  • the migration module includes:
  • the upper layer interface determining submodule is adapted to determine each upper layer interface belonging to the acquired data, and the genus For each upper layer interface of the synchronization data;
  • a data acquisition sub-module configured to invoke, by the abstract interface class, an underlying operation interface of the first system to acquire data from the first system for each upper-layer interface belonging to the acquired data;
  • the data synchronization submodule is adapted to invoke the lower layer operation interface of the second system through the abstract interface class for each upper layer interface belonging to the synchronization data to synchronize the data acquired from the first system to the second system.
  • the data acquisition submodule includes:
  • the first low-level operation interface determining sub-module is adapted to directly invoke the first abstract class interface corresponding to the first upper-layer interface in the abstract interface class as the first upper-level interface in each of the upper-layer interfaces belonging to the acquired data.
  • the first upper layer interface is a common interface of each system;
  • the first lower layer operation interface calls the submodule, and is adapted to generate, by the abstract interface class, a first interface class for the first system, and call the first interface class in the second upper layer interface of each upper layer interface that belongs to the acquired data.
  • An underlying operation interface corresponding to each second upper layer interface; the second upper layer interface is not an interface common to each system.
  • the data synchronization submodule comprises:
  • the second lower layer operation interface determining submodule is adapted to directly invoke the second abstract class interface corresponding to the third upper layer interface in the abstract interface class as the lower layer operation interface for the third upper layer interface in each upper layer interface belonging to the synchronization data.
  • the third upper layer interface is a common interface of each system;
  • the second lower layer operation interface calls the submodule, and is adapted to generate, by the abstract interface class, a second interface class for the second system, for the fourth upper layer interface among the upper layer interfaces belonging to the synchronization data, where the second interface class is called.
  • An underlying operation interface corresponding to each fourth upper layer interface; the fourth upper layer interface is not an interface common to each system.
  • the data acquisition submodule includes:
  • a metadata acquisition submodule configured to execute, by a scanner of the management component, an upper layer interface for acquiring metadata, wherein the upper layer interface for obtaining the metadata invokes an underlying operation interface of the first system for acquiring metadata through the abstract class interface, from the first The system acquires each metadata corresponding to the migration task;
  • a task distribution sub-module adapted to be generated by a distributor of the management component, based on each metadata, and distributed to a receiver of each working component;
  • the actual data acquisition sub-module is adapted to perform, by the receiver of the working component, the upper layer interface for acquiring the actual data according to the receiving the synchronization task, where the upper layer interface that obtains the actual data invokes the bottom layer of the first system to obtain the actual data through the abstract class interface. Obtain an interface, and obtain actual data of the migration task from the first system.
  • the data synchronization submodule comprises:
  • An actual data synchronization sub-module configured to execute, by the synchronizer of the working component, each upper-layer interface of the synchronization data, where each upper-layer interface of the synchronization data invokes an underlying operation interface of the second system through an abstract interface class, and synchronizes the actual data to The second system.
  • the management component is in one server of the cluster, and the working component is in each server of the cluster.
  • the task distribution submodule includes:
  • a data size extraction submodule adapted to extract the amount of data recorded in the metadata
  • a first metadata distribution submodule configured to directly distribute the metadata as a synchronization task to a receiver of a working component in a server where the management component is located if the amount of data recorded in the metadata is less than a first threshold
  • the second metadata distribution sub-module is adapted to distribute the metadata as a synchronization task to a receiver of a working component in a server with the least load if the amount of data recorded by the metadata is not less than the first threshold.
  • the task distribution submodule includes:
  • a data size extraction submodule adapted to extract the amount of data recorded in the metadata
  • a first metadata distribution submodule configured to directly distribute the metadata as a synchronization task to a receiver of a working component in a server where the management component is located if the amount of data recorded in the metadata is less than a first threshold
  • a third metadata distribution submodule adapted to distribute the metadata generation synchronization task to a working component in a server with the least load if the amount of data recorded in the metadata is not less than a first threshold but less than a second threshold receiver;
  • a metadata cutting molecular module adapted to call an upper layer interface for a large file slice if the amount of data recorded in the metadata is not less than a second threshold, wherein the upper layer interface for the large file slice calls the second system through the abstract interface class The underlying operation interface for the large file slice, the large file corresponding to the metadata is sliced;
  • the slice information of the different slices of the metadata is generated and distributed to the receiver of the working component of the plurality of servers with low load.
  • data is transmitted between the servers of the cluster by using a JGroups communication protocol.
  • the actual data synchronization submodule comprises:
  • the first actual data synchronization sub-module is adapted to be executed by the synchronizer of the working component according to the speed limit instruction, and the thread corresponding to the speed limit instruction is used to execute each upper layer interface of the synchronization data, and each upper layer interface of the synchronization data is called by the abstract interface class.
  • the underlying operational interface of the second system synchronizes the actual data to the second system.
  • the method further includes:
  • the migration determination module is adapted to determine whether the migration of the data in the first system to the second system is successful; if it fails, the migration module is repeatedly executed until the number of retries reaches the retry threshold or succeeds within the retry threshold.
  • the method further includes:
  • the pause command judgment module is adapted to determine whether a pause command is received; if a pause is received The instruction enters the pause recording sub-module;
  • a pause recording module adapted to suspend migration of data of the first system to the second system, and record the migrated data location in the first system at the time of suspension;
  • the method further includes:
  • the restart instruction determining module is adapted to determine whether a restart command is received; if the restart command is received, the migration module is re-entered according to the migrated data position in the first system at the time of the pause.
  • the method further includes:
  • An application installation module adapted to install an application in a client that simulates each of the underlying operational interfaces for a specified file.
  • the method further includes:
  • the data monitoring module is adapted to monitor whether the client file system changes with the data that was migrated to the second system last time; if the change occurs, enter the change data synchronization module;
  • the change data synchronization module is adapted to invoke each upper layer interface corresponding to the migration task, and each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to synchronize the change data of the client file system. To the second system.
  • the method further includes:
  • a signature submodule adapted to invoke an upper layer interface of the signature, and invoke an underlying operation interface of the signature of the first system by the upper interface of the signature to sign the data;
  • the method further includes:
  • the signature verification sub-module is adapted to invoke an upper-layer interface of the signature authentication, and the upper-layer interface of the signature authentication is called by the upper-layer interface of the signed authentication to perform signature verification on the data.
  • the method further includes:
  • a first file list obtaining module configured to invoke an upper layer interface of the file list for the first system, and invoke an upper layer operation interface of the first system by an upper interface of the calling file list to obtain a file list of the first system ;
  • a second file list obtaining module configured to invoke an upper layer interface of the file list for the second system, and invoke an upper layer operation interface of the second system by the upper interface of the calling file list to obtain a file list of the second system ;
  • the difference prompting module is adapted to compare the file list of the first system with the file list of the second system, and prompt the difference between the two to the client.
  • the migration task is sent by a client, where the migration task includes identifier information of the first system, identity information of the user in the first system, file information of the data to be migrated in the first system, and identifier of the second system. Information, the identity information of the user in the second system.
  • the method further includes:
  • the first identity verification module is adapted to invoke an upper layer interface for the first system, and the upper layer interface of the identity authentication invokes an underlying operation interface of the first system by using an abstract interface class to identify identity information of the user in the first system. authenticating;
  • a second identity verification module configured to invoke an upper layer interface for the second system, and the upper layer interface of the identity verification system invokes an underlying operation interface of the second system by using the abstract interface class to identify the identity information of the user in the second system authenticating.
  • a migration task of migrating data in the first system to the second system is received; then each upper layer interface corresponding to the migration task is invoked, and the first system and the second system are invoked by the upper layer interface through the abstract interface class.
  • any data terminal including cloud storage products that have not yet supported data migration, such as Baidu object storage, Jinshan object storage, and bidirectional transmission migration of data with any other data end.
  • FIG. 1 is a flow chart showing the steps of an embodiment of a data migration method of the present application
  • 1A is a schematic diagram of data migration performed by five systems of the present application.
  • FIG. 1B is a schematic diagram of a data migration process interface call of five systems of the present application.
  • FIG. 2 is a flow chart showing the steps of an embodiment of a data migration method of the present application.
  • 2C is a schematic diagram of a data migration structure of the present application.
  • 2D is a schematic diagram of a distributed mode data migration data flow of the present application.
  • FIG. 3 is a structural block diagram of an embodiment of a data migration apparatus of the present application.
  • FIG. 4 is a structural block diagram of an embodiment of a data migration apparatus of the present application.
  • One of the core concepts of the embodiments of the present application is to receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the plurality of systems, and then invoke Corresponding to the upper layer interfaces of the migration task, and the upper layer interfaces invoke the bottom layer operation interfaces of the first system and the second system by the upper interface interfaces to migrate the data of the first system to the second system.
  • any two data ends can be used for bidirectional data transmission and migration, including inter-bucket migration within the data end.
  • the data migration technology described in the present application has a good extension function, and can implement any data terminal, including a cloud storage product that does not support data migration, such as Baidu object storage, Jinshan object storage, and any other data terminal. Two-way transfer of data.
  • FIG. 1 a flow chart of steps of an embodiment of a data migration method of the present application is shown, which may specifically include the following steps:
  • Step 110 Receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the plurality of systems.
  • One method of data migration of the present application is a method of migrating data in one system to another system, for any two systems of multiple systems.
  • the system may be a storage server, or a storage space, a cloud storage product, etc., such as a client file system (Local), AWS (Amazon Web Services, Amazon Cloud Services) S3, Facebook Cloud OSS (Object Storage Service) , object storage service), Microsoft cloud computing platform Azure storage data service (Blob), Qi Niuyun storage (QiNiu) and so on.
  • FIG. 1A is a schematic diagram of data migration performed by the foregoing five systems. It can be seen that any two of the multiple systems can perform bidirectional data transmission and migration, including data migration between different buckets within the system.
  • Bucket is a storage space used by users to manage stored objects, similar to the root directory or disk partition in the file system.
  • the data migration method of the present application has a good extended function, and other cloud storage products that have not been added to the five systems shown in FIG. 1A, such as Baidu Object Storage (BOS) and Kingsoft Standard Storage Service (Kingsoft Standard Storage Service, KS3), etc., can also be easily added, and can be bidirectionally transmitted and migrated with any other system.
  • BOS Baidu Object Storage
  • Kingsoft Standard Storage Service Kingsoft Standard Storage Service
  • the first system and the second system can be two systems of the same type, for example, the OSS of Facebook Cloud, then the data of one OSS is migrated to another OSS, such as the data in the OSS of a region. Migrate to OSS in another region; of course, the first system and the second system can also be two different types of systems, such as OSS and Blob respectively, then the OSS data is migrated to the Blob, or it can be For two different in the same system The storage area, then the data of one storage interval in the same system is migrated to another storage interval.
  • the migration data for example, whether to migrate all the data in the first system to the second system, or only a part of the migration, it can be set in the migration task.
  • the server receiving the data migration task can obtain the above information from the migration task.
  • the migration task is sent by a client, where the migration task includes identification information of the first system, identity information of the user in the first system, and data to be migrated in the first system.
  • the identification information of the first system refers to information that can uniquely identify the first system, such as the access domain name of the first system, and the identity information of the user in the first system refers to the identity information of the current first system user, such as the username.
  • the identification information of the second system refers to information that can uniquely identify the second system, such as the IP address of the second system, and the identity information of the user in the second system refers to the identity information of the current second system user, such as the user name.
  • the file information of the data to be migrated in the first system may include information such as a name, a storage location, a data size, and the like of the file to be migrated in the first system.
  • the embodiment of the present application may provide a migration configuration page in the client, where the user may configure related information as the identification information of the first system of the migrated object, and the user in the first system.
  • the migration object ID is all, and all the data of the first system to be migrated.
  • you can configure the file or The storage path of the several files in the first storage system is identified as a migration object. Of course, you can also configure other requirements information.
  • some storage systems are sub-areas. For example, if there is a storage center in Beijing and a storage center in Shanghai, you can also configure the storage system. Region.
  • the specific configuration items may be set according to requirements, which are not limited in the embodiment of the present application.
  • step 120 the upper layer interfaces corresponding to the migration task are invoked, and the upper layer interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second system.
  • multiple servers may be connected to form a large processing cluster.
  • the entire cluster has a master server and a plurality of slave servers.
  • the master receives the migration task of migrating data from the first system to the second system, and then receives the data through a reasonable distribution policy.
  • the migration task to the distribution is distributed to Slave for processing. After the Slave processes the received migration task, the migration result can be fed back to the Master.
  • the embodiment of the present application can also implement data migration by using a single server.
  • the preferred number of applications herein samples the aforementioned clusters.
  • FIG. 1B is a schematic diagram of a data migration process interface call to the foregoing five systems.
  • the Peer interface defines the functions that each specific system for data migration must have. For example, get(), put(), respectively, refers to downloading and uploading data, which is an interface that needs to be called to perform actual data transmission;
  • mkRoot(), rmRoot() Create and delete the root directory.
  • the object of operation is the bucket in the system; the bucket storage space is a storage space used by the user to manage the stored Object, similar to the root directory or disk partition in the file system.
  • getObj(), getAllObjs() Get a single metadata, get all the metadata, mainly use these interfaces to get the metadata of the data to be transmitted; Metadata Metadata as the descriptive data of the actual stored data, only contains some basic information For example, the key, data size, timestamp, etc. that identify the uniqueness of the data.
  • beginPartPut(), partPut(), endPartPut() mainly handles the resume transmission of large points and the slice transfer function of large files;
  • Rm(), rmAll() Deletes a single data and deletes all data.
  • getAllObjs() for some upper-level interfaces, such as getAllObjs(), rmAll(), their implementation is implemented by calling other underlying operation interfaces such as getObj(), rm(), which is true for any system that performs data migration. It is generic and can be implemented in the AbstractPeer layer shown in Figure 2B.
  • the underlying operation interface such as the underlying operation interface formed by Local, OSS, S3, QiNiu, and Blob in FIG. 1B, may be implemented in the layer of AbstractPeer. Therefore, each upper layer interface corresponding to the migration task can be invoked, and the upper layer operation interface of the first system and the second system is invoked by each upper layer interface through the abstract interface class to migrate the data of the first system to the second system.
  • each upper interface corresponding to the migration task may be invoked, including an upper interface for acquiring data in the Local and used to synchronize the acquired data to the OSS.
  • the upper interface in the middle, and then the upper interface of the data in the Local can be firstly invoked by the corresponding abstract interface class corresponding to the GetObj() interface and the get() interface in the Local to obtain the migration data in the Local, and then
  • the upper interface for synchronizing the acquired data to the OSS sequentially calls the put() interface in the OSS through the corresponding abstract interface class corresponding to the OSS to transfer the migrated data acquired from the Local to the OSS.
  • An abstract interface class layer AbstractPeer is constructed.
  • the abstract interface class layer can inherit the upper interface layer.
  • the AbstractPeer can be inherited, so that the process of migrating data between the respective systems can be realized.
  • the abstract interface class layer AbstractPeer for the new storage system, only the underlying operation interface layer of the lower layer needs to inherit the abstract interface class layer AbstractPeer, so that the new storage system and any other storage system before can be realized.
  • step 120 includes:
  • Sub-step 121 determining each upper-layer interface belonging to the acquired data, and each upper-layer interface belonging to the synchronous data.
  • the upper layer interface that is used to obtain data from the first system is an upper layer interface that belongs to the acquired data
  • the upper layer interface that synchronizes the acquired data to the second system is an upper layer interface that belongs to the synchronous data.
  • the upper layer interface that acquires the data and the upper layer interface that belongs to the synchronous data may be one or more upper layer interfaces, which is not limited in this embodiment.
  • the migration task includes the identification information of the first system from which the data is to be migrated, and the file information of a file to be migrated.
  • the upper interface getObj in the peer of FIG. 1B needs to be called.
  • the user identity information of the first system such as the account name and password of the user in the first system, and the upper layer interface for performing identity verification may also be included.
  • the second system since the file is migrated from the first system, usually the second system does not have a corresponding folder, then the mkRoot() in the peer of FIG. 1B needs to be called to create a directory, and the peer of FIG. 1B.
  • the put() upload data in .
  • mkRoot() and put() belong to the upper interface of the synchronous data.
  • the user identity information of the second system such as the account name and password of the user in the second system, and the upper layer interface for performing identity verification may also be included.
  • Sub-step 122 For each upper-layer interface belonging to the acquired data, the underlying operation interface of the first system is invoked through the abstract interface class to acquire data from the first system.
  • the upper layer interface invokes the underlying operation interface of the first system through the abstract interface class to migrate the data of the first system to the second system.
  • the process of migrating to the second system includes obtaining data from the first system.
  • the upper layer interface belonging to the acquired data may be utilized, and the underlying operation interface of the first system is invoked through the abstract interface class to acquire data from the first system.
  • the server in the embodiment of the present application may send the identifier information of the first system as one of the parameters to the upper layer interface for acquiring data, and then the upper layer interface invokes the abstract interface class, and the abstract interface class may go according to the identifier information. Call the corresponding underlying operational interface of the first system.
  • sub-step 122 includes:
  • Sub-step 1221 for the first upper-layer interface in each upper-layer interface that belongs to the acquired data, the first abstract class interface corresponding to the first upper-layer interface in the abstract interface class is directly invoked as the lower-level operation interface; the first upper-layer interface A common interface for each system.
  • an interface common to each system included in each upper layer interface that acquires data for example, getAllObjs(), rmAll(), may be referred to as a first upper layer interface.
  • Their implementation is implemented by calling other low-level operation interfaces. Therefore, for the first upper-level interface in each upper-layer interface that belongs to the acquired data, the first abstract class interface corresponding to the first upper-layer interface in the abstract interface class can be used as the bottom layer. The interface is manipulated to be called directly.
  • the first abstract class interface getAllObjs() in the abstract interface class is implemented by calling the underlying operation interface getObj(), so that the first abstract class interface getAllObjs can be directly As the underlying operation interface call, it is equivalent to calling the underlying operation interface getObj().
  • the server of the embodiment of the present application may send the identifier information of the first system as one of the parameters to the upper layer interface for acquiring the data, and then the upper layer interface invokes the abstract interface class, if the abstract interface class has the first abstract class corresponding to the upper layer interface.
  • the interface, the first abstract class interface can be executed according to parameters such as the identification information.
  • Sub-step 1222 for the second upper layer in each upper layer interface belonging to the acquired data a first interface class for the first system is generated by the abstract interface class, and an underlying operation interface corresponding to each second upper layer interface is invoked in the first interface class; the second upper layer interface is not common to each system interface.
  • each upper layer interface common to each system it can be called a second upper layer interface.
  • the first interface class for the first system needs to be generated through the abstract interface class, and then called in the first interface class.
  • the lower layer operation interface corresponding to each second upper layer interface.
  • the server of the embodiment of the present application may send the identifier information of the first system as one of the parameters to the upper interface of the data, and then the upper interface invokes the abstract interface class. If the abstract interface class does not have the first abstract class corresponding to the upper interface. The interface generates an interface class corresponding to the first system according to the identifier information, and then invokes the bottom interface corresponding to the upper interface in the first interface class.
  • Sub-step 123 for each upper-layer interface belonging to the synchronization data, the lower-layer operation interface of the second system is called by the abstract interface class to synchronize the data acquired from the first system to the second system.
  • the upper layer interface invokes the underlying operation interface of the first system through the abstract interface class to migrate the data of the first system to the second system.
  • the process of migrating data of the first system to the second system includes synchronizing data acquired from the first system to the second system.
  • each of the upper layer interfaces belonging to the synchronization data may be utilized, and the underlying operation interface of the second system is invoked through the abstract interface class to synchronize the data acquired from the first system to the second system.
  • the server in the embodiment of the present application may send the identifier information of the second system as one of the parameters to the upper interface of the data, and then the upper interface invokes the abstract interface class, if the upper interface does not correspond to the upper interface.
  • the first abstract class interface, the abstract interface class can call the corresponding underlying operation interface of the second system according to the identification information.
  • the embodiment of the present application may directly The substantial data is transmitted to the storage interface of the synchronization data of the second system, so that the migration between the buckets inside the two storage systems can be realized. For example, after calling the first system's get() to obtain the data, the data is directly transmitted to the second system's put(), and the processing is performed in the memory, and the data is directly migrated to the second system. Will not be stored to the local disk.
  • the embodiment of the present application may set a first function to monitor whether get() obtains data, and if the data is obtained, the data is passed to put().
  • a transmission pipeline is constructed between the underlying interface that acquires the data and the underlying operation of the synchronous data, and is used to transmit two types of interfaces.
  • the data between them, after each data transfer, is not stored locally and does not accumulate in memory.
  • sub-step 123 includes:
  • Sub-step 1231 for the third upper-layer interface in each of the upper-layer interfaces belonging to the synchronization data, the second abstract class interface corresponding to the third upper-layer interface in the abstract interface class is directly invoked as the lower-level operation interface; the third upper-layer interface A common interface for each system.
  • the upper layer interface common to each system in each upper layer interface belonging to the synchronous data is referred to as a third upper layer interface
  • the second abstract class interface corresponding to the third upper layer interface in the abstract interface class is directly called as the lower layer operation interface.
  • the server of the embodiment of the present application may send the identifier information of the second system as one of the parameters to the upper layer interface for acquiring the data, and then the upper layer interface invokes the abstract interface class, and if the abstract interface class has the second abstract class corresponding to the upper layer interface.
  • the interface, the second abstract class interface can be executed according to the identifier information and other parameters. This step is similar to the previous step 1221, and details are not described herein again.
  • Sub-step 1232 for the fourth upper-layer interface in each of the upper-layer interfaces belonging to the synchronization data, generating a second interface class for the second system by using the abstract interface class, and calling the fourth upper-layer interface corresponding to the second interface class in the second interface class
  • the lower layer operation interface; the fourth upper layer interface is not an interface common to each system.
  • an upper layer interface that is not common to each system among the upper layer interfaces belonging to the synchronization data is referred to as a fourth upper layer interface
  • a second interface class for the second system is generated by the abstract interface class, in the second interface class.
  • the underlying operation interface corresponding to each fourth upper layer interface is called.
  • the server in the embodiment of the present application may send the identifier information of the second system as one of the parameters to the upper layer interface for acquiring the data. If the abstract interface class does not have the second abstract class interface corresponding to the upper layer interface, then the upper layer The interface then calls the abstract interface class, and the abstract interface class can call the corresponding underlying operation interface of the second system according to the identification information. This step is similar to the previous step 1222, and details are not described herein again.
  • step 120 the method further includes:
  • Step 130 Determine whether the migration of data in the first system to the second system is successful; if it fails, repeat step 120 until the number of retries reaches the retry threshold or succeeds within the retry threshold.
  • the data migration process may be unsuccessful, and the data migration is unsuccessful. Therefore, in the embodiment of the present application, it is determined whether the data in the first system is migrated to the second system.
  • the successful migration means that all the data that needs to be migrated in the migration instruction is migrated to the second system. Specifically, it can be determined whether the size of the data that has been migrated is equal to the data size set in the migration instruction, or whether the data that has been migrated and the data set in the migration instruction are consistent with other existing available technologies.
  • the embodiment is not limited.
  • determining whether the data in the first system is migrated to the second system is successful may be performed in various manners, for example, by monitoring the success of the first system downloading notification, and monitoring the success of the second system. If you receive any unsuccessful notifications, the migration is confirmed to have failed.
  • the embodiments of the present application do not limit them.
  • a retry threshold may be set, and if the number of retry attempts exceeds the retry threshold and the migration is not successfully performed, the current migration task is abandoned, and then the migration may be performed. The reason for the unsuccessful move.
  • the retry threshold may be set according to requirements, which is not limited in this embodiment of the present application.
  • step 120 the method further includes:
  • step 140 it is determined whether a pause command is received; if a pause command is received, then step 150 is entered.
  • the pause command can be sent to the server executing the migration instruction. Therefore, it can be judged whether or not a pause command is received.
  • Step 150 Suspend migration of data of the first system to the second system, and record the migrated data location in the first system at the time of suspension.
  • the data of the first system is migrated to the second system, and at the same time, in order to avoid moving the data of the first system to the second system again, starting from the initial situation,
  • the data that has been migrated to the second system in one system is again migrated to the second system again, thereby wasting time, and the migration of the first system in the first system can be recorded while the data of the first system is suspended to the second system.
  • the data location to continue the data migration starting from the migrated data location.
  • the suspended upper layer interface may also be invoked, and then the abstract interface class respectively invokes the suspended underlying operation interface of the first system and the suspended underlying operation interface of the second system to implement the migration suspension process. .
  • step 160 it is determined whether a restart command is received; if a restart command is received, the process proceeds to step 120 based on the migrated data position in the first system at the time of the pause.
  • the upper layer interface corresponding to the migration task is re-executed according to the migrated data position in the first system at the pause time. And the upper layer operation interface of the first system and the second system is called by each upper layer interface through the abstract interface class, so that the number of the first system is According to the migration to the second system.
  • the data of the first system refers to data in the first system that has not been migrated to the second system.
  • the restarted upper layer interface may also be invoked, and then the abstract interface class respectively invokes the restarted underlying operation interface of the first system and the restarted underlying operation interface of the second system to implement the migration suspension.
  • the method when the first system is a client file system, before step 110, the method further includes:
  • Step 100 Install an application in the client, the application simulating each underlying operation interface for the specified file.
  • the present application can provide services for the client by installing an application in the client. Therefore, in the embodiment of the present application, if the first system is a client file system, an application is first installed in the client, and the application simulates each underlying operation interface for the specified file, as shown in local in FIG. 1B. Each underlying operational interface. In turn, the client user can access each of the underlying operational interfaces that can edit the specified file corresponding to the application by accessing the application.
  • step 120 the method further includes:
  • Step 170 Monitor whether the client file system changes with the data that was migrated to the second system last time; if a change occurs, proceed to step 180.
  • the client application may be logged in multiple times by multiple users, and during the login process, the data in the client file system may be modified. If the data in the client file system is changed, the corresponding The location needs to be updated to the second system. So, for the client file system, you can monitor whether the client file system is migrated with the most recent one. The data to the second system is changed. If a change occurs, the change needs to be updated to the second system. Specifically, each upper layer interface corresponding to the migration task may be invoked, and each upper layer interface invokes the abstract interface class. The underlying operational interface of a system and a second system to synchronize the change data of the client file system to the second system.
  • Step 180 Calling each upper layer interface corresponding to the migration task, and calling, by each upper layer interface, an underlying operation interface of the first system and the second system through an abstract interface class, to synchronize the change data of the client file system to the second system.
  • the method further includes:
  • Sub-step 124 the upper interface of the signature is invoked, and the underlying operation interface of the signature of the first system is invoked by the upper interface of the signature to sign the data.
  • the migration data in the first system is completely migrated to the second system, and may be first
  • the data in the system that needs to be migrated is signed to identify the data that needs to be migrated to the second system.
  • the upper interface of the signature may be invoked, and the underlying operation interface of the signature of the first system is invoked by the upper interface of the signature to sign the data.
  • the data may be signed in the first system, and the data may be signed when the data is moved out of the first system and not synchronized to the second system.
  • the method further includes:
  • Sub-step 125 the upper layer interface of the signature authentication is invoked, and the underlying operation interface of the signature authentication of the second system is called by the upper layer interface of the signed authentication, and the data is signature verified.
  • the migrated data can be signature verified. If the signature verification is successful, it indicates that the data that has been migrated to the second system is consistent with the data that needs to be migrated to the second system in the first system, and if the signature verification is unsuccessful, for example, part of the data that has been migrated to the second system. If the signature does not correspond to the signature in the first system, it indicates that the migration fails and can be repeated. Step 120, until the number of retries reaches the retry threshold or succeeds within the retry threshold.
  • the upper layer interface of the signature authentication may be invoked, and the lower layer interface of the signature authentication of the second system is invoked by the upper layer interface of the signed authentication to perform signature verification on the data.
  • step 120 the method further includes:
  • step 190 the upper layer interface of the file list is called for the first system, and the upper layer interface of the calling file list is invoked by the abstract interface class to call the bottom operating interface of the first system, and the file list of the first system is obtained.
  • step 1110 the upper layer interface of the file list is called for the second system, and the upper layer interface of the calling file list is called by the abstract interface class to call the bottom layer operation interface of the second system, and the file list of the second system is obtained.
  • Step 1120 Compare the file list of the first system with the file list of the second system, and prompt the difference between the two to the client.
  • the method further includes:
  • Sub-step 126 the upper layer interface of the first system is called by the upper layer interface of the first system, and the upper layer interface of the first system is used to verify the identity information of the user in the first system.
  • Sub-step 127 the upper layer interface of the second system is called by the upper layer interface of the second system, and the upper layer interface of the second system is called by the abstract interface class to verify the identity information of the user in the second system.
  • a migration task of migrating data in the first system to the second system may be received, where the first system and the second system are any two of the plurality of systems, and then the corresponding The upper layer interfaces of the task are migrated, and the upper layer operation interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second system.
  • the data migration technology described in the present application has a good extension function, and can implement any data terminal, including a cloud storage product that does not support data migration, such as Baidu object storage, Jinshan object storage, and data data of any other data end. Two-way transfer migration.
  • FIG. 2 a flow chart of steps of an embodiment of a data migration method of the present application is shown, which may specifically include the following steps:
  • Step 210 Receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the plurality of systems.
  • the user can select any two of them, one of which is the first system and the other as the second system.
  • Step 220 Determine each upper layer interface belonging to the acquired data, and each upper layer interface belonging to the synchronous data.
  • Step 230 Perform, by the scanner of the management component, an upper layer interface for acquiring metadata, where the upper layer interface for acquiring the metadata invokes an underlying operation interface of the first system for acquiring metadata through the abstract class interface, and the migration is obtained from the first system.
  • the metadata corresponding to the task.
  • FIG. 2C is a schematic diagram of a data migration structure of the present application. It consists of two separate components: Master and Slave. Master is responsible for scanning data sources and task distribution, and Slave is responsible for receiving tasks and performing real data migration.
  • the Master includes a scanner (Scanner) and a distributor (Dispatcher); the slave includes a receiver (Receiver) and a synchronizer (Syncer).
  • the master composed of the scanner and the distributor may be a management component, and the slave composed of the receiver and the synchronizer is a working component.
  • the architecture can operate in both stand-alone environments as well as distributed data transfers. In stand-alone mode, Master and Slave serve as two components in a process at the same time.
  • the preferred sampling clustering manner of the embodiment of the present application is a server with Master+Slave and each server with Slave.
  • Metadata Queue The scanner is responsible for traversing the metadata of the data source and storing it in the queue "Metadata Queue".
  • Metadata only contains some basic information, such as the key that identifies the uniqueness of the data, the size of the data, and the timestamp.
  • the Metadata Queue may be a blocking queue, and the difference between the blocking queue and the normal queue is that when the queue is empty, the operation of acquiring data from the queue will be blocked, or when When the queue is full, the operation of adding data to the queue will be blocked.
  • threads that attempt to add new data to a full blocking queue are also blocked until other threads make the queue free again, such as removing one or more data from the queue, or completely emptying the queue.
  • the metadata is first stored in the blocking queue in order until the blocking queue is full, and then the metadata in the blocking queue is sequentially removed and synchronized to the second system, and each time a metadata is removed, correspondingly A metadata can be extracted from the first system and stored in the blocking queue, so that the speed of data migration can be effectively balanced.
  • the scanner of the management component may execute an upper layer interface for acquiring metadata, and then the upper layer interface for obtaining the metadata may invoke the bottom layer operation interface of the first system to obtain the metadata through the abstract class interface, from the first system. Get the metadata corresponding to the current migration task.
  • the dispatcher is responsible for extracting the metadata from Scanner's Metadata Queue and dispatching the metadata as a task to the Receiver's Receiver (Receiver) according to the distribution algorithm. Every time a task is distributed, it will be recorded. After the Slave completes a task, it will report to the Master for summary. The Master can know the task number and server load of all the Slave nodes in the current cluster in real time.
  • the task in the stand-alone mode, can be directly distributed to the slave of the machine;
  • the Dispatcher distribution algorithm in the mode can be as follows:
  • the data is less than 1MB, it is distributed directly to the server's slave. Because for small data, distributed to other servers through the network, the total time spent is not as direct as the implementation of this server.
  • the server receiver is responsible for receiving tasks distributed from the Dispatcher, storing them in their own Metadata Queue, and waiting for processing.
  • the component that the synchronizer actually performs for data transfer. It continuously retrieves the metadata of the data to be transferred from Receiver's Metadata Queue and puts it into a different worker thread pool to perform the transfer task.
  • the work pool thread is a multi-threaded processing form, the task is added to the queue during processing, and then these tasks are automatically started after the thread is created.
  • Thread pool threads are background threads. The benefits of using a work pool thread include: reducing the time spent creating and destroying threads and the consumption of system resources; if you do not use a thread pool, it may cause the system to create a large number of threads resulting in excessive system memory consumption and need to be in different threads. "Over-switching" between.
  • there may be two Work Thread Pools one is responsible for processing relatively small data, and one time can complete the transmission of files corresponding to the entire metadata.
  • the other is responsible for processing relatively large data, such as receiving large slice data corresponding to large metadata.
  • a threshold may be preset. If the metadata is smaller than the threshold, the metadata is processed by a working thread pool responsible for processing relatively small data. If the metadata is not less than the threshold, the processing is relatively large.
  • the worker thread pool of data processes the metadata. Of course, there may be only one working thread pool, and the entire data is processed uniformly. For example, if the large file is not sliced, the embodiment of the present application is not limited.
  • FIG. 2D is a schematic diagram of a distributed mode data migration data flow.
  • the migration of data in cloud storage A to cloud storage B can be divided into the following four steps:
  • Scanner scans the data source "Cloud Storage A” to obtain metadata.
  • Dispatcher obtains metadata from Scanner and dispatches it to Slave's Receiver according to the distribution algorithm.
  • the management component is in a server of the cluster, and the work component is in each server of the cluster.
  • data is transmitted between the servers of the cluster using the JGroups communication protocol.
  • JGroups is written in pure Java and is a reliable message dissemination toolkit. It can be used to easily build a multi-node communication cluster, and each node can communicate with each other and cooperate. With JGroups, users can build a peer-to-peer network or a master-Slave cluster mode.
  • the distributed cluster of the present application adopts the Master-Slave mode.
  • JGroups' discovery protocol you only need to configure the IP+ port address of the master node in each node in advance, and then agree on a unified cluster name.
  • each Slave node is started in turn, so that each Slave node can find the Master node and automatically join the cluster. If a slave node exits, the FD (Failure Detection) protocol of JGroups can automatically detect the exited Slave node and remove the node information from the entire cluster.
  • FD Fallure Detection
  • JGroups provides two message propagation mechanisms:
  • the message of the byte stream is directly transmitted.
  • One node can send bytes to another node (byte) stream data. This mode generally requires the sender to serialize the data to be transmitted into a byte array, and then deserialize the data back to the receiver.
  • Step 240 Generate, by the distributor of the management component, a synchronization task based on each metadata and distribute it to the receiver of each working component.
  • the synchronization task is a type of data migration task.
  • the synchronization task may also include the identification information of the first system, the identity information of the user in the first system, the file information of the data to be migrated in the first system, and the identification information of the second system.
  • the data to be migrated refers to metadata.
  • step 240 includes:
  • Sub-step 241 extracts the amount of data recorded in the metadata.
  • the parameter of the size of the data amount can be included in each metadata, and therefore, the amount of data recorded therein can be directly extracted from the metadata.
  • Sub-step 242 if the amount of data recorded in the metadata is less than the first threshold, the metadata is directly distributed as a synchronization task to the receiver of the working component in the server where the management component is located.
  • the first threshold may be set according to requirements, and the embodiment of the present application is not limited.
  • the Dispatcher will be distributed directly to the server's Slave Receiver. Because for small data, distributed to other servers through the network, the total time spent is not as direct as the implementation of this server.
  • Sub-step 243 if the amount of data recorded by the metadata is not less than the first threshold, the metadata is distributed as a synchronization task to the receiver of the working component in the server with the least load.
  • the Metadata is distributed as a synchronization task to the receiver of the working component in the least loaded server.
  • the Dispatcher distributes the data to the Receiver of the slave of the other server.
  • step 240 includes:
  • Sub-step 241 extracts the amount of data recorded in the metadata.
  • Sub-step 242 if the amount of data recorded in the metadata is less than the first threshold, the metadata is directly distributed as a synchronization task to the receiver of the working component in the server where the management component is located.
  • Sub-step 243 if the amount of data recorded in the metadata is not less than the first threshold but less than the second threshold, then the metadata generation synchronization task is distributed to the receiver of the working component in the server with the least load.
  • a second threshold may be additionally set.
  • the second threshold may be set according to requirements, but the second threshold must be greater than
  • the first threshold is not limited in this embodiment of the present application.
  • the metadata generation synchronization task is distributed to the receiver of the working component in the server with the least load.
  • Sub-step 244 if the amount of data recorded in the metadata is not less than the second threshold, then The upper layer interface for slicing a large file, the upper layer interface of the large file slice is called by the abstract interface class to call the bottom layer operation interface of the large file slice of the second system, and the large file corresponding to the metadata is sliced.
  • the large file corresponding to the metadata may be sliced, which is large.
  • the file is divided into several smaller slices, which is equivalent to dividing the metadata into corresponding numbers of data slices.
  • the upper layer interface of the large file slice can be called, and the upper layer interface of the large file slice can be called by the abstract interface class to call the bottom layer operation interface of the large file slice of the second system, and the large file corresponding to the metadata is sliced. .
  • the large file can be divided into a plurality of slices with the same data size, or can be randomly cut into a plurality of slices with different data sizes, but the data volume of each slice needs to be less than the second threshold, and if there is data volume If the slice is not smaller than the second threshold, the slice needs to be cut again.
  • the number of the slices can be set according to requirements, and the embodiment of the present application is not limited.
  • Sub-step 245 generating synchronization tasks for the sliced information of the different slices of the metadata, and distributing them to the receivers of the working components of the plurality of servers with low load.
  • each synchronization task is distributed to the receiver of the working component of the server from low to high, corresponding to the number of synchronization tasks. For example, if five synchronization tasks are generated, the five servers with the lowest load are selected according to the load of each server in the current cluster, and then five synchronization tasks are respectively distributed to the working components of the five servers.
  • the receiver causes the receiver of the working component of each server to receive a synchronization task, and the synchronization tasks received by the receivers of the working components of any two servers are not the same synchronization task.
  • five synchronization tasks can be randomly distributed to the receivers of the working components of the five servers, or in a certain order, for example, according to the data volume of the synchronization tasks from large to small, and the load of the server from low to high,
  • the five synchronization tasks are distributed to the receivers of the working components of the five servers, which are not limited in this embodiment.
  • Step 250 The upper interface of the actual data is executed by the receiver of the working component according to the received synchronization task, and the upper layer interface that obtains the actual data invokes the bottom layer acquisition interface of the first system to obtain the actual data through the abstract class interface.
  • a system acquires actual data of the migration task.
  • the synchronization task includes the file information in the first system of the metadata to be migrated, but the synchronization task does not include the metadata to be migrated, that is, the actual data that needs to be migrated by the current synchronization task.
  • the actual data that needs to be migrated for the synchronization task is still stored in the first system. Therefore, the actual data that needs to be migrated by the synchronization task needs to be obtained from the first system.
  • the receiver of the working component of the server responsible for executing the synchronization task can execute the upper layer interface for acquiring the actual data after receiving the synchronization task.
  • the upper layer interface that obtains the actual data can call the first system to obtain the actual data underlying acquisition interface through the abstract class interface, and obtain the actual data of the migration task from the first system, that is, the actual data that needs to be migrated by the synchronization task.
  • Step 260 Perform, by the synchronizer of the working component, each upper layer interface of the synchronization data, and each upper layer interface of the synchronization data invokes an underlying operation interface of the second system through an abstract interface class, and synchronizes the actual data to the second system.
  • the actual data can be synchronized to the second system.
  • the upper layer interface of the synchronization data can be executed by the synchronizer of the working component, and each upper layer interface of the synchronization data is The actual data can be synchronized to the second system by calling the underlying operational interface of the second system through the abstract interface class.
  • step 260 includes:
  • Sub-step 261 by the synchronizer of the working component, according to the speed limit instruction, calling and the speed limit instruction
  • the corresponding thread executes each upper layer interface of the synchronization data, and each upper layer interface of the synchronization data calls the lower layer operation interface of the second system through the abstract interface class to synchronize the actual data to the second system.
  • a speed limit command may be input, which includes a data transmission speed in an acceptable data migration process, which may be a specific speed limit value, or may be
  • the synchronizer of the working component can further call each upper layer interface of the synchronous data according to the speed limit instruction according to the speed limit instruction, and each upper layer interface of the synchronous data can call the second system through the abstract interface class.
  • the bottom layer operation interface synchronizes the actual data to the second system according to the speed limit value or the data transmission speed in the speed range.
  • the thread corresponding to the speed limit command may exist in the foregoing working thread pool, or may be a separately created thread, which is not limited in this embodiment of the present application.
  • a migration task of migrating data in the first system to the second system may be received, where the first system and the second system are any two of the plurality of systems, and then the corresponding The upper layer interfaces of the task are migrated, and the upper layer operation interfaces of the first system and the second system are invoked by the upper layer interfaces through the abstract interface class to migrate the data of the first system to the second system.
  • any two data ends can perform bidirectional data transmission and migration, including inter-bucket migration within the data end.
  • the data migration technology described in the present application has a good extension function, and can implement any data terminal, including a cloud storage product that does not support data migration, such as Baidu object storage, Jinshan object storage, and any other data terminal. Two-way transfer of data.
  • the present application can support distributed data transmission, which can greatly improve the efficiency of data migration.
  • the OSS of the two regions in the first system and the second system are transmitted from the data of one bucket of one regional OSS to the bucket of another region, and the test method adopts a common network.
  • the network path is first downloaded from a bucket, and then uploaded to another bucket through the internal network link.
  • the transmission adopts a distributed cluster environment: Facebook Cloud ECS (Elastic Compute Service), and a single server is configured as a 4-core CPU, 4G memory, and 500 Mbps intranet bandwidth, for a total of 5 nodes.
  • the test example is shown in the following table (1):
  • the efficiency unit is MB/s (megabits per second).
  • the body can include the following modules:
  • the task receiving module 310 is adapted to receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the plurality of systems.
  • the migration module 320 is adapted to invoke each upper layer interface corresponding to the migration task, and the upper layer interface invokes the bottom layer operation interface of the first system and the second system by the upper interface interface to migrate the data of the first system to the second system.
  • the migration module 320 includes:
  • the upper interface determining submodule is adapted to determine each upper layer interface belonging to the acquired data, and each upper layer interface belonging to the synchronous data.
  • the data acquisition submodule is adapted to invoke the lower layer operation interface of the first system through the abstract interface class for each upper layer interface belonging to the acquired data to acquire data from the first system.
  • the data synchronization submodule is adapted to invoke the lower layer operation interface of the second system through the abstract interface class for each upper layer interface belonging to the synchronization data to synchronize the data acquired from the first system to the second system.
  • the data acquisition submodule includes:
  • the first low-level operation interface determining sub-module is adapted to directly invoke the first abstract class interface corresponding to the first upper-layer interface in the abstract interface class as the first upper-level interface in each of the upper-layer interfaces belonging to the acquired data.
  • the first upper layer interface is a common interface of each system.
  • the first lower layer operation interface calls the submodule, and is adapted to generate, by the abstract interface class, a first interface class for the first system, and call the first interface class in the second upper layer interface of each upper layer interface that belongs to the acquired data.
  • An underlying operation interface corresponding to each second upper layer interface; the second upper layer interface is not an interface common to each system.
  • the data synchronization submodule includes:
  • the third upper layer interface in the layer interface is directly called by the second abstract class interface corresponding to the third upper layer interface in the abstract interface class; the third upper layer interface is a common interface of each system.
  • the second lower layer operation interface calls the submodule, and is adapted to generate, by the abstract interface class, a second interface class for the second system, for the fourth upper layer interface among the upper layer interfaces belonging to the synchronization data, where the second interface class is called.
  • An underlying operation interface corresponding to each fourth upper layer interface; the fourth upper layer interface is not an interface common to each system.
  • the method further includes:
  • the migration determination module is adapted to determine whether the migration of the data in the first system to the second system is successful; if it fails, the migration module is repeatedly executed until the number of retries reaches the retry threshold or succeeds within the retry threshold.
  • the method further includes:
  • the pause instruction judgment module is adapted to determine whether a pause instruction is received; if the pause instruction is received, enter the pause record sub-module.
  • the pause recording module is adapted to suspend migration of data of the first system to the second system, and record the migrated data location in the first system at the time of suspension.
  • the method further includes:
  • the restart instruction determining module is adapted to determine whether a restart command is received; if the restart command is received, the migration module is re-entered according to the migrated data position in the first system at the time of the pause.
  • the method when the first system is a client file system, before the task receiving module 310, the method further includes:
  • the application installation module 300 is adapted to install an application in the client that simulates each of the underlying operational interfaces for the specified file.
  • the method further includes:
  • the data monitoring module is adapted to monitor whether the client file system changes with the data that was recently migrated to the second system; if a change occurs, the change data synchronization module is entered.
  • the change data synchronization module is adapted to invoke each upper layer interface corresponding to the migration task, and each upper layer interface invokes an underlying operation interface of the first system and the second system through an abstract interface class to synchronize the change data of the client file system. To the second system.
  • the method further includes:
  • the signature submodule is adapted to invoke the upper interface of the signature, and the lower layer interface of the signature is called by the upper interface of the signature to sign the data.
  • the method further includes:
  • the signature verification sub-module is adapted to invoke an upper-layer interface of the signature authentication, and the upper-layer interface of the signature authentication is called by the upper-layer interface of the signed authentication to perform signature verification on the data.
  • the method further includes:
  • a first file list obtaining module configured to invoke an upper layer interface of the file list for the first system, and invoke an upper layer operation interface of the first system by an upper interface of the calling file list to obtain a file list of the first system .
  • a second file list obtaining module configured to invoke an upper layer interface of the file list for the second system, and invoke an upper layer operation interface of the second system by the upper interface of the calling file list to obtain a file list of the second system .
  • the difference prompting module is adapted to compare the file list of the first system with the file list of the second system, and prompt the difference between the two to the client.
  • the method further includes:
  • a first identity verification module adapted to invoke an upper layer interface for authentication of the first system
  • the upper layer interface of the identity verification system invokes the bottom layer operation interface of the first system through the abstract interface class to verify the identity information of the user in the first system.
  • a second identity verification module configured to invoke an upper layer interface for the second system, and the upper layer interface of the identity verification system invokes an underlying operation interface of the second system by using the abstract interface class to identify the identity information of the user in the second system authenticating.
  • the embodiment of the present application may receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the multiple systems, and then invoke the corresponding migration task.
  • Each upper layer interface, and each upper layer interface invokes the first system and the bottom layer operation interface of the second system through the abstract interface class to migrate the data of the first system to the second system.
  • any two data ends can be used for bidirectional data transmission and migration, including inter-bucket migration within the data end.
  • the data migration technology described in the present application has a good extension function, and can implement any data terminal, including a cloud storage product that does not support data migration, such as Baidu object storage, Jinshan object storage, and any other data terminal. Two-way transfer of data.
  • FIG. 4 a structural block diagram of an embodiment of a data migration apparatus of the present application is shown, which may specifically include the following modules:
  • the task receiving module 410 is adapted to receive a migration task of migrating data in the first system to the second system; the first system and the second system are any two of the plurality of systems.
  • the migration module 420 is adapted to invoke each upper layer interface corresponding to the migration task, and the upper layer interface invokes the bottom layer operation interface of the first system and the second system by the upper interface interface to migrate the data of the first system to the second system. Specifically include:
  • the upper interface determining submodule 421 is adapted to determine each upper layer interface belonging to the acquired data and each upper layer interface belonging to the synchronous data.
  • the data acquisition sub-module 422 is adapted to invoke the underlying operation interface of the first system through the abstract interface class for each upper-layer interface belonging to the acquired data to acquire data from the first system. Specifically include:
  • the metadata obtaining sub-module 4221 is adapted to execute, by the scanner of the management component, an upper layer interface for acquiring metadata, where the upper layer interface for obtaining the metadata invokes the bottom layer operation interface of the first system for acquiring the metadata through the abstract class interface, A system acquires each metadata corresponding to the migration task.
  • the task distribution sub-module 4222 is adapted to generate a synchronization task based on each metadata by a distributor of the management component and distribute it to a receiver of each working component.
  • the task distribution sub-module 4222 includes:
  • the data size extraction sub-module is adapted to extract the amount of data recorded in the metadata.
  • the first metadata distribution sub-module is adapted to directly distribute the metadata as a synchronization task to a receiver of a working component in a server where the management component is located, if the amount of data recorded in the metadata is less than the first threshold.
  • the second metadata distribution sub-module is adapted to distribute the metadata as a synchronization task to a receiver of a working component in a server with the least load if the amount of data recorded by the metadata is not less than the first threshold.
  • the task distribution sub-module 4222 includes:
  • the data size extraction sub-module is adapted to extract the amount of data recorded in the metadata.
  • the first metadata distribution sub-module is adapted to directly distribute the metadata as a synchronization task to a receiver of a working component in a server where the management component is located, if the amount of data recorded in the metadata is less than the first threshold.
  • a third metadata distribution sub-module adapted to not record the amount of data in the metadata
  • the metadata generation synchronization task is distributed to the receiver of the working component in the server with the least load.
  • a metadata cutting molecular module adapted to call an upper layer interface for a large file slice if the amount of data recorded in the metadata is not less than a second threshold, wherein the upper layer interface for the large file slice calls the second system through the abstract interface class
  • the slice information of the different slices of the metadata is generated and distributed to the receiver of the working component of the plurality of servers with low load.
  • the actual data acquisition sub-module 4223 is adapted to perform, by the receiver of the working component, the upper layer interface for acquiring the actual data according to the received synchronization task, where the upper layer interface that obtains the actual data invokes the first system to obtain the actual data through the abstract class interface.
  • the bottom layer acquires an interface, and acquires actual data of the migration task from the first system.
  • the data synchronization sub-module 423 is adapted to invoke the underlying operation interface of the second system through the abstract interface class for each upper-layer interface belonging to the synchronization data to synchronize the data acquired from the first system to the second system. Specifically include:
  • the actual data synchronization sub-module 4231 is adapted to execute each upper-layer interface of the synchronization data by the synchronizer of the working component, and each upper-layer interface of the synchronization data invokes an underlying operation interface of the second system through an abstract interface class to synchronize the actual data. To the second system.
  • the actual data synchronization submodule 4231 includes:
  • the first actual data synchronization sub-module is adapted to be executed by the synchronizer of the working component according to the speed limit instruction, and the thread corresponding to the speed limit instruction is used to execute each upper layer interface of the synchronization data, and each upper layer interface of the synchronization data is called by the abstract interface class.
  • the underlying operational interface of the second system synchronizes the actual data to the second system.
  • the data in the first system may be received to be migrated to the second system.
  • a migration task the first system and the second system are any two of the plurality of systems, and then call each upper layer interface corresponding to the migration task, and the first system and the first system are invoked by the upper layer interface through the abstract interface class.
  • the underlying operational interface of the second system to migrate data from the first system to the second system.
  • any two data ends can perform bidirectional data transmission and migration, including inter-bucket migration within the data end.
  • the data migration technology described in the present application has a good extension function, and can implement any data terminal, including a cloud storage product that does not support data migration, such as Baidu object storage, Jinshan object storage, and any other data terminal. Two-way transfer of data.
  • the present application supports both single-machine data transmission and distributed data transmission, which can greatly improve the efficiency of data migration.
  • the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
  • embodiments of the embodiments of the present application can be provided as a method, apparatus, or computer program product. Therefore, the embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, embodiments of the present application may employ computer program products embodied on one or more computer usable synchronization media (including but not limited to disk synchronizers, CD-ROMs, optical synchronizers, etc.) having computer usable program code embodied therein. form.
  • computer usable synchronization media including but not limited to disk synchronizers, CD-ROMs, optical synchronizers, etc.
  • the computer device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • the memory may include non-permanent synchronizers in the computer readable medium, random access synchronizer (RAM) and/or non-volatile memory, such as read-only synchronizer (ROM) or flash (flash RAM).
  • RAM random access synchronizer
  • ROM read-only synchronizer
  • flash RAM flash RAM
  • Memory is an example of a computer readable medium.
  • Computer readable media including both permanent and non-persistent, removable and non-removable media can be synchronized by any method or technique.
  • the information can be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer synchronization media include, but are not limited to, phase change memory (PRAM), static random access synchronizer (SRAM), dynamic random access synchronizer (DRAM), other types of random access synchronizers (RAM), Read-only synchronizer (ROM), electrically erasable programmable read-only synchronizer (EEPROM), flash memory or other memory technology, CD-ROM read-only synchronizer (CD-ROM), digital versatile disc (DVD) Or other optical sync, magnetic tape cartridge, tape magnetic disk sync or other magnetic sync device or any other non-transport medium that can be used to synchronize information that can be accessed by the computing device.
  • computer readable media does not include non-persistent computer readable media, such as modulated data signals and carrier waves.
  • Embodiments of the present application are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations. Or a combination of processes and/or blocks in a block diagram.
  • the computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing terminal device to generate a server such that instructions are executed by a processor of a computer or other programmable data processing terminal device Means are provided for implementing the functions specified in one or more of the flow or in one or more blocks of the flow chart.
  • the computer program instructions can also be synchronized in a computer readable synchronizer capable of directing a computer or other programmable data processing terminal device to operate in a particular manner such that synchronization of instructions in the computer readable synchronizer produces an article of manufacture including the instruction device
  • the instruction means implements the functions specified in one or more blocks of the flow or in a flow or block diagram of the flowchart.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种数据迁移方法和装置，涉及计算机技术领域。所述方法包括：接收将第一系统中的数据迁移至第二系统的迁移任务（110，210）；调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统（120）。首先，实现了任意两个数据端可以进行双向的数据传输和迁移。其次，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。

Description

一种数据迁移方法和装置 技术领域
本申请涉及计算机技术领域，特别是涉及一种数据迁移方法和一种数据迁移装置。
背景技术
随着国内外云计算平台、市场的快速发展，云存储产品也越来越多。在云计算平台上，云服务器是最基础、最重要的产品，其次就是云存储。各大云服务商都有自己的云存储产品，如AWS(Amazon Web Services，亚马逊云服务)的S3、阿里云的OSS(Object Storage Service，对象存储服务)、微软云计算平台Azure的存储数据服务(Blob)。另外，还有像七牛云存储这种专门提供云存储服务的服务商。
虽然云存储产品种类很多，它们在功能和使用方法上都有一些共通点，但具体到API(Application Programming Interface，应用程序编程接口)调用、SDK(Software Development Kit，软件开发工具包)的编程上，又都有一些差异和各自的特点。在实际使用过程中，有时因为整体业务的搬迁，可能会从一个云服务商切换到另一个服务商，这时候就需要相应地把云存储的数据进行迁移同步；或者有时为了数据的安全性，不仅在同个服务商提供的多个云存储产品上同步备份数据，甚至会把数据备份到另一服务商提供的云存储产品，减少风险；又或者将客户端文件系统中的数据同步迁移到某一云服务商提供的云存储产品中。
但是，先有的数据迁移技术或工具，只能针对性地在某一服务商提供的云存储产品与用户之间进行数据传输，不支持其它云存储产品；或者是将其他存储系统中的数据迁移到某一服务商提供的云存储产品，不能进行任意两个存储系统之间的数据的双向迁移。
发明内容
鉴于上述问题，提出了本申请实施例以便提供一种克服上述问题或者至少部分地解决上述问题的一种数据迁移方法和相应的一种数据迁移装置。
本申请公开了一种数据迁移方法，其特征在于，包括：
接收将第一系统中的数据迁移至第二系统的迁移任务；
调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
优选地，所述调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤，包括：
确定属于获取数据的各上层接口，以及属于同步数据的各上层接口；
对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据；
对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
优选地，对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口的步骤，包括：
对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口；
对于属于获取数据的各上层接口中的第二上层接口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二上层接口不为各系统通用的接口。
优选地，对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口的步骤，包括：
对于属于同步数据的各上层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口；
对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
优选地，所述对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据的步骤，包括：
由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据；
由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器；
由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
优选地，所述对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统的步骤，包括：
由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
优选地，所述管理组件在集群的一台服务器中，所述工作组件在集群 的各台服务器中。
优选地，所述由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器的步骤，包括：
提取元数据中记录的数据量大小；
如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器；
优选地，所述由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器的步骤，包括：
提取元数据中记录的数据量大小；
如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器；
如果元数据中记录的数据量不小于第二阈值时，则调用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片；
将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
优选地，所述集群的各台服务器中之间采用JGroups通信协议传输数据。
优选地，所述由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将 所述实际数据同步至第二系统的步骤，包括：
由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
优选地，调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
判断将第一系统中的数据迁移至第二系统是否成功；
如果失败，则重复执行调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤，直至重试次数达到重试阈值或者在重试阈值内成功。
优选地，在调用对应所述迁移任务的各上层接口，通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的过程中，还包括：
判断是否接收到暂停指令；
如果接收到暂停指令，则暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置；
则进一步的，在暂停将第一系统的数据迁移至第二系统之后，还包括：
判断是否接收到重启指令；
如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新调用对应所述迁移任务的各上层接口，通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤。
优选地，当第一系统为客户端文件系统时，在接收将第一系统中的数 据迁移至第二系统的迁移任务的步骤之前，还包括：
在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
优选地，所述调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；
如果出现变更，则调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
优选地，所述对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据的步骤之后，还包括：
调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名；
进一步的，对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统的步骤之后，还包括：
调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
优选地，在调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文 件列表；
针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表；
比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
优选地，所述迁移任务由客户端发送，所述迁移任务包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。
优选地，在确定属于获取数据的各上层接口，以及属于同步数据的各上层接口的步骤之后，还包括：
针对第一系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证；
针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身份信息进行验证。
本申请还公开了一种数据迁移装置，其特征在于，包括：
任务接收模块，适于接收将第一系统中的数据迁移至第二系统的迁移任务；
迁移模块，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
优选地，所述迁移模块，包括：
上层接口确定子模块，适于确定属于获取数据的各上层接口，以及属 于同步数据的各上层接口；
数据获取子模块，适于对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据；
数据同步子模块，适于对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
优选地，所述数据获取子模块，包括：
第一底层操作接口确定子模块，适于对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口；
第一底层操作接口调用子模块，适于对于属于获取数据的各上层接口中的第二上层接口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二上层接口不为各系统通用的接口。
优选地，所述数据同步子模块，包括：
第二底层操作接口确定子模块，适于对于属于同步数据的各上层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口；
第二底层操作接口调用子模块，适于对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
优选地，所述数据获取子模块，包括：
元数据获取子模块，适于由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据；
任务分发子模块，适于由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器；
实际数据获取子模块，适于由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
优选地，所述数据同步子模块，包括：
实际数据同步子模块，适于由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
优选地，所述管理组件在集群的一台服务器中，所述工作组件在集群的各台服务器中。
优选地，所述任务分发子模块，包括：
数据量大小提取子模块，适于提取元数据中记录的数据量大小；
第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
第二元数据分发子模块，适于如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器。
优选地，所述任务分发子模块，包括：
数据量大小提取子模块，适于提取元数据中记录的数据量大小；
第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
第三元数据分发子模块，适于如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器；
元数据切分子模块，适于如果元数据中记录的数据量不小于第二阈值时，则调用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片；
将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
优选地，所述集群的各台服务器中之间采用JGroups通信协议传输数据。
优选地，所述实际数据同步子模块，包括：
第一实际数据同步子模块，适于由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
优选地，在所述迁移模块之后，还包括：
迁移判断模块，适于判断将第一系统中的数据迁移至第二系统是否成功；如果失败，则重复执行迁移模块，直至重试次数达到重试阈值或者在重试阈值内成功。
优选地，在所述迁移模块之后，还包括：
暂停指令判断模块，适于判断是否接收到暂停指令；如果接收到暂停 指令，则进入暂停记录子模块；
暂停记录模块，适于暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置；
则进一步的，在所述暂停指令判断模块之后，还包括：
重启指令判断模块，适于判断是否接收到重启指令；如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新进入迁移模块。
优选地，当第一系统为客户端文件系统时，在任务接收模块之前，还包括：
应用程序安装模块，适于在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
优选地，在所述迁移模块之后，还包括：
数据监控模块，适于监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；如果出现变更，则进入变更数据同步模块；
变更数据同步模块，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
优选地，在所述数据获取子模块之后，还包括：
签名子模块，适于调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名；
进一步的，在所述数据同步子模块之后，还包括：
签名验证子模块，适于调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
优选地，在所述迁移模块之后，还包括：
第一文件列表获取模块，适于针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文件列表；
第二文件列表获取模块，适于针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表；
差异提示模块，适于比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
优选地，所述迁移任务由客户端发送，所述迁移任务包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。
优选地，在所述上层接口确定子模块之后，还包括：
第一身份验证模块，适于针对第一系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证；
第二身份验证模块，适于针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身份信息进行验证。
本申请实施例包括以下优点：
本申请实施例，接收将第一系统中的数据迁移至第二系统的迁移任务；然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
首先，实现了任意两个数据端可以进行双向的数据传输和迁移。
其次，本申请所述的数据迁移技术具有良好的扩展功能，可以实现将 任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
附图说明
图1是本申请的一种数据迁移方法实施例的步骤流程图；
图1A是本申请的一种五个系统进行数据迁移的示意图；
图1B是本申请的一种五个系统的数据迁移过程接口调用示意图；
图2是本申请的一种数据迁移方法实施例的步骤流程图；
图2C是本申请的一种数据迁移结构示意图；
图2D是本申请的一种分布式模式数据迁移数据流示意图；
图3是本申请的一种数据迁移装置实施例的结构框图；
图4是本申请的一种数据迁移装置实施例的结构框图。
具体实施方式
为使本申请的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本申请作进一步详细的说明。
本申请实施例的核心构思之一在于，接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统，然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
首先，实现了任意两个数据端可以进行双向的数据传输和迁移，包括数据端内部的bucket间迁移。
其次，本申请所述的数据迁移技术具有良好的扩展功能，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
实施例一
参照图1，示出了本申请的一种数据迁移方法实施例的步骤流程图，具体可以包括如下步骤：
步骤110，接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统。
本申请的一种数据迁移方法是将一个系统中的数据迁移至另一个系统的方法，是针对多个系统中的任一两个系统。其中的系统可以是具有存储功能的服务器，或者存储空间、云存储产品等，例如客户端文件系统(Local)、AWS(Amazon Web Services，亚马逊云服务)的S3、阿里云的OSS(Object Storage Service，对象存储服务)、微软云计算平台Azure的存储数据服务(Blob)、七牛云存储(QiNiu)等。如图1A为前述五个系统进行数据迁移的示意图。可以看出有，多个系统中的任意两个系统可以进行双向的数据传输和迁移，包括系统内部的不同存储区间(bucket)之间的数据迁移。其中，Bucket是一个用户用来管理所存储对象的存储空间，类似于文件系统里面的根目录或者磁盘分区。而且本申请的数据迁移方法具有良好的扩展功能，对于其它尚未加入图1A所示五个系统的云存储产品，如百度对象存储(Baidu Object Storage，BOS)、金山对象存储(Kingsoft Standard Storage Service，KS3)等，也可以很方便地加入，并且可以跟其它任意一个系统进行数据的双向传输与迁移。
为了区分两个系统，可以定义将第一系统中的数据迁移至第二系统。其中的第一系统和第二系统可以为两个相同类型的系统，例如都为阿里云的OSS，那么此时是将一个OSS的数据迁移至另一个OSS，比如将一个地区的OSS中的数据迁移至另一个地区的OSS中；当然，第一系统和第二系统也可以为两个不同类型的系统，例如分别为OSS和Blob，那么此时是将OSS的数据迁移至Blob，或者也可以为同一系统中的两个不同的 存储区，那么此时是将同一系统中的一个存储区间的数据迁移至另一个存储区间。
对于具体的第一系统和第二系统，以及迁移数据，例如是将第一系统中的全部数据迁移至第二系统，还是只迁移一部分，可以在迁移任务中设定。接收数据迁移任务的服务器可以从迁移任务中获取上述信息。
在本申请的另一个优选地实施例中，所述迁移任务由客户端发送，所述迁移任务包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。其中，第一系统的标识信息是指可以唯一标识第一系统的信息，例如第一系统的访问域名等；用户在第一系统的身份信息是指当前第一系统用户的身份信息，例如用户名等；第二系统的标识信息是指可以唯一标识第二系统的信息，例如第二系统的IP地址等；用户在第二系统的身份信息是指当前第二系统用户的身份信息，例如用户名等；待迁移数据在第一系统中的文件信息可以包括待迁移数据存储在第一系统中的文件的名称、存储位置、数据大小等信息。
在实际应用中，本申请实施例可以在客户端中提供一个迁移配置页面，用户可以在该迁移配置页面中配置相关信息，作为被迁移对象的第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、作为迁移目标对象的第二系统的标识信息、用户在第二系统的身份信息，当然还包括迁移对象标识。其中，如果用户需要迁移第一系统的全部数据，则可以配置迁移对象标识为all，all标识迁移第一系统的全部数据；如果用户需要迁移某个或者某些歌文件，则可以配置该文件或者该几个文件在第一存储系统中的存储路径作为迁移对象标识。当然，还可以配置其他需求的信息，比如有的存储系统是分区域，比如北京有一个存储中心，上海有一个存储中心，那么还可以配置该存储系统所在区域 Region。具体的配置项可以根据需求设置，本申请实施例不对其加以限制。
步骤120，调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
在本申请实施例中，可以将多个服务器连接起来形成一个大的处理集群。整个集群有一台控制节点(Master)服务器和多台工作节点(Slave)服务器，Master用以接收将第一系统中的数据迁移至第二系统的迁移任务，然后可以通过合理的分发策略，把接收到的迁移任务分发给Slave进行处理。Slave处理完接收到的迁移任务后，可以把迁移结果反馈给Master。当然，本申请实施例也可以用单台服务器实现数据的迁移。优选的本申请数量采样前述集群的方式。
如图1B为对前述五个系统的数据迁移过程接口调用示意图。其中，上层(Peer)接口定义了每个具体的进行数据迁移的系统都必须具有的功能。例如，get()，put()，分别是指下载、上传数据，这是进行实际数据传输需要调用的接口；
mkRoot()，rmRoot()：建立、删除根目录。操作的对象就是系统中的Bucket；Bucket存储空间，是一个用户用来管理所存储Object的存储空间，类似于文件系统里面的根目录或者磁盘分区。
getObj()，getAllObjs()：获取单个元数据、获取全部元数据，主要就是利用这些接口，获取要传输的数据的元数据；元数据Metadata作为实际存储数据的描述型数据，只包含一些基础信息，如：标识数据唯一性的Key、数据大小、时间戳等。
beginPartPut()，partPut()，endPartPut()：主要处理断点续传、大文件的切片传输功能；
rm()，rmAll()：删除单个数据，删除全部数据。
其中，对于某一些上层接口，如getAllObjs()，rmAll()，它们的实现是通过调用其它底层操作接口如getObj()，rm()来实现，这对于任何一个进行数据迁移的系统来说都是通用的，可以在图2B中所示的抽象接口类(AbstractPeer)层中实现。
所以在本申请实施例中，可以在AbstractPeer这一层实现调用底层操作接口，例如图1B中的Local、OSS、S3、QiNiu、Blob构成的底层的操作接口。所以，可以调用对应迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
例如，若某一迁移任务是要将Local中的数据迁移至OSS，则可以调用对应该迁移任务的各上层接口，包括用以获取Local中数据的上层接口和用以将获取的数据同步到OSS中的上层接口，然后首先可以由获取Local中数据的上层接口通过与之对应的抽象接口类依次调用Local中的getObj()接口、get()接口，以获取Local中的迁移数据，然后可以由用以将获取的数据同步到OSS中的上层接口通过与之对应的抽象接口类依次调用OSS中的put()接口，将从Local中获取的迁移数据传输至OSS。
由于本申请设置一个统一的上层接口层peer，在其中构建了各种操作接口。又构建了一个抽象接口类层AbstractPeer，该抽象接口类层可以继承上层接口层。然后对于各个存储系统(如前述Local、OSS、S3、QiNiu、Blob)本身的底层操作接口，可以继承该AbstractPeer，从而可以实现各个系统之间互相迁移数据的过程。同时，由于有抽象接口类层AbstractPeer，对于新的存储系统，只需要将其底层的底层操作接口层继承抽象接口类层AbstractPeer，即可实现该新的存储系统与之前的其他任意一个存储系统之间的数据迁移过程。
在本申请的另一个优选的实施例中，步骤120包括：
子步骤121，确定属于获取数据的各上层接口，以及属于同步数据的各上层接口。
在本申请实施例中，定义用以从第一系统中获取数据的上层接口为属于获取数据的上层接口，用以将获取的数据同步至第二系统的上层接口为属于同步数据的上层接口。在实际应用中，根据具体需求的不同，属于获取数据的上层接口和属于同步数据的上层接口都可能为一个或多个上层接口，对此本申请实施例不加以限制。
在实际应用中，接收到一个迁移任务后，首先要确定去某个存储系统获取数据要调用的各上层接口，以及将数据存储至另一系统要调用的上传接口。
比如，对于迁移任务中包括了要从中迁移数据的第一系统的标识信息，以及待迁移的一个文件的文件信息，如果不需要断点续传，则需要调用图1B的peer中的上层接口getObj()获取元数据，还要调用图1B的peer中上层接口get()获取实际数据，那么getObj()和get()属于获取数据的上层接口。当然实际应用中，还可以包括对第一系统的用户身份信息，如用户在第一系统中的账户名和密码，进行身份验证的上层接口。
而对于第二系统，由于该文件是从第一系统迁移过来，通常情况下第二系统是没有相应文件夹的，那么需要调用图1B的peer中的mkRoot()建立目录，以及图1B的peer中的put()上传数据。那么mkRoot()和put()属于同步数据的上层接口。当然实际应用中，还可以包括对第二系统的用户身份信息，如用户在第二系统中的账户名和密码，进行身份验证的上层接口。
子步骤122，对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据。
如前述，在实际应用中，上层接口是通过抽象接口类调用第一系统的底层操作接口，以将第一系统的数据迁移至第二系统。在将第一系统的数 据迁移至第二系统的过程中包括从第一系统中获取数据。此时，可以利用属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据。
在实际应用中，本申请实施例的服务器可以将第一系统的标识信息作为参数之一传给获取数据的上层接口，那么上层接口再调用抽象接口类，抽象接口类则可以根据该标识信息去调用第一系统的相应的底层操作接口。
在本申请的另一个优选的实施例中，子步骤122包括：
子步骤1221，对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口。
在实际应用中，对于属于获取数据的各上层接口中包括的各系统通用的接口，例如，getAllObjs()，rmAll()，可以称为第一上层接口。它们的实现是通过调用其它底层操作接口来实现的，因此对于属于获取数据的各上层接口中的第一上层接口，可以以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用。
例如，对于第一上层接口getAllObjs()，在抽象接口类中第一抽象类接口getAllObjs()是通过调用底层操作接口getObj()实现的，所以，此时可以直接将第一抽象类接口getAllObjs()作为底层操作接口调用，相当于调用了底层操作接口getObj()。
本申请实施例的服务器可以将第一系统的标识信息作为参数之一传给获取数据的上层接口，那么上层接口再调用抽象接口类，如果抽象接口类中有该上层接口对应的第一抽象类接口，则该第一抽象类接口可以根据该标识信息等参数执行。
子步骤1222，对于属于获取数据的各上层接口中的第二上层接 口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二上层接口不为各系统通用的接口。
而对于为各系统通用的各上层接口，可以称为第二上层接口，对于此类上层接口，则需要通过抽象接口类生成针对第一系统的第一接口类，然后在第一接口类中调用与各第二上层接口对应的底层操作接口。
本申请实施例的服务器可以将第一系统的标识信息作为参数之一传给获取数据的上层接口，那么上层接口再调用抽象接口类，如果抽象接口类中没有该上层接口对应的第一抽象类接口，则由抽象接口类根据该标识信息生成对应第一系统的第一接口类，然后在该第一接口类中调用上层接口对应的底层接口。
子步骤123，对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
如前述，在实际应用中，上层接口是通过抽象接口类调用第一系统的底层操作接口，以将第一系统的数据迁移至第二系统。在将第一系统的数据迁移至第二系统的过程中包括将从第一系统获取的数据同步至第二系统。此时，可以利用属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
在实际应用中，本申请实施例的服务器可以将第二系统的标识信息作为参数之一传给获取数据的上层接口，那么上层接口再调用抽象接口类，如果抽象接口类中没有对应该上层接口的第一抽象类接口，抽象接口类则可以根据该标识信息去调用第二系统的相应的底层操作接口。
在本申请实施例中，对于步骤122，在第一系统的获取数据的底层操作接口获取到需要迁移到第二系统的实质数据后，本申请实施例可以直接 将该实质数据传给第二系统的同步数据的存储接口，从而可以实现两个存储系统内部的bucket间的迁移。比如，在调用第一系统的get()获取到数据后，直接将该数据传给第二系统的put()，其处理过程均在内存中进行，那么该数据则直接迁移到第二系统，不会存储到本地磁盘中。在实际应用中，本申请实施例可以设定一个第一函数监控get()是否获取到数据，如果获取到数据，则将该数据传给put()。在本申请实施例中，获取到数据的底层接口传输给同步数据的底层操作接口后，在获取到数据的底层接口和同步数据的底层操作之间架构了一个传输管道，用于传输两类接口之间的数据，其每份数据传输后，不会在本地存储，也不会累积到内存中。
在本申请的另一个优选的实施例中，子步骤123包括：
子步骤1231，对于属于同步数据的各上层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口。
此时，将属于同步数据的各上层接口中各系统通用的上层接口称为第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用。本申请实施例的服务器可以将第二系统的标识信息作为参数之一传给获取数据的上层接口，那么上层接口再调用抽象接口类，如果抽象接口类中有该上层接口对应的第二抽象类接口，则该第二抽象类接口可以根据该标识信息等参数执行。该步骤与前述的步骤1221类似，在此不再赘述。
子步骤1232，对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
此时，将属于同步数据的各上层接口中不为各系统通用的上层接口称为第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口。在实际应用中，本申请实施例的服务器可以将第二系统的标识信息作为参数之一传给获取数据的上层接口，如果抽象接口类中没有对应该上层接口的第二抽象类接口，那么上层接口再调用抽象接口类，抽象接口类则可以根据该标识信息去调用第二系统的相应的底层操作接口。该步骤与前述的步骤1222类似，在此不再赘述。
在本申请的另一个优选的实施例中，在步骤120之后，还包括：
步骤130，判断将第一系统中的数据迁移至第二系统是否成功；如果失败，则重复执行步骤120，直至重试次数达到重试阈值或者在重试阈值内成功。
在实际应用中，在数据迁移的过程中可能会出现干扰或错误等，导致数据迁移不成功，所以在本申请实施例中，会判断将将第一系统中的数据迁移至第二系统是否成功，所谓迁移成功，是指将需要迁移指令中需要进行迁移的数据全部迁移至第二系统中。具体地，可以通过判断已经迁移的数据大小是否等于迁移指令中设定的数据大小，或者是其他现有的可用技术比较已经迁移的数据和迁移指令中设定的数据是否一致，对此本申请实施例不加以限定。
当然，判断将第一系统中的数据迁移至第二系统是否成功可以采用多种方式，比如：可以通过监控第一系统下载成功与否的通知，以及监控第二系统上层成功与否的通知，如果收到任意一个不成功的通知，则确认迁移失败。当然，还有其他方式，本申请实施例不对其加以限制。
而且，在本申请实施例中，可以设置一个重试阈值，如果重试的次数超过重试阈值仍未成功迁移，则放弃执行当前迁移任务，进而可以检查迁 移不成功的原因。重试阈值可以根据需求设定，对此本申请实施例不加以限定。
在本申请的另一个优选的实施例中，在步骤120之后，还包括：
步骤140，判断是否接收到暂停指令；如果接收到暂停指令，则进入步骤150。
在数据迁移过程中，若因为某些原因需要暂停数据迁移，则此时可以发送暂停指令至执行迁移指令的服务器。所以，可以判断是否接收到暂停指令。
步骤150，暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置。
如果接收到暂停指令，则会暂停继续将第一系统的数据迁移至第二系统，同时，为了避免在再次开始将第一系统的数据迁移至第二系统时，又从初始情况开始，将第一系统中已经迁移至第二系统的数据再次重复迁移至第二系统，从而浪费时间，可以在暂停将第一系统的数据迁移至第二系统的同时，记录暂停时刻第一系统中已迁移的数据位置，以便从已迁移的数据位置开始，继续进行数据迁移。
在实际应用中，本申请实施例中，也可以调用暂停的上层接口，然后通过抽象接口类分别调用第一系统的暂停的底层操作接口和第二系统的暂停的底层操作接口，实现迁移暂停过程。
步骤160，判断是否接收到重启指令；如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新进入步骤120。
在接收到暂停指令之后，可以进一步判断判断是否接收到重启指令，如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新执行调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数 据迁移至第二系统。此时第一系统的数据是指第一系统中的尚未被迁移至第二系统中的数据。
在实际应用中，本申请实施例中，也可以调用重启的上层接口，然后通过抽象接口类分别调用第一系统的重启的底层操作接口和第二系统的重启的底层操作接口，实现迁移暂停。
在本申请的另一个优选的实施例中，当第一系统为客户端文件系统时，在步骤110之前，还包括：
步骤100，在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
若第一系统为客户端文件系统，那么此时将客户端文件系统中的数据迁移至第二系统，可以理解为将客户端文件系统中的数据备份至第二系统。对于客户端而言，本申请为了实现客户端与各个系统之间是互通的，为了适应图1B的逻辑架构，本申请可以通过在客户端中安装应用程序，为客户提供服务。所以，在本申请实施例中，若第一系统为客户端文件系统，则首先在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口，如图1B中的local下的各个底层操作接口。进而，客户端用户可以通过访问应用程序，调用可编辑该应用程序对应的指定文件的各底层操作接口。
在本申请的另一个优选的实施例中，在步骤120之后，还包括：
步骤170，监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；如果出现变更，则进入步骤180。
在实际应用中，客户端的应用程序可能会被多个用户多次登录，而且在登录过程中，可以对客户端文件系统中的数据进行修改，如若客户端文件系统中的数据被变更，则相应地需要将变更部分更新至第二系统中。所以，针对客户端文件系统，可以监控客户端文件系统是否与最近一次迁移 到第二系统的数据出现变更，如果出现变更，则需要将变更更新至第二系统中，具体地，可以调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
步骤180，调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
在本申请的另一个优选的实施例中，在子步骤122之后，还包括：
子步骤124，调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名。
在本申请实施例中，为了保证数据迁移前后的一致性，也即保证在执行完一次数据迁移任务，是将第一系统中的迁移数据完整的迁移至第二系统中，可以首先对第一系统中的需要进行迁移的数据进行签名，以标识需要迁移至第二系统的数据。具体地，可以调用签名的上层接口，并由签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名。可以在第一系统中对数据进行签名，也可以在将数据移出第一系统，且尚未同步至第二系统时，对数据进行签名，对此本申请实施例不加以限定。
进一步地，在子步骤123之后，还包括：
子步骤125，调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
理论上，若将需要迁移至第二系统中数据进行签名，则在数据迁移至第二系统后，可以对已迁移的数据进行签名验证。若签名验证成功，则表明已迁移至第二系统中的数据与第一系统中需要迁移至第二系统中的数据一致，而若签名验证不成功，例如已迁移至第二系统中的部分数据与其在第一系统中的签名不对应等情况，则表明此时迁移失败，可以重复执行 步骤120，直至重试次数达到重试阈值或者在重试阈值内成功。
具体地，可以调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对数据进行签名验证。
在本申请的另一个优选的实施例中，在步骤120之后，还包括：
步骤190，针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文件列表。
步骤1110，针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表。
步骤1120，比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
在本申请的另一个优选的实施例中，在子步骤121之后，还包括：
子步骤126，针对第一系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证。
子步骤127，针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身份信息进行验证。
在本申请实施例中，可以接收将第一系统中的数据迁移至第二系统的迁移任务，所述第一系统和第二系统为多个系统中的任意两个系统，然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
从而实现了任意两个数据端可以进行双向的数据传输和迁移；其次， 本申请所述的数据迁移技术具有良好的扩展功能，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
实施例二
参照图2，示出了本申请的一种数据迁移方法实施例的步骤流程图，具体可以包括如下步骤：
步骤210，接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统。
如图1A中的示例，用户可以选择其中的任意两个系统，将其中一个系统作为第一系统，另一个作为第二系统。
步骤220，确定属于获取数据的各上层接口，以及属于同步数据的各上层接口。
步骤230，由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据。
如图2C为本申请的一个数据迁移结构示意图。其由两大独立的组件构成：Master和Slave。Master主要负责扫描数据源和任务分发，Slave负责接收任务，并执行真正的数据迁移工作。Master包括扫描器(Scanner)、分发器(Dispatcher)；Slave包括接收器(Receiver)、同步器(Syncer)。在本申请实施例中，可以称由扫描器、分发器构成的Master为管理组件，由接收器和同步器构成的Slave为工作组件。该结构既可以在单机环境运行，也支持分布式数据传输。在单机模式下，Master和Slave作为一个进程内的两个组件同时提供服务，而在分布式模式下，整个集群里面有一台服务器中同时存在Master和Slave，其余的服务器中只存在Slave；或者是集群里面有一台服务器中只存在Master，其余的服务器中只存在Slave。 因为Master本身并不占用太多内存空间，所以Master和Slave可以共存于一个服务器中，以节省集群中的服务器成本。本申请实施例优选的采样集群的方式，即一个带Master+Slave的服务器和各个带Slave的服务器。
其中，扫描器负责遍历获取数据源的元数据(metadata)，并存放到队列“Metadata Queue(元数据队列)”里面。Metadata作为实际存储数据的描述型数据，只包含一些基础信息，如：标识数据唯一性的Key、数据量大小、时间戳等。另外，在本申请另一个优选地实施例中，Metadata Queue可以为阻塞队列，阻塞队列与普通队列的区别在于，当队列是空的时，从队列中获取数据的操作将会被阻塞，或者当队列是满时，往队列里添加数据的操作会被阻塞。同样，试图往已满的阻塞队列中添加新数据的线程同样也会被阻塞，直到其他的线程使队列重新变得空闲起来，如从队列中移除一个或者多个数据，或者完全清空队列。则在数据迁移的过程中，首先将元数据按序存放至阻塞队列，直至阻塞队列已满，然后依次将阻塞队列中的元数据移出并同步至第二系统，每移出一个元数据则相应地可以从第一系统中提取一个元数据存入阻塞队列，从而可以有效的均衡数据迁移的速度。
在本申请实施例中，管理组件的扫描器可以执行获取元数据的上层接口，进而获取元数据的上层接口可以通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取当前迁移任务对应的各元数据。
分发器负责从Scanner的Metadata Queue里面取出metadata，根据分发算法，把metadata作为任务派送给Slave组件的接收器(Receiver)。每分发一个任务都会进行记录，Slave完成一个任务后都会汇报给Master进行汇总，Master可以实时知道当前整个集群里面所有Slave节点的任务数和服务器负载情况。
其中，在单机模式下可以直接把任务分发给本机的Slave；而在分布式 模式下Dispatcher的分发算法，可以如下：
如果数据小于1MB，就直接分发给本服务器的Slave。因为对于小数据而言，通过网络分发到其它服务器执行，所耗费的总时间还不如直接在本服务器执行。
服务器接收器负责接收来自Dispatcher分发的任务，存放在自己的Metadata Queue里面，等待处理。
同步器真正执行数据传输的组件。它不断地从Receiver的Metadata Queue里面获取要传输数据的metadata，放到不同的工作线程池里面执行传输任务。
其中，工作池线程是一种多线程处理形式，处理过程中将任务添加到队列，然后在创建线程后自动启动这些任务。线程池线程都是后台线程。使用工作池线程的好处包括：减少在创建和销毁线程上所花的时间以及系统资源的消耗；如不使用线程池，有可能造成系统创建大量线程而导致消耗过多系统内存以及需要在不同线程之间“过度切换”。
在本申请实施例中，工作线程池(Work Thread Pool)可以有两个，一个负责处理比较小的数据，一次性可以完成整个元数据对应的文件的传输。另一个则负责处理比较大的数据，例如接收比较大的元数据对应的大文件的切片数据，对于大文件而言，需要把所有切片传输完成后，才能在第二系统中完成整个大文件的拼接还原，其中，可以预置一个阈值，若元数据小于该阈值，则由负责处理比较小的数据的工作线程池处理该元数据，若元数据不小于该阈值，则由负责处理比较大的数据的工作线程池处理该元数据。当然，工作线程池可以只有一个，对全部数据统一处理，例如若不对大文件进行切片处理，对此本申请实施例不加以限定。
如图2D为分布式模式数据迁移数据流示意图。其中，将云存储A中的数据迁移至云存储B，主要可以分为以下四步：
(1)Scanner扫描数据源“云存储A”，获取metadata。
(2)Dispatcher从Scanner获取metadata，根据分发算法派送给Slave的Receiver。
(3)Syncer从Receiver获取metadata，开始真正的数据传输。从数据源“云存储A”下载获取实际数据，不经过任何中间文件保存，直接上传到目的数据端“云存储B”，实现端到端的直接传输。
(4)Sycner把每个传输结果回传给Receiver，Receiver再汇报给Master的Dispatcher进行汇总处理。
在本申请另一个优选的实施例中，所述管理组件在集群的一台服务器中，所述工作组件在集群的各台服务器中。
在本申请另一个优选的实施例中，所述集群的各台服务器中之间采用JGroups通信协议传输数据。
JGroups由纯Java编写实现，是一个可靠的消息传播工具包。利用它可以简单地构建起一个多节点的通信集群，各个节点间可以互相通信，进行协作。利用JGroups，用户可以自己搭建一个对等节点网络，或者是Master-Slave的集群模式。
本申请的分布式集群，采用的是Master-Slave模式。利用JGroups的发现协议，只要预先在每个节点配置好Master节点的IP+端口地址，然后约定好一个统一的集群名称。先启动Master节点后，再依次启动各个Slave节点，这样每个Slave节点就都能发现找到Master节点并自动加入到这个集群里面。如果某个Slave节点退出，利用JGroups的FD(Failure detection)协议能够自动探测到该退出的Slave节点，并把该节点的信息从整个集群中移除。
在实际应用中，JGroups提供两种消息传播机制：
(1)字节流的消息直传。一个节点可以往另外一个节点发送字节 (byte)流数据。这种模式一般需要发送端先把要传输的数据序列化成byte数组，然后在接收端再反序列化回数据。
(2)类方法的直接RPC(Remote Procedure Call Protocol，远程过程调用协议)调用。JGroups支持在一个节点直接远程调用另外一个节点里面某个类的方法，就跟普通的方法调用一样。这种方法可以避免对于消息的序列化处理，更加方便。
因此，本申请的Dispatcher和Receiver之间的数据传输就是通过JGroups的RPC功能来实现的。
步骤240，由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器。
同步任务是数据迁移任务的一种，同步任务中同样可以包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。其中的待迁移数据是指个元数据。
在本申请另一个优选的实施例中，步骤240，包括：
子步骤241，提取元数据中记录的数据量大小。
如前述，在各元数据中可以包含数据量大小的参数，因此，可以直接从元数据中提取其中记录的数据量大小。
子步骤242，如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器。
如果元数据中记录的数据量小于第一阈值，则说明该元数据不会给服务器增加过多的负载量，所以可以直接将该元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器。其中，第一阈值可以根据需求设定，对此本申请实施例不加以限定。
比如，如果数据小于1MB，Dispatcher就直接分发给本服务器的Slave的Receiver。因为对于小数据而言，通过网络分发到其它服务器执行，所耗费的总时间还不如直接在本服务器执行。
子步骤243，如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器。
对于分布式模式下的数据迁移，如果元数据记录的数据量不小于第一阈值，此时为了均衡集群中各服务器的负载量，即让各服务器的负载量不会相差太大，可以将该元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器。
比如，如果数据大于等于1MB，Dispatcher就将该数据分发给其他服务器的slave的Receiver。
在本申请另一个优选的实施例中，步骤240，包括：
子步骤241，提取元数据中记录的数据量大小。
子步骤242，如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器。
子步骤243，如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器。
在本申请实施例中，为了避免过大的元数据在传输的过程中效率太低，所以可以另外设置一个第二阈值，同样的，第二阈值可以根据需求设定，但是第二阈值必须大于第一阈值，对此本申请实施例不加以限定。
如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器。
子步骤244，如果元数据中记录的数据量不小于第二阈值时，则调 用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片。
而若元数据中记录的数据量不小于第二阈值，则说明此时该元数据的数据量过大，为了保证数据迁移的速度，可以将该元数据对应的大文件进行切片处理，将大文件分为若干个较小的切片，相当于将元数据分为相应个数的数据切片。具体地，可以通过调用对大文件切片的上层接口，对大文件切片的上层接口可以通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片。
其中，可以将大文件分成若干个数据量大小相同的切片，也可以随机切成若干个数据量大小不完全相同的切片，但是需要保证各切片的数据量小于第二阈值，若其中存在数据量不小于第二阈值的切片，则需要对该切片再次进行分切。而且，每次切片的数量可以根据需求设定，对此本申请实施例不加以限定。
子步骤245，将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
在将元数据对应的大文件切片后，则可以对该元数据的不同切片的切片信息，也即各切片中包含的数据，生成不同的同步任务，然后可以按照当前集群中各服务器的负载从低到高的顺序，分别将各同步任务分发给负载从低到高，对应同步任务个数的服务器的工作组件的接收器。例如，若生成5个同步任务，则按照当前集群中各服务器的负载从低到高的顺序，选出负载最低的5个服务器，然后将5个同步任务分别分发到5个服务器的工作组件的接收器，使每个服务器的工作组件的接收器都接收到一个同步任务，且任意两个服务器的工作组件的接收器接收到的同步任务不为同一个同步任务。
其中，可以随机将5个同步任务分发到5个服务器的工作组件的接收器，也可以按照一定顺序，例如按照同步任务的数据量从大到小，服务器的负载从低到高的对应关系，将5个同步任务分发到5个服务器的工作组件的接收器，对此本申请实施例不加以限定。
步骤250，由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
在本申请实施例中，同步任务中包含了待迁移的元数据第一系统中的文件信息，但是同步任务中并不包含待迁移的元数据，也即当前同步任务所需要迁移的实际数据。同步任务所需要迁移的实际数据仍存储于第一系统中。所以，需要从第一系统中获取同步任务所需要迁移的实际数据，具体地，可以由负责执行该同步任务的服务器的工作组件的接收器在收到同步任务后，执行获取实际数据的上层接口，获取实际数据的上层接口即可以通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取迁移任务的实际数据，也即同步任务所需要迁移的实际数据。
步骤260，由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
在获取了同步任务对用需要迁移的实际数据之后，即可以将实际数据同步至第二系统，具体地，可以由工作组件的同步器执行同步数据的各上层接口，同步数据的各上层接口即可以通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
在本申请另一个优选的实施例中，步骤260，包括：
子步骤261，由工作组件的同步器根据限速指令，调用与限速指令 相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
为了调节数据迁移过程中的数据传输速度，在本申请实施例中，可以输入限速指令，其中包含需要可以接受的数据迁移过程中的数据传输速度，其可以为具体的限速数值，也可以为一个速度范围，进而可以由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，同步数据的各上层接口可以通过抽象接口类调用第二系统的底层操作接口，将所述实际数据按照限速数值的速度或者是以速度范围内的数据传输速递将实际数据同步至第二系统。其中，与限速指令相应的线程可以存在于前述的工作线程池中，也可以为另外创建的一个线程，对此本申请实施例不加以限定。
在本申请实施例中，可以接收将第一系统中的数据迁移至第二系统的迁移任务，所述第一系统和第二系统为多个系统中的任意两个系统，然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
从而实现了任意两个数据端可以进行双向的数据传输和迁移，包括数据端内部的bucket间迁移。其次，本申请所述的数据迁移技术具有良好的扩展功能，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
另外，本申请可以支持分布式数据传输，可以极大提升数据迁移的效率。比以第一系统和第二系统为两个地区的OSS为例，从一个地区OSS的一个bucket的数据传输到另一个地区的bucket中，测试方法采用普通网 络路径，先从一个bucket下载出来，通过内网络链路，再上传存储到另外一个bucket。其传输采用分布式集群环境：阿里云杭州ECS(Elastic Compute Service,云服务器)，单台服务器配置为4核CPU、4G内存、500Mbps内网带宽，总共5个节点。其测试示例图下表(一)：
Figure PCTCN2017077722-appb-000001
表(一)
其中，效率一列单位是MB/s(兆比特每秒)。
可以看到，随着数据规模的增加，本申请的分布式传输的效率优势更加明显，跟集群的节点规模成正比例关系。
需要说明的是，对于方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请实施例并不受所描述的动作顺序的限制，因为依据本申请实施例，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作并不一定是本申请实施例所必须的。
实施例三
参照图3，示出了本申请的一种数据迁移装置实施例的结构框图，具 体可以包括如下模块：
任务接收模块310，适于接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统。
迁移模块320，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
在本申请另一个优选地实施例中，所述迁移模块320，包括：
上层接口确定子模块，适于确定属于获取数据的各上层接口，以及属于同步数据的各上层接口。
数据获取子模块，适于对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据。
数据同步子模块，适于对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
在本申请另一个优选地实施例中，所述数据获取子模块，包括：
第一底层操作接口确定子模块，适于对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口。
第一底层操作接口调用子模块，适于对于属于获取数据的各上层接口中的第二上层接口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二上层接口不为各系统通用的接口。
在本申请另一个优选地实施例中，所述数据同步子模块，包括：
第二底层操作接口确定子模块，适于对于属于同步数据的各上 层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口。
第二底层操作接口调用子模块，适于对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
在本申请另一个优选地实施例中，在所述迁移模块320之后，还包括：
迁移判断模块，适于判断将第一系统中的数据迁移至第二系统是否成功；如果失败，则重复执行迁移模块，直至重试次数达到重试阈值或者在重试阈值内成功。
在本申请另一个优选地实施例中，在所述迁移模块320之后，还包括：
暂停指令判断模块，适于判断是否接收到暂停指令；如果接收到暂停指令，则进入暂停记录子模块。
暂停记录模块，适于暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置。
则进一步的，在所述暂停指令判断模块之后，还包括：
重启指令判断模块，适于判断是否接收到重启指令；如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新进入迁移模块。
在本申请另一个优选地实施例中，当第一系统为客户端文件系统时，在任务接收模块310之前，还包括：
应用程序安装模块300，适于在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
在本申请另一个优选地实施例中，在所述迁移模块320之后，还包括：
数据监控模块，适于监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；如果出现变更，则进入变更数据同步模块。
变更数据同步模块，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
在本申请另一个优选地实施例中，在所述数据获取子模块之后，还包括：
签名子模块，适于调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名。
进一步的，在所述数据同步子模块之后，还包括：
签名验证子模块，适于调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
在本申请另一个优选地实施例中，在所述迁移模块320之后，还包括：
第一文件列表获取模块，适于针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文件列表。
第二文件列表获取模块，适于针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表。
差异提示模块，适于比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
在本申请另一个优选地实施例中，在所述上层接口确定子模块之后，还包括：
第一身份验证模块，适于针对第一系统调用身份验证的上层接口， 并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证。
第二身份验证模块，适于针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身份信息进行验证。
本申请实施例可以接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统，然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
首先，实现了任意两个数据端可以进行双向的数据传输和迁移，包括数据端内部的bucket间迁移。
其次，本申请所述的数据迁移技术具有良好的扩展功能，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
实施例四
参照图4，示出了本申请的一种数据迁移装置实施例的结构框图，具体可以包括如下模块：
任务接收模块410，适于接收将第一系统中的数据迁移至第二系统的迁移任务；所述第一系统和第二系统为多个系统中的任意两个系统。
迁移模块420，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。具体包括：
上层接口确定子模块421，适于确定属于获取数据的各上层接口，以及属于同步数据的各上层接口。
数据获取子模块422，适于对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据。具体包括：
元数据获取子模块4221，适于由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据。
任务分发子模块4222，适于由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器。
在本申请另一个优选地实施例中，所述任务分发子模块4222，包括：
数据量大小提取子模块，适于提取元数据中记录的数据量大小。
第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器。
第二元数据分发子模块，适于如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器。
在本申请另一个优选地实施例中，所述任务分发子模块4222，包括：
数据量大小提取子模块，适于提取元数据中记录的数据量大小。
第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器。
第三元数据分发子模块，适于如果元数据中记录的数据量不 小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器。
元数据切分子模块，适于如果元数据中记录的数据量不小于第二阈值时，则调用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片。
将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
实际数据获取子模块4223，适于由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
数据同步子模块423，适于对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。具体包括：
实际数据同步子模块4231，适于由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
在本申请另一个优选地实施例中，所述实际数据同步子模块4231，包括：
第一实际数据同步子模块，适于由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
在本申请实施例中，可以接收将第一系统中的数据迁移至第二系统的 迁移任务，所述第一系统和第二系统为多个系统中的任意两个系统，然后调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
从而实现了任意两个数据端可以进行双向的数据传输和迁移，包括数据端内部的bucket间迁移。其次，本申请所述的数据迁移技术具有良好的扩展功能，可以实现将任意的数据终端，包括尚未支持数据迁移的云存储产品，如百度对象存储、金山对象存储，与其它任意一个数据端进行数据的双向传输迁移。
另外，本申请同时支持单机数据传输以及分布式数据传输，可以极大提升数据迁移的效率。
对于装置实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。
本领域内的技术人员应明白，本申请实施例的实施例可提供为方法、装置、或计算机程序产品。因此，本申请实施例可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用同步介质(包括但不限于磁盘同步器、CD-ROM、光学同步器等)上实施的计算机程序产品的形式。
在一个典型的配置中，所述计算机设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。内存可能包括计算机可读介质中的非永久性同步器，随机存取同步器(RAM)和/或非易失性内存等形式，如只读同步器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息同步。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的同步介质的例子包括，但不限于相变内存(PRAM)、静态随机存取同步器(SRAM)、动态随机存取同步器(DRAM)、其他类型的随机存取同步器(RAM)、只读同步器(ROM)、电可擦除可编程只读同步器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读同步器(CD-ROM)、数字多功能光盘(DVD)或其他光学同步、磁盒式磁带，磁带磁磁盘同步或其他磁性同步设备或任何其他非传输介质，可用于同步可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括非持续性的电脑可读媒体(transitory media)，如调制的数据信号和载波。
本申请实施例是参照根据本申请实施例的方法、终端设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/ 或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理终端设备的处理器以产生一个服务器，使得通过计算机或其他可编程数据处理终端设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可同步在能引导计算机或其他可编程数据处理终端设备以特定方式工作的计算机可读同步器中，使得同步在该计算机可读同步器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理终端设备上，使得在计算机或其他可编程终端设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程终端设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本申请实施例的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请实施例范围的所有变更和修改。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术 语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者终端设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者终端设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者终端设备中还存在另外的相同要素。
以上对本申请所提供的一种数据迁移方法和一种数据迁移装置，进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (38)

  1. 一种数据迁移方法，其特征在于，包括：
    接收将第一系统中的数据迁移至第二系统的迁移任务；
    调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
  2. 根据权利要求1所述的方法，其特征在于，所述调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤，包括：
    确定属于获取数据的各上层接口，以及属于同步数据的各上层接口；
    对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据；
    对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
  3. 根据权利要求2所述的方法，其特征在于，对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口的步骤，包括：
    对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口；
    对于属于获取数据的各上层接口中的第二上层接口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二上层接口不为各系统通用的接口。
  4. 根据权利要求2或3所述的方法，其特征在于，对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口的步骤， 包括：
    对于属于同步数据的各上层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口；
    对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
  5. 根据权利要求2所述的方法，其特征在于，所述对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据的步骤，包括：
    由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据；
    由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器；
    由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
  6. 根据权利要求5所述的方法，其特征在于，所述对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统的步骤，包括：
    由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
  7. 根据权利要求5或6所述的方法，其特征在于，所述管理组件在 集群的一台服务器中，所述工作组件在集群的各台服务器中。
  8. 根据权利要求7所述的方法，其特征在于，所述由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器的步骤，包括：
    提取元数据中记录的数据量大小；
    如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
    如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器；
  9. 根据权利要求7所述的方法，其特征在于，所述由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器的步骤，包括：
    提取元数据中记录的数据量大小；
    如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
    如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器；
    如果元数据中记录的数据量不小于第二阈值时，则调用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据对应的大文件切片；
    将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
  10. 根据权利要求8所述的方法，其特征在于，所述集群的各台服务器中之间采用JGroups通信协议传输数据。
  11. 根据权利要求6所述的方法，其特征在于，所述由工作组件的同步器执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统的步骤，包括：
    由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
  12. 根据权利要求1或2所述的方法，其特征在于，调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
    判断将第一系统中的数据迁移至第二系统是否成功；
    如果失败，则重复执行调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤，直至重试次数达到重试阈值或者在重试阈值内成功。
  13. 根据权利要求1或2所述的方法，其特征在于，在调用对应所述迁移任务的各上层接口，通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的过程中，还包括：
    判断是否接收到暂停指令；
    如果接收到暂停指令，则暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置；
    则进一步的，在暂停将第一系统的数据迁移至第二系统之后，还包括：
    判断是否接收到重启指令；
    如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置 开始，重新调用对应所述迁移任务的各上层接口，通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤。
  14. 根据权利要求1或2所述的方法，其特征在于，当第一系统为客户端文件系统时，在接收将第一系统中的数据迁移至第二系统的迁移任务的步骤之前，还包括：
    在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
  15. 根据权利要求14所述的方法，其特征在于，所述调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
    监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；
    如果出现变更，则调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
  16. 根据权利要求2所述的方法，其特征在于，所述对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据的步骤之后，还包括：
    调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名；
    进一步的，对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统的步骤之后，还包括：
    调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
  17. 根据权利要求1所述的方法，其特征在于，在调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统的步骤之后，还包括：
    针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文件列表；
    针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表；
    比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
  18. 根据权利要求2所述的方法，其特征在于，所述迁移任务由客户端发送，所述迁移任务包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。
  19. 根据权利要求18所述的方法，其特征在于，在确定属于获取数据的各上层接口，以及属于同步数据的各上层接口的步骤之后，还包括：
    针对第一系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证；
    针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身 份信息进行验证。
  20. 一种数据迁移装置，其特征在于，包括：
    任务接收模块，适于接收将第一系统中的数据迁移至第二系统的迁移任务；
    迁移模块，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将第一系统的数据迁移至第二系统。
  21. 根据权利要求20所述的装置，其特征在于，所述迁移模块，包括：
    上层接口确定子模块，适于确定属于获取数据的各上层接口，以及属于同步数据的各上层接口；
    数据获取子模块，适于对于属于获取数据的各上层接口，通过抽象接口类调用第一系统的底层操作接口，以从第一系统获取数据；
    数据同步子模块，适于对于属于同步数据的各上层接口，通过抽象接口类调用第二系统的底层操作接口，以将从第一系统获取的数据同步至第二系统。
  22. 根据权利要求21所述的装置，其特征在于，所述数据获取子模块，包括：
    第一底层操作接口确定子模块，适于对于属于获取数据的各上层接口中的第一上层接口，以抽象接口类中与第一上层接口对应的第一抽象类接口作为底层操作接口以直接调用；所述第一上层接口为各系统通用的接口；
    第一底层操作接口调用子模块，适于对于属于获取数据的各上层接口中的第二上层接口，通过抽象接口类生成针对第一系统的第一接口类，在所述第一接口类中调用与各第二上层接口对应的底层操作接口；所述第二 上层接口不为各系统通用的接口。
  23. 根据权利要求21或22所述的装置，其特征在于，所述数据同步子模块，包括：
    第二底层操作接口确定子模块，适于对于属于同步数据的各上层接口中的第三上层接口，以抽象接口类中与第三上层接口对应的第二抽象类接口作为底层操作接口以直接调用；所述第三上层接口为各系统通用的接口；
    第二底层操作接口调用子模块，适于对于属于同步数据的各上层接口中的第四上层接口，通过抽象接口类生成针对第二系统的第二接口类，在所述第二接口类中调用与各第四上层接口对应的底层操作接口；所述第四上层接口不为各系统通用的接口。
  24. 根据权利要求21所述的装置，其特征在于，所述数据获取子模块，包括：
    元数据获取子模块，适于由管理组件的扫描器执行获取元数据的上层接口，所述获取元数据的上层接口通过抽象类接口调用第一系统的获取元数据的底层操作接口，从第一系统获取所述迁移任务对应的各元数据；
    任务分发子模块，适于由管理组件的分发器，基于各元数据生成同步任务并分发给各工作组件的接收器；
    实际数据获取子模块，适于由工作组件的接收器根据接收到同步任务，执行获取实际数据的上层接口，所述获取实际数据的上层接口通过抽象类接口调用第一系统的获取实际数据的底层获取接口，从第一系统获取所述迁移任务的实际数据。
  25. 根据权利要求24所述的装置，其特征在于，所述数据同步子模块，包括：
    实际数据同步子模块，适于由工作组件的同步器执行同步数据的各上 层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
  26. 根据权利要求24或25所述的装置，其特征在于，所述管理组件在集群的一台服务器中，所述工作组件在集群的各台服务器中。
  27. 根据权利要求26所述的装置，其特征在于，所述任务分发子模块，包括：
    数据量大小提取子模块，适于提取元数据中记录的数据量大小；
    第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
    第二元数据分发子模块，适于如果元数据记录的数据量不小于第一阈值，则将所述元数据作为同步任务分发给负载最小的服务器中的工作组件的接收器。
  28. 根据权利要求26所述的装置，其特征在于，所述任务分发子模块，包括：
    数据量大小提取子模块，适于提取元数据中记录的数据量大小；
    第一元数据分发子模块，适于如果元数据中记录的数据量小于第一阈值，直接将所述元数据作为同步任务分发给管理组件所在的服务器中的工作组件的接收器；
    第三元数据分发子模块，适于如果元数据中记录的数据量不小于第一阈值但是小于第二阈值时，则将所述元数据生成同步任务分发给负载最小的服务器中的工作组件的接收器；
    元数据切分子模块，适于如果元数据中记录的数据量不小于第二阈值时，则调用对大文件切片的上层接口，所述对大文件切片的上层接口通过抽象接口类调用第二系统的对大文件切片的底层操作接口，将所述元数据 对应的大文件切片；
    将对所述元数据的不同切片的切片信息生成同步任务，并分发给负载低的多个服务器的工作组件的接收器。
  29. 根据权利要求27所述的装置，其特征在于，所述集群的各台服务器中之间采用JGroups通信协议传输数据。
  30. 根据权利要求25所述的装置，其特征在于，所述实际数据同步子模块，包括：
    第一实际数据同步子模块，适于由工作组件的同步器根据限速指令，调用与限速指令相应的线程执行同步数据的各上层接口，所述同步数据的各上层接口通过抽象接口类调用第二系统的底层操作接口，将所述实际数据同步至第二系统。
  31. 根据权利要求20或21所述的装置，其特征在于，在所述迁移模块之后，还包括：
    迁移判断模块，适于判断将第一系统中的数据迁移至第二系统是否成功；如果失败，则重复执行迁移模块，直至重试次数达到重试阈值或者在重试阈值内成功。
  32. 根据权利要求20或21所述的装置，其特征在于，在所述迁移模块之后，还包括：
    暂停指令判断模块，适于判断是否接收到暂停指令；如果接收到暂停指令，则进入暂停记录子模块；
    暂停记录模块，适于暂停将第一系统的数据迁移至第二系统，并记录暂停时刻第一系统中已迁移的数据位置；
    则进一步的，在所述暂停指令判断模块之后，还包括：
    重启指令判断模块，适于判断是否接收到重启指令；如果接收到重启指令，则根据暂停时刻第一系统中已迁移的数据位置开始，重新进入迁移 模块。
  33. 根据权利要求20或21所述的装置，其特征在于，当第一系统为客户端文件系统时，在任务接收模块之前，还包括：
    应用程序安装模块，适于在客户端中安装应用程序，所述应用程序针对指定文件模拟各底层操作接口。
  34. 根据权利要求33所述的装置，其特征在于，在所述迁移模块之后，还包括：
    数据监控模块，适于监控客户端文件系统是否与最近一次迁移到第二系统的数据出现变更；如果出现变更，则进入变更数据同步模块；
    变更数据同步模块，适于调用对应所述迁移任务的各上层接口，并由各上层接口通过抽象接口类调用第一系统和第二系统的底层操作接口，以将客户端文件系统的变更数据同步至第二系统。
  35. 根据权利要求21所述的装置，其特征在于，在所述数据获取子模块之后，还包括：
    签名子模块，适于调用签名的上层接口，并由所述签名的上层接口调用第一系统的签名的底层操作接口，对所述数据进行签名；
    进一步的，在所述数据同步子模块之后，还包括：
    签名验证子模块，适于调用签名认证的上层接口，并由所签名认证的上层接口调用第二系统的签名认证的底层操作接口，对所述数据进行签名验证。
  36. 根据权利要求20所述的装置，其特征在于，在所述迁移模块之后，还包括：
    第一文件列表获取模块，适于针对第一系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第一系统的底层操作接口，获取第一系统的文件列表；
    第二文件列表获取模块，适于针对第二系统调用文件列表的上层接口，并由所述调用文件列表的上层接口通过抽象接口类调用第二系统的底层操作接口，获取第二系统的文件列表；
    差异提示模块，适于比较第一系统的文件列表和第二系统的文件列表，并将两者之间的差异提示给客户端。
  37. 根据权利要求21所述的装置，其特征在于，所述迁移任务由客户端发送，所述迁移任务包括第一系统的标识信息、用户在第一系统的身份信息、待迁移数据在第一系统中的文件信息、第二系统的标识信息、用户在第二系统的身份信息。
  38. 根据权利要求37所述的装置，其特征在于，在所述上层接口确定子模块之后，还包括：
    第一身份验证模块，适于针对第一系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第一系统的底层操作接口，对用户在第一系统的身份信息进行验证；
    第二身份验证模块，适于针对第二系统调用身份验证的上层接口，并由所述身份验证的上层接口通过抽象接口类调用第二系统的底层操作接口，对用户在第二系统的身份信息进行验证。
PCT/CN2017/077722 2016-03-31 2017-03-22 一种数据迁移方法和装置 Ceased WO2017167100A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US16/147,376 US11354283B2 (en) 2016-03-31 2018-09-28 Data migration method and apparatus
US17/659,448 US11940965B2 (en) 2016-03-31 2022-04-15 Data migration method and apparatus
US18/587,138 US20240193136A1 (en) 2016-03-31 2024-02-26 Data migration method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610202889.0A CN107291750B (zh) 2016-03-31 2016-03-31 一种数据迁移方法和装置
CN201610202889.0 2016-03-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/147,376 Continuation US11354283B2 (en) 2016-03-31 2018-09-28 Data migration method and apparatus

Publications (1)

Publication Number Publication Date
WO2017167100A1 true WO2017167100A1 (zh) 2017-10-05

Family

ID=59963473

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/077722 Ceased WO2017167100A1 (zh) 2016-03-31 2017-03-22 一种数据迁移方法和装置

Country Status (4)

Country Link
US (3) US11354283B2 (zh)
CN (1) CN107291750B (zh)
TW (1) TWI739804B (zh)
WO (1) WO2017167100A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113312147A (zh) * 2021-06-11 2021-08-27 中国科学技术大学 一种对象存储跨集群海量数据迁移方法及系统
US11354283B2 (en) 2016-03-31 2022-06-07 Alibaba Group Holding Limited Data migration method and apparatus

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021647B (zh) * 2017-11-30 2020-10-16 苏州浪潮智能科技有限公司 一种文件迁移方法、系统、介质及设备
CN109358899B (zh) * 2018-12-05 2021-07-30 江苏风云科技服务有限公司 共享货架中资源版本的更新方法及装置
CN112019585A (zh) * 2019-05-31 2020-12-01 北京车和家信息技术有限公司 云存储管理方法、系统、电子设备及计算机可读存储介质
CN112073811B (zh) * 2019-06-10 2023-02-28 海信视像科技股份有限公司 一种文件传输调度方法及显示设备
CN110413418B (zh) * 2019-06-25 2022-04-05 北京三快在线科技有限公司 缓存同步装置及方法，缓存同步系统、电子设备
CN112559537B (zh) 2019-09-25 2022-04-29 阿里巴巴集团控股有限公司 云存储区间更新方法、装置、电子设备及计算机存储介质
CN111459415B (zh) * 2020-04-15 2023-05-23 腾讯云计算(长沙)有限责任公司 跨云数据迁移方法、装置、设备及存储介质
MX2022015104A (es) * 2020-05-29 2023-03-01 Pollen Inc Reenrutamiento de recursos para gestionar plataformas.
US11954532B2 (en) 2020-05-29 2024-04-09 Pollen, Inc. Reconciling resource management platforms
CN112600803B (zh) * 2020-12-02 2022-07-19 上海哔哩哔哩科技有限公司 Web端数据签名方法、装置及计算机设备
CN112905984A (zh) * 2021-03-09 2021-06-04 浙江网商银行股份有限公司 权限控制方法、装置及电子设备
US20230133014A1 (en) * 2021-10-29 2023-05-04 Netapp, Inc. Methods and Systems for Storage Virtual Machine Migration Between Clusters of a Networked Storage System
CN113986886B (zh) * 2021-11-10 2025-09-16 京东科技信息技术有限公司 数据结构服务器集群配置变更的数据迁移方法及装置
US12526617B2 (en) * 2021-12-08 2026-01-13 Palo Alto Networks, Inc. Targeted PDU capture by a network device for roaming detection in wireless networks
CN114443623A (zh) * 2022-02-14 2022-05-06 浪潮云信息技术股份公司 面向混合云场景的数据迁移任务调度系统
CN114579180B (zh) * 2022-04-28 2022-08-09 北京聚通达科技股份有限公司 数据迁移的方法、装置、电子设备及存储介质
CN115134349B (zh) * 2022-06-06 2024-02-23 蚂蚁区块链科技(上海)有限公司 一种执行传输任务的方法、装置、介质及设备
US12487966B2 (en) 2022-11-18 2025-12-02 Bank Of America Corporation Dynamic file selection process to transfer required data file between different servers in near real-time
CN116974987A (zh) * 2023-07-31 2023-10-31 济南浪潮数据技术有限公司 一种监控和迁移状态机的方法、系统、设备和介质
CN119806395B (zh) * 2024-11-29 2025-11-04 天翼云科技有限公司 分布式存储系统的数据管理方法、电子设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101079902A (zh) * 2007-06-29 2007-11-28 清华大学 海量数据分级存储方法
US20130080488A1 (en) * 2011-09-23 2013-03-28 Alibaba Group Holding Limited Management Apparatus and Method of Distributed Storage System
CN103605728A (zh) * 2013-11-18 2014-02-26 浪潮(北京)电子信息产业有限公司 一种数据分级存储方法及系统
CN103856724A (zh) * 2012-12-05 2014-06-11 成都索贝数码科技股份有限公司 一种制作应急系统
CN104424283A (zh) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 一种数据迁移的系统和数据迁移的方法

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6249866B1 (en) * 1997-09-16 2001-06-19 Microsoft Corporation Encrypting file system and method
GB0011954D0 (en) * 2000-05-17 2000-07-05 Univ Surrey Protocol stacks
US20040098419A1 (en) * 2002-11-18 2004-05-20 International Business Machines Corporation Method and apparatus for a migration assistant
US7805583B1 (en) * 2003-04-23 2010-09-28 Emc Corporation Method and apparatus for migrating data in a clustered computer system environment
US7448022B1 (en) * 2004-02-10 2008-11-04 Prasad Ram Dynamic software composition in a component-based software system
US7624283B2 (en) * 2006-02-13 2009-11-24 International Business Machines Corporation Protocol for trusted platform module recovery through context checkpointing
US8214811B2 (en) * 2006-10-23 2012-07-03 International Business Machines Corporation Instantiating an interface or abstract class in application code
CN101282345B (zh) * 2008-03-03 2011-12-07 北京航空航天大学 一种基于形式化描述和状态迁移的xml转换方法
JP5331555B2 (ja) * 2009-04-23 2013-10-30 株式会社日立製作所 データ移行システムおよびデータ移行方法
JP5284905B2 (ja) * 2009-08-12 2013-09-11 富士通株式会社 データ移行方法、及びプログラム
US20110295925A1 (en) * 2010-03-30 2011-12-01 Jason Lieblich Systems and methods for selecting an alternative computing infrastructure
JP5595530B2 (ja) * 2010-10-14 2014-09-24 株式会社日立製作所 データ移行システム及びデータ移行方法
US8370300B2 (en) * 2011-05-05 2013-02-05 Hitachi, Ltd. Method and apparatus of data protection
US8914381B2 (en) * 2012-02-16 2014-12-16 Apple Inc. Correlation filter
US9298521B1 (en) * 2013-04-29 2016-03-29 Seagate Technology Llc Command sets and functions
US9098364B2 (en) * 2013-07-09 2015-08-04 Oracle International Corporation Migration services for systems
US9461969B2 (en) * 2013-10-01 2016-10-04 Racemi, Inc. Migration of complex applications within a hybrid cloud environment
US10313243B2 (en) * 2015-02-24 2019-06-04 Commvault Systems, Inc. Intelligent local management of data stream throttling in secondary-copy operations
WO2016138474A1 (en) * 2015-02-26 2016-09-01 Bittitan, Inc. Data migration systems and methods including archive migration
US10324914B2 (en) * 2015-05-20 2019-06-18 Commvalut Systems, Inc. Handling user queries against production and archive storage systems, such as for enterprise customers having large and/or numerous files
US10157196B2 (en) * 2015-08-12 2018-12-18 Salesforce.Com, Inc. Intra-platform data migration
US10331656B2 (en) * 2015-09-25 2019-06-25 Microsoft Technology Licensing, Llc Data migration validation
US10496291B2 (en) * 2015-12-18 2019-12-03 Sap Se Maintaining data integrity during data migration
CN107291750B (zh) 2016-03-31 2020-11-06 阿里巴巴集团控股有限公司 一种数据迁移方法和装置
CN107247594B (zh) * 2017-06-12 2020-10-16 武汉斗鱼网络科技有限公司 逻辑系统及其实现方法、计算设备及计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101079902A (zh) * 2007-06-29 2007-11-28 清华大学 海量数据分级存储方法
US20130080488A1 (en) * 2011-09-23 2013-03-28 Alibaba Group Holding Limited Management Apparatus and Method of Distributed Storage System
CN103856724A (zh) * 2012-12-05 2014-06-11 成都索贝数码科技股份有限公司 一种制作应急系统
CN104424283A (zh) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 一种数据迁移的系统和数据迁移的方法
CN103605728A (zh) * 2013-11-18 2014-02-26 浪潮(北京)电子信息产业有限公司 一种数据分级存储方法及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11354283B2 (en) 2016-03-31 2022-06-07 Alibaba Group Holding Limited Data migration method and apparatus
US11940965B2 (en) 2016-03-31 2024-03-26 Alibaba Group Holding Limited Data migration method and apparatus
CN113312147A (zh) * 2021-06-11 2021-08-27 中国科学技术大学 一种对象存储跨集群海量数据迁移方法及系统
CN113312147B (zh) * 2021-06-11 2022-12-30 中国科学技术大学 一种对象存储跨集群海量数据迁移方法及系统

Also Published As

Publication number Publication date
TW201741901A (zh) 2017-12-01
US20190034422A1 (en) 2019-01-31
TWI739804B (zh) 2021-09-21
CN107291750A (zh) 2017-10-24
US20220245101A1 (en) 2022-08-04
US11940965B2 (en) 2024-03-26
CN107291750B (zh) 2020-11-06
US11354283B2 (en) 2022-06-07
US20240193136A1 (en) 2024-06-13

Similar Documents

Publication Publication Date Title
CN107291750B (zh) 一种数据迁移方法和装置
CN110462589B (zh) 本地装置协调器中的按需代码执行
CN108027828B (zh) 与无状态同步节点的托管文件同步
CN112035062B (zh) 云计算的本地存储的迁移方法、计算机设备及存储介质
WO2019184164A1 (zh) 自动部署Kubernetes从节点的方法、装置、终端设备及可读存储介质
CN107357660A (zh) 一种虚拟资源的分配方法及装置
Han et al. {MetaSync}: File Synchronization Across Multiple Untrusted Storage Services
CN106878363B (zh) 一种信息处理方法、装置及系统
WO2019057055A1 (zh) 一种任务处理方法、装置、电子设备及存储介质
CN108200219A (zh) 数据同步方法、装置、服务器及存储介质
WO2024016624A1 (zh) 多集群访问方法和系统
EP3545408A1 (en) Localized device coordinator with on-demand code execution capabilities
CN113541987A (zh) 一种更新配置数据的方法和装置
CN106941522B (zh) 轻量级分布式计算平台及其数据处理方法
US12086158B2 (en) Hybrid cloud asynchronous data synchronization
CN111431951B (zh) 一种数据处理方法、节点设备、系统及存储介质
CN113297158B (zh) 一种云安全产品管理方法、装置、设备及存储介质
CN111596933A (zh) 文件处理方法、装置、电子设备及计算机可读存储介质
CN117234687A (zh) 多云场景下异构算力调度的方法、装置、设备及存储介质
US11455181B1 (en) Cross-network connector appliances
CN114979104A (zh) 一种实现无服务器的系统、方法、计算机设备及存储介质
CN114327866A (zh) 一种分布式镜像库的资源调度方法、系统及相关装置
US12373309B2 (en) Providing status notifications of file-copy operations to backup clients
Casciaro Development of a local cloud system based on P2P file sharing: the myP2PSync file synchronization system
US10223019B1 (en) Policy-driven volume expansion

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17773130

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17773130

Country of ref document: EP

Kind code of ref document: A1