WO2024082857A1 - Procédé et système de migration de données et appareil associé - Google Patents

Procédé et système de migration de données et appareil associé Download PDF

Info

Publication number
WO2024082857A1
WO2024082857A1 PCT/CN2023/117343 CN2023117343W WO2024082857A1 WO 2024082857 A1 WO2024082857 A1 WO 2024082857A1 CN 2023117343 W CN2023117343 W CN 2023117343W WO 2024082857 A1 WO2024082857 A1 WO 2024082857A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
storage area
migration
server
migrated
Prior art date
Application number
PCT/CN2023/117343
Other languages
English (en)
Chinese (zh)
Inventor
黄潘潘
Original Assignee
超聚变数字技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 超聚变数字技术有限公司 filed Critical 超聚变数字技术有限公司
Publication of WO2024082857A1 publication Critical patent/WO2024082857A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • the embodiments of the present application relate to the technical field of data migration, and in particular, to a method, system and related apparatus for data migration.
  • the present application provides a method, system and related devices for data migration, which can not only realize online data migration, but also improve the efficiency of data migration and provide a good user experience.
  • the present application provides a data migration method, which is applied to a computing device, comprising:
  • the first data is processed according to the write request, and the first data is migrated from the source data storage area to the target storage area for storage;
  • the first data is stored in a source data storage area, and the source data storage area and the target storage area are located in the same computing device or in different computing devices.
  • a migration operation of the first data may be triggered by a read request or a write request for the first data.
  • the first data can be sent and received between the different computing devices.
  • the migration efficiency can be improved on the basis of realizing online migration.
  • the method after receiving a read request or a write request for the first data, the method further includes:
  • the first data is processed according to the read request or the write request.
  • the first data when the first data does not need to be migrated or has already been migrated, the first data can be directly processed for read requests or write requests without migrating the first data from the source data storage area to the target storage area.
  • the first data in various scenarios can be fully covered, various situations can be taken into consideration, and multiple migrations can be avoided.
  • the storage address of the first data in the source data storage area is a first address
  • the processing of the first data according to the read request and migrating the first data from the source data storage area to the target storage area for storage specifically includes:
  • the first data may be read from the first address according to the read request and stored in a second address in the target storage area.
  • the first data may be stored in a block storage, object storage, or other storage methods, which are not limited thereto.
  • the data block or the single object needs to be split according to the size of the data actually read so that the partial data after the splitting process is exactly the first data.
  • the first data may be returned to the user.
  • the storage address of the first data in the source data storage area is a first address
  • the storage method of the first data is block storage
  • the processing of the first data according to the write request and migrating the first data from the source data storage area to the target storage area specifically includes:
  • the first data is read from the first address according to the write request and stored in the second address of the target storage area; and the first data stored in the first address and the second address is written;
  • the data block is data that is migrated once in the block storage; the alignment means that the size of the first data is an integer multiple of the size of the data block.
  • a prerequisite for responding to a write request is that write-write mutual exclusion must be satisfied, that is, the data targeted by two write requests cannot have an intersection.
  • the user may be notified of the completion of the response to the write request.
  • the computing device when the computing device determines that the size of the data block of the first data is not aligned, the computing device continues to acquire a mutex lock of the data block;
  • the second data is read from the first location and stored in a second address in the target storage area.
  • the storage method of the first data is object storage, and the storage address of the first data in the source data storage area is a first address.
  • the processing of the first data according to the write request and migrating the first data from the source data storage area to the target storage area specifically includes:
  • the first data is read from the first address and stored in the second address in the target storage area, and a write process is performed on the first data stored in the first address and the second address.
  • the write processing operation of the object storage can only act on the entire object, so there is no need to perform alignment judgment between the first data and a single object.
  • the migration metadata record table is updated, and the migration metadata record table records the migration status of the first data.
  • the migration status includes any one of the following: the first data needs to be migrated but has not been migrated, has been migrated, and does not need to be migrated.
  • the state of the migration metadata record table may be converted from "N" to "Y".
  • a background migration task may be started to determine the data to be migrated, and the system begins to migrate the data blocks or single objects to be migrated in sequence in the background.
  • an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, comprising computer instructions.
  • the computer instructions When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect.
  • the electronic device provided in the second aspect and the computer-readable storage medium provided in the third aspect are both used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.
  • FIG1 is a schematic diagram of a data migration system according to an embodiment of the present application.
  • FIG. 2 is a schematic block diagram of the structure of a server 100 provided in an embodiment of the present application.
  • FIG3 is a flow chart of a method for data migration in response to a read request provided in an embodiment of the present application.
  • FIG4 is a flow chart of another data migration method in response to a read request provided in an embodiment of the present application.
  • FIG5 is a flow chart of a data migration method in response to a write request provided in an embodiment of the present application.
  • FIG6 is a flow chart of another method for data migration in response to a write request provided in an embodiment of the present application.
  • FIG. 7 is a flow chart of a background data migration method provided in an embodiment of the present application.
  • FIG8 is a flow chart of another background data migration method provided in an embodiment of the present application.
  • first and second are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated.
  • a feature defined as “first” or “second” may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality” means two or more.
  • GUI graphical user interface
  • Data migration can refer to the process of selecting, preparing, extracting, and transforming data and permanently transferring it from one computer storage system to another. There are many reasons for data migration, which may include but are not limited to server or storage device replacement, maintenance or upgrade, application migration, website integration, disaster recovery, and data center migration. In some implementations, data migration can be understood as copying data stored in one computer storage system and copying it to another computer storage system.
  • some data migration methods do not support online migration. For example, if a user needs to access the data being migrated, he needs to wait until the data migration is completed before accessing the data. In other words, the electronic device cannot respond to the read and write requests for the data during the data migration process.
  • Some data migration methods are less efficient. For example, when the system uses snapshots to migrate data, it is necessary to export data from the source data storage area and then import the exported data into the target storage area after the migration. If during the data migration process, a user operation triggers a write request resulting in new data, and the new data also needs to be migrated, then it is necessary to repeatedly export the new data from the source data storage area and then import the exported new data into the target storage area after the migration.
  • an embodiment of the present application provides a data migration method.
  • data migration is divided into two forms: foreground migration and background migration.
  • the foreground migration method specifically refers to: firstly determining a data set to be migrated, wherein the data set to be migrated may be determined by a manager.
  • the system receives a read request or a write request for first data from a user, the system starts to migrate the first data.
  • the first data belongs to the data set to be migrated.
  • the system When the system receives a read request for the operation of the first data from the user, the system reads the first data from the source data storage area, migrates it to the target storage area, and then returns the first data in the target storage area to the user.
  • the system When the system receives a write request from the user for an operation on the first data, the system reads the first data from the source data storage area, migrates it to the target storage area, then performs double-write processing on the first data in the source data storage area and the first data in the target storage area, and then returns the first data in the target storage area to the user; or the system first reads the first data, performs write processing, stores the first data after the write processing in the source data storage area, and then migrates the first data after the write processing to the source data storage area.
  • the background migration method specifically refers to: firstly determining the data to be migrated, and then the system starts to migrate the data blocks or single objects to be migrated in sequence in the background.
  • foreground migration has a higher priority than background migration. That is, when the system receives a read request or a write request from a user for the first data, the system may preferentially select the foreground migration method for migration. When the system does not receive any read request or write request, the system may only select the background migration method to migrate the data targeted by the migration instruction.
  • a storage address at which the first data is stored in the source data storage area may be referred to as a first address
  • a storage address at which the first data is stored in the target data storage area may be referred to as a second address.
  • data after the first data is written may be referred to as second data.
  • the above method can ensure that when the system receives a user's read request or write request for the first data, the system only needs to read the first data once in the source data storage area to ensure that the user's read request or write request and migration task are implemented.
  • the migration efficiency can be improved compared with the method in which the system first receives the migration instruction and then receives the read request or write request. This is because the system cannot guarantee that the data received by the migration instruction and the data received by the read request or write request are the same data during the migration process. This will cause the system to read a certain data twice in the source data storage area when implementing the user's read request or write request and the migration task.
  • the data migration method provided in the embodiment of the present application can not only realize the online migration of data, but also improve the efficiency of data migration and provide a good user experience.
  • Figure 1 is a schematic diagram of the framework of a data migration system provided in an embodiment of the present application.
  • the data migration system may include but is not limited to a source data storage area, a target data storage area, a data migration tool, a storage medium, etc.
  • the data migration system may be applied to application scenarios such as local data migration to the cloud, data migration between clouds, and cloud data migration to the local.
  • the above-mentioned source data storage area and target data storage area may refer to the logical storage location of data blocks or single objects in the server, and the above-mentioned storage medium may refer to the physical storage location of data blocks or single objects in the server.
  • the source data storage area may refer to a database storing unmigrated data.
  • Common databases may include but are not limited to relational database manager (My Structured Query Language, MYSQL), Oracle database, Microsoft SQL Server, etc.
  • the types of data stored in the source data storage area may include but are not limited to block storage, file storage, object storage, etc.
  • the source data storage area may be used as a volume or a logical unit number (Logical Unit Number, lun), and the data stored in the source data storage area may be divided into data blocks of the same size (for example, a data block may be 1MB in size).
  • the data when data stored in a source data storage area is migrated, the data may be migrated in units of data blocks.
  • the source data storage area when the data stored in the source data storage area is stored in the form of object storage, the source data storage area can be used as a bucket, and the data stored in the source data storage area is a single object. In other words, data migration can be performed by unit amount of data.
  • the target data storage area may refer to a database storing migrated data.
  • the types of data stored in the target data storage area may include but are not limited to block storage, file storage, and object storage, etc. The detailed description of the storage type can refer to the above content and will not be repeated here.
  • the source data storage area and the target data storage area may be located on the same hardware device or on different hardware devices.
  • the rectangle a shown in the source data storage area and the rectangle b shown in the source data storage area shown in Figure 1 can represent a data block or a single object.
  • multiple data blocks or single objects need to be migrated.
  • the data block or single object of rectangle a can be migrated to rectangle b.
  • the data block or single object represented by rectangle b is the data of the data block or single object of rectangle a after migration, and the data is consistent in content.
  • the data migration tool can capture requests, distribute data, store and update migration metadata record tables, and perform background migration tasks, etc.
  • the capture of the request can be used to monitor a read request or a write request from a user for the first data, so as to start the foreground migration task.
  • the capture of the request can be completed at the Device Mapper layer in Linux.
  • the capture of the request can also be completed at the client. This is not limited in the embodiments of the present application.
  • the data migration tool after completing the capture of the request, obtains the logical address of the first data targeted by the request from the source data storage area, and reads the first data from the storage medium according to the logical address.
  • the source data storage area and the target data storage area are logical storage locations of the first data
  • the storage medium is the physical storage location of the first data
  • the storage medium may refer to a carrier for storing the first data, such as a floppy disk, an optical disk, a high-density digital video disc (Digital Video Disc, DVD), a hard disk, a flash memory, a USB flash drive, a compact flash (Compact Flash, CF) card, a secure digital memory card (Secure Digital Memory Card, SD card), a multimedia card (Multi-Media Card, MMC card), a smart media (Smart Media, SM) card, a memory stick (Memory Stick), an extreme digital-picture card (Extreme Digital-Picture Card, xD card), etc.
  • a carrier for storing the first data such as a floppy disk, an optical disk, a high-density digital video disc (Digital Video Disc, DVD), a hard disk, a flash memory, a USB flash drive, a compact flash (Compact Flash, CF) card, a secure digital memory card (Secure Digital Memory Card, SD card), a multimedia card
  • the data distribution can be used to perform migration processing on the first data after the data migration tool reads the first data.
  • the data migration tool can determine whether the first data needs to be migrated, that is, whether the first data belongs to the data set to be migrated. If the first data belongs to the data set to be migrated, the data migration tool can migrate the first data to the target storage area.
  • the migration metadata record table is used to record the migration status of the data (e.g., not migrated, migrated, and initial state).
  • the data migration tool can store and update the migration metadata record table. Specifically, reference can be made to the migration metadata record tables shown in Table 1 and Table 2.
  • Table 1 exemplarily shows a block storage migration metadata record table
  • Table 2 exemplarily shows an object storage migration metadata record table.
  • the migration metadata record table may include an index and a migration status.
  • the migration metadata record table may also include any one or more of the following items: the identifier of the server where the data is stored when it has not been migrated, the identifier of the server where the data is stored after it has been migrated, the logical location where the data is stored when it has not been migrated, and the logical location where the data is stored after it has been migrated, etc. In the embodiments of the present application, this is not limited.
  • the indexes shown in Table 1 and Table 2 may be used to represent a structure for sorting data in the source data storage area.
  • the indexes may be used to quickly view specific data in the source data storage area.
  • the index “Block 1" represents the index of data blocks of the same size (for example, 1MB) in data stored in the form of block storage
  • the index "Obeject 1" represents the index of a single object in data stored in the form of object storage.
  • the index of data is related to the storage type of the data.
  • the index of the data is volume ID + logical address
  • the index of the data is file system ID (File system ID) + directory (Directory) + file name (File name) + offset address (Offset)
  • the index of the data is key (Key) + version number (Version).
  • the migration status shown in Table 1 and Table 2 indicates whether the data in the source data storage area is in a migration state.
  • the status "Y" indicates that the data indicated by the specific index in the row is currently in a state where migration is required and has been completed.
  • the status "N" indicates that the data indicated by the specific index in the row is currently in a state where the data needs to be migrated but has not been migrated, or needs to be migrated and is being migrated but has not been completely migrated.
  • the status "NA" indicates that the data indicated by the specific index in the row is currently in a state where migration is not required and is not in an unmigrated state.
  • the data migration tool can also change the state of the data in real time. For example, when data 1 needs to be migrated and after the migration is completed, the state of data 1 can be changed from “N” to "Y". Generally, data 1 will not be changed from "Y" to "N”.
  • the same data indicated by the indexes in different migration metadata record tables may be in different states in different tables.
  • the state of data 1 in table A is "Y”
  • the state in table B is "N"
  • data 1 has been migrated from the logical location in table A where it was stored before migration to the logical location where it is stored after migration, but has not yet been migrated from the logical location in table B where it was stored before migration to the logical location where it is stored after migration.
  • Logical location of storage is
  • the server 100 stores a migration metadata record table that records the status of the data of the migration unit
  • the migration metadata record table is updateable and persistent. Therefore, in the embodiment of the present application, breakpoint resumption and retry migration can be supported. For example, when a network problem occurs and the data cannot be copied, the data copying is interrupted, and the original migration task is not affected. When the network is restored, the data copying continues from the interruption point.
  • the migration tool continues to restore the original migration process and migrates the unmigrated data and dirty blocks to the target.
  • the data blocks or single objects processed at the moment the network problem occurs are reprocessed to achieve accurate migration tasks.
  • multiple tasks can be migrated to different targets.
  • Two different migration tasks for the same source data storage area are independent of each other and will not affect each other.
  • For simultaneous migration of different source data storage areas there will be multiple migration tasks in the background, and there will also be migration metadata tables for different migration tasks.
  • Multiple migration tasks are also independent of each other and will not affect each other.
  • the background migration task is different from the foreground migration task. It refers to scanning the data to be migrated in real time and in sequence, determining the unmigrated unit data in the data to be migrated, and migrating the unmigrated unit data (such as data blocks or single objects) in the data to be migrated.
  • the data to be migrated is 100 units of data, namely data 1 to data 100. If there is no read request or write request for any of the data from data 1 to data 100, the background migration task starts scanning from data 1 to data 100, and determines that data 1 is unmigrated data when scanning data 1, and migrates data 1. The remaining 99 data are migrated in the same way.
  • the foreground migration task will migrate data 10 first, and then the background migration task will scan from data 1 to data 100, and then migrate data 1-data 9, data 11-data 100 in order, and when scanning data 1, it will be determined that data 1 is not migrated data, and data 1 will be migrated. The remaining 98 data will be migrated in the same way, but when scanning data 10, it will be determined that data 10 is migrated data, and data 10 will not be migrated.
  • the server 100 provided in the embodiment of the present application is as follows.
  • FIG. 2 is a schematic diagram of the structure of the server 100 provided in an embodiment of the present application.
  • the server 100 may include: one or more processors 101, a memory 102, a communication interface 103, a transmitter 105, a receiver 106, a coupler 107, and an antenna 108.
  • the above components may be connected via a bus 104 or other means, and FIG2 takes the connection via a bus as an example. Among them:
  • the processor 101 can be used to read and execute computer-readable instructions. Specifically, the processor 101 can be used to call a program stored in the memory 102, such as an implementation program of the data migration method provided in the embodiment of the present application on the server 100, and execute the instructions contained in the program. The processor 101 can also be used to determine whether migration is required based on the migration metadata record table.
  • the memory 102 is coupled to the processor 101 and is used to store various software programs and/or multiple sets of instructions.
  • the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.
  • the memory 102 may store information such as the data set to be migrated and the migration metadata record table.
  • the memory 102 may store an operating system (hereinafter referred to as system), such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc.
  • system such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc.
  • the memory 102 may also store a network communication program, which may be used to communicate with clients and other servers.
  • the communication interface 103 can be used for communication between the server 100 and other devices.
  • the communication interface 103 may be a 3G communication interface, a long term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN Communication interface, WAN communication interface, etc.
  • LTE long term evolution
  • the server 100 may also be configured with a wired communication interface 103 to support wired communication.
  • the transmitter 105 and the receiver 106 can be regarded as a wireless modem.
  • the transmitter 105 can be used to transmit the signal output by the processor 101.
  • the receiver 106 can be used to receive the signal.
  • the number of the transmitter 105 and the number of the receiver 106 can be one or more.
  • the antenna 108 may be used to convert electromagnetic energy in the transmission line into electromagnetic waves in free space, or to convert electromagnetic waves in free space into electromagnetic energy in the transmission line.
  • the coupler 107 may be used to divide the mobile communication signal into multiple paths and distribute the signals to multiple receivers 106 .
  • the antenna 108 of the network device may be implemented as a large-scale antenna array.
  • server 100 shown in FIG. 2 is only one implementation of the embodiment of the present application. In actual applications, the server 100 may also include more or fewer components, which is not limited here.
  • the server stored in the source data storage area and the target storage area can be the same or two different servers.
  • the data migration tool may be run on a server stored in a source data storage area, or on a server stored in a target storage area, or on other servers other than the above.
  • the data migration tool may be run simultaneously on the server stored in the source data storage area and the server stored in the target storage area. In the embodiments of the present application, this is not limited.
  • the processor 101 can be used to respond to the user's read requests and write requests, and intercept and process the requests.
  • the processor 101 may also determine whether the data needs to be migrated according to the migration metadata record table.
  • the processor 101 may also update the state information of the data in the migration metadata record table after the data migration is completed.
  • the memory 102 may be used to store information such as data to be migrated, data that has been migrated, and a migration metadata record table.
  • the communication interface 103 may be used to establish a communication connection with an electronic device used by a user.
  • the transmitter 105 can be used to return the data required by the user to the electronic device used by the user, the receiver 106 can be used to receive the user's read request or write request, and the receiver 106 can also be used to receive migration instructions for the first data. For example, the receiver 106 can also be used to receive migration instructions for the first data sent by the control center.
  • server stored in the source data storage area and the server stored in the target storage area in the data migration system in Figure 1 are not the same server 100 shown in Figure 2, the server stored in the source data storage area is server 110, and the server stored in the target storage area is server 120.
  • the data migration tool can run on server 110 and server 120 at the same time, and server 100 includes server 110 and server 120.
  • the processor 101 of the server 110 can be used to respond to the user's read request and write request and intercept the request.
  • the processor 101 of the server 110 can also determine whether the data needs to be migrated according to the migration metadata record table.
  • the processor 101 of the server 110 can update the migration metadata record table after sending the data to be migrated.
  • the memory 102 of the server 110 may be used to store data to be migrated and information such as a migration metadata record table.
  • the communication interface 103 of the server 110 may be used to establish a communication connection with the server 120 .
  • the transmitter 105 of the server 110 may be used to send the data to be migrated to the server 120 .
  • the receiver 106 of the server 110 may be used to receive a read request or a write request from a user.
  • the receiver 106 of the server 110 may also be used to receive a migration instruction for the first data sent by the control center.
  • the processor 101 of the server 120 may update the migration metadata record table after receiving the data to be migrated sent by the server 110 .
  • the memory 102 of the server 120 may be used to store information such as migrated data and migration metadata record tables.
  • the communication interface 103 of the server 120 may be used to establish a communication connection with the server 110 .
  • the transmitter 105 of the server 120 may be used to send a prompt message to the server 110 indicating that the data to be migrated has been migrated, and the receiver 106 of the server 120 may be used to receive the data to be migrated sent by the server 110 .
  • server 100 may be referred to as a computing device.
  • the electronic device used by the user may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and/or a smart city device.
  • PDA personal digital assistant
  • AR augmented reality
  • VR virtual reality
  • AI artificial intelligence
  • wearable device a vehicle-mounted device
  • smart home device a smart home device and/or a smart city device.
  • a method for data migration provided in an embodiment of the present application is described below. This embodiment takes the case where the server 100 responds to a user's read request for data to be migrated as an example to introduce the method for data migration.
  • Fig. 3 shows a schematic flow chart of a data migration method in response to a read request provided in an embodiment of the present application.
  • the data storage mode is block storage
  • the migration unit is a data block as an example.
  • the server 100 responds to a user's read request for first data.
  • the user can input an operation to the server 100 according to his/her needs, and the server 100 can respond with a read request for the first data.
  • the read request for the first data may include, but is not limited to: the logical address of the first data in the source data storage area, the index, and the data payload length, etc., wherein the data payload length refers to the effective payload data length excluding the header and the length in the read request information.
  • the server 100 may receive a migration instruction for the first data sent by the control center before responding to the read request.
  • the system may issue a migration instruction due to the remaining storage space of the server 100, the server 100 being upgraded, or the server 100 being damaged, or may issue a migration instruction based on an input operation by a manager. This is not limited in the embodiments of the present application.
  • the status of the first data in the migration metadata record table will not be "NA” but "Y” or "N".
  • the state of the first data may refer to:
  • the state of the first data is the state of the first data divided into corresponding multiple data blocks; if the size of the first data is less than the data block, the state of the first data is the state of the data block corresponding to the first data.
  • the server 100 may determine whether the first data has been migrated based on the migration metadata record table.
  • the status corresponding to the index carried in the read request for the first data is "Y", it can be determined that the first data has completed migration; if in the migration metadata record table, the status corresponding to the index carried in the read request for the first data is "N”, it can be determined that the first data has not been migrated or is in the process of migrating and has not yet completed migration.
  • the server 100 determines that the first data has been migrated, the status of the first data in the migration metadata record table is "Y", and the first data does not need to be migrated from the source data storage area to the target storage area again, and S304 is executed.
  • the server 100 determines that the first data has not been migrated, but it can be known from S303 that there is a migration instruction for the first data, the state of the first data in the migration metadata record table is "N", and S305 is executed.
  • the migration metadata record table may record that the state of the first data may be "N+".
  • S304 Read the first data from the source data storage area and return it to the user.
  • the server 100 may read the first data from the source data storage area based on the read request and return the first data to the user.
  • the cloud can obtain the logical address of the first data based on the read request of the first data sent by the local user, and then obtain the physical address of the first data based on the mapping relationship between the logical address and the physical address, and read the first data from the physical address, and then the cloud sends the read first data to the local user.
  • S305 Read the first data from the source data storage area, migrate the first data to the target storage area, update the migration metadata record table, and then return the first data to the user.
  • the server 100 may migrate the first data from the source data storage area to the target storage area. For example, when the source data storage area and the target storage area are stored on the same server 100, the logical address of the first data in the source data storage area is L1, and the physical address corresponding to the logical address L1 is W1, and the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the first data from the source data storage area to the target storage area, which may mean changing the logical address of the first data from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to map the logical address L2 to the physical address W1.
  • the two physical addresses stored in the server 100 before and after the first data migration may be the same or different.
  • the server 100 may update the migration metadata record table.
  • the state of the migration metadata record table may be converted from "N" to "Y".
  • the server 100 needs to split the data block according to the size of the data actually read, so that the part of the data after the splitting process is exactly the first data.
  • the size of the first data that the user wants to read is 0.5MB
  • the size of the data block is 1MB.
  • the size of the data block read out by the server 100 from the source data storage area is 1MB, wherein the 1MB data block includes 0.5MB of first data and 0.5MB of other data.
  • the server 100 needs to divide the 1MB data block, select 0.5MB of first data therefrom, and return the 0.5MB of first data to the user.
  • Fig. 4 shows a flow chart of another data migration method in response to a read request provided in an embodiment of the present application.
  • the data storage mode is object storage
  • the migration unit is a single object as an example.
  • the server 100 responds to a user's read request for first data.
  • the user may perform input operations to the server 100 according to his/her own needs, and the server 100 may respond with a read request for the first data.
  • the read request for the first data may include but is not limited to: the logical address of the first data in the source data storage area, index and data payload length, etc., wherein the data payload length refers to the effective payload data length excluding the header and length in the read request information.
  • the server 100 may receive a migration request for the first data from the control center before responding to the read request. instruction.
  • the system may issue a migration instruction due to the remaining storage space of the server 100, the server 100 being upgraded, or the server 100 being damaged, or may issue a migration instruction based on an input operation by a manager. This is not limited in the embodiments of the present application.
  • the status of the first data in the migration metadata record table will not be "NA” but "Y” or "N".
  • the state of the first data may refer to:
  • the state of the first data is the state of the first data divided into corresponding multiple single objects; if the size of the first data is less than a single object, the state of the first data is the state of the single object corresponding to the first data.
  • the server 100 may determine whether the first data has been migrated based on the migration metadata record table.
  • the state corresponding to the index carried in the read request for the first data is "Y"
  • the state corresponding to the index carried in the read request for the first data is "N"
  • the server 100 determines that the first data has been migrated, the status of the first data in the migration metadata record table is "Y", and the first data does not need to be migrated from the source data storage area to the target storage area again, and S404 is executed.
  • the server 100 determines that the first data has not been migrated, but it can be known from S403 that there is a migration instruction for the first data, the state of the first data in the migration metadata record table is "N", and S405 is executed.
  • the migration metadata record table may record that the state of the first data may be "N+".
  • S404 Read the first data from the source data storage area and return it to the user.
  • the server 100 may read the first data from the source data storage area based on the read request and return the first data to the user.
  • the cloud can obtain the logical address of the first data based on the read request of the first data sent by the local user, and then obtain the physical address of the first data based on the mapping relationship between the logical address and the physical address, and read the first data from the physical address, and then the cloud sends the read first data to the local user.
  • S405 Read the first data from the source data storage area, migrate the first data to the target storage area, update the migration metadata record table, and then return the first data to the user.
  • the server 100 may migrate the first data from the source data storage area to the target storage area. For example, when the source data storage area and the target storage area are stored on the same server 100, the logical address of the first data in the source data storage area is L1, and the physical address corresponding to the logical address L1 is W1, and the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the first data from the source data storage area to the target storage area, which may mean changing the logical address of the first data from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to the logical address L2 mapping relationship.
  • the two physical addresses stored in the server 100 before and after the first data migration may be the same or different.
  • the server 100 may update the migration metadata record table.
  • the state of the migration metadata record table may be converted from "N" to "Y".
  • the server 100 needs to split the single object according to the size of the data actually read, so that the portion of data after the splitting process is exactly the first data.
  • the size of the first data that the user wants to read is 0.5MB
  • the size of the single object is 1MB.
  • the size of the single object read by the server 100 from the source data storage area is 1MB, where the 1MB single object includes 0.5MB of first data and 0.5MB of other data.
  • the server 100 needs to split the 1MB single object, select 0.5MB of first data from it, and return the 0.5MB of first data to the user.
  • online migration can be realized, and it can be ensured that when the system receives a user's read request for the first data, the system only needs to read the first data once in the source data storage area to ensure that the user's read request and migration task are realized. In this way, the migration efficiency can be improved.
  • This embodiment takes the case where the server 100 responds to a user's write request for data to be migrated as an example to introduce the data migration method.
  • Fig. 5 shows a schematic flow chart of a data migration method in response to a write request provided in an embodiment of the present application.
  • the data storage mode is block storage
  • the migration unit is a data block as an example.
  • the server 100 responds to a user's write request for first data.
  • the user may perform input operations to the server 100 according to his/her own needs, and the server 100 may respond with a write request for the first data.
  • the write request for the first data may include but is not limited to: a logical address of the first data in the source data storage area, an index, a data payload length, and a modification content, etc.
  • the server 100 needs to satisfy write-write mutual exclusion in responding to the user's write request for the first data, that is, the data targeted by the two write requests cannot have an intersection.
  • S502 Determine whether the first data needs to be migrated. If not, write the first data to the source data storage area, and store the written first data in the source data storage area.
  • the server 100 may receive a migration instruction for the first data issued by the system before responding to the write request.
  • the system may issue a migration instruction due to the remaining storage space of the server 100, the server 100 being upgraded, or the server 100 being damaged, or may issue a migration instruction based on an input operation by a manager. This is not limited in the embodiments of the present application.
  • the status of the first data in the migration metadata record table will not be "NA” but "Y” or "N".
  • the server 100 determines that the first data does not need to be migrated, that is, there is no migration instruction for the first data, the status of the first data in the migration metadata record table is "NA", and the server 100 can directly process the first data according to the write request to obtain the first data after writing, and store the first data after writing in the source data storage area, that is, the logical address of the first data and the first data after writing is the same.
  • the server 100 may notify the user of the completion of the write request. For example, if the user deletes document M, the server 100 may The server 100 notifies the user that the write request for the document M has been completed by displaying a prompt message (eg, a pop-up window) such as a text message "Document M has been deleted".
  • a prompt message eg, a pop-up window
  • the server 100 is not limited to the way in which the write request has been completed.
  • the state of the first data may refer to:
  • the state of the first data is a state in which the first data is divided into a corresponding plurality of data blocks.
  • the state of the first data is the state of the data block corresponding to the first data.
  • S503 Determine whether the first data has been migrated. If so, double-write the first data in the source data storage area and the target storage area.
  • the server 100 may determine whether the first data has been migrated based on the migration metadata record table.
  • the status corresponding to the index carried in the write request for the first data is "Y"
  • the status corresponding to the index carried in the write request for the first data is "N”
  • the server 100 determines that the first data has completed migration, the status of the first data in the migration metadata record table is "Y", and the first data does not need to be migrated from the source data storage area to the target storage area again.
  • the server 100 can process the first data in the source data storage area and the first data in the target storage area respectively according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 can notify the user of the completion of the write request, which is not described in detail here.
  • the server 100 determines that the first data has not been migrated, but it can be known from S502 that there is a migration instruction for the first data, the state of the first data in the migration metadata record table is "N", and S504 is executed.
  • the migration metadata record table may record that the state of the first data may be "N+".
  • the server 100 determines that the first data is being migrated but has not been completely migrated, the first data does not need to be migrated from the source data storage area to the target storage area again.
  • the server 100 can process the first data in the source data storage area and the first data in the target storage area according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 can notify the user of the completion of the write request, which will not be described in detail here.
  • S504 Determine whether the data block to be migrated is aligned with the first data.
  • the migration unit when the data is stored in block storage, the migration unit is a data block, and the server 100 can determine whether the data block is aligned with the first data.
  • the data block is not aligned with the first data.
  • the size of the first data is 0.5MB and the size of the data block is 1MB
  • the data block is not aligned with the first data.
  • the migration unit of the first data is 1.5, and the data block is not aligned with the first data.
  • the size of the first data is not an integer multiple of the data block, the data block is not aligned with the first data.
  • the data block is aligned with the first data.
  • the size of the first data is 1MB and the size of the data block is 1MB
  • the data block is aligned with the first data.
  • the migration unit of the first data is 2, and the data block is aligned with the first data.
  • the size of the first data is an integer multiple of the data block, the data block is aligned with the first data.
  • the size of the data block is consistent with the size of the first data or the size of the first data is the same as the size of the first data.
  • the size is an integer multiple of the size of the data block, and S505 is executed.
  • the first type is that the size of the first data is smaller than the data block size, in which case S506 is executed;
  • the second type is that the size of the first data is larger than the data block size and is not an integer multiple of the data block size.
  • the first data can be divided into two parts, the first part can be data whose size is the maximum integer multiple of the data block size, and the second part can be data from the first data without the first part.
  • the first part of the data is executed in S505, and the second part of the data is executed in S506.
  • S505 Migrate the first data to the target storage area, double-write the first data in the source data storage area and the target storage area, and update the migration metadata record table.
  • the server 100 may migrate the first data from a source data storage area to a target storage area.
  • the logical address of the first data in the source data storage area is L1
  • the physical address corresponding to the logical address L1 is W1
  • the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the first data from the source data storage area to the target storage area, which may mean changing the logical address of the first data from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to map the logical address L2 to the physical address W1.
  • server 110 stored in the source data storage area when the server stored in the source data storage area and the server stored in the target storage area are not the same server, that is, the server stored in the source data storage area is server 110, and the server stored in the target storage area is server 120, server 110 stored in the source data storage area sends the first data stored at logical address L1 and physical address W1 to server 120 stored in the target storage area, and server 120 stores the first data in the target storage area.
  • the logical address of the first data in server 120 stored in the target storage area is L2, and the physical address is W2.
  • the two physical addresses stored in the server before and after the migration of the first data may be the same or different. That is, the storage medium where the first data is physically stored in the source data storage area and the storage medium where the first data is physically stored in the target data storage area may be the same or different. In the embodiment of the present application, this is not limited.
  • the server 100 may double-write the first data in the source data storage area and the first data in the target storage area. Specifically, the server 100 may process the first data in the source data storage area and the first data in the target storage area respectively according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 may update the migration metadata record table.
  • the state of the first data in the migration metadata record table may be converted from "N" to "Y".
  • the server 100 may notify the user of information on completion of the write request, which may be referred to the above content and will not be elaborated here.
  • S506 Write the first data in the source data storage area, migrate the written first data to the target storage area, and update the migration metadata record table.
  • the first data in S506 is not aligned with the size of the data block, so that the data targeted by other write requests may overlap with the first data targeted by this write request, that is, the same area may be written. Therefore, before writing the first data in the source data storage area, it is necessary to obtain a mutex lock in the data block. If the acquisition of the mutex lock fails, it is repeated until the acquisition succeeds.
  • the acquisition of the mutex lock successfully may mean stopping the write operation on the first data in other threads and only executing the write operation in S506. In this way, the write-write conflict phenomenon can be avoided.
  • the server 100 may write the first data in the source data storage area. Specifically, the server 100 may write the first data stored in the source data storage area according to the write request, and store the first data after writing in the source data storage area. In some implementations, the first data after writing may be referred to as the second data.
  • the server 100 may migrate the first data from the source data storage area to the target storage area.
  • the logical address of the first data in the source data storage area is L1
  • the physical address corresponding to the logical address L1 is W1
  • the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the first data from the source data storage area to the target storage area, which may mean changing the logical address of the first data from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to map the logical address L2 to the physical address W1.
  • server 110 stored in the source data storage area when the server stored in the source data storage area and the server stored in the target storage area are not the same but on different servers 100, that is, the server stored in the source data storage area is server 110 and the server stored in the target storage area is server 120, server 110 stored in the source data storage area sends the first data stored in the logical address L1 to server 120 stored in the target storage area, and server 120 stores the first data in the target source target storage area.
  • the logical address of the first data in server 120 stored in the target source target storage area is L2.
  • the two physical addresses stored in the server 100 before and after the first data is migrated may be the same or different. That is, the storage medium where the first data is physically stored in the source data storage area and the storage medium where the first data is physically stored in the target data storage area may be the same or different. In the embodiment of the present application, this is not limited.
  • the server 100 may update the migration metadata record table.
  • the state of the first data in the migration metadata record table may be converted from "N" to "Y".
  • the server 100 may notify the user of the completion of the write request, which will not be elaborated here.
  • online migration can be realized, and because the migration of data is triggered by a write request, it can be ensured that when the system receives a user's write request for first data, the system only needs to read the first data once in the source data storage area to ensure that the user's write request and migration task are realized. In this way, the migration efficiency can be improved.
  • Fig. 6 shows a flow chart of another data migration method in response to a write request provided in an embodiment of the present application.
  • the data storage mode is object storage
  • the migration unit is a single object as an example.
  • S601 The server 100 responds to a user's write request for first data.
  • the user can input an operation to the server 100 according to his/her needs, and the server 100 can respond with a write request for the first data.
  • the write request for the first data may include but is not limited to: the logical address of the first data in the source data storage area, index, data payload length, and modified content, etc.
  • the server 100 needs to satisfy write-write mutual exclusion in responding to the user's write request for the first data, that is, the data targeted by the two write requests cannot have an intersection.
  • S602 Determine whether the first data needs to be migrated. If not, write the first data in the source data storage area and store the written first data in the source data storage area.
  • the server 100 may receive a migration instruction for the first data issued by the system before responding to the write request.
  • the system may issue a migration instruction due to the remaining storage space of the server 100, the server 100 being upgraded, or the server 100 being damaged, or may issue a migration instruction based on an input operation by a manager. This is not limited in the embodiments of the present application.
  • the status of the first data in the migration metadata record table will not be "NA” but "Y” or "N".
  • the server 100 determines that the first data does not need to be migrated, that is, there is no migration instruction for the first data, the status of the first data in the migration metadata record table is "NA", and the server 100 can directly write the first data according to the write request to obtain the first data after writing, and store the first data after writing in the source data storage area, that is, the logical address of the first data before the write process and the first data after the write process are the same.
  • the server 100 may notify the user of the completion of the write request. For example, if the user deletes document M, the server 100 may The server 100 notifies the user that the write request for the document M has been completed by displaying a prompt message (eg, a pop-up window) such as a text message "Document M has been deleted".
  • a prompt message eg, a pop-up window
  • the server 100 is not limited to the way in which the write request has been completed.
  • the state of the first data may mean: if the size of the first data is greater than or equal to a single object, then the state of the first data is the state of the first data divided into corresponding multiple single objects; if the size of the first data is smaller than a single object, then the state of the first data is the state of the single object corresponding to the first data.
  • S603 Determine whether the first data has been migrated, and if so, double-write the first data in the source data storage area and the target storage area.
  • the server 100 may determine whether the first data has been migrated based on the migration metadata record table.
  • the status corresponding to the index carried in the write request for the first data is "Y"
  • the status corresponding to the index carried in the write request for the first data is "N”
  • the server 100 determines that the first data has completed migration, the status of the first data in the migration metadata record table is "Y", and the first data does not need to be migrated from the source data storage area to the target storage area again.
  • the server 100 can process the first data in the source data storage area and the first data in the target storage area respectively according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 can notify the user of the completion of the write request, which is not described in detail here.
  • the server 100 determines that the first data has not been migrated, but it can be known from S602 that there is a migration instruction for the first data, the state of the first data in the migration metadata record table is "N", and S604 is executed.
  • the migration metadata record table may record that the state of the first data may be "N+".
  • the server 100 determines that the first data is being migrated but has not been completely migrated, the first data does not need to be migrated from the source data storage area to the target storage area again.
  • the server 100 can write the first data in the source data storage area and the first data in the target storage area according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 can notify the user of the completion of the write request, which will not be described in detail here.
  • S604 Migrate the first data to the target storage area, double-write the first data in the source data storage area and the target storage area, and update the migration metadata record table.
  • the first data since the first data is stored in an object storage manner and writing to an object generally involves writing to the entire object, there is no need to determine whether the first data is aligned with a single object as in S504. Instead, when the first data needs to be migrated and has not been migrated, the write operation is directly responded to and the migration task of the first data is completed.
  • the server 100 may migrate the first data from the source data storage area to the target storage area. For example, when the source data storage area and the target storage area are stored on the same server 100, the logical address of the first data in the source data storage area is L1, and the physical address corresponding to the logical address L1 is W1, and the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the first data from the source data storage area to the target storage area, which may mean changing the logical address of the first data from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to map the logical address L2 to the physical address W1.
  • server 110 stored in the source data storage area when the server stored in the source data storage area is not the same as the server stored in the target storage area, that is, the source
  • server 110 stored in the source data storage area sends the first data stored at logical address L1 and physical address W1 to server 120 stored in the target storage area
  • server 120 stores the first data in the target storage area.
  • the logical address of the first data in server 120 stored in the target storage area is L2, and the physical address is W2.
  • the two physical addresses stored in the server before and after the migration of the first data may be the same or different. That is, the storage medium where the first data is physically stored in the source data storage area and the storage medium where the first data is physically stored in the target data storage area may be the same or different. In the embodiment of the present application, this is not limited.
  • the server 100 may double-write the first data in the source data storage area and the first data in the target storage area.
  • the server 100 can process the first data in the source data storage area and the first data in the target storage area respectively according to the write request, and store the first data in the source data storage area after writing in the source data storage area, and store the first data in the target storage area after writing in the target storage area.
  • the server 100 may update the migration metadata record table.
  • the state of the first data in the migration metadata record table may be converted from "N" to "Y".
  • the server 100 may notify the user of the completion of the write request, which will not be elaborated here.
  • This embodiment takes the case where there is no user read request or write request for the data to be migrated as an example to introduce the data migration method.
  • FIG7 shows a flow chart of a background data migration method provided in an embodiment of the present application.
  • the flow chart takes block storage as an example. It is worth noting that if the source data storage area targeted by the background data migration method has a migration instruction, the following specific steps will no longer determine whether there is a migration instruction.
  • S701 sequentially traverse the source data storage area to be migrated, and determine whether the currently scanned data block has been migrated.
  • the server 100 may sequentially traverse the source data storage area to be migrated. For example, if the source data storage area to be migrated is data 1 to data 100, which has 100 data blocks in total, the server 100 scans from data 1 to data 100, and determines whether each data block has been migrated.
  • the server 100 may determine whether the currently scanned data block has completed migration.
  • the status corresponding to the currently scanned data block is "Y" in the migration metadata record table, it can be determined that the currently scanned data block has completed migration;
  • the state corresponding to the currently scanned data block is "N" in the migration metadata record table, it can be determined that the currently scanned data block has not been migrated or is being migrated and the migration has not yet been completed.
  • the server 100 determines that the currently scanned data block has completed migration, the status of the currently scanned data block in the migration metadata record table is "Y", and the currently scanned data block does not need to be migrated from the source data storage area to the target storage area again, so the next data block of the current data block in the source data storage area is scanned.
  • the source data storage area to be migrated is data 1-data 100, which has 100 data blocks in total.
  • the server 100 determines that data 1 has been migrated, and then the server 100 scans data 2 to determine whether data 2 has been migrated.
  • the server 100 determines that the currently scanned data block has not been migrated, the status of the currently scanned data block in the migration metadata record table is "N", and S502 is executed.
  • S702 Determine whether the currently scanned data block is being processed.
  • the server 100 may determine whether the currently scanned data block is being processed, that is, determine whether the currently scanned data block is being migrated.
  • the migration metadata record table may record that the status of the currently scanned data block may be "N+".
  • S701 is executed to scan the next data block of the current data block in the source data storage area.
  • the source data storage area to be migrated is data 1-data 100, which has 100 data blocks in total.
  • the server 100 determines that data 1 is being migrated, and then the server 100 scans data 2 to determine whether data 2 has completed migration.
  • S703 is executed.
  • S703 Read the currently scanned data block from the source data storage area, migrate the data block to the target storage area, and update the migration metadata record table.
  • the server 100 may migrate the currently scanned data block from the source data storage area to the target storage area.
  • the logical address of the currently scanned data block in the source data storage area is L1
  • the physical address corresponding to the logical address L1 is W1
  • the mapping relationship is that the logical address L1 maps the physical address W1.
  • the server 100 migrates the currently scanned data block from the source data storage area to the target storage area, which may mean changing the logical address of the currently scanned data block from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to map the logical address L2 to the physical address W1.
  • the two physical addresses of the currently scanned data block stored in the server 100 before and after the migration may be the same or different.
  • the server 100 may update the migration metadata record table.
  • the state of the migration metadata record table may be converted from "N" to "Y".
  • FIG8 shows a flow chart of a background data migration method provided in an embodiment of the present application.
  • the flow chart takes object storage as an example. It is worth noting that if the source data storage area targeted by the background data migration method has a migration instruction, the following specific steps will no longer determine whether there is a migration instruction.
  • the server 100 may sequentially traverse the source data storage area to be migrated.
  • the source data storage area to be migrated is data 1 to data 100, which contains 100 objects in total.
  • the server 100 scans from data 1 to data 100, and determines whether each object has been migrated.
  • the server 100 may determine whether the migration of a single object currently being scanned has been completed.
  • the status corresponding to the currently scanned single object is "Y" in the migration metadata record table, it can be determined that the currently scanned single object has completed migration;
  • the status corresponding to the currently scanned single object is "N" in the migration metadata record table, it can be determined that the currently scanned single object has not been migrated or is being migrated and the migration has not yet been completed.
  • the server 100 determines that the migration of the single object currently being scanned has been completed, the status of the single object currently being scanned in the migration metadata record table is "Y", and the single object currently being scanned does not need to be migrated from the source data storage area to the target storage area again, so the next data block of the current single object in the source data storage area is scanned.
  • the source data storage area to be migrated is data 1-data 100, which contains 100 objects in total.
  • the server 100 determines that data 1 has been migrated, and then the server 100 scans data 2 to determine whether data 2 has been migrated.
  • the server 100 determines that the currently scanned single object has not been migrated, the status of the currently scanned single object in the migration metadata record table is "N", and S802 is executed.
  • S802 Determine whether the single object currently being scanned is being processed.
  • the server 100 may determine whether the currently scanned single object is being processed. Whether the object is being migrated.
  • the migration metadata record table may record that the status of the currently scanned single object may be "N+".
  • S801 is executed to scan the next object of the current single object in the source data storage area.
  • the source data storage area to be migrated is data 1-data 100, which contains 100 objects in total.
  • the server 100 determines that data 1 is being migrated, and then the server 100 scans data 2 to determine whether data 2 has completed migration.
  • the server 100 may migrate a single object currently being scanned from a source data storage area to a target storage area.
  • the logical address of the currently scanned single object in the source data storage area is L1
  • the physical address corresponding to the logical address L1 is W1
  • the mapping relationship is that the logical address L1 maps to the physical address W1.
  • the server 100 migrates the currently scanned single object from the source data storage area to the target storage area, which may refer to changing the logical address of the currently scanned single object from the logical address L1 in the source data storage area to the logical address L2 in the target storage area, and changing the mapping relationship to that the logical address L2 maps to the physical address W1.
  • the two physical addresses of the single object currently being scanned stored in the server 100 before and after the migration may be the same or different.
  • the server 100 may update the migration metadata record table.
  • the state of the migration metadata record table may be converted from "N" to "Y".
  • the corresponding data migration is completed during the data reading and writing process, and the remaining data is migrated through the background migration task.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated.
  • the available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.
  • SSD solid state drive
  • the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium.
  • the program When the program is executed, it can include the processes of the above-mentioned method embodiments.
  • the aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, etc. The medium of the code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un procédé et un système de migration de données, et un appareil associé. Le procédé comprend les étapes suivantes : lorsqu'un serveur (100) reçoit une demande de lecture ou une demande d'écriture pour des premières données, le serveur (100) déclenche une opération de migration pour les premières données, puis répond au traitement de la demande de lecture ou de la demande d'écriture pour les premières données après que la migration est accomplie. La présente solution permet de réaliser une migration en ligne de données, l'efficacité de migration de données peut également être améliorée, et la convivialité d'utilisation est bonne.
PCT/CN2023/117343 2022-10-20 2023-09-06 Procédé et système de migration de données et appareil associé WO2024082857A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211292048.5 2022-10-20
CN202211292048.5A CN115729463A (zh) 2022-10-20 2022-10-20 数据迁移的方法、系统及相关装置

Publications (1)

Publication Number Publication Date
WO2024082857A1 true WO2024082857A1 (fr) 2024-04-25

Family

ID=85293958

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/117343 WO2024082857A1 (fr) 2022-10-20 2023-09-06 Procédé et système de migration de données et appareil associé

Country Status (2)

Country Link
CN (1) CN115729463A (fr)
WO (1) WO2024082857A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115729463A (zh) * 2022-10-20 2023-03-03 超聚变数字技术有限公司 数据迁移的方法、系统及相关装置

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102073462A (zh) * 2010-11-29 2011-05-25 华为技术有限公司 虚拟存储迁移方法、系统和虚拟机监控器
US20130041977A1 (en) * 2011-08-11 2013-02-14 Fujitsu Limited Computer product, migration executing apparatus, and migration method
CN104899218A (zh) * 2014-03-06 2015-09-09 腾讯科技（深圳）有限公司 数据读写方法及数据读写装置
CN107402722A (zh) * 2017-07-04 2017-11-28 杭州宏杉科技股份有限公司 一种数据迁移方法及存储设备
CN111427514A (zh) * 2020-03-27 2020-07-17 杭州宏杉科技股份有限公司 视图迁移方法和装置
CN112578997A (zh) * 2019-09-30 2021-03-30 华为技术有限公司 一种数据迁移方法、系统及相关设备
US20210286541A1 (en) * 2020-03-13 2021-09-16 EMC IP Holding Company LLC Techniques for data migration
CN113867645A (zh) * 2021-09-30 2021-12-31 苏州浪潮智能科技有限公司 数据迁移和数据读写方法、装置、计算机设备及存储介质
CN115729463A (zh) * 2022-10-20 2023-03-03 超聚变数字技术有限公司 数据迁移的方法、系统及相关装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102073462A (zh) * 2010-11-29 2011-05-25 华为技术有限公司 虚拟存储迁移方法、系统和虚拟机监控器
US20130041977A1 (en) * 2011-08-11 2013-02-14 Fujitsu Limited Computer product, migration executing apparatus, and migration method
CN104899218A (zh) * 2014-03-06 2015-09-09 腾讯科技（深圳）有限公司 数据读写方法及数据读写装置
CN107402722A (zh) * 2017-07-04 2017-11-28 杭州宏杉科技股份有限公司 一种数据迁移方法及存储设备
CN112578997A (zh) * 2019-09-30 2021-03-30 华为技术有限公司 一种数据迁移方法、系统及相关设备
US20210286541A1 (en) * 2020-03-13 2021-09-16 EMC IP Holding Company LLC Techniques for data migration
CN111427514A (zh) * 2020-03-27 2020-07-17 杭州宏杉科技股份有限公司 视图迁移方法和装置
CN113867645A (zh) * 2021-09-30 2021-12-31 苏州浪潮智能科技有限公司 数据迁移和数据读写方法、装置、计算机设备及存储介质
CN115729463A (zh) * 2022-10-20 2023-03-03 超聚变数字技术有限公司 数据迁移的方法、系统及相关装置

Also Published As

Publication number Publication date
CN115729463A (zh) 2023-03-03

Similar Documents

Publication Publication Date Title
CN109254733B (zh) 用于存储数据的方法、装置和系统
US7660867B2 (en) Virtual computer system and virtual computer migration control method
US10514947B2 (en) Container management apparatus, container management method, and nonvolatile recording medium
JP5541149B2 (ja) スナップショット採取プログラム、サーバおよびスナップショット採取方法
JP2019519025A (ja) 分散システムにおける範囲の分割および移動
US8386707B2 (en) Virtual disk management program, storage device management program, multinode storage system, and virtual disk managing method
CN112395264B (zh) 分布式存储系统中逻辑目标与卷之间映射的处理方法
US11797494B2 (en) Snapshot management in distributed file systems
WO2024082857A1 (fr) Procédé et système de migration de données et appareil associé
US11580148B2 (en) Document storage and management
CN113806300B (zh) 数据存储方法、系统、装置、设备及存储介质
US20210374107A1 (en) Distributed file system and distributed file managing method
US11157456B2 (en) Replication of data in a distributed file system using an arbiter
US20230101774A1 (en) Techniques for performing clipboard-to-file paste operations
US11010408B2 (en) Hydration of a hierarchy of dehydrated files
TWI571754B (zh) 用來進行檔案同步控制之方法與裝置
CN114490540B (zh) 数据存储方法、介质、装置和计算设备
US20050278501A1 (en) Method and apparatus for caching storage system
US8943019B1 (en) Lookup optimization during online file system migration
CN114780043A (zh) 基于多层缓存的数据处理方法及装置、电子设备
CN109343928B (zh) 虚拟化集群中虚拟机的虚拟内存文件重定向方法及其系统
US10685046B2 (en) Data processing system and data processing method
CN111399753A (zh) 写入图片的方法和装置
US11379147B2 (en) Method, device, and computer program product for managing storage system
WO2024001280A1 (fr) Procédé de perception de flux de données et appareil associé