CN111273872A - Data migration method, device, equipment and medium - Google Patents
Data migration method, device, equipment and medium Download PDFInfo
- Publication number
- CN111273872A CN111273872A CN202010093246.3A CN202010093246A CN111273872A CN 111273872 A CN111273872 A CN 111273872A CN 202010093246 A CN202010093246 A CN 202010093246A CN 111273872 A CN111273872 A CN 111273872A
- Authority
- CN
- China
- Prior art keywords
- storage unit
- data
- source
- target
- volume
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a data migration method, a data migration device, data migration equipment and a data migration medium, and relates to the technical field of data storage access. The specific implementation mode is as follows: dividing the storage space of a source volume into a plurality of source storage units, and dividing the storage space of a target volume into a plurality of target storage units; establishing a position corresponding relation between each source storage unit and each target storage unit; when a data access request is acquired, determining a source storage unit where data influencing the data access request is located in a source volume; copying the data of the source storage unit where the influence data are located to a corresponding target storage unit according to the corresponding position relation; and responding to the data access request according to the copied data in the target storage unit. According to the technical scheme of the embodiment of the application, automatic and convenient data migration is realized under the condition that the data service is not stopped, the target volume replaces the source volume to execute the corresponding service, and the migration of the service is synchronously realized.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a data storage access technology.
Background
At present, a lot of business applications need to support a large amount of user data storage, and business service providers may store user data in a cloud server to support business operation. When a storage system upgrade is faced, data in one storage system needs to be smoothly migrated to a new storage system.
In the prior art, the data migration manner may be based on a file system level data migration or a snapshot type data migration. However, file system level data migration is dependent on the upper level file system and sometimes also on manual operation and control; data migration in the form of snapshots requires stopping the business service and restarting the service after the data migration is completed, which is unacceptable in many business service scenarios.
Disclosure of Invention
The embodiment of the application provides a data migration method, a data migration device, data migration equipment and a data migration medium, so that automatic and convenient data migration is achieved under the condition that data service is not stopped.
In a first aspect, an embodiment of the present application provides a data migration method, including:
dividing the storage space of a source volume into a plurality of source storage units, and dividing the storage space of a target volume into a plurality of target storage units;
establishing a position corresponding relation between each source storage unit and each target storage unit;
when a data access request is acquired, determining a source storage unit where influence data of the data access request are located in the source volume;
according to the position corresponding relation, if the influence data does not exist in a target storage unit corresponding to a source storage unit where the influence data is located, copying the data of the source storage unit where the influence data is located to the corresponding target storage unit;
and responding to the data access request according to the copied data in the target storage unit.
The method comprises the steps of dividing the storage space of a source volume into a plurality of source storage units and dividing the storage space of a target volume into a plurality of target storage units; establishing a position corresponding relation between each source storage unit and each target storage unit; when a data access request is acquired, determining a source storage unit where data influencing the data access request is located in a source volume; according to the position corresponding relation, if the influence data does not exist in the target storage unit corresponding to the source storage unit where the influence data is located, copying the data of the source storage unit where the influence data is located to the corresponding target storage unit; and responding to the data access request according to the copied data in the target storage unit. According to the technical scheme, the storage spaces of the source volume and the target volume are divided, so that data migration is performed according to the divided storage units, and the condition that the service is stopped due to mass migration is avoided. Meanwhile, the position corresponding relation between the source storage unit in the source volume and the target storage unit in the target volume is established, and the influence data of the data access request is automatically migrated from the source storage unit of the source volume to the corresponding target storage unit in the target volume according to the position corresponding relation, so that a specific file system is not needed, manual operation control is not needed, and the convenience of data migration is improved. And responding to the data access request according to the copied data in the target storage unit, thereby replacing the source volume with the target volume to execute corresponding business and synchronously realizing the migration of business services.
Optionally, the sizes of the source storage units are equal, the sizes of the target storage units are equal, and the sizes of the source storage units and the target storage units are equal.
In an optional implementation manner in the foregoing application, the sizes of each source storage unit and each target storage unit are limited, so that the accuracy of the data migration process can be effectively ensured, and the occurrence of situations such as repeated copying or missed copying of data is avoided.
Optionally, the location correspondence includes a start address of the source storage unit and a start address of the target storage unit.
In an optional implementation manner of the foregoing application, the location correspondence is refined to include the start address of the source storage unit and the start address of the target storage unit, so that the storage units can be conveniently searched and matched according to the location correspondence, and accuracy of subsequent data migration is ensured.
Optionally, the location correspondence further includes an offset address of the source storage unit and an offset address of the target storage unit.
In an optional implementation manner of the foregoing application, the offset address of the source storage unit and the offset address of the target storage unit are added to the position corresponding relationship, so that when the two offset address values are not equal, the target storage unit is accurately located, and therefore, accurate execution of subsequent data migration is guaranteed.
Optionally, before acquiring the data access request, the method further includes:
and if the migration instruction of the data from the source volume to the target volume is acquired, switching and mounting the data access volume of the application layer from the source volume to the target volume in the logic storage layer.
In an optional implementation manner of the foregoing application, before the data access request is obtained, when a migration instruction of data from the source volume to the target volume is obtained, the data access volume of the application program is mounted from the source volume to the target volume in the logical storage layer, so that switching of the data access volume is realized, and an effect of service function migration is achieved.
Optionally, after copying the data of the source storage unit where the influence data is located into the corresponding target storage unit according to the position correspondence relationship, the method further includes:
marking the target storage unit which is copied completely;
correspondingly, when a data access request is acquired, after determining a source storage unit in which the data affected by the data access request is located in the source volume, the method further includes:
and according to the position corresponding relation, if the mark of the target storage unit corresponding to the source storage unit of the influence data is copy-completed, the copy operation is abandoned, and the data access request is directly responded according to the data in the target storage unit.
An alternative embodiment of the above application avoids duplicate copying of data by, prior to copying the affected data into the target storage unit of the target source, when the flag identifying the target storage unit is copy complete, aborting the copy operation until responding to the data access request.
Optionally, the method further includes:
and if the marks of all the target storage units are identified to be copied, deleting the position corresponding relation between the source storage unit and the target storage unit.
In an optional implementation manner of the foregoing application, in the data migration process, when it is additionally identified that the marks of the target storage unit are all copied, the position corresponding relationship between the source storage unit and the target storage unit is deleted, so as to reduce the data storage amount, and meanwhile, implicitly release information that all data are migrated, thereby avoiding repeated mark search and comparison operations under the condition that all data are migrated, and reducing the data processing amount in the service function execution process.
Optionally, the location correspondence between the source storage unit and the target storage unit, the correspondence between the impact data and the source storage unit, and the mark are stored as meta information of the target volume.
In an optional implementation manner of the foregoing application, the location correspondence between the source storage unit and the target storage unit, the correspondence between the influence data and the source storage unit, and the mark are stored as meta information of the target volume, so that the data is searched or marked based on the correspondence and the mark.
In a second aspect, an embodiment of the present application further provides a data migration apparatus, including:
the storage unit dividing module is used for dividing the storage space of the source volume into a plurality of source storage units and dividing the storage space of the target volume into a plurality of target storage units;
the corresponding relation establishing module is used for establishing the position corresponding relation between each source storage unit and the target storage unit;
the source storage unit determining module is used for determining a source storage unit where the influence data of the data access request are located in the source volume when the data access request is acquired;
the influence data copying module is used for copying the data of the source storage unit where the influence data are located into the corresponding target storage unit if the influence data do not exist in the target storage unit corresponding to the source storage unit where the influence data are located according to the position corresponding relation;
and the access request response module is used for responding to the data access request according to the copied data in the target storage unit.
In a third aspect, an embodiment of the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a data migration method provided in the embodiments of the first aspect.
In a fourth aspect, embodiments of the present application further provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a data migration method provided in the first aspect.
Other effects of the above-described alternative will be described below with reference to specific embodiments.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1A is a flowchart of a data migration method according to a first embodiment of the present application;
FIG. 1B is a flow chart of another data migration method according to an embodiment of the present application;
FIG. 1C is a schematic diagram of a data access process in the first embodiment of the present application;
fig. 2 is a flowchart of a data migration method in the second embodiment of the present application;
fig. 3 is a block diagram of a data migration apparatus according to a third embodiment of the present application;
fig. 4 is a block diagram of an electronic device for implementing the data migration method according to the embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one
Fig. 1A is a flowchart of a data migration method in a first embodiment of the present application. The method is executed by a data migration device, and the device is realized through software and/or hardware and is specifically configured in electronic equipment. The electronic device may be a device having a certain control function for each storage space involved in the data migration process, and the device may be one of the storage spaces involved in the data migration, or may be another electronic device. The electronic device may be a server.
A method of data migration as shown in fig. 1A, comprising:
s101, dividing the storage space of the source volume into a plurality of source storage units, and dividing the storage space of the target volume into a plurality of target storage units.
The volume is a unit of storage space, and may include a storage space of a set size as needed. The source storage unit is used as a minimum unit of data migration in the source volume; accordingly, the target storage unit is used as the minimum unit of received data in the target volume.
The memory cells can be divided in a uniform or random division manner. Because each storage unit is the minimum unit for performing data migration once, in order to facilitate effective control of the migration process and avoid repeated reading and/or repeated writing of data, the storage space of the source volume is preferably divided into a plurality of source storage units with the same size by adopting a uniform division mode; accordingly, the storage space of the target volume is divided into a plurality of target storage units of equal size.
It is understood that, in order to ensure that the target storage unit in the target volume can match the source storage unit in the source volume and ensure the accuracy of data migration, the size of the target storage unit is usually set to be an integral multiple of the source storage unit. Typically, the source storage unit and the target storage unit are equal in size.
To avoid the impact of the data migration process on the service function, generally, the sizes of the source storage unit and the target storage unit are smaller, so as to ensure that the single data migration process does not affect the user I/O (input/output) access. For example, the size of the memory cell is 1 MB.
S102, establishing the position corresponding relation between the source storage unit and the target storage unit.
The position corresponding relation comprises a starting address of the source storage unit and a starting address of the target storage unit.
When the sizes of the source storage unit and the target storage unit are not equal, optionally, the location correspondence further includes an offset address of the source storage unit and an offset address of the target storage unit. Or, optionally, the location correspondence may further include a termination address of the source storage unit and a termination address of the target storage unit. In order to ensure the accuracy of data migration and avoid the occurrence of repeated read or write requests, typically, a one-to-one correspondence relationship between the source storage unit and the target storage unit is established.
It is to be understood that, for the convenience of subsequent lookup of the location correspondence, after the location correspondence between each of the source storage units and the target storage unit is established, the location correspondence between each of the source storage units and the target storage unit may be stored as meta information of the target volume.
S103, when the data access request is acquired, determining a source storage unit where the data influencing the data access request is located in the source volume.
The data access request may be a data read request or a data write request. The data affected by the data access request, i.e. the data affected by the data access, at least includes the data to be read and/or the data to be written.
Optionally, the source storage unit in which the influence data of the data access request is located in the source volume may be determined by pre-establishing a corresponding relationship between the influence data and the source storage unit, and correspondingly, the source storage unit in which the influence data of the data access request is located in the source volume is determined according to the corresponding relationship. Since the storage location of the data accessed by the upper layer in the physical storage space is generally maintained by the logical storage layer, the data stored in the source volume can be known based on the data storage location recorded in the past as to which source storage unit the data is stored.
S104, according to the position corresponding relation, if the influence data does not exist in the target storage unit corresponding to the source storage unit where the influence data is located, copying the data of the source storage unit where the influence data is located into the corresponding target storage unit.
Specifically, a target storage unit having a position corresponding relationship with a source storage unit where the influence data is located is determined according to the position corresponding relationship, and if the influence data does not exist in the target storage unit corresponding to the source storage unit where the influence data is located, the data of the source storage unit where the influence data is located is copied to the determined target storage unit.
It can be understood that when a user initiates a data access request, data in the source storage unit where the data is affected by the data access request is immediately migrated to the corresponding target storage unit, so that the access requirement of the user on the real-time performance of the data is met.
In an optional implementation manner of the embodiment of the present application, in order to facilitate subsequent use of searching for a location correspondence, after data is copied to a corresponding target storage unit, the location correspondence between the source storage unit and the target storage unit in the copy operation may also be stored as meta information of a target volume.
And S105, responding to the data access request according to the copied data in the target storage unit.
Illustratively, if the data access request is a data reading request, directly performing a data reading operation from the copied target storage unit of the target volume; and if the data access request is a data write request, and the data to be written in the data write request is directly written into the copied target storage unit of the target volume, so that the data in the target storage unit of the target volume can be added, deleted or modified.
The method comprises the steps of dividing the storage space of a source volume into a plurality of source storage units and dividing the storage space of a target volume into a plurality of target storage units; establishing a position corresponding relation between each source storage unit and each target storage unit; when a data access request is acquired, determining a source storage unit where data influencing the data access request is located in a source volume; copying data influencing the source storage unit where the data are located to a corresponding target storage unit according to the position corresponding relation; and responding to the data access request according to the copied data in the target storage unit. According to the technical scheme, the storage spaces of the source volume and the target volume are divided, so that data migration is performed according to the divided storage units, and the condition that the service is stopped due to mass migration is avoided. Meanwhile, the position corresponding relation between the source storage unit in the source volume and the target storage unit in the target volume is established, and the influence data of the data access request is automatically migrated from the source storage unit of the source volume to the corresponding target storage unit in the target volume according to the position corresponding relation, so that a specific file system is not needed, manual operation control is not needed, and the convenience of data migration is improved. And responding to the data access request according to the copied data in the target storage unit, thereby replacing the source volume with the target volume to execute corresponding business and synchronously realizing the migration of business services.
On the basis of the technical solutions of the foregoing embodiments, in order to implement migration of a service function, before acquiring a data access request, the method further includes, in step S103a, switching and mounting, in the logical storage layer, a data access volume of the application layer from a source volume to a target volume if a migration instruction of data from the source volume to the target volume is acquired. As shown in fig. 1B.
It is worth noting that since the data access volume is a single entry of the access operation, the consistency of the user data is ensured through the switching of the data access volume.
Referring to the schematic diagram of the data access process shown in fig. 1C, when mount switching is not implemented, after the data access request is obtained, since the data access volume of the application layer is still the source volume, the source volume access is performed first. After the mount switching is realized, after the data access request is acquired, the data access volume of the application layer is switched to the target volume, so that the data access operation can be directly performed in the target volume.
It should be noted that, in order to facilitate accurate execution of the data access operation in the target volume, the corresponding relationship between the impact data and the source storage unit may also be stored as the meta information of the target volume, so that the target volume can determine, according to the corresponding relationship between the impact data and the source storage unit, the current source storage unit corresponding to the impact data of the current data access request, and then determine, according to the corresponding relationship between the positions of the source storage unit and the target storage unit, the current target storage unit corresponding to the current source storage unit, so as to perform the data access operation in the current target storage unit.
Example two
Fig. 2 is a flowchart of a data migration method in the second embodiment of the present application, and the second embodiment of the present application performs optimization and improvement on the basis of the technical solutions of the foregoing embodiments.
Further, after the operation "copying the data of the source storage unit where the influence data is located to the corresponding target storage unit according to the position corresponding relation", adding "a mark for completing copying of the target storage unit after completing copying"; correspondingly, after determining the source storage unit where the influence data of the data access request is located in the source volume when the data access request is acquired, additionally determining the corresponding relationship according to the position, if the mark of the target storage unit corresponding to the source storage unit where the influence data is located is copied, abandoning the copying operation, and directly responding to the data access request according to the data in the target storage unit so as to effectively monitor the data migration condition between the source volume and the storage unit corresponding to the target volume.
A method of data migration as shown in fig. 2, comprising:
s201, dividing the storage space of the source volume into a plurality of source storage units, and dividing the storage space of the target volume into a plurality of target storage units.
S202, establishing the position corresponding relation between each source storage unit and the target storage unit.
S203, when the data access request is acquired, determining a source storage unit where the data influencing the data access request is located in the source volume.
S204, judging whether the mark of the target storage unit having the position corresponding relation with the source storage unit is copy completion or not, and if not, executing S205; otherwise, S208 is performed.
Wherein the flag is used to identify the completion of copying the data in the target storage unit.
In order to facilitate accurate control of the copying completion condition, the mark of the target storage unit can be stored as the meta-information of the target volume, so as to determine whether the target storage unit completes data copying according to the search comparison of the meta-information.
S205, copying the data of the source storage unit where the influence data are located to the corresponding target storage unit according to the position corresponding relation.
And when the target storage unit is determined to be not copied according to the mark, copying the data influencing the source storage unit where the data is located into the target storage unit having a position corresponding relationship with the source storage unit.
Preferably, the flag may further record the copying status, and then may wait for the response of data access after the current copy is completed. Since the space of the storage unit is small, copying is usually completed quickly, and thus the copying state may not be recorded.
S206, marking the copy completion of the target storage unit which completes the copy.
In order to ensure consistency between the marking condition and the actual copying condition of the target storage unit, after the data of the source storage unit is copied to the corresponding target storage unit, the target storage unit which is copied is marked for copying completion.
For example, the mark of the target storage unit may be stored as meta-information of the target volume, thereby implementing real-time update of the meta-information of the target volume.
And S207, responding to the data access request according to the copied data in the target storage unit.
And S208, responding to the data access request according to the data in the target storage unit.
And when the target storage unit is determined to be completely copied according to the mark, abandoning the copying operation, and directly responding to the data access request according to the data in the target storage unit.
It can be understood that when all target storage units of the target volume are copied, and each time a data access request is received, the determination of copying completion needs to be performed according to the flag, which will result in a certain memory usage and prolong the data access time. In order to avoid the above situation, on the basis of the technical solutions of the above embodiments, if it is recognized that the marks of all the target storage units are copied, the location correspondence between the source storage unit and the target storage unit is deleted, so that data access can be directly performed in the target storage unit of the target volume without intervention of the source storage unit.
In order to ensure the accuracy of data access in the target storage unit of the target volume, before deleting the position corresponding relationship between the source storage unit and the target storage unit, the corresponding relationship between the influence data and the target storage unit can be reestablished according to the position corresponding relationship between the source storage unit and the target storage unit and the corresponding relationship between the influence data and the source storage unit, the target volume is used as a new source volume, the reestablished corresponding relationship between the influence data and the target storage unit is used as the new corresponding relationship between the influence data and the source storage unit, and the corresponding relationship between the influence data and the source storage unit in the meta information is replaced.
According to the technical scheme of the embodiment of the application, the copy completion mark is carried out on the target storage unit which completes data migration, the copy completion state is determined according to the mark before the next data copy is carried out, the copy operation is abandoned when the copy is completed, repeated copy of data is avoided, the resource occupation amount during data access is saved, and the data access efficiency is improved.
EXAMPLE III
Fig. 3 is a block diagram of a data migration apparatus according to a third embodiment of the present application. The embodiment of the application is suitable for the condition that data in one storage system is smoothly migrated to another storage system when the storage system is updated or upgraded, and the device is realized through software and/or hardware and is specifically configured in the electronic equipment. The electronic device may be a device having a certain control function for each storage system involved in the data migration process, and the device may be one of the storage systems involved in the data migration, or may be another electronic device. The electronic device may be a server.
A data migration apparatus 300 as shown in fig. 3, comprising: a storage unit dividing module 301, a corresponding relation establishing module 302, a source storage unit determining module 303, an influence data copying module 304 and an access request responding module 305. Wherein the content of the first and second substances,
a storage unit dividing module 301, configured to divide a storage space of a source volume into a plurality of source storage units, and divide a storage space of a target volume into a plurality of target storage units;
a corresponding relationship establishing module 302, configured to establish a location corresponding relationship between each of the source storage units and the target storage unit;
a source storage unit determining module 303, configured to determine, when a data access request is obtained, a source storage unit in which influence data of the data access request is located in the source volume;
an influence data copying module 304, configured to copy, according to the position correspondence, data of a source storage unit in which the influence data is located into a corresponding target storage unit if the influence data does not exist in the target storage unit corresponding to the source storage unit in which the influence data is located;
an access request response module 305, configured to respond to the data access request according to the copied data in the target storage unit.
According to the embodiment of the application, the storage space of a source volume is divided into a plurality of source storage units through a storage unit dividing module, and the storage space of a target volume is divided into a plurality of target storage units; establishing a position corresponding relation between each source storage unit and each target storage unit through a corresponding relation establishing module; when the data access request is acquired through the source storage unit determining module, determining a source storage unit where data influencing the data access request is located in a source volume; copying the data of the source storage unit where the influence data are located to a corresponding target storage unit through the influence data copying module according to the corresponding position relation; and responding the data access request according to the copied data in the target storage unit through the access request response module. According to the technical scheme, the storage spaces of the source volume and the target volume are divided, so that data migration is performed according to the divided storage units, and the condition that the service is stopped due to mass migration is avoided. Meanwhile, the position corresponding relation between the source storage unit in the source volume and the target storage unit in the target volume is established, and the influence data of the data access request is automatically migrated from the source storage unit of the source volume to the corresponding target storage unit in the target volume according to the position corresponding relation, so that a specific file system is not needed, manual operation control is not needed, and the convenience of data migration is improved. And responding to the data access request according to the copied data in the target storage unit, thereby replacing the source volume with the target volume to execute corresponding business and synchronously realizing the migration of business services.
Furthermore, the sizes of the source storage units are equal, the sizes of the target storage units are equal, and the sizes of the source storage units and the target storage units are equal.
Further, the location correspondence includes a start address of the source storage unit and a start address of the target storage unit.
Further, the location correspondence relationship further includes an offset address of the source storage unit and an offset address of the target storage unit.
Further, the apparatus further includes a mount switching module, configured to:
before acquiring a data access request, if acquiring a migration indication of data from the source volume to the target volume, switching and mounting a data access volume of an application layer from the source volume to the target volume in a logic storage layer.
Further, the apparatus further comprises a marking module configured to:
after copying the data of the source storage unit where the influence data are located to the corresponding target storage unit according to the position corresponding relation, marking the target storage unit which is copied with a copy completion;
correspondingly, the apparatus further comprises a mark verification unit for:
when a data access request is acquired, after determining a source storage unit where influence data of the data access request is located in the source volume, according to the position corresponding relationship, if a target storage unit corresponding to the source storage unit where the influence data is located is marked as copy completion, discarding the copy operation, directly triggering an access request response module 305, and executing an operation of responding to the data access request according to data in the target storage unit.
Further, the apparatus further includes a correspondence deletion module configured to:
and if the marks of all the target storage units are identified to be copied, deleting the position corresponding relation between the source storage unit and the target storage unit.
Further, the apparatus further comprises a meta information storage module configured to:
and storing the position corresponding relation between the source storage unit and the target storage unit, the corresponding relation between the influence data and the source storage unit and the mark as the meta information of the target volume.
The data migration device can execute the data migration method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the data migration method.
Example four
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 4 is a block diagram of an electronic device implementing the data migration method according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 4, the electronic apparatus includes: one or more processors 401, memory 402, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 4, one processor 401 is taken as an example.
The memory 402, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the data migration method in the embodiment of the present application (for example, the storage unit dividing module 301, the correspondence relation establishing module 302, the source storage unit determining module 303, the influence data copying module 304, and the access request responding module 305 shown in fig. 3). The processor 401 executes various functional applications of the server and data processing by running non-transitory software programs, instructions, and modules stored in the memory 402, that is, implements the data migration method in the above-described method embodiments.
The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by use of the electronic device implementing the data migration method, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 402 may optionally include memory located remotely from the processor 401, and these remote memories may be connected over a network to an electronic device implementing the data migration method. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device implementing the data migration method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected by a bus or other means, and fig. 4 illustrates an example of a connection by a bus.
The input device 403 may receive input numeric or character information and generate key signal inputs related to user settings and function control of an electronic apparatus implementing the data migration method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 404 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
The method comprises the steps of dividing the storage space of a source volume into a plurality of source storage units and dividing the storage space of a target volume into a plurality of target storage units; establishing a position corresponding relation between each source storage unit and each target storage unit; when a data access request is acquired, determining a source storage unit where data influencing the data access request is located in a source volume; copying data influencing the source storage unit where the data are located to a corresponding target storage unit according to the position corresponding relation; and responding to the data access request according to the copied data in the target storage unit. According to the technical scheme, the storage spaces of the source volume and the target volume are divided, so that data migration is performed according to the divided storage units, and the condition that the service is stopped due to mass migration is avoided. Meanwhile, the position corresponding relation between the source storage unit in the source volume and the target storage unit in the target volume is established, and the influence data of the data access request is automatically migrated from the source storage unit of the source volume to the corresponding target storage unit in the target volume according to the position corresponding relation, so that a specific file system is not needed, manual operation control is not needed, and the convenience of data migration is improved. And responding to the data access request according to the copied data in the target storage unit, thereby replacing the source volume with the target volume to execute corresponding business and synchronously realizing the migration of business services.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (11)
1. A method of data migration, comprising:
dividing the storage space of a source volume into a plurality of source storage units, and dividing the storage space of a target volume into a plurality of target storage units;
establishing a position corresponding relation between each source storage unit and each target storage unit;
when a data access request is acquired, determining a source storage unit where influence data of the data access request are located in the source volume;
according to the position corresponding relation, if the influence data does not exist in a target storage unit corresponding to a source storage unit where the influence data is located, copying the data of the source storage unit where the influence data is located into the corresponding target storage unit;
and responding to the data access request according to the copied data in the target storage unit.
2. The method of claim 1, wherein the size of each of the source storage units is equal, the size of each of the target storage units is equal, and the size of the source storage unit and the size of the target storage unit are equal.
3. The method according to claim 1 or 2, wherein the location correspondence comprises a start address of the source storage unit and a start address of a target storage unit.
4. The method of claim 3, wherein the location correspondence further comprises an offset address of the source storage unit and an offset address of the target storage unit.
5. The method of claim 1, wherein prior to obtaining the data access request, further comprising:
and if the migration instruction of the data from the source volume to the target volume is acquired, switching and mounting the data access volume of the application layer from the source volume to the target volume in the logic storage layer.
6. The method according to claim 1, wherein after copying the data of the source storage unit where the influence data is located into the corresponding target storage unit according to the location correspondence, the method further comprises:
marking the target storage unit which is copied completely;
correspondingly, when a data access request is acquired, after determining a source storage unit in which the data affected by the data access request is located in the source volume, the method further includes:
and according to the position corresponding relation, if the mark of the target storage unit corresponding to the source storage unit of the influence data is completely copied, directly responding to the data access request according to the data in the target storage unit.
7. The method of claim 6, further comprising:
and if the marks of all the target storage units are identified to be copied, deleting the position corresponding relation between the source storage unit and the target storage unit.
8. The method of claim 6, wherein the location correspondence of the source storage unit and the target storage unit, the correspondence between the impact data and the source storage unit, and the label are stored as meta information of the target volume.
9. A data migration apparatus, comprising:
the storage unit dividing module is used for dividing the storage space of the source volume into a plurality of source storage units and dividing the storage space of the target volume into a plurality of target storage units;
the corresponding relation establishing module is used for establishing the position corresponding relation between each source storage unit and the target storage unit;
the source storage unit determining module is used for determining a source storage unit where the influence data of the data access request are located in the source volume when the data access request is acquired;
the influence data copying module is used for copying the data of the source storage unit where the influence data are located into the corresponding target storage unit if the influence data do not exist in the target storage unit corresponding to the source storage unit where the influence data are located according to the position corresponding relation;
and the access request response module is used for responding to the data access request according to the copied data in the target storage unit.
10. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of data migration as claimed in any one of claims 1 to 8.
11. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a data migration method as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010093246.3A CN111273872A (en) | 2020-02-14 | 2020-02-14 | Data migration method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010093246.3A CN111273872A (en) | 2020-02-14 | 2020-02-14 | Data migration method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111273872A true CN111273872A (en) | 2020-06-12 |
Family
ID=70999551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010093246.3A Pending CN111273872A (en) | 2020-02-14 | 2020-02-14 | Data migration method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111273872A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111913663A (en) * | 2020-07-29 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Storage volume online migration method and device and storage volume online migration system |
| CN112084173A (en) * | 2020-08-31 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Data migration method and device and storage medium |
| CN112667148A (en) * | 2020-11-30 | 2021-04-16 | 上海七牛信息技术有限公司 | Method and system for cluster storage data online migration |
| CN113050895A (en) * | 2021-04-22 | 2021-06-29 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for online switching of disk types |
| CN113254424A (en) * | 2021-06-11 | 2021-08-13 | 腾讯科技（深圳）有限公司 | Data service migration method, device and equipment |
| CN114091121A (en) * | 2022-01-18 | 2022-02-25 | 苏州浪潮智能科技有限公司 | Message abstract storage method, system, storage medium and equipment |
| WO2022174734A1 (en) * | 2021-02-18 | 2022-08-25 | 北京沃东天骏信息技术有限公司 | Method and apparatus for storing data |
| CN116028433A (en) * | 2022-05-16 | 2023-04-28 | 荣耀终端有限公司 | Data migration method and electronic equipment |
| CN117389713A (en) * | 2023-12-13 | 2024-01-12 | 苏州元脑智能科技有限公司 | Storage system application service data migration method, device, equipment and medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5835954A (en) * | 1996-09-12 | 1998-11-10 | International Business Machines Corporation | Target DASD controlled data migration move |
| US6145066A (en) * | 1997-11-14 | 2000-11-07 | Amdahl Corporation | Computer system with transparent data migration between storage volumes |
| US20060236056A1 (en) * | 2005-04-19 | 2006-10-19 | Koji Nagata | Storage system and storage system data migration method |
| CN101025703A (en) * | 2006-02-17 | 2007-08-29 | 国际商业机器公司 | Apparatus, system, and method for switching a volume address association in a point-in-time copy relationship |
| CN102929748A (en) * | 2012-11-08 | 2013-02-13 | 华为技术有限公司 | Method and device for data backup |
| CN103902233A (en) * | 2012-12-27 | 2014-07-02 | 中国电信股份有限公司 | Virtual machine system on-line storage migration method and device and server |
| CN106445421A (en) * | 2016-10-14 | 2017-02-22 | 郑州云海信息技术有限公司 | Storage system data migration method and system |
| CN106953893A (en) * | 2017-02-15 | 2017-07-14 | 北京百度网讯科技有限公司 | Data Migration between cloud storage system |
| CN107402722A (en) * | 2017-07-04 | 2017-11-28 | 杭州宏杉科技股份有限公司 | A kind of data migration method and storage device |
| US10296255B1 (en) * | 2015-12-16 | 2019-05-21 | EMC IP Holding Company LLC | Data migration techniques |
-
2020
- 2020-02-14 CN CN202010093246.3A patent/CN111273872A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5835954A (en) * | 1996-09-12 | 1998-11-10 | International Business Machines Corporation | Target DASD controlled data migration move |
| US6145066A (en) * | 1997-11-14 | 2000-11-07 | Amdahl Corporation | Computer system with transparent data migration between storage volumes |
| US20060236056A1 (en) * | 2005-04-19 | 2006-10-19 | Koji Nagata | Storage system and storage system data migration method |
| CN101025703A (en) * | 2006-02-17 | 2007-08-29 | 国际商业机器公司 | Apparatus, system, and method for switching a volume address association in a point-in-time copy relationship |
| CN102929748A (en) * | 2012-11-08 | 2013-02-13 | 华为技术有限公司 | Method and device for data backup |
| CN103902233A (en) * | 2012-12-27 | 2014-07-02 | 中国电信股份有限公司 | Virtual machine system on-line storage migration method and device and server |
| US10296255B1 (en) * | 2015-12-16 | 2019-05-21 | EMC IP Holding Company LLC | Data migration techniques |
| CN106445421A (en) * | 2016-10-14 | 2017-02-22 | 郑州云海信息技术有限公司 | Storage system data migration method and system |
| CN106953893A (en) * | 2017-02-15 | 2017-07-14 | 北京百度网讯科技有限公司 | Data Migration between cloud storage system |
| CN107402722A (en) * | 2017-07-04 | 2017-11-28 | 杭州宏杉科技股份有限公司 | A kind of data migration method and storage device |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111913663A (en) * | 2020-07-29 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Storage volume online migration method and device and storage volume online migration system |
| CN112084173A (en) * | 2020-08-31 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Data migration method and device and storage medium |
| CN112667148A (en) * | 2020-11-30 | 2021-04-16 | 上海七牛信息技术有限公司 | Method and system for cluster storage data online migration |
| WO2022174734A1 (en) * | 2021-02-18 | 2022-08-25 | 北京沃东天骏信息技术有限公司 | Method and apparatus for storing data |
| CN113050895A (en) * | 2021-04-22 | 2021-06-29 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for online switching of disk types |
| CN113254424A (en) * | 2021-06-11 | 2021-08-13 | 腾讯科技（深圳）有限公司 | Data service migration method, device and equipment |
| CN113254424B (en) * | 2021-06-11 | 2023-02-28 | 腾讯科技（深圳）有限公司 | Data service migration method, device and equipment |
| CN114091121A (en) * | 2022-01-18 | 2022-02-25 | 苏州浪潮智能科技有限公司 | Message abstract storage method, system, storage medium and equipment |
| CN116028433A (en) * | 2022-05-16 | 2023-04-28 | 荣耀终端有限公司 | Data migration method and electronic equipment |
| CN116028433B (en) * | 2022-05-16 | 2023-10-20 | 荣耀终端有限公司 | Data migration method and electronic equipment |
| CN117389713A (en) * | 2023-12-13 | 2024-01-12 | 苏州元脑智能科技有限公司 | Storage system application service data migration method, device, equipment and medium |
| CN117389713B (en) * | 2023-12-13 | 2024-02-23 | 苏州元脑智能科技有限公司 | Storage system application service data migration method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111273872A (en) | Data migration method, device, equipment and medium | |
| CN107533507B (en) | Method and system for managing data in storage device | |
| CN110780890B (en) | System upgrading method, device, electronic equipment and medium | |
| US9003149B2 (en) | Transparent file system migration to a new physical location | |
| US9417899B2 (en) | Memory page de-duplication in a computer system that includes a plurality of virtual machines | |
| US20180367379A1 (en) | Online upgrade method, apparatus, and system | |
| US9397953B2 (en) | Operation managing method for computer system, computer system and computer-readable storage medium having program thereon | |
| US10956335B2 (en) | Non-volatile cache access using RDMA | |
| EP3598301A1 (en) | Cloud management platform, virtual machine management method, system thereof | |
| WO2020259289A1 (en) | Resource allocation method and apparatus, electronic device and storage medium | |
| US10620871B1 (en) | Storage scheme for a distributed storage system | |
| KR20210040864A (en) | File directory traversal method, apparatus, device, and medium | |
| CN112565356A (en) | Data storage method and device and electronic equipment | |
| US20120284457A1 (en) | Selecting an auxiliary storage medium for writing data of real storage pages | |
| CN111767169B (en) | Data processing method, device, electronic equipment and storage medium | |
| US8738873B2 (en) | Interfacing with a point-in-time copy service architecture | |
| CN107329798B (en) | Data replication method and device and virtualization system | |
| US11055813B2 (en) | Method, electronic device and computer program product for expanding memory of GPU | |
| CN113032088A (en) | Dirty page recording method and device, electronic equipment and computer readable medium | |
| CN110728773A (en) | Image storage method and device and electronic equipment | |
| US10530870B2 (en) | Direct volume migration in a storage area network | |
| CN115562871A (en) | Memory allocation management method and device | |
| CN111597192B (en) | Database switching control method and device and electronic equipment | |
| CN112306372B (en) | Method, apparatus and program product for processing data | |
| CN114726657A (en) | Method and device for interrupt management and data receiving and sending management and intelligent network card |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |