CN107943572A - Data migration method, device, computer equipment and storage medium - Google Patents
Data migration method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN107943572A CN107943572A CN201711140525.5A CN201711140525A CN107943572A CN 107943572 A CN107943572 A CN 107943572A CN 201711140525 A CN201711140525 A CN 201711140525A CN 107943572 A CN107943572 A CN 107943572A
- Authority
- CN
- China
- Prior art keywords
- migration
- migrating
- migrating objects
- task
- objects
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
- G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Retry When Errors Occur (AREA)
Abstract
The present invention relates to a kind of data migration method, device, computer equipment and storage medium, the described method includes：Obtain migrating objects；Corresponding migration task is performed to the migrating objects according to default tasks carrying logic, to be migrated to the migrating objects, the tasks carrying logic includes the execution logic between multiple tasks；During being migrated, the corresponding current task state of the migrating objects and current transition state are recorded；If stopping migration during Data Migration is performed, when restarting migration, the corresponding current task state of migrating objects and current transition state when stopping migration being obtained；The migrating objects are migrated according to the corresponding current task state of migrating objects and current transition state when stopping migrating.The above method can be reduced migration workload, improve the efficiency of migration.
Description
Technical field
The present invention relates to data processing field, is situated between more particularly to data migration method, device, computer equipment and storage
Matter.
Background technology
With the development of science and technology and the increase of information content, can all there are the need migrated to data under many scenes
Ask, such as when the NAS (Network Attached Storage, network attached storage) of storing image files has licked standard guarantor
After repairing the period of service, the image file on NAS volumes can be moved in cloud storage and be stored.Data are moved from source storage device
When moving on to target storage device, the situation for stopping migration often occurring, such as meets with the abnormal shape such as network disconnection, unit exception
Condition etc..In conventional art, when occurring abnormal, if the transition process set is not completed, it is necessary to by data from source storage device weight
New to move to target storage device, migration heavy workload, transport efficiency are low.
The content of the invention
Based on this, it is necessary to for it is above-mentioned the problem of, there is provided a kind of data migration method, device, computer equipment and deposit
Storage media.
A kind of data migration method, the described method includes：Obtain migrating objects；According to default tasks carrying logic to institute
State migrating objects and perform corresponding migration task, to be migrated to the migrating objects, the tasks carrying logic includes more
Execution logic between a task；During being migrated, record the corresponding current task state of the migrating objects with
And current transition state；If stopping migration during Data Migration is performed, when restarting migration, obtain stopping and move
The corresponding current task state of migrating objects and current transition state during shifting；According to migrating objects when stopping migrating
Corresponding current task state and current transition state migrate the migrating objects.
It is in one embodiment, described that corresponding task is performed to the migrating objects according to default tasks carrying logic,
To include the step of migration to the migrating objects：Replication task is performed to replicate the migrating objects from source storage device
To target storage device；Verification of data task is performed to be matched the migrating objects after migration with the migrating objects before migration,
Matching result is obtained, the matching result includes successful match or it fails to match；According to the matching result execution flag task,
The migration results of the migrating objects are marked in the source storage device；If it fails to match, reparation task is performed from the source
Again the migrating objects are migrated in storage device.
In one embodiment, the current transition state include being carrying out, etc. pending, pause and mistake, it is described
According to stop migrate when the corresponding current task state of migrating objects and current transition state to the migrating objects into
The step of row migration, includes：When stopping migrating, the corresponding current transition state of the migrating objects is to be carrying out or wait
During execution, the corresponding current task state of the migrating objects corresponding task when stopping migration being performed；When stopping migration when institute
State the corresponding current transition state of migrating objects for mistake when, migrate the migrating objects again from the storage device of source；When stopping
When the corresponding current transition state of the migrating objects is pause when only migrating, if pause condition releases, when performing stopping migration
The corresponding task of the corresponding current task state of the migrating objects.
In one embodiment, the execution verification of data task is to the migrating objects after migration and the migrating objects before migration
The step of being matched, obtaining matching result includes：Breathed out according to the preset attribute information of the migrating objects before migration
It is uncommon to calculate, obtain the first cryptographic Hash；Hash calculation is carried out according to the preset attribute information of the migrating objects after migration, is obtained
Second cryptographic Hash；First cryptographic Hash and second cryptographic Hash are matched, if unanimously, matching result be into
Work(, if inconsistent, matching result is failure.
In one embodiment, the method further includes：Migration instruction is received, the migration instruction carries migration strategy, institute
Stating migration strategy includes migrating objects mark and transition condition；Described each move is obtained according to the migrating objects Data Identification
The attribute information of object is moved, and rope is generated according to institute's each corresponding attribute information of migrating objects and the transition condition
Draw information sequence；The step of acquisition migrating objects, includes:According to the index information retrieval migrating objects.
A kind of data migration device, described device include：Migrating objects acquisition module, for obtaining migrating objects；Task
Execution module, for performing corresponding migration task to the migrating objects according to default tasks carrying logic, with to described
Migrating objects are migrated, and the tasks carrying logic includes the execution logic between multiple tasks；State recording module, is used for
During being migrated, the corresponding current task state of the migrating objects and current transition state are recorded；State obtains
Modulus block, if for stopping migration during Data Migration is performed, when restarting migration, when obtaining stopping migration
The corresponding current task state of migrating objects and current transition state；Transferring module, for according to stopping migration when institute
State the corresponding current task state of migrating objects and current transition state migrates the migrating objects.
In one embodiment, the task execution module includes：Execution unit is replicated, for performing replication task by described in
Migrating objects copy to target storage device from source storage device；Execution unit is checked, will be moved for performing verification of data task
Migrating objects after shifting are matched with the migrating objects before migration, obtain matching result, the matching result include matching into
Work(or it fails to match；Execution unit is marked, for according to the matching result execution flag task, in the source storage device
Mark the migration results of the migrating objects；Execution unit is repaired, if for it fails to match, reparation task is performed and is deposited from the source
The migrating objects are migrated again in storage equipment.
In one embodiment, the current transition state include being carrying out, etc. pending, pause and mistake, it is described
Transferring module is used for：When stopping migrating, the corresponding current transition state of the migrating objects is pending to be carrying out or waiting
When, perform the corresponding current task state of the migrating objects corresponding task when stopping migration；Moved described in when stopping migrating
When the corresponding current transition state of shifting object is mistake, the migrating objects are migrated again from the storage device of source；When stopping is moved
When the corresponding current transition state of the migrating objects is suspends during shifting, if pause condition releases, perform described in when stopping migrating
The corresponding task of the corresponding current task state of migrating objects.
A kind of computer equipment, including memory and processor, are stored with computer program, the meter in the memory
When calculation machine program is performed by the processor so that the processor performs the step of above-mentioned data migration method.
A kind of computer-readable recording medium, it is characterised in that calculating is stored with the computer-readable recording medium
Machine program, when the computer program is executed by processor so that the processor performs the step of above-mentioned data migration method.
Above-mentioned data migration method, device, computer equipment and storage medium, when carrying out Data Migration, obtain migration
Migrating objects are performed corresponding migration task by object according to default tasks carrying logic, to be migrated to migrating objects,
And during being migrated, the corresponding current task state of migrating objects and current transition state are recorded, if performing
Stop migration during Data Migration, then it is corresponding according to migrating objects when stopping migrating when restarting to be migrated
Current task state and current transition state migrate migrating objects.Since migration is divided into multiple tasks, and set
Tasks carrying logic, and record the corresponding current task state of migrating objects during migration and currently migrate shape
State, therefore after migration is restarted, can be according to migrating objects corresponding current task state when stopping migrating and current
Transition state is migrated, and reduces migration workload, improves the efficiency of migration.
Brief description of the drawings
Fig. 1 is the implementation environment figure of the data migration method provided in one embodiment；
Fig. 2 is the flow chart of data migration method in one embodiment；
Fig. 3 is to perform corresponding task to migrating objects according to default tasks carrying logic in one embodiment, with right
The flow chart that migrating objects are migrated；
Fig. 4 is that verification of data task is performed in one embodiment to the migrating objects after migration and the migrating objects before migration
Matched, obtain the flow chart of matching result；
Fig. 5 is the flow chart of data migration method in one embodiment；
Fig. 6 is the structure diagram of data migration device in one embodiment；
Fig. 7 is the structure diagram of task execution module in one embodiment；
Fig. 8 is the structure diagram that execution unit is checked in one embodiment；
Fig. 9 is the structure diagram of data migration unit in one embodiment；
Figure 10 is the internal structure block diagram of one embodiment Computer equipment.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, it is right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
It is appreciated that such as explanation without exception, term used in this application " first ", " second " etc. can be used for herein
Various elements are described, but unless stated otherwise, these elements should not be limited by these terms.These terms are only used to by first member
Part is distinguished with another element.For example, in the case where not departing from scope of the present application, the first cryptographic Hash can be known as
Second cryptographic Hash, and similarly, the second cryptographic Hash can be known as the first cryptographic Hash.In this application, the migrating objects before migration
Migrating objects in the storage device of finger source, the migrating objects after migration refer to move to target storage device from the storage device of source after
The migrating objects being stored in target storage device.
Fig. 1 is the implementation environment figure of the data migration method provided in one embodiment, as shown in Figure 1, in the implementation ring
In border, including source storage device 110, computer equipment 120 and target storage device 130.
The migrating objects in need migrated are stored in source storage device storage 110, when needing to be migrated, are calculated
Machine equipment 120 receives migration instruction, obtains the migrating objects in source storage device 110, migrating objects then are moved to target
In storage device 130.Computer equipment 120 can be independent physical server or terminal or multiple physical services
The server cluster that device is formed, can be to provide the basic cloud computing service such as Cloud Server, cloud database, cloud storage and CDN
Cloud Server, the present invention are not limited again.
As shown in Fig. 2, in one embodiment it is proposed that a kind of data migration method, the data migration method can answer
For in above-mentioned computer equipment 120, specifically may comprise steps of：
Step S202, obtains migrating objects.
Specifically, migrating objects refer to the data to be migrated, and migrating objects can be image file, video file, sound
The data such as frequency file and text.Migrating objects can have one or more, can include a NAS in one embodiment
The all or part of data stored on disk or multiple NAS disks.When needing to carry out Data Migration, can be stored from corresponding source
Migrating objects are obtained in equipment.
Migrating objects are performed corresponding migration task, with to migration by step S204 according to default tasks carrying logic
Object is migrated, and tasks carrying logic includes the execution logic between multiple tasks.
Tasks carrying logic refers to the relation between task and task, be, for example, execution sequence between task and task and
The execution of particular task is using the implementing result of other tasks as execution condition, the execution parameter of particular task is according to other tasks
Implementing result generation etc..Such as B tasks are carried out after the completion of A tasks, perform D tasks if the failure of C tasks carryings.Moved
Before shifting, the process of migration is divided into multiple tasks, and the tasks carrying logic between these tasks is set.
In one embodiment, Data Migration can be divided into replication task, verification task, mark task and reparation to appoint
Business.Replication task is used to migrating objects copying to target storage device from source storage device；After verification task is used for migration
Migrating objects matched with the migrating objects before migration, obtain matching result, matching result be, for example, successful match or
It fails to match；Mark task is used for the migration results for marking migrating objects in the storage device of source according to matching result, works as matching
During success, mark migration results is migrate successfully, and when it fails to match, mark migration results fail for migration.Reparation task is used
In if it fails to match, migrating objects are migrated again from the storage device of source.
Step S206, during being migrated, records the corresponding current task state of migrating objects and currently moves
Shifting state.
Specifically, current task state record be being currently located in transition process for task, be, for example, downloading task or
Person marks task dispatching.Transition state refers to the practice condition of migrating objects, for example, can include being carrying out, etc. pending, pause
And mistake etc..
In one embodiment, transition state can perform adjustment with task.With the progress of task, the migration of data
State may also be changed.Since migration generally has multiple migrating objects, in order to improve transport efficiency, each task can
To open multiple processes, to carry out parallel migration to migrating objects.Therefore, after the completion of a task, if next task into
Number of passes is inadequate, then the corresponding transition state of migrating objects is pending by being carrying out being adjusted to etc..For example, when to a migration pair
After completing replication task, if verification task process number is inadequate, the migrating objects of the migrating objects is in waits.
In one embodiment, each task is corresponding can be the same or different into number of passes, specifically can be according to task
The execution time set, the length and process number for performing the time can be with correlations.For example, the replication task time
Long, verification task time is short, then the process for carrying out the advance ratio verification task of replication task is more.
In one embodiment, in transition process, number of processes can be adjusted according to the portfolio of source storage device.Work as industry
When being engaged in big, the quantity of process is reduced, to reduce the pressure of source storage device.When portfolio is small, increase the number of process
Amount, to improve the speed of migration.
In one embodiment, transition state can be adjusted according to migration adjust instruction, can send stopping migration and refer to
Order, stops carrying the mark for the migrating objects for needing to stop migration in migration instruction, then the mark of the migrating objects is corresponding moves
The transition state for moving object is adjusted to suspend.
Step S208, if stopping migration during Data Migration is performed, when restarting to be migrated, obtains
Stop the corresponding current task state of migrating objects and current transition state during migration.
Specifically, stop referring to that time for stopping migration during migration, it can be due to cause there occurs abnormal to stop migration
Or receive stop migration instruction be caused.When re-starting migration, migration pair when stopping migration being obtained
As corresponding current task state and current transition state.
Step S210, according to the corresponding current task state of migrating objects and current transition state when stopping migrating to moving
Object is moved to be migrated.
Specifically, get stopping migration when the corresponding current task state of migrating objects and current transition state after,
Migrating objects are migrated with reference to the corresponding current task state of migrating objects and current transition state when stopping migrating.
In one embodiment, transition state can include being carrying out, etc. pending, pause and mistake etc..Holding
During the capable migration task for referring to migrating objects is carrying out, the task where waiting pending finger migrating objects is not performing, but
If after the completion of the corresponding task of the preceding migrating objects that sort, start to continue to execute migration task.Pause, which refers to migrating objects, to be needed
Suspend migration, wait pause condition to be further continued for being migrated after releasing.Mistake refers to migrating objects after before migration with migrating
Data Matching failure or into target storage device download migrating objects failure, it is necessary to carry out reparation task to data again into
Row migration.When stopping migrating, the corresponding current transition state of migrating objects is is carrying out or when pending, execution stops
The corresponding current task state of migrating objects corresponding task when only migrating；Migrating objects when stopping migrating are corresponding current
When transition state is suspends, if restart migration, pause condition does not release also, then keeps the transition state of pause, not right
The migrating objects are migrated, and are released if suspending condition when restarting migration, and it is corresponding to perform migrating objects when stopping migration
The corresponding task of current task state.When stopping migrating, the corresponding current transition state of migrating objects is mistake, execution is repaiied
Multiple task migrates migrating objects again from the storage device of source.
For example, if the tasks carrying logical order of migration is task A- task B- tasks C.If be abnormal stopping migration
Task status be B, during transition state is waits, then task B is performed to migrating objects when restarting migration.If it is abnormal
Task status when causing to stop migration is B, and transition state is pause, and transition state is that pause is because temporarily during stopping migration
Stop condition for before July 18 to the data without migration, if it is July 17, not to the migration then to restart migration
Object is migrated.If being July 19 when restarting migration, task B is performed to migrating objects.
In one embodiment, migrating objects can be moved in one or more target storage devices.For example, it will move
Move object and move to main cluster and calamity at the same time for cluster, further lift transport efficiency, and main collection pocket transmission failure when
Time enables calamity to be responded in time for cluster.Data after migration can be stored by the way of object storage.
In one embodiment, record migration log information in transition process, makes whole transition process more transparence.
In one embodiment, as shown in figure 3, step S204 holds migrating objects according to default tasks carrying logic
The corresponding task of row, may comprise steps of with carrying out migration to migrating objects：
S302, performs replication task and migrating objects is copied to target storage device from source storage device.
Specifically, replicate and refer to the migrating objects being stored in the storage device of source being sent in target storage device, together
When can also retain data in the storage device of source.Source storage device can be the same or different with target storage device, example
Such as can be by a zoned migration of migrating objects slave device to another subregion, or from a virtual machine (vm) migration to another
Virtual machine.
S304, performs verification of data task and is matched the migrating objects after migration with the migrating objects before migration, obtain
To matching result.
Specifically, can be according to the migrating objects attribute informations such as the sizes of data, title and the content pair of migrating objects
Checked before migration with the migrating objects after migration.Can also be respectively according to moving after the migrating objects before migration and migration
Move object properties information and carry out Hash operation, the cryptographic Hash being calculated is matched.Hash calculation is random length
Input is transformed into the computational methods of the output of certain length by hashing algorithm, and hashing algorithm can be MD5 (Message-
Digest Algorithm5, message digest algorithm 5) algorithm and SHA (Secure Hash Standard, Secure Hash Standard)
Algorithm etc., is not limited specifically.
S306, according to matching result execution flag task, the migration results of mark migrating objects in the storage device of source.
Specifically, migration results include migration failure and migrate successfully.If matching result is successful match, deposited in source
Migration results are marked in storage equipment to migrate successfully.If matching result for it fails to match, marks migration in the storage device of source
As a result it is migration failure.
S308, if it fails to match, performs reparation task and migrates migrating objects again from the storage device of source.
Specifically, task is repaired for when it fails to match, the migrating objects in the storage device of source to be moved to target again
In storage device.
In one embodiment, if successful match, set to further confirm that migrating objects have migrated to target storage
In standby, can also carry out downloading task, migrating objects downloaded into target storage device, if failed download, illustrate to migrate not into
Work(migrates again, it is necessary to perform reparation task.If download successfully and successful match, execution flag task, the storage of mark source are set
The migration results of standby upper migrating objects are to migrate successfully.
In one embodiment, when service server receives the request of data for obtaining migrating objects, source can be obtained
Migration results in storage device, if the migration results of the migrating objects in the storage device of source store to migrate successfully to target
Data are obtained in equipment.
Fig. 4 shows that in one embodiment step S304 performs verification of data task to the migrating objects after migration
Matched with the migrating objects before migration, obtain the flow chart of realizing of matching result, specifically may comprise steps of：
Step S402, carries out Hash calculation according to the preset attribute information of the migrating objects before migration, obtains the first Hash
Value.
Step S404, carries out Hash calculation according to the preset attribute information of the migrating objects after migration, obtains the second Hash
Value.
Step S406, the first cryptographic Hash and the second cryptographic Hash are matched, if unanimously, matching result be successfully,
If inconsistent, matching result is failure.
Specifically, attribute information refers to the property of migrating objects, such as can include the size information and name of migrating objects
Claim etc..It is appreciated that when carrying out Hash calculation, the preset attribute letter of the migrating objects after migrating objects and migration before migration
The type of breath is identical, if such as make use of migration before migrating objects size information calculate the first cryptographic Hash, counting
Also the size information of the migrating objects after migration must be utilized when calculating the second cryptographic Hash.Hash calculation is the input random length
The computational methods of the output of certain length are transformed into by hashing algorithm, hashing algorithm can be MD5 (Message-digest
Algorithm5, message digest algorithm 5) algorithm and SHA (Secure Hash Standard, Secure Hash Standard) algorithm etc.,
It is not limited specifically.When obtaining the first cryptographic Hash and the second cryptographic Hash, whether compare the first cryptographic Hash and the second cryptographic Hash
Unanimously.
In certain embodiments, the attribute of the migrating objects on after the migrating objects before migration and migration can also be believed
Breath is checked, to further confirm that whether migrating objects are correctly moved in target storage device.
In one embodiment, as shown in figure 5, data migration method can also comprise the following steps：
Step S502, receive migration instruction, migration instruction carry migration strategy, migration strategy include migrating objects identify with
And transition condition.
Specifically, migrating objects mark can be specific object identity, or data type or data storage
Position etc. can be used for the character for identifying migrating objects.Transition condition level, which refers to, when to be migrated.Transition condition
Such as can be including one or more in the size of data requirement of time, the priority of migrating objects or the migrating objects migrated
It is a.The time of migration is to be immediately performed migration or perform migration in preset time.The priority of migrating objects refers to for instruction
Which high priority data is migrated.For example, for insurance industry, the data of life insurance are more important than production danger, then can set life insurance
Data priority data priority more dangerous than production is high.
In one embodiment, transition condition can also include the corresponding target storage address of data type.Such as life insurance
The corresponding target storage address of data is the address of A storage devices, and it is B storage devices to produce the corresponding target storage device of dangerous data
Address.
Step S504, according to migrating objects Data Identification obtain migrating objects attribute information, and according to attribute information with
And transition condition generation index information sequence.
Specifically, the attribute information of the migrating objects of acquisition can be determined according to transition condition.If for example, transition condition bag
Include which data type is preferentially migrated, then the attribute information of migrating objects includes data type, if transition condition is including excellent
During the small migrating objects of first migrating data amount, then the attribute information of the migrating objects obtained includes the size of migrating objects.When
To after attribute information, index information sequence is generated according to attribute information and transition condition.Index information in index information sequence
It is ranked up according to the attribute information of migrating objects and transition condition.For example, when transition condition is priority migration life insurance, will
The sequence of the data of life insurance is preceding.When transition condition is the big migrating objects of priority migration data volume, according to migrating objects
Data volume from big to small arranges index information.When obtaining index information sequence, moved according to index information retrieval
Move object.Such as migrating objects are obtained according to the sequence of index information.
In one embodiment, when further including the corresponding target storage address of data type in transition condition, acquisition
Attribute information includes data type, and generates index information sequence, index according to the corresponding target storage address of data type
Information sequence includes the corresponding target storage address of each migrating objects, therefore, when being migrated, according to migrating objects pair
The target storage address answered moves to migrating objects in corresponding target storage device.
As shown in fig. 6, in one embodiment, there is provided a kind of data migration device, the data migration device can collect
Into in above-mentioned computer equipment 120, including：
Migrating objects acquisition module 602, for obtaining migrating objects.
Task execution module 604, appoints for according to default tasks carrying logic migrating objects to be performed with corresponding migration
Business, to be migrated to migrating objects, tasks carrying logic includes the execution logic between multiple tasks.
State recording module 606, for during being migrated, recording the corresponding current task state of migrating objects
And current transition state.
State acquisition module 608, if for stopping migration during Data Migration is performed, is restarting to migrate
When, obtain the corresponding current task state of migrating objects and current transition state when stopping migration.
Transferring module 610, for according to migrating objects corresponding current task state when stopping migrating and current migration
State migrates migrating objects.
In one embodiment, current transition state include be carrying out, etc. it is pending, pause and mistake, migrate mould
Block is used for：When stopping migrating, the corresponding current transition state of migrating objects is is carrying out or when pending, execution stops
The corresponding current task state of migrating objects corresponding task when only migrating；When stopping migrating, migrating objects are corresponding currently moves
When shifting state is mistake, migrating objects are migrated again from the storage device of source；When stopping migrating, migrating objects are corresponding current
When transition state is suspends, if pause condition releases, perform migrating objects corresponding current task state when stopping migrating and correspond to
Task.
In one embodiment, as shown in fig. 7, task execution module 604 includes：
Execution unit 702 is replicated, migrating objects are copied into target storage from source storage device for performing replication task
Equipment.
Execution unit 704 is checked, for performing verification of data task by the migrating objects after migration and the migration before migration
Object is matched, and obtains matching result.
Execution unit 706 is marked, for according to matching result execution flag task, migration pair to be marked in the storage device of source
The migration results of elephant.
Execution unit 708 is repaired, if for it fails to match, performs the migration migration again from the storage device of source of reparation task
Object.
In one embodiment, as shown in figure 8, verification execution unit 704 includes：
First computation subunit 802, for carrying out Hash calculation according to the preset attribute information of the migrating objects before migration,
Obtain the first cryptographic Hash.
Second computation subunit 804, carries out Hash calculation according to the preset attribute information of the migrating objects after migration, obtains
Second cryptographic Hash.
Coupling subelement 806, for the first cryptographic Hash and the second cryptographic Hash to be matched, if unanimously, matching knot
Fruit is that successfully, if inconsistent, matching result is failure.
In one embodiment, as shown in figure 9, data migration device can also include：
Command reception module 902 is migrated, for receiving migration instruction, migration instruction carries migration strategy, migration strategy bag
Include migrating objects mark and transition condition.
Index information generation module 904, for obtaining the attribute information of migrating objects according to migrating objects Data Identification, and
According to attribute information and transition condition generation index information sequence.
Migrating objects acquisition module 602 is specifically used for according to index information retrieval migrating objects.
As shown in Figure 10, it is the cut-away view of one embodiment Computer equipment, which passes through system
Connect bus couple processor, non-volatile memory medium, built-in storage and network interface.Wherein, the computer equipment is non-
Volatile storage medium can storage program area and computer program, which is performed, and may be such that processor is held
A kind of data migration method of row.The processor of the computer equipment is used to provide calculating and control ability, supports whole computer
The operation of equipment.Computer program can be stored in the built-in storage, when which is executed by processor, may be such that place
Manage device and perform a kind of data migration method.The network interface of computer equipment is used to carry out network service, such as receives migration instruction,
Send and stop control instruction etc..It will be understood by those skilled in the art that the structure shown in Figure 10, it is only and application scheme
The block diagram of relevant part-structure, does not form the restriction for the computer equipment being applied thereon to application scheme, specifically
Computer equipment can include than more or fewer components shown in figure, either combine some components or with difference
Component arrangement.
In one embodiment, the data migration device that the application provides can be implemented as a kind of shape of computer program
Formula, computer program can be run in computer equipment as shown in Figure 10, and the non-volatile memory medium of computer equipment can
Storage forms each program module of the data migration device, such as migrating objects acquisition module 602, tasks carrying in Fig. 6
Module 604, state recording module 606, state acquisition module 608 and transferring module 610.Each program module includes computer
Program, computer program are used for the Data Migration for each embodiment of the application for making computer equipment perform described in this specification
Step in method.For example, computer equipment can obtain mould by the migrating objects in data migration device as shown in Figure 6
Block 602 obtains migrating objects, and correspondence is performed to migrating objects according to default tasks carrying logic by task execution module 604
Migration task, to be migrated to migrating objects, tasks carrying logic include multiple tasks between execution logic, pass through shape
State logging modle 606 records the corresponding current task state of migrating objects and currently migrates shape during being migrated
State, if stopping migration during Data Migration is performed by state acquisition module 608, when restarting migration, is obtained
The corresponding current task state of migrating objects and current transition state when taking the stopping to migrate, by transferring module 610 according to stopping
The corresponding current task state of migrating objects and current transition state migrate migrating objects when only migrating.
In one embodiment it is proposed that a kind of computer equipment, computer equipment include memory, processor and storage
On a memory and the computer program that can run on a processor, processor realize following steps when performing computer program：
Obtain migrating objects；Corresponding migration task is performed to migrating objects according to default tasks carrying logic, with to migrating objects
Migrated, tasks carrying logic includes the execution logic between multiple tasks；During being migrated, record migration pair
As corresponding current task state and current transition state；If stopping migration during Data Migration is performed, in weight
When newly starting migration, the corresponding current task state of migrating objects and current transition state when stopping migration being obtained；According to stopping
The corresponding current task state of migrating objects and current transition state migrate migrating objects when only migrating.
In one embodiment, it is corresponding to migrating objects execution according to default tasks carrying logic performed by processor
Task, to be migrated to migrating objects the step of, include：Replication task is performed to copy to migrating objects from source storage device
Target storage device；Perform verification of data task to be matched the migrating objects after migration with the migrating objects before migration, obtain
To matching result, matching result includes successful match or it fails to match；According to matching result execution flag task, in source, storage is set
The migration results of standby upper mark migrating objects；If it fails to match, the migration migration again from the storage device of source of reparation task is performed
Object.
In one embodiment, current transition state include be carrying out, etc. it is pending, pause and mistake, processor institute
What is performed carries out migrating objects according to migrating objects corresponding current task state when stopping migrating and current transition state
The step of migration, includes：When stopping migrating, the corresponding current transition state of migrating objects is pending to be carrying out or waiting
When, perform the corresponding current task state of migrating objects corresponding task when stopping migration；The migrating objects pair when stopping migrating
When the current transition state answered is mistake, migrating objects are migrated again from the storage device of source；The migrating objects when stopping migrating
When corresponding current transition state is suspends, if pause condition releases, it is corresponding as predecessor to perform migrating objects when stopping migrating
The corresponding task of business state.
In one embodiment, before the execution verification of data task performed by processor is to the migrating objects after migration and migration
Migrating objects the step of being matched, obtaining matching result include：According to the preset attribute information of the migrating objects before migration
Hash calculation is carried out, obtains the first cryptographic Hash；Hash calculation is carried out according to the preset attribute information of the migrating objects after migration, is obtained
To the second cryptographic Hash；First cryptographic Hash and the second cryptographic Hash are matched, if unanimously, matching result is successfully, if not
Unanimously, then matching result is failure.
In one embodiment, following steps are also performed when processor performs computer program：Migration instruction is received, migration refers to
Order carries migration strategy, and migration strategy includes migrating objects mark and transition condition；Obtained according to migrating objects Data Identification
The attribute information of each migrating objects, and according to the corresponding attribute information of each migrating objects of institute and transition condition generation index
Information sequence；The step of obtaining migrating objects includes：According to index information retrieval migrating objects.
In one embodiment, there is provided a kind of computer-readable recording medium, is stored with computer-readable recording medium
Computer program, when computer program is executed by processor so that processor performs following steps：Obtain migrating objects；According to
Default tasks carrying logic performs corresponding migration task, to be migrated to migrating objects, tasks carrying to migrating objects
Logic includes the execution logic between multiple tasks；During being migrated, the corresponding current task of record migrating objects
State and current transition state；If stopping migration during Data Migration is performed, when restarting migration, obtain
Stop the corresponding current task state of migrating objects and current transition state during migration；According to migrating objects pair when stopping migrating
The current task state and current transition state answered migrate migrating objects.
In one embodiment, it is corresponding to migrating objects execution according to default tasks carrying logic performed by processor
Task, to be migrated to migrating objects the step of, include：Replication task is performed to copy to migrating objects from source storage device
Target storage device；Perform verification of data task to be matched the migrating objects after migration with the migrating objects before migration, obtain
To matching result, matching result includes successful match or it fails to match；According to matching result execution flag task, in source, storage is set
The migration results of standby upper mark migrating objects；If it fails to match, the migration migration again from the storage device of source of reparation task is performed
Object.
In one embodiment, current transition state include be carrying out, etc. it is pending, pause and mistake, processor institute
What is performed carries out migrating objects according to migrating objects corresponding current task state when stopping migrating and current transition state
The step of migration, includes：When stopping migrating, the corresponding current transition state of migrating objects is pending to be carrying out or waiting
When, perform the corresponding current task state of migrating objects corresponding task when stopping migration；The migrating objects pair when stopping migrating
When the current transition state answered is mistake, migrating objects are migrated again from the storage device of source；The migrating objects when stopping migrating
When corresponding current transition state is suspends, if pause condition releases, it is corresponding as predecessor to perform migrating objects when stopping migrating
The corresponding task of business state.
In one embodiment, before the execution verification of data task performed by processor is to the migrating objects after migration and migration
Migrating objects the step of being matched, obtaining matching result include：According to the preset attribute information of the migrating objects before migration
Hash calculation is carried out, obtains the first cryptographic Hash；Hash calculation is carried out according to the preset attribute information of the migrating objects after migration, is obtained
To the second cryptographic Hash；First cryptographic Hash and the second cryptographic Hash are matched, if unanimously, matching result is successfully, if not
Unanimously, then matching result is failure.
In one embodiment, following steps are also performed when processor performs computer program：Migration instruction is received, migration refers to
Order carries migration strategy, and migration strategy includes migrating objects mark and transition condition；Obtained according to migrating objects Data Identification
The attribute information of each migrating objects, and according to the corresponding attribute information of each migrating objects of institute and transition condition generation index
Information sequence；The step of obtaining migrating objects includes：According to index information retrieval migrating objects.
One of ordinary skill in the art will appreciate that realize all or part of flow in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, which can be stored in a computer-readable storage and be situated between
In matter, the program is upon execution, it may include such as the flow of the embodiment of above-mentioned each method.Wherein, foregoing storage medium can be
The non-volatile memory mediums such as magnetic disc, CD, read-only memory (Read-Only Memory, ROM), or random storage note
Recall body (Random Access Memory, RAM) etc..
Each technical characteristic of above example can be combined arbitrarily, to make description succinct, not to above-described embodiment
In each technical characteristic it is all possible combination be all described, as long as however, lance is not present in the combination of these technical characteristics
Shield, is all considered to be the scope of this specification record.
Above example only expresses the several embodiments of the present invention, its description is more specific and detailed, but can not
Therefore it is interpreted as the limitation to the scope of the claims of the present invention.It should be pointed out that for those of ordinary skill in the art,
Without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the protection model of the present invention
Enclose.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (10)
- A kind of 1. data migration method, it is characterised in that the described method includes：Obtain migrating objects；Corresponding migration task is performed to the migrating objects according to default tasks carrying logic, with to the migrating objects into Row migration, the tasks carrying logic include the execution logic between multiple tasks；During being migrated, the corresponding current task state of the migrating objects and current transition state are recorded；If stopping migration during Data Migration is performed, when restarting migration, obtain and moved described in when stopping migration Move the corresponding current task state of object and current transition state；According to migrating objects corresponding current task state when stopping migrating and current transition state to the migration pair As being migrated.
- 2. according to the method described in claim 1, it is characterized in that, it is described according to default tasks carrying logic to the migration Object performs corresponding task, to include the step of migration to the migrating objects：Perform replication task and the migrating objects are copied into target storage device from source storage device；Perform verification of data task to be matched the migrating objects after migration with the migrating objects before migration, obtain matching knot Fruit, the matching result includes successful match or it fails to match；According to the matching result execution flag task, the migration knot of the migrating objects is marked in the source storage device Fruit；If it fails to match, perform reparation task and migrate the migrating objects again from the source storage device.
- 3. according to the method described in claim 1, it is characterized in that, the current transition state includes being carrying out, waiting and holding Row, pause and mistake, it is described according to the migrating objects corresponding current task state when stopping migrating and current migration The step of state migrates the migrating objects includes：When stopping migrating, the corresponding current transition state of the migrating objects is is carrying out or when pending, execution stops The corresponding current task state of the migrating objects corresponding task when only migrating；When stopping migrating, the corresponding current transition state of the migrating objects is mistake, migrated again from the storage device of source The migrating objects；When stopping migrating, the corresponding current transition state of the migrating objects is suspends, if pause condition releases, execution stops The corresponding current task state of the migrating objects corresponding task when only migrating.
- 4. according to the method described in claim 2, it is characterized in that, the execution verification of data task is to the migration pair after migration As including the step of matched with the migrating objects before migration, obtain matching result：Hash calculation is carried out according to the preset attribute information of the migrating objects before migration, obtains the first cryptographic Hash；Hash calculation is carried out according to the preset attribute information of the migrating objects after migration, obtains the second cryptographic Hash；First cryptographic Hash and second cryptographic Hash are matched, if unanimously, matching result is successfully, if differing Cause, then matching result is failure.
- 5. according to the method described in claim 1, it is characterized in that, the method further includes：Migration instruction is received, the migration instruction carries migration strategy, and the migration strategy includes migrating objects and identifies and move Shifting condition；Obtain the attribute information of each migrating objects according to the migrating objects Data Identification, and each moved according to institute is described Move the corresponding attribute information of object and transition condition generation index information sequence；The step of acquisition migrating objects, includes:According to the index information retrieval migrating objects.
- 6. a kind of data migration device, it is characterised in that described device includes：Migrating objects acquisition module, for obtaining migrating objects；Task execution module, for performing corresponding migration task to the migrating objects according to default tasks carrying logic, To be migrated to the migrating objects, the tasks carrying logic includes the execution logic between multiple tasks；State recording module, for during being migrated, record the corresponding current task state of the migrating objects with And current transition state；State acquisition module, if for stopping migration during Data Migration is performed, when restarting migration, obtains Stop the corresponding current task state of migrating objects and current transition state during migration；Transferring module, for according to the corresponding current task state of migrating objects and current transition state when stopping migrating The migrating objects are migrated.
- 7. device according to claim 6, it is characterised in that the task execution module includes：Execution unit is replicated, the migrating objects are copied to target storage from source storage device for performing replication task sets It is standby；Execution unit is checked, carries out the migrating objects after migration and the migrating objects before migration for performing verification of data task Matching, obtains matching result, the matching result includes successful match or it fails to match；Execution unit is marked, for according to the matching result execution flag task, in the source storage device described in mark The migration results of migrating objects；Execution unit is repaired, if for it fails to match, reparation task is performed and is moved again described in migration from the source storage device Move object.
- 8. device according to claim 6, it is characterised in that the current transition state includes being carrying out, waiting and holding Row, pause and mistake, the transferring module are used for：When stopping migrating, the corresponding current transition state of the migrating objects is is carrying out or when pending, execution stops The corresponding current task state of the migrating objects corresponding task when only migrating；When stopping migrating, the corresponding current transition state of the migrating objects is mistake, migrated again from the storage device of source The migrating objects；When stopping migrating, the corresponding current transition state of the migrating objects is suspends, if pause condition releases, execution stops The corresponding current task state of the migrating objects corresponding task when only migrating.
- 9. a kind of computer equipment, it is characterised in that including memory and processor, computer journey is stored with the memory Sequence, when the computer program is performed by the processor so that any one of 1 to 5 power of processor perform claim requirement Profit requires the step of data migration method.
- 10. a kind of computer-readable recording medium, it is characterised in that be stored with computer on the computer-readable recording medium Program, when the computer program is executed by processor so that the processor perform claim requires any one of 1 to 5 right It is required that the step of data migration method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711140525.5A CN107943572B (en) | 2017-11-16 | 2017-11-16 | Data migration method, device, computer equipment and storage medium |
| PCT/CN2018/077106 WO2019095576A1 (en) | 2017-11-16 | 2018-02-24 | Data migration method, apparatus, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711140525.5A CN107943572B (en) | 2017-11-16 | 2017-11-16 | Data migration method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107943572A true CN107943572A (en) | 2018-04-20 |
| CN107943572B CN107943572B (en) | 2019-09-03 |
Family
ID=61931496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711140525.5A Active CN107943572B (en) | 2017-11-16 | 2017-11-16 | Data migration method, device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107943572B (en) |
| WO (1) | WO2019095576A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109450997A (en) * | 2018-10-26 | 2019-03-08 | 平安科技（深圳）有限公司 | Data cross-terminal moving method, device, computer equipment and storage medium |
| CN109766392A (en) * | 2019-01-21 | 2019-05-17 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN109884897A (en) * | 2019-03-21 | 2019-06-14 | 中山大学 | A kind of matching of unmanned plane task and computation migration method based on deeply study |
| CN110209344A (en) * | 2018-07-26 | 2019-09-06 | 腾讯科技（深圳）有限公司 | The method, apparatus and computer readable storage medium of migrating objects storage |
| CN110335012A (en) * | 2019-05-30 | 2019-10-15 | 广东民航机场建设有限公司 | Project task state switching method, device, computer equipment and storage medium |
| CN113032369A (en) * | 2021-03-26 | 2021-06-25 | 山东英信计算机技术有限公司 | Data migration method, device and medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131179B (en) * | 2020-09-23 | 2023-11-21 | 平安科技（深圳）有限公司 | Task state detection method, device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120303739A1 (en) * | 2011-05-27 | 2012-11-29 | James Michael Ferris | Systems and methods for determining consistencies in staged replication data to improve data migration efficiency in cloud based networks |
| CN103902593A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团河南有限公司 | Data transfer method and device |
| CN105528368A (en) * | 2014-09-30 | 2016-04-27 | 北京金山云网络技术有限公司 | A database migration method and device |
| CN105824846A (en) * | 2015-01-09 | 2016-08-03 | 阿里巴巴集团控股有限公司 | Data migration method and device |
-
2017
- 2017-11-16 CN CN201711140525.5A patent/CN107943572B/en active Active
-
2018
- 2018-02-24 WO PCT/CN2018/077106 patent/WO2019095576A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120303739A1 (en) * | 2011-05-27 | 2012-11-29 | James Michael Ferris | Systems and methods for determining consistencies in staged replication data to improve data migration efficiency in cloud based networks |
| CN103902593A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团河南有限公司 | Data transfer method and device |
| CN105528368A (en) * | 2014-09-30 | 2016-04-27 | 北京金山云网络技术有限公司 | A database migration method and device |
| CN105824846A (en) * | 2015-01-09 | 2016-08-03 | 阿里巴巴集团控股有限公司 | Data migration method and device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209344A (en) * | 2018-07-26 | 2019-09-06 | 腾讯科技（深圳）有限公司 | The method, apparatus and computer readable storage medium of migrating objects storage |
| CN110209344B (en) * | 2018-07-26 | 2022-02-18 | 腾讯科技（深圳）有限公司 | Method, device and computer readable storage medium for migrating object storage |
| CN109450997A (en) * | 2018-10-26 | 2019-03-08 | 平安科技（深圳）有限公司 | Data cross-terminal moving method, device, computer equipment and storage medium |
| CN109450997B (en) * | 2018-10-26 | 2022-09-30 | 平安科技（深圳）有限公司 | Data cross-terminal migration method and device, computer equipment and storage medium |
| CN109766392A (en) * | 2019-01-21 | 2019-05-17 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN109884897A (en) * | 2019-03-21 | 2019-06-14 | 中山大学 | A kind of matching of unmanned plane task and computation migration method based on deeply study |
| CN109884897B (en) * | 2019-03-21 | 2022-03-25 | 中山大学 | Unmanned aerial vehicle task matching and calculation migration method based on deep reinforcement learning |
| CN110335012A (en) * | 2019-05-30 | 2019-10-15 | 广东民航机场建设有限公司 | Project task state switching method, device, computer equipment and storage medium |
| CN113032369A (en) * | 2021-03-26 | 2021-06-25 | 山东英信计算机技术有限公司 | Data migration method, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019095576A1 (en) | 2019-05-23 |
| CN107943572B (en) | 2019-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107943572B (en) | Data migration method, device, computer equipment and storage medium | |
| US9251235B1 (en) | Log-based synchronization | |
| US9417976B2 (en) | Preventing migration of a virtual machine from affecting disaster recovery of replica | |
| US20170124103A1 (en) | Method and Apparatus for Creating System Disk Snapshot of Virtual Machine | |
| CN105022656B (en) | virtual machine snapshot management method and device | |
| US10467424B2 (en) | File system content based security | |
| US11640341B1 (en) | Data recovery in a multi-pipeline data forwarder | |
| CN107391629A (en) | Data migration method, system, server and computer-readable storage medium between cluster | |
| US9552407B1 (en) | Log-based synchronization with conditional append | |
| CN109145051A (en) | The data summarization method and device and electronic equipment of distributed data base | |
| CN105574217A (en) | Data synchronization method and device of distributed relational database | |
| US12056153B2 (en) | Replicating large statements with low latency | |
| US10956446B1 (en) | Log-based synchronization with inferred context | |
| US20230315584A1 (en) | Backing up data for a namespace assigned to a tenant | |
| CN109165210A (en) | A kind of method and device of cluster Hbase Data Migration | |
| CN109542860B (en) | Service data management method based on HDFS and terminal equipment | |
| CN108536822A (en) | Data migration method, device, system and storage medium | |
| CN109471901A (en) | A kind of method of data synchronization and device | |
| US11023493B2 (en) | Intelligently scheduling resynchronization jobs in a distributed object-based storage system | |
| CN103176843B (en) | The file migration method and apparatus of MapReduce distributed system | |
| KR20160145250A (en) | Shuffle Embedded Distributed Storage System Supporting Virtual Merge and Method Thereof | |
| CN110677497B (en) | Network medium distribution method and device | |
| JP4715777B2 (en) | Replication method, storage apparatus, and program | |
| WO2021185059A1 (en) | Data migration method and apparatus for database | |
| CN113282551B (en) | Data processing method, system and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |