CN112527765A - Data migration method and device - Google Patents
Data migration method and device Download PDFInfo
- Publication number
- CN112527765A CN112527765A CN202011398952.5A CN202011398952A CN112527765A CN 112527765 A CN112527765 A CN 112527765A CN 202011398952 A CN202011398952 A CN 202011398952A CN 112527765 A CN112527765 A CN 112527765A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- migration
- determining
- data object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013508 migration Methods 0.000 title claims abstract description 165
- 230000005012 migration Effects 0.000 title claims abstract description 165
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 11
- 230000008520 organization Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data migration method and a device, wherein the method comprises the following steps: receiving a data migration request, wherein the data migration request comprises a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database; determining a first data structure of the source data object and a second data structure of the target data object, respectively; determining a data migration rule according to the first data structure and the second data structure; and storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule. Therefore, the overall uniformity of the data in the database is improved, the data in the database does not need to be sorted additionally, and the data sorting cost is reduced.
Description
Technical Field
The present application relates to the field of network technologies, and in particular, to a data migration method and apparatus.
Background
The DataX is an offline synchronization tool for heterogeneous data sources of an Ali open source, and aims to realize stable and efficient data synchronization functions among various heterogeneous data sources including a relational database (MySQL, Oracle and the like), an HDFS (Hadoop distributed file system), a Hive (Hive), an ODPS (ODPS), an HBase, an FTP (File transfer protocol) and the like. The data source can be used for synchronizing data through the data source, wherein the data source can be used for erecting a 'bridge'; the original data synchronization link in a network form between data sources is changed into a divergent data synchronization link taking DataX as the center, and the efficiency and the stability of data synchronization between heterogeneous data sources are improved.
However, although this method can achieve data synchronization between heterogeneous data sources, as the amount of data to be synchronized increases, the complexity of the data sources increases, and the overall uniformity of the data decreases. And this approach can only support data synchronization between locally running heterogeneous data sources.
Therefore, a data migration method is needed to ensure the overall uniformity of data source data while performing data migration.
Disclosure of Invention
Embodiments of the present invention provide a data migration processing method and apparatus, which can ensure the overall uniformity of data source data while performing data migration.
In a first aspect, an embodiment of the present invention provides a data migration method, where the method includes:
receiving a data migration request, wherein the data migration request comprises a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database;
determining a first data structure of the source data object and a second data structure of the target data object, respectively;
determining a data migration rule according to the first data structure and the second data structure;
and storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule.
In the method, a data migration rule is determined according to the first data structure and the second data structure, and each data of the source data object in the first database is stored into the target data object in the second database according to the data migration rule. In this manner, the data of the source data object may be caused to be stored in a data structure that conforms to the target data object. Compared with the prior art, the method has the advantages that each data of the source data object is directly stored in the second database, so that the data in the second database is messy, and the overall uniformity is lacked. By the method and the device, the target data in the second database can be sorted in the storage process, and the overall uniformity of the data is improved. And the data in the second database does not need to be sorted additionally, so that the data sorting cost is reduced.
Optionally, determining a data migration rule according to the first data structure and the second data structure includes: determining the name and the type of each first data item in the first data structure, and determining the name and the type of each second data item in the second data structure; if the name of a first data item is the same as that of the second data item, determining to migrate the data under the first data item to the second data item with the same name; if the name of a first data item is different from the names of all second data items, and the type of the first data item is the same as that of the second data items, determining to migrate the data in the first data item to the second data items with the same type; if the name of a first data item is different from the name of each second data item, and the type of the first data item is different from the type of each second data item, adding the first data items in sequence after the last second data item of the second data structure.
In the method, the data name, the data type and the data sequence in the data structure of the source data and the data structure of the target data are matched. And determining a data migration rule by using a matching algorithm with a data name of first priority, a data type of second priority and a data sequence of third priority. The data migration rule can be determined simply, quickly and accurately. Therefore, each data of the source data object is migrated according to the data migration rule, and the data integrity of the target data object in the second database is improved.
Optionally, respectively determining the first data structure of the source data object and the second data structure of the target data object includes: searching a configuration record according to the source data object and the target data object, and respectively acquiring first connection information of the first database and second connection information of the second database; acquiring the first data structure from the first database according to the first connection information; and acquiring the second data structure from the second database according to the second connection information.
In the method, the configuration record is searched according to the source data object and the target data object, and the first connection information and the second connection information are obtained, where the first connection information and the second connection information may include information such as a server address, a file path, or a database name, so that the data structure of the source data object is obtained from the first database according to the first connection information, and the data structure of the target data object is obtained from the second database according to the second connection information.
Optionally, receiving a data migration request includes: and receiving a data migration request in an HTTP format through a preset API (application programming interface).
In the method, the data migration request in the HTTP format is received through the preset API, so that the remote operation of data migration can be realized.
Optionally, before determining the first data structure of the source data object and the second data structure of the target data object respectively, the method further includes: and verifying the data migration request, and determining that the data migration request conforms to a verification rule.
In the method, the data migration request is verified, and the data migration request is determined to accord with the verification rule. The legality of the data migration request is guaranteed, and the safety of data migration operation is further guaranteed.
Optionally, after determining the data migration rule according to the first data structure and the second data structure, the method further includes: displaying the data migration rule; and receiving the adjustment execution of the data migration rule by the user to obtain an updated data migration rule.
In the method, by displaying the data migration rule, the user can correspondingly adjust the data migration rule according to the displayed data migration rule, so that the adjusted data migration rule is more in line with the requirement. Further, the data migration is carried out according to the adjusted data migration rule, so that the overall data uniformity and the availability of the database can be further improved.
Optionally, after obtaining each data of the source data object from the first database, before storing the data into the target data object of the second database according to the data migration rule, the method further includes: and converting each data in the source data object using the first data language into each data in the target data object using the second data language.
In the above method, each data in the source data object that uses the first data language is converted into each data in the target data object that uses the second data language. Therefore, the situation that the database cannot perform corresponding processing on the data in the database due to the fact that the data languages of the data in the database are inconsistent is prevented. The uniformity of data languages in the database and the availability of data in the database are ensured.
In a second aspect, an embodiment of the present invention provides a data apparatus method, where the apparatus includes:
the data migration system comprises a receiving and sending module, a sending and receiving module and a processing module, wherein the receiving and sending module is used for receiving a data migration request which comprises a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database;
a determining module for determining a first data structure of the source data object and a second data structure of the target data object, respectively;
the determining module is further configured to determine a data migration rule according to the first data structure and the second data structure;
and the migration module is used for storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule.
In a third aspect, an embodiment of the present application further provides a computing device, including: a memory for storing a program; a processor for calling the program stored in said memory and executing the method as described in the various possible designs of the first aspect according to the obtained program.
In a fourth aspect, embodiments of the present application further provide a computer-readable non-transitory storage medium including a computer-readable program which, when read and executed by a computer, causes the computer to perform the method as described in the various possible designs of the first aspect.
These and other implementations of the present application will be more readily understood from the following description of the embodiments.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a block diagram of a data migration architecture according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a data migration method according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of a data migration method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides a system architecture for data migration, and as shown in fig. 1, the data migration system includes a network middle station, a generated task configuration unit, a verification unit, a log unit, a task submitting or terminating unit, a task pool, a data migration component, and an executor. And the user inputs the source data object and the target data object to generate a data migration request by opening a data migration interface. The network interface is packaged in the network middle station and used for providing the calling function of the data migration system for the user. The network central station receives a data migration request, sends the data migration request to a verification unit, verifies the validity of the data migration request, sends the data migration request to a generated task configuration unit after the verification is passed, the generated task configuration unit searches configuration records according to a source data object and a target data object in the data migration request, obtains first connection information of a first database and second connection information of a second database, the generated task configuration unit generates a data migration task according to the source data object, the target data object, the first connection information, the second connection information and the like, submits the data migration task to a submission or termination task unit, the submission or termination task unit sends the task to a task pool, and a data migration component waits for obtaining the data migration task.
After the data migration component acquires the data migration task, determining a query command which can be identified and correspondingly executed by the first database through an actuator of the first database according to the first connection information and the source data object, and acquiring a first data structure of the source data object from the first database; determining a query command which can be identified and correspondingly executed by the second database through an actuator of the second database, and acquiring a second data structure of the target data object from the second database according to the second connection information and the target data object; the data migration component determines a data migration rule according to the first data structure and the second data structure; the data migration component displays the data migration rule to a user through a network middlebox and receives the data migration rule adjusted by the user; and the data migration component acquires each data corresponding to the source data object from the first database through the first connection information and the source data object, converts each data in the first data language into each data in the target data object using the second data language, and stores each data converted into the second data language into the second database according to the adjusted data migration rule. The log unit records the state of the process and other related parameters, so that a user can conveniently search the running state, the related parameters and the like subsequently. It should be noted that, if the user does not adjust the data migration rule, the data migration is still performed according to the data migration rule determined by the first data structure and the second data structure. Wherein, if the data migration component is replaced by the DataX, the remote invocation of the DataX can be realized.
Based on this, an embodiment of the present application provides a flow of a data migration method, as shown in fig. 2, including:
and 204, storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule.
In the method, a data migration rule is determined according to the first data structure and the second data structure, and each data of the source data object in the first database is stored into the target data object in the second database according to the data migration rule. In this manner, the data of the source data object may be caused to be stored in a data structure that conforms to the target data object. Compared with the prior art, the method has the advantages that each data of the source data object is directly stored in the second database, so that the data in the second database is messy, and the overall uniformity is lacked. By the method and the device, the target data in the second database can be sorted in the storage process, and the overall uniformity of the data is improved. And the data in the second database does not need to be sorted additionally, so that the data sorting cost is reduced.
The embodiment of the application provides a method for determining a data migration rule, wherein the method for determining the data migration rule according to the first data structure and the second data structure comprises the following steps: determining the name and the type of each first data item in the first data structure, and determining the name and the type of each second data item in the second data structure; if the name of a first data item is the same as that of the second data item, determining to migrate the data under the first data item to the second data item with the same name; if the name of a first data item is different from the names of all second data items, and the type of the first data item is the same as that of the second data items, determining to migrate the data in the first data item to the second data items with the same type; if the name of a first data item is different from the name of each second data item, and the type of the first data item is different from the type of each second data item, adding the first data items in sequence after the last second data item of the second data structure. That is, a first priority of matching in terms of data item names, a second priority of matching in terms of data item types, and a third priority of matching in terms of data item orders. To better explain the data migration rule determining method, for example, if the source data object is table 1, and table 1 is a text intelligent recognition accuracy result record, the first data structure of the source data object is as follows: the noun class: activity name, street name, organization name, date, character name, program name, and column in that order: identifying the model name: f1 and F2 in sequence; as shown in table 1:
TABLE 1
If the target data object is table 2, and table 2 is the result record of intelligent character recognition accuracy, the second data structure of the target data object is as follows: the noun class: organization name, date, campaign name, date, and columns in that order: identifying the model name: sequentially A1; as shown in table 2:
| A1 | |
| organization name | 89.64％ |
| Date | 94.00％ |
| Activity name | 93.00％ |
| Address | 92.80％ |
TABLE 2
And if the name of the first data item in the table 1 is the same as that of the second data item in the table 2, determining to migrate the data under the first data item to the second data item with the same name, wherein the step of correspondingly transferring the row data of the 'activity name' of the first data item to the row of the 'activity name' of the second data item, the step of correspondingly transferring the row data of the 'organization name' of the first data item to the row of the 'organization name' of the second data item, and the step of correspondingly transferring the row data of the 'date' of the first data item to the row of the 'date' of the second data item. If the name of the first data item in table 1 is different from the names of the second data items in table 2, and the type of the first data item is the same as the type of the second data items in table 2, determining to migrate the data in the first data item to the second data item with the same type, including: the line data of the first data item 'street name' is correspondingly transferred to the line of the second data item 'address', the address and the street name are different in name but belong to the address type. If the name of the first data item in table 1 is different from the name of each second data item in table 2, and the type of the first data item in table 1 is different from the type of each second data item in table 2, the first data items are sequentially added after the last second data item in the second data structure. The method comprises the following steps: the 'person name' and 'program name' names are both different and of different types, the first data item is sequentially added after the last second data item 'address' of table 2. Obtaining a table 3 corresponding to the target data object after data migration, where the table 3 is also a record of the intelligent character recognition accuracy result, and the data structure of the target data object after data migration is as follows: the noun class: organization name, date, activity name, address, character name, program name, and columns in that order: identifying the model name: a1, F2 and F1 in sequence; as shown in table 2:
| A1 | F2 | F1 | |
| organization name | 89.64％ | 92.64％ | 95.21％ |
| Date | 94.00％ | 92.37％ | 93.57％ |
| Activity name | 93.00％ | 88.03％ | 92.00％ |
| Address | 92.80％ | 92.07％ | 93.80％ |
| Character name | 86.67％ | 95.21％ | |
| Program title | 89.64％ | 84.00％ |
TABLE 3
It should be noted here that the above example is only used for explaining the method for determining the data migration rule, and does not limit the determination of the data migration rule, and the data structure in this example is a data structure in a table, and also only explains the data structure, and does not limit the data structure.
An embodiment of the present application provides a method for obtaining a data structure, which respectively determines a first data structure of a source data object and a second data structure of a target data object, and includes: searching a configuration record according to the source data object and the target data object, and respectively acquiring first connection information of the first database and second connection information of the second database; acquiring the first data structure from the first database according to the first connection information; and acquiring the second data structure from the second database according to the second connection information. That is, the data migration system includes a configuration record, the configuration record stores connection information of each database, and the connection information may include information such as a server IP and a file path; the configuration record may be searched according to the source data object and the target data object, the first connection information and the second connection information may be obtained, and further, a connection may be established with the first database and the second database. The first database and the second database can be the same type of database, can also be different types of databases, and can only refer to the storage positions of the data; for example, the first database may be a mysql database, the second database may be a hive database, the first database and the second database may both be sftp databases, the first database may be an hdfs file, the second database may be a local file, and the first database and the second database may both be hdfs files. The specific form of the first database and the second database is not limited herein.
The embodiment of the application provides a method for receiving a data migration request, which receives the data migration request in an HTTP format through a preset API (application program interface). That is to say, a preset API interface may be encapsulated in the data migration system, so that the data migration system may receive a data migration request in the HTTP format, break through a barrier that only local operations are possible, and implement remote operations.
The embodiment of the present application provides a verification method for receiving a data migration request, before determining a first data structure of a source data object and a second data structure of a target data object respectively, the method further includes: and verifying the data migration request, and determining that the data migration request conforms to a verification rule. Here, the security of data migration can be improved by checking the request format, the password, or the like of the data migration request.
The embodiment of the present application provides another method for determining a data migration rule, where after determining a data migration rule according to the first data structure and the second data structure, the method further includes: displaying the data migration rule; and receiving the adjustment execution of the data migration rule by the user to obtain an updated data migration rule. That is, in the above example of the method for determining the data migration rule, after how to obtain the data migration rule in table 3 is obtained, the data migration rule may be displayed to the user, and the user performs corresponding adjustment to obtain the adjusted data migration rule.
An embodiment of the present application provides another data migration method, where after obtaining data of the source data object from the first database, before storing the data in the target data object of the second database according to the data migration rule, the method further includes: and converting each data in the source data object using the first data language into each data in the target data object using the second data language. That is, if the first database and the second database respectively apply the first data language and the second data language, in order to prevent the second database from being unable to identify operations for migrating data, etc., the first data language of the migrating data is converted into the second data language for storage, so as to ensure the availability of the migrating data in the second database.
Based on the above method flows, an embodiment of the present application provides a flow of a data migration method, as shown in fig. 3, including:
And 307, acquiring each data of the source data object in the first database according to the first connection information.
And 309, storing each data applying the second data language into a target data object of the second database according to the adjusted data migration rule.
It should be noted that, the above-mentioned steps of the flow are not exclusive, and for example, step 307 may be executed before step 306.
Based on the same concept, an embodiment of the present invention provides a data migration apparatus, and fig. 4 is a schematic diagram of the data migration apparatus provided in the embodiment of the present application, as shown in fig. 4, including:
a transceiver module 401, configured to receive a data migration request, where the data migration request includes a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database;
a determining module 402, configured to determine a first data structure of the source data object and a second data structure of the target data object, respectively;
the determining module 402 is further configured to determine a data migration rule according to the first data structure and the second data structure;
a migration module 403, configured to store each data of the source data object obtained from the first database into the target data object of the second database according to the data migration rule.
Optionally, the determining module 402 is specifically configured to determine a name and a type of each first data item in the first data structure, and determine a name and a type of each second data item in the second data structure; if the name of a first data item is the same as that of the second data item, determining to migrate the data under the first data item to the second data item with the same name; if the name of a first data item is different from the names of all second data items, and the type of the first data item is the same as that of the second data items, determining to migrate the data in the first data item to the second data items with the same type; if the name of a first data item is different from the name of each second data item, and the type of the first data item is different from the type of each second data item, adding the first data items in sequence after the last second data item of the second data structure.
Optionally, the determining module 402 is specifically configured to search configuration records according to the source data object and the target data object, and respectively obtain first connection information of the first database and second connection information of the second database; acquiring the first data structure from the first database according to the first connection information; and acquiring the second data structure from the second database according to the second connection information.
Optionally, the transceiver module 401 is specifically configured to receive a data migration request in an HTTP format through a preset API interface.
Optionally, the determining module 402 is further configured to verify the data migration request, and determine that the data migration request meets a verification rule.
Optionally, the determining module 402 is further configured to display the data migration rule;
and receiving the adjustment execution of the data migration rule by the user to obtain an updated data migration rule.
The migration module 403 is further configured to convert each data in the source data object using the first data language into each data in the target data object using the second data language.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method of data migration, the method comprising:
receiving a data migration request, wherein the data migration request comprises a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database;
determining a first data structure of the source data object and a second data structure of the target data object, respectively;
determining a data migration rule according to the first data structure and the second data structure;
and storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule.
2. The method of claim 1,
determining a data migration rule according to the first data structure and the second data structure, including:
determining the name and the type of each first data item in the first data structure, and determining the name and the type of each second data item in the second data structure;
if the name of a first data item is the same as that of the second data item, determining to migrate the data under the first data item to the second data item with the same name;
if the name of a first data item is different from the names of all second data items, and the type of the first data item is the same as that of the second data items, determining to migrate the data in the first data item to the second data items with the same type;
if the name of a first data item is different from the name of each second data item, and the type of the first data item is different from the type of each second data item, adding the first data items in sequence after the last second data item of the second data structure.
3. The method of claim 1,
determining a first data structure of the source data object and a second data structure of the target data object, respectively, comprising:
searching a configuration record according to the source data object and the target data object, and respectively acquiring first connection information of the first database and second connection information of the second database;
acquiring the first data structure from the first database according to the first connection information;
and acquiring the second data structure from the second database according to the second connection information.
4. The method of claim 1, wherein receiving a data migration request comprises:
and receiving a data migration request in an HTTP format through a preset API (application programming interface).
5. The method of claim 1, wherein determining the first data structure of the source data object and the second data structure of the target data object, respectively, is preceded by:
and verifying the data migration request, and determining that the data migration request conforms to a verification rule.
6. The method of claim 1,
after determining the data migration rule according to the first data structure and the second data structure, the method further includes:
displaying the data migration rule;
and receiving the adjustment execution of the data migration rule by the user to obtain an updated data migration rule.
7. The method of claim 4, wherein after obtaining the data of the source data object from the first database, before storing the data in the target data object of the second database according to the data migration rule, the method further comprises:
and converting each data in the source data object using the first data language into each data in the target data object using the second data language.
8. An apparatus for data migration, the apparatus comprising:
the data migration system comprises a receiving and sending module, a sending and receiving module and a processing module, wherein the receiving and sending module is used for receiving a data migration request which comprises a source data object and a target data object; the source data object is stored in a first database, and the target data object is stored in a second database;
a determining module for determining a first data structure of the source data object and a second data structure of the target data object, respectively;
the determining module is further configured to determine a data migration rule according to the first data structure and the second data structure;
and the migration module is used for storing each data of the source data object acquired from the first database into the target data object of the second database according to the data migration rule.
9. A computer-readable storage medium, characterized in that it stores a program which, when run on a computer, causes the computer to carry out the method of any one of claims 1 to 7.
10. A computer device, comprising:
a memory for storing a computer program;
a processor for calling a computer program stored in said memory to execute the method of any of claims 1 to 7 in accordance with the obtained program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011398952.5A CN112527765B (en) | 2020-12-01 | 2020-12-01 | Data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011398952.5A CN112527765B (en) | 2020-12-01 | 2020-12-01 | Data migration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112527765A true CN112527765A (en) | 2021-03-19 |
| CN112527765B CN112527765B (en) | 2024-05-24 |
Family
ID=74998095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011398952.5A Active CN112527765B (en) | 2020-12-01 | 2020-12-01 | Data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112527765B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050131970A1 (en) * | 2003-12-15 | 2005-06-16 | International Business Machines Corporation | Customizable data translation method and system |
| CN101754221A (en) * | 2008-12-19 | 2010-06-23 | 中国移动通信集团山东有限公司 | Data transmission method between heterogeneous systems and data transmission system |
| CN102521252A (en) * | 2011-11-17 | 2012-06-27 | 四川长虹电器股份有限公司 | Access method of remote data |
| CN109002474A (en) * | 2018-06-15 | 2018-12-14 | 北京信安世纪科技股份有限公司 | A kind of data migration method and device |
| US20200021556A1 (en) * | 2018-07-16 | 2020-01-16 | Amazon Technologies, Inc. | Address migration service |
| CN111008188A (en) * | 2019-10-29 | 2020-04-14 | 平安科技（深圳）有限公司 | Data migration method and device, computer equipment and storage medium |
| CN111352993A (en) * | 2019-03-20 | 2020-06-30 | 深圳市鸿合创新信息技术有限责任公司 | Cross-platform data synchronization method and device and electronic equipment |
| CN111538795A (en) * | 2020-04-27 | 2020-08-14 | 天津市融创软通科技股份有限公司 | Intelligent interaction system of relational database and object database |
| CN111585823A (en) * | 2020-05-20 | 2020-08-25 | 石高峰 | Communication network optimization method and device based on block chain and edge calculation |
-
2020
- 2020-12-01 CN CN202011398952.5A patent/CN112527765B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050131970A1 (en) * | 2003-12-15 | 2005-06-16 | International Business Machines Corporation | Customizable data translation method and system |
| CN101754221A (en) * | 2008-12-19 | 2010-06-23 | 中国移动通信集团山东有限公司 | Data transmission method between heterogeneous systems and data transmission system |
| CN102521252A (en) * | 2011-11-17 | 2012-06-27 | 四川长虹电器股份有限公司 | Access method of remote data |
| CN109002474A (en) * | 2018-06-15 | 2018-12-14 | 北京信安世纪科技股份有限公司 | A kind of data migration method and device |
| US20200021556A1 (en) * | 2018-07-16 | 2020-01-16 | Amazon Technologies, Inc. | Address migration service |
| CN111352993A (en) * | 2019-03-20 | 2020-06-30 | 深圳市鸿合创新信息技术有限责任公司 | Cross-platform data synchronization method and device and electronic equipment |
| CN111008188A (en) * | 2019-10-29 | 2020-04-14 | 平安科技（深圳）有限公司 | Data migration method and device, computer equipment and storage medium |
| CN111538795A (en) * | 2020-04-27 | 2020-08-14 | 天津市融创软通科技股份有限公司 | Intelligent interaction system of relational database and object database |
| CN111585823A (en) * | 2020-05-20 | 2020-08-25 | 石高峰 | Communication network optimization method and device based on block chain and edge calculation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112527765B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108282527B (en) | Generate the distributed system and method for Service Instance | |
| CN107391735A (en) | Business datum source tracing method, device, system and storage device based on block chain | |
| CN103139172B (en) | A kind of service implementation method and device | |
| CN108133007A (en) | A kind of method of data synchronization and system | |
| CN106557486A (en) | A kind of storage method and device of data | |
| CN106897342B (en) | Data verification method and equipment | |
| CN110543469B (en) | Database version management method and server | |
| CN111143470A (en) | Cross-platform database data synchronization comparison method and device | |
| CN106547646B (en) | Data backup and recovery method and data backup and recovery device | |
| WO2019047516A1 (en) | Method and device for information output | |
| US9313284B2 (en) | Smart posting with data analytics and semantic analysis to improve a message posted to a social media service | |
| CN111435367A (en) | Knowledge graph construction method, system, equipment and storage medium | |
| CN112015806A (en) | Method and device for storing data by block chain | |
| CN114372102A (en) | Data analysis method and device, storage medium and electronic equipment | |
| CN113408254A (en) | Page form information filling method, device, equipment and readable medium | |
| CN112559526A (en) | Data table export method and device, computer equipment and storage medium | |
| WO2018066152A1 (en) | Data integration device and data integration method | |
| CN117216114A (en) | Data stream association method, device, equipment and storage medium thereof | |
| CN112527765B (en) | Data migration method and device | |
| US8498963B2 (en) | Method and system for data synchronization | |
| CN109189864B (en) | Method, device and equipment for determining data synchronization delay | |
| CN106294700A (en) | The storage of a kind of daily record and read method and device | |
| CN109561126A (en) | A kind of method of data synchronization and device, storage medium, processor | |
| CN109543024B (en) | Text processing method and device | |
| US20180196886A1 (en) | Devices and methods for implementing dynamic collaborative workflow systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |