CN116303337A - Data migration method, device, equipment and computer storage medium - Google Patents
Data migration method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN116303337A CN116303337A CN202211557662.XA CN202211557662A CN116303337A CN 116303337 A CN116303337 A CN 116303337A CN 202211557662 A CN202211557662 A CN 202211557662A CN 116303337 A CN116303337 A CN 116303337A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- migration
- target
- initial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013508 migration Methods 0.000 title claims abstract description 259
- 230000005012 migration Effects 0.000 title claims abstract description 259
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000006243 chemical reaction Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 13
- 238000012795 verification Methods 0.000 claims description 11
- 230000008676 import Effects 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013496 data integrity verification Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses a data migration method, a device, equipment and a computer storage medium, wherein the data migration method comprises the steps of obtaining a data structure of a target database and a plurality of preset data structures, wherein the preset data structures are data structures of the database; generating a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, wherein the migration templates correspond to the preset data structures one by one, and the migration templates are used for indicating the corresponding relation between the preset data structures and the data structures of the target database; acquiring a data structure of an initial database; determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the plurality of migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template; and importing the data information in the initial database into the target database according to the target migration template.
Description
Technical Field
The present application belongs to the field of data migration, and in particular, relates to a method, an apparatus, a device, and a computer storage medium for data migration.
Background
With the rapid development of information technology, after a period of time, a service system is used, and a new service system is generally used to replace an old service system due to technology update or service change. However, the existing data migration platform requires that the old service system and the new service system are consistent in terms of requirements of data specifications, which results in that service systems inconsistent in data specifications cannot perform data migration.
Aiming at the technical problem that the service systems with inconsistent data specifications cannot perform data migration because the prior art does not support data migration among the service systems with inconsistent data specifications, no solution is obtained at present.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a computer storage medium for data migration, which can solve the technical problem that the service systems with inconsistent data specifications cannot perform data migration because the prior art does not support data migration between the service systems with inconsistent data specifications.
In one aspect, an embodiment of the present application provides a method for data migration, where the method includes: acquiring a data structure of a target database and a plurality of preset data structures, wherein the preset data structures are the data structures of the database; generating a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, wherein the migration templates correspond to the preset data structures one by one, and the migration templates are used for indicating the corresponding relation between the preset data structures and the data structures of the target database; acquiring a data structure of an initial database; determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the plurality of migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template; and importing the data information in the initial database into the target database according to the target migration template.
In another aspect, an embodiment of the present application provides an apparatus for data migration, where the apparatus includes: the first acquisition unit is used for acquiring a data structure of the target database and a plurality of preset data structures, wherein the preset data structures are data structures of the database; the generation unit is used for generating a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, wherein the migration templates correspond to the preset data structures one by one, and the migration templates are used for indicating the corresponding relation between the preset data structures and the data structures of the target database; the second acquisition unit is used for acquiring the data structure of the initial database; the determining unit is used for determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the plurality of migration templates, wherein the data structure of the initial database is the same as the preset data structure corresponding to the target migration template; and the importing unit is used for importing the data information in the initial database into the target database according to the target migration template.
In yet another aspect, an embodiment of the present application provides a device for data migration, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method as described above at data migration.
In yet another aspect, embodiments of the present application provide a computer storage medium having stored thereon computer program instructions that, when executed by a processor, perform a method of data migration as above.
According to the data migration method, device, equipment and computer storage medium, a migration template (migration specification) is customized for a target database according to different preset data structures; the method and the system realize that different initial databases can find corresponding migration templates (migration specifications) according to the data structures of the initial databases, and further migrate data information of the initial databases into a target database based on the corresponding migration templates (migration specifications), so that the technical problem that in the prior art, data migration cannot be performed on service systems with inconsistent data specifications because only service systems with consistent data specification requirements can be performed; and further realizing data migration among service systems of asynchronous data specifications.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described, and it is possible for a person skilled in the art to obtain other drawings according to these drawings without inventive effort.
FIG. 1 is a flow chart illustrating a method for data migration according to one embodiment of the present application;
FIG. 2 is a second flow chart of a method for data migration according to one embodiment of the present application;
FIG. 3 is a schematic diagram of an apparatus for data migration according to one embodiment of the present application;
FIG. 4 is a schematic diagram II of an apparatus for data migration according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data migration device according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application are described in detail below to make the objects, technical solutions and advantages of the present application more apparent, and to further describe the present application in conjunction with the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
In order to solve the problems in the prior art, embodiments of the present application provide a method, an apparatus, a device, and a computer storage medium for data migration. The following first describes a method for data migration provided in an embodiment of the present application.
FIG. 1 is a flow chart illustrating a method for data migration according to one embodiment of the present application.
As shown in fig. 1, the data migration method includes the following steps:
s101, acquiring a data structure of a target database and a plurality of preset data structures, wherein the preset data structures are the data structures of the database.
The target database may be a database of any one of the micro-service application systems in the new business system in the "micro-service + containerized" architecture mode.
S102, generating a plurality of migration templates according to a data structure of a target database and a plurality of preset data structures, wherein the migration templates correspond to the preset data structures one by one, and the migration templates are used for indicating the corresponding relation between the preset data structures and the data structures of the target database;
the migration template for indicating a correspondence between the preset data structure and the data structure of the target database may refer to: the mapping relation and the format conversion relation between each data element in the data structure and each data element in the data structure of the target database are preset.
S103, acquiring a data structure of an initial database;
s104, determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the plurality of migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template;
s105, importing the data information in the initial database into the target database according to the target migration template.
When there are a plurality of initial databases, in S104, a corresponding target migration template is sequentially matched for each initial database, and in S105, data information in the initial databases is sequentially imported into the target databases according to the target migration templates matched with the initial databases.
According to the embodiment of the application, the migration template (migration specification) is customized for the target database according to different preset data structures; and further, different initial databases can find corresponding migration templates (migration specifications) according to the data structures of the initial databases, and then data information of the initial databases is migrated into the target databases based on the corresponding migration templates (migration specifications), so that the technical problem that in the prior art, data migration cannot be performed on service systems with inconsistent data specifications because only service systems with consistent data specifications are subjected to data migration is solved.
When the old service system is updated to the new service system and the new service system adopts a micro service and containerization architecture mode, the data migration method provided by the embodiment of the application can be sequentially executed for the database of each micro service application system in the new service system, so that the data information in the old service system is migrated to the new service system.
Also to be described is: when the data migration method provided by the embodiment of the application is executed for the database of each micro service application system in the new service system in turn, the migration template generated by the database of each micro service application system further includes: data dependencies between databases of the respective micro service application systems.
Also to be described is: when an old business system is upgraded to a new business system and the old business system involves multiple databases, the method can be performed for each old business system database in turn: acquiring a data structure of an initial database; determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the plurality of migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template; and importing the data information in the initial database into the target database according to the target migration template so as to migrate the data information in the databases of the plurality of old service systems into the new service system. The initial database may be one or a plurality of databases.
When the data structure of the target database is updated and adjusted, dynamic adjustment of the migration specification can be achieved by synchronously adjusting the migration template of the target database, so that the data migration method provided by the embodiment of the application has higher flexibility.
In one embodiment, importing the data information in the initial database into the target database according to the target migration template may be achieved by:
s1051, constructing a database table corresponding to the target database according to the data structure of the target database.
S1052, importing the data information in the initial database into the database table according to the target migration template corresponding to the initial database.
S1053, importing the data information in the database table into the corresponding target database.
According to the method and the device for data integration, the database table for the target database pair is constructed between the initial database and the target database, so that data information to be migrated to the target database in the initial database can be integrated in the database table outside the target database, and data verification/data supplement/data adjustment can be performed on the migration database before the data information in the initial database is migrated to the target database, so that migration quality is improved.
In one embodiment, according to the data structure of the target database, constructing the database table corresponding to the target database may be implemented as follows:
s1051a, analyzing a data structure of a target database to obtain a data element set of the target database, wherein the data element represents one type of data in the target database;
s1051a, generating a database table corresponding to the target database according to the data element set of the target database.
The database table in the embodiment of the application is mainly constructed based on the data element set of the target database, and at this time, after the data information migrated from the initial database is filled in the database table, the target database can directly obtain the specific data information of each data element based on the database table, so that the technical effect of efficient migration is realized.
In one embodiment, according to the target migration template corresponding to each initial database, the data information in the plurality of initial databases is sequentially imported into the database table by the following steps:
s1052a, determining attribute information of each field in the initial database according to the data structure of the initial database;
s1052b, based on the attribute information of each field in the initial database and the attribute information of each field in the target migration template, establishes a mapping relationship and a conversion rule between the field in the target migration data template and the field in the initial database.
It should be noted that: the conversion rule may be a format conversion rule or a content conversion rule, for example: after the contents of the plurality of fields in the initial database are converted, the plurality of fields correspond to a certain field in the target database, which is not particularly limited in this application.
S1052c, importing the data information in the initial database into the database table based on the mapping relation and the conversion rule between the fields in the target migration data template and the fields in the initial database.
According to the method and the device, the mapping relation and the conversion rule between the fields in the target migration data template and the fields in the initial database are established, and the data information in the initial database is imported into the database table according to the mapping relation and the conversion rule between the fields in the target migration data template and the fields in the initial database, so that data migration among databases with different data structures and different data specifications is achieved.
In one example, importing data information in an initial database into a database table based on correspondence between fields in a target migration data template and fields in the initial database and conversion rules may be accomplished by: determining a target field and a conversion rule of the target field, which have a corresponding relation with the field in the target migration data template, in the initial database based on the corresponding relation between the field in the target migration data template and the field in the initial database; performing data conversion on the target field based on the conversion rule of the target field; and importing the target field after data conversion into a database table.
In one example, importing data information in an initial database into a database table based on correspondence between fields in a target migration data template and fields in the initial database and conversion rules may be accomplished by: generating an migration script based on the corresponding relation between the fields in the target migration data template and the fields in the initial database and the conversion rule, importing the migration script into the initial database, and executing the migration script in the initial database, wherein the migration script is used for determining a target field with the corresponding relation with the fields in the target migration data template in the initial database, performing data conversion on the target field, and storing the target field after data conversion in a migration platform in the form of a custom format file, wherein a database table of the target database is built in the migration platform; at this time, the migration platform fills in the data information in the custom format file into the database table.
It should be noted that: the migration script exports the target field after data conversion in a custom format file, so that a worker can supplement the custom format file with data, and the part of the content which cannot be supplemented with data through the system rule can be supplemented by the worker.
In one embodiment, before importing the data information in the database table into the target database, the method further comprises: carrying out service logic check and data integrity check on the data information in the database table; if the database table passes the verification, continuing to execute the data information in the database table to be imported into the target database.
In the embodiment of the application, before the data information in the database table is imported to the target database, service logic verification is performed on the data information in the database table, so that the data integrity and service logic of the data information imported into the template database are ensured, and the migration quality of data migration is ensured.
In one example, if the database table fails to pass the verification, the data information in the database is adjusted according to the verification result, or the migration template is adjusted according to the verification result, and the data information is reintroduced into the database table according to the adjusted migration template.
In one embodiment, importing data information in a database table into a target database includes:
s1053a, obtaining a data structure of a database table;
s1053b, constructing an migration script and a rollback script according to the data structure of the target database and the data structure of the database table;
s1053c, importing the migration script and the rollback script into a target database;
s1053d, executing the migration script in the target database to import the data information in the database table into the target database.
S1053e, executing the rollback script to delete the data information imported by the migration script when the migration script is abnormal in execution.
In the embodiment of the application, when the data information in the database table is migrated to the target database, not only the migration script but also the corresponding rollback script are generated. At this time, if an accident occurs in the process of migrating the data information in the database table to the target database, the rollback operation of data migration can be performed, so as to ensure the data stability of the target database.
In one embodiment, after importing the data information in the database table into the target database, the method further comprises: judging whether the data information in the database table is successfully imported into the target database; after the data information in the database table is successfully imported into the target database, deleting the migration execution script and the migration rollback script in the target database.
In one embodiment, after importing the data information in the database table into the target database, the method further comprises: judging whether the data information in the database table is successfully imported into the target database; and deleting the database table corresponding to the target database after the data information in the database table is successfully imported into the target database.
In summary, according to the data migration method provided by the embodiment of the application, the physical isolation between the target database and the initial database is realized by setting the database table of the target database, so that the data security of the target database is ensured; by setting a migration template for the target database, the personalized requirement of the target database on the migration data on the data specification is realized; by setting the migration script and the rollback script, the technical effect of smoothly rollback the migration data under the condition of abnormal migration is realized.
Taking an example of upgrading an old service system to a new service system, and adopting a "micro service+containerization" architecture mode for the new service system, the method for data migration provided by the foregoing embodiment of the present application is described:
fig. 2 is a schematic flow chart of a data migration method according to another embodiment of the present application.
As shown in fig. 2, the data migration method includes the steps of:
s201: and creating a data migration template according to the structure types of different micro-service databases of the target system.
Because the target system is developed based on the micro-service architecture, each micro-service database in the target system is relatively independent, and further different migration templates are required to be generated for each micro-service database; meanwhile, the migration template covers the data dependency relationship among the micro-service databases so as to be suitable for data migration of all the micro-service databases in the target system.
In addition, the migration template can cover the normalization requirement of the target system on the migration data when different source systems (source objects of data migration) migrate the data to the target system. Specifically, in the actual data migration process, each source system may select a migration template according to the database structure type of the source system.
S202: and creating each micro-service database table in the middle of the migration platform according to the data structure of each micro-service database of the target system. Specifically, each micro-service database structure of the target system is analyzed to obtain a database structure element set, and a micro-service database table is generated according to the data structure element set to obtain a migration platform intermediate library.
S203: and carrying out mapping relation configuration on the migration template and the database structure of the source system. Specifically, a conversion rule is generated by creating a correspondence between fields in the migration template and fields in a database of the source system, so as to form data migration configuration, and a migration configuration task is generated.
S204: the database of the source system is connected, and the effective data with mapping relation with each micro-service database is derived from the database of the source system. Specifically, a query script generated by a migration configuration task is executed in a source database, effective data in the source database is obtained by executing the query script, the effective data is subjected to data conversion, a custom format file is generated, and the custom format file is stored in a database table in a data migration platform.
S205: and carrying out service logic and data integrity check on the data in the database table. Specifically, after the effective data in each source database are imported into the database table of the migration platform, checking the data in the database table, and if the data pass the check, obtaining the version number of the data migration; if the verification fails, the migration data/migration template needs to be changed according to the error prompt information.
S206: and generating an migration script, wherein the migration script is used for importing the data in the database table which is successfully checked into the corresponding micro-service database, and taking the version number of the data migration obtained after the database table is successfully checked as the version number of the migration script.
S207: based on the version number of the data migration, determining a corresponding migration script for the micro-service database, and executing the migration script in the database table of the micro-service database to complete the data migration.
S208: and deleting the database table in the migration platform after the migration data is successfully imported into the micro-service database of the target system.
Fig. 3 is a schematic structural diagram of a data migration apparatus according to another embodiment of the present application.
As shown in fig. 3, the data migration apparatus includes:
the migration template making unit 310 is configured to create a data migration template according to the structure types of different micro service databases of the target system.
The database table creating unit 320 is configured to create each micro service database table in the migration platform according to the data structure of each micro service database of the target system.
The migration task configuration unit 330 is configured to perform mapping relationship configuration on the migration template and the database structure of the source system. Specifically, a conversion rule is generated by creating a correspondence between fields in the migration template and fields in a database of the source system, so as to form data migration configuration, and a migration configuration task is generated.
The migration data collection unit 340 is configured to connect databases of the source system, and derive valid data having a mapping relationship with each micro service database from the databases of the source system. Specifically, a query script generated by a migration configuration task is executed in a source database, effective data in the source database is obtained by executing the query script, the effective data is subjected to data conversion, a custom format file is generated, and the custom format file is stored in a database table in a data migration platform.
And the migration data verification unit 350 is configured to perform service logic and data integrity verification on the data in the database table. Specifically, after the effective data in each source database are imported into the database table of the migration platform, checking the data in the database table, and if the data pass the check, obtaining the version number of the data migration; if the verification fails, the migration data/migration template needs to be changed according to the error prompt information.
The migration script generating unit 360 is configured to generate an migration script, where the migration script is configured to import data in the database table that is successfully checked into the corresponding micro-service database, and use the version number of the data migration obtained after the database table is successfully checked as the version number of the migration script.
The migration data importing unit 370 is configured to determine a corresponding migration script for the micro service database based on the version number of the data migration, and execute the migration script in the database table of the micro service database to complete the data migration.
A migration data deleting unit 380 for deleting the database table in the migration platform after the migration data is successfully imported into the micro service database of the target system
Fig. 4 is a schematic structural diagram of an apparatus for data migration according to another embodiment of the present application. As shown in fig. 4, the apparatus may include:
a first obtaining unit 410, configured to obtain a data structure of a target database and a plurality of preset data structures, where the preset data structures are data structures of the database;
the generating unit 420 is configured to generate a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, where the migration templates correspond to the preset data structures one by one, and the migration templates are used to indicate a correspondence between the preset data structures and the data structures of the target database;
a second obtaining unit 430, configured to obtain a data structure of the initial database;
the determining unit 440 is configured to determine a target migration template that is matched with the initial database according to a data structure of the initial database and preset data structures corresponding to the plurality of migration templates, where the data structure of the initial database is the same as the preset data structure corresponding to the target migration template;
and the importing unit 450 is used for importing the data information in the initial database into the target database according to the target migration template.
According to the embodiment of the application, the migration template (migration specification) is customized for the target database according to different preset data structures; and further, different initial databases can find corresponding migration templates (migration specifications) according to the data structures of the initial databases, and then data information of the initial databases is migrated into a target database based on the corresponding migration templates (migration specifications), so that the technical problem that in the prior art, data migration can not be performed on old service systems and new service systems which are inconsistent in data specifications because only old service systems and new service systems which are consistent in data specification are subjected to data migration is solved.
In one embodiment, the importing unit 450 includes: the construction subunit is used for constructing a database table corresponding to the target database according to the data structure of the target database; the first importing subunit is used for importing the data information in the initial database into the database table according to the target migration template corresponding to the initial database; and the second importing subunit is used for importing the data information in the database table into the target database.
In one embodiment, a building subunit comprises: the analysis module is used for analyzing the data structure of the target database to obtain a data element set of the target database, wherein the data elements represent one type of data in the target database; and the generating module is used for generating a database table corresponding to the target database according to the data element set of the target database.
In one embodiment, a first import subunit includes: the determining module is used for determining attribute information of each field in the initial database according to the data structure of the initial database; the first construction module is used for establishing a mapping relation and a conversion rule between the fields in the target migration data template and the fields in the initial database based on the attribute information of each field in the initial database and the attribute information of each field in the target migration template; the first importing module is used for importing the data information in the initial database into the database table based on the mapping relation and the conversion rule between the fields in the target migration data template and the fields in the initial database.
In one embodiment, the apparatus further comprises: the verification subunit is used for carrying out service logic verification and data integrity verification on the data information in the database table before the data information in the database table is imported into the target database; and the execution subunit is used for continuously executing the data information in the database table to be imported into the target database if the database table passes the verification.
In one embodiment, the second import subunit comprises: the acquisition module is used for acquiring the data structure of the database table; the second construction module is used for constructing an migration script and a rollback script according to the data structure of the target database and the data structure of the database table; the second import module is used for importing the migration script and the rollback script into the target database; the first execution module is used for executing the migration script in the target database so as to import the data information in the database table into the target database; and the second execution module is used for executing the rollback script under the condition that the execution of the migration script is abnormal so as to delete the data information imported by the migration script.
In one embodiment, the apparatus further comprises: the first judging module is used for judging whether the data information in the database table is successfully imported into the target database after the data information in the database table is imported into the target database; and the first deleting unit is used for deleting the migration execution script and the migration rollback script in the target database after the data information in the database table is successfully imported into the target database.
In one embodiment, the apparatus further comprises: the second judging module is used for judging whether the data information in the database table is successfully imported into the target database after the data information in the database table is imported into the target database; and the second deleting unit is used for deleting the database table corresponding to the target database after the data information in the database table is successfully imported into the target database.
Fig. 5 shows a schematic hardware structure of a device for data migration according to an embodiment of the present application.
The device at which data is migrated may include a processor 501 and memory 502 storing computer program instructions.
In particular, the processor 501 may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of embodiments of the present application.
The memory may include Read Only Memory (ROM), random Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors) it is operable to perform the operations described with reference to a method according to an aspect of the present application.
The processor 501 implements the apparatus method of any one of the above embodiments by reading and executing computer program instructions stored in the memory 502.
In one example, the device for data migration may also include a communication interface 503 and a bus 510. As shown in fig. 5, the processor 501, the memory 502, and the communication interface 503 are connected to each other by a bus 510 and perform communication with each other.
The communication interface 503 is mainly used to implement communication between each module, apparatus, unit and/or device in the embodiments of the present application.
In addition, in conjunction with the data migration apparatus in the foregoing embodiments, embodiments of the present application may provide a computer storage medium to be implemented. The computer storage medium has stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement a method of data migration in any of the above embodiments.
It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be different from the order in the embodiments, or several steps may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, which are intended to be included in the scope of the present application.
Claims (11)
1. A method of data migration, the method comprising:
acquiring a data structure of a target database and a plurality of preset data structures, wherein the preset data structures are the data structures of the database;
generating a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, wherein the migration templates correspond to the preset data structures one by one, and the migration templates are used for indicating the corresponding relation between the preset data structures and the data structures of the target database;
acquiring a data structure of an initial database;
determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template;
and importing the data information in the initial database into a target database according to the target migration template.
2. The method of claim 1, wherein importing the data information in the initial database into a target database according to the target migration template comprises:
constructing a database table corresponding to the target database according to the data structure of the target database;
according to the target migration template corresponding to the initial database, importing the data information in the initial database into the database table;
and importing the data information in the database table into the target database.
3. The method of claim 2, wherein constructing a database table corresponding to the target database according to the data structure of the target database comprises:
analyzing the data structure of the target database to obtain a data element set of the target database, wherein the data element represents one type of data in the target database;
and generating a database table corresponding to the target database according to the data element set of the target database.
4. The method of claim 2, wherein importing the data information in the initial database into the database table according to the target migration template corresponding to the initial database comprises:
determining attribute information of each field in the initial database according to the data structure of the initial database;
establishing a mapping relation and a conversion rule between the fields in the target migration data template and the fields in the initial database based on the attribute information of each field in the initial database and the attribute information of each field in the target migration template;
and importing the data information in the initial database into the database table based on the mapping relation and the conversion rule between the fields in the target migration data template and the fields in the initial database.
5. The method of claim 2, wherein prior to importing the data information in the database table into the target database, the method further comprises:
carrying out service logic check and data integrity check on the data information in the database table;
if the database table passes the verification, continuing to execute the step of importing the data information in the database table into the target database.
6. The method of claim 2, wherein importing data information in the database table into the target database comprises:
acquiring a data structure of the database table;
constructing an migration script and a rollback script according to the data structure of the target database and the data structure of the database table;
importing the migration script and the rollback script into the target database;
executing an migration script in the target database to import the data information in the database table into the target database;
and executing the rollback script under the condition that the execution of the migration script is abnormal so as to delete the data information imported by the migration script.
7. The method of claim 6, wherein after importing the data information in the database table into the target database, the method further comprises:
judging whether the data information in the database table is successfully imported into the target database;
and deleting the migration execution script and the migration rollback script in the target database after the data information in the database table is successfully imported into the target database.
8. The method of claim 2, wherein after importing the data information in the database table into the target database, the method further comprises:
judging whether the data information in the database table is successfully imported into the target database;
and deleting the database table corresponding to the target database after the data information in the database table is successfully imported into the target database.
9. An apparatus for data migration, the apparatus comprising:
the first acquisition unit is used for acquiring a data structure of a target database and a plurality of preset data structures, wherein the preset data structures are data structures of the database;
the generation unit is used for generating a plurality of migration templates according to the data structure of the target database and a plurality of preset data structures, wherein the migration templates are in one-to-one correspondence with the preset data structures, and the migration templates are used for indicating the corresponding relations between the preset data structures and the data structures of the target database;
the second acquisition unit is used for acquiring the data structure of the initial database;
the determining unit is used for determining a target migration template matched with the initial database according to the data structure of the initial database and preset data structures corresponding to the migration templates, wherein the data structure of the initial database is identical to the preset data structure corresponding to the target migration template;
and the importing unit is used for importing the data information in the initial database into the target database according to the target migration template.
10. An apparatus for data migration, the apparatus comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the method at data migration as claimed in any one of claims 1-8.
11. A computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method of data migration according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557662.XA CN116303337A (en) | 2022-12-06 | 2022-12-06 | Data migration method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557662.XA CN116303337A (en) | 2022-12-06 | 2022-12-06 | Data migration method, device, equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116303337A true CN116303337A (en) | 2023-06-23 |
Family
ID=86827518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211557662.XA Pending CN116303337A (en) | 2022-12-06 | 2022-12-06 | Data migration method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116303337A (en) |
-
2022
- 2022-12-06 CN CN202211557662.XA patent/CN116303337A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109344154B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN115952760A (en) | Method, device and equipment for simulating digital-analog circuit and computer storage medium | |
| CN112541688B (en) | Service data verification method and device, electronic equipment and computer storage medium | |
| CN116303337A (en) | Data migration method, device, equipment and computer storage medium | |
| CN114510902B (en) | Simulation result verification method, device, equipment and computer storage medium | |
| CN114372072A (en) | Data storage method, device, equipment, computer storage medium and program product | |
| CN115186741A (en) | Method, device and equipment for verifying POI fusion data | |
| CN115809228A (en) | Data comparison method and device, storage medium and electronic equipment | |
| CN114677211A (en) | Credit parameter updating method, apparatus, electronic device, medium, and program product | |
| CN113886278B (en) | Method and device for automatically exporting and verifying requirement attribute | |
| CN114661604A (en) | Data generation method, device, equipment and computer storage medium | |
| CN117891500A (en) | Method, device, equipment, storage medium and program product for generating interface document | |
| CN115391447A (en) | Data export method, device, equipment, medium and product | |
| CN114168564A (en) | Data processing method, device, equipment and computer storage medium | |
| CN108763363B (en) | Method and device for checking record to be written | |
| CN117312149A (en) | Method, apparatus, device, storage medium and program product for generating dummy data | |
| CN116775673A (en) | Data integration method of multiple data sources and device, equipment, medium and product thereof | |
| WO2017083149A1 (en) | Systems and methods for inferring landmark delimiters for log analysis | |
| CN114879947A (en) | Method, device, equipment and medium for field configuration | |
| CN114116772A (en) | Data access method and device, electronic equipment and storage medium | |
| CN116701403A (en) | Database management method, device, electronic equipment, program unit and storage medium | |
| CN114637795A (en) | Report processing method, device, equipment, medium and product | |
| CN117369911A (en) | Component display method, device, equipment, medium and product | |
| CN116186063A (en) | Data query method, device, equipment and computer storage medium | |
| CN114090014A (en) | Program splitting method, device, equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |