CN117520309A - Cross-database type data migration method and device - Google Patents
Cross-database type data migration method and device Download PDFInfo
- Publication number
- CN117520309A CN117520309A CN202311580452.7A CN202311580452A CN117520309A CN 117520309 A CN117520309 A CN 117520309A CN 202311580452 A CN202311580452 A CN 202311580452A CN 117520309 A CN117520309 A CN 117520309A
- Authority
- CN
- China
- Prior art keywords
- data
- identification information
- database
- source
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000005012 migration Effects 0.000 title claims abstract description 48
- 238000013508 migration Methods 0.000 title claims abstract description 48
- 230000011218 segmentation Effects 0.000 claims description 35
- 230000015654 memory Effects 0.000 claims description 26
- 238000009795 derivation Methods 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 12
- 230000008676 import Effects 0.000 claims description 9
- 238000000638 solvent extraction Methods 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 24
- 238000006243 chemical reaction Methods 0.000 abstract description 7
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data migration and discloses a data migration method and device of a cross-database type. In the process, the source data in the source database is segmented by sentences through adding identifiers to the source data, so that segmented source data is formed, and due to the configured export rule, the segmented source data is more flexible in the export process, and the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion are avoided, so that in the process of importing the intermediate file, the source data is segmented, and when the intermediate file is analyzed, target data can be generated according to the type of the target database, and the data migration across the database is completed on the basis of ensuring the data integrity and accuracy of the source data.
Description
Technical Field
The invention relates to the technical field of data migration, in particular to a method and a device for data migration across database types.
Background
In modern information systems, data migration for localization of information is a very important task. However, because of the differences in syntax, data types, and functionality between different database management systems, data migration across database types often faces difficulties and challenges.
In the related art, the support of certain data types by domestic databases and boss database manufacturers, or the representation of data types, and the specific functions and characteristics of databases such as storage processes, triggers, indexes, etc. are different due to the fact that different database management systems may have different support of data types and specific functions and characteristics. When the data is migrated from the source database to the target database, the data integrity and accuracy of the source data are lost without corresponding adjustment, so that data migration across the data cannot be completed.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for data migration across database types, so as to solve the technical problem of data migration failure caused by differences between different databases in data migration.
In a first aspect, the present invention provides a method for migrating data across database types, the method comprising: adding an identifier to source data of a source database to form segmented source data; configuring export rules; exporting the segmentation source data based on the export rule to generate an intermediate file; and importing the intermediate file into a target database, and analyzing the intermediate file to form target data.
With reference to the first aspect, in one possible implementation manner of the first aspect, deriving the segmentation source data based on the derivation rule, and generating the intermediate file includes: deriving segmentation source data based on the derivation rule; the type of the split source data is mapped to an intermediate data type, and an intermediate file is generated based on the intermediate data type.
With reference to the first aspect, in one possible implementation manner of the first aspect, deriving the rule includes: partial derivation, based on a derivation rule, of deriving segmentation source data, comprising: determining first data identification information based on the partial derivation; determining second identification information of the segmentation source data based on the identifier; the first segmentation source data is derived based on the first identification information and the second identification information.
With reference to the first aspect, in one possible implementation manner of the first aspect, deriving the rule includes: setting a protection column, deriving the segmentation source data based on the derivation rule, comprising: determining third data identification information based on the set protection column; determining second identification information of the split data based on the identifier; the second partitioned data source is derived based on the third data identification information and the second identification information.
With reference to the first aspect, in one possible implementation manner of the first aspect, deriving the rule includes: father-son association matching, deriving segmentation source data based on a derivation rule, comprising: determining fourth data identification information based on the parent-child association matching; traversing the segmentation source data based on the fourth data identification information, and determining fifth data identification information corresponding to the fourth data identification information; determining second identification information of the split data based on the identifier; and deriving a third split data source based on the fourth data identification information, the fifth data identification information and the second identification information.
With reference to the first aspect, in one possible implementation manner of the first aspect, importing the intermediate file into the target database, and forming the target data by parsing the intermediate file includes: analyzing the intermediate file, and determining the identifier contained in the intermediate file and the identification information corresponding to the export rule; acquiring a database type of a target database; based on the identifier and the identification information, the target data is formed in a newly added sentence corresponding to the database type.
With reference to the first aspect, in one possible implementation manner of the first aspect, forming, based on the identifier and the identification information, the target data in a new sentence corresponding to the database type includes: determining import data based on the identifier and the identification information; mapping the type of the imported data into an imported data type matched with the database type; and forming target data through the new sentence based on the imported data and the imported data type.
In a second aspect, the present invention provides a data migration apparatus across database types, the apparatus comprising: the segmentation module is used for adding identifiers to the source data of the source database to form segmented source data; the acquisition module is used for acquiring the configured export rule; the export module is used for exporting the segmentation source data based on export rules to generate an intermediate file; and the importing module is used for importing the intermediate file into the target database and forming target data by analyzing the intermediate file.
In a third aspect, the present invention provides a computer device comprising: the system comprises a memory and a processor, wherein the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions so as to execute the data migration method of the first aspect or any implementation mode corresponding to the first aspect.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon computer instructions for causing a computer to perform the above-described method of data migration across database types of the first aspect or any of its corresponding embodiments.
The technical scheme of the invention has the following advantages:
the method divides the source data in a form by adding the identifier to the source data and generates the intermediate file based on the export rule, so that the target data is formed by analyzing the intermediate file to realize data migration. In the process, the source data in the source database is divided by adding identifiers to the source data, so that the divided source data is formed, and due to the configured export rule, the divided source data is more flexible in the export process, and the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion are avoided, so that in the process of importing the intermediate file, the source data is divided, and when the intermediate file is analyzed, the target data can be generated according to the type of the target database, and the data migration across the database is completed on the basis of ensuring the data integrity and accuracy of the source data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic illustration of an application of a cross-data migration system provided in accordance with an embodiment of the present invention;
FIG. 2 is a flow chart of a method for migrating data across database types according to an embodiment of the present invention;
FIG. 3 is a block diagram of yet another cross-database type data migration apparatus provided in accordance with an embodiment of the present invention;
fig. 4 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The present embodiment provides a cross-data migration system, as shown in fig. 1, where the system includes: a source database 101, a target database 102, an intermediate file 103. By converting the source data in the source database 101 into the intermediate file 103, the cross-data migration system is caused to convert the intermediate file 103 into the target data of the target database 102 according to the database type of the target data 102. It should be understood that the source database 101, the target database 102 and the intermediate file 103 may be disposed on the same device, or may be disposed in a distributed manner, partially on different devices. The intermediate file 103 may be transferred to the target database 102 by network transfer, manual copying to a specified conversion directory, or other means, which is not specifically limited in this embodiment.
In accordance with an embodiment of the present invention, there is provided an embodiment of a data migration method across database types, it being noted that the steps shown in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
The embodiment provides a data migration method of a cross-database type, as shown in fig. 2, including the following steps:
s201, adding an identifier to source data of a source database to form segmented source data.
Specifically, adding an identifier to source data of a source database to form split source data refers to adding an identifier to each row or each column of source data in the source database, so that table data is split by the identifier to form a piece of table data, that is, the split source data is formed by adding the identifier, wherein the identifier may be a special character, a specific character string or other characters, which is not particularly limited in this embodiment, so long as the identifier may be used to split the source data in the source database.
S202, acquiring configured export rules.
Specifically, the export rule refers to a rule set in a process of generating an intermediate file from source data in a source database, and the export rule includes: one or more of partial export, setting protection columns, parent-child association matches.
And S203, exporting the segmentation source data based on the export rule to generate an intermediate file.
Specifically, exporting the segmentation source data based on the export rule, and generating the intermediate file refers to exporting the segmentation source data based on the export rule, mapping the type of the segmentation source data to an intermediate data type, and generating the intermediate file.
S204, importing the intermediate file into a target database, and analyzing the intermediate file to form target data.
Specifically, importing the intermediate file into a target database, analyzing the intermediate file to form target data, namely analyzing the intermediate file, determining identifiers and identification information contained in the intermediate file, and forming the target data according to the type of the target database.
According to the data migration method across database types, the identifier is added to the source data, the source data is divided in a form, and the intermediate file is generated based on the export rule, so that the target data is formed by analyzing the intermediate file, and data migration is achieved. In the process, the source data in the source database is divided by adding identifiers to the source data, so that the divided source data is formed, and due to the configured export rule, the divided source data is more flexible in the export process, and the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion are avoided, so that in the process of importing the intermediate file, the source data is divided, and when the intermediate file is analyzed, the target data can be generated according to the type of the target database, and the data migration across the database is completed on the basis of ensuring the data integrity and accuracy of the source data.
In an alternative embodiment, the exporting the split source data based on the export rule to generate the intermediate file includes:
(1) Based on the export rule, the segmentation source data is exported.
In an alternative embodiment, deriving the rule includes: partial derivation, based on a derivation rule, of deriving segmentation source data, comprising:
first data identification information is determined based on the partial derivation.
In particular, partial export refers to exporting only partial data in the source database, i.e., data migration of only partial data. Determining the first data identification information based on the partial derivation refers to determining data identification information of derived source data, wherein the first data identification information refers to identification information of source data corresponding to the derived data, such as a data row id or a data column id of the source data.
Second identification information of the segmentation source data is determined based on the identifier.
Specifically, based on the identifier, it is determined that the second identification information of the split source data is identification information of a corresponding data row or data column, i.e., the second identification information of the split source data, such as a data row id or a data column id, exists in each row or no column of the split source data because the identifier is used for performing the sentence split of the source data row by row or column by column.
The first segmentation source data is derived based on the first identification information and the second identification information.
Specifically, deriving the first divided source data based on the first identification information and the second identification information refers to which source data the data derived by the first identification information and the second identification information determining section are specifically, and on the basis of which the divided source data having the same identification information is derived, that is, the first divided source data is derived.
By implementing the embodiment, the split source data is screened by determining the first identification information corresponding to partial export in the export rule and the second identification information corresponding to the split source data, so that only partial source data is exported, and a data basis is provided for realizing data migration of the partial source data. In the process, the source data is segmented by adding identifiers to the source data and dividing the source data in the source database by different sentences, so that the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion can be effectively avoided, and the data migration of part of the source data is completed on the basis of ensuring the data integrity and accuracy of the source data.
In an alternative embodiment, deriving the rule includes: setting a protection column, deriving the segmentation source data based on the derivation rule, comprising:
third data identification information is determined based on setting the protection column.
Specifically, the protection column refers to protecting a certain or some column data in the source database, so that when the corresponding source data is migrated from the source database to the target database, if the same data identifier exists in the target database as that in the source database, the protection column is not covered, and the corresponding protection column is independently stored. Determining the third data identification information based on the setting of the protection column refers to determining the data identification information of the protection column, wherein the second data identification information refers to identification information corresponding to the protection column, such as a data column id or a special character, a specific character string or other character contained in the data column.
Second identification information of the split data is determined based on the identifier.
Specifically, referring to the description about the second identification information in the above embodiment, the description is omitted here.
The second partitioned data source is derived based on the third data identification information and the second identification information.
Specifically, deriving the second split data source based on the third data identification information and the second identification information refers to determining which source data belongs to the protection column through the second identification information and the third identification information, and deriving the split source data corresponding to the protection column based on the determination, that is, deriving the second split source data.
By implementing the embodiment, the third identification information corresponding to the protection column and the second identification information corresponding to the split source data are set in the export rule, so that the source data corresponding to the protection column are protected, the source data corresponding to the protection column cannot be lost in data migration, and a data basis is provided for realizing data migration of the source data corresponding to the protection column. In the process, the source data is segmented by adding identifiers to the source data and dividing the source data in the source database by different sentences, so that the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion can be effectively avoided, and the data migration of part of the source data is completed on the basis of ensuring the data integrity and accuracy of the source data.
In an alternative embodiment, deriving the rule includes: father-son association matching, deriving segmentation source data based on a derivation rule, comprising:
fourth data identification information is determined based on the parent-child association matches.
Specifically, the father-son association matching refers to the association relationship between the main key and the external key existing in the source database, namely, the data table with the association relationship in the source database. Determining fourth data identification information based on the parent-child association matching refers to determining data identification information matched with the primary key based on a data structure of the source database, wherein the fourth data identification information refers to identification information corresponding to the primary key, such as a data row id, a data column id or special characters, specific character strings or other characters contained in data. By determining the fourth data identification information, it is possible to determine the source data table corresponding to the fourth data identification information, and if the data table corresponding to the primary key is table a and the data table corresponding to the foreign key is table B, the table corresponding to the fourth data identification information is table a.
Based on the fourth data identification information, the segmentation source data is traversed, and fifth data identification information corresponding to the fourth data identification information is determined.
Specifically, traversing the split source data based on the fourth data identification information, determining fifth data identification information corresponding to the fourth data identification information refers to determining data identification information matched with the foreign key based on a data structure of the source database, wherein the fifth data identification information refers to identification information corresponding to the foreign key, such as a data row id, a data column id, or special characters, specific character strings, or other characters contained in the data. By determining the fifth data identification information, it is possible to determine the source data table corresponding to the fifth data identification information, and if the data table corresponding to the primary key is table a and the data table corresponding to the foreign key is table B, the table corresponding to the fifth data identification information is table B.
Second identification information of the split data is determined based on the identifier.
Specifically, referring to the description about the second identification information in the above embodiment, the description is omitted here.
And deriving a third split data source based on the fourth data identification information, the fifth data identification information and the second identification information.
Specifically, deriving the third split data source based on the fourth data identification information, the fifth data identification information and the second identification information means determining which split source data are source data including an association relationship according to the fourth data identification information, the fifth data identification information and the second identification information, and when one of the split source data corresponding to the parent-child association matching is derived on the basis of the determination, the other split source data should be derived, that is, the third split source data is derived.
According to the embodiment, the split source data are screened by determining the fourth identification information, the fifth identification information and the second identification information corresponding to the split source data, wherein the fourth identification information and the fifth identification information correspond to the father-son association matching in the export rule, so that the source data with the association relationship are exported together, the source data with the association relationship keep the original association relationship in the data migration process, and a data basis is provided for realizing the data migration of the source data corresponding to the father-son association matching. In the process, the source data is segmented by adding identifiers to the source data and dividing the source data in the source database by different sentences, so that the problems of data loss, truncation or format damage and the like possibly occurring during data type conversion can be effectively avoided, and the data migration of part of the source data is completed on the basis of ensuring the data integrity and accuracy of the source data.
(2) The type of the split source data is mapped to an intermediate data type, and an intermediate file is generated based on the intermediate data type.
Specifically, the intermediate data type refers to the process of uniformly mapping data in the source database into fixed type data, such as text type in HighGo, clob type in Kingbase and longtext type in MySQL, which cannot be directly applied to export and import during migration, and in the process of exporting the source data active database, the data type can be uniformly mapped into the clob type through JSON format, that is, based on the intermediate data type, the intermediate file is generated. And mapping the clob type back to the data type corresponding to the target database according to the type of the target database when the clob type is imported to the target database. Thus, the problem that the support degree of the data types or the expression modes of the data types are different is solved. It should be understood that the intermediate data types include, but are not limited to, a clob type, which may be set according to actual conditions, which is not particularly limited in this embodiment.
In an alternative embodiment, importing the intermediate file into the target database, and forming the target data by parsing the intermediate file includes:
(1) And analyzing the intermediate file, and determining the identification information corresponding to the identifier contained in the intermediate file and the export rule.
Specifically, parsing the intermediate file, determining the identifier included in the intermediate file and the identification information corresponding to the derived rule refer to determining which segmented source data is included through the identifier, and determining which corresponding rule is included in the derived source data through the identification information, such as one or more of partial derivation, setting a protection column, and father-son association matching.
(2) And acquiring the database type of the target database.
Specifically, the database type of the target database refers to the type of the target database, such as HighGo, kingbas, so as to provide a data basis for the subsequent modification of the mapping relationship of the intermediate data types by determining the database type of the target database.
(3) Based on the identifier and the identification information, the target data is formed in a newly added sentence corresponding to the database type.
In an alternative embodiment, forming the target data in the newly added sentence corresponding to the database type based on the identifier and the identification information includes:
the imported data is determined based on the identifier and the identification information.
Specifically, determining the imported data based on the identifier and the identification information refers to determining a port and address information of the imported source data, where the port and the address information are used to represent a storage address of the intermediate file, and the storage address of the intermediate file is related to a transmission mode of the intermediate file to the target database, which is described in detail in the related description of the above embodiment, and will not be repeated. The identification information is used for indicating whether the exported partitioned source data contains a protection column, father-son association matching and other export rules, if the protection column exists, if the target database has the same column name as the source data during import, the data corresponding to the protection column is not covered, but the data items are independently and newly added for storage; if the father-son association matching exists, the data tables corresponding to the main key and the external key are reserved during import, so that the corresponding relation between the main key and the external key in the source data is reserved.
The type of import data is mapped to an import data type that matches the database type.
Specifically, mapping the type of imported data to an imported data type matching the database type refers to adjusting the mapping of the intermediate data type to the data type corresponding to the target database. If the target database is HighGo and the intermediate data type is colb type, the intermediate data type may be mapped back to text type.
And forming target data through the new sentence based on the imported data and the imported data type.
Specifically, forming the target data through the new sentence based on the imported data and the imported data type refers to creating the target data in the target database in a new sentence mode after determining the content of the imported data and the imported data type, so as to complete the data migration from the source data in the source database to the target database.
By implementing the embodiment, the content of the imported data is determined by analyzing the intermediate file, and the mapping type of the intermediate data is adjusted by acquiring the type of the target database, so that the mapping relation of the intermediate data type is adjusted to the data type corresponding to the target database, namely the imported data type is determined, and therefore, the data migration across the databases is completed in the form of a newly added statement. In the process, when the intermediate file is analyzed, the mapping relation of the intermediate data type is adjusted, so that the target data can be generated according to the type of the target database, and the data migration across the database is completed on the basis of the data integrity and accuracy of the source data.
The embodiment also provides a database-crossing data migration device, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a data migration apparatus across database types, as shown in fig. 3, including:
the splitting module 301 is configured to add an identifier to source data of the source database to form split source data. The specific process may be referred to the related description of step S201 in the above embodiment, and will not be repeated here.
An obtaining module 302, configured to obtain the configured export rule. The specific process may be referred to the related description of step S202 in the above embodiment, and will not be repeated here.
The export module 303 is configured to export the segmentation source data based on the export rule, and generate an intermediate file. The specific process may be referred to the related description of step S203 in the above embodiment, and will not be repeated here.
The import module 304 is configured to import the intermediate file into the target database, and form target data by parsing the intermediate file. The specific process may be referred to the related description of step S204 in the above embodiment, and will not be repeated here.
The cross-database type data migration apparatus in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit ) circuits, processors and memories that execute one or more software or firmware programs, and/or other devices that can provide the above-described functionality.
The embodiment of the invention also provides computer equipment, which is provided with the data migration device of the cross-database type shown in the figure 3.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. X, the computer device includes: one or more processors 401, memory 402, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 401 is illustrated in fig. 4.
The processor 401 may be a central processor, a network processor or a combination thereof. Wherein the processor 401 may further comprise a hardware chip. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 402 stores instructions executable by the at least one processor 401 to cause the at least one processor 401 to perform the method shown in implementing the above embodiments.
Memory 402 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 402 may optionally include memory located remotely from processor 401, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 402 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; memory 402 may also include a combination of the above types of memory. The computer device also includes a communication interface 403 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (10)
1. A method of data migration across database types, the method comprising:
adding an identifier to source data of a source database to form segmented source data;
acquiring a configured export rule;
exporting the segmentation source data based on the export rule to generate an intermediate file;
and importing the intermediate file into a target database, and analyzing the intermediate file to form target data.
2. The method of claim 1, wherein the deriving the split source data based on the derivation rules, generating an intermediate file, comprises:
deriving segmentation source data based on the derivation rule;
and mapping the type of the segmentation source data into an intermediate data type, and generating an intermediate file based on the intermediate data type.
3. The method of claim 2, wherein the deriving the rule comprises: partial derivation, said deriving partitioning source data based on said derivation rules, comprising:
determining first data identification information based on the partial derivation;
determining second identification information of the segmentation source data based on the identifier;
and deriving first segmentation source data based on the first identification information and the second identification information.
4. The method of claim 2, wherein the deriving the rule comprises: setting a protection column, wherein the deriving the segmentation source data based on the derivation rule comprises:
determining third data identification information based on the set protection column;
determining second identification information of the segmentation data based on the identifier;
and deriving a second partitioned data source based on the third data identification information and the second identification information.
5. The method of claim 2, wherein the deriving the rule comprises: -parent-child association matching, said deriving segmentation source data based on said derivation rule, comprising:
determining fourth data identification information based on the parent-child association matching;
traversing the segmentation source data based on the fourth data identification information, and determining fifth data identification information corresponding to the fourth data identification information;
determining second identification information of the segmentation data based on the identifier;
and deriving a third split data source based on the fourth data identification information, the fifth data identification information and the second identification information.
6. The method of claim 1, wherein importing the intermediate file into a target database and forming target data by parsing the intermediate file comprises:
analyzing the intermediate file, and determining the identifier contained in the intermediate file and the identification information corresponding to the export rule;
acquiring a database type of a target database;
and forming target data by using a new sentence corresponding to the database type based on the identifier and the identification information.
7. The method of claim 6, wherein forming the target data in the newly added sentence corresponding to the database type based on the identifier and the identification information comprises:
determining import data based on the identifier and the identification information;
mapping the type of the imported data into an imported data type matched with the database type;
and forming target data through a new sentence based on the imported data and the imported data type.
8. A cross-database type data migration apparatus, the apparatus comprising:
the segmentation module is used for adding identifiers to the source data of the source database to form segmented source data;
the acquisition module is used for acquiring the configured export rule;
the export module is used for exporting the segmentation source data based on the export rule to generate an intermediate file;
and the importing module is used for importing the intermediate file into a target database and forming target data by analyzing the intermediate file.
9. A computer device, comprising:
a memory and a processor communicatively coupled to each other, the memory having stored therein computer instructions that, upon execution, perform the method of cross-database type data migration of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the cross-database type data migration method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311580452.7A CN117520309A (en) | 2023-11-24 | 2023-11-24 | Cross-database type data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311580452.7A CN117520309A (en) | 2023-11-24 | 2023-11-24 | Cross-database type data migration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117520309A true CN117520309A (en) | 2024-02-06 |
Family
ID=89747394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311580452.7A Pending CN117520309A (en) | 2023-11-24 | 2023-11-24 | Cross-database type data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117520309A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118585513A (en) * | 2024-08-07 | 2024-09-03 | 天翼物联科技有限公司 | Data migration method, device, computer equipment and storage medium |
-
2023
- 2023-11-24 CN CN202311580452.7A patent/CN117520309A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118585513A (en) * | 2024-08-07 | 2024-09-03 | 天翼物联科技有限公司 | Data migration method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109062952B (en) | Data query method and device and electronic equipment | |
| CN107122368B (en) | Data verification method and device and electronic equipment | |
| CN111177231A (en) | Report generation method and report generation device | |
| US8695006B2 (en) | Resource management method | |
| CN111324577B (en) | Yml file reading and writing method and device | |
| CN110866029B (en) | sql statement construction method, device, server and readable storage medium | |
| CN112912870B (en) | Conversion of tenant identifiers | |
| CN117520309A (en) | Cross-database type data migration method and device | |
| CN113468204A (en) | Data query method, device, equipment and medium | |
| CN114281803A (en) | Data migration method, device, equipment, medium and program product | |
| CN113901083A (en) | Heterogeneous data source operation resource analysis positioning method and equipment based on multiple analyzers | |
| CN117349267A (en) | Database migration processing method and system | |
| CN113703777B (en) | Code generation method, device, storage medium and equipment based on database table | |
| US10223086B2 (en) | Systems and methods for code parsing and lineage detection | |
| CN108664505B (en) | Method and device for exporting database table structure | |
| CN114764558A (en) | SQL dialect conversion method, device, system and storage medium | |
| CN112883088B (en) | Data processing method, device, equipment and storage medium | |
| US11182272B2 (en) | Application state monitoring | |
| WO2020024778A1 (en) | Method, system and device for modifying xml file in batch and computer-readable storage medium | |
| JP2019008799A (en) | System and method for merging source data from source application into target data of target application | |
| CN114185871A (en) | Data migration method, data migration device, medium, and electronic device | |
| CN114925142A (en) | Multi-type database compatibility method, device, equipment and medium of ORM framework | |
| KR102046567B1 (en) | Real-time DDL generation method for standard dictionary-based metadata change management | |
| CN110334098A (en) | A kind of database combining method and system based on script | |
| CN112749189A (en) | Data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |