CN114185873A - Data migration method, device, server and storage medium - Google Patents
Data migration method, device, server and storage medium Download PDFInfo
- Publication number
- CN114185873A CN114185873A CN202111549860.7A CN202111549860A CN114185873A CN 114185873 A CN114185873 A CN 114185873A CN 202111549860 A CN202111549860 A CN 202111549860A CN 114185873 A CN114185873 A CN 114185873A
- Authority
- CN
- China
- Prior art keywords
- source
- target
- data table
- field
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application is applicable to the technical field of artificial intelligence, and provides a data migration method, a device, a server and a storage medium, wherein the method comprises the following steps: receiving migration configuration information; determining source types of a source database and a source database according to a source address, and determining a target type of a target database and a target type of the target database according to a target address; traversing each source data table in a source database, acquiring a table name of the corresponding source data table aiming at the traversed corresponding source data table, extracting a field name of each source field in the corresponding source data table, and generating a migration configuration file aiming at the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, a source type and a target type; and migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database. According to the method and the device, data migration among the databases is realized in an information configuration mode, and the improvement of data migration efficiency is facilitated.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a data migration method, an apparatus, a server, and a storage medium.
Background
In the software development process, data migration between data sources is often required, that is: data is exported from the database as the source, and the exported data is imported into the database as the destination.
In the related art, because the types of different databases may be different, the structures of the data tables in the databases may be different, and the field values in the data tables may also be different, in the related art, if the difference between the two databases is manually processed, the operation is cumbersome, which may result in a reduction in data migration efficiency.
Disclosure of Invention
In view of this, embodiments of the present application provide a data migration method, an apparatus, a server, and a storage medium, so as to solve the problem in the related art that efficiency of data migration between databases is not high enough.
A first aspect of an embodiment of the present application provides a data migration method, including:
receiving migration configuration information, wherein the migration configuration information comprises a source address and a target address, and the source address is the address of a source database and the target address is the address of a target database;
determining source types of a source database and a source database according to a source address, determining target types of a target database and a target database according to a target address, and establishing connection with the source database and the target database respectively;
traversing each source data table in a source database, acquiring a table name of the corresponding source data table aiming at the traversed corresponding source data table, extracting a field name of each source field in the corresponding source data table, and generating a migration configuration file aiming at the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, a source type and a target type;
and migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
Further, generating a migration configuration file for the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type, including:
and combining the field names of the source fields in the corresponding source data table to generate a source field character string, and writing the table name, the source field character string, the source type and the target type of the corresponding source data table into a preset configuration template to obtain a migration configuration file for the corresponding source data table.
Further, after generating the migration configuration file for the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type, the method further includes:
determining a target table name aiming at a corresponding source data table and a target field name aiming at each source field in a corresponding source data table according to a table naming rule and a field naming rule aiming at a target database which are stored in advance;
and combining the target field names of the source fields in the corresponding source data table to generate a target field character string, and writing the target table name and the target field character string of the corresponding source data table into the migration configuration file.
Further, migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database, including:
aiming at each source data table in a source database, establishing a target data table corresponding to the corresponding source data table in a target database according to the migration configuration file corresponding to the corresponding source data table, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table;
and respectively reading the value of each source field from the corresponding source data table, converting the read value of the source field according to the format conversion relation corresponding to the source type and the target type, and writing the converted value into the corresponding target field in the target database.
Further, according to the migration configuration file corresponding to the corresponding source data table, establishing a target data table corresponding to the corresponding source data table in the target database, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table, including:
if the migration configuration file corresponding to the corresponding source data table comprises a target table name corresponding to the table name of the corresponding source data table and target field names corresponding to the source fields in the corresponding source data table, the migration configuration file corresponding to the corresponding source data table searches for the target table name corresponding to the table name of the corresponding source data table and records the target table name as a first name, searches for the target field names corresponding to the source fields in the corresponding source data table and records the target field names as a second name, establishes a target data table named as the first name in a target database, and establishes the target fields named as the second name in the established target data table.
Further, the method further comprises:
aiming at each target data table in the target database, determining the corresponding table position of the corresponding target data table in the source database and the corresponding field position of each target field in the corresponding target data table in the source data according to the migration configuration file corresponding to the corresponding target data table;
and carrying out consistency comparison on the value of each field in the corresponding target data table and the value of the corresponding field in the corresponding source data table according to the corresponding table position and the corresponding field position, and generating prompt information for prompting that the comparison result is inconsistent aiming at the fields with inconsistent comparison.
Further, establishing connections with the source database and the target database respectively includes:
if the migration configuration information further comprises a source login account, a source password, a target login account and a target password, connection with the source database is established according to the source address, the source login account and the source password, and connection with the target database is established according to the target address, the target login account and the target password.
A second aspect of an embodiment of the present application provides a data migration apparatus, including:
the information receiving unit is used for receiving migration configuration information, wherein the migration configuration information comprises a source address and a target address, the source address is the address of a source database, and the target address is the address of a target database;
the connection establishing unit is used for determining the source types of the source database and the source database according to the source address, determining the target types of the target database and the target database according to the target address, and respectively establishing connection with the source database and the target database;
the file generation unit is used for traversing each source data table in the source database, acquiring the table name of the corresponding source data table aiming at the traversed corresponding source data table, extracting the field name of each source field in the corresponding source data table, and generating a migration configuration file aiming at the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type and the target type;
and the migration execution unit is used for migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
Further, the file generating unit is specifically configured to: and combining the field names of the source fields in the corresponding source data table to generate a source field character string, and writing the table name, the source field character string, the source type and the target type of the corresponding source data table into a preset configuration template to obtain a migration configuration file for the corresponding source data table.
Further, the file generating unit is specifically further configured to: determining a target table name aiming at a corresponding source data table and a target field name aiming at each source field in a corresponding source data table according to a table naming rule and a field naming rule aiming at a target database which are stored in advance; and combining the target field names of the source fields in the corresponding source data table to generate a target field character string, and writing the target table name and the target field character string of the corresponding source data table into the migration configuration file.
Further, the migration execution unit is specifically configured to: aiming at each source data table in a source database, establishing a target data table corresponding to the corresponding source data table in a target database according to the migration configuration file corresponding to the corresponding source data table, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table; and respectively reading the value of each source field from the corresponding source data table, converting the read value of the source field according to the format conversion relation corresponding to the source type and the target type, and writing the converted value into the corresponding target field in the target database.
Further, in the migration execution unit, according to the migration configuration file corresponding to the corresponding source data table, establishing a target data table corresponding to the corresponding source data table in the target database, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table, including: if the migration configuration file corresponding to the corresponding source data table comprises a target table name corresponding to the table name of the corresponding source data table and target field names corresponding to the source fields in the corresponding source data table, the migration configuration file corresponding to the corresponding source data table searches for the target table name corresponding to the table name of the corresponding source data table and records the target table name as a first name, searches for the target field names corresponding to the source fields in the corresponding source data table and records the target field names as a second name, establishes a target data table named as the first name in a target database, and establishes the target fields named as the second name in the established target data table.
Further, the device also comprises a position determining unit and a data verifying unit.
The position determining unit is used for determining the corresponding table position of the corresponding target data table in the source database and the corresponding field position of each target field in the corresponding target data table in the source data according to the migration configuration file corresponding to the corresponding target data table aiming at each target data table in the target database;
and the data verification unit is used for carrying out consistency comparison on the value of each field in the corresponding target data table and the value of the corresponding field in the corresponding source data table according to the corresponding table position and the corresponding field position, and generating prompt information for prompting that the comparison result is inconsistent aiming at the fields with inconsistent comparison.
Further, the connection establishing unit is specifically configured to: if the migration configuration information further comprises a source login account, a source password, a target login account and a target password, connection with the source database is established according to the source address, the source login account and the source password, and connection with the target database is established according to the target address, the target login account and the target password.
A third aspect of embodiments of the present application provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the server, where the processor implements the steps of the data migration method provided in the first aspect when executing the computer program.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data migration method provided in the first aspect.
The data migration method, the data migration device, the server and the storage medium have the following beneficial effects that: data migration between databases can be automatically realized in an information configuration mode, and the data migration efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the embodiments or the related technical descriptions will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart of an implementation of a data migration method provided in an embodiment of the present application;
FIG. 2 is a flowchart illustrating an implementation of a migration execution process according to an embodiment of the present disclosure;
FIG. 3 is a flowchart of an implementation for verifying migrated data according to an embodiment of the present disclosure;
FIG. 4 is a block diagram of a data migration apparatus according to an embodiment of the present disclosure;
fig. 5 is a block diagram of a server according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
In the embodiment of the application, effective data migration between databases is realized based on an artificial intelligence technology.
The data migration method according to the embodiment of the present application may be executed by a server. When the data migration method is executed by the server, the execution subject is the server.
It should be noted that the server may include, but is not limited to, a server, a computer, a mobile phone, a tablet, a wearable smart device, and the like. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), and a big data and artificial intelligence platform.
Referring to fig. 1, fig. 1 shows a flowchart of an implementation of a data migration method according to an embodiment of the present application, including:
The migration configuration information includes a source address and a destination address. The source address is the address of the source database, and the target address is the address of the target database.
The migration configuration information is generally information for configuring a data migration process. In practice, the migration configuration information may also include, but is not limited to: a source login account, a source password, a target login account, a target password, a source type, a target type, etc.
The source login account is usually a login account of the source database. The source password is typically a login password of the source database. The target login account is typically a login account of the target database. The target password is typically a login password of the target database. The source type is typically the database type of the source database. The object type is typically a database type of the object database.
In practice, database types may include, but are not limited to, MySQL, MariaDB, Cassandra, MongoDB, Dynamo, LevelDB. Wherein MySQL and MariaDB are relational databases, Cassandra and MongoDB are non-relational databases, and Dynamo and LevelDB are key value databases.
The source database and the target database are servers for storing data.
Here, the execution subject of the data migration method is typically a server. The execution main body may receive migration configuration information directly input by a user, or may receive migration configuration information input by the user through a user terminal.
Wherein the source type is generally a database type of the source database. The object type is typically a database type of the object database.
Here, the execution body may find the source database by using the source address, and may determine the database type of the source database in various ways, for example, the database type of the source database may be determined by a function specific to the database, a string processing way, an annotation way, and the like. Similarly, the executing agent may also find the target database by using the target address, and determine the database type of the target database.
Then, the execution body may establish a communication connection with the source database and a communication connection with the target database. For example, the execution body may directly use the address of the source database to initiate a communication connection request to the source database, thereby establishing a communication connection with the source database.
In some optional implementations of various embodiments of the present application, the establishing connections with the source database and the target database respectively may include: if the migration configuration information further comprises a source login account, a source password, a target login account and a target password, connection with the source database is established according to the source address, the source login account and the source password, and connection with the target database is established according to the target address, the target login account and the target password.
The source login account is usually a login account of the source database. The source password is typically a login password of the source database. The target login account is typically a login account of the target database. The target password is typically a login password of the target database.
Here, the execution body may log in the source database by using the login password and the login account of the source database, so as to establish a communication connection with the source database. Similarly, the executing agent may log in the target database by using the login password and the login account of the target database, so as to establish a communication connection with the target database.
The source data table is usually a data table in a source database. The source field generally refers to a field in a source data table. Each data table usually includes a plurality of fields, for example, a data table for recording the scores of students in a certain grade may include a "name" field, a "class" field, a "Chinese score" field, a "math score" field, and the like.
Here, the execution body may traverse each source data table in the source database. For each source data table in the source database, the execution body may obtain a table name of the source data table and a field name of each source field in the source data table. Then, the execution body may generate a migration profile of the source data table by using the table name of the source data table, the field name of each source field, the source type, and the target type. For example, the migration configuration file may be generated by combining the above four.
In some optional implementation manners of this embodiment, the generating a migration configuration file for a corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type may include: and combining the field names of the source fields in the corresponding source data table to generate a source field character string, and writing the table name, the source field character string, the source type and the target type of the corresponding source data table into a preset configuration template to obtain a migration configuration file for the corresponding source data table.
The preset configuration template is generally a preset template for configuring information. In practice, there are usually multiple variables in the template, and when information is written to the template, it is usually the corresponding variable locations of the template where the information is written.
Here, for the corresponding source data table, the execution body may combine field names of source fields in the source data table to generate a source field string. If there are 3 fields in the source data table, and the field names are "name", "sex", and "age", respectively, the generated source field string may be: "name-sex-age". Then, the execution body may write the table name of the source data table, the generated source field character string, the source type, and the target type into corresponding variable positions of a preset configuration template, respectively, to obtain a migration configuration file for the source data table. In this way, one migration profile is available for each source data table.
In some optional implementations, after the migration configuration file for the corresponding source data table is generated according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type, the following steps one to two may also be included.
Step one, according to a table naming rule and a field naming rule which are stored in advance and aim at a target database, determining a target table name aiming at a corresponding source data table, and determining a target field name aiming at each source field in the corresponding source data table.
The table naming rule is a rule that is generally set in advance and used for naming a data table in the target database. The field naming rule is a rule for naming a field in a target database.
Here, the execution subject may process the table name of the source data table by using the table naming rule, so as to obtain the table name adapted to the target database. And processing the field name in the source database by adopting a field naming rule to obtain the field name matched with the target database. In this way, for each source data table in the source database, the target table name of the data table in the target database and the target field name of each field in the data table in the target database may be obtained.
The embodiment can realize the self-adaptive modification of the table names of the source data tables in the source database and the self-adaptive modification of the field names in the source data tables.
And step two, combining the target field names of the source fields in the corresponding source data tables to generate target field character strings, and writing the target table names and the target field character strings of the corresponding source data tables into the migration configuration files.
Here, for each source data table, the execution agent may combine the obtained target field names to generate a target field string, and write the generated target field string into the migration configuration file. In this way, it can be realized that, in the migration configuration file, the table name of each source data table in the source database in the target database and the field name of each field in the target database are recorded. Therefore, the corresponding data can be found based on the name mapping relation before and after data migration, so that the data consistency verification is realized, and whether the data migration is wrong or not is determined in time.
And 104, migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
Here, the execution body may migrate each source data table to the target database according to all configuration information in the migration configuration file. For example, for each source data table in the source database, the execution main body may establish a target data table corresponding to the source data table in the target database according to the migration configuration file corresponding to the source data table, establish a target field corresponding to each source field of the source data table in the established target data table, and then store the value of each field in the source data table into the corresponding target field of the corresponding target data table.
The method provided by the embodiment can automatically realize data migration between databases in an information configuration mode, and is beneficial to improving the data migration efficiency.
In some optional implementation manners of this embodiment, the migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database includes the following steps 201 to 202. Fig. 2 is a flowchart of an implementation of a migration execution process according to an embodiment of the present application.
Step 201, for each source data table in the source database, according to the migration configuration file corresponding to the corresponding source data table, a target data table corresponding to the corresponding source data table is established in the target database, and a target field corresponding to each source field of the corresponding source data table is established in the established target data table.
Here, the execution subject may first create, for each source data table, a target data table in the target database having the same table name as that of the source data table, and create target fields in the target data table having the same field names as those of the source data table.
The format conversion relationship is a mapping relationship that is generally set in advance and performs numerical conversion between the database of the source type and the database of the target type.
Here, for each source field in the corresponding source data table, the execution body may convert a value of the source field and then migrate the converted value to a target field, thereby implementing migration of the source data table to a target database.
In some optional implementation manners of this embodiment, the establishing, according to the migration configuration file corresponding to the corresponding source data table, a target data table corresponding to the corresponding source data table in the target database, and establishing, in the established target data table, a target field corresponding to each source field of the corresponding source data table may include: if the migration configuration file corresponding to the corresponding source data table comprises a target table name corresponding to the table name of the corresponding source data table and target field names corresponding to the source fields in the corresponding source data table, the migration configuration file corresponding to the corresponding source data table searches for the target table name corresponding to the table name of the corresponding source data table and records the target table name as a first name, searches for the target field names corresponding to the source fields in the corresponding source data table and records the target field names as a second name, establishes a target data table named as the first name in a target database, and establishes the target fields named as the second name in the established target data table.
Here, for each source data table, if the migration profile corresponding to the source data table includes a target table name corresponding to the table name of the source data table and a target field name corresponding to each source field in the source data table. At this time, the execution body may find a target table name corresponding to the table name of the source data table from the configuration file, which is denoted as a first name for convenience of description, and find a target field name corresponding to each source field in the source data table from the configuration file, which is denoted as a second name for convenience of description. It is noted that one second name may be obtained for each source field. Thereafter, the execution agent may establish a target data table with a table name of the first name in the target database and an object field with a field name of the second name in the target data table by using the first name.
According to the embodiment, the target data table and the target field can be established in the target database according to the target table name and the target field name configured in the migration configuration file, and the method is more flexible and practical.
In some optional implementations of this embodiment, the data migration method may further include the following steps 301 to 302. Fig. 3 is a flowchart for implementing verification on migrated data according to an embodiment of the present disclosure.
And the migration configuration file corresponding to the target data table and the migration configuration file corresponding to the corresponding source data table are the same file.
Here, for each target data table migrated into the target database, since the migration profile records the corresponding relationship between the corresponding source data table and the target data table, the execution subject may determine to obtain the position of the target data table in the source data table and determine to obtain the position of each field in the target data table in the corresponding source data table by using the migration profile corresponding to the target data table.
Here, the execution subject may use, for each target data table, a value of the target field for each target field in the target data table, and perform consistency comparison with a value of the target field in the corresponding field of the source data table. For example, the data may be used for direct comparison, or the hash value may be used for comparison. And if the comparison result indicates that the two are consistent, the field data is considered to be correct in migration. If the comparison result indicates that the two are inconsistent, the field data can be migrated with errors.
For example, for a field, the value of the field in the source database is 11/16/2021, and the value under the target database is 2021.11.16. The formats of the two fields are different, and at this time, the comparison between the two fields is inconsistent, and the execution main body can generate prompt information aiming at the inconsistency of the value comparison of the field. Therefore, the developer can process the prompt information in time, and the efficiency and the accuracy of data migration are improved.
Referring to fig. 4, fig. 4 is a block diagram illustrating a data migration apparatus 400 according to an embodiment of the present disclosure. The data migration apparatus in this embodiment includes units for executing the steps in the embodiments corresponding to fig. 1 to fig. 3. Please refer to fig. 1-3 and the related descriptions of the embodiments corresponding to fig. 1-3. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 4, the data migration apparatus 400 includes:
an information receiving unit 401, configured to receive migration configuration information, where the migration configuration information includes a source address and a target address, where the source address is an address of a source database, and the target address is an address of a target database;
a connection establishing unit 402, configured to determine source types of the source database and the source database according to the source address, determine target types of the target database and the target database according to the target address, and establish connections with the source database and the target database, respectively;
a file generating unit 403, configured to traverse each source data table in the source database, obtain a table name of the corresponding source data table for the traversed corresponding source data table, extract a field name of each source field in the corresponding source data table, and generate a migration configuration file for the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type;
the migration executing unit 404 is configured to migrate each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
As an embodiment of the present application, the file generating unit 403 is specifically configured to: and combining the field names of the source fields in the corresponding source data table to generate a source field character string, and writing the table name, the source field character string, the source type and the target type of the corresponding source data table into a preset configuration template to obtain a migration configuration file for the corresponding source data table.
As an embodiment of the present application, the file generating unit 403 is further specifically configured to: determining a target table name aiming at a corresponding source data table and a target field name aiming at each source field in a corresponding source data table according to a table naming rule and a field naming rule aiming at a target database which are stored in advance; and combining the target field names of the source fields in the corresponding source data table to generate a target field character string, and writing the target table name and the target field character string of the corresponding source data table into the migration configuration file.
As an embodiment of the present application, the migration executing unit 404 is specifically configured to: aiming at each source data table in a source database, establishing a target data table corresponding to the corresponding source data table in a target database according to the migration configuration file corresponding to the corresponding source data table, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table; and respectively reading the value of each source field from the corresponding source data table, converting the read value of the source field according to the format conversion relation corresponding to the source type and the target type, and writing the converted value into the corresponding target field in the target database.
As an embodiment of the present application, in the migration executing unit 404, according to the migration configuration file corresponding to the corresponding source data table, establishing a target data table corresponding to the corresponding source data table in the target database, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table, including: if the migration configuration file corresponding to the corresponding source data table comprises a target table name corresponding to the table name of the corresponding source data table and target field names corresponding to the source fields in the corresponding source data table, the migration configuration file corresponding to the corresponding source data table searches for the target table name corresponding to the table name of the corresponding source data table and records the target table name as a first name, searches for the target field names corresponding to the source fields in the corresponding source data table and records the target field names as a second name, establishes a target data table named as the first name in a target database, and establishes the target fields named as the second name in the established target data table.
As an embodiment of the present application, the apparatus further includes a position determination unit and a data verification unit (not shown in the figure).
The position determining unit is used for determining the corresponding table position of the corresponding target data table in the source database and the corresponding field position of each target field in the corresponding target data table in the source data according to the migration configuration file corresponding to the corresponding target data table aiming at each target data table in the target database;
and the data verification unit is used for carrying out consistency comparison on the value of each field in the corresponding target data table and the value of the corresponding field in the corresponding source data table according to the corresponding table position and the corresponding field position, and generating prompt information for prompting that the comparison result is inconsistent aiming at the fields with inconsistent comparison.
As an embodiment of the present application, the connection establishing unit 402 is specifically configured to: if the migration configuration information further comprises a source login account, a source password, a target login account and a target password, connection with the source database is established according to the source address, the source login account and the source password, and connection with the target database is established according to the target address, the target login account and the target password.
The device provided by the embodiment can automatically realize data migration between databases in an information configuration mode, and is beneficial to improving the data migration efficiency.
It should be understood that, in the structural block diagram of the data migration apparatus shown in fig. 4, each unit is used to execute each step in the embodiment corresponding to fig. 1 to fig. 3, and each step in the embodiment corresponding to fig. 1 to fig. 3 has been explained in detail in the above embodiment, and specific reference is made to the relevant description in the embodiments corresponding to fig. 1 to fig. 3 and fig. 1 to fig. 3, which is not repeated herein.
Fig. 5 is a block diagram of a server according to another embodiment of the present application. As shown in fig. 5, the server 500 of this embodiment includes: a processor 501, a memory 502 and a computer program 503, such as a program for a data migration method, stored in the memory 502 and executable on the processor 501. The processor 501 executes the computer program 503 to implement the steps in the embodiments of the data migration methods, such as the steps 101 to 104 shown in fig. 1. Alternatively, when the processor 501 executes the computer program 503, the functions of the units in the embodiment corresponding to fig. 4, for example, the functions of the units 401 to 404 shown in fig. 4, are implemented, for which reference is specifically made to the relevant description in the embodiment corresponding to fig. 4, which is not repeated herein.
Illustratively, the computer program 503 may be divided into one or more units, which are stored in the memory 502 and executed by the processor 501 to accomplish the present application. One or more elements may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of computer program 503 in server 500. For example, the computer program 503 may be divided into an information receiving unit, a connection establishing unit, a file generating unit, and a migration executing unit, and the specific functions of the units are as described above.
The server may include, but is not limited to, a processor 501, a memory 502. Those skilled in the art will appreciate that fig. 5 is merely an example of a server 500, and does not constitute a limitation on server 500, and may include more or fewer components than shown, or some components in combination, or different components, e.g., a turntable device may also include input output devices, network access devices, buses, etc.
The Processor 501 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 502 may be an internal storage unit of the server 500, such as a hard disk or a memory of the server 500. The memory 502 may also be an external storage device of the server 500, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the server 500. Further, memory 502 may also include both internal storage units of server 500 and external storage devices. The memory 502 is used for storing computer programs and other programs and data required by the turntable device. The memory 502 may also be used to temporarily store data that has been output or is to be output.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated module, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. The computer readable storage medium may be non-volatile or volatile. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method of data migration, the method comprising:
receiving migration configuration information, wherein the migration configuration information comprises a source address and a target address, the source address is the address of a source database, and the target address is the address of a target database;
determining source types of a source database and a source database according to the source address, determining target types of a target database and a target database according to the target address, and establishing connection with the source database and the target database respectively;
traversing each source data table in the source database, acquiring a table name of the corresponding source data table aiming at the traversed corresponding source data table, extracting a field name of each source field in the corresponding source data table, and generating a migration configuration file aiming at the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type and the target type;
and migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
2. The data migration method according to claim 1, wherein the generating a migration configuration file for the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type includes:
combining the field names of the source fields in the corresponding source data table to generate a source field character string, and writing the table name, the source field character string, the source type and the target type of the corresponding source data table into a preset configuration template to obtain a migration configuration file for the corresponding source data table.
3. The data migration method according to claim 2, wherein after the generating the migration configuration file for the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type, and the target type, further comprises:
determining a target table name for a corresponding source data table and a target field name for each source field in a corresponding source data table according to a table naming rule and a field naming rule which are stored in advance and aim at the target database;
and combining the target field names of the source fields in the corresponding source data table to generate a target field character string, and writing the target table name and the target field character string of the corresponding source data table into the migration configuration file.
4. The data migration method according to claim 1, wherein the migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database includes:
aiming at each source data table in the source database, establishing a target data table corresponding to the corresponding source data table in the target database according to the migration configuration file corresponding to the corresponding source data table, and establishing a target field corresponding to each source field of the corresponding source data table in the established target data table;
and respectively reading the value of each source field from the corresponding source data table, converting the read value of the source field according to the format conversion relation corresponding to the source type and the target type, and writing the converted value into the corresponding target field in the target database.
5. The data migration method according to claim 4, wherein the creating, in the target database, a target data table corresponding to the corresponding source data table and creating, in the created target data table, a target field corresponding to each source field of the corresponding source data table according to the migration configuration file corresponding to the corresponding source data table includes:
if the migration configuration file corresponding to the corresponding source data table comprises a target table name corresponding to the table name of the corresponding source data table and target field names corresponding to the source fields in the corresponding source data table, searching the target table name corresponding to the table name of the corresponding source data table from the migration configuration file corresponding to the corresponding source data table, recording the target table name as a first name, searching the target field names corresponding to the source fields in the corresponding source data table, recording the target field names as a second name, establishing a target data table with the name as the first name in the target database, and establishing the target field with the name as the second name in the established target data table.
6. The data migration method according to claim 4, wherein the method further comprises:
for each target data table in the target database, determining a corresponding table position of the corresponding target data table in the source database and a corresponding field position of each target field in the corresponding target data table in the source data according to the migration configuration file corresponding to the corresponding target data table;
and according to the corresponding table position and the corresponding field position, carrying out consistency comparison on the value of each field in the corresponding target data table and the value of the corresponding field in the corresponding source data table, and generating prompt information for prompting that the comparison result is inconsistent aiming at the fields with inconsistent comparison.
7. The data migration method according to any one of claims 1 to 6, wherein the establishing connections with the source database and the target database respectively comprises:
if the migration configuration information further comprises a source login account, a source password, a target login account and a target password, establishing connection with the source database according to the source address, the source login account and the source password, and establishing connection with the target database according to the target address, the target login account and the target password.
8. An apparatus for data migration, the apparatus comprising:
an information receiving unit, configured to receive migration configuration information, where the migration configuration information includes a source address and a target address, where the source address is an address of a source database, and the target address is an address of a target database;
the connection establishing unit is used for determining the source types of the source database and the source database according to the source address, determining the target types of the target database and the target database according to the target address, and respectively establishing connection with the source database and the target database;
the file generation unit is used for traversing each source data table in the source database, acquiring the table name of the corresponding source data table aiming at the traversed corresponding source data table, extracting the field name of each source field in the corresponding source data table, and generating a migration configuration file aiming at the corresponding source data table according to the table name of the corresponding source data table, the field name of each source field, the source type and the target type;
and the migration execution unit is used for migrating each source data table to the target database according to the migration configuration file corresponding to each source data table in the source database.
9. A server comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111549860.7A CN114185873A (en) | 2021-12-17 | 2021-12-17 | Data migration method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111549860.7A CN114185873A (en) | 2021-12-17 | 2021-12-17 | Data migration method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114185873A true CN114185873A (en) | 2022-03-15 |
Family
ID=80544304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111549860.7A Pending CN114185873A (en) | 2021-12-17 | 2021-12-17 | Data migration method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114185873A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114816578A (en) * | 2022-05-11 | 2022-07-29 | 上海柯林布瑞信息技术有限公司 | Method, device and equipment for generating program configuration file based on configuration table |
| CN115543969A (en) * | 2022-11-28 | 2022-12-30 | 建信金融科技有限责任公司 | Data migration method, device, equipment and medium |
-
2021
- 2021-12-17 CN CN202111549860.7A patent/CN114185873A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114816578A (en) * | 2022-05-11 | 2022-07-29 | 上海柯林布瑞信息技术有限公司 | Method, device and equipment for generating program configuration file based on configuration table |
| CN114816578B (en) * | 2022-05-11 | 2024-05-17 | 上海柯林布瑞信息技术有限公司 | Program configuration file generation method, device and equipment based on configuration table |
| CN115543969A (en) * | 2022-11-28 | 2022-12-30 | 建信金融科技有限责任公司 | Data migration method, device, equipment and medium |
| CN115543969B (en) * | 2022-11-28 | 2023-03-10 | 建信金融科技有限责任公司 | Data migration method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2745518C9 (en) | Data isolation in the blockchain network | |
| CN114185873A (en) | Data migration method, device, server and storage medium | |
| CN108334609B (en) | Method, device, equipment and storage medium for realizing JSON format data access in Oracle | |
| CN109558525B (en) | Test data set generation method, device, equipment and storage medium | |
| CN111352935B (en) | Index creating method, device and equipment in block chain type account book | |
| CN104639517A (en) | Method and device for verifying identities by aid of human body biological characteristics | |
| CN111435367B (en) | Knowledge graph construction method, system, equipment and storage medium | |
| CN113505128A (en) | Method, device and equipment for creating data table and storage medium | |
| WO2019161645A1 (en) | Shell-based data table extraction method, terminal, device, and storage medium | |
| CN110399306B (en) | Automatic testing method and device for software module | |
| US20160019266A1 (en) | Query generating method and query generating device | |
| CN111897828A (en) | Data batch processing implementation method, device, equipment and storage medium | |
| CN110362630B (en) | Data management method, device, equipment and computer readable storage medium | |
| CN115858488A (en) | Parallel migration method and device based on data governance and readable medium | |
| CN114116691A (en) | Data import checking method, system, device and computer readable medium | |
| US11449461B2 (en) | Metadata-driven distributed dynamic reader and writer | |
| KR100762712B1 (en) | Method for transforming of electronic document based on mapping rule and system thereof | |
| CN110704635B (en) | Method and device for converting triplet data in knowledge graph | |
| CN116860941A (en) | Question answering method and device, electronic equipment and storage medium | |
| CN111124541A (en) | Configuration file generation method, device, equipment and medium | |
| CN103440231A (en) | Equipment and method for comparing texts | |
| CN113741864B (en) | Automatic semantic service interface design method and system based on natural language processing | |
| CN115757174A (en) | Database difference detection method and device | |
| CN110795915A (en) | Method, system, equipment and computer-readable storage medium for modifying xml file in batch | |
| US11115189B2 (en) | Verifying a blockchain-type ledger |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |