CN110297813B - Data migration method and device, computer equipment and storage medium - Google Patents

Data migration method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN110297813B
CN110297813B CN201910430064.8A CN201910430064A CN110297813B CN 110297813 B CN110297813 B CN 110297813B CN 201910430064 A CN201910430064 A CN 201910430064A CN 110297813 B CN110297813 B CN 110297813B
Authority
CN
China
Prior art keywords
migration
target
data
field
data table
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910430064.8A
Other languages
Chinese (zh)
Other versions
CN110297813A (en
Inventor
严炜
陈旭波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Bank Co Ltd
Original Assignee
Ping An Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Bank Co Ltd filed Critical Ping An Bank Co Ltd
Priority to CN201910430064.8A priority Critical patent/CN110297813B/en
Publication of CN110297813A publication Critical patent/CN110297813A/en
Application granted granted Critical
Publication of CN110297813B publication Critical patent/CN110297813B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/214Database migration support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data migration method, a data migration device, computer equipment and a storage medium, wherein the method comprises the following steps: determining a target database with access authority as read-write authority based on the data migration instruction; displaying a data table to be migrated and a field to be migrated corresponding to the target database on a visual operation interface; determining a target immigration data table based on the data table to be immigrated, and determining a target immigration field based on the field to be immigration; acquiring a target migration scheme corresponding to the target migration field, wherein the target migration scheme comprises a target migration rule, a target migration data table and a target migration field which are determined based on the original migration rule; and acquiring a corresponding target code generation template according to the target migration rule, processing the target migration data table, the target migration field, the target migration data table and the target migration field by adopting the target code generation template, acquiring and executing the data migration code corresponding to the target migration rule, and acquiring a data migration result. The method improves the accuracy of data migration.

Description

Data migration method and device, computer equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data migration method and apparatus, a computer device, and a storage medium.
Background
The data migration process refers to a process of migrating data from one database to another database, and is a means for realizing data backup. In the traditional data migration process, data migration is mainly carried out in an SQL (structured query language) code mode or realized in a batch processing mode, errors are easy to occur in the data migration modes, and the error position is not easy to be found. For example, when data of a plurality of fields in one data table is migrated to another data table through the SQL code, if one field is missing in the SQL code, it takes a long time to find the error position. Or when the data migration is realized through a batch processing mode, each field of the database maps one line of codes, the code amount is large in the batch processing process, the error probability is high in the compiling process, and the error position is not easy to find.
Disclosure of Invention
Embodiments of the present invention provide a data migration method and apparatus, a computer device, and a storage medium, so as to solve the problems that an error is easy to occur and an error position is not easy to be found in a current data migration process.
A method of data migration, comprising:
receiving a data migration instruction, and determining a target database with access authority as read-write authority based on the data migration instruction;
displaying at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table in a hierarchical structure display mode on a visual operation interface;
determining a target migration data table based on at least one to-be-migrated data table, and determining a target migration field based on at least one to-be-migrated field;
receiving a migration rule editing request, and entering a migration rule editing interface corresponding to the target migration field based on the migration rule editing request, wherein the migration rule editing interface displays at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table;
acquiring a target migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the target migration scheme comprises a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table and a target migration field determined based on at least one to-be-migrated field;
acquiring a corresponding target code generation template according to the target migration rule, and processing the target migration data table, the target migration field, the target migration data table and the target migration field by adopting the target code generation template to acquire a data migration code corresponding to the target migration rule;
and executing the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring a data migration result.
A data migration apparatus, comprising:
the target database determining module is used for receiving a data migration instruction and determining a target database with access right as read-write right based on the data migration instruction;
the data display module is used for displaying at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table in a hierarchical structure display mode on a visual operation interface;
the target migration field determining module is used for determining a target migration data table based on at least one to-be-migrated data table and determining a target migration field based on at least one to-be-migrated field;
the migration rule editing request module is used for receiving a migration rule editing request, entering a migration rule editing interface corresponding to the target migration field based on the migration rule editing request, wherein the migration rule editing interface displays at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table;
a target migration scheme obtaining module, configured to obtain, based on the migration rule editing interface, a target migration scheme corresponding to the target migration field, where the target migration scheme includes a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table, and a target migration field determined based on at least one to-be-migrated field;
a data migration code obtaining module, configured to obtain a corresponding target code generation template according to the target migration rule, process the target migration data table, the target migration field, the target migration data table, and the target migration field by using the target code generation template, and obtain a data migration code corresponding to the target migration rule;
and the data migration result acquisition module is used for executing the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring a data migration result.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the data migration method when executing the computer program.
A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the above-mentioned data migration method.
According to the data migration method, the data migration device, the computer equipment and the storage medium, after the target database is determined, the at least one to-be-migrated data table and the at least one to-be-migrated field corresponding to the target database are rapidly and comprehensively displayed on the visual operation interface in a hierarchical structure display mode, so that the determination efficiency and accuracy of the subsequent target migration data table and the target migration field are guaranteed, errors in the input process of developers are avoided, and the time is saved. And determining a target migration data table based on at least one to-be-migrated data table and determining a target migration field based on at least one to-be-migrated field, so that the required target migration data table and the required target migration field can be rapidly determined, and the determination efficiency of the target migration scheme is improved. And comprehensively and intuitively displaying at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table on a migration rule editing interface so as to quickly determine a target migration scheme corresponding to the target migration field, so that the determination process of the target migration scheme is simpler and more convenient, and the determination efficiency is improved. And acquiring a corresponding target code generation template according to the target migration rule, processing corresponding information in a target migration scheme by adopting the target code generation template, and acquiring a data migration code corresponding to the target migration rule, so that the acquisition efficiency of the data migration code is improved, the code error is avoided, and the accuracy of the data migration code is improved. And executing the data migration code, and migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, so that data migration is rapidly realized, and the efficiency of data migration is guaranteed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a diagram of an application environment of a data migration method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data migration method according to an embodiment of the present invention;
FIG. 3 is another flow chart of a data migration method according to an embodiment of the present invention;
FIG. 4 is another flow chart of a data migration method according to an embodiment of the present invention;
FIG. 5 is another flow chart of a data migration method in an embodiment of the present invention;
FIG. 6 is another flow chart of a method of data migration in an embodiment of the present invention;
FIG. 7 is a diagram illustrating a data migration apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The data migration method provided by the embodiment of the invention can be applied to the application environment shown in fig. 1. Specifically, the data migration method is applied to a data migration system, where the data migration system includes a client and a server as shown in fig. 1, and the client and the server communicate with each other through a network to improve the accuracy of data migration. The client is also called a developer side, and refers to a program corresponding to the server and providing local services for the client. The client may be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.
In an embodiment, as shown in fig. 2, a data migration method is provided, which is described by taking the application of the method to the server in fig. 1 as an example, and includes the following steps:
s201: and receiving a data migration instruction, and determining the target database with the access authority as the read-write authority based on the data migration instruction.
The data migration instruction specifically refers to an instruction for triggering the data migration system to perform data migration, that is, an instruction for implementing data migration from the source database to the target database, so as to implement the purpose of migrating and backing up data in the source database to the target database. The source database is used for storing data to be migrated; accordingly, the target database is a database for storing data that needs to be migrated. Access rights refer to the rights of the data migration system to access the database. The access authority includes a read authority (i.e., an authority that can read data in the database), a write authority (i.e., an authority that can write data to the database), and a read-write authority (i.e., an authority that can read data in the database as well as write data to the database). The access authority of the data migration system to the target database is limited to the read-write authority, so that the data migration system can access the data in the target database to know which data tables and other information are contained in the target database; and data can be written into the target database to ensure that the target database meets the requirements of the data transferee.
In this embodiment, step S201 specifically includes the following steps:
s2011: the server receives a data migration instruction sent by the client, controls the client to enter a visual operation interface based on the data migration instruction, and displays at least one original database and the access authority of each original database on the visual operation interface.
In this embodiment, data migration is performed on the visual operation interface, so that the creation process of the migration scheme can be visually displayed on the visual operation interface, and therefore the migration scheme can be conveniently viewed by the developer, and the migration scheme can be quickly and efficiently written. The migration scheme may be understood as a scheme for migrating the contents of certain fields in a particular data table in the source database to corresponding fields in a corresponding data table in the target database. Specifically, a developer may click a data migration button of the client to trigger a data migration instruction, so that the server of the data migration system controls the client to enter the visual operation interface after receiving the data migration instruction.
Wherein, the original database refers to a database which is connected with the data migration system in a communication way. The access right of each original database specifically refers to the right of the data migration system to access the original database, and specifically includes a read right, a write right and a read-write right. It is to be understood that, before step S2011, that is, before the server receives the data migration instruction sent by the client, the server of the data migration system may further perform the following steps: (1) receiving parameter configuration information corresponding to each original database input by a client, wherein the parameter configuration information comprises an IP address, an Sid number, a port number, a login user name and a login password of the original database. Wherein Sid (Security Identifiers) is a unique number for identifying the user and the computer account, and in this embodiment, refers to a number for uniquely identifying the database to be connected. The IP address, the Sid number, the port number, the login user name, the login password and other parameter configuration information of the original database are necessary parameters for realizing the communication connection between the data migration system and the original database, and are necessary parameters for enabling the data migration system to be smoothly connected with the original database. (2) And according to the parameter configuration information corresponding to each original database, enabling the data migration system to be in communication connection with the original databases. The server judges whether the original database corresponds to the only original database according to the IP address, the Sid number and the port number of the original database, and performs identity authentication according to the login user name and the login password; if the IP address, the Sid number and the port number correspond to the unique original database and the login user name and the login password pass the authentication, the data migration system is communicated with the original database; otherwise, displaying a reconnection interface to enable the developer to input the parameter configuration information again, and repeating the steps to ensure that the data migration system can be connected to each original database, thereby ensuring the smooth data migration. (3) And acquiring the authority configuration information of an original database which is in communication connection with the data migration system, and storing the authority configuration information in a database authority information table. The authority configuration information of the original database is information of access authority configured in advance for each original database by a system administrator. The database permission information table is an information table used for recording access permission corresponding to at least one original database connected with the data migration system. It can be understood that, when the server controls the client to enter the visual operation interface after receiving the data migration instruction, at least one original database in the database permission information table and the corresponding access permission thereof may be displayed on the visual operation interface.
S2012: and determining a target database with the access authority as the read-write authority from at least one original database with the access authority as the read-write authority.
Specifically, when a client enters a visual operation interface, at least one original database which is in communication connection with a server of the data migration system is displayed on the visual operation interface, and the access authority of each original database is displayed; the developer can select one of the original databases with the access right as the read-write right as the target database, so that the target database with the access right as the read-write right can be quickly determined.
It can be understood that when the data migration instruction enters the visualized operation interface, the access authority of each original database in the data authority information table is called, and at least one original database corresponding to the access authority such as the read-write authority, the read authority, the write authority and the like is displayed in a tree structure on the visualized operation interface, so that developers can conveniently and quickly select and determine the target database. And configuring a page monitoring event on the visual operation interface, wherein when a developer clicks and selects any one original database, the page monitoring event returns monitoring information to the server, and the monitoring information comprises a database identifier corresponding to the clicked and selected original database. And the server determines the target database according to monitoring information returned by the page monitoring event, so that the target database with the access right as the read-write right is quickly determined.
Further, in order to avoid that a developer determines an original database with an access right which is not a read-write right as a target database, the original database with the access right which is not the read-write right in the visual operation interface can be set to be in an inoperable mode, namely, in a non-clickable mode in the process of determining the target database; and setting the original database with the access right of the read-write right in the visual operation interface as an operable mode, namely a clickable mode, so as to ensure that the target database with the access right of the read-write right can be quickly determined.
S202: and displaying at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table in a hierarchical structure display mode on a visual operation interface.
The data table to be migrated refers to a data table which is stored in the target database and can realize data migration. The field to be migrated refers to a field of migratable data in the data table to be migrated. The hierarchical structure display mode is a mode in which all the data to be displayed are displayed hierarchically according to a preset hierarchical relationship. For example, all the to-be-migrated data tables stored in the target database are displayed on a first-layer display interface of the visual operation interface, and at least one to-be-migrated field corresponding to each to-be-migrated data table is displayed on a second-layer display interface corresponding to each to-be-migrated data table. Generally, when a developer selects and determines a target database, a visual operation interface corresponding to the target database can be entered, all data tables to be migrated are visually displayed on a first-layer display interface of the visual operation interface, each data table to be migrated is correspondingly provided with a pull-down button or a + button, and the developer clicks the pull-down button or the + button to display all fields to be migrated corresponding to the data table to be migrated.
Specifically, after a target database is determined, the server enters a visual operation interface corresponding to the target database, displays all the data tables and the fields to be migrated on the visual operation interface in a hierarchical structure display mode, so as to quickly and comprehensively display all the data tables and the fields to be migrated, and avoid data migration errors caused by errors in inputting the data tables and the fields to be migrated when a migration scheme is created.
S203: and determining a target migration data table based on the at least one to-be-migrated data table, and determining a target migration field based on the at least one to-be-migrated field.
The target migration data table refers to a data table in which the migrated data needs to be stored in the data migration operation. The target immigration field is a field which is arranged in the target immigration data table and is used for storing data needing immigration. Specifically, a developer can select a determined target immigration data table from at least one immigration data table to be displayed on the visual operation interface, and after the target immigration data table is determined, a target immigration field is determined from at least one immigration field corresponding to the target immigration data table. Or, if the developer does not select the target immigration data table requiring the immigration data in the at least one data table to be immigrated displayed on the visual operation interface, or does not select the target immigration field requiring the immigration data, the developer can also independently create the target immigration data table and/or the target immigration field to ensure that the data migration can be realized.
Specifically, a developer selects one of at least one to-be-migrated field corresponding to each target migration data table displayed on the visual operation interface, and determines the selected field as a target migration field, so that the target migration field is quickly acquired. For example, the field to be migrated may be determined as the target migration field by a right-click selection method or a drag method.
S204: receiving a migration rule editing request, entering a migration rule editing interface corresponding to a target migration field based on the migration rule editing request, wherein the migration rule editing interface displays at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table.
The migration rule editing request is used for triggering a request for entering the migration rule editing interface. Specifically, each target migration field determined on the visual operation interface corresponds to an editing button, and when a developer clicks the editing button, a migration rule editing request corresponding to the target migration field can be triggered, so that the server controls the client to enter the migration rule editing interface corresponding to the target migration field based on the migration rule editing request after receiving the migration rule editing request. Specifically, a migration rule editing interface corresponding to the target immigration field is entered based on the target immigration field, that is, the migration rule editing interface is an interface dedicated to configuring a migration rule corresponding to the target immigration field.
The original migration rule is a preset rule for realizing data migration. Each original migration rule corresponds to an original code generation template, the original code generation template is a code which is edited and packaged in advance and can realize the data migration function, and the original code generation template provides a data interface for receiving actual parameters required by realizing the original code generation template.
In this embodiment, the original migration rule includes, but is not limited to, a copy migration rule, a join migration rule, and a split migration rule. The copy and migration rule is to copy the content of a field in the source database to a field in the target database, for example, copy the content of the serialno field in table a in the source database a to the serialno field or id field in table B in the target data table B, where the names of the fields in the two data tables may be the same or different, where table a is a target migrated data table and table B is a target migrated data table. The splicing migration rule is to splice the contents of at least two fields in the source database to a certain field in the target database, for example, splice the contents of the firstName field and the second name field in the table a in the source database a to the name field in the table B in the target database B. The split migration rule splits the content of a field in the source database into at least two fields in the target database, for example, splits the content of the name field in table a in the source database a into the firstName field and the secondName field in table B in the target database B.
The source database with the read permission refers to a database with the read permission of the data migration system, and particularly relates to a database capable of realizing data migration. The data table to be migrated refers to a data table which is stored in a source database with read permission and can realize data migration. The to-be-migrated field refers to a field of migratable data in the to-be-migrated data table.
S205: and acquiring a target migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the target migration scheme comprises a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table and a target migration field determined based on at least one to-be-migrated field.
The target migration scheme refers to a migration scheme created for a target immigration field, and the migration scheme determines information such as a target migration rule, a target immigration data table and a target immigration field corresponding to the target immigration field. Because at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table are displayed on the migration rule editing interface, developers can quickly select and determine a required target migration rule from the at least one original migration rule, quickly select and determine a required target migration data table from the at least one to-be-migrated data table, and quickly select a required target migration field from the at least one to-be-migrated field, so that a target migration scheme corresponding to a target migration field is formed, the operation process is simpler and more convenient, the probability of errors in manually inputting corresponding information is reduced, and the determination efficiency and accuracy of the target migration scheme are improved.
S206: and acquiring a corresponding target code generation template according to the target migration rule, and processing the target migration data table, the target migration field, the target migration data table and the target migration field by adopting the target code generation template to acquire a data migration code corresponding to the target migration rule.
Wherein the target code generation template is a template for generating execution code, which is generated in advance and stored in the server. Because the data migration in this embodiment is directed to data migration between databases, the target code generation template capable of implementing the data migration function can be pre-edited by using SQL statements, so that the corresponding target generation template is directly called in the following process to quickly generate the corresponding data migration code, the generation efficiency and accuracy of the data migration code are improved, and the influence on the data migration efficiency due to editing errors of developers during the data migration process is avoided.
In the server, an original code generation template corresponding to each original migration rule is configured in advance, and the original code generation template comprises an SQL statement capable of realizing the corresponding function (such as copying, splicing and splitting) of the original migration rule and a form parameter applied in the SQL statement. And after determining the corresponding target code generation template according to the target migration rule, taking the target migration data table, the target migration field, the target migration data table and the target migration field as actual parameters corresponding to the corresponding form parameters, thereby quickly acquiring the corresponding data migration code. Because the SQL statements corresponding to the functions of the original migration rule are written in advance, after the target migration scheme is determined, the target migration data table, the target migration field, the target migration data table and the target migration field in the target migration scheme can be used as actual parameters to be substituted into the target code generation template, the data migration code corresponding to the target migration rule is obtained, the obtaining efficiency and the obtaining accuracy of the data migration code are improved, and errors in the code editing process are avoided.
S207: and executing a data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to a target migration rule, and acquiring a data migration result.
The data to be migrated corresponding to the target migration field specifically refers to data recorded in the target migration data table and corresponding to the target migration field, and refers to data that needs to be migrated and stored in the target migration data table. The padding area refers to an area in the target immigration data table for storing data of the target immigration field.
Specifically, on a server of the data migration system, a data migration code is executed, data to be migrated corresponding to a target migration field in a target migration data table is read first, data processing (such as copying, splicing or splitting processing) is performed on the data to be migrated according to the target migration rule, the processed data to be migrated is obtained, and the processed data to be migrated is stored in a filling area corresponding to the target migration field in the target migration data table to obtain a data migration result, so that fast migration and data backup according to the data to be migrated are achieved, and data security is guaranteed.
In the data migration method provided by this embodiment, after the target database is determined, the at least one to-be-migrated data table and the at least one to-be-migrated field corresponding to the target database are quickly and comprehensively displayed in a hierarchical structure display manner on the visual operation interface, which is helpful for ensuring the determination efficiency and accuracy of the subsequent target migration data table and target migration field, and avoiding errors in the input process of developers, so as to save time. And determining a target migration data table based on the at least one to-be-migrated data table and determining a target migration field based on the at least one to-be-migrated field, so that the required target migration data table and the required target migration field can be rapidly determined, and the determination efficiency of the target migration scheme is improved. And comprehensively and intuitively displaying at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table on a migration rule editing interface so as to quickly determine a target migration scheme corresponding to a target migration field, so that the determination process of the target migration scheme is simpler and more convenient, and the determination efficiency is improved. And acquiring a corresponding target code generation template according to the target migration rule, processing corresponding information in the target migration scheme by adopting the target code generation template, and acquiring a data migration code corresponding to the target migration rule, so that the acquisition efficiency of the data migration code is improved, the code error is avoided, and the accuracy of the data migration code is improved. And executing a data migration code, and migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to a target migration rule, so that data migration is rapidly realized, and the efficiency of data migration is guaranteed.
In an embodiment, as shown in fig. 3, step S202, namely, displaying, in a hierarchical display manner on the visual operation interface, at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table, includes:
s301: and traversing the folders in the target data table by using a folder traversal script to obtain at least one original folder.
The folder traversal script refers to a script which is edited in advance and used for traversing the folder in the target data table. The original folder is a folder arranged in the target database and used for storing the data table to be migrated.
In this embodiment, the folder in the target data table is traversed by using the folder traversal script, and specifically, all folder information in the target database may be queried by using the folder traversal script including the traversal statement of EXEC master, dbo, xp _ folders'd: \ so as to quickly determine at least one original folder, where the folder information includes a folder name and an address.
S302: and traversing the data table in each original folder by adopting a data table traversal script to obtain at least one data table to be migrated corresponding to each original folder.
The data table traversal script refers to a script which is edited in advance and used for traversing the data table in each original folder.
In this embodiment, the data table in each original folder is traversed BY using the data table traversal script, and specifically, all the data tables to be migrated in each original folder may be queried BY using the data table traversal script that includes a traversal statement of SELECT NAME FROM system query term. And traversing the data tables to be migrated in each original folder by adopting a data table traversal script so as to quickly, simply and conveniently acquire at least one data to be migrated and reduce complicated manual searching operation.
S303: and traversing the fields in each data table to be migrated by adopting a field traversal script, and acquiring at least one field to be migrated corresponding to each data table to be migrated.
The field traversal script is a script which is edited in advance and is used for traversing each field to be migrated into the data table.
In this embodiment, the field in each data table to be migrated is traversed by using the field traversal script, and specifically, all the fields to be migrated in each data table to be migrated may be queried by using the field traversal script including the traversal statement of SELECT NAME FROM system library world ID (OBJECT _ ID ('bcusemer'). And traversing each field to be migrated in the data table to be migrated by adopting a field traversal script so as to accelerate the acquisition of all the fields to be migrated and reduce the tedious manual search operation.
S304: and displaying at least one original folder, at least one data table to be migrated and at least one field to be migrated corresponding to the target database in a hierarchical structure display mode on a visual operation interface by adopting a page asynchronous loading mode.
The page asynchronous loading is specifically a mode of loading the page by adopting Ajax. The Ajax, namely "Asynchronous JavaScript And XML" (Asynchronous JavaScript And XML), refers to a web page development technology for creating interactive web page applications. Ajax is a technique for creating a fast dynamic web page, which is a technique that can update a part of a web page without reloading the entire web page. On a visual operation interface of the data migration system, the server and the target database exchange a small amount of data, Ajax can enable the webpage to be updated asynchronously, and therefore, a certain part of the webpage can be updated without reloading the whole webpage, and the speed of loading at least one original folder, at least one data table to be migrated and at least one field to be migrated on the visual operation interface is increased.
In this embodiment, a page asynchronous loading manner is adopted to load the original folder, the data table to be migrated, and the field to be migrated in each target database, so that the loading speed of the data table to be migrated and the field to be migrated is increased, and the creation efficiency of the migration scheme is improved. And displaying at least one original folder, at least one data table to be migrated and at least one field to be migrated on a visual operation interface in a hierarchical structure display mode so as to visually and clearly display the data tables to be migrated and the field to be migrated to developers for watching, thereby facilitating the subsequent determination of the target migration data table and the target migration field.
In the data migration method provided by this embodiment, the folder traversal script, the data table traversal script, and the field traversal script are respectively used for traversal, so that at least one original folder, at least one to-be-migrated data table, and at least one to-be-migrated field are simply, conveniently, and quickly acquired, cumbersome manual operations are reduced, and determination efficiency of the original folder, the to-be-migrated data table, and the to-be-migrated field is improved. And displaying at least one original folder, at least one data table to be migrated and at least one field to be migrated in a hierarchical structure display mode on a visual operation interface in a page asynchronous loading mode, and visually and clearly displaying on the visual operation interface.
In an embodiment, the target immigration data table and the target immigration field are determined, and the target immigration data table and the target immigration field can be directly determined by the data table to be immigrated and displayed on the visual operation interface, or a target immigration data table containing the target immigration field can be newly built in the target database by a developer. Therefore, as shown in fig. 4, step S203, namely determining a target migration data table based on at least one to-be-migrated data table, and determining a target migration field based on at least one to-be-migrated field, includes:
s401: and acquiring a migration information configuration request, wherein the migration information configuration request comprises a configuration migration data table name and a configuration migration field.
The migration information configuration request refers to a request for configuring information related to migration data. The configuration immigration data table name is a table name of an immigration data table which is configured by a developer autonomously and used for storing migrated data. The configuration immigration field is an immigration field configured by a developer autonomously to store migrated data.
Specifically, a developer may enter the autonomic configuration interface by clicking an autonomic configuration button on the visual operation interface, enter the configuration immigration data table name and the configuration immigration field on the autonomic configuration interface, and click a confirmation button, so that the client sends the formed immigration information configuration request to the server, and the server may obtain the immigration information configuration request. It can be understood that the migration information configuration request can clearly and intuitively reflect that in the data migration process, the developer wants to store the configuration migration data table and the configuration migration field of the migrated data, which may be the same as the data table to be migrated and the field to be migrated, and reflect that the developer wants to transfer the migrated data to the existing data table to be migrated and the field to be migrated in the target database; or the target migration data and the target migration field are different from the data table to be migrated and reflect that a developer wants to newly build the corresponding target migration data and the corresponding target migration field in the target database.
S402: and if the to-be-migrated data table corresponding to the name of the configuration migration data table does not exist in at least one to-be-migrated data table, judging whether the configuration migration field comprises a unique index field.
Wherein a unique index field may be understood as an index that may uniquely identify a particular piece of data. For example, the data such as the identification number of the user, the bank card number, the insurance policy number and the like can be unique indexes.
Specifically, the to-be-migrated data table corresponding to the configured migration data table name does not exist in the to-be-migrated data table, that is, the target migration data table cannot be directly determined according to the configured migration data table name, and a target migration data table corresponding to the configured migration data table name needs to be created in the target database. Because the data must meet the constraint of the association relationship between the two data tables when migrating from the target migration data table to the target migration data table, the association relationship may be embodied that the two data tables, the target migration data table and the target migration data table, have the same field, and the content of the same field may uniquely determine one piece of data, that is, it is required to ensure that the two data tables, the target migration data table and the target migration data table, contain the same unique index field. Therefore, in order to ensure that the data migration is successful, whether a target migration data table corresponding to the configuration migration data table name can be created based on the configuration migration field can be determined by judging whether the configuration migration field comprises a unique index field.
S403: and if the configuration immigration field comprises the unique index field, creating a target immigration data table corresponding to the configuration immigration data table name based on the configuration immigration field, and enabling the target immigration data table to carry the newly-built label.
The new label is a label for indicating that the target immigration data table is the new data table.
Specifically, if at least one to-be-migrated data table does not have a to-be-migrated data table corresponding to the configured migration data table name, and the configured migration field in the migration information configuration request includes a unique index field, a preset data table creation statement template is called, the configured migration data table name and the configured migration field replace corresponding form parameters in the data table creation statement template to form a target data table creation statement, the target data table creation statement is executed, so that a target migration data table corresponding to the configured migration data table name is created, and the configured migration field is determined as the target migration field in the target migration data table. For example, the data table creation statement template is create table XX (aa type 1auto _ increment primary; bb type 2 … … zz type z), XX may be replaced by a configuration immigration data table name, aa, bb, zz, and the like are replaced by a configuration immigration field to obtain a target data table creation statement, and the statement is created according to the target data table, that is, a target immigration data table corresponding to the configuration immigration data table name may be created in the target database, so as to ensure that the target database for implementing data immigration is stored in the target database. Therefore, the new label is carried in the target immigration data table, so that the subsequent data migration operation is facilitated.
It can be understood that, if at least one to-be-migrated data table does not have a to-be-migrated data table corresponding to the configured migration data table name, and the configured migration field in the migration information configuration request does not include the unique index field, modification reminding information is generated to remind a developer to modify the configured migration field, thereby ensuring that the target migration data table is generated. It can be understood that, when there is no to-be-migrated data table corresponding to the configured migration data table name in at least one to-be-migrated data table, it indicates that a developer wants to create a target migration data table for storing migrated data in the target database, and data having storage significance is stored for ensuring the created target migration data table, so that when it is determined that the configured migration field does not include the unique index field, the developer is prompted to modify the configured migration field through the generated modification prompting information.
Correspondingly, after step S401, that is, after acquiring the migration information configuration request, the data migration method further includes:
s404: and if the to-be-migrated data table corresponding to the configured migration data table name exists in at least one to-be-migrated data table, determining the to-be-migrated data table as a target migration data table, and determining the configured migration field as a target migration field.
It is understood that the configuration immigration field may be the same as or different from the existing field to be immigrated in the target immigration data table. And when the data table to be migrated corresponding to the configured migrated data table name exists in at least one data table to be migrated, directly determining the data table to be migrated as a target migrated data table. After the target immigration data table is determined, comparing a field to be immigrated in the target immigration data table with a configured immigration field; if the field to be migrated in the target migration data table has a field corresponding to the configured migration field, the field to be migrated in the target migration data table can be directly determined as the target migration field, so that the acquisition efficiency of the target migration field is improved; if the field to be migrated in the target migration data table does not have the field corresponding to the configured migration field, processing the configured migration field through a preset field adding statement template to obtain a target field adding statement, namely creating the configured migration field in the target migration data table and determining the configured migration field as the target migration field. The field addition statement template may be an alter table name add (field 1 type (length), field 2 type (length), and field 3 type (length)). In this embodiment, since the to-be-migrated data table is an existing data table, it should include the unique index field, and the target migration field may not include the unique index field.
In the data migration method provided in this embodiment, by acquiring the name of the configured immigration data table and the configured immigration field, it is determined whether a to-be-immigration data table corresponding to the name of the configured immigration data table exists, and if the to-be-immigration data table corresponding to the name of the configured immigration data table exists, the to-be-immigration data table is determined as the target immigration data table, so as to accelerate the acquisition efficiency of the target immigration data table, and the configured immigration field is determined as the target immigration field, thereby ensuring that data migration can be performed subsequently. And if the data table to be migrated corresponding to the configured migrated data table name does not exist and the configured migrated field comprises the unique index field, creating a target migrated data table corresponding to the configured migrated data table name based on the configured migrated field, ensuring that the target database stores the target migrated data table for realizing data migration, and facilitating subsequent data migration operation.
In an embodiment, as shown in fig. 5, in step S205, that is, based on the migration rule editing interface, obtaining the target migration scheme corresponding to the target migration field includes:
s501: and acquiring a preselected migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the preselected migration scheme comprises a preselected migration rule determined based on at least one original migration rule, a preselected migration data table determined based on at least one to-be-migrated data table and a preselected migration field determined based on at least one to-be-migrated field.
The pre-selected migration rule is an original migration rule which is autonomously selected and determined by a developer through a migration rule editing interface of the client. The pre-selected data sheet is a data sheet to be migrated which is automatically selected and determined by a developer through a migration rule editing interface of the client. And the pre-selected emigration field is a field to be emigration which is autonomously selected and determined by a developer through a migration rule editing interface of the client. As can be appreciated, the server obtains the pre-selected migration schemes corresponding to the target migration-in fields based on the migration rule editing interface, so as to quickly determine the target migration schemes from the pre-selected migration schemes, thereby implementing data migration.
S502: and if the target immigration data table carries the newly-built label, determining the preselected migration scheme as the target migration scheme.
It can be understood that if the new tag is carried in the target immigration data table, it indicates that the target immigration data table is the new data table, and when data is immigrated from the pre-selected immigration data table to the target immigration data table, the data is not constrained by the association relationship of the existing data in the target immigration data table, so that the association relationship check is not required, and the determination efficiency of the target migration scheme is improved.
S503: and if the target immigration data table does not carry the newly-built tag and the field content corresponding to the preselected immigration field corresponds to the field content of the unique index field in the target immigration data table, determining the preselected migration scheme as the target migration scheme.
If the target immigration data table does not carry the newly-built tag, the target immigration data table is the existing data table, when data is immigrated from the preselected immigration data table to the target immigration data table, the data is restrained by an incidence relation between the two data tables, the incidence relation can be embodied that the two data tables have the same field, and the content of the same field can uniquely determine one piece of data.
Specifically, if the field content of the preselected migration field corresponds to the field content of the unique index field in the target migration data table, and the preselected migration field conforms to the constraint of the association relationship between the two data tables, the preselected migration scheme is determined as the target migration scheme, so as to ensure the smooth operation of the data migration process. For example, the selected pre-selected immigration field in the pre-selected immigration data table a includes a policy field for storing a policy number, a name field for storing a name, and a time field for storing an order time, and the target immigration data table includes a policy field (a unique index field, other field names may be used) for storing a policy number and a predetermined target immigration field, at which time it is assumed that the field contents corresponding to the presence of the pre-selected immigration field correspond to the field contents of the unique index field in the target immigration data table.
S504: and if the new label is not carried by the target immigration data table, and the field content corresponding to the non-preselected immigration field does not correspond to the field content of the unique index field in the target immigration data table, generating modification reminding information.
The modification reminding information is information which is sent to developers and reminds the developers to modify the preselected emigration field. And modifying the preselected migration field by the developer according to the modification reminding information to ensure that a target migration scheme is generated so as to perform data migration. For example, the preselected immigration field selected in the preselected immigration data table a includes a policy field for storing a policy number, a name field for storing a name, and a time field for storing an order time, and the target immigration data table includes an identification number field (a unique index field, other field names may be used) for storing an identification number and a predetermined target immigration field.
It can be understood that, if the target immigration data table does not carry the newly created tag, and there is no field content corresponding to the preselected immigration field corresponding to the field content of the unique index field in the target immigration data table, that is, the preselected immigration field in the preselected immigration scheme is a useless field. For example, selected pre-selected immigration fields in pre-selected immigration data table a include a name field for storing a name and a time field for storing an order time. At this time, if the target immigration data table includes the policy field (unique index field) and the target immigration field, only the name field and the time field are processed by using the preselected migration rule, and the formed data does not include data corresponding to the unique index field, so that the data does not have uniqueness and can be understood as meaningless data, and therefore, a developer needs to be reminded to modify the preselected immigration field.
In the data migration method provided in this embodiment, a preselected migration scheme corresponding to a target immigration field is obtained based on a migration rule editing interface, and if a newly-built tag is carried by a target immigration data table, or a newly-built tag is not carried by a target immigration data table and there is a field content corresponding to the preselected immigration field corresponding to a field content of a unique index field in the target immigration data table, the preselected migration scheme is determined as the target migration scheme, so as to quickly obtain the target migration scheme, and improve the determination efficiency of the target migration scheme, so as to perform data migration subsequently. And if the target immigration data table does not carry the newly-built label and the field content corresponding to the preselected immigration field does not exist corresponds to the field content of the unique index field in the target immigration data table, generating modification reminding information, and modifying the preselected immigration field according to the modification reminding information, so that errors of a finally generated target migration scheme are avoided, and the smooth data migration process is ensured.
In an embodiment, as shown in fig. 6, in step S207, executing a data migration code, migrating data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to a target migration rule, and obtaining a data migration result, where the data migration result includes:
s601: and executing a data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to a target migration rule, and acquiring the pre-submitted migration data.
The pre-submitted migration data refers to data to be migrated which is pre-submitted to the target database. Specifically, transaction management of the target database is utilized to execute a data migration code, data to be migrated (namely, field content) corresponding to the target migrated field is processed according to a target migration rule, and the processed data to be migrated is migrated to a filling area corresponding to the target migrated field to form pre-submitted migration data, wherein the pre-submitted migration data is not formally submitted to the target database. The operation is only to pre-submit the data to be migrated to the target migration database, obtain the pre-submitted migration data, to determine whether the pre-submitted migration data is accurate, and actually, the data to be migrated is not updated to the filling area.
S602: and verifying the pre-submitted migration data by adopting a data verification interface corresponding to the target migration rule to obtain a data verification result.
The data verification interface is an interface used for verifying the pre-submitted migration data. The data check interface may be an interface for checking whether the pre-committed migration data on the filler area conforms to a data type, a data length, or other data information corresponding to the target migration rule. The data verification result refers to a result of whether the pre-submitted migration data verified by the data verification interface is correct or not. It can be understood that the data verification interface corresponds to the target migration rule, so that the data verification result has referential property, and the result of pre-submitting migration data is ensured to be identical with the real data migration result.
S603: and if the data verification result is that the verification is passed, submitting the pre-submitted migration data to a target database, and acquiring the data migration result.
When the data verification result is that the verification is passed, the test that the pre-submitted migration data passes through the data verification interface is proved to have accuracy, at the moment, a commit (commit) operation in the transaction management of the target database is executed, and the pre-submitted migration data is submitted to the target database, so that a correct data migration result is obtained, and the error rate of data migration is reduced.
And performing pre-submission processing in the target database, namely filling the data to be migrated into a filling area corresponding to a target migration field in a target migration data table, performing operation by using transaction management of the target database, wherein the operation is only pre-submitted into the target database and is not updated actually, and only by clicking a submit button on an interface, the submission operation on the target database is triggered, the pre-submitted migration data is submitted into the target database, and a data migration result is obtained.
S604: and if the data verification result is that the verification fails, executing a preset rollback mechanism, and rolling back to the position before executing the data migration code.
The rollback mechanism is a mechanism for triggering the program to return to the state before the data migration code is executed when the data verification result fails. Specifically, if the data verification result is that the verification fails, a preset rollback mechanism is executed, and the data is rolled back to a position before the data migration code is executed, so that a developer can check and modify the data in the data migration process, and the accuracy of the data migration is ensured. For example, the rollback mechanism may be a return instruction, and when the data check result is obtained as a check failure, the return instruction is triggered to a position before the data migration code is executed, so as to ensure data consistency.
When the pre-submission operation is carried out on the target database, error check can be carried out on the pre-submitted migration data which is pre-submitted to the target database so as to ensure the accuracy of the data which is finally uploaded to the target database, and if the data check result is that the check does not pass, a rollback mechanism is executed. The error check mainly depends on the control of a database, when the pre-submission operation is executed, the pre-submission migration data generated by the system is real and write-once operation is executed in the database, so that the conflict of field length control and a main key can be immediately found and returned to report errors, the system analyzes error codes and error information of the database and then intuitively returns the error information on a visual operation interface. For example, if the length of the field of the migrated table is not enough, the error is reported in the execution process, the error is fed back by the database, and based on the error information, the mapping rule can be located. If no problem exists in the execution process, namely the data verification result is that the verification is passed, the migration is successful only by clicking submission by a developer finally.
In the data migration method provided in this embodiment, a data migration code is executed in the data migration system, data pre-submission is performed, and pre-submitted migration data is acquired, so as to verify whether the pre-submitted migration data is correct, and ensure accuracy of the finally migrated pre-submitted migration data. If the data verification result of the pre-submitted migration data is that the verification is passed, submitting operation is executed to obtain a data migration result so as to realize rapid migration of the data and ensure the accuracy of the data migration; and if the data verification result of the pre-submitted migration data is that the verification is failed, executing rollback operation, rolling back all the data to the position before executing the data migration code, and returning execution error information so as to facilitate the detection and correction of developers and avoid errors of the migration data.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by functions and internal logic of the process, and should not limit the implementation process of the embodiments of the present invention in any way.
In an embodiment, a data migration apparatus is provided, where the data migration apparatus corresponds to the data migration method in the foregoing embodiment one to one. As shown in fig. 7, the data migration apparatus includes a target database determination module 701, a data display module 702, a target migration field determination module 703, a migration rule edit request module 704, a target migration scheme acquisition module 705, a data migration code acquisition module 706, and a data migration result acquisition module 707. The functional modules are explained in detail as follows:
and a target database determining module 701, configured to receive the data migration instruction, and determine, based on the data migration instruction, a target database whose access right is a read-write right.
The data display module 702 is configured to display, in a hierarchical structure display manner on the visual operation interface, at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table.
The target immigration field determining module 703 is configured to determine a target immigration data table based on the at least one to-be-immigration data table, and determine a target immigration field based on the at least one to-be-immigration field.
The migration rule edit request module 704 is configured to receive a migration rule edit request, enter a migration rule edit interface corresponding to a target migration field based on the migration rule edit request, and display at least one original migration rule and at least one to-be-migrated data table in a source database having a read right and at least one to-be-migrated field corresponding to each to-be-migrated data table on the migration rule edit interface.
The target migration scheme obtaining module 705 is configured to obtain, based on the migration rule editing interface, a target migration scheme corresponding to the target migration field, where the target migration scheme includes a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table, and a target migration field determined based on at least one to-be-migrated field.
And the data migration code acquisition module 706 is configured to acquire a corresponding target code generation template according to the target migration rule, process the target migration data table, the target migration field, the target migration data table, and the target migration field by using the target code generation template, and acquire a data migration code corresponding to the target migration rule.
The data migration result obtaining module 707 is configured to execute the data migration code, migrate the data to be migrated corresponding to the target migration field to the filling area corresponding to the target migration field according to the target migration rule, and obtain the data migration result.
Preferably, the data display module 702 includes: the system comprises an original folder acquisition unit, a to-be-migrated data table acquisition unit, a to-be-migrated field acquisition unit and a data display unit.
And the original folder acquisition unit is used for traversing the folders in the target data table by adopting a folder traversal script to acquire at least one original folder.
And the data table to be migrated acquiring unit is used for traversing the data table in each original folder by adopting a data table traversal script and acquiring at least one data table to be migrated corresponding to each original folder.
And the field to be migrated acquiring unit is used for traversing the field in each data table to be migrated by adopting the field traversal script and acquiring at least one field to be migrated corresponding to each data table to be migrated.
And the data display unit is used for displaying at least one original folder, at least one to-be-migrated data table and at least one to-be-migrated field corresponding to the target database in a hierarchical structure display mode on the visual operation interface in a page asynchronous loading mode.
Preferably, the target immigration field determining module 703 includes: the system comprises a migration information configuration request unit, a first judgment processing unit and a target migration data table creating unit.
And the immigration information configuration request unit is used for acquiring an immigration information configuration request, and the immigration information configuration request comprises a configuration immigration data table name and a configuration immigration field.
The first judging and processing unit is used for judging whether the configuration immigration field comprises a unique index field or not if the data table to be immigrated corresponding to the configuration immigration data table name does not exist in at least one data table to be immigrated.
And the target immigration data table creating unit is used for creating a target immigration data table corresponding to the configured immigration data table name based on the configured immigration field if the configured immigration field comprises the unique index field, so that the target immigration data table carries the newly-built label.
Preferably, after migrating the information configuration request unit, the data migration apparatus further includes: a second judgment processing unit.
And the second judgment processing unit is used for determining the data table to be migrated as a target migration data table and determining the configured migration field as the target migration field if the data table to be migrated corresponding to the configured migration data table name exists in the at least one data table to be migrated.
Preferably, the target migration scheme obtaining module 705 includes: a pre-selected migration scheme acquisition unit, a first migration scheme determination unit, and a second migration scheme determination unit.
And the preselected migration scheme acquisition unit is used for acquiring a preselected migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the preselected migration scheme comprises a preselected migration rule determined based on at least one original migration rule, a preselected migration data table determined based on at least one to-be-migrated data table and a preselected migration field determined based on at least one to-be-migrated field.
And the first migration scheme determining unit is used for determining the preselected migration scheme as the target migration scheme if the target migration data table carries the newly-built tag.
And the second migration scheme determining unit is used for determining the preselected migration scheme as the target migration scheme if the target migration data table does not carry the newly-built tag and the field content of the preselected migration field corresponds to the field content of the unique index field in the target migration data table.
Preferably, the data migration result obtaining module 707 includes: the device comprises a pre-submission processing unit, a data verification result acquisition unit and a data submission unit.
And the pre-submission processing unit is used for executing the data migration code, migrating the data to be migrated corresponding to the target migration field to the filling area corresponding to the target migration field according to the target migration rule, and acquiring the pre-submission migration data.
And the data verification result acquisition unit is used for verifying the pre-submitted migration data by adopting a data verification interface corresponding to the target migration rule to acquire a data verification result.
And the data submitting unit is used for submitting the pre-submitted migration data to the target database to obtain a data migration result if the data verification result is that the verification is passed.
Preferably, after the data verification result obtaining unit, the data migration apparatus further includes: the rollback mechanism executes the unit.
And the rollback mechanism execution unit is used for executing a preset rollback mechanism if the data verification result is that the verification fails, and rolling back to the position before the data migration code is executed.
For specific limitations of the data migration apparatus, reference may be made to the above limitations of the data migration method, which will not be described herein again. The modules in the data migration apparatus may be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for executing data adopted or generated in the data migration method process, such as a target migration data table. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data migration method.
In an embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the data migration method in the foregoing embodiments is implemented, for example, in S201-S207 shown in fig. 2, or in fig. 3 to 6, which is not described herein again to avoid repetition. Alternatively, the processor implements the functions of each module/unit in the embodiment of the data migration apparatus when executing the computer program, for example, the functions of the target database determining module 701, the data displaying module 702, the target immigration field determining module 703, the migration rule editing request module 704, the target migration scheme obtaining module 705, the data migration code obtaining module 706, and the data migration result obtaining module 707 shown in fig. 7, which are not described herein again to avoid repetition.
In an embodiment, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the data migration method in the foregoing embodiments is implemented, for example, S201 to S207 shown in fig. 2, or shown in fig. 3 to fig. 6, which is not described herein again to avoid repetition. Alternatively, the processor implements the functions of each module/unit in the embodiment of the data migration apparatus when executing the computer program, for example, the functions of the target database determining module 701, the data displaying module 702, the target immigration field determining module 703, the migration rule editing request module 704, the target migration scheme obtaining module 705, the data migration code obtaining module 706, and the data migration result obtaining module 707 shown in fig. 7, which are not described herein again to avoid repetition.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (9)

1. A method of data migration, comprising:
receiving a data migration instruction, and determining a target database with access authority as read-write authority based on the data migration instruction;
displaying at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table in a hierarchical structure display mode on a visual operation interface;
determining a target migration data table based on at least one to-be-migrated data table, and determining a target migration field based on at least one to-be-migrated field;
receiving a migration rule editing request, and entering a migration rule editing interface corresponding to the target migration field based on the migration rule editing request, wherein the migration rule editing interface displays at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table;
acquiring a target migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the target migration scheme comprises a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table and a target migration field determined based on at least one to-be-migrated field;
acquiring a corresponding target code generation template according to the target migration rule, and processing the target migration data table, the target migration field, the target migration data table and the target migration field by adopting the target code generation template to acquire a data migration code corresponding to the target migration rule;
executing the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring a data migration result;
the obtaining of the target migration scheme corresponding to the target migration field based on the migration rule editing interface includes:
acquiring a preselected migration scheme corresponding to the target migration field based on the migration rule editing interface, wherein the preselected migration scheme comprises a preselected migration rule determined based on at least one original migration rule, a preselected migration data table determined based on at least one to-be-migrated data table and a preselected migration field determined based on at least one to-be-migrated field;
if the target immigration data table carries the newly-built label, determining the preselected migration scheme as a target migration scheme;
and if the target immigration data table does not carry the newly-built tag and the field content of the preselected immigration field corresponds to the field content of the unique index field in the target immigration data table, determining the preselected migration scheme as the target migration scheme.
2. The data migration method according to claim 1, wherein the displaying, in a hierarchical display manner on a visual operation interface, at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table includes:
traversing the folders in the target data table by adopting a folder traversal script to obtain at least one original folder;
traversing the data table in each original folder by adopting a data table traversal script to acquire at least one data table to be migrated corresponding to each original folder;
traversing the fields in each data table to be migrated by adopting a field traversal script, and acquiring at least one field to be migrated corresponding to each data table to be migrated;
and displaying at least one original folder, at least one data table to be migrated and at least one field to be migrated corresponding to the target database in a hierarchical structure display mode on a visual operation interface by adopting a page asynchronous loading mode.
3. The data migration method according to claim 1, wherein said determining a target migration data table based on at least one of said to-be-migrated data tables and determining a target migration field based on at least one of said to-be-migrated fields comprises:
acquiring a migration information configuration request, wherein the migration information configuration request comprises a configuration migration data table name and a configuration migration field;
if at least one data table to be migrated does not have a data table to be migrated corresponding to the name of the configuration data table to be migrated, judging whether the configuration migration field comprises a unique index field;
and if the configuration immigration field comprises a unique index field, creating a target immigration data table corresponding to the configuration immigration data table name based on the configuration immigration field, and enabling the target immigration data table to carry a newly-built label.
4. The data migration method according to claim 3, wherein after the obtaining migration information configuration request, the data migration method further comprises:
if the to-be-migrated data table corresponding to the configured migration data table name exists in at least one to-be-migrated data table, determining the to-be-migrated data table as a target migration data table, and determining the configured migration field as a target migration field.
5. The data migration method according to claim 1, wherein the executing the data migration code to migrate the data to be migrated corresponding to the target migration field to the filling area corresponding to the target migration field according to the target migration rule, and obtain the data migration result includes:
executing the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring pre-submitted migration data;
verifying the pre-submitted migration data by adopting a data verification interface corresponding to the target migration rule to obtain a data verification result;
and if the data verification result is that the verification is passed, submitting the pre-submitted migration data to the target database, and acquiring a data migration result.
6. The data migration method according to claim 5, wherein after said obtaining the data verification result, the data migration method further comprises:
and if the data verification result is that the verification is not passed, executing a preset rollback mechanism, rolling back to execute the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring the data migration result before.
7. A data migration apparatus, comprising:
the target database determining module is used for receiving a data migration instruction and determining a target database with access right as read-write right based on the data migration instruction;
the data display module is used for displaying at least one to-be-migrated data table corresponding to the target database and at least one to-be-migrated field corresponding to each to-be-migrated data table in a hierarchical structure display mode on a visual operation interface;
the target migration field determining module is used for determining a target migration data table based on at least one to-be-migrated data table and determining a target migration field based on at least one to-be-migrated field;
the migration rule editing request module is used for receiving a migration rule editing request, entering a migration rule editing interface corresponding to the target migration field based on the migration rule editing request, wherein the migration rule editing interface displays at least one original migration rule and at least one to-be-migrated data table in a source database with read permission and at least one to-be-migrated field corresponding to each to-be-migrated data table;
a target migration scheme obtaining module, configured to obtain, based on the migration rule editing interface, a target migration scheme corresponding to the target migration field, where the target migration scheme includes a target migration rule determined based on at least one original migration rule, a target migration data table determined based on at least one to-be-migrated data table, and a target migration field determined based on at least one to-be-migrated field;
a data migration code obtaining module, configured to obtain a corresponding target code generation template according to the target migration rule, process the target migration data table, the target migration field, the target migration data table, and the target migration field by using the target code generation template, and obtain a data migration code corresponding to the target migration rule;
the data migration result acquisition module is used for executing the data migration code, migrating the data to be migrated corresponding to the target migration field to a filling area corresponding to the target migration field according to the target migration rule, and acquiring a data migration result;
wherein the target migration scheme acquisition module comprises:
a preselected migration scheme obtaining unit, configured to obtain, based on the migration rule editing interface, a preselected migration scheme corresponding to the target migration field, where the preselected migration scheme includes a preselected migration rule determined based on at least one original migration rule, a preselected migration data table determined based on at least one to-be-migrated data table, and a preselected migration field determined based on at least one to-be-migrated field;
a first migration scheme determining unit, configured to determine the preselected migration scheme as a target migration scheme if the target migration data table carries the newly-created tag;
and the second migration scheme determining unit is used for determining the preselected migration scheme as the target migration scheme if the target migration data table does not carry the newly-built tag and the field content of the preselected migration field corresponds to the field content of the unique index field in the target migration data table.
8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data migration method according to any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the data migration method according to any one of claims 1 to 6.
CN201910430064.8A 2019-05-22 2019-05-22 Data migration method and device, computer equipment and storage medium Active CN110297813B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910430064.8A CN110297813B (en) 2019-05-22 2019-05-22 Data migration method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910430064.8A CN110297813B (en) 2019-05-22 2019-05-22 Data migration method and device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110297813A CN110297813A (en) 2019-10-01
CN110297813B true CN110297813B (en) 2022-09-02

Family

ID=68027056

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910430064.8A Active CN110297813B (en) 2019-05-22 2019-05-22 Data migration method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110297813B (en)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110784353B (en) * 2019-10-30 2022-06-28 中信科移动通信技术股份有限公司 Network element equipment configuration data migration method and device
CN112817930A (en) * 2019-11-15 2021-05-18 北京沃东天骏信息技术有限公司 Data migration method and device
CN112948472A (en) * 2019-11-26 2021-06-11 北京沃东天骏信息技术有限公司 Data processing method, device, equipment and computer readable medium
CN111125213A (en) * 2019-11-29 2020-05-08 北京数起科技有限公司 Data acquisition method, device and system
CN111177113B (en) * 2019-12-06 2023-07-25 中国平安财产保险股份有限公司 Data migration method, device, computer equipment and storage medium
CN111125059B (en) * 2019-12-17 2022-08-12 南京甄视智能科技有限公司 Data migration method and device, storage medium and server
CN111008037A (en) * 2019-12-17 2020-04-14 苏州浪潮智能科技有限公司 Code migration method, equipment and storage medium
CN111104393B (en) * 2019-12-17 2024-02-20 深圳追一科技有限公司 Data migration method, device, electronic equipment and storage medium
CN111177246B (en) * 2019-12-27 2023-03-14 腾讯云计算(北京)有限责任公司 Service data processing method and device
CN111367889B (en) * 2020-03-09 2023-08-04 中国工商银行股份有限公司 Cross-cluster data migration method and device based on webpage interface
CN111415749A (en) * 2020-03-12 2020-07-14 深圳中兴网信科技有限公司 Information processing method, information processing apparatus, and computer-readable storage medium
CN111596899A (en) * 2020-05-12 2020-08-28 重庆锐云科技有限公司 Database migration method, system, equipment and storage medium based on Java development
CN111694517B (en) * 2020-05-28 2024-02-13 新浪技术(中国)有限公司 Distributed data migration method, system and electronic equipment
CN111680023B (en) * 2020-06-08 2023-07-21 中国银行股份有限公司 Migration data checking method and device
CN111522798B (en) * 2020-06-18 2020-10-23 腾讯科技(深圳)有限公司 Data synchronization method, device, equipment and readable storage medium
CN111782144A (en) * 2020-06-23 2020-10-16 上海传英信息技术有限公司 Intelligent terminal, data storage method and computer readable storage medium
CN111966665B (en) * 2020-08-17 2024-01-19 中国银行股份有限公司 Data migration testing method and device
CN111966597B (en) * 2020-08-18 2024-04-16 中国银行股份有限公司 Test data generation method and device
CN111984631A (en) * 2020-09-02 2020-11-24 深圳壹账通智能科技有限公司 Production data migration method and device, computer equipment and storage medium
CN112148713B (en) * 2020-09-29 2024-04-16 中国银行股份有限公司 Method and device for data migration between heterogeneous databases
CN112632032B (en) * 2020-12-18 2022-12-27 华人运通(上海)云计算科技有限公司 Data migration method and device, storage medium and terminal equipment
CN112817935A (en) * 2021-01-22 2021-05-18 平安普惠企业管理有限公司 Data processing method, device and equipment based on field type and storage medium
CN112860666A (en) * 2021-02-08 2021-05-28 土巴兔集团股份有限公司 Data migration method and device, computer equipment and storage medium
CN113342823A (en) * 2021-06-30 2021-09-03 招商局金融科技有限公司 Data changing method and device, computer equipment and storage medium
CN113901031A (en) * 2021-10-21 2022-01-07 掌阅科技股份有限公司 Database migration method, electronic device and storage medium
CN116431688B (en) * 2022-11-14 2024-05-03 北京远舢智能科技有限公司 Data processing method and device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108604278A (en) * 2015-10-23 2018-09-28 甲骨文国际公司 Self-described configuration with the support to shared data table
CN108647279A (en) * 2018-05-03 2018-10-12 山东浪潮通软信息科技有限公司 Sheet disposal method, apparatus, medium and storage control based on field multiplexing
CN109241029A (en) * 2018-08-08 2019-01-18 新智能源系统控制有限责任公司 A kind of method and apparatus for realizing SCADA system data library smooth migration
CN109359086A (en) * 2018-08-31 2019-02-19 中国科学院计算技术研究所 A kind of fast state services migrating method and system based on dynamic Compound RPC
CN109376033A (en) * 2018-09-21 2019-02-22 平安科技(深圳)有限公司 Snap backup method, device, computer equipment and storage medium

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101227424A (en) * 2007-12-20 2008-07-23 腾讯科技(深圳)有限公司 Information exhibiting method as well as subscriber terminal
CN101714151B (en) * 2009-10-28 2012-07-04 福建新大陆通信科技股份有限公司 SQL database-based set top box data storage method
US9176997B2 (en) * 2013-06-28 2015-11-03 Sap Se Universe migration from one database to another
CN106055670A (en) * 2016-06-06 2016-10-26 中国工商银行股份有限公司 Inter-system data migration method and device
US10866938B2 (en) * 2016-07-29 2020-12-15 Sap Se Automated reshaping and restructuring of storage structures
US11314799B2 (en) * 2016-07-29 2022-04-26 Splunk Inc. Event-based data intake and query system employing non-text machine data
CN106407404B (en) * 2016-09-22 2019-09-24 成都快乐家网络技术有限公司 Date storage method, data managing method and system, database, client
CN106776876A (en) * 2016-11-29 2017-05-31 用友网络科技股份有限公司 Data migration method and data mover system
US10437799B2 (en) * 2016-12-02 2019-10-08 International Business Machines Corporation Data migration using a migration data placement tool between storage systems based on data access
CN106874389B (en) * 2017-01-11 2023-04-07 腾讯科技(深圳)有限公司 Data migration method and device
US11269822B2 (en) * 2017-10-09 2022-03-08 Sap Se Generation of automated data migration model
US10649978B2 (en) * 2017-11-08 2020-05-12 International Business Machines Corporation Preview processing and automatic correction of bulk data for software as a service applications
CN109542882B (en) * 2018-12-05 2020-11-06 南京中孚信息技术有限公司 Database migration method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108604278A (en) * 2015-10-23 2018-09-28 甲骨文国际公司 Self-described configuration with the support to shared data table
CN108647279A (en) * 2018-05-03 2018-10-12 山东浪潮通软信息科技有限公司 Sheet disposal method, apparatus, medium and storage control based on field multiplexing
CN109241029A (en) * 2018-08-08 2019-01-18 新智能源系统控制有限责任公司 A kind of method and apparatus for realizing SCADA system data library smooth migration
CN109359086A (en) * 2018-08-31 2019-02-19 中国科学院计算技术研究所 A kind of fast state services migrating method and system based on dynamic Compound RPC
CN109376033A (en) * 2018-09-21 2019-02-22 平安科技(深圳)有限公司 Snap backup method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN110297813A (en) 2019-10-01

Similar Documents

Publication Publication Date Title
CN110297813B (en) Data migration method and device, computer equipment and storage medium
CN110209650B (en) Data normalization and migration method and device, computer equipment and storage medium
US9836297B2 (en) Computer implemented method and system for automatically deploying and versioning scripts in a computing environment
CN109308285A (en) Database script management method, device, computer equipment and storage medium
CN110941630A (en) Database operation and maintenance method, device and system
WO2019037418A1 (en) Code management method and apparatus, computer device and computer-readable storage medium
CN109657177A (en) The generation method of the page, device, storage medium and computer equipment after upgrading
CN109493048A (en) Financial accounting method, apparatus, equipment and storage medium based on block chain
CN107169000A (en) Static resource dissemination method and device
US20100106541A1 (en) Analyzing the Readiness of a Template
CN116560683A (en) Software updating method, device, equipment and storage medium
US11741000B2 (en) Method and system for verifying resulting behavior of graph query language
EP2797001A2 (en) System and method for creating variants in a test database during various test stages
CN111522881B (en) Service data processing method, device, server and storage medium
US11294666B1 (en) Data structures for managing configuration versions of cloud-based applications
CN112215015A (en) Translation text revision method, translation text revision device, computer equipment and storage medium
US10838947B2 (en) Consistency check for foreign key definition
US20230195792A1 (en) Database management methods and associated apparatus
CN115934129A (en) Software project updating method and device, computer equipment and storage medium
CN115185821A (en) Version labeling method, system, equipment and storage medium in program test
US11301246B2 (en) Automatically generating continuous integration pipelines
CN112435007A (en) Js-based code submission method, device, computer equipment and storage medium
CN114116664A (en) Database table building statement processing method and device, computer equipment and storage medium
CN112434505A (en) Template generation method and device, computer equipment and storage medium
CN113448617B (en) Data object management method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant