CN112380186A - Domestic database migration method based on active adaptation - Google Patents

Domestic database migration method based on active adaptation Download PDF

Info

Publication number
CN112380186A
CN112380186A CN202011229058.5A CN202011229058A CN112380186A CN 112380186 A CN112380186 A CN 112380186A CN 202011229058 A CN202011229058 A CN 202011229058A CN 112380186 A CN112380186 A CN 112380186A
Authority
CN
China
Prior art keywords
database
data
migration
domestic
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011229058.5A
Other languages
Chinese (zh)
Inventor
董昌朝
庄莉
梁懿
陈又咏
林君君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Information and Telecommunication Co Ltd
Fujian Yirong Information Technology Co Ltd
Great Power Science and Technology Co of State Grid Information and Telecommunication Co Ltd
Original Assignee
State Grid Information and Telecommunication Co Ltd
Fujian Yirong Information Technology Co Ltd
Great Power Science and Technology Co of State Grid Information and Telecommunication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Information and Telecommunication Co Ltd, Fujian Yirong Information Technology Co Ltd, Great Power Science and Technology Co of State Grid Information and Telecommunication Co Ltd filed Critical State Grid Information and Telecommunication Co Ltd
Priority to CN202011229058.5A priority Critical patent/CN112380186A/en
Publication of CN112380186A publication Critical patent/CN112380186A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/214Database migration support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462Approximate or statistical queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a domestic database migration method based on active adaptation, which belongs to the field of computational service databases and comprises a source database, a database migration system and a database migration system, wherein a signal output end of the source database is in signal connection with a signal receiving end of the database migration system; the invention provides a one-stop service for migration between a non-domestic database and a domestic database, thereby greatly reducing the migration cost of the databases; meanwhile, by using a data log synchronization technology, the hot migration of the database can be realized without influencing the operation condition of the system, so that the system has wider applicability.

Description

Domestic database migration method based on active adaptation
Technical Field
The invention belongs to the technical field of computing service databases, and particularly relates to a domestic database migration method based on active adaptation.
Background
The domestic database migration is to migrate a traditional non-domestic relational database (such as Oracle) from a traditional X86 series CPU running environment to a domestic database supporting a domestic CPU and an operating system. The traditional migration method is that different migration tools are selected according to different domestic databases to perform database migration, manual repair is performed on a task which fails in migration, then comparison and verification are performed on data before and after migration, and finally performance tests of an application system and the databases are performed.
The traditional database migration has several defects, and the cold migration of the database causes that the service must be interrupted; the migration can not be carried out according to different domestic databases, and different migration tools are required to be selected; the migration process is unstable, and the database objects which are failed in migration need to be repaired after migration; the migration result cannot be actively verified after the migration is completed; performance tests cannot be performed on the migrated database.
Disclosure of Invention
The invention aims to provide a domestic database migration method based on active adaptation, aiming at solving the problems that the traditional database migration in the prior art has several defects, and the cold migration of the database causes that the service must be interrupted; the migration can not be carried out according to different domestic databases, and different migration tools are required to be selected; the migration process is unstable, and the database objects which are failed in migration need to be repaired after migration; the migration result cannot be actively verified after the migration is completed; and the performance test can not be carried out on the migrated database.
In order to achieve the purpose, the invention provides the following technical scheme:
a domestic database system based on active adaptation comprises a source database, a database migration system and a performance testing module, wherein a signal output end of the source database is in signal connection with a signal receiving end of the database migration system, a signal output end of the database migration system is in signal connection with the signal receiving end, the database migration system is used for migrating data in the source database into the database, and an active adaptation module, a data synchronization module, a grammar conversion module, a data verification module and a performance testing module are arranged in the database migration system.
As a preferred scheme of the invention, the method for migrating the domestic database based on active adaptation comprises the following steps:
s1: creating a user, database/mode in a target database;
s2: the data in the database is migrated through an actively-adaptive domestic database migration system;
s3: checking statistical data before and after migration after the migration is completed, and ensuring that all migration tasks are completed smoothly;
s4: the application system tests to ensure that the database has no abnormality in the use process of the application system;
s5: and (4) performance testing, wherein after the migration is finished, in order to ensure that the database system is stable and reliable, a migration tool is used for performing the performance testing of the domestic database.
As a preferred scheme of the present invention, the system further includes a first obtaining module, and the first obtaining unit is configured to obtain a difference file between a domestic database and a non-domestic database product after the domestic database is built, and then select data for migration.
As a preferable aspect of the present invention, in step S2, the migrated data is a database object such as a data table, a view, a stored procedure, a function, and the like.
As a preferable scheme of the present invention, the data synchronization module is adopted during the migration in step S2, so that the use of the database migration system is not easily affected during the migration of data.
As a preferred scheme of the present invention, in the step S2, when the data is migrated, the syntax incompatible item is replaced by the syntax conversion module to be applicable to the syntax of the domestic database, so that the migrated data can be better applicable to the domestic database.
As a preferred scheme of the present invention, after the data migration is completed in step S3, a data log synchronization function is started to ensure that the migrated database and the production library are synchronized.
As a preferred scheme of the present invention, the active adaptation module further includes a data acquisition module, an instance selection module, a parameter configuration module, an information acquisition module, a data export module, and a data import module.
As a preferred scheme of the present invention, an application method of a home-made database migration system based on active adaptation is as follows:
a: the active adaptation module adapts a mainstream non-domestic database for a source database, adapts a mainstream domestic database for a target database, acquires data of the source database from the non-domestic database to the domestic database through a program library, performs syntax conversion on the acquired data through a syntax conversion module, and then migrates the data into the source database;
b: after the migration is completed, the data verification module verifies the source database and the target database, and the database objects including data table data, data tables, views, storage processes, functions and the like are subjected to statistical comparison to obtain a conclusion;
c: the data synchronization module is used for ensuring that the transferred database and the production library are kept synchronous without influencing the continuous use of the production system;
d: the performance test module can be used for performing performance test on the target database and performing performance difference analysis on the target database and the source database.
As a preferred aspect of the present invention,
compared with the prior art, the invention has the beneficial effects that:
1. the invention provides one-stop service for migration between non-domestic databases and domestic databases, thereby greatly reducing the migration cost of the databases.
2. The invention can realize the hot migration of the database without influencing the running condition of the system by using the data log synchronization technology.
3. The invention provides a unified connection pool and a data fragment access strategy, and the data access performance is improved.
4. The invention provides the data verification and performance test after migration, and ensures the complete migration of the database and the reliability after migration.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a functional block diagram of a domestic data migration method based on active adaptation in the present invention;
FIG. 2 is a flow chart of a domestic data migration method based on active adaptation in the present invention;
fig. 3 is a functional block diagram of the active adaptation module of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1-3, the present invention provides the following technical solutions:
a home database system based on active adaptation is characterized by comprising a source database, a database migration system and a database, wherein a signal output end of the source database is in signal connection with a signal receiving end of the database migration system, a signal output end of the database migration system is in signal connection with the signal receiving end, the database migration system is used for migrating data in the source database into the database, an active adaptation module, a data synchronization module, a grammar conversion module, a data verification module and a performance test module are arranged in the database migration system, the active adaptation module adapts to a mainstream non-home database for the source database, adapts to a mainstream home database for a target database, acquires database objects such as a data table, a view, a storage process and a function in the source database through a data acquisition module, and replaces grammar incompatible items into grammar suitable for the home database through a grammar conversion module, the data after migration can be better suitable for a domestic database, a data synchronization module is adopted at the same time, the use of a database migration system is not easily influenced when the data are migrated, the collected data are exported through a data export module and then migrated into the data import module, after the data migration is completed, a data verification module verifies a source database and a target database after the migration is completed, and database objects including data of a data table, the data table, a view, a storage process, a function and the like are subjected to statistical comparison to obtain a conclusion; then, starting a data log synchronization function through a data synchronization module to ensure that the transferred database and the production library are kept synchronous; finally, the performance test of the target database is carried out through the database performance test, and the performance difference analysis is carried out with the source database, so that the one-stop service for the migration between the non-domestic database and the domestic database is provided, and the migration cost of the database is greatly reduced; meanwhile, by using a data log synchronization technology, the hot migration of the database can be realized without influencing the running condition of the system; a unified connection pool and a data fragment access strategy are provided, and the data access performance is improved; the data verification and performance test after migration is provided, and the complete migration of the database and the reliability after migration are guaranteed.
Specifically, referring to fig. 1, a method for migrating a domestic database based on active adaptation includes the following steps:
s1: creating a user, database/mode in a target database;
s2: the data in the database is migrated through an actively-adaptive domestic database migration system;
s3: checking statistical data before and after migration after the migration is completed, and ensuring that all migration tasks are completed smoothly;
s4: the application system tests to ensure that the database has no abnormality in the use process of the application system;
s5: and (4) performance testing, wherein after the migration is finished, in order to ensure that the database system is stable and reliable, a migration tool is used for performing the performance testing of the domestic database.
Specifically referring to fig. 3, the active adaptation module further includes a data acquisition module, an instance selection module, a parameter configuration module, an information acquisition module, a data export module, and a data import module.
In this embodiment: the system comprises a data acquisition module, an instance selection module and an instance selection unit, wherein the data acquisition module is used for acquiring data in a source database, and the instance selection module and the instance selection unit are used for selecting a source database instance needing to export data and a target database instance used for importing data; the instance selection module is used for configuring export parameters of the source database instance and import parameters of the target database instance; the information acquisition module is used for acquiring the related information of the source database instance and the target database instance from the database management platform; the data export module is used for exporting the data from the source database instance by utilizing the export parameters to generate backup data; and the data import module is used for importing the backup data into the target database instance by using the import parameters.
Specifically, an application method of a home database migration system based on active adaptation is as follows:
a: the active adaptation module adapts a mainstream non-domestic database for a source database, adapts a mainstream domestic database for a target database, acquires data of the source database from the non-domestic database to the domestic database through a program library, performs syntax conversion on the acquired data through a syntax conversion module, and then migrates the data into the source database;
b: after the migration is completed, the data verification module verifies the source database and the target database, and the database objects including data table data, data tables, views, storage processes, functions and the like are subjected to statistical comparison to obtain a conclusion;
c: the data synchronization module is used for ensuring that the transferred database and the production library are kept synchronous without influencing the continuous use of the production system;
d: the performance test module can be used for performing performance test on the target database and performing performance difference analysis on the target database and the source database.
The working principle and the using process of the invention are as follows: a method for migrating a domestic database based on active adaptation comprises the following steps:
s1: creating a user, database/mode in a target database;
s2: the data in the database is migrated through an actively-adaptive domestic database migration system;
s3: checking statistical data before and after migration after the migration is completed, and ensuring that all migration tasks are completed smoothly;
s4: the application system tests to ensure that the database has no abnormality in the use process of the application system;
s5: performing performance test, namely performing the performance test of the domestic database by using a migration tool in order to ensure the stability and reliability of the database system after the migration is finished; the invention provides a one-stop service for migration between a non-domestic database and a domestic database, thereby greatly reducing the migration cost of the databases; meanwhile, by using a data log synchronization technology, the hot migration of the database can be realized without influencing the operation condition of the system, so that the system has wider applicability.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A domestic database system based on active adaptation is characterized by comprising a source database, a database migration system and a data transmission system, wherein a signal output end of the source database is in signal connection with a signal receiving end of the database migration system, a signal output end of the database migration system is in signal connection with the signal receiving end, the database migration system is used for migrating data in the source database into the database, and an active adaptation module, a data synchronization module, a grammar conversion module, a data verification module and a performance test module are arranged in the database migration system.
2. The method for migrating the domestic database based on active adaptation according to claim 1, comprising the following steps:
s1: creating a user, database/mode in a target database;
s2: the data in the database is migrated through an actively-adaptive domestic database migration system;
s3: checking statistical data before and after migration after the migration is completed, and ensuring that all migration tasks are completed smoothly;
s4: the application system tests to ensure that the database has no abnormality in the use process of the application system;
s5: and (4) performance testing, wherein after the migration is finished, in order to ensure that the database system is stable and reliable, a migration tool is used for performing the performance testing of the domestic database.
3. The domestic database system based on active adaptation as claimed in claim 1, further comprising a first obtaining module, wherein said first obtaining unit is configured to obtain difference files between domestic database products and non-domestic database products after the domestic database is built, and then select data for migration.
4. The method for migrating domestic database based on active adaptation according to claim 3, wherein in step S2, the migrated data is database objects such as data tables, views, stored procedures, functions, etc.
5. The method for migrating the domestic database based on active adaptation according to claim 4, wherein a data synchronization module is adopted during the migration in step S2, so that the use of the database migration system is not easily affected during the migration of data.
6. The method for migrating domestic database based on active adaptation according to claim 5, wherein the syntax incompatible item is replaced by the syntax transformation module to be applicable to the syntax of the domestic database during data migration in step S2, so that the migrated data can be better applicable to the domestic database.
7. The method for migrating the domestic database based on active adaptation as claimed in claim 6, wherein after the data migration is completed in step S3, a data log synchronization function is started to ensure that the migrated database and the production library are kept synchronized.
8. The domestic database system based on active adaptation as claimed in claim 1, wherein the active adaptation module further comprises a data acquisition module, an instance selection module, a parameter configuration module, an information acquisition module, a data export module and a data import module.
9. The method for migrating the domestic database based on active adaptation according to claim 8, wherein the application method of the database migration system is as follows:
a: the active adaptation module adapts a mainstream non-domestic database for a source database, adapts a mainstream domestic database for a target database, acquires data of the source database from the non-domestic database to the domestic database through a program library, performs syntax conversion on the acquired data through a syntax conversion module, and then migrates the data into the source database;
b: after the migration is completed, the data verification module verifies the source database and the target database, and the database objects including data table data, data tables, views, storage processes, functions and the like are subjected to statistical comparison to obtain a conclusion;
c: the data synchronization module is used for ensuring that the transferred database and the production library are kept synchronous without influencing the continuous use of the production system;
d: the performance test module can be used for performing performance test on the target database and performing performance difference analysis on the target database and the source database.
CN202011229058.5A 2020-11-06 2020-11-06 Domestic database migration method based on active adaptation Pending CN112380186A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011229058.5A CN112380186A (en) 2020-11-06 2020-11-06 Domestic database migration method based on active adaptation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011229058.5A CN112380186A (en) 2020-11-06 2020-11-06 Domestic database migration method based on active adaptation

Publications (1)

Publication Number Publication Date
CN112380186A true CN112380186A (en) 2021-02-19

Family

ID=74579791

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011229058.5A Pending CN112380186A (en) 2020-11-06 2020-11-06 Domestic database migration method based on active adaptation

Country Status (1)

Country Link
CN (1) CN112380186A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113111050A (en) * 2021-04-27 2021-07-13 山东福生佳信科技股份有限公司 Database comparison method and device

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009032016A (en) * 2007-07-26 2009-02-12 Hitachi Software Eng Co Ltd Database selective migration system
CN104077401A (en) * 2014-07-04 2014-10-01 用友软件股份有限公司 Database data migration device and method
CN106980679A (en) * 2017-03-30 2017-07-25 广东电网有限责任公司信息中心 A kind of information system Domestic Database migration remodeling method and device
CN108241618A (en) * 2016-12-23 2018-07-03 航天信息股份有限公司 The method and device of database migration, the method and device of service routine migration
CN108932282A (en) * 2018-05-18 2018-12-04 腾讯科技(深圳)有限公司 A kind of database migration method, apparatus and storage medium
CN109933416A (en) * 2019-02-01 2019-06-25 广东安可云科技有限公司 A kind of moving method, device and storage medium using data
CN110532246A (en) * 2019-08-29 2019-12-03 上海众恒信息产业股份有限公司 Database migration tool and moving method
CN110688367A (en) * 2019-09-27 2020-01-14 浪潮软件集团有限公司 Universal database migration adaptation method and system
CN110990482A (en) * 2019-11-11 2020-04-10 中国建设银行股份有限公司 Data synchronization method and device between asynchronous databases
CN111258989A (en) * 2020-02-14 2020-06-09 腾讯科技(深圳)有限公司 Database migration evaluation method and device, storage medium and computer equipment
US10803031B1 (en) * 2015-12-30 2020-10-13 Amazon Technologies, Inc. Migrating data between databases

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009032016A (en) * 2007-07-26 2009-02-12 Hitachi Software Eng Co Ltd Database selective migration system
CN104077401A (en) * 2014-07-04 2014-10-01 用友软件股份有限公司 Database data migration device and method
US10803031B1 (en) * 2015-12-30 2020-10-13 Amazon Technologies, Inc. Migrating data between databases
CN108241618A (en) * 2016-12-23 2018-07-03 航天信息股份有限公司 The method and device of database migration, the method and device of service routine migration
CN106980679A (en) * 2017-03-30 2017-07-25 广东电网有限责任公司信息中心 A kind of information system Domestic Database migration remodeling method and device
CN108932282A (en) * 2018-05-18 2018-12-04 腾讯科技(深圳)有限公司 A kind of database migration method, apparatus and storage medium
CN109933416A (en) * 2019-02-01 2019-06-25 广东安可云科技有限公司 A kind of moving method, device and storage medium using data
CN110532246A (en) * 2019-08-29 2019-12-03 上海众恒信息产业股份有限公司 Database migration tool and moving method
CN110688367A (en) * 2019-09-27 2020-01-14 浪潮软件集团有限公司 Universal database migration adaptation method and system
CN110990482A (en) * 2019-11-11 2020-04-10 中国建设银行股份有限公司 Data synchronization method and device between asynchronous databases
CN111258989A (en) * 2020-02-14 2020-06-09 腾讯科技(深圳)有限公司 Database migration evaluation method and device, storage medium and computer equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WEIXIN_48960094: ""数据迁移,数据库迁移工具,数据库实时同步方案"", 《HTTPS://BLOG.CSDN.NET/WEIXIN_48960094/ARTICLE/DETAILS/107033886》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113111050A (en) * 2021-04-27 2021-07-13 山东福生佳信科技股份有限公司 Database comparison method and device
CN113111050B (en) * 2021-04-27 2023-07-07 山东福生佳信科技股份有限公司 Database comparison method and device

Similar Documents

Publication Publication Date Title
CN106570086B (en) Data migration system and data migration method
CN109189852B (en) Data synchronization method and device for data synchronization
US20180081956A1 (en) Method for automatically synchronizing multi-source heterogeneous data resources
CN110737594B (en) Database standard conformance testing method and device for automatically generating test cases
WO2018001200A1 (en) Data processing method, cluster manager, resource manager and data processing system
CN115712623B (en) Batch data fault-tolerant collection method based on captured metadata change
CN104462169A (en) Method and a system for merging data into a database table
CN109063005B (en) Data migration method and system, storage medium and electronic device
CN110063042A (en) A kind of response method and its terminal of database failure
CN115292307A (en) Data synchronization system, method and corresponding computer equipment and storage medium
CN112380186A (en) Domestic database migration method based on active adaptation
CN112948359A (en) Non-stop client data migration method based on database sub-base sub-table
CN117539849A (en) Data migration method and device of heterogeneous database, storage medium and electronic device
CN111209652B (en) Method and device for constructing time sequence of static equipment model of power system
CN100474250C (en) Program downloading system and method
KR102464740B1 (en) Test automation framework for dbms for analysis of bigdata and method of test automation
CN112612812A (en) Method and device for processing multi-system data
CN112000671A (en) Block chain-based database table processing method, device and system
CN112084259A (en) Database real-time synchronization method based on extended field characteristic marking
CN111008141A (en) Automatic incremental deployment test environment method
CN113688147B (en) Data processing method and system
CN110688426B (en) Second-level heartbeat synchronization method for block chain big database
TWI824155B (en) Dynamic intelligent test method, system, computer and storage medium
CN117093649A (en) Incremental data real-time comparison repair method and device, computer equipment and storage medium
CN116775460A (en) Solid state disk testing method, system, electronic setting and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210219

RJ01 Rejection of invention patent application after publication