CN111221800A - Database migration method and device, electronic equipment and storage medium - Google Patents
Database migration method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111221800A CN111221800A CN201911319464.8A CN201911319464A CN111221800A CN 111221800 A CN111221800 A CN 111221800A CN 201911319464 A CN201911319464 A CN 201911319464A CN 111221800 A CN111221800 A CN 111221800A
- Authority
- CN
- China
- Prior art keywords
- data
- library
- new
- old
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005012 migration Effects 0.000 title claims abstract description 88
- 238000013508 migration Methods 0.000 title claims abstract description 85
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012795 verification Methods 0.000 claims abstract description 46
- 230000002159 abnormal effect Effects 0.000 claims description 76
- 238000012544 monitoring process Methods 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 7
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 230000002547 anomalous effect Effects 0.000 claims description 2
- 230000001360 synchronised effect Effects 0.000 abstract description 10
- 238000004590 computer program Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention provides a database migration method, a database migration device, electronic equipment and a storage medium. The method is capable of synchronizing the entire amount of data in the old library to the new library, and writing the resulting new data to the old library, when the full synchronization is determined to be completed, the new data is incrementally synchronized from the old library to the new library, so that the total amount of migration data when the new library is online is reduced, the waiting time of the new library online is shortened, after the new library is on line, real-time data are respectively written into the old library and the new library, and when the new library is not completely stable, still using the old library in combination to ensure that the data reading and writing are normal, further checking the data in the new library, when the verification is passed, the new library is controlled to execute the read-write command, the old library is controlled to be off-line, and then finish the migration of the database, not merely improved the speed of the data handling, guaranteed the normal execution of the read-write operation at the same time, and through checking, have further guaranteed the stability of the new storehouse.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a database migration method and apparatus, an electronic device, and a storage medium.
Background
Database migration is a problem often encountered in project development, for example, migration of some old systems, bottleneck of database performance, too high price of a database in use, and the like require database migration. Currently, de-O (off-Oracle) has become a big trend, and more enterprises migrate data from the Oracle database to the MySQL database.
In the prior art, in order to implement database migration, a new database is first built, then codes are changed, when the new database is online, all application programs need to be stopped, data is synchronized again, and after data synchronization is completed, a new version is released, and the data is directly migrated to the new database.
The above method has several disadvantages. Firstly, all data need to be synchronized, so that the time spent is long, and often several hours are spent, thereby greatly reducing the availability of the system; secondly, data are directly migrated to a new database, some format errors or slight changes of the data are difficult to find in time due to differences among the databases, if verification is carried out after online, a new version is required to be published for solving the problems, and the risk is higher for a system with higher requirements on the importance of the data and the availability of the system.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a database migration method, apparatus, electronic device and storage medium, which can not only improve the data migration speed, but also ensure the normal execution of read/write operations, and further ensure the stability of a new database through verification.
A method of database migration, the method comprising:
receiving a database migration instruction, wherein the database migration instruction is used for triggering data migration from an old library to a new library;
synchronizing a full amount of data in the old library to the new library;
detecting whether new data is generated in real time;
when new data is generated, the generated new data is written into the old library;
incrementally synchronizing the new data from the old library to the new library upon determining that full-scale synchronization is complete;
acquiring real-time data after the new library is online;
writing the real-time data into the old library and the new library respectively;
verifying the data in the new library;
and when the verification is passed, controlling the new library to execute a read-write instruction, and controlling the old library to be offline.
According to a preferred embodiment of the present invention, said incrementally synchronizing said new data from said old library to said new library comprises:
acquiring an operation log of the old library;
analyzing the operation log to obtain changed data;
determining the changed data as the new data;
synchronizing the new data to the new library.
According to a preferred embodiment of the present invention, in incrementally synchronizing the new data from the old library to the new library, the method further comprises:
when incremental synchronization is carried out, real-time monitoring is carried out based on an exception handling mechanism;
when data abnormity is monitored, capturing abnormal data;
the captured data is skipped.
According to a preferred embodiment of the invention, the method further comprises:
recording the captured data as abnormal data;
determining a type of the anomalous data after incrementally synchronizing the new data from the old library to the new library;
when the abnormal data is failed to be written, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and supplementing the called data to the new library; or
And when the abnormal data is a write-in error, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updating the abnormal data according to the called data.
According to a preferred embodiment of the present invention, when the real-time data is written into the old library and the new library respectively, the method further comprises:
incrementally synchronizing data from the old library to the new library is stopped.
According to a preferred embodiment of the invention, the method further comprises:
before the configuration time point within the preset time, carrying out full quantity comparison on the data in the new library and the data in the old library to obtain a full quantity comparison result;
after the configuration time point within the preset time, performing increment comparison on the data in the new library and the data in the old library to obtain an increment comparison result;
monitoring the logs of the new library within a preset time to obtain a monitoring result;
and when the full quantity comparison result, the increment comparison result and the monitoring result are not abnormal, determining that the new library passes the verification.
According to the preferred embodiment of the present invention, the controlling the new library to execute the read-write command and the controlling the old library to be offline includes:
acquiring a configuration list, wherein the configuration list stores the corresponding relation between the read-write instruction type and a database;
determining a target database from the new library and the old library according to the configuration list;
executing a read-write instruction in the target database;
after the time is configured, when the new library is determined to be read and written normally, controlling the new library to execute a read and write instruction;
and cleaning the data of the old library, and controlling the old library to be offline.
A database migration apparatus, the apparatus comprising:
the system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a database migration instruction, and the database migration instruction is used for triggering data migration from an old database to a new database;
a synchronization unit for synchronizing the data in the old library to the new library in full;
the detection unit is used for detecting whether new data are generated in real time;
a write unit for writing the generated new data into the old bank when new data is generated;
the synchronization unit is further used for incrementally synchronizing the new data from the old library to the new library when the full synchronization is determined to be completed;
the acquisition unit is used for acquiring real-time data after the new library is online;
the writing unit is further configured to write the real-time data into the old library and the new library respectively;
the verification unit is used for verifying the data in the new library;
and the control unit is used for controlling the new library to execute a read-write instruction and controlling the old library to be offline when the verification is passed.
According to a preferred embodiment of the present invention, the synchronization unit incrementally synchronizing the new data from the old library to the new library comprises:
acquiring an operation log of the old library;
analyzing the operation log to obtain changed data;
determining the changed data as the new data;
synchronizing the new data to the new library.
According to a preferred embodiment of the invention, the apparatus further comprises:
the monitoring unit is used for carrying out real-time monitoring based on an exception handling mechanism when the new data is subjected to increment synchronization from the old library to the new library and the increment synchronization is carried out;
the capturing unit is used for capturing abnormal data when the data are monitored to be abnormal;
a skipping unit for skipping the captured data.
According to a preferred embodiment of the invention, the apparatus further comprises:
a recording unit for recording the captured data as abnormal data;
a determining unit, configured to determine a type of the abnormal data after incrementally synchronizing the new data from the old library to the new library;
the entry supplementing unit is used for calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data when the abnormal data is failed to be written, and additionally recording the called data to the new library; or
And the updating unit is used for calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data and updating the abnormal data according to the called data when the abnormal data is a write error.
According to a preferred embodiment of the invention, the apparatus further comprises:
and the stopping unit is used for stopping incremental synchronization of data from the old library to the new library when the real-time data is written into the old library and the new library respectively.
According to a preferred embodiment of the invention, the apparatus further comprises:
the comparison unit is used for carrying out full-quantity comparison on the data in the new library and the data in the old library before the configuration time point in the preset time to obtain a full-quantity comparison result;
the comparison unit is further configured to perform incremental comparison on the data in the new library and the data in the old library after the configuration time point within the preset time to obtain an incremental comparison result;
the monitoring unit is used for monitoring the logs of the new library within a preset time to obtain a monitoring result;
the determining unit is further configured to determine that the new library passes verification when none of the full-quantity comparison result, the incremental comparison result, and the monitoring result is abnormal.
According to a preferred embodiment of the present invention, the control unit is specifically configured to:
acquiring a configuration list, wherein the configuration list stores the corresponding relation between the read-write instruction type and a database;
determining a target database from the new library and the old library according to the configuration list;
executing a read-write instruction in the target database;
after the time is configured, when the new library is determined to be read and written normally, controlling the new library to execute a read and write instruction;
and cleaning the data of the old library, and controlling the old library to be offline.
An electronic device, the electronic device comprising:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the database migration method.
A computer-readable storage medium having stored therein at least one instruction, the at least one instruction being executable by a processor in an electronic device to implement the database migration method.
According to the technical scheme, the method can acquire the old library and the new library when a database migration instruction is received, synchronize the total amount of data in the old library to the new library, detect whether new data is generated in real time, write the generated new data into the old library when new data is generated, incrementally synchronize the new data from the old library to the new library when the total amount synchronization is completed, reduce the total amount of migration data when the new library is online, further shorten the waiting time of online of the new library, acquire real-time data after the new library is online, write the real-time data into the old library and the new library respectively, further still combine and use the old library when the new library is not completely stable so as to ensure normal data reading and writing, further verify the data in the new library, and control the new library to execute the reading and writing instruction when the new library passes the verification, and controlling the old library to be offline, and further completing the migration of the database, thereby not only improving the speed of data migration, but also ensuring the normal execution of read-write operation, and further ensuring the stability of the new library through verification.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the database migration method of the present invention.
FIG. 2 is a functional block diagram of a preferred embodiment of the database migration apparatus of the present invention.
FIG. 3 is a schematic structural diagram of an electronic device implementing the database migration method according to a preferred embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a flow chart of a preferred embodiment of the database migration method according to the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The database migration method is applied to one or more electronic devices, where the electronic devices are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.
The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a cloud computing (cloud computing) based cloud consisting of a large number of hosts or network servers.
The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
And S10, receiving a database migration instruction.
Wherein the database migration instruction is to trigger migration of data from an old library to a new library.
In at least one embodiment of the invention, the database migration instruction may be triggered by an associated staff member, for example: developers, etc.
In at least one embodiment of the present invention, the old library may include an Oracle database or the like, and the new library may include a MySQL database or the like, which is not limited by the present invention.
In at least one embodiment of the invention, the electronic device may retrieve the new library and the old library from the database migration instruction.
For example: the database migration instructions may be used to migrate data from an Oracle database to a MySQL database.
And S11, synchronizing the data in the old library to the new library in a full scale mode.
In at least one embodiment of the invention, in order to save the time consumed for data synchronization after the new library is online, the electronic equipment synchronizes the whole amount of data in the old library to the new library before the new library is online.
Specifically, the electronic device may employ a designated tool to synchronize the entire amount of data in the old library to the new library, which is not limited by the present invention.
Wherein the designated tools may include, but are not limited to: KETTLE, HKRONLINES SYNCNavigator, etc.
S12, detecting whether new data is generated in real time.
In at least one embodiment of the present invention, each system or terminal connected to the old library will continue to generate new data because it is always running, and this newly generated data also needs to be written into the old library and the new library for easy calling.
Therefore, in order to avoid that newly generated data is missed to be written, the electronic device detects whether new data is generated in real time.
S13, when new data is generated, the generated new data is written into the old library.
In at least one embodiment of the invention, the electronic device first writes the new data generated to the old library, since the new library has not been formally used online.
Through the implementation mode, the old library can still be used for storing data before the new library is formally on-line.
S14, when it is determined that full-volume synchronization is complete, incrementally synchronizing the new data from the old library to the new library.
In at least one embodiment of the invention, after the full amount synchronization is completed, the incremental synchronization is performed, so that the data is prevented from being repeatedly synchronized to the new library, and the data in the new library is prevented from being redundant.
In the prior art, a one-time full-volume synchronization mode is usually adopted for database migration, and this migration mode not only needs to spend a lot of time for data writing and reduce the availability of the system, but also needs to stop the application program being used for data initialization before the new library is formally online, which affects the normal operation of the application program.
In comparison, the data is also subjected to incremental synchronization before the new library is online, so that the new data does not need to be migrated when the new library is online, the total amount of migrated data is reduced, the online waiting time of the new library is further shortened, the availability of a system is improved, meanwhile, an application program corresponding to the new data does not need to be stopped, and the user experience is improved.
Preferably, the electronic device incrementally synchronizing the new data from the old library to the new library comprises:
and the electronic equipment acquires the operation log of the old library, analyzes the operation log to obtain changed data, determines the changed data as the new data and synchronizes the new data to the new library.
Specifically, the electronic device may employ a configuration tool to synchronize the new data to the new library, which is not limited by the present invention.
For example: the configuration tools may include, but are not limited to: oracle Golden Gate et al.
Specifically, the electronic device builds a synchronous link (gg link) from the old library to the new library by using the Oracle Golden Gate.
Further, the electronic device analyzes the operation log and determines changed data from the operation log, the electronic device determines the changed data as the new data, and then the electronic device opens the Oracle Golden Gate and performs incremental synchronization of the new data based on the gg link.
Through the embodiment, data is not omitted due to incremental synchronization based on the operation logs, and the Oracle Golden Gate is structured data replication based on the logs and reads the operation logs of the old library as a synchronization basis, so that the consumption of a Central Processing Unit (CPU) of a system is low, and the influence on the performance of the system is small.
In at least one embodiment of the invention, the electronic device may add transactions to write operations of the old bank and the new bank, respectively.
Wherein the transaction is the smallest unit of work for the database operation, and is a series of operations performed as a single logical unit of work, and the operations are submitted to the system as a whole, either all or none of the operations are performed, that is: the transaction is a set of non-repartitionable operation sets (working logic units), the electronic device adds transactions to the branch codes for operating the new library and the branch codes for operating the old library respectively, and then the new library and the old library cannot affect each other, so that when data writing of the new library is abnormal, data writing of the old library cannot be affected, and the operations of the new library and the old library are independent.
In at least one embodiment of the invention, in incrementally synchronizing the new data from the old library to the new library, the method further comprises:
when the increment synchronization is carried out, the electronic equipment carries out real-time monitoring based on an exception handling mechanism, and when data exception is monitored, the electronic equipment captures the data exception and further skips the captured data.
Specifically, when performing the incremental synchronization, the electronic device may execute the exception handling mechanism using a try-catch statement.
Further, the electronic device performs real-time monitoring by adopting try statements, and captures the abnormal data by adopting catch statements.
In the prior art, when data abnormality is monitored, increment synchronization is terminated, so that the efficiency of the increment synchronization is influenced, and the migration efficiency of a database is further influenced.
In this embodiment, the electronic device skips the captured data and continues to perform incremental synchronization to improve the efficiency of data synchronization.
However, since the exception is captured, the electronic device also needs to handle the exception, and further, the quality of data is not affected.
Specifically, the method further comprises:
the electronic device records the captured data as abnormal data, and after incrementally synchronizing the new data from the old library to the new library, the electronic device determines the type of the abnormal data, specifically:
(1) and when the abnormal data is failed to be written, the electronic equipment calls the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and additionally records the called data to the new library.
(2) And when the abnormal data is a write-in error, the electronic equipment calls the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updates the abnormal data according to the called data.
Wherein the types of the abnormal data include, but are not limited to: write failures, write errors, etc.
Further, a piece of data can be uniquely determined by the data identification.
It can be understood that, if the abnormal data is a write failure, it indicates that corresponding data may not be written in the new library, and therefore, the electronic device retrieves data corresponding to the abnormal data from the old library based on the data identifier, and additionally records the retrieved data in the new library, thereby avoiding data omission and ensuring the comprehensiveness of the data in the new library.
If the abnormal data is a write error, the abnormal data indicates that the error data possibly occurs in the new library, so that the electronic equipment calls the data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, updates the abnormal data according to the called data, and further avoids data errors so as to ensure the accuracy of the data in the new library.
Of course, in other embodiments, the electronic device may further obtain an execution log of incremental synchronization, obtain information such as writing time of the abnormal data from the execution log, and retrieve data corresponding to the abnormal data from the old library according to the obtained information, which is not limited in the present invention.
And S15, acquiring real-time data after the new library is online.
In at least one embodiment of the present invention, since new data (such as application data, operation data, and the like) is generated after the new library is online, in order to ensure the integrity of the data, the electronic device needs to write the data into the database in real time.
Therefore, after the new library is on line, the electronic equipment acquires real-time data so as to store the data acquired in real time in time.
And S16, writing the real-time data into the old library and the new library respectively.
In at least one embodiment of the present invention, since the new library is already online, the electronic device does not need to use other tools to write the real-time data into the new library, but only needs to directly write the real-time data into the new library by using the built-in code of the new library.
In at least one embodiment of the present invention, when the real-time data is written into the old library and the new library, respectively, the method further comprises:
the electronic device stops incrementally synchronizing data from the old library to the new library.
It can be understood that, after the new library comes online, the electronic device will perform synchronous writing of data into the new and old databases through the code layer, and at this time, if the incremental synchronization of data from the old library to the new library is not stopped, the repeated data from the old library is synchronized to the new library, so that the new library has redundant data.
Further, in this embodiment, the electronic device may also stop the relevant application program to avoid data loss caused by the running of the program.
Through the implementation mode, data redundancy can be effectively avoided, and the quality of data in the new library is improved.
And S17, checking the data in the new library.
In at least one embodiment of the present invention, in order to ensure the accuracy and comprehensiveness of the data in the new library, the electronic device verifies the data in the new library, and a specific verification manner will be described later.
And S18, when the verification is passed, controlling the new library to execute a read-write instruction, and controlling the old library to be offline.
In at least one embodiment of the present invention, when the verification is passed, the electronic device may further wait for a period of time, control the new library to execute the read-write command, and control the old library to go offline, so as to further ensure that the new library is completely stable.
In at least one embodiment of the invention, the method further comprises:
before a configuration time point within a preset time, the electronic device performs full comparison on data in the new library and data in the old library to obtain a full comparison result, further, after the configuration time point within the preset time, the electronic device performs incremental comparison on the data in the new library and the data in the old library to obtain an incremental comparison result, the electronic device monitors logs of the new library within the preset time to obtain a monitoring result, and when the full comparison result, the incremental comparison result and the monitoring result are not abnormal, the electronic device determines that the new library passes verification.
The preset time and the configuration time point may be determined by performing big data analysis according to historical data, which is not limited in the present invention.
Specifically, the electronic device acquires all verification time periods from the historical data, determines the latest verification time and the earliest verification time from all the verification time periods, and determines the preset time by using the earliest verification time as the starting time and the latest verification time as the ending time.
Further, the electronic device may process the data within the preset time by using a least square method to obtain an optimal solution, and determine the optimal solution as the configuration time point.
Through the implementation mode, firstly, the detection of the data in the new library is realized by combining a full quantity comparison technology and an increment comparison technology, and then, the verification process of the data is further perfected by combining a log monitoring technology so as to realize the accurate verification of the data in the new library.
In at least one embodiment of the present invention, the controlling the new library to execute the read-write command and the controlling the old library to be offline includes:
the electronic equipment obtains a configuration list, wherein the configuration list stores a corresponding relation between a read-write instruction type and a database, the electronic equipment determines a target database from the new database and the old database according to the configuration list, and executes a read-write instruction in the target database, and after configuration time, when the new database is determined to be read and written normally, the electronic equipment controls the new database to execute the read-write instruction, cleans the data of the old database, and controls the old database to be offline.
For example: and the electronic equipment modifies the parameters of the configuration file, so that a part of types of read-write instructions are executed in the new library, and the other part of types of read-write instructions are executed in the old library, and then the execution range of the new library on the read-write instructions is gradually expanded until the whole system reads from the new library, and after the read-write functions are normal, the configuration parameters are modified, and data is only written into the new library and is not written into the old library.
The configuration list may be configured by performing big data analysis according to historical data, which is not limited in the present invention.
Specifically, the electronic device obtains historical configuration data, calculates the probability of each read-write command type operating in a certain database according to the historical configuration data, determines the database corresponding to the maximum probability of each read-write command type operating from the calculated probabilities, and records the corresponding relationship between the corresponding database and each read-write command type, and the electronic device further configures the configuration list according to the corresponding relationship.
For example: and when the probability of the read-write operation of the task A in the old library is the maximum, the electronic equipment records that the read-write operation of the task A is executed in the old library.
And the configuration time can be configured in a user-defined manner.
Different from the mode of directly migrating the data to the new library in the prior art, the scheme can gradually open the new library, and avoids abnormal data generation after the new library is directly opened.
According to the technical scheme, the method can acquire the old library and the new library when a database migration instruction is received, synchronize the total amount of data in the old library to the new library, detect whether new data is generated in real time, write the generated new data into the old library when new data is generated, incrementally synchronize the new data from the old library to the new library when the total amount synchronization is completed, reduce the total amount of migration data when the new library is online, further shorten the waiting time of online of the new library, acquire real-time data after the new library is online, write the real-time data into the old library and the new library respectively, further still combine and use the old library when the new library is not completely stable so as to ensure normal data reading and writing, further verify the data in the new library, and control the new library to execute the reading and writing instruction when the new library passes the verification, and controlling the old library to be offline, and further completing the migration of the database, thereby not only improving the speed of data migration, but also ensuring the normal execution of read-write operation, and further ensuring the stability of the new library through verification.
FIG. 2 is a functional block diagram of the database migration apparatus according to the preferred embodiment of the present invention. The database migration apparatus 11 includes a receiving unit 110, a synchronizing unit 111, a detecting unit 112, a writing unit 113, an obtaining unit 114, a checking unit 115, a control unit 116, a monitoring unit 117, a capturing unit 118, a skipping unit 119, a recording unit 120, a determining unit 121, an additional recording unit 122, an updating unit 123, a stopping unit 124, a comparing unit 125, and a monitoring unit 126. The module/unit referred to in the present invention refers to a series of computer program segments that can be executed by the processor 13 and that can perform a fixed function, and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
The receiving unit 110 receives a database migration instruction.
Wherein the database migration instruction is to trigger migration of data from an old library to a new library.
In at least one embodiment of the invention, the database migration instruction may be triggered by an associated staff member, for example: developers, etc.
In at least one embodiment of the present invention, the old library may include an Oracle database or the like, and the new library may include a MySQL database or the like, which is not limited by the present invention.
In at least one embodiment of the invention, the new library and the old library may be retrieved from the database migration instruction.
For example: the database migration instructions may be used to migrate data from an Oracle database to a MySQL database.
The synchronization unit 111 synchronizes the full amount of data in the old library to the new library.
In at least one embodiment of the present invention, in order to save the time consumed for data synchronization after the new library is online, the synchronization unit 111 synchronizes the entire amount of data in the old library to the new library before the new library is online.
Specifically, the synchronization unit 111 may employ a specific tool to synchronize the data in the old library to the new library in full, which is not limited in the present invention.
Wherein the designated tools may include, but are not limited to: KETTLE, HKRONLINES SYNCNavigator, etc.
The detection unit 112 detects whether new data is generated in real time.
In at least one embodiment of the present invention, each system or terminal connected to the old library will continue to generate new data because it is always running, and this newly generated data also needs to be written into the old library and the new library for easy calling.
Therefore, in order to avoid that newly generated data is missed to be written, the detection unit 112 detects whether new data is generated in real time.
When new data is generated, the write unit 113 writes the generated new data to the old bank.
In at least one embodiment of the present invention, since the new bank has not been formally used online, the writing unit 113 first writes the generated new data to the old bank.
Through the implementation mode, the old library can still be used for storing data before the new library is formally on-line.
When it is determined that the full-scale synchronization is completed, the synchronization unit 111 incrementally synchronizes the new data from the old bank to the new bank.
In at least one embodiment of the invention, after the full amount synchronization is completed, the incremental synchronization is performed, so that the data is prevented from being repeatedly synchronized to the new library, and the data in the new library is prevented from being redundant.
In the prior art, a one-time full-volume synchronization mode is usually adopted for database migration, and this migration mode not only needs to spend a lot of time for data writing and reduce the availability of the system, but also needs to stop the application program being used for data initialization before the new library is formally online, which affects the normal operation of the application program.
In comparison, the data is also subjected to incremental synchronization before the new library is online, so that the new data does not need to be migrated when the new library is online, the total amount of migrated data is reduced, the online waiting time of the new library is further shortened, the availability of a system is improved, meanwhile, an application program corresponding to the new data does not need to be stopped, and the user experience is improved.
Preferably, the synchronization unit 111 incrementally synchronizing the new data from the old library to the new library comprises:
the synchronization unit 111 obtains the operation log of the old library, analyzes the operation log to obtain changed data, and determines the changed data as the new data and synchronizes the new data to the new library.
Specifically, the synchronization unit 111 may employ a configuration tool to synchronize the new data to the new library, which is not limited by the present invention.
For example: the configuration tools may include, but are not limited to: oracle Golden Gate et al.
Specifically, the synchronization unit 111 uses the Oracle Golden Gate to build a synchronization link (gg link) from the old library to the new library.
Further, the synchronization unit 111 parses the operation log, determines changed data from the operation log, the synchronization unit 111 determines the changed data as the new data, and then the synchronization unit 111 opens the Oracle Golden Gate and performs incremental synchronization of the new data based on the gg link.
Through the embodiment, data is not omitted due to incremental synchronization based on the operation logs, and the Oracle Golden Gate is structured data replication based on the logs and reads the operation logs of the old library as a synchronization basis, so that the consumption of a Central Processing Unit (CPU) of a system is low, and the influence on the performance of the system is small.
In at least one embodiment of the invention, the write unit 113 may add transactions to the write operations of the old bank and the new bank, respectively.
Wherein the transaction is the smallest unit of work for the database operation, and is a series of operations performed as a single logical unit of work, and the operations are submitted to the system as a whole, either all or none of the operations are performed, that is: the transaction is a set of non-repartitionable operation sets (working logic units), and the writing unit 113 adds transactions separately to the branch codes for operating the new library and the branch codes for operating the old library, so that the new library and the old library will not affect each other, and thus, when data writing in the new library is abnormal, data writing in the old library will not be affected, and the operations of the new library and the old library are independent from each other.
In at least one embodiment of the present invention, when the synchronization unit 111 incrementally synchronizes the new data from the old library to the new library, the monitoring unit 117 performs real-time monitoring based on an exception handling mechanism, when data exception is monitored, the capturing unit 118 captures the data of the exception, and the skipping unit 119 further skips the captured data.
In particular, when performing incremental synchronization, the exception handling mechanism may be performed using try-catch statements.
Further, the monitoring unit 117 performs real-time monitoring using try statements, and the capturing unit 118 captures data of the exception using catch statements.
In the prior art, when data abnormality is monitored, increment synchronization is terminated, so that the efficiency of the increment synchronization is influenced, and the migration efficiency of a database is further influenced.
In this embodiment, the skipping unit 119 skips the captured data and continues to perform incremental synchronization to improve the efficiency of data synchronization.
However, since the exception is captured, the exception is handled, and the quality of data is not affected.
Specifically, the recording unit 120 records the captured data as abnormal data, and after incrementally synchronizing the new data from the old library to the new library, the determining unit 121 determines the type of the abnormal data, specifically:
(1) when the abnormal data is a write failure, the additional recording unit 122 calls data corresponding to the abnormal data from the old library based on the data identifier of the abnormal data, and additionally records the called data in the new library.
(2) When the abnormal data is a write error, the updating unit 123 retrieves data corresponding to the abnormal data from the old library based on the data identifier of the abnormal data, and updates the abnormal data according to the retrieved data.
Wherein the types of the abnormal data include, but are not limited to: write failures, write errors, etc.
Further, a piece of data can be uniquely determined by the data identification.
It can be understood that, if the abnormal data is a write failure, it indicates that corresponding data may not be written in the new library, and therefore, the entry supplementing unit 122 retrieves data corresponding to the abnormal data from the old library based on the data identifier, and additionally records the retrieved data in the new library, thereby avoiding data omission and ensuring the comprehensiveness of the data in the new library.
If the abnormal data is a write error, it indicates that the new library may have error data, so the updating unit 123 retrieves data corresponding to the abnormal data from the old library based on the data identifier of the abnormal data, and updates the abnormal data according to the retrieved data, thereby avoiding data error and ensuring the accuracy of the data in the new library.
Of course, in other embodiments, an execution log with incremental synchronization may also be obtained, information such as the writing time of the abnormal data may also be obtained from the execution log, and data corresponding to the abnormal data may be called from the old library according to the obtained information, which is not limited in the present invention.
After the new library comes online, the acquisition unit 114 acquires real-time data.
In at least one embodiment of the present invention, since new data (such as application data, operation data, and the like) is generated after the new library is online, in order to ensure the integrity of the data, the writing unit 113 needs to write the data into the database in real time.
Therefore, after the new library is online, the acquiring unit 114 acquires real-time data so as to save the real-time acquired data in time.
The write unit 113 writes the real-time data into the old bank and the new bank, respectively.
In at least one embodiment of the present invention, since the new library is online, the writing unit 113 does not need to use other tools to write the real-time data into the new library, but only needs to directly write the real-time data into the new library by using the built-in code of the new library.
In at least one embodiment of the present invention, the stop unit 124 stops incrementally synchronizing data from the old library to the new library as the real-time data is written to the old library and the new library, respectively.
It is understood that after the new library comes online, the writing unit 113 performs synchronous writing of data into the old and new databases through a code layer, and at this time, if the incremental synchronization of data from the old library to the new library is not stopped, the repeated data from the old library is synchronized to the new library, so that the new library has redundant data.
Further, in this embodiment, the stopping unit 124 may also stop the relevant application program to avoid data loss caused by the running of the program.
Through the implementation mode, data redundancy can be effectively avoided, and the quality of data in the new library is improved.
The verification unit 115 verifies the data in the new library.
In at least one embodiment of the present invention, in order to ensure the accuracy and comprehensiveness of the data in the new library, the verification unit 115 verifies the data in the new library, and a specific verification manner will be described later.
When the verification is passed, the control unit 116 controls the new library to execute the read-write command, and controls the old library to be offline.
In at least one embodiment of the present invention, when the verification is passed, the control unit 116 may further wait for a period of time, control the new library to execute the read/write command, and control the old library to go offline, so as to further ensure that the new library is completely stable.
In at least one embodiment of the present invention, before a configuration time point within a preset time, the comparing unit 125 performs full comparison on data in the new library and data in the old library to obtain a full comparison result, further, after the configuration time point within the preset time, the comparing unit 125 performs incremental comparison on data in the new library and data in the old library to obtain an incremental comparison result, and within the preset time, the monitoring unit 126 monitors a log of the new library to obtain a monitoring result, and when none of the full comparison result, the incremental comparison result, and the monitoring result is abnormal, the determining unit 121 determines that the new library passes verification.
The preset time and the configuration time point may be determined by performing big data analysis according to historical data, which is not limited in the present invention.
Specifically, the determining unit 121 obtains all verification time periods from the historical data, and determines the latest verification time and the earliest verification time from all verification time periods, and the determining unit 121 determines the preset time by using the earliest verification time as the start time and the latest verification time as the end time.
Further, the determining unit 121 may process the data within the preset time by using a least square method to obtain an optimal solution, and determine the optimal solution as the configuration time point.
Through the implementation mode, firstly, the detection of the data in the new library is realized by combining a full quantity comparison technology and an increment comparison technology, and then, the verification process of the data is further perfected by combining a log monitoring technology so as to realize the accurate verification of the data in the new library.
In at least one embodiment of the present invention, the controlling unit 116 controls the new library to execute the read/write command, and controls the old library to be offline includes:
the control unit 116 obtains a configuration list, the configuration list stores a corresponding relationship between a read-write instruction type and a database, the control unit 116 determines a target database from the new database and the old database according to the configuration list, and executes a read-write instruction in the target database, after configuration time, when it is determined that the new database is read and written normally, the control unit 116 controls the new database to execute the read-write instruction, cleans up the old database, and controls the old database to be offline.
For example: the control unit 116 modifies the parameters of the configuration file to make a part of types of read-write instructions execute in the new library and another part of types of read-write instructions execute in the old library, and then gradually expands the execution range of the read-write instructions by the new library until the whole system reads from and writes to the new library, and after the read-write functions are normal, the configuration parameters are modified to write data only to the new library and not to write to the old library.
The configuration list may be configured by performing big data analysis according to historical data, which is not limited in the present invention.
Specifically, the control unit 116 obtains historical configuration data, and calculates a probability of each read/write command type operating in a certain database according to the historical configuration data, the control unit 116 determines a database corresponding to a maximum probability of each read/write command type operating from the calculated probabilities, and records a corresponding relationship between the corresponding database and each read/write command type, and the control unit 116 further configures the configuration list according to the corresponding relationship.
For example: when the probability that the read-write operation of the task a is in the old library is the maximum, the control unit 116 records that the read-write operation of the task a is executed in the old library.
And the configuration time can be configured in a user-defined manner.
Different from the mode of directly migrating the data to the new library in the prior art, the scheme can gradually open the new library, and avoids abnormal data generation after the new library is directly opened.
According to the technical scheme, the method can acquire the old library and the new library when a database migration instruction is received, synchronize the total amount of data in the old library to the new library, detect whether new data is generated in real time, write the generated new data into the old library when new data is generated, incrementally synchronize the new data from the old library to the new library when the total amount synchronization is completed, reduce the total amount of migration data when the new library is online, further shorten the waiting time of online of the new library, acquire real-time data after the new library is online, write the real-time data into the old library and the new library respectively, further still combine and use the old library when the new library is not completely stable so as to ensure normal data reading and writing, further verify the data in the new library, and control the new library to execute the reading and writing instruction when the new library passes the verification, and controlling the old library to be offline, and further completing the migration of the database, thereby not only improving the speed of data migration, but also ensuring the normal execution of read-write operation, and further ensuring the stability of the new library through verification.
Fig. 3 is a schematic structural diagram of an electronic device implementing the database migration method according to a preferred embodiment of the present invention.
The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as a database migration program, stored in the memory 12 and executable on the processor 13.
It will be understood by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation to the electronic device 1, the electronic device 1 may have a bus-type structure or a star-type structure, the electronic device 1 may further include more or less hardware or software than those shown in the figures, or different component arrangements, for example, the electronic device 1 may further include an input and output device, a network access device, and the like.
It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those that can be adapted to the present invention, should also be included in the scope of the present invention, and are included herein by reference.
The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, for example a removable hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of a database migration program, etc., but also to temporarily store data that has been output or is to be output.
The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (for example, executing a database migration program, etc.) stored in the memory 12 and calling data stored in the memory 12.
The processor 13 executes an operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in the above-mentioned respective database migration method embodiments, such as steps S10, S11, S12, S13, S14, S15, S16, S17, S18 shown in fig. 1.
Alternatively, the processor 13, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example:
receiving a database migration instruction, wherein the database migration instruction is used for triggering data migration from an old library to a new library;
synchronizing a full amount of data in the old library to the new library;
detecting whether new data is generated in real time;
when new data is generated, the generated new data is written into the old library;
incrementally synchronizing the new data from the old library to the new library upon determining that full-scale synchronization is complete;
acquiring real-time data after the new library is online;
writing the real-time data into the old library and the new library respectively;
verifying the data in the new library;
and when the verification is passed, controlling the new library to execute a read-write instruction, and controlling the old library to be offline.
Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a receiving unit 110, a synchronizing unit 111, a detecting unit 112, a writing unit 113, an acquiring unit 114, a verifying unit 115, a controlling unit 116, a monitoring unit 117, a capturing unit 118, a skipping unit 119, a recording unit 120, a determining unit 121, an entry complementing unit 122, an updating unit 123, a stopping unit 124, a comparing unit 125, and a monitoring unit 126.
The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device) or a processor (processor) to execute parts of the methods according to the embodiments of the present invention.
The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.
Although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
Fig. 3 only shows the electronic device 1 with components 12-13, and it will be understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
With reference to fig. 1, the memory 12 of the electronic device 1 stores a plurality of instructions to implement a database migration method, and the processor 13 executes the plurality of instructions to implement:
receiving a database migration instruction, wherein the database migration instruction is used for triggering data migration from an old library to a new library;
synchronizing a full amount of data in the old library to the new library;
detecting whether new data is generated in real time;
when new data is generated, the generated new data is written into the old library;
incrementally synchronizing the new data from the old library to the new library upon determining that full-scale synchronization is complete;
acquiring real-time data after the new library is online;
writing the real-time data into the old library and the new library respectively;
verifying the data in the new library;
and when the verification is passed, controlling the new library to execute a read-write instruction, and controlling the old library to be offline.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A method of database migration, the method comprising:
receiving a database migration instruction, wherein the database migration instruction is used for triggering data migration from an old library to a new library;
synchronizing a full amount of data in the old library to the new library;
detecting whether new data is generated in real time;
when new data is generated, the generated new data is written into the old library;
incrementally synchronizing the new data from the old library to the new library upon determining that full-scale synchronization is complete;
acquiring real-time data after the new library is online;
writing the real-time data into the old library and the new library respectively;
verifying the data in the new library;
and when the verification is passed, controlling the new library to execute a read-write instruction, and controlling the old library to be offline.
2. The database migration method of claim 1, wherein said incrementally synchronizing said new data from said old library to said new library comprises:
acquiring an operation log of the old library;
analyzing the operation log to obtain changed data;
determining the changed data as the new data;
synchronizing the new data to the new library.
3. The database migration method of claim 1, wherein, in incrementally synchronizing the new data from the old library to the new library, the method further comprises:
when incremental synchronization is carried out, real-time monitoring is carried out based on an exception handling mechanism;
when data abnormity is monitored, capturing abnormal data;
the captured data is skipped.
4. The database migration method according to claim 1, wherein said method further comprises:
recording the captured data as abnormal data;
determining a type of the anomalous data after incrementally synchronizing the new data from the old library to the new library;
when the abnormal data is failed to be written, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and supplementing the called data to the new library; or
And when the abnormal data is a write-in error, calling data corresponding to the abnormal data from the old library based on the data identification of the abnormal data, and updating the abnormal data according to the called data.
5. The method of database migration according to claim 1, wherein when writing the real-time data to the old library and the new library, respectively, the method further comprises:
incrementally synchronizing data from the old library to the new library is stopped.
6. The database migration method according to claim 1, wherein said method further comprises:
before the configuration time point within the preset time, carrying out full quantity comparison on the data in the new library and the data in the old library to obtain a full quantity comparison result;
after the configuration time point within the preset time, performing increment comparison on the data in the new library and the data in the old library to obtain an increment comparison result;
monitoring the logs of the new library within a preset time to obtain a monitoring result;
and when the full quantity comparison result, the increment comparison result and the monitoring result are not abnormal, determining that the new library passes the verification.
7. The database migration method according to claim 1, wherein said controlling said new library to execute read and write commands and controlling said old library to go offline comprises:
acquiring a configuration list, wherein the configuration list stores the corresponding relation between the read-write instruction type and a database;
determining a target database from the new library and the old library according to the configuration list;
executing a read-write instruction in the target database;
after the time is configured, when the new library is determined to be read and written normally, controlling the new library to execute a read and write instruction;
and cleaning the data of the old library, and controlling the old library to be offline.
8. An apparatus for database migration, the apparatus comprising:
the system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a database migration instruction, and the database migration instruction is used for triggering data migration from an old database to a new database;
a synchronization unit for synchronizing the data in the old library to the new library in full;
the detection unit is used for detecting whether new data are generated in real time;
a write unit for writing the generated new data into the old bank when new data is generated;
the synchronization unit is further used for incrementally synchronizing the new data from the old library to the new library when the full synchronization is determined to be completed;
the acquisition unit is used for acquiring real-time data after the new library is online;
the writing unit is further configured to write the real-time data into the old library and the new library respectively;
the verification unit is used for verifying the data in the new library;
and the control unit is used for controlling the new library to execute a read-write instruction and controlling the old library to be offline when the verification is passed.
9. An electronic device, characterized in that the electronic device comprises:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the database migration method of any of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executable by a processor in an electronic device to implement the database migration method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319464.8A CN111221800A (en) | 2019-12-19 | 2019-12-19 | Database migration method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319464.8A CN111221800A (en) | 2019-12-19 | 2019-12-19 | Database migration method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111221800A true CN111221800A (en) | 2020-06-02 |
Family
ID=70829534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911319464.8A Pending CN111221800A (en) | 2019-12-19 | 2019-12-19 | Database migration method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221800A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015351A (en) * | 2020-10-19 | 2020-12-01 | 北京易真学思教育科技有限公司 | Data migration method and device, storage medium and electronic equipment |
| CN112527777A (en) * | 2020-12-18 | 2021-03-19 | 福建天晴数码有限公司 | Log-tracing-based database expansion method and device |
| CN112632032A (en) * | 2020-12-18 | 2021-04-09 | 华人运通(上海)云计算科技有限公司 | Data migration method and device, storage medium and terminal equipment |
| CN112925770A (en) * | 2021-03-09 | 2021-06-08 | 京东数字科技控股股份有限公司 | Database capacity expansion method, device, equipment and medium |
| CN113254421A (en) * | 2021-05-31 | 2021-08-13 | 重庆富民银行股份有限公司 | Database double-writing based cross-library account migration method |
| CN113468148A (en) * | 2021-08-13 | 2021-10-01 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| CN107122355A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
| CN108900497A (en) * | 2018-06-25 | 2018-11-27 | 江苏欧软信息科技有限公司 | Method of data synchronization and system between a kind of heterogeneous system |
| CN110427422A (en) * | 2019-05-23 | 2019-11-08 | 武汉达梦数据库有限公司 | Data consistency verification method, equipment and storage medium when data synchronous abnormality |
| US10509696B1 (en) * | 2017-08-16 | 2019-12-17 | Amazon Technologies, Inc. | Error detection and mitigation during data migrations |
-
2019
- 2019-12-19 CN CN201911319464.8A patent/CN111221800A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| CN107122355A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
| US10509696B1 (en) * | 2017-08-16 | 2019-12-17 | Amazon Technologies, Inc. | Error detection and mitigation during data migrations |
| CN108900497A (en) * | 2018-06-25 | 2018-11-27 | 江苏欧软信息科技有限公司 | Method of data synchronization and system between a kind of heterogeneous system |
| CN110427422A (en) * | 2019-05-23 | 2019-11-08 | 武汉达梦数据库有限公司 | Data consistency verification method, equipment and storage medium when data synchronous abnormality |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015351A (en) * | 2020-10-19 | 2020-12-01 | 北京易真学思教育科技有限公司 | Data migration method and device, storage medium and electronic equipment |
| CN112527777A (en) * | 2020-12-18 | 2021-03-19 | 福建天晴数码有限公司 | Log-tracing-based database expansion method and device |
| CN112632032A (en) * | 2020-12-18 | 2021-04-09 | 华人运通(上海)云计算科技有限公司 | Data migration method and device, storage medium and terminal equipment |
| CN112925770A (en) * | 2021-03-09 | 2021-06-08 | 京东数字科技控股股份有限公司 | Database capacity expansion method, device, equipment and medium |
| CN113254421A (en) * | 2021-05-31 | 2021-08-13 | 重庆富民银行股份有限公司 | Database double-writing based cross-library account migration method |
| CN113468148A (en) * | 2021-08-13 | 2021-10-01 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
| CN113468148B (en) * | 2021-08-13 | 2023-02-17 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111221800A (en) | Database migration method and device, electronic equipment and storage medium | |
| WO2021169260A1 (en) | System board card power supply test method, apparatus and device, and storage medium | |
| US9479395B2 (en) | Model framework to facilitate robust programming of distributed workflows | |
| CN103827834B (en) | A kind of moving method of internal storage data, computing machine and device | |
| CN107111595B (en) | Method, device and system for detecting early boot errors | |
| US20120254662A1 (en) | Automated test system and automated test method | |
| US9921905B2 (en) | Resource integrity during partial backout of application updates | |
| WO2021072880A1 (en) | Method for asynchronously creating internal snapshot of virtual machine, apparatus, system and storage medium | |
| TWI759719B (en) | Flash memory controller and method used in flash memory controller | |
| US20060112308A1 (en) | Selection of status data from synchronous redundant devices | |
| CN114168222A (en) | Method and device for acquiring starting time, terminal equipment and storage medium | |
| CN108431781A (en) | The self diagnosis of the mistake of device driver detection and automatic diagnostic data are collected | |
| US9740430B2 (en) | Optimizing parallel build of application | |
| CN110704236B (en) | On-line checking method for chip flash data and computer storage medium | |
| US8984333B2 (en) | Automatic computer storage medium diagnostics | |
| CN111694684A (en) | Abnormal construction method and device of storage equipment, electronic equipment and storage medium | |
| CN116126832A (en) | Database switching method, switching device, electronic equipment and storage medium | |
| DE102022127895A1 (en) | FAST ECC MEMORY ERRORS CORRECTION | |
| US20150067252A1 (en) | Communicating outstanding maintenance tasks to improve disk data integrity | |
| US20220121390A1 (en) | Accelerated non-volatile memory device inspection and forensics | |
| CN113392090A (en) | Data verification method, device, equipment and medium based on database migration | |
| CN115248627A (en) | Method, system and computer program product for storing finite state machine state data | |
| CN111324543B (en) | Method and device for testing IE (Internet Explorer information) information based on Dual BIOS (basic input/output System) | |
| CN113722212B (en) | CPLD upgrading test method, device, equipment and medium | |
| JP2002229811A (en) | Control method of logical partition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |