CN113590643B - Data synchronization method, device, equipment and storage medium based on dual-track database - Google Patents
Data synchronization method, device, equipment and storage medium based on dual-track database Download PDFInfo
- Publication number
- CN113590643B CN113590643B CN202111139449.2A CN202111139449A CN113590643B CN 113590643 B CN113590643 B CN 113590643B CN 202111139449 A CN202111139449 A CN 202111139449A CN 113590643 B CN113590643 B CN 113590643B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- synchronization
- dual
- track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data synchronization method, a device, equipment and a storage medium based on a dual-track database. The method comprises the following steps: intercepting a double-track database execution statement, and obtaining a target execution statement of a target database according to the double-track database execution statement; updating the data in the target database according to the target execution statement; acquiring a preset mapping relation between data in a target database and data in the double-track database; acquiring synchronous information corresponding to the updated data in the target database; and synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation. The method can ensure the accuracy of data synchronization.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a data synchronization method, apparatus, device, and storage medium based on a dual-track database.
Background
With the development of computer technology, each software system is updated and iterated, and the situation of architecture upgrading using different bottom databases exists.
In the traditional technology, the simultaneous access use needs to be supported during the switching commissioning of the new system and the old system, and the dual-rail operation is realized. During the operation of the dual tracks of the system, the data synchronization of the old database and the new database needs to be ensured.
However, data synchronization of the existing old and new databases is performed in a message queue manner, that is, after data update is performed on one database, updated data is obtained, and the other database is updated in a message queue manner, so that the same information stored in the two databases is ensured, but when the same data is concurrently modified in the two databases, synchronization of the data cannot be realized, and a problem occurs in data synchronization.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data synchronization method, apparatus, device and storage medium based on a dual-track database, which can improve the accuracy of data synchronization.
A dual-track database-based data synchronization method, the method comprising:
intercepting a double-track database execution statement, and obtaining a target execution statement of a target database according to the double-track database execution statement;
updating the data in the target database according to the target execution statement;
acquiring a preset mapping relation between data in a target database and data in the double-track database;
acquiring synchronous information corresponding to the updated data in the target database;
and synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation.
In one embodiment, the obtaining synchronization information corresponding to the updated data in the target database includes:
acquiring synchronous information of each datum in the target database;
and screening to obtain updated data according to the synchronous information.
In one embodiment, the synchronization information includes a synchronization interval time; the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes:
counting all updated data in the synchronization interval time;
synchronizing the counted update data into the dual-track database according to the mapping relation.
In one embodiment, the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes:
acquiring the current resource use condition of the target database;
acquiring the number of synchronous threads according to the current resource use condition;
and starting corresponding synchronous threads according to the number of the synchronous threads, and synchronizing the updated data into the dual-track database according to the synchronous information and the mapping relation through the started synchronous threads.
In one embodiment, the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes:
adding a data lock to the update data;
synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation;
and deleting the data lock after the synchronization is completed.
In one embodiment, after synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship, the method includes:
and acquiring a synchronization result of the updating data, and storing the synchronization result into synchronization information corresponding to the updating data.
In one embodiment, the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes:
judging whether the number of times of failure of the synchronization result in the synchronization information of the updated data is less than a preset value;
and if so, synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation, otherwise, outputting the updating data.
A dual-track database-based data synchronization apparatus, the apparatus comprising:
the intercepting module is used for intercepting a double-track database execution statement and obtaining a target execution statement of a target database according to the double-track database execution statement;
the updating module is used for updating the data in the target database according to the target execution statement;
the mapping relation acquisition module is used for acquiring a preset mapping relation between data in a target database and data in the double-track database;
the synchronous information acquisition module is used for acquiring synchronous information corresponding to the updated data in the target database;
and the synchronization module is used for synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method in any of the above embodiments when the processor executes the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method in any of the above-mentioned embodiments.
According to the data synchronization method, the device, the equipment and the storage medium based on the double-track database, the update of the data is realized through the target database, the updated data of the target database is resynchronized to the double-track database according to the preset mapping relation between the data in the target database and the data in the double-track database and the synchronization information of the updated data, so that the update of the data of the double-track database can be realized only by introducing one target database, only the execution statement of the double-track database needs to be intercepted, the introduction of extra database introduction cost is not needed, in addition, the update of the double-track database is in the target database, the main keys are unified, and the accuracy of the data is ensured.
Drawings
FIG. 1 is a diagram of an application environment of a dual-track database-based data synchronization method in one embodiment;
FIG. 2 is a flow diagram of a dual-track database-based data synchronization method in one embodiment;
FIG. 3 is an architecture diagram of a database in one embodiment;
FIG. 4 is a mapping diagram of a mapping table in one embodiment;
FIG. 5 is a flowchart illustrating a method for dual-track database-based data synchronization according to another embodiment;
FIG. 6 is a timing diagram of a dual-rail database-based data synchronization method in one embodiment;
FIG. 7 is a block diagram of a dual-track database-based data synchronization apparatus according to an embodiment;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data synchronization method based on the dual-track database can be applied to the application environment shown in fig. 1. The server 102 is in communication with the operation end 104 and the target database 106 through a network, wherein the server 102 can intercept an execution statement of the operation end 104 on the dual-track database, and obtain a target execution statement of the target database according to the dual-track database execution statement, so that the server 102 updates data in the target database 106 according to the target execution statement, and further the server 102 obtains a preset mapping relationship between the data in the target database and the data in the dual-track database; acquiring synchronous information corresponding to updated data in a target database; the update data is synchronized into the dual-track database according to the synchronization information and the mapping relationship, wherein the dual-track database includes a first database 108 and a second database 110. Therefore, data updating is achieved through the target database, the updated data of the target database is resynchronized to the double-track database according to the preset mapping relation between the data in the target database and the data in the double-track database and the synchronization information of the updated data, so that the data updating of the double-track database can be achieved only by introducing one target database, only execution statements of the double-track database need to be intercepted, extra database introduction cost does not need to be introduced, in addition, the updating of the double-track database is achieved in the target database, the main keys are unified, and the accuracy of the data is guaranteed.
The operation end 104 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 102 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in fig. 2, a data synchronization method based on a dual-track database is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
s202: intercepting the execution statements of the double-track database, and obtaining target execution statements of the target database according to the execution statements of the double-track database.
In particular, the dual-rail database refers to a scenario that two databases are used at the same time, for example, the dual-rail operation needs to support the use of simultaneous access during the switching commissioning of the old and new systems. During the operation of the dual tracks of the system, the data synchronization of the old database and the new database needs to be ensured. The target database is a database intermediate storage layer created by transformation based on database driving, and the database intermediate storage layer integrates data in the dual-track database through a mapping relation.
Specifically, the server modifies the database drive, so that execution statements of the dual-track database can be intercepted, and the execution statements of the dual-track database are processed to obtain target execution statements of the target database, so that the application program only needs to add corresponding dependency packages and configuration, the application program can be prevented from being introduced into the intermediate storage layer of the database, and additional modification cost for introducing the intermediate storage layer of the database is avoided.
The processing of the dual-track database executed by the dual-track database may be to replace the name of the dual-track database in the statement with the name of the target database.
S204: and updating the data in the target database according to the target execution statement.
Specifically, the target database executes the target execution statement to complete the update of the data in the target database.
S206: and acquiring a preset mapping relation between the data in the target database and the data in the dual-track database.
Specifically, the mapping relationship is preset, and as shown in fig. 3 and fig. 4, an application scenario is defined as a dual-track running system, the dual-track running system, for example, two databases respectively corresponding to the dual-track system a and the dual-track system B are a database a and a database B, and the intermediate storage layer of the databases integrates the database a and the database B to obtain the corresponding mapping relationship.
The server relates tables of the database A and the database B through mapping, and creates a table E in the intermediate storage layer of the database for a table C and a table D with the same table name and field name in the database A and the database B, wherein the table E is the same as the table C and the table D. And adding the association of the table E, the table C and the table D in the mapping table one by one. For a table C and a table D with different table names or field names in the database A and the database B, a table E is created in the middle storage layer of the database, and the table E is the union of the table C and the table D. And adding the association of the table E, the table C and the table D in the mapping table one by one. For table C, which exists only in the a or B databases, table E is created in the intermediate storage layer of the database, which is the same as table C. The association of table E and table C is added one by one in the mapping table. The mapping association table is generated as in fig. 4.
The management mechanism of the mapping relation mainly comprises two aspects, namely, automatically creating the mapping relation with the consistent table name and field name in the first database and the second database through a metadata management table of the database; and secondly, the data with different table names or field names in the first database and the second database can be mapped and configured or the automatically mapped configuration data in the first database can be modified through manual configuration. For the configured mapping relation, when the table name, the field name and the field type are unchanged, the mapping can be automatically synchronized by refreshing; for the case of table name or field name modification, it is necessary to automatically create a mapping relationship or perform manual configuration according to the above case.
S208: and acquiring synchronous information corresponding to the updated data in the target database.
Specifically, the synchronization information is synchronization information of each data in the target database, where each data in the target database stores the synchronization information correspondingly, and the server may determine to update the data according to the synchronization information. Specifically, the synchronization information may include at least one of a synchronization status, a synchronization time field, a number of synchronization failures, a synchronization failure reason, and a synchronization failure correspondence library. The synchronization information at least includes a synchronization status, and the synchronization time field is used for determining the synchronization time of the data to synchronize the data in batches within the same synchronization time. The number of synchronization failures, the reason for the synchronization failure, and the synchronization failure correspondence library may include at least one to facilitate re-synchronization.
S210: and synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation.
Specifically, the server determines a dual-track database corresponding to the update data according to the mapping relationship, and then synchronizes the update data to a data table in the corresponding dual-track database.
Specifically, if the updated data c of the data table b of the database a is determined according to the mapping relationship, the data is synchronized to the data c of the data table b of the database a to complete the data synchronization.
And the database drive is used for transforming to create a database intermediate storage layer, and the database intermediate storage layer integrates data in the dual-track database through a mapping relation. The double-track system can read and write data from the intermediate storage layer of the database without application transformation by modifying the database drive, the intermediate storage layer of the database is distributed to the double-track database in real time through a distribution strategy, real-time synchronization of the data is realized, and the data integrity is ensured through real-time data synchronization confirmation. Because the double-track system directly operates the intermediate storage layer of the database, the problem of concurrent operation of the same data can be solved through a locking mechanism, and the uniformity of the primary key can also be ensured.
According to the data synchronization method based on the double-track database, data updating is achieved through the target database, the updated data of the target database is resynchronized to the double-track database according to the preset mapping relation between the data in the target database and the data in the double-track database and the synchronization information of the updated data, so that the data updating of the double-track database can be achieved only by introducing one target database, only execution statements of the double-track database need to be intercepted, extra database introduction cost does not need to be introduced, in addition, the updating of the double-track database is achieved in the target database, the main keys are unified, and the accuracy of the data is guaranteed.
In one embodiment, obtaining synchronization information corresponding to updated data in a target database includes: acquiring synchronous information of each datum in a target database; and screening to obtain updated data according to the synchronous information.
Specifically, in this embodiment, the synchronization information includes a synchronization state, so that the server acquires the synchronization information of each data in the target database, that is, acquires the synchronization state of each data, and thus may acquire the data whose synchronization state is unsynchronized to obtain the update data, so as to update the update data to the corresponding dual-track database.
The obtained updating data comprises the data which is really updated in the time period and the updating data which fails to be synchronized in the previous time period, so that the updating data which fails to be synchronized can be updated again, and the success rate of synchronization is improved.
In one embodiment, the synchronization information includes a synchronization interval time; synchronizing the update data into the dual-track database according to the synchronization information and the mapping relation, comprising: counting all updated data in the synchronization interval time; and synchronizing the counted updated data into the dual-rail database according to the mapping relation.
Specifically, the synchronization time interval refers to a period of time, and the server collects update data of the target database in the period of time and synchronizes the update data to the dual-track database together, wherein optionally, the server may acquire the update data in real time in the synchronization time interval and store the update data in a corresponding synchronization queue, so that when the synchronization time interval is reached, the update data in the synchronization queue are synchronized together to the dual-track database, and preferably, in order to improve synchronization efficiency, the number of the synchronization queues is two, and each of the two synchronization queues corresponds to one of the dual-track databases.
Considering that the updating frequency of the tables is different under different service scenes, and because the probability of modifying the same data table within a certain period of time is higher, a synchronization strategy can be set for each table; and the synchronous interval time of the corresponding table is set, so that the synchronous pushing of the same table for multiple times in a short time is avoided. Through the database intermediate layer, the dual-track system does not directly operate the first database and the second database, only pays attention to the real-time accuracy of the data in the database intermediate layer, has low requirement on the real-time performance of the data in the first database and the second database, and only needs to ensure the final consistency of the data. And the middle layer of the database is synchronized every minute, and a table needing synchronization in the interval section is inquired according to a configuration rule to synchronize data.
Therefore, because the probability of modifying the same data table within a certain period of time is higher, a synchronization strategy can be set for each table; and the synchronous interval time of the corresponding table is set, so that the synchronous pushing of the same table for multiple times in a short time is avoided.
In one embodiment, synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes: acquiring the current resource use condition of a target database; acquiring the number of synchronous threads according to the current resource use condition; and starting the corresponding synchronous threads according to the number of the synchronous threads, and synchronizing the updated data into the dual-track database according to the synchronous information and the mapping relation through the started synchronous threads.
Specifically, the current resource usage includes at least one of a current CPU occupancy, a current memory usage, and a number of connections of the intermediate storage tier of the database. Preferably, the server determines the number of distribution threads, that is, the number of synchronization threads, according to the current CPU occupancy rate of the server in the intermediate storage tier of the database, the current memory usage rate, and the number of connections of the intermediate storage tier of the database, for example, a preset threshold value and a corresponding processing thread number of the server. And selecting a proper thread number according to the comparison between the current three factors of the server and the threshold, equally distributing the data to be synchronized to each thread, and performing synchronous data distribution processing. When the pressure of the server in the intermediate storage layer of the database is high, the number of the synchronous data threads is reduced, and the pressure of the server is prevented from being increased.
The distribution rule of the number of the synchronous threads is mainly distributed according to the performance of the current intermediate database server. The number of distribution threads is determined according to the current CPU occupancy rate (factor A), the current memory utilization rate (factor B) and the connection number of the intermediate storage layers of the database (factor C), and a threshold value and the corresponding processing thread number can be preset. And selecting a proper thread number according to the comparison between the current three factors of the server and the threshold, equally distributing the data to be synchronized to each thread, and performing synchronous data distribution processing.
In one embodiment, synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes: adding the updated data to a data lock; synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation; after synchronization is complete, the data lock is deleted.
Specifically, in this embodiment, during data distribution, for each piece of data being synchronized, a locking operation is performed on the piece of data in the middle layer of the database, the data are synchronized to the database a and the database B according to the mapping relationship, and an unlocking operation is performed after the synchronization result is recorded.
Therefore, the synchronization failure caused by the data being operated in the synchronization process is avoided.
In one embodiment, after synchronizing the update data into the dual-rail database according to the synchronization information and the mapping relationship, the method includes: and acquiring a synchronization result of the update data, and storing the synchronization result into synchronization information corresponding to the update data.
Specifically, in this embodiment, the intermediate storage layer of the database records the result of synchronizing the database a and the database B in real time, and the consistency of the data is ensured through a real-time confirmation mechanism.
In one embodiment, synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes: judging whether the number of times of failure of the synchronization result in the synchronization information of the updated data is less than a preset value; if yes, the updating data are synchronized into the dual-track database according to the synchronization information and the mapping relation, and if not, the updating data are output.
Specifically, in this embodiment, for the record of synchronization failure, when the number of failures is less than 5 times, the next task still performs synchronization. When the failure times exceed 5 times, the system administrator is informed to manually troubleshoot the problems.
Specifically, referring to fig. 5 and fig. 6, fig. 5 is a schematic flowchart of a dual-rail database-based data synchronization method in another embodiment, and fig. 6 is a timing diagram of the dual-rail database-based data synchronization method in an embodiment.
Wherein, an intermediate layer of the database, namely the data to be synchronized in the target data, is obtained according to the synchronization strategy, whether the synchronization failure times of the data to be synchronized is less than a preset value, for example, 5 times is judged, if the synchronization failure times reach 5 times, an administrator is informed, otherwise, a preset thread number is obtained according to the current performance of the server, a corresponding synchronization thread is created, the data to be synchronized is distributed to each synchronization thread, for example, the data to be synchronized is evenly distributed to each synchronization thread, the server traverses the data to be synchronized in the target database, adds a data lock to the data to be synchronized, obtains a preset mapping relation, specifically, reads the corresponding mapping relation from the mapping table for each synchronization thread according to the distributed target data in parallel, synchronizes the data to be synchronized to the database A and the database B according to the mapping relation in parallel, and returns the synchronization state of the data to be synchronized, and unlocking the data to be synchronized.
For example, the data stored in the database is data of an application system, and the application system no longer reads data from the database a and the database B, but reads data from the intermediate storage layer of the database, that is, reads data from the target database, and updates the data of the intermediate storage layer of the database.
And the server reads the updating data according to the synchronization time interval, and distributes the updating data to the corresponding synchronization thread to synchronize the updating data to the database A and the database B. Aiming at the abnormal condition occurring in the synchronization process, the dual-track system does not directly operate the first database and the second database through the database middle layer, only the real-time accuracy of the data in the database middle layer is emphasized, the requirement on the real-time performance of the data in the first database and the second database is not high, and only the final consistency of the data is ensured. And the database primary key is mainly managed in the intermediate layer of the database, the first database and the second database are mainly synchronous data, the data synchronization fails, the data of the intermediate database is not influenced, the data in the synchronization module of the intermediate database is still in an unsynchronized state, and the data can be inserted into or updated into the first database and the second database according to the primary key only by pushing again through the synchronization module after the intermediate database is recovered. Due to the consistency of the primary key, duplication or confusion of the first database or the second database data is not caused.
In the embodiment, because the double-track application database does not directly operate data, the data are operated uniformly through the intermediate storage layer of the database, and the primary keys are uniformly arranged in the intermediate storage layer of the database; when the same data is modified concurrently, management is performed through transactions stored in the middle of the database, and the dual-track application database only focuses on the finally synchronized data. And the timeliness and the data integrity of the data are ensured through a real-time distribution strategy and data consistency check of an intermediate storage layer of the database.
It should be understood that although the steps in the flowcharts of fig. 2, 5 and 6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 5 and 6 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 7, there is provided a dual-track database-based data synchronization apparatus, including: an interception module 701, an update module 702, a mapping relationship acquisition module 703, a synchronization information acquisition module 704, and a synchronization module 705, wherein:
the intercepting module 701 is used for intercepting the execution statements of the double-track database and obtaining target execution statements of the target database according to the execution statements of the double-track database;
an update module 702, configured to update data in the target database according to the target execution statement;
a mapping relation obtaining module 703, configured to obtain a mapping relation between data in a preset target database and data in a dual-track database;
a synchronization information obtaining module 704, configured to obtain synchronization information corresponding to updated data in a target database;
and a synchronization module 705 for synchronizing the update data into the dual-rail database according to the synchronization information and the mapping relationship.
In one embodiment, the synchronization information obtaining module 704 includes:
the synchronous information acquisition unit is used for acquiring the synchronous information of each datum in the target database;
and the updating data acquisition unit is used for screening and obtaining the updating data according to the synchronous information.
In one embodiment, the synchronization information includes a synchronization interval time; the synchronization module 705 includes:
a counting unit for counting all the updated data in the synchronization interval time;
and the synchronization unit is used for synchronizing the counted updated data into the dual-rail database according to the mapping relation.
In one embodiment, the synchronization module 705 includes:
the resource use condition acquisition unit is used for acquiring the current resource use condition of the target database;
the quantity obtaining unit is used for obtaining the quantity of the synchronous threads according to the current resource use condition;
and the first synchronization unit is used for starting the corresponding synchronization threads according to the number of the synchronization threads and synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation through the started synchronization threads.
In one embodiment, the synchronization module 705 includes:
the locking unit is used for adding the updated data into the data lock;
the second synchronization unit is used for synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation;
and the deleting unit is used for deleting the data lock after the synchronization is finished.
In one embodiment, the apparatus further includes:
and the storage module is used for acquiring the synchronization result of the update data and storing the synchronization result into the synchronization information corresponding to the update data.
In one embodiment, the synchronization module 705 includes:
the judging unit is used for judging whether the number of times of failure of the synchronization result in the synchronization information of the updating data is less than a preset value;
and the third synchronization unit is used for synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation if the updating data is in the positive state, and otherwise, outputting the updating data.
The above-mentioned data synchronization apparatus based on dual-rail database can refer to the above definition of the data synchronization method based on dual-rail database, and is not described herein again. The modules in the dual-track database-based data synchronization device can be implemented in whole or in part by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a dual-track database-based data synchronization method.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a dual-track database-based data synchronization method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program: intercepting a double-track database execution statement, and obtaining a target execution statement for a target database according to the double-track database execution statement; updating data in the target database according to the target execution statement; acquiring a preset mapping relation between data in a target database and data in a double-track database; acquiring synchronous information corresponding to updated data in a target database; and synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation.
In one embodiment, the obtaining synchronization information corresponding to the updated data in the target database, which is implemented when the processor executes the computer program, includes: acquiring synchronous information of each datum in a target database; and screening to obtain updated data according to the synchronous information.
In one embodiment, the synchronization information implemented when the computer program is executed by the processor includes a synchronization interval time; the synchronization of the update data into the dual-rail database according to the synchronization information and the mapping relation, which is realized when the processor executes the computer program, includes: counting all updated data in the synchronization interval time; and synchronizing the counted updated data into the dual-rail database according to the mapping relation.
In one embodiment, the synchronizing of the update data into the dual-rail database according to the synchronization information and the mapping relationship, which is implemented when the processor executes the computer program, includes: acquiring the current resource use condition of a target database; acquiring the number of synchronous threads according to the current resource use condition; and starting the corresponding synchronous threads according to the number of the synchronous threads, and synchronizing the updated data into the dual-track database according to the synchronous information and the mapping relation through the started synchronous threads.
In one embodiment, the synchronizing of the update data into the dual-rail database according to the synchronization information and the mapping relationship, which is implemented when the processor executes the computer program, includes: adding the updated data to a data lock; synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation; after synchronization is complete, the data lock is deleted.
In one embodiment, after the processor, when executing the computer program, synchronizes the update data into the dual-rail database according to the synchronization information and the mapping relationship, the method includes: and acquiring a synchronization result of the update data, and storing the synchronization result into synchronization information corresponding to the update data.
In one embodiment, the synchronizing of the update data into the dual-rail database according to the synchronization information and the mapping relationship, which is implemented when the processor executes the computer program, includes: judging whether the number of times of failure of the synchronization result in the synchronization information of the updated data is less than a preset value; if yes, the updating data are synchronized into the dual-track database according to the synchronization information and the mapping relation, and if not, the updating data are output.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: intercepting a double-track database execution statement, and obtaining a target execution statement for a target database according to the double-track database execution statement; updating data in the target database according to the target execution statement; acquiring a preset mapping relation between data in a target database and data in a double-track database; acquiring synchronous information corresponding to updated data in a target database; and synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation.
In one embodiment, the computer program, when executed by a processor, implements obtaining synchronization information corresponding to updated data in a target database, including: acquiring synchronous information of each datum in a target database; and screening to obtain updated data according to the synchronous information.
In one embodiment, the synchronization information implemented when the computer program is executed by the processor includes a synchronization interval time; the computer program, when executed by the processor, implements synchronizing the update data into the dual-rail database according to the synchronization information and the mapping relationship, including: counting all updated data in the synchronization interval time; and synchronizing the counted updated data into the dual-rail database according to the mapping relation.
In one embodiment, the computer program, when executed by a processor, implements synchronizing update data into a dual-rail database according to synchronization information and mapping relationships, comprising: acquiring the current resource use condition of a target database; acquiring the number of synchronous threads according to the current resource use condition; and starting the corresponding synchronous threads according to the number of the synchronous threads, and synchronizing the updated data into the dual-track database according to the synchronous information and the mapping relation through the started synchronous threads.
In one embodiment, the computer program, when executed by a processor, implements synchronizing update data into a dual-rail database according to synchronization information and mapping relationships, comprising: adding the updated data to a data lock; synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation; after synchronization is complete, the data lock is deleted.
In one embodiment, the computer program, when executed by the processor, after synchronizing the update data into the dual-rail database according to the synchronization information and the mapping relationship, includes: and acquiring a synchronization result of the update data, and storing the synchronization result into synchronization information corresponding to the update data.
In one embodiment, the computer program, when executed by a processor, implements synchronizing update data into a dual-rail database according to synchronization information and mapping relationships, comprising: judging whether the number of times of failure of the synchronization result in the synchronization information of the updated data is less than a preset value; if yes, the updating data are synchronized into the dual-track database according to the synchronization information and the mapping relation, and if not, the updating data are output.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for synchronizing data based on a dual-track database, the method comprising:
intercepting a double-track database execution statement, and obtaining a target execution statement of a target database according to the double-track database execution statement, wherein the double-track database is used for storing new and old data and comprises a first database and a second database;
updating the data in the target database according to the target execution statement;
acquiring a preset mapping relation between data in a target database and data in the double-track database;
acquiring synchronous information corresponding to updated data in the target database, wherein the synchronous information comprises a synchronous state;
acquiring updating data in real time in a synchronization time interval and storing the updating data into a corresponding synchronization queue; the two synchronous queues respectively correspond to one of the double-track databases;
and synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation.
2. The method of claim 1, wherein the obtaining synchronization information corresponding to the updated data in the target database comprises:
acquiring synchronous information of each datum in the target database;
and screening to obtain updated data according to the synchronous information.
3. The method of claim 1, wherein the synchronization information comprises a synchronization interval time; the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship includes:
counting all updated data in the synchronization interval time;
synchronizing the counted update data into the dual-track database according to the mapping relation.
4. The method according to any one of claims 1 to 3, wherein the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship comprises:
acquiring the current resource use condition of the target database;
acquiring the number of synchronous threads according to the current resource use condition;
and starting corresponding synchronous threads according to the number of the synchronous threads, and synchronizing the updated data into the dual-track database according to the synchronous information and the mapping relation through the started synchronous threads.
5. The method according to any one of claims 1 to 3, wherein the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship comprises:
adding a data lock to the update data;
synchronizing the updated data into the dual-track database according to the synchronization information and the mapping relation;
and deleting the data lock after the synchronization is completed.
6. The method according to any one of claims 1 to 3, wherein after synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship, the method comprises:
and acquiring a synchronization result of the updating data, and storing the synchronization result into synchronization information corresponding to the updating data.
7. The method of claim 6, wherein the synchronizing the update data into the dual-track database according to the synchronization information and the mapping relationship comprises:
judging whether the number of times of failure of the synchronization result in the synchronization information of the updated data is less than a preset value;
and if so, synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation, otherwise, outputting the updating data.
8. A data synchronization apparatus based on a dual-track database, the apparatus comprising:
the system comprises an interception module, a first database and a second database, wherein the interception module is used for intercepting a double-track database execution statement and obtaining a target execution statement of a target database according to the double-track database execution statement, the double-track database is used for storing new and old data, and comprises the first database and the second database;
the updating module is used for updating the data in the target database according to the target execution statement;
the mapping relation acquisition module is used for acquiring a preset mapping relation between data in a target database and data in the double-track database;
the synchronous information acquisition module is used for acquiring synchronous information corresponding to the updated data in the target database, and the synchronous information comprises a synchronous state;
the synchronization module is used for acquiring the updating data in real time within a synchronization time interval and storing the updating data into a corresponding synchronization queue; the two synchronous queues respectively correspond to one of the double-track databases; and synchronizing the updating data into the dual-track database according to the synchronization information and the mapping relation.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111139449.2A CN113590643B (en) | 2021-09-28 | 2021-09-28 | Data synchronization method, device, equipment and storage medium based on dual-track database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111139449.2A CN113590643B (en) | 2021-09-28 | 2021-09-28 | Data synchronization method, device, equipment and storage medium based on dual-track database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113590643A CN113590643A (en) | 2021-11-02 |
| CN113590643B true CN113590643B (en) | 2022-03-18 |
Family
ID=78242114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111139449.2A Active CN113590643B (en) | 2021-09-28 | 2021-09-28 | Data synchronization method, device, equipment and storage medium based on dual-track database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590643B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114896261B (en) * | 2022-06-22 | 2024-04-05 | 中国平安财产保险股份有限公司 | Database upgrading method and device, computer equipment and storage medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8868497B2 (en) * | 2009-06-12 | 2014-10-21 | Sap Ag | Correlation aware synchronization for near real-time decision support |
| US9129000B2 (en) * | 2010-04-30 | 2015-09-08 | International Business Machines Corporation | Method and system for centralized control of database applications |
| CN102063502B (en) * | 2011-01-06 | 2013-07-31 | 天津神舟通用数据技术有限公司 | Method for realizing synchronization of data in heterogeneous database |
| CN102129478B (en) * | 2011-04-26 | 2012-10-03 | 广州从兴电子开发有限公司 | Database synchronization method and system thereof |
| US10838827B2 (en) * | 2015-09-16 | 2020-11-17 | Richard Banister | System and method for time parameter based database restoration |
| US10585874B2 (en) * | 2016-04-25 | 2020-03-10 | International Business Machines Corporation | Locking concurrent commands in a database management system |
| EP3252618B1 (en) * | 2016-05-30 | 2021-04-28 | Sap Se | Database integration system |
| CN109885581B (en) * | 2019-03-14 | 2021-09-17 | 苏州达家迎信息技术有限公司 | Database synchronization method, device, equipment and storage medium |
| CN112306994A (en) * | 2020-11-10 | 2021-02-02 | 北京沃东天骏信息技术有限公司 | Database data migration method and device and storage medium |
-
2021
- 2021-09-28 CN CN202111139449.2A patent/CN113590643B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113590643A (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102392944B1 (en) | Data backup methods, storage media and computing devices | |
| CN107958010B (en) | Method and system for online data migration | |
| CN109683826A (en) | Expansion method and device for distributed memory system | |
| CN111399764B (en) | Data storage method, data reading device, data storage equipment and data storage medium | |
| CN110727724B (en) | Data extraction method and device, computer equipment and storage medium | |
| CN106648994B (en) | Method, equipment and system for backing up operation log | |
| CN107423404B (en) | Flow instance data synchronous processing method and device | |
| US20230418811A1 (en) | Transaction processing method and apparatus, computing device, and storage medium | |
| CN107040576A (en) | Information-pushing method and device, communication system | |
| CN111177144A (en) | Table synchronization method, device, equipment, storage medium and system | |
| CN113760513A (en) | Distributed task scheduling method, device, equipment and medium | |
| CN111737227A (en) | Data modification method and system | |
| CN114443908A (en) | Graph database construction method, system, terminal and storage medium | |
| CN111291062B (en) | Data synchronous writing method and device, computer equipment and storage medium | |
| CN113590643B (en) | Data synchronization method, device, equipment and storage medium based on dual-track database | |
| CN114443294A (en) | Big data service component deployment method, system, terminal and storage medium | |
| CN107391539B (en) | Transaction processing method, server and storage medium | |
| CN109462661A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| CN111090701B (en) | Service request processing method, device, readable storage medium and computer equipment | |
| CN116303789A (en) | Parallel synchronization method and device for multi-fragment multi-copy database and readable medium | |
| CN116974983A (en) | Data processing method, device, computer readable medium and electronic equipment | |
| JP2023546818A (en) | Transaction processing method, device, electronic device, and computer program for database system | |
| CN110569231B (en) | Data migration method, device, equipment and medium | |
| CN115292394A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN116107801A (en) | Transaction processing method and related product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |