CN107741965B - Database synchronous processing method and device, computing equipment and computer storage medium - Google Patents
Database synchronous processing method and device, computing equipment and computer storage medium Download PDFInfo
- Publication number
- CN107741965B CN107741965B CN201710923159.4A CN201710923159A CN107741965B CN 107741965 B CN107741965 B CN 107741965B CN 201710923159 A CN201710923159 A CN 201710923159A CN 107741965 B CN107741965 B CN 107741965B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- synchronized
- synchronous
- buffer area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database synchronous processing method, a device, a computing device and a computer storage medium, wherein the database synchronous processing method comprises the following steps: writing data to be synchronized in a main database into a buffer area of the main database, and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database; and transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer area of the master database, and updating the synchronous data record in the buffer area of the slave database. According to the technical scheme provided by the invention, the corresponding time stamp is set for the data to be synchronized, so that the synchronous data record of the slave database can be effectively updated, the transmission condition of the data to be synchronized from the buffer area of the master database to the slave database is clearly recorded, the data coverage transmission is avoided, and the synchronous processing efficiency of the database is improved.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a database synchronous processing method, a database synchronous processing device, a computing device and a computer storage medium.
Background
There are many database storage schemes in the prior art, and in some storage schemes, the database includes a master database and at least one slave database, and data in the master database needs to be synchronized to the slave database. Taking the Redis storage scheme as an example, in the process of synchronizing data in the master database to the slave database, the data in the master database needs to be written into the master database buffer, and then the data needs to be synchronized to the slave database according to the data written in the master database buffer. However, when the synchronization process is disconnected due to a network disconnection or the like, the data in the master database needs to be written into the master database buffer again, and then the data transmission is resumed from the first data written in the master database buffer, so that the data that has been transmitted before the synchronization process is disconnected also needs to be retransmitted. Much unnecessary time is consumed in this process, and thus there is a problem in that the database synchronization process is inefficient.
Disclosure of Invention
In view of the above, the present invention has been made to provide a database synchronization processing method, apparatus, computing device and computer storage medium that overcome or at least partially solve the above-mentioned problems.
According to an aspect of the present invention, there is provided a database synchronization processing method, including:
writing data to be synchronized in a main database into a buffer area of the main database, and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database; and
and transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer area of the master database, and updating the synchronous data record in the buffer area of the slave database.
According to another aspect of the present invention, there is provided a database synchronization processing apparatus including:
the write-in module is suitable for writing the data to be synchronized in the main database into a buffer area of the main database and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database;
the transmission module is suitable for transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer area of the master database; and
and the updating module is suitable for updating the synchronous data record in the slave database buffer area.
According to yet another aspect of the present invention, there is provided a computing device comprising: the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the database synchronous processing method.
According to still another aspect of the present invention, there is provided a computer storage medium having at least one executable instruction stored therein, where the executable instruction causes a processor to perform operations corresponding to the database synchronization processing method.
According to the technical scheme provided by the invention, data to be synchronized in a master database is written into a master database buffer area, a corresponding time stamp is set for the written data to be synchronized in the master database buffer area, then the data to be synchronized is transmitted into a slave database according to the data to be synchronized written in the master database buffer area, and a synchronous data record is updated in the slave database buffer area. According to the technical scheme provided by the invention, the corresponding time stamp is set for the data to be synchronized, so that the synchronous data record of the slave database can be effectively updated, the transmission condition of the data to be synchronized from the buffer area of the master database to the slave database is clearly recorded, the data overlay transmission is avoided, and the synchronous processing efficiency of the database is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow diagram illustrating a database synchronization processing method according to one embodiment of the invention;
FIG. 2 is a flow diagram illustrating a database synchronization processing method according to another embodiment of the invention;
FIG. 3 is a flow diagram illustrating a database synchronization processing method according to yet another embodiment of the invention;
fig. 4 is a block diagram showing the structure of a database synchronization processing apparatus according to an embodiment of the present invention;
fig. 5 is a block diagram showing the configuration of a database synchronization processing apparatus according to another embodiment of the present invention;
FIG. 6 shows a schematic structural diagram of a computing device according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Fig. 1 is a flow chart illustrating a database synchronization processing method according to an embodiment of the present invention, and as shown in fig. 1, the method includes the following steps:
and step S100, writing the data to be synchronized in the master database into a master database buffer, and setting a corresponding time stamp for the written data to be synchronized in the master database buffer.
In the prior art, a master database has a master database buffer, and a slave database does not have a corresponding buffer, and in order to enable the slave database to record synchronous data records, the invention provides the slave database with a slave database buffer for storing the synchronous data records. The method provided by the invention is executed according to the master database buffer area of the master database and the slave database buffer area set for the slave database.
When synchronization is needed, data to be synchronized in the main database is written into a buffer area of the main database, and in addition, in order to determine a synchronous continuous transmission position, a corresponding time stamp is set for the written data to be synchronized in the buffer area of the main database. Specifically, a corresponding time stamp is set for each piece of data to be synchronized, and thus, the data to be synchronized corresponds to the time stamp one to one. Wherein the accuracy of the set time stamp is of the order of microseconds.
A person skilled in the art can select a specific setting mode according to actual needs to set a corresponding timestamp for the data to be synchronized, which is not limited herein. For example, the data content of the data to be synchronized can be distinguished from the set timestamp by a preset symbol, specifically, the preset symbol can be added after the data content of the data to be synchronized, and then the corresponding timestamp is set after the preset symbol, so that for a certain piece of data to be synchronized, the content before the preset symbol is the data content of the piece of data to be synchronized, and the content after the preset symbol is the corresponding timestamp. The skilled person can set the preset symbol according to the actual need, and the preset symbol is not limited herein. For example, the preset symbols may be three apostrophes.
Step S101, according to the data to be synchronized written in the buffer area of the master database, the data to be synchronized is transmitted to the slave database, and the synchronous data record is updated in the buffer area of the slave database.
Specifically, the data to be synchronized is transmitted to the slave database according to the writing sequence of the data to be synchronized in the buffer area of the master database, and the synchronization data record is updated in the buffer area of the slave database. Wherein, the synchronous data record is updated in real time according to the current transmission completion condition. Specifically, synchronizing the data records includes: the data to be synchronized and the corresponding time stamp are transmitted at present.
For example, after the data to be synchronized 1 is transmitted to the slave database, the synchronization data record in the slave database buffer is updated to the synchronization data record 1, wherein the synchronization data record 1 includes the data to be synchronized 1 and the corresponding timestamp time 1; then, when the data to be synchronized 2 is transferred to the slave database, the synchronization data record 1 in the slave database buffer is updated to be the synchronization data record 2, wherein the synchronization data record 2 comprises the data to be synchronized 2 and the corresponding timestamp time 2. In this way, the transmission condition of the data to be synchronized from the buffer area of the master database to the slave database can be clearly and effectively recorded.
Specifically, when the data to be synchronized is transmitted to the slave database, the slave database can filter the timestamp from the received data to be synchronized, only the data content of the data to be synchronized is stored in the slave database, and the timestamp corresponding to the data to be synchronized is not stored in the slave database, so that the storage space of the slave database is not consumed to store the timestamp, and the waste of the storage space of the slave database is avoided. For example, if a preset symbol is added after the data content of the data to be synchronized in step S100 and then a corresponding timestamp is set after the preset symbol, when the data to be synchronized is transmitted to the slave database, the slave database may take only the content before the preset symbol (i.e., the data content of the data to be synchronized) from the received data to be synchronized and store the content, and discard the content after the preset symbol (i.e., the timestamp corresponding to the data to be synchronized).
According to the database synchronization processing method provided by the embodiment, data to be synchronized in the master database is written into the master database buffer, a corresponding time stamp is set for the written data to be synchronized in the master database buffer, then the data to be synchronized is transmitted to the slave database according to the data to be synchronized written in the master database buffer, and the synchronization data record is updated in the slave database buffer. According to the technical scheme provided by the invention, the corresponding time stamp is set for the data to be synchronized, so that the synchronous data record of the slave database can be effectively updated, the transmission condition of the data to be synchronized from the buffer area of the master database to the slave database is clearly recorded, the data overlay transmission is avoided, and the synchronous processing efficiency of the database is improved.
Fig. 2 is a flow chart illustrating a database synchronization processing method according to another embodiment of the present invention, and as shown in fig. 2, the method includes the following steps:
step S200, writing the data to be synchronized in the main database into a buffer area of the main database, and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database.
When synchronization is needed, data to be synchronized in the main database is written into a buffer area of the main database, and in order to determine a synchronous continuous transmission position, a corresponding time stamp is set for the written data to be synchronized in the buffer area of the main database. Specifically, a corresponding time stamp is set for each piece of data to be synchronized, and thus, the data to be synchronized corresponds to the time stamp one to one. Wherein the accuracy of the set time stamp is of the order of microseconds.
Step S201, according to the synchronous data record in the buffer area of the slave database, the synchronous continuous transmission position is determined in the buffer area of the master database.
When the data to be synchronized is transmitted to the slave database, the synchronization data record is updated in the slave database buffer, so that the synchronization continuous transmission position can be effectively and quickly determined in the master database buffer according to the synchronization data record in the slave database buffer. Specifically, according to the synchronous data record, the data to be synchronized matched with the synchronous data record is inquired in the buffer area of the main database, so that the synchronous continuous transmission position is determined.
By utilizing the method provided by the invention, when the synchronization process is interrupted and then recovered, the synchronous continuous transmission position can be effectively and quickly determined in the buffer area of the main database, and then the transmission of the data to be synchronized is continued according to the synchronous continuous transmission position without starting to transmit the data again from the first data written in the buffer area of the main database. Specifically, in case of network interruption or the like, the synchronization process is interrupted, the data to be synchronized cannot be continuously transmitted to the slave database, and when the synchronization process is resumed, since the synchronization data record is recorded in the slave database buffer, the synchronization resuming position may be determined in the master database buffer according to the synchronization data record in the slave database buffer in step S201.
Step S202, according to the determined synchronous continuous transmission position, transmitting the data to be synchronized in the buffer area of the master database to the slave database, and updating the synchronous data record in the buffer area of the slave database.
After the synchronous continuous transmission position is determined, the data to be synchronized in the buffer area of the master database is continuously transmitted to the slave database according to the determined synchronous continuous transmission position, and the synchronous data record is updated in the buffer area of the slave database, so that the data of the master database is not required to be written into the buffer area of the master database again even when the synchronous processing is recovered, and the data transmission is not required to be started again from the first data written into the buffer area of the master database. Wherein synchronizing the data records comprises: the data to be synchronized and the corresponding time stamp are transmitted at present.
According to the database synchronization processing method provided by the embodiment, the transmission condition of the data to be synchronized transmitted from the master database buffer area to the slave database can be clearly recorded, the synchronization continuous transmission position is effectively determined, the data transmission is continuously carried out according to the synchronization continuous transmission position, the data of the master database does not need to be written into the master database buffer area again even when the synchronization processing is recovered, and the data transmission is not required to be carried out again from the first data written into the master database buffer area, so that the data coverage type transmission is avoided, the database synchronization processing efficiency is improved, and the database synchronization processing mode is optimized.
Fig. 3 is a flow chart illustrating a database synchronization processing method according to another embodiment of the present invention, and as shown in fig. 3, the method includes the following steps:
step S300, writing the data to be synchronized in the master database into the master database buffer, and setting a corresponding time stamp for the written data to be synchronized in the master database buffer.
The method provided by the invention is executed according to the master database buffer area of the master database and the slave database buffer area set for the slave database. In order to facilitate the determination of the synchronous continuous transmission position, the invention sets a corresponding time stamp for the written data to be synchronized in the buffer of the master database. Specifically, a corresponding time stamp is set for each data to be synchronized. The accuracy of the set time stamp is in microsecond level, so that the data to be synchronized and the time stamp can be in one-to-one correspondence.
Step S301, the data to be synchronized in the buffer of the master database and the corresponding time stamp are transmitted to the slave database.
And during synchronization, according to the writing sequence of the data to be synchronized in the buffer area of the master database, transmitting the data to be synchronized and the corresponding time stamp in the buffer area of the master database to the slave database, and extracting the data to be synchronized from the received data to be synchronized and the corresponding time stamp from the slave database for storage.
Step S302, according to the data to be synchronized which is currently transmitted and the corresponding time stamp, the synchronous data record is updated in the slave database buffer area.
The synchronization data record is used to record the data to be synchronized that is currently finished transmitting, and the synchronization data record may include: the data to be synchronized and the corresponding time stamp are transmitted at present. Assuming that the data to be synchronized, which is currently finished to be transmitted, is the data to be synchronized 2, the data to be synchronized 2 and the corresponding time stamp 2 of the data to be synchronized 2 can be recorded in the synchronization data record.
Step S303 reads the synchronized data record from the database buffer.
For example, when the synchronization process is resumed after an interruption, the synchronization data record needs to be read from the database buffer in order to determine the synchronization continuation position.
Step S304, according to the data to be synchronized which is currently finished to be transmitted in the synchronous data record and the corresponding time stamp, the data to be synchronized which is matched with the synchronous data record is inquired in the buffer area of the main database.
After the synchronized data records are read, the master database buffer is queried for data to be synchronized that matches the synchronized data records. Specifically, the data to be synchronized matched with the synchronized data record is the data to be synchronized whose corresponding timestamp is consistent with the timestamp recorded in the synchronized data record, or the data to be synchronized matched with the synchronized data record is the data to be synchronized whose corresponding timestamp is consistent with the timestamp recorded in the synchronized data record and whose data content is consistent with the data content of the data to be synchronized recorded in the synchronized data record. Since the data to be synchronized corresponds to the time stamps one-to-one, the master database buffer may preferably be queried for data to be synchronized whose corresponding time stamps are consistent with the time stamps recorded in the synchronized data records.
And step S305, determining a synchronous continuous transmission position in the buffer area of the main database according to the data to be synchronized, which is obtained by query and is matched with the synchronous data record.
Specifically, a location corresponding to the data to be synchronized that matches the synchronized data record may be determined as a synchronized resume location in the master database buffer.
And step S306, transmitting the data to be synchronized in the buffer area of the master database to the slave database according to the determined synchronous continuous transmission position.
For example, data to be synchronized 1, data to be synchronized 2, data to be synchronized 3, data to be synchronized 4, data to be synchronized 5, and the like are sequentially written in the master database buffer, the data to be synchronized, which is obtained by query and matches with the synchronization data record, is the data to be synchronized 3, the position corresponding to the data to be synchronized 3 is determined as a synchronization continuous transmission position in the master database buffer, the data to be synchronized 4, the data to be synchronized 5, and the like in the master database buffer are sequentially transmitted to the slave database according to the synchronization continuous transmission position, and the synchronization data record is updated in the slave database buffer after the transmission of each data to be synchronized is completed, so that the data to be synchronized, which is currently completed in transmission, is recorded.
In addition, in the case that the synchronization process is interrupted and not recovered for a long time, the data to be synchronized in the master database, which needs to be written into the master database buffer, may overwrite the data to be synchronized in the master database buffer, which is not currently completed to be transmitted. In order to solve the problem, optionally, when the data to be synchronized in the master database, which needs to be written into the master database buffer, is to cover the data to be synchronized currently not completed for transmission in the master database buffer, the storage space of the master database buffer is expanded, and the expanded storage space is used for writing the data to be synchronized, which is to cover the data to be synchronized currently not completed for transmission in the master database buffer, so as to solve the problem that the data to be synchronized currently not completed for transmission in the master database buffer is covered. The size of the expanded storage space can be set by those skilled in the art according to actual needs, and is not limited herein. After the synchronous processing is recovered, a synchronous continuous transmission position is determined according to the data to be synchronized which is currently finished to be transmitted in the synchronous data record and the corresponding timestamp, and then continuous transmission is carried out according to the synchronous continuous transmission position. Alternatively, when the data to be synchronized written in the master database buffer is all transferred to the slave database, the size of the storage space of the expanded master database buffer may be restored to the size of the storage space of the original master database buffer.
According to the database synchronous processing method provided by the embodiment, the synchronous continuous transmission position can be quickly and accurately determined through the recorded data which are currently transmitted and the corresponding timestamp, and the data transmission is continued according to the synchronous continuous transmission position, so that data coverage type transmission is avoided, the time required by the database synchronous processing is greatly shortened, the database synchronous processing efficiency is improved, and the database synchronous processing mode is optimized.
Fig. 4 is a block diagram showing a configuration of a database synchronization processing apparatus according to an embodiment of the present invention, wherein the apparatus operates according to a master database buffer provided in a master database and a slave database buffer provided for a slave database. As shown in fig. 4, the apparatus includes: a write module 410, a transfer module 420, and an update module 430.
The writing module 410 is adapted to: and writing the data to be synchronized in the main database into a buffer area of the main database, and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database.
The transmission module 420 is adapted to: and transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer area of the master database.
The update module 430 is adapted to: the synchronized data records are updated in the slave database buffer.
Wherein synchronizing the data records comprises: the data to be synchronized and the corresponding time stamp are transmitted at present.
According to the database synchronization processing device provided by the embodiment, the corresponding timestamp is set for the data to be synchronized, so that the synchronization data record of the slave database can be effectively updated, the transmission condition of the data to be synchronized transmitted from the buffer area of the master database to the slave database is clearly recorded, the data coverage transmission is avoided, and the database synchronization processing efficiency is improved.
Fig. 5 is a block diagram showing a configuration of a database synchronization processing apparatus according to another embodiment of the present invention, wherein the apparatus operates according to a master database buffer provided in a master database and a slave database buffer provided for a slave database. As shown in fig. 5, the apparatus includes: a setup module 510, a write module 520, a transfer module 530, and an update module 540.
The setup module 510 is adapted to: a slave database buffer is provided for the slave database for storing the synchronized data records.
The writing module 520 is adapted to: and writing the data to be synchronized in the main database into a buffer area of the main database, and setting a corresponding time stamp for the written data to be synchronized in the buffer area of the main database. Wherein the accuracy of the set time stamp is of the order of microseconds. The data to be synchronized corresponds to the time stamps one to one.
The transmission module 530 is adapted to: and transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer area of the master database.
Optionally, the transmission module 530 is further adapted to: determining a synchronous continuous transmission position in the buffer area of the master database according to the synchronous data record in the buffer area of the slave database; and transmitting the data to be synchronized in the buffer area of the master database to the slave database according to the determined synchronous continuous transmission position.
Optionally, the transmission module 530 is further adapted to: reading the synchronized data record from the database buffer; inquiring data to be synchronized matched with the synchronous data records in a buffer area of a main database according to the data to be synchronized which is currently transmitted and the corresponding timestamp in the synchronous data records; and determining a synchronous continuous transmission position in the buffer area of the main database according to the data to be synchronized, which is obtained by query and is matched with the synchronous data record.
Optionally, the transmission module 530 is further adapted to: and transmitting the data to be synchronized in the buffer area of the master database and the corresponding time stamp to the slave database.
The update module 540 is adapted to: and updating the synchronous data record in the slave database buffer area according to the currently transmitted data to be synchronized and the corresponding timestamp.
According to the database synchronous processing device provided by the embodiment, the synchronous continuous transmission position can be quickly and accurately determined through the recorded data which are transmitted currently and the corresponding time stamp, and the data transmission is continued according to the synchronous continuous transmission position, so that the data coverage type transmission is avoided, the time required by the database synchronous processing is greatly shortened, the database synchronous processing efficiency is improved, and the database synchronous processing mode is optimized.
The invention also provides a nonvolatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the database synchronous processing method in any method embodiment.
Fig. 6 is a schematic structural diagram of a computing device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.
As shown in fig. 6, the computing device may include: a processor (processor)602, a communication Interface 604, a memory 606, and a communication bus 608.
Wherein:
the processor 602, communication interface 604, and memory 606 communicate with one another via a communication bus 608.
A communication interface 604 for communicating with network elements of other devices, such as clients or other servers.
The processor 602 is configured to execute the program 610, and may specifically perform relevant steps in the foregoing database synchronization processing method embodiment.
In particular, program 610 may include program code comprising computer operating instructions.
The processor 602 may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 606 for storing a program 610. Memory 606 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 610 may specifically be configured to enable the processor 602 to execute the database synchronization processing method in any of the above-described method embodiments. For specific implementation of each step in the program 610, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing database synchronization processing embodiment, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in accordance with embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (10)
1. A database synchronization processing method comprises the following steps:
writing data to be synchronized in a main database into a main database buffer, and setting a corresponding time stamp for the written data to be synchronized in the main database buffer; and
according to the data to be synchronized written in the buffer area of the master database, transmitting the data to be synchronized to a slave database, and updating a synchronous data record in the buffer area of the slave database;
wherein, the transmitting the data to be synchronized to the slave database according to the data to be synchronized written in the buffer of the master database further comprises:
determining a synchronous continuous transmission position in the buffer area of the master database according to a synchronous data record in the buffer area of the slave database, wherein the synchronous data record comprises data to be synchronized and corresponding time stamps, which are currently transmitted; and
according to the determined synchronous continuous transmission position, transmitting the data to be synchronized in the buffer area of the master database to the slave database;
the determining a synchronous resume location in the master database buffer from the synchronous data records in the slave database buffer further comprises:
reading the synchronized data record from the slave database buffer;
inquiring the data to be synchronized matched with the synchronous data record in the buffer area of the main database according to the data to be synchronized which is currently transmitted in the synchronous data record and the corresponding timestamp;
and determining the synchronous continuous transmission position in the buffer area of the main database according to the data to be synchronized, which is obtained by query and is matched with the synchronous data record.
2. The method of claim 1, wherein the transferring the data to be synchronized into a slave database in accordance with the data to be synchronized written in the master database buffer, and updating a synchronized data record in the slave database buffer further comprises:
transmitting the data to be synchronized in the buffer area of the master database and the corresponding time stamp to the slave database;
and updating the synchronous data record in the slave database buffer area according to the currently transmitted data to be synchronized and the corresponding timestamp.
3. The method of claim 1, wherein the timestamp is accurate to the order of microseconds.
4. The method according to any one of claims 1-3, wherein the method further comprises:
a slave database buffer is provided for the slave database for storing the synchronized data records.
5. A database synchronization processing apparatus comprising:
the system comprises a writing module, a synchronization module and a synchronization module, wherein the writing module is suitable for writing data to be synchronized in a main database into a main database buffer area and setting a corresponding time stamp for the written data to be synchronized in the main database buffer area;
the transmission module is suitable for transmitting the data to be synchronized to a slave database according to the data to be synchronized written in the buffer area of the master database; and
an update module adapted to update the synchronized data records in the slave database buffer;
the transmission module is further adapted to determine a synchronous continuous transmission position in the master database buffer according to a synchronous data record in the slave database buffer, wherein the synchronous data record comprises data to be synchronized and corresponding time stamp, which are currently finished to be transmitted; and
according to the determined synchronous continuous transmission position, transmitting the data to be synchronized in the buffer area of the master database to the slave database;
the transmission module is further adapted to read the synchronized data record from the slave database buffer; and
inquiring the data to be synchronized matched with the synchronous data record in the buffer area of the main database according to the data to be synchronized which is currently transmitted in the synchronous data record and the corresponding timestamp; and
and determining the synchronous continuous transmission position in the buffer area of the main database according to the data to be synchronized, which is obtained by query and is matched with the synchronous data record.
6. The apparatus of claim 5, wherein the transmission module is further adapted to: transmitting the data to be synchronized in the buffer area of the master database and the corresponding time stamp to the slave database;
the update module is further adapted to: and updating the synchronous data record in the slave database buffer area according to the currently transmitted data to be synchronized and the corresponding timestamp.
7. The apparatus of claim 5, wherein the timestamp is accurate to the order of microseconds.
8. The apparatus of any of claims 5-7, wherein the apparatus further comprises:
and the setting module is suitable for setting a slave database buffer area for the slave database, and is used for storing the synchronous data records.
9. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, and is characterized in that the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the database synchronization processing method according to any one of claims 1-4.
10. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the database synchronization processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710923159.4A CN107741965B (en) | 2017-09-30 | 2017-09-30 | Database synchronous processing method and device, computing equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710923159.4A CN107741965B (en) | 2017-09-30 | 2017-09-30 | Database synchronous processing method and device, computing equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107741965A CN107741965A (en) | 2018-02-27 |
| CN107741965B true CN107741965B (en) | 2021-06-29 |
Family
ID=61236674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710923159.4A Expired - Fee Related CN107741965B (en) | 2017-09-30 | 2017-09-30 | Database synchronous processing method and device, computing equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107741965B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109819048B (en) * | 2019-02-27 | 2022-03-15 | 北京字节跳动网络技术有限公司 | Data synchronization method, device, terminal and storage medium |
| CN110990435A (en) * | 2019-12-03 | 2020-04-10 | 秒针信息技术有限公司 | Data synchronization method, device and computer readable storage medium |
| CN111221861B (en) * | 2019-12-31 | 2023-06-02 | 武汉中海庭数据技术有限公司 | Method and device for synchronizing local cache and database data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102760126A (en) * | 2011-04-26 | 2012-10-31 | 阿里巴巴集团控股有限公司 | Data synchronization method in distributive system, and distributive system |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN104881494A (en) * | 2015-06-12 | 2015-09-02 | 北京奇虎科技有限公司 | Method, device and system for performing data synchronization with Redis server |
| CN105205175A (en) * | 2015-10-15 | 2015-12-30 | 北京农信互联科技有限公司 | Data operation method and system for distributed database cluster |
| US9507842B2 (en) * | 2013-04-13 | 2016-11-29 | Oracle International Corporation | System for replication-driven repository cache invalidation across multiple data centers |
-
2017
- 2017-09-30 CN CN201710923159.4A patent/CN107741965B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102760126A (en) * | 2011-04-26 | 2012-10-31 | 阿里巴巴集团控股有限公司 | Data synchronization method in distributive system, and distributive system |
| US9507842B2 (en) * | 2013-04-13 | 2016-11-29 | Oracle International Corporation | System for replication-driven repository cache invalidation across multiple data centers |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN104881494A (en) * | 2015-06-12 | 2015-09-02 | 北京奇虎科技有限公司 | Method, device and system for performing data synchronization with Redis server |
| CN105205175A (en) * | 2015-10-15 | 2015-12-30 | 北京农信互联科技有限公司 | Data operation method and system for distributed database cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107741965A (en) | 2018-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108234641B (en) | Data reading and writing method and device based on distributed consistency protocol | |
| CN108268501B (en) | Service processing method and device in online data migration process | |
| CN107741965B (en) | Database synchronous processing method and device, computing equipment and computer storage medium | |
| CN107590207B (en) | Data synchronization method and device and electronic equipment | |
| CN105630632A (en) | Virtual machine recovery method and virtual machine management device | |
| CN104462477A (en) | Data synchronizing method, device and system based on timestamp | |
| CN103049355B (en) | Method and equipment for database system recovery | |
| CN108304334B (en) | Application processor and integrated circuit including interrupt controller | |
| CN112015595B (en) | Master-slave database switching method, computing device and storage medium | |
| CN109656797B (en) | Log data association method and device | |
| CN105677256A (en) | Virtual disk system based on local caching and scheduling method | |
| CN107368563B (en) | Database data deleting method and device, electronic equipment and storage medium | |
| CN106682021B (en) | Database migration method and device | |
| EP3542519B1 (en) | Faster data transfer with remote direct memory access communications | |
| CN108205588B (en) | Data synchronization method and device based on master-slave structure | |
| CN104683486A (en) | Method and device for processing synchronous messages in distributed system and distributed system | |
| CN111552673B (en) | File processing method and device, electronic equipment and storage medium | |
| CN103761060B (en) | Data processing method and server | |
| CN111400213B (en) | Method, device and system for transmitting data | |
| CN109660310B (en) | Clock synchronization method and device, computing equipment and computer storage medium | |
| CN104461779A (en) | Distributed data storage method, device and system | |
| CN112035418A (en) | Multi-computer room synchronization method, computing device and computer storage medium | |
| CN102193884B (en) | Data transfer circuit and method | |
| CN112988880A (en) | Data synchronization method and device, electronic equipment and computer storage medium | |
| CN110765392A (en) | Data loading method and device, storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210629 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |