CN108205588A - Based on hypotactic method of data synchronization and device - Google Patents
Based on hypotactic method of data synchronization and device Download PDFInfo
- Publication number
- CN108205588A CN108205588A CN201711477361.5A CN201711477361A CN108205588A CN 108205588 A CN108205588 A CN 108205588A CN 201711477361 A CN201711477361 A CN 201711477361A CN 108205588 A CN108205588 A CN 108205588A
- Authority
- CN
- China
- Prior art keywords
- daily record
- data
- length
- node
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention discloses one kind to be based on hypotactic method of data synchronization and device.Wherein, method includes：When the first daily record that host node is read positioned at host node side occurs to read abnormal, according to abnormal conditions are read, determine that daily record elapses length；Host node elapses length according to the daily record and skips synch command to from node transmission daily record；Synch command is skipped from node according to the daily record, after the current sync offset of the second daily record of node side, fills positioned at described and elapses the matched blank log content of length with the daily record.Using this programme, it can be in main and subordinate node data synchronization process, ensure that host node has identical daily record synchronisation offset with from node, so as to avoid the loss of the data in synchronizing process, especially in the case of frequent main and subordinate node switches, the situation of loss of data can be greatly reduced, so as to improve the consistency of main and subordinate node data.
Description
Technical field
The present invention relates to field of computer technology, and in particular to one kind is based on hypotactic method of data synchronization and dress
It puts.
Background technology
In Database Systems, the hypotactic mode of generally use come realize reading and writing data detach, data restore etc., from
And the reliability of data is improved, reduce service node load and promotes the read-write efficiency of data.
In hypotactic Database Systems, the data to realize main and subordinate node synchronize, and host node can sequence transmission position
Journal file in host node side, the journal file sent from node according to host node are literary positioned at the daily record from node side to update
Part, so as to fulfill the synchronization of daily record, and then convenient for realizing that the data of main and subordinate node are synchronized according to journal file.
However, when the journal file that host node reading is recorded in host node side occurs to read abnormal, host node will be direct
The abnormal area is skipped to continue to send journal file, and the journal file from node before abnormal area is received to from node
Afterwards, directly receive the journal file after abnormal area, so as to cause host node side the offset that synchronizes of journal file with from section
The daily record synchronisation offset of point side is inconsistent, so as to easily cause the loss of data, especially after the frequent switching of main and subordinate node,
The situation of loss of data can be caused frequently to occur, so as to reduce the consistency of main and subordinate node data.
Invention content
In view of the above problems, it is proposed that the present invention overcomes the above problem in order to provide one kind or solves at least partly
State problem based on hypotactic method of data synchronization and device.
According to an aspect of the invention, there is provided a kind of be based on hypotactic method of data synchronization, including：
When the first daily record that host node is read positioned at host node side occurs to read abnormal, according to abnormal conditions are read, really
Settled date will elapses length；
Host node elapses length according to the daily record and skips synch command to from node transmission daily record；
Synch command is skipped from node according to the daily record, in the current sync positioned at second daily record from node side
After offset, filling elapses the matched blank log content of length with the daily record.
According to another aspect of the present invention, it provides one kind and is based on hypotactic data synchronization unit, including：
Determining module, when the first daily record suitable for being read when host node positioned at host node side occurs to read abnormal, according to reading
Abnormal conditions are taken, determine that daily record elapses length；
Order sending module elapses length according to the daily record suitable for host node and skips synchronous life to from node transmission daily record
It enables；
Fill module, suitable for skipping synch command according to the daily record from node, positioned at described from the second of node side
After the current sync offset of daily record, filling elapses the matched blank log content of length with the daily record.
According to another aspect of the invention, a kind of computing device is provided, including：Processor, memory, communication interface and
Communication bus, the processor, the memory and the communication interface complete mutual communication by the communication bus；
For the memory for storing an at least executable instruction, it is above-mentioned that the executable instruction performs the processor
Based on the corresponding operation of hypotactic method of data synchronization.
In accordance with a further aspect of the present invention, provide a kind of computer storage media, be stored in the storage medium to
A few executable instruction, the executable instruction make processor perform as above-mentioned based on hypotactic method of data synchronization correspondence
Operation.
According to provided by the invention based on hypotactic method of data synchronization and device, it is located at main section when host node is read
When first daily record of point side occurs to read abnormal, according to abnormal conditions are read, determine that daily record elapses length；Host node is according to daily record
Passage length skips synch command to from node transmission daily record；Synch command is skipped from node according to daily record, positioned at from node
After the current sync offset of second daily record of side, filling elapses the matched blank log content of length with daily record.Using we
Case can ensure that host node has identical daily record synchronisation offset with from node in main and subordinate node data synchronization process, so as to
The loss of the data in synchronizing process is avoided, especially in the case of frequent main and subordinate node switches, data can be greatly reduced and lose
The situation of mistake, so as to improve the consistency of main and subordinate node data.
Above description is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, below the special specific embodiment for lifting the present invention.
Description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this field
Technical staff will become clear.Attached drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings：
Fig. 1 is shown to be shown according to the flow provided by one embodiment of the present invention based on hypotactic method of data synchronization
It is intended to；
Fig. 2 shows the flows based on hypotactic method of data synchronization provided according to a further embodiment of the invention
Schematic diagram；
Fig. 3 is shown according to the structural frames provided by one embodiment of the present invention based on hypotactic data synchronization unit
Figure；
Fig. 4 shows the structure diagram according to computing device provided by one embodiment of the present invention.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Completely it is communicated to those skilled in the art.
Fig. 1 is shown to be shown according to the flow provided by one embodiment of the present invention based on hypotactic method of data synchronization
It is intended to.As shown in Figure 1, this method includes：
Step S110, it is different according to reading when the first daily record that host node is read positioned at host node side occurs to read abnormal
Reason condition determines that daily record elapses length.
Wherein, record has the first daily record in host node, and the behaviour performed for data in host node is had recorded in the first daily record
The information such as variation of data in work and/or host node.Wherein, the present embodiment does not limit the type of the first daily record, for example,
First daily record can be binlog daily records (binary log) or redo log daily records (redoing log) etc..
Specifically, since situations such as artificial damage or abnormal power-down can lead to occur daily record data damage in daily record, from
And host node can be caused to occur to read exception when reading positioned at the first daily record of host node side.It is located at main section when host node is read
When first daily record of point side occurs to read abnormal, it can determine that daily record elapses length according to abnormal conditions are read.For example, when reading
When data content is abnormal, it may be determined that the corresponding data length of the data content determines that daily record elapses according to the data length
Length.
Step S120, host node elapse length according to daily record and skip synch command to from node transmission daily record.
Host node elapses length according to the daily record determined in step S110 and skips synch command to from node transmission daily record.
In specific implementation process, for main and subordinate node when daily record synchronizes, host node usually sends daily record synchronization with sync orders to from node
Order, and in this step, when the first daily record that host node is read positioned at host node side occurs to read abnormal, different from reading
The sync orders sent under normal conditions, but synch command is skipped, also, synchronous life is skipped in daily record to daily record is sent from node
Include daily record passage length information in order.
Step S130 skips synch command from node according to daily record, positioned at the current same of the second daily record from node side
After walking offset, filling elapses the matched blank log content of length with daily record.
Specifically, record has the second daily record from node, is skipped together when from node in the daily record for receiving host node transmission
After step command, after the current sync offset of the second daily record, filling elapses the matched blank log content of length with daily record.
Such as, however, it is determined that daily record passage length is 8 bytes, and the current sync offset of the second daily record is 100 bytes, then in the second daily record
Fill in the blanks log content in 101st byte to the 108th byte, also, the second daily record current offset refreshes as 108 bytes.
Optionally, when being multiple from node, synch command each is skipped from node according to the daily record received, respective
The second daily record current sync offset after, filling with daily record elapse the matched blank log content of length.
According to provided by the invention based on hypotactic method of data synchronization, when host node is read positioned at host node side
When first daily record occurs to read abnormal, according to abnormal conditions are read, determine that daily record elapses length；Host node is elapsed according to daily record to be grown
It spends to from node transmission daily record and skips synch command；Synch command is skipped from node according to daily record, positioned at from the of node side
After the current sync offset of two daily records, filling elapses the matched blank log content of length with daily record.It, can be using this programme
In main and subordinate node daily record synchronizing process, ensure that host node has identical daily record synchronisation offset with from node, so as to avoid
Especially in the case of frequent main and subordinate node switches, the feelings of loss of data can be greatly reduced in the loss of data in synchronizing process
Shape, so as to improve the consistency of main and subordinate node data.
Fig. 2 shows the flows based on hypotactic method of data synchronization provided according to a further embodiment of the invention
Schematic diagram.As shown in Fig. 2, this method includes：
Step S210, it is different according to reading when the first daily record that host node is read positioned at host node side occurs to read abnormal
Reason condition determines that daily record elapses length.
Wherein, record has the first daily record in host node, and the behaviour performed for data in host node is had recorded in the first daily record
The information such as variation of data in work and/or host node.For example, the first daily record can be binlog daily records or redo log days
Will etc..
Specifically, since situations such as artificial damage or abnormal power-down can lead to occur block corruptions in daily record, so as to meeting
Host node is caused to occur to read exception when reading positioned at the first daily record of host node side.It is located at host node side when host node is read
The first daily record when occurring to read abnormal, according to abnormal conditions are read, determine that daily record elapses length.
Wherein it is determined that the method for daily record passage length includes but not limited to one kind in implementation below or at least two
Combination.
Embodiment one, record has the data length and data content of pieces of data in the first daily record.When reading data
When content occurs abnormal, according to the abnormal corresponding data length of data content is read, determine that daily record elapses length.For example,
The storage mode of data is in first daily record：Store the data length of the data in the initial position of every data, close to
The data content of the data is stored after data length.If when reading data5 data, the number of the data can be normally read
It is 80 bytes according to length, and when reading data content generation reading exception, it is determined that the data content is corresponding
Data length is 80 bytes, so as to further determine that daily record passage length is 80 bytes.
Embodiment two for ease of being managed to daily record data, is divided into multiple data blocks in the first daily record, per number
According at least one data are included in block, the length of each data block can be the same or different.Then recorded in the first daily record
There are the data length and data content of each data block.So as to according to the data length for reading abnormal data block, determine
Daily record elapses length.For example, when the generation for reading certain data in Block2 reads abnormal, it can be according to the number of the Block2
According to length, determine that daily record elapses length.Optionally, the data length of data block recorded in the first daily record is preset length,
Then it can determine that daily record elapses length according to the quantity and preset length for reading abnormal data block.For example, preset length is 40
Byte, when exception is read in the generation for reading certain data in Block2, host node is skipped Block2 and is read in Block3
Data, when reading the data in Block3 and also reading abnormal, host node skips Block3 and reads data in Block4,
When reading the data in Block4 and not reading abnormal, it is determined that daily record passage length is 2 times of preset length, i.e. 80 words
Section.
Optionally, in specific implementation process, record has the data length and number of each data block in the first daily record
According to content, also, also record has the data length and data content of pieces of data in the first daily record.When host node is reading the
The data length of certain data in one daily record does not occur to read exception, and is read in the data content for reading the data
When abnormal, it is determined that the corresponding data length of the data content elapses length for daily record.When host node is in the first daily record is read
The data length of certain data when occurring to read abnormal, it may be determined that the corresponding data block of the data, according to the data block
Length determines that daily record elapses length.
Step S220, host node elapse length according to daily record and skip synch command to from node transmission daily record.
Host node elapses length according to the daily record determined in step S210 and skips synch command to from node transmission daily record.
In specific implementation process, for main and subordinate node in daily record synchronizing process, host node usually sends daily record with sync orders to from node
Include specific operation and the key values of operation data and value values in synch command, wherein sync orders.And this step
In rapid, then length elapsed according to daily record and skip synch command to from node transmission daily record.Optionally, daily record skips synch command and is
SKIP orders include daily record passage length information in SKIP orders.
Step S230 skips synch command from node according to daily record, positioned at the current same of the second daily record from node side
After walking offset, filling elapses the matched blank log content of length with daily record.
Specifically, record has the second daily record from node, is skipped together when from node in the daily record for receiving host node transmission
After step command, after the current sync offset of the second daily record, filling elapses the matched blank log content of length with daily record.
Such as, however, it is determined that daily record passage length is 8 bytes, and the current sync offset of the second daily record is 100 bytes, then in the second daily record
Fill in the blanks log content in 101st byte to the 108th byte, also, the second daily record current offset refreshes as 108 bytes.
Optionally, when being multiple from node, synch command each is skipped from node according to the daily record received, respective
The second daily record current sync offset after, filling with daily record elapse the matched blank log content of length.
Optionally, the matched blank log content of length can be elapsed with daily record every preset time or in each filling
Later, lead to the consistency of the first daily record of checksum algorithm checks and the second daily record.When the first daily record and the second daily record are inconsistent
When, the second daily record is verified, to ensure the consistency of the first daily record and the second daily record.
Further, in specific implementation process, from described in node successively the second daily record of the basis from node side
Operation to performing corresponding operation from the data in node, synchronizes so as to fulfill the data between main and subordinate node.
Step S240, host node update the current sync offset of the first daily record, according to working as updated first daily record
Preamble offset carries out data synchronization.
Specifically, when host node is after the first daily record generation reading exception is read, host node updates working as the first daily record
Preamble offset, and determine daily record passage length, skip synch command to daily record is sent from node.It is skipped together sending daily record
After step command, sync orders are sent to from node according to the current sync offset of updated first daily record, for principal and subordinate
Node carries out subsequent data synchronization.
According to provided by the invention based on hypotactic method of data synchronization, when host node is read positioned at host node side
When first daily record occurs to read abnormal, according to the abnormal corresponding data length of data content is read, daily record passage length is determined
Degree or the data length according to the data block for reading exception determine that daily record elapses length；Host node elapses length according to daily record
Synch command is skipped to daily record is sent from node；Synch command is skipped from node according to daily record, positioned at from the second of node side
After the current sync offset of daily record, filling elapses the matched blank log content of length with daily record；Host node updates first
The current sync offset of will according to the current sync offset of updated first daily record, carries out daily record synchronization.Using we
Case can ensure that host node has identical daily record synchronisation offset with from node in main and subordinate node data synchronization process, so as to
The loss of the data in synchronizing process is avoided, especially in the case of frequent main and subordinate node switches, data can be greatly reduced and lose
The situation of mistake, so as to improve the consistency of main and subordinate node data.
Fig. 3 is shown according to the structural frames provided by one embodiment of the present invention based on hypotactic data synchronization unit
Figure.As shown in figure 3, the device includes：Determining module 31, order sending module 32 and filling module 33.
Determining module 31, when the first daily record suitable for being read when host node positioned at host node side occurs to read abnormal, according to
Abnormal conditions are read, determine that daily record elapses length.
Order sending module 32 elapses length according to daily record suitable for host node and skips synchronous life to from node transmission daily record
It enables.
Fill module 33, suitable for skipping synch command according to the daily record from node, positioned at described from the of node side
After the current sync offset of two daily records, filling elapses the matched blank log content of length with daily record.
Optionally, the data length and data content for having pieces of data are recorded in the first daily record；Then determining module 31 into
One step is suitable for：According to the abnormal corresponding data length of data content is read, the daily record passage length is determined.
Optionally, the data length and data content for having each data block are recorded in the first daily record；Then determining module 31
It is further adapted for：According to the data length for reading abnormal data block, the daily record passage length is determined.
Further alternative, the data length of the data block recorded in the first daily record is preset length；Then determining module
31 are further adapted for：According to the quantity and the preset length for reading abnormal data block, the daily record passage length is determined.
Optionally, described device further includes update module (not shown), is elapsed suitable for host node according to the daily record
Length updates the current sync offset of the first daily record, according to the current sync offset of updated first daily record, into line number
According to synchronization.
Optionally, described device further includes：Data simultaneous module (not shown), suitable for from node according to described second
Daily record carries out data synchronization.
Wherein, the specific embodiment based on hypotactic data synchronization unit in the present embodiment, can refer to Fig. 1 and
Shown in Fig. 2 to be based on illustrating accordingly in hypotactic method of data synchronization embodiment, this will not be repeated here.
According to provided by the invention based on hypotactic data synchronization unit, by determining module when host node reads position
When the first daily record of host node side occurs to read abnormal, according to abnormal conditions are read, determine that daily record elapses length；And utilize life
It enables sending module that host node is made to elapse length according to daily record and skips synch command to from node transmission daily record；Finally by fill mould
Block makes to skip synch command from node according to daily record, after the current sync offset positioned at the second daily record from node side, fills out
It fills and elapses the matched blank log content of length with daily record.Using this programme, can ensure in main and subordinate node data synchronization process
Host node has identical daily record synchronisation offset with from node, so as to avoid the loss of the data in synchronizing process, especially exists
In the case of frequent main and subordinate node switching, the situation of loss of data can be greatly reduced, so as to improve the one of main and subordinate node data
Cause property.
A kind of nonvolatile computer storage media is provided according to one embodiment of the invention, the computer storage is situated between
Matter is stored with an at least executable instruction, the computer executable instructions can perform in above-mentioned any means embodiment based on master
From the method for data synchronization of structure.
Fig. 4 is shown according to a kind of structure diagram of computing device provided by one embodiment of the present invention, present invention tool
Body embodiment does not limit the specific implementation of computing device.
As shown in figure 4, the computing device can include：Processor (processor) 402, communication interface
(Communications Interface) 404, memory (memory) 406 and communication bus 408.
Wherein：
Processor 402, communication interface 404 and memory 406 complete mutual communication by communication bus 408.
Communication interface 404, for communicating with the network element of miscellaneous equipment such as client or other servers etc..
Processor 402 for performing program 410, can specifically perform above-mentioned based on hypotactic method of data synchronization
Correlation step in embodiment.
Specifically, program 410 can include program code, which includes computer-managed instruction.
Processor 402 may be central processor CPU or specific integrated circuit ASIC (Application
Specific Integrated Circuit) or be arranged to implement the embodiment of the present invention one or more integrate electricity
Road.The one or more processors that computing device includes can be same type of processor, such as one or more CPU；Also may be used
To be different types of processor, such as one or more CPU and one or more ASIC.
Memory 406, for storing program 410.Memory 406 may include high-speed RAM memory, it is also possible to further include
Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.
Program 410 specifically can be used for so that processor 402 performs following operation：
When the first daily record that host node is read positioned at host node side occurs to read abnormal, according to abnormal conditions are read, really
Settled date will elapses length；
Host node elapses length according to the daily record and skips synch command to from node transmission daily record；
Synch command is skipped from node according to the daily record, in the current sync positioned at second daily record from node side
After offset, filling elapses the matched blank log content of length with the daily record.
In a kind of optional embodiment, record has the data length and data of pieces of data in first daily record
Content；Then program 410 specifically can be used for so that processor 402 performs following operation：
According to the abnormal corresponding data length of data content is read, the daily record passage length is determined.
In a kind of optional embodiment, record has the data length and number of each data block in first daily record
According to content；
Then program 410 specifically can be used for so that processor 402 performs following operation：
According to the data length for reading abnormal data block, the daily record passage length is determined.
In a kind of optional embodiment, the data length of data block recorded in first daily record is default length
Degree；
Then program 410 specifically can be used for so that processor 402 performs following operation：
According to the quantity and the preset length for reading abnormal data block, the daily record passage length is determined.
In a kind of optional embodiment, program 410 specifically can be used for so that processor 402 performs following operation：
Host node updates the current sync offset of the first daily record, is deviated according to the current sync of updated first daily record
Amount carries out data synchronization.
In a kind of optional embodiment, program 410 specifically can be used for so that processor 402 performs following operation：
Data synchronization is carried out from node according to second daily record.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with teaching based on this.As described above, required by constructing this kind of system
Structure be obvious.In addition, the present invention is not also directed to any certain programmed language.It should be understood that it can utilize various
Programming language realizes the content of invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the specification provided in this place, numerous specific details are set forth.It is to be appreciated, however, that the implementation of the present invention
Example can be put into practice without these specific details.In some instances, well known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of each inventive aspect,
Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. required guarantor
Shield the present invention claims the more features of feature than being expressly recited in each claim.More precisely, as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim is in itself
Separate embodiments all as the present invention.
Those skilled in the art, which are appreciated that, to carry out adaptively the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.It can be the module or list in embodiment
Member or component be combined into a module or unit or component and can be divided into addition multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it may be used any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification is (including adjoint power
Profit requirement, abstract and attached drawing) disclosed in each feature can be by providing the alternative features of identical, equivalent or similar purpose come generation
It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included certain features rather than other feature, but the combination of the feature of different embodiments means in of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
One of meaning mode can use in any combination.
The all parts embodiment of the present invention can be with hardware realization or to be run on one or more processor
Software module realize or realized with combination thereof.It will be understood by those of skill in the art that it can use in practice
Microprocessor or digital signal processor (DSP) are according to embodiments of the present invention based on the synchronization of hypotactic data to realize
The some or all functions of some or all components in device.The present invention is also implemented as being retouched here for performing
The some or all equipment or program of device (for example, computer program and computer program product) for the method stated.
Such program for realizing the present invention can may be stored on the computer-readable medium or can have one or more signal
Form.Such signal can be downloaded from internet website obtain either providing on carrier signal or with it is any its
He provides form.
It should be noted that the present invention will be described rather than limits the invention, and ability for above-described embodiment
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference mark between bracket should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real
It is existing.If in the unit claim for listing equipment for drying, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any sequence.These words can be explained and run after fame
Claim.
The invention discloses：A1. it is a kind of to be based on hypotactic method of data synchronization, including：
When the first daily record that host node is read positioned at host node side occurs to read abnormal, according to abnormal conditions are read, really
Settled date will elapses length；
Host node elapses length according to the daily record and skips synch command to from node transmission daily record；
Synch command is skipped from node according to the daily record, in the current sync positioned at second daily record from node side
After offset, filling elapses the matched blank log content of length with the daily record.
A2. the method according to A1, wherein, record has the data length and number of pieces of data in first daily record
According to content；
It is described according to abnormal conditions, determine that daily record passage length further comprises：
According to the abnormal corresponding data length of data content is read, the daily record passage length is determined.
A3. the method according to A1, wherein, in first daily record record have each data block data length and
Data content；
It is described according to abnormal conditions, determine that daily record passage length further comprises：
According to the data length for reading abnormal data block, the daily record passage length is determined.
A4. the method according to A3, wherein, the data length of data block recorded in first daily record is default
Length；
The data length according to the data block for reading exception determines that the daily record passage length further comprises：Root
According to the quantity and the preset length for reading abnormal data block, the daily record passage length is determined.
A5. according to the method described in any one of A1-A4, wherein, the method further includes：
Host node updates the current sync offset of the first daily record, is deviated according to the current sync of updated first daily record
Amount carries out data synchronization.
A6. according to the method described in any one of A1-A5, wherein, the method further includes：
Data synchronization is carried out from node according to second daily record.
The invention also discloses：B7. it is a kind of to be based on hypotactic data synchronization unit, including：
Determining module, when the first daily record suitable for being read when host node positioned at host node side occurs to read abnormal, according to reading
Abnormal conditions are taken, determine that daily record elapses length；
Order sending module elapses length according to the daily record suitable for host node and skips synchronous life to from node transmission daily record
It enables；
Fill module, suitable for skipping synch command according to the daily record from node, positioned at described from the second of node side
After the current sync offset of daily record, filling elapses the matched blank log content of length with the daily record.
B8. the device according to B7, wherein, record has the data length and number of pieces of data in first daily record
According to content；
The determining module is further adapted for：According to the abnormal corresponding data length of data content is read, institute is determined
State daily record passage length.
B9. the device according to B7, wherein, in first daily record record have each data block data length and
Data content；
The determining module is further adapted for：According to the data length for reading abnormal data block, determine that the daily record pushes away
Move length.
B10. the device according to B9, wherein, the data length of data block recorded in first daily record is pre-
If length；
The determining module is further adapted for：According to the quantity and the preset length for reading abnormal data block, really
The fixed daily record passage length.
B11. according to the device described in any one of B7-B10, wherein, described device further includes：
Update module elapses the current sync offset of length the first daily record of update suitable for host node according to the daily record,
According to the current sync offset of updated first daily record, data synchronization is carried out.
B12. according to the device described in any one of B7-B11, wherein, described device further includes：
Data simultaneous module, suitable for carrying out data synchronization according to second daily record from node.
The invention also discloses：C13. a kind of computing device, including：Processor, memory, communication interface and communication are total
Line, the processor, the memory and the communication interface complete mutual communication by the communication bus；
For the memory for storing an at least executable instruction, the executable instruction makes the processor perform such as
Described in any one of A1-A6 based on the corresponding operation of hypotactic method of data synchronization.
The invention also discloses：D14. a kind of computer storage media, being stored at least one in the storage medium can hold
Row instruction, the executable instruction make processor perform being synchronized based on hypotactic data as described in any one of A1-A6
The corresponding operation of method.
Claims (10)
1. one kind is based on hypotactic method of data synchronization, including：
When the first daily record that host node is read positioned at host node side occurs to read abnormal, according to abnormal conditions are read, day is determined
Will elapses length；
Host node elapses length according to the daily record and skips synch command to from node transmission daily record；
Synch command is skipped from node according to the daily record, in the current sync offset positioned at second daily record from node side
After amount, filling elapses the matched blank log content of length with the daily record.
2. according to the method described in claim 1, wherein, in first daily record record have pieces of data data length and
Data content；
It is described according to abnormal conditions, determine that daily record passage length further comprises：
According to the abnormal corresponding data length of data content is read, the daily record passage length is determined.
3. according to the method described in claim 1, wherein, in first daily record record have the data length of each data block with
And data content；
It is described according to abnormal conditions, determine that daily record passage length further comprises：
According to the data length for reading abnormal data block, the daily record passage length is determined.
4. according to the method described in claim 3, wherein, the data length of data block recorded in first daily record is pre-
If length；
The data length according to the data block for reading exception determines that the daily record passage length further comprises：According to reading
The quantity of abnormal data block and the preset length are taken, determines the daily record passage length.
5. according to the method described in any one of claim 1-4, wherein, the method further includes：
Host node updates the current sync offset of the first daily record, according to the current sync offset of updated first daily record,
Carry out data synchronization.
6. according to the method described in any one of claim 1-5, wherein, the method further includes：
Data synchronization is carried out from node according to second daily record.
7. one kind is based on hypotactic data synchronization unit, including：
Determining module, it is different according to reading when the first daily record suitable for being read when host node positioned at host node side occurs to read abnormal
Reason condition determines that daily record elapses length；
Order sending module elapses length according to the daily record suitable for host node and skips synch command to from node transmission daily record；
Module is filled, suitable for skipping synch command according to the daily record from node, positioned at second daily record from node side
Current sync offset after, filling with the daily record elapse the matched blank log content of length.
8. device according to claim 7, wherein, in first daily record record have pieces of data data length and
Data content；
The determining module is further adapted for：According to the abnormal corresponding data length of data content is read, the day is determined
Will elapses length.
9. a kind of computing device, including：Processor, memory, communication interface and communication bus, the processor, the storage
Device and the communication interface complete mutual communication by the communication bus；
For the memory for storing an at least executable instruction, the executable instruction makes the processor perform right such as will
Ask described in any one of 1-6 based on the corresponding operation of hypotactic method of data synchronization.
10. a kind of computer storage media, an at least executable instruction, the executable instruction are stored in the storage medium
Make processor perform as described in any one of claim 1-6 based on the corresponding operation of hypotactic method of data synchronization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477361.5A CN108205588B (en) | 2017-12-29 | 2017-12-29 | Data synchronization method and device based on master-slave structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477361.5A CN108205588B (en) | 2017-12-29 | 2017-12-29 | Data synchronization method and device based on master-slave structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108205588A true CN108205588A (en) | 2018-06-26 |
| CN108205588B CN108205588B (en) | 2021-04-09 |
Family
ID=62605122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711477361.5A Active CN108205588B (en) | 2017-12-29 | 2017-12-29 | Data synchronization method and device based on master-slave structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108205588B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109388677A (en) * | 2018-08-23 | 2019-02-26 | 顺丰科技有限公司 | Method of data synchronization, device, equipment and its storage medium between cluster |
| CN110688254A (en) * | 2019-09-06 | 2020-01-14 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113515574A (en) * | 2021-05-17 | 2021-10-19 | 聚好看科技股份有限公司 | Data synchronization method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500229A (en) * | 2013-10-24 | 2014-01-08 | 北京奇虎科技有限公司 | Database synchronization method and database system |
| US20140181016A1 (en) * | 2012-12-21 | 2014-06-26 | Zetta, Inc. | Asynchronous replication correctness validation |
| CN104504062A (en) * | 2014-12-22 | 2015-04-08 | 浙江宇视科技有限公司 | Data synchronization method and device of primary and standby databases |
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
-
2017
- 2017-12-29 CN CN201711477361.5A patent/CN108205588B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181016A1 (en) * | 2012-12-21 | 2014-06-26 | Zetta, Inc. | Asynchronous replication correctness validation |
| CN103500229A (en) * | 2013-10-24 | 2014-01-08 | 北京奇虎科技有限公司 | Database synchronization method and database system |
| CN104504062A (en) * | 2014-12-22 | 2015-04-08 | 浙江宇视科技有限公司 | Data synchronization method and device of primary and standby databases |
| CN105912628A (en) * | 2016-04-07 | 2016-08-31 | 北京奇虎科技有限公司 | Synchronization method and device for master database and slave database |
Non-Patent Citations (2)
| Title |
|---|
| 何小川: ""MySQL数据库主从复制的实现"", 《广东通信技术》 * |
| 谈书才 等: "" 基于Replication和LVS的MySQL分布式数据库研究"", 《中国科技信息》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109388677A (en) * | 2018-08-23 | 2019-02-26 | 顺丰科技有限公司 | Method of data synchronization, device, equipment and its storage medium between cluster |
| CN110688254A (en) * | 2019-09-06 | 2020-01-14 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN110688254B (en) * | 2019-09-06 | 2022-06-03 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113515574A (en) * | 2021-05-17 | 2021-10-19 | 聚好看科技股份有限公司 | Data synchronization method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108205588B (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TW522320B (en) | Apparatus and method for recovering a failed database data set | |
| CN108234641A (en) | Data read-write method and device based on distributed consensus protocol realization | |
| CN105550229B (en) | The method and apparatus of distributed memory system data reparation | |
| CN112000355B (en) | Method and device for firmware upgrading and household appliance | |
| CN111124277B (en) | Deep learning data set caching method, system, terminal and storage medium | |
| CN111124755B (en) | Fault recovery method and device for cluster nodes, electronic equipment and storage medium | |
| CN108205588A (en) | Based on hypotactic method of data synchronization and device | |
| CN105262831A (en) | Method, device, and synchronizing system for synchronizing data among memory systems | |
| CN104516966A (en) | High-availability solving method and device of database cluster | |
| CN110389858B (en) | Method and device for recovering faults of storage device | |
| CN108055343A (en) | For the method for data synchronization and device of computer room | |
| CN104537046A (en) | Method and device for complementing data | |
| CN106709066B (en) | Data synchronization method and device | |
| CN106789741A (en) | The consuming method and device of message queue | |
| CN104765652A (en) | Data backup and data recovery method and device | |
| US9037539B2 (en) | Data synchronization | |
| CN108228812A (en) | Adaptive host node switching method and device | |
| CN108228789A (en) | The synchronous abnormality restoration methods and device triggered from node | |
| CN110083372B (en) | Block chain data version upgrading method | |
| CN105453095A (en) | Function setting method | |
| CN105550230A (en) | Method and device for detecting failure of node of distributed storage system | |
| CN108200157A (en) | The daily record synchronous method and device that host node triggering retracts | |
| CN112131177A (en) | Data migration method and device, storage medium and electronic equipment | |
| CN110825758B (en) | Transaction processing method and device | |
| CN107864209A (en) | The method, apparatus and server of data write-in |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |