CN106101229B - Data synchronization network and method - Google Patents
Data synchronization network and method Download PDFInfo
- Publication number
- CN106101229B CN106101229B CN201610424362.2A CN201610424362A CN106101229B CN 106101229 B CN106101229 B CN 106101229B CN 201610424362 A CN201610424362 A CN 201610424362A CN 106101229 B CN106101229 B CN 106101229B
- Authority
- CN
- China
- Prior art keywords
- synchronization
- data
- node
- synchronous
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000001360 synchronised effect Effects 0.000 claims abstract description 217
- 230000000977 initiatory effect Effects 0.000 claims abstract description 20
- 230000005540 biological transmission Effects 0.000 claims description 4
- 239000003999 initiator Substances 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims 1
- 101000651709 Homo sapiens SCO-spondin Proteins 0.000 description 9
- 102100027296 SCO-spondin Human genes 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000007812 deficiency Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Synchronisation In Digital Transmission Systems (AREA)
Abstract
the invention provides a data synchronization network and a method, wherein a plurality of nodes of the network respectively comprise a data synchronization unit, and the data synchronization unit comprises the following steps: the synchronization initiating subunit is configured to initiate data synchronization at a fixed time, and specifically includes: selecting a close-range synchronization object node according to a first preset condition, generating a synchronization request list according to each synchronized identification bit corresponding to the synchronization object node, sending the synchronization request list, receiving returned synchronization data, and setting the synchronized identification bit corresponding to the synchronization object node of each local group of data; and the synchronous receiving subunit is used for receiving the synchronous request list, generating synchronous data according to the received synchronous request list and the owned identification bits corresponding to the synchronous request nodes, returning the synchronous data, and setting the corresponding owned identification bits. The invention guarantees timeliness by setting the preset conditions, generates the synchronous request list by the corresponding synchronous identification bits, and generates the synchronous data by the synchronous request list and the corresponding owned identification bits, thereby realizing high-efficiency data synchronization.
Description
Technical Field
The present application relates to the field of data synchronization technologies, and in particular, to a data synchronization network and method.
Background
Whether a network of a traditional client-server (CS) architecture or a peer-to-peer network, which is currently more popular, stores data that is of significant need on specific or non-specific nodes of the network. The CS fabric network stores data in a single or multiple server nodes, while the peer-to-peer network may store data in any node. For both the CS-structured network and the peer-to-peer network of a plurality of server nodes, there is a need to keep the data stored in each node consistent, i.e., to synchronize the data.
With existing network synchronization mechanisms, such as a stepped file system (DFS), Cloud Storage (Cloud Storage), etc., Rsync and RDC are two most common data synchronization algorithms. In practical application, both algorithms have certain deficiencies in the timeliness and efficiency performance of data synchronization.
disclosure of Invention
in view of the above-mentioned drawbacks and deficiencies of the prior art, it is desirable to provide a data synchronization network and method with enhanced timeliness and efficiency.
in a first aspect, the present invention provides a data synchronization network, where the network includes a plurality of nodes, each node includes a data synchronization unit, and the data synchronization unit includes:
a synchronization initiating subunit, configured to initiate data synchronization to a synchronization target node at regular time, where initiating data synchronization to the synchronization target node includes: selecting a close-range synchronization object node according to a first preset condition, generating a synchronization request list according to local groups of data corresponding to synchronization identification bits of the selected synchronization object node, sending the synchronization request list to the synchronization object node, receiving synchronization data returned by the synchronization object node, and setting the synchronization identification bits of the local groups of data corresponding to the synchronization object node;
the synchronous receiving subunit is used for receiving a synchronous request list sent by a synchronous request node, generating synchronous data according to the received synchronous request list and local groups of data corresponding to owned identification bits of the synchronous request node, setting the owned identification bits of the local groups of data corresponding to the synchronous request node, and sending the synchronous data to the synchronous request node;
each group of data is respectively provided with a plurality of synchronized identification bits corresponding to the synchronized object nodes and a plurality of owned identification bits corresponding to the synchronized request nodes, the synchronized identification bits are used for identifying the synchronized corresponding synchronized object nodes of the group of data, and the owned identification bits are used for identifying the synchronized request nodes corresponding to the group of data.
In a second aspect, the present invention provides a data synchronization method, including:
Selecting a close-range synchronization object node according to a first preset condition;
Generating a synchronization request list according to each set of local data corresponding to each synchronized identification bit of the selected synchronization object node;
sending the synchronization request list to the synchronization object node;
Receiving the synchronous data returned by the synchronous object node;
setting synchronized identification bits corresponding to the synchronization object nodes of local groups of data; the steps are circulated regularly;
Each group of data is provided with a plurality of synchronized identification bits respectively corresponding to the synchronized object nodes, and the synchronized identification bits are used for identifying the synchronized corresponding synchronized object nodes of the group of data.
in a third aspect, the present invention provides another data synchronization method, including: receiving a synchronous request list sent by a synchronous request node;
generating synchronous data according to the received synchronous request list and local groups of data corresponding to owned identification bits of the synchronous request nodes, and setting the owned identification bits of the local groups of data corresponding to the synchronous request nodes;
Sending the synchronization data to the synchronization request node;
Each group of data is provided with a plurality of owned identification bits respectively corresponding to the synchronous request nodes, and the owned identification bits are used for identifying that the corresponding synchronous request nodes own the group of data.
The data synchronization network and the method provided by the embodiments of the present invention set a synchronized identification bit corresponding to each synchronization object node and used for identifying the synchronization object node corresponding to the group of data that has been synchronized for each group of data stored in each node in the network, and an owned identification bit corresponding to each synchronization request node and used for identifying the synchronization request node corresponding to the synchronization request node that has owned the group of data, and by setting a first preset condition, the timeliness is ensured while selecting a synchronization object node in a close range, a synchronization request list is generated by each synchronized identification bit corresponding to the selected synchronization object node, and by generating synchronization data by the received synchronization request list and each owned identification bit corresponding to the synchronization request node, high-efficiency data synchronization is achieved;
the data synchronization network and the data synchronization method provided by some embodiments of the present invention further improve the efficiency of data synchronization by selecting a synchronization target node from the plurality of data source nodes according to a second preset condition;
The data synchronization network and the method provided by some embodiments of the invention further ensure stronger timeliness by setting more strict preset conditions.
Drawings
other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
Fig. 1 is a schematic structural diagram of a data synchronization network according to an embodiment of the present invention.
fig. 2 is a flowchart of a data synchronization method according to an embodiment of the present invention.
fig. 3 is a flowchart of another data synchronization method according to an embodiment of the invention.
fig. 4 is a flowchart of step S12 in the data synchronization method shown in fig. 2.
Fig. 5 is a flowchart of a preferred embodiment of the data synchronization method shown in fig. 2.
Fig. 6 is a flowchart of a preferred embodiment of the data synchronization method shown in fig. 2.
Fig. 7 is a flowchart of step S25 in the data synchronization method shown in fig. 3.
Fig. 8 is a schematic structural diagram of a node in a data synchronization network according to a preferred embodiment of the present invention.
Detailed Description
the present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
it should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 is a schematic structural diagram of a data synchronization network according to an embodiment of the present invention.
As shown in fig. 1, in the present embodiment, the data synchronization network provided by the present invention includes several nodes 10. The node 10 comprises a data synchronization unit 20, the data synchronization unit 20 comprising a synchronization transmitter unit 21 and a synchronization receiver subunit 22.
The synchronization initiating subunit 21 is configured to initiate data synchronization to a synchronization target node at regular time. The initiating data synchronization to the synchronization object node comprises: the method comprises the steps of selecting a close-range synchronization object node according to a first preset condition, generating a synchronization request list according to local groups of data corresponding to synchronization identification bits of the selected synchronization object node, sending the synchronization request list to the synchronization object node, receiving synchronization data returned by the synchronization object node, and setting the synchronization identification bits of the local groups of data corresponding to the synchronization object node.
the synchronization receiving subunit 22 is configured to receive a synchronization request list sent by a synchronization request node, generate synchronization data according to the received synchronization request list and local groups of data corresponding to owned identification bits of the synchronization request node, set the owned identification bits of the local groups of data corresponding to the synchronization request node, and send the synchronization data to the synchronization request node.
each group of data is respectively provided with a plurality of synchronized identification bits corresponding to the synchronized object nodes and a plurality of owned identification bits corresponding to the synchronized request nodes, the synchronized identification bits are used for identifying the synchronized corresponding synchronized object nodes of the group of data, and the owned identification bits are used for identifying the synchronized request nodes corresponding to the group of data.
fig. 2 is a flowchart of a data synchronization method according to an embodiment of the present invention. The data synchronization method shown in fig. 2 can be applied to each synchronization initiating subunit 21 of the data synchronization network shown in fig. 1.
As shown in fig. 2, in this embodiment, a data synchronization method provided by the present invention includes:
S12: selecting a close-range synchronization object node according to a first preset condition;
s13: generating a synchronization request list according to each set of local data corresponding to each synchronized identification bit of the selected synchronization object node;
S14: sending the synchronization request list to the synchronization object node;
s15: receiving the synchronous data returned by the synchronous object node;
s16: setting synchronized identification bits corresponding to the synchronization object nodes of local groups of data; the timing jumps to step S12 to loop.
each group of data is provided with a plurality of synchronized identification bits respectively corresponding to the synchronized object nodes, and the synchronized identification bits are used for identifying the synchronized corresponding synchronized object nodes of the group of data.
Fig. 3 is a flowchart of another data synchronization method according to an embodiment of the invention. The data synchronization method shown in fig. 3 can be applied to each synchronization receiving subunit 22 of the data synchronization network shown in fig. 1.
As shown in fig. 3, in this embodiment, another data synchronization method provided by the present invention includes:
s23: receiving a synchronous request list sent by a synchronous request node;
s25: generating synchronous data according to the received synchronous request list and local groups of data corresponding to owned identification bits of the synchronous request nodes, and setting the owned identification bits of the local groups of data corresponding to the synchronous request nodes;
S27: and sending the synchronous data to the synchronous request node.
Each group of data is provided with a plurality of owned identification bits respectively corresponding to the synchronous request nodes, and the owned identification bits are used for identifying that the corresponding synchronous request nodes own the group of data.
Specifically, taking a node a locally storing five groups of data, i.e. a, b, c, d, and e, as an example:
the node A can be used as a synchronous request node for actively initiating data synchronization, and for five groups of data a, b, c, d and e, each group of data is respectively provided with a data synchronization node and a data synchronization nodethe synchronized flag SYNC _ STATE _ ACKNOWLEDGED _ TO _ PEER corresponding TO each synchronization object node, hereinafter abbreviated as SSATP, for example, the data a corresponds TO the synchronized flag SSATP of the node BaBdata b corresponds to the synchronized flag SSATP of node CbCEtc.;
The node a may also be a synchronization object node for receiving a data synchronization request, and for five groups of data, a, b, c, d, and e, each group of data is further provided with an owned flag SYNC _ STATE _ PEER _ OWN corresponding to each synchronization request node, which is referred to as SSPO hereinafter, for example, data d corresponds to an owned flag SSPO of the node RdRdata e corresponds to the owned flag SSPO of node TeTAnd the like.
Taking the node B with five groups of data a, B, c, f and g, the node a having synchronized the data a and B with the node B, the node a initiating data synchronization at regular time, and selecting the node B as a synchronization object node as an example:
In step S12, the synchronization initiating subunit 21 of the node a selects the node B in the close range as the synchronization target node according to the preset time limit condition.
in the present embodiment, the synchronous transmission unit 21 takes the xor result of the node ID values as the distance between two nodes, and in more embodiments, different distance calculation methods can be adopted according to actual requirements, so that the same technical effect can be achieved.
In step S13, the synchronization initiating subunit 21 of node a corresponds to the synchronized identification bits SSATP of the selected synchronization target node B according to the local sets of data a and BaB、SSATPbBA synchronization request list is generated. Specifically, the node A has synchronized data a and B to the node B, unsynchronized data c, d, e, data a and B are respectively provided with synchronized identification bits SSATP corresponding to the node BaB、SSATPbBthe data c, d, e are not provided with the synchronized flag bit corresponding to the node B, and the synchronization initiating subunit 21 stores the related information of the data c, d, e in the synchronization request list, respectively.
In this embodiment, the related information is a hash value of the group of data, and in further embodiments, the related information may be set as other corresponding information according to actual requirements, for example, check codes generated by different algorithms, so long as the related information can be used as a unique identifier of corresponding data and a smaller data size is maintained, the same technical effect can be achieved.
In step S14, the synchronization transmission driver unit 21 of the node a transmits the synchronization request list generated in step S13 to the synchronization reception subunit 22 of the node B.
In step S15, the synchronization transmitter unit 21 of node a receives the synchronization data returned by the synchronization receiver subunit 22 of node B, including data f and g.
in step S16, the synchronization initiator unit 21 of node a sets the synchronized flag bits of data c, d, e, f, g corresponding to node B to synchronized, and jumps to step S12 for the next data synchronization.
Specifically, in some embodiments, after receiving the synchronization data returned by the node B in step S15, the synchronization initiating subunit 21 sets the synchronized identification bit corresponding to the node B for all local data for which the synchronized identification bit corresponding to the node B is not set, that is, sets the synchronized identification bit corresponding to the node B for all data c, d, e, f, and g;
In other embodiments, after receiving the synchronization data returned by the node B at step S15, the synchronization initiating subunit 21 sets the synchronized identification bits SSATP corresponding to the node B to the data c, d, and e corresponding to the synchronization request list at step S16cB、SSATPdB、SSATPeBand sets synchronized identification bits SSATP corresponding to the node B to the received data f and gfB、SSATPgB。
Correspondingly, when the node B receives the synchronization request list sent by the node a in step S14:
in step S23, the synchronization receiving subunit 22 of the node B receives the synchronization request list transmitted by the synchronization initiating subunit 21 of the node a.
In step S25, the synchronization receiving subunit 22 of node B corresponds to the owned identification bits SSPO of the synchronization requesting node a according to the received synchronization request list and the local sets of data a and BaA、SSPObAAnd generating synchronous data, and setting local groups of data corresponding to the owned identification bits of the synchronous request nodes.
specifically, the node a can know that the node a owns the data a and b through the owned identification bits; the synchronous request list stores the related information of the data c, d and e, the node A can know that the node A owns the data c through comparison with the local groups of data a, b, c, f and g, the synchronous data comprising the data f and g is finally generated, and the data c, f and g are set to correspond to the owned identification bits SSPO of the node AcA、SSPOfA、SSPOgA：
in some embodiments, the synchronization receiving subunit 22 of the node B sets the owned identification bit corresponding to the node a for all data for which the owned identification bit corresponding to the synchronization requesting node a is not set locally after generating the synchronization data;
in other embodiments, when comparing the received synchronization request list with the local groups of data, the synchronization receiving subunit 22 of node B sets the owned identification bit SSPO corresponding to the synchronization request node a for the data c obtained by the comparisoncA(ii) a And after generating the synchronous data, setting the owned identification bits SSPO corresponding to the synchronous request node A to the data f and g corresponding to the synchronous datafA、SSPOgA。
in step S27, the synchronization receiving subunit 22 of the node B transmits the synchronization data generated in step S25 to the synchronization requesting node A.
In the above embodiment, a synchronized identification bit corresponding to each synchronization target node and used for identifying a synchronization target node corresponding to the group of data synchronized is set for each group of data stored in each node in the network, and an owned identification bit corresponding to each synchronization request node and used for identifying a synchronization request node corresponding to the synchronization target node and having the group of data synchronized is set for each group of data stored in each node in the network.
in a preferred embodiment, the selecting the close-range synchronization object node according to the first preset condition includes:
calculating the distance between each node and the current node according to a preset rule, and selecting a plurality of nodes closest to the current node;
sequentially comparing the selected nodes with a first preset condition according to the distance: if the node is in accordance with the synchronization object node, selecting the node as a synchronization object node; if none of the matches, stopping the synchronization.
Fig. 4 is a flowchart of step S12 in the data synchronization method shown in fig. 2. The method shown in fig. 4 can be applied to the synchronization driver unit 21 of the data synchronization network provided in the above preferred embodiment.
As shown in fig. 4, in a preferred embodiment, step S12 includes:
s123: calculating the distance between each node and the current node according to a preset rule, and selecting a plurality of nodes closest to the current node;
s125: sequentially comparing the selected nodes with a first preset condition according to the distance: if the node is in accordance with the synchronization object node, selecting the node as a synchronization object node; if none of the matches, stopping the synchronization.
specifically, in the present embodiment, in step S123, the synchronization driver unit 21 of the node a selects the three nodes B, C and D closest to the node a by calculating the exclusive or result of the own ID and the IDs of the other nodes in the network as the distance between the two nodes.
in step S125, the nodes B, C and D selected in step S125 are sequentially compared with a first preset condition, and if they are met, the node is selected as a synchronization target node, and if they are not met, the synchronization is stopped, and the next synchronization is waited.
Specifically, in this embodiment, the first preset condition includes:
1. the selected node is not the current node. For example, the selected node cannot be the current node a.
2. The interval between the current synchronization time and the last synchronization time is smaller than the preset synchronization time interval. For example, setting the synchronization interval to 10 minutes, node a must initiate the next synchronization within 10 minutes after initiating one synchronization.
3. The interval between the current synchronization time and the last synchronization failure time of the selected node is not less than the preset synchronization failure time interval, and the failure times are less than the preset failure time threshold. For example, the synchronization failure interval is set to 2 minutes, the failure time threshold is set to 5 times, if the synchronization initiated by the node B by the node a fails, the synchronization to the node B may be reinitiated every 2 minutes, and if the failure time reaches 5 times, the node a does not initiate synchronization to the node B any more.
4. and the interval between the current synchronization time and the last synchronization success time of the selected node is not less than the preset synchronization success time interval. For example, the synchronization success interval is set to 30 minutes, i.e. node a does not initiate synchronization to node B any more within 30 minutes after initiating synchronization success to node B, during which node a may initiate synchronization to the two other nodes C and D that are closest.
Corresponding to the first preset condition, in this embodiment, after step S14, if node a does not receive the synchronization data returned by node B and synchronization fails, the last synchronization time of node a and the last synchronization failure time of node B are updated, and the number of synchronization failures of node a to node B is increased by one; if the synchronization data returned by the node B is successfully received in step S15, the last synchronization time and the last synchronization success time for the node B are updated.
in this embodiment, the number of the nodes closest to each other is selected to be 3, the distance is calculated by taking an exclusive or value of the ID, and the set first preset condition is the 3 conditions. In more embodiments, different numbers of nodes closest to each other may be selected according to actual requirements, different algorithms are used to calculate the distances between the nodes, and first preset conditions of different logic strategies are set.
in a preferred embodiment, the generating a synchronization request list according to the local sets of data corresponding to the synchronized identification bits of the selected synchronization object node includes performing the following determinations for each set of data:
judging whether the synchronization object node is synchronized according to the synchronized identification bit of the group of data corresponding to the selected synchronization object node: and if the data is not synchronized, adding the hash value of the data into the synchronization request list.
fig. 5 is a flowchart of a preferred embodiment of the data synchronization method shown in fig. 2. The method shown in fig. 5 can be applied to the synchronization driver unit 21 of the data synchronization network provided in the above preferred embodiment.
as shown in fig. 5, in a preferred embodiment, step S13 includes the following determinations for each set of data:
S131: judging whether the synchronization object node is synchronized according to the synchronized identification bit of the group of data corresponding to the selected synchronization object node: and if the data is not synchronized, adding the hash value of the data into the synchronization request list.
In a preferred embodiment, the synchronization initiator unit 21 is further configured to select a synchronization object node from the plurality of data source nodes according to a second preset condition.
fig. 6 is a flowchart of a preferred embodiment of the data synchronization method shown in fig. 2. The method shown in fig. 6 can be applied to the synchronization driver unit 21 of the data synchronization network provided in the above preferred embodiment.
As shown in fig. 6, in a preferred embodiment, step S12 is preceded by:
S11: selecting a synchronization object node from the plurality of data source nodes according to a second preset condition:
If so, the process goes to step S13.
Specifically, for node a, each synchronization object node that has been successfully synchronized before is the data source node. The advantage of selecting the synchronization object node from the data source nodes is that: because the synchronization is successful before, on one hand, the success rate of the synchronization is relatively high, on the other hand, the relative data difference is small, and the synchronization efficiency is higher.
specifically, in this embodiment, the second preset condition includes:
1. The interval between the current synchronization time and the last synchronization time is smaller than the preset synchronization time interval. Similarly to the first preset condition, for example, if the synchronization interval is set to 10 minutes, the node a must initiate the next synchronization within 10 minutes after initiating the synchronization.
2. And the interval between the current synchronization time and the last synchronization failure time of the data source node is not less than a preset synchronization failure time interval, and the failure times are less than a preset failure time threshold. Similarly to the first preset condition, for example, the synchronization failure time interval is set to 2 minutes, and the failure time threshold is set to 5 times, if the synchronization initiated by the node a for the data source node T fails, the node a may reinitiate synchronization for the node T every 2 minutes, and if the failure time reaches 5 times, the node a does not initiate synchronization to the data source node T any more.
3. And the interval between the current synchronization time and the last synchronization success time of the data source node is not less than a preset synchronization success time interval. Similarly to the first preset condition, taking the synchronization success time interval set to 2 hours as an example, the node a does not initiate synchronization to the node S any more within 2 hours after initiating the synchronization success to the node S, during which the node a may initiate synchronization to other data source nodes meeting the second preset condition.
Corresponding to the second preset condition, in this embodiment, after step S14, if node a does not receive the synchronization data returned by node T and synchronization fails, the last synchronization time of node a and the last synchronization failure time of node T are updated, and the number of synchronization failures of node a to node T is increased by one; if the synchronization data returned by the node S is successfully received in step S15, the last synchronization time and the last synchronization success time for the node S are updated.
In further embodiments, the second preset condition of different logic strategies may be set according to actual requirements.
The embodiment further selects the synchronization object node from the plurality of data source nodes according to the second preset condition, so that the efficiency of data synchronization is improved, and stronger timeliness is guaranteed by setting a stricter preset condition.
in a preferred embodiment, the generating synchronization data according to the received synchronization request list and local sets of data corresponding to owned identification bits of the synchronization request node, and the setting of the local sets of data corresponding to owned identification bits of the synchronization request node includes:
Respectively judging whether the synchronous request node has all groups of data according to the synchronous request list: if the synchronous request node is owned, setting the owned identification bit of the group of data corresponding to the synchronous request node;
And adding each group of data which is not provided with the owned identification bit corresponding to the synchronous request node into synchronous data, and respectively setting the owned identification bits corresponding to the synchronous request node.
Fig. 7 is a flowchart of step S25 in the data synchronization method shown in fig. 3. The method shown in fig. 7 can be applied to the synchronous receiving subunit 22 of the data synchronization network provided in the above preferred embodiment.
as shown in fig. 7, in a preferred embodiment, step S25 includes:
s251: respectively judging whether the synchronous request node has all groups of data according to the synchronous request list: if the synchronous request node is owned, setting the owned identification bit of the group of data corresponding to the synchronous request node;
s253: and adding each group of data which is not provided with the owned identification bit corresponding to the synchronous request node into synchronous data, and respectively setting the owned identification bits corresponding to the synchronous request node.
Specifically, taking the example that the node B stores five groups of data a, B, c, f, and g, the node a has synchronized the data a and B with the node B, and the node B receives the synchronization request list including the information related to the data c, d, and e sent by the node a:
In step S251, the synchronous receiving subunit 22 of the node B sequentially determines whether the node a has owned local groups of data according to the received synchronous request list, compares the local groups of data with the synchronous request list to obtain the owned data c of the node a, and sets the owned flag SSPO of the node a corresponding to the data ccA。
in step S253, the node B' S synchronized reception sub-ticketThe element 22 adds the data f and g, which do not have the owned flag bit set corresponding to the node A, to the synchronization data, and sets the data f and g corresponding to the owned flag bit SSPO of the node A, respectivelyfAand SSPOgA。
In a preferred embodiment, the synchronization request list includes hash values of a plurality of sets of data, and the determining whether the synchronization request node has each set of data according to the synchronization request list includes:
Comparing the hash value of each group of data with the hash value in the synchronization request list: if the synchronous request list comprises the hash value of the data of the group, the synchronous request node already has the data of the group; otherwise, it is not owned.
Fig. 8 is a schematic structural diagram of a node in a data synchronization network according to a preferred embodiment of the present invention.
As shown in fig. 8, in a preferred embodiment, the data synchronization unit 20 further includes:
and the identification bit management unit 23 is configured to manage synchronized identification bits and owned identification bits of local groups of data.
The synchronous receiving subunit 21 sets the synchronized identification bit by calling the identification bit management unit 23; the synchronous reception subunit 22 sets the owned flag by calling the flag management unit 23.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
the units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, for example, the synchronization transmitter unit 21 and the synchronization receiver unit 22 may be software programs provided in a computer or a mobile intelligent device; or may be a hardware chip that performs data synchronization separately. Where the names of these units or modules do not in some cases constitute a limitation on the units or modules themselves, for example, the data synchronization unit 20 may also be described as a "synchronization storage unit for keeping data stored between nodes consistent".
As another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above-described embodiments; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the formula input methods described herein.
the above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by a person skilled in the art that the scope of the invention as referred to in the present application is not limited to the embodiments with a specific combination of the above-mentioned features, but also covers other embodiments with any combination of the above-mentioned features or their equivalents without departing from the inventive concept. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (14)
1. A data synchronization network, said network comprising a plurality of nodes, wherein said nodes comprise data synchronization units, said data synchronization units comprising:
A synchronization initiating subunit, configured to initiate data synchronization to a synchronization target node at regular time, where initiating data synchronization to the synchronization target node includes: selecting a close-range synchronization object node according to a first preset condition, generating a synchronization request list according to local groups of data corresponding to synchronization identification bits of the selected synchronization object node, sending the synchronization request list to the synchronization object node, receiving synchronization data returned by the synchronization object node, and setting the synchronization identification bits of the local groups of data corresponding to the synchronization object node;
The synchronous receiving subunit is used for receiving a synchronous request list sent by a synchronous request node, generating synchronous data according to the received synchronous request list and local groups of data corresponding to owned identification bits of the synchronous request node, setting the owned identification bits of the local groups of data corresponding to the synchronous request node, and sending the synchronous data to the synchronous request node;
Each group of data is respectively provided with a plurality of synchronized identification bits respectively corresponding to the synchronized object nodes and a plurality of owned identification bits respectively corresponding to the synchronized request nodes, the synchronized identification bits are used for identifying the synchronized corresponding synchronized object nodes of the group of data, and the owned identification bits are used for identifying that the corresponding synchronized request nodes have the group of data;
Wherein the generating of the synchronization request list according to the local groups of data corresponding to the synchronized identification bits of the selected synchronization object node comprises the following steps of:
Judging whether the synchronization object node is synchronized according to the synchronized identification bit of the group of data corresponding to the selected synchronization object node: if not, adding the hash value of the data into the synchronization request list;
Wherein the generating of the synchronization data according to the received synchronization request list and the local groups of data corresponding to the owned identification bits of the synchronization request node, and the setting of the local groups of data corresponding to the owned identification bits of the synchronization request node comprises:
Respectively judging whether the synchronous request node has all groups of data according to the synchronous request list: if the synchronous request node is owned, setting the owned identification bit of the group of data corresponding to the synchronous request node;
And adding each group of data which is not provided with the owned identification bit corresponding to the synchronous request node into synchronous data, and respectively setting the owned identification bits corresponding to the synchronous request node.
2. The data synchronization network according to claim 1, wherein the selecting the close-range synchronization object node according to the first preset condition comprises:
Calculating the distance between each node and the current node according to a preset rule, and selecting a plurality of nodes closest to the current node; sequentially comparing the selected nodes with a first preset condition according to the distance: if the node is in accordance with the synchronization object node, selecting the node as a synchronization object node; if none of the matches, stopping the synchronization.
3. The data synchronization network of claim 2, wherein the first preset condition comprises:
The selected node is not the current node;
the interval between the current synchronization time and the last synchronization time is smaller than a preset synchronization time interval;
The interval between the current synchronization time and the last synchronization failure time of the selected node is not less than a preset synchronization failure time interval, and the failure times are less than a preset failure time threshold;
And the interval between the current synchronization time and the last synchronization success time of the selected node is not less than the preset synchronization success time interval.
4. A data synchronization network according to any of claims 1-3, wherein the synchronization initiator unit is further configured to select a synchronization object node from a plurality of data source nodes according to a second predetermined condition.
5. the data synchronization network of claim 4, wherein the second preset condition comprises:
The interval between the current synchronization time and the last synchronization time is smaller than a preset synchronization time interval;
the interval between the current synchronization time and the last synchronization failure time of the data source node is not less than a preset synchronization failure time interval, and the failure times are less than a preset failure time threshold;
And the interval between the current synchronization time and the last synchronization success time of the data source node is not less than a preset synchronization success time interval.
6. The data synchronization network of claim 1, wherein the synchronization request list comprises hash values of a plurality of sets of data, and the determining whether the synchronization request node has each set of data according to the synchronization request list comprises:
comparing the hash value of each group of data with the hash value in the synchronization request list: if the synchronous request list comprises the hash value of the data of the group, the synchronous request node already has the data of the group;
Otherwise, it is not owned.
7. The data synchronization network of claim 1, wherein the data synchronization unit further comprises:
the identification bit management unit is used for managing each synchronized identification bit and each owned identification bit of each local group of data; the synchronous receiving subunit sets the synchronized identification bit by calling the identification bit management unit; and the synchronous receiving subunit sets the owned identification bit by calling the identification bit management unit.
8. A method for synchronizing data, the method comprising:
selecting a close-range synchronization object node according to a first preset condition;
generating a synchronization request list according to each set of local data corresponding to each synchronized identification bit of the selected synchronization object node;
sending the synchronization request list to the synchronization object node;
Receiving the synchronous data returned by the synchronous object node;
Setting synchronized identification bits corresponding to the synchronization object nodes of local groups of data;
The steps are circulated regularly;
Wherein each group of data is respectively provided with a plurality of synchronized identification bits which respectively correspond to the synchronized object nodes and are used for identifying the synchronized corresponding synchronized object nodes of the group of data,
the generating of the synchronization request list according to the local groups of data corresponding to the synchronized identification bits of the selected synchronization object node comprises the following steps of:
Judging whether the synchronization object node is synchronized according to the synchronized identification bit of the group of data corresponding to the selected synchronization object node: if not, adding the hash value of the data into the synchronization request list,
the receiving the synchronization data returned by the synchronization object node comprises:
the receiving synchronous receiving subunit respectively judges whether the synchronous request node has all groups of data according to the synchronous request list: if the synchronous request node has the identification bit, setting the group of data corresponding to the owned identification bit of the synchronous request node, adding the group of data which is not set with the owned identification bit corresponding to the synchronous request node into synchronous data, and respectively setting the owned identification bit corresponding to the synchronous request node and then sending the synchronous data.
9. the data synchronization method according to claim 8, wherein the selecting the close-range synchronization object node according to the first preset condition comprises:
Calculating the distance between each node and the current node according to a preset rule, and selecting a plurality of nodes closest to the current node; sequentially comparing the selected nodes with a first preset condition according to the distance: if the node is in accordance with the synchronization object node, selecting the node as a synchronization object node; if none of the matches, stopping the synchronization.
10. the data synchronization method according to claim 9, wherein the first preset condition comprises: the selected node is not the current node; the interval between the current synchronization time and the last synchronization time is smaller than a preset synchronization time interval; the interval between the current synchronization time and the last synchronization failure time of the selected node is not less than a preset synchronization failure time interval, and the failure times are less than a preset failure time threshold; and the interval between the current synchronization time and the last synchronization success time of the selected node is not less than the preset synchronization success time interval.
11. The data synchronization method according to any one of claims 8 to 10, wherein before the selecting the synchronization object node in the close range according to the first preset condition, the method further comprises:
Selecting a synchronization object node from the plurality of data source nodes according to a second preset condition: and if the synchronization request is selected, jumping to the step of generating a synchronization request list according to each set of local data corresponding to each synchronized identification bit of the selected synchronization object node.
12. The data synchronization method according to claim 11, wherein the second preset condition comprises: the interval between the current synchronization time and the last synchronization time is smaller than a preset synchronization time interval; the interval between the current synchronization time and the last synchronization failure time of the data source node is not less than a preset synchronization failure time interval, and the failure times are less than a preset failure time threshold; and the interval between the current synchronization time and the last synchronization success time of the data source node is not less than a preset synchronization success time interval.
13. a method for synchronizing data, the method comprising:
Receiving a synchronous request list sent by a synchronous request node; generating synchronous data according to the received synchronous request list and local groups of data corresponding to owned identification bits of the synchronous request nodes, and setting the owned identification bits of the local groups of data corresponding to the synchronous request nodes; sending the synchronization data to the synchronization request node; each group of data is provided with a plurality of owned identification bits respectively corresponding to the synchronous request nodes, and the owned identification bits are used for identifying that the corresponding synchronous request nodes own the group of data;
Wherein, the receiving of the synchronization request list sent by the synchronization request node includes:
the synchronous transmission unit is used for judging whether the synchronous object node is synchronized according to the synchronous identification bit of the group of data corresponding to the selected synchronous object node, and if the synchronous object node is not synchronized, the synchronous transmission unit is used for adding the hash value of the group of data into a synchronous request list and transmitting the synchronous request list;
wherein the generating of the synchronization data according to the received synchronization request list and the local groups of data corresponding to the owned identification bits of the synchronization request node, and the setting of the local groups of data corresponding to the owned identification bits of the synchronization request node comprises:
Respectively judging whether the synchronous request node has all groups of data according to the synchronous request list: if the synchronous request node is owned, setting the owned identification bit of the group of data corresponding to the synchronous request node;
adding each group of data which is not provided with the owned identification bit corresponding to the synchronous request node into synchronous data, and respectively setting the owned identification bits corresponding to the synchronous request node;
the synchronization request list comprises hash values of a plurality of groups of data, and the respectively judging whether the synchronization request node has each group of data according to the synchronization request list comprises:
comparing the hash value of each group of data with the hash value in the synchronization request list:
if the synchronous request list comprises the hash value of the data of the group, the synchronous request node already has the data of the group; otherwise, it is not owned.
14. the data synchronization method according to claim 13, wherein the synchronization request list includes hash values of a plurality of sets of data, and the determining whether the synchronization request node has each set of data according to the synchronization request list comprises:
comparing the hash value of each group of data with the hash value in the synchronization request list:
If the synchronous request list comprises the hash value of the data of the group, the synchronous request node already has the data of the group; otherwise, it is not owned.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610424362.2A CN106101229B (en) | 2016-06-15 | 2016-06-15 | Data synchronization network and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610424362.2A CN106101229B (en) | 2016-06-15 | 2016-06-15 | Data synchronization network and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106101229A CN106101229A (en) | 2016-11-09 |
| CN106101229B true CN106101229B (en) | 2019-12-13 |
Family
ID=57846644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610424362.2A Expired - Fee Related CN106101229B (en) | 2016-06-15 | 2016-06-15 | Data synchronization network and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106101229B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714941A (en) * | 2008-10-06 | 2010-05-26 | 联想(北京)有限公司 | Data updating method and data cache server |
| CN101826073A (en) * | 2009-03-06 | 2010-09-08 | 华为技术有限公司 | Synchronous method, apparatus and system for distributed database |
| CN102143194A (en) * | 2010-06-10 | 2011-08-03 | 华为技术有限公司 | Data synchronization method and system, immediate data node and terminal data node |
| CN102685257A (en) * | 2012-06-11 | 2012-09-19 | 深圳中兴网信科技有限公司 | Data synchronous processing system, data synchronous processing method and server system |
| CN103176867A (en) * | 2013-04-02 | 2013-06-26 | 浪潮电子信息产业股份有限公司 | Fast file differential backup method |
| CN103299602A (en) * | 2011-01-14 | 2013-09-11 | 苹果公司 | Data synchronization |
| CN104301990A (en) * | 2014-10-10 | 2015-01-21 | 大唐移动通信设备有限公司 | Data synchronizing method and device |
| CN104317971A (en) * | 2014-11-19 | 2015-01-28 | 亚信科技（南京）有限公司 | Real-time system parameter data non-blocking updating and access method |
| CN105227683A (en) * | 2015-11-11 | 2016-01-06 | 中国建设银行股份有限公司 | A kind of LDAP company-data synchronous method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7542432B2 (en) * | 2005-10-27 | 2009-06-02 | Alcatel Lucent | Resource matched topology database synchronization in communications networks having topology state routing protocols |
-
2016
- 2016-06-15 CN CN201610424362.2A patent/CN106101229B/en not_active Expired - Fee Related
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714941A (en) * | 2008-10-06 | 2010-05-26 | 联想(北京)有限公司 | Data updating method and data cache server |
| CN101826073A (en) * | 2009-03-06 | 2010-09-08 | 华为技术有限公司 | Synchronous method, apparatus and system for distributed database |
| CN102143194A (en) * | 2010-06-10 | 2011-08-03 | 华为技术有限公司 | Data synchronization method and system, immediate data node and terminal data node |
| CN103299602A (en) * | 2011-01-14 | 2013-09-11 | 苹果公司 | Data synchronization |
| CN102685257A (en) * | 2012-06-11 | 2012-09-19 | 深圳中兴网信科技有限公司 | Data synchronous processing system, data synchronous processing method and server system |
| CN103176867A (en) * | 2013-04-02 | 2013-06-26 | 浪潮电子信息产业股份有限公司 | Fast file differential backup method |
| CN104301990A (en) * | 2014-10-10 | 2015-01-21 | 大唐移动通信设备有限公司 | Data synchronizing method and device |
| CN104317971A (en) * | 2014-11-19 | 2015-01-28 | 亚信科技（南京）有限公司 | Real-time system parameter data non-blocking updating and access method |
| CN105227683A (en) * | 2015-11-11 | 2016-01-06 | 中国建设银行股份有限公司 | A kind of LDAP company-data synchronous method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106101229A (en) | 2016-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11494404B2 (en) | Data synchronization in a distributed data storage system | |
| CN107888562B (en) | Data verification and transceiving method, node and system for parallel link access to interconnection chain | |
| KR101752928B1 (en) | Swarm-based synchronization over a network of object stores | |
| US20170075941A1 (en) | Consensus system and method for adding data to a blockchain | |
| EP2378718B1 (en) | Method, node and system for controlling version in distributed system | |
| CN110190949B (en) | Method, system, apparatus and storage medium for large data volume resource uplink | |
| JP6488195B2 (en) | Communication device for time synchronization | |
| CN111159289B (en) | Method and device for synchronizing blocks | |
| JP7509797B2 (en) | Short Transaction Identifier Collision Detection and Reconciliation | |
| CN107231400B (en) | Data synchronization method and device | |
| JP2019504533A5 (en) | ||
| CN105530284A (en) | Method for synchronizing file | |
| EP3125501A1 (en) | File synchronization method, server, and terminal | |
| US20130124937A1 (en) | Method and apparatus for transmitting data in device-to-device service system | |
| CN104348859B (en) | File synchronisation method, device, server, terminal and system | |
| CN105933393A (en) | Intra-local area network data synchronization method, device and user terminal | |
| CN110557416B (en) | Multi-node collaborative blocking method and system | |
| CN111611321A (en) | Data storage method and device and block chain system | |
| CN110633168A (en) | Data backup method and system for distributed storage system | |
| WO2014206034A1 (en) | Data decomposition and synchronization method and system | |
| WO2015062082A1 (en) | Communication access method and user equipment | |
| US10362105B1 (en) | Generating probalistic data structures in gossip protocols | |
| WO2009103212A1 (en) | Method, system and device of data synchronization | |
| US8799212B2 (en) | Repository synchronization in a ranked repository cluster | |
| CN114491656A (en) | Method and apparatus in a blockchain network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20191213 |