CN110543448A - data synchronization method, device, equipment and computer readable storage medium - Google Patents

data synchronization method, device, equipment and computer readable storage medium Download PDF

Info

Publication number
CN110543448A
CN110543448A CN201910662462.2A CN201910662462A CN110543448A CN 110543448 A CN110543448 A CN 110543448A CN 201910662462 A CN201910662462 A CN 201910662462A CN 110543448 A CN110543448 A CN 110543448A
Authority
CN
China
Prior art keywords
file
data synchronization
data
hash table
local
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910662462.2A
Other languages
Chinese (zh)
Inventor
王强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
OneConnect Smart Technology Co Ltd
Original Assignee
OneConnect Smart Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by OneConnect Smart Technology Co Ltd filed Critical OneConnect Smart Technology Co Ltd
Priority to CN201910662462.2A priority Critical patent/CN110543448A/en
Publication of CN110543448A publication Critical patent/CN110543448A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • G06F16/137Hash-based
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/176Support for shared access to files; File sharing support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/178Techniques for file synchronisation in file systems

Abstract

The application provides a data synchronization method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: when a data synchronization request is received, generating a corresponding local file hash table according to the data files in the local shared file directory; determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request; and sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file. The application relates to data synchronization of a block chain, which can improve the data synchronization efficiency and reliability of a gateway node in the block chain.

Description

data synchronization method, device, equipment and computer readable storage medium
Technical Field
The present application relates to the field of block chains, and in particular, to a data synchronization method, apparatus, device, and computer readable storage medium.
Background
After the alliance chain is established, in order to facilitate the service system to be connected, a gateway node needs to be established on the alliance chain, the service system accesses the alliance chain through the gateway node, the gateway node increases correspondingly with the increase of the members of the alliance chain, after the gateway node is newly added, the public configuration information on the alliance chain and a digital Certificate (CA) of the blockchain node need to be synchronized, otherwise, the newly added gateway node cannot work normally, and in addition, when the public configuration information in the gateway node and the digital Certificate of the blockchain node change, the other gateway nodes also need to be synchronized.
At present, a common gateway node data synchronization scheme is to implement data synchronization by a shared storage (cloud disk or network disk) and a message queue notification manner, but the shared storage is centralized, when a storage service has a problem, data synchronization cannot be implemented, the reliability is low, when a message queue is synchronized, a message middleware needs to be built, when the message middleware has a problem, data synchronization cannot be implemented, the reliability is low, much time is spent to find data to be synchronized, the data synchronization efficiency is low, and therefore how to improve the data synchronization efficiency and reliability of a gateway node is a problem to be solved urgently at present.
Disclosure of Invention
The present application mainly aims to provide a data synchronization method, device, apparatus, and computer readable storage medium, which aim to improve data synchronization efficiency and reliability of a gateway node.
In a first aspect, the present application provides a data synchronization method, including the steps of:
when a data synchronization request is received, generating a corresponding local file hash table according to the data files in the local shared file directory;
Determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
And sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
In a second aspect, the present application further provides a data synchronization apparatus, including:
the generating module is used for generating a corresponding local file hash table according to the data files in the local shared file directory when receiving the data synchronization request;
The determining module is used for determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
and the sending module is used for sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the data synchronization method as described above.
in a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, where the computer program, when executed by a processor, implements the steps of the data synchronization method as described above.
the application provides a data synchronization method, a device, equipment and a computer readable storage medium, the application forms a shared file directory in each gateway node of a alliance chain according to the same structure, when a data synchronization request is received, a local file hash table is generated based on data files under the shared file directory, a target data file needing to be synchronized can be accurately and rapidly determined based on the local file hash table and the file hash table in the data synchronization request, then the target data file is sent to a corresponding gateway node, the gateway node performs data synchronization operation based on the target data file, the whole scheme does not need to add new hardware, does not need additional shared storage, does not need to use intermediate components such as message queues and the like, the reliability of data synchronization is effectively improved, and simultaneously, the data needing to be synchronized can be rapidly compared through data difference verification of the file hash table, the data synchronization efficiency of the gateway node is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data synchronization method according to an embodiment of the present application;
Fig. 2 is a schematic flowchart of another data synchronization method according to an embodiment of the present application;
Fig. 3 is a schematic flowchart of another data synchronization method according to an embodiment of the present application;
Fig. 4 is a schematic block diagram of a data synchronization apparatus according to an embodiment of the present application;
Fig. 5 is a schematic block diagram of another data synchronization apparatus provided in an embodiment of the present application;
Fig. 6 is a schematic block diagram of another data synchronization apparatus provided in an embodiment of the present application;
Fig. 7 is a block diagram schematically illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The embodiment of the application provides a data synchronization method, a data synchronization device, computer equipment and a computer readable storage medium. The data synchronization method can be applied to each gateway node of a alliance chain, the gateway node is a server accessed to the alliance chain, and the server can be a single server or a server cluster consisting of a plurality of servers.
some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a data synchronization method according to an embodiment of the present application.
As shown in fig. 1, the data synchronization method is used for synchronizing data between gateway nodes, wherein the data synchronization method includes steps S101 to S103.
step S101, when a data synchronization request is received, a corresponding local file hash table is generated according to data files in a local shared file directory.
the gateway nodes in the alliance chain can provide an alliance chain access function for a service system, a shared file directory is created in each gateway node of the alliance chain according to the same shared file directory structure, data files are stored in the shared file directory, each gateway node in the alliance chain realizes data communication through a gossip protocol, and the data files include but are not limited to data transmission protocol addresses (gossip addresses) of the gateway nodes, CA certificates of the block chain nodes and intelligent contracts in the alliances.
It should be noted that the shared file directory structure may be set based on actual situations, and this is not specifically limited in this application. Optionally, the shared file directory structure is as follows:
Shared file directory of/Share #
Gossip address of/GtAddr # gateway node
/orgA/gateway.addr
/orgB/gateway.addr
CA certificate of/Ca # block chain node
/orgA/peer1.cert
/orgB/peer2.cert
Intelligent contracts in ChainCode # alliance
/cc1/cc1.go
/cc2/cc2.go
the shared file directory structure comprises a folder for storing the gossip address of the gateway node, a folder for storing the CA certificate of the block chain node and a folder for storing the intelligent contract in the alliance.
and each gateway node in the alliance chain randomly selects the gossip address of one gateway node under the shared file directory at regular time, generates a corresponding file hash table according to the data file under the shared file directory, then generates a data synchronization request carrying the gossip address and the file hash table, and sends the data synchronization request to the corresponding other gateway nodes. When one gateway node receives a data synchronization request sent by another gateway node, the gateway node receiving the data synchronization request generates a corresponding local file hash table according to the data file in the local shared file directory.
In an embodiment, the file hash table is generated in a specific manner: the method comprises the steps of obtaining file content, file updating time, file names and storage paths of each data file from a local shared file directory, calculating file hash values corresponding to the file content of each data file, generating a corresponding local file hash table according to the file name, the storage path, the file updating time of each data file and the file hash values corresponding to the file content of each data file, namely taking the storage path + the file name as a key value, namely key, and the file hash value and the file updating time as data, namely value, obtaining key value pair representations of each data file, and writing the key value pair representations of each data file into hash map, so as to generate the corresponding file hash table. Hashmap is the implementation of a Map interface based on hash tables. It should be noted that the algorithms for calculating the Hash value of the file include, but are not limited to, an MD4 Algorithm, an MD5 Algorithm, and an SHA-1(Secure Hash Algorithm 1) Algorithm, and may be set based on actual situations, which is not specifically limited in this application.
for example, a file hash table is as follows:
The file hash table comprises two data files, namely a gossip address of a gateway node A and a CA certificate of a block chain node B, wherein the storage path + file name of the gossip address of the gateway node A is GtAddr, the file hash of the gossip address of the gateway node A is Mdfei2883kdaN, the file update time of the gossip address of the gateway node A is 2019-03-1209:08:02, the storage path + file name of the CA certificate of the block chain node B is Ca-orgB-peer1.cert, the file hash of the CA certificate of the block chain node B is Ihfeid 8233 kdadN, and the file update time of the CA certificate of the block chain node B is 2019-03-1209:08: 02.
In the concrete implementation, when a gateway node is newly added in a alliance chain, an alliance master gateway node stores the gossip address of the newly added gateway node into a shared file directory and acquires a shared file directory structure, and acquires a gateway node list (in which the name and the gossip address of each gateway node are stored) from the shared file directory, then based on the gossip address of the newly added gateway node, the shared file directory structure and the gateway node list are sent to the newly added gateway node, the newly added gateway node sends a full data synchronization request to any rest of the gateway nodes in the gateway node list after receiving the shared file directory structure and the gateway node list sent by the alliance master node, the gateway node receiving the full data synchronization request sends all data files in the shared file directory as target data files to be synchronized to the newly added gateway node, and after receiving the target data file, the newly added gateway node stores the target data file according to the shared file directory structure to form a shared file directory.
And step S102, determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
After the local file hash table is generated, the gateway node receiving the data synchronization request determines a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request, namely, the file hash table is obtained from the data synchronization request, a target file name and a target storage path of the data file which is not located in the file hash table are obtained from the local file hash table, then, according to the target storage path, the data file corresponding to the target file name is read from the local shared file directory, and the read data file is determined as the target data file.
For example, the local file hash table stores file names, storage paths, and file hashes of 5 data files, as shown in table 1, and the file hash table stores file names, storage paths, and file hashes of 3 data files, as shown in table 2.
Table 1 is a local file hash table
Filename + storage path File hashing
GtAddr-orgA-gateway.addr1 Mdfei2883kdaD
GtAddr-orgB-gateway.addr2 Cbbas1547abcF
GtAddr-orgC-gateway.addr3 Bqatx7895xieE
GtAddr-orgD-gateway.addr4 Fvdte5496xfgV
GtAddr-orgE-gateway.addr5 Enjik6874kdaL
table 2 is a file hash table
Filename + storage path file hashing
GtAddr-orgA-gateway.addr1 Mdfei2883kdaD
GtAddr-orgC-gateway.addr3 Bqatx7895xieE
GtAddr-orgE-gateway.addr5 Fazcs5496cvgT
As can be seen from comparing table 1 and table 2, the file hash table in table 2 lacks file names, storage paths, and file hashes of 2 data files, that is, lacks GtAddr-orgB-gateway.addr2 and GtAddr-orgD-gateway.addr4, and lacks Cbbas1547abcF and Fvdte5496xfgV, and further has a file hash of one data file that is inconsistent, that is, the file hash of GtAddr-orgE-gateway.addr5 is inconsistent, the corresponding file hash in table 1 is Enjik6874kdaL, and the corresponding file hash in table 2 is Fazcs5496cvgT, so that target file names of gateway.addr2, gateway.addr4, and gateway.addr5 can be obtained, and the target storage paths are GtAddr, gtatab, GtAddr, and GtAddr.
step S103, the target data file is sent to a corresponding gateway node according to the data synchronization request, so that the gateway node executes data synchronization operation based on the target data file.
After determining a target data file to be synchronized, the gateway node receiving the data synchronization request sends the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs data synchronization operation based on the target data file, that is, a data communication protocol address, namely a gossip address, is obtained from the data synchronization request, the target data file is sent to the gateway node corresponding to the gossip address, and the gateway node receiving the target data file stores the target data file in a local shared file directory to achieve data synchronization.
The data synchronization method provided by the above embodiment forms the shared file directory in the same structure in each gateway node of the federation chain, generating a local file hash table based on the data files under the shared file directory upon receiving the data synchronization request, and can accurately and quickly determine the target data file to be synchronized based on the local file hash table and the file hash table in the data synchronization request, then the target data file is sent to the corresponding gateway node, the gateway node executes data synchronization operation based on the target data file, the whole scheme does not need to add new hardware, does not need additional shared storage, does not need to use middleware such as message queue and the like, effectively improves the reliability of data synchronization, meanwhile, data needing to be synchronized can be quickly compared through data difference verification of the file hash table, and the data synchronization efficiency of the gateway node is effectively improved.
Referring to fig. 2, fig. 2 is a schematic flow chart of another data synchronization method according to an embodiment of the present disclosure.
as shown in fig. 2, the data synchronization method includes steps S201 to 204.
step S201, when a data synchronization request is received, generating a corresponding local file hash table according to the data file in the local shared file directory.
and each gateway node in the alliance chain randomly selects the gossip address of one gateway node under the shared file directory at regular time, generates a corresponding file hash table according to the data file under the shared file directory, then generates a data synchronization request carrying the gossip address and the file hash table, and sends the data synchronization request to the corresponding other gateway nodes. When one gateway node receives a data synchronization request sent by another gateway node, the gateway node receiving the data synchronization request generates a corresponding local file hash table according to the data file in the local shared file directory.
step S202, determining whether to trigger a data synchronization instruction according to the local file hash table and the file hash table in the data synchronization request.
after the local file hash table is generated, the gateway node receiving the data synchronization request determines whether to trigger the data synchronization instruction according to the local file hash table and the file hash table in the data synchronization request, specifically, obtains the file hash table from the data synchronization request, and determines whether the file hash table is the same as the local file hash table, if the file hash table is the same as the local file hash table, the data synchronization instruction is not triggered, and if the file hash table is different from the local file hash table, the data synchronization instruction is triggered.
The specific way for judging whether the file hash table is the same as the local file hash table is as follows: counting the number of file hashes stored in a file hash table, recording the number as a first file hash number, counting the number of file hashes stored in a local file hash table, recording the number as a second file hash number, then judging whether the first file hash number is smaller than the second file hash number, if the first file hash number is smaller than the second file hash number, determining that the file hash table is different from the local file hash table, if the first file hash number is equal to the second file hash number, further judging whether the file hashes of the file hash table, which are the same as key values (file name + storage path) in the local file hash table, are the same one by one, if at least one file hash of the same key values (file name + storage path) is different, determining that the file hash table is different from the local file hash table, if the file hashes of the same key values (file name + storage path) are the same, it may be determined that the file hash table is the same as the local file hash table.
Step S203, if a data synchronization instruction is triggered, determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
if the data synchronization instruction is triggered, the gateway node receiving the data synchronization request determines a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request, namely, the file hash table is obtained from the data synchronization request, a target file name and a target storage path of the data file which is not located in the file hash table are obtained from the local file hash table, then the data file corresponding to the target file name is read from the local shared file directory according to the target storage path, and the read data file is determined as the target data file. And if the data synchronization instruction is not triggered, sending a response to the corresponding gateway node, and the gateway node continuously and regularly generating a data synchronization request.
Step S204, the target data file is sent to the corresponding gateway node according to the data synchronization request, so that the gateway node executes data synchronization operation based on the target data file.
after determining a target data file to be synchronized, the gateway node receiving the data synchronization request sends the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs data synchronization operation based on the target data file, that is, a data communication protocol address, namely a gossip address, is obtained from the data synchronization request, the target data file is sent to the gateway node corresponding to the gossip address, and the gateway node receiving the target data file stores the target data file in a local shared file directory to achieve data synchronization.
In the data synchronization method provided in the above embodiment, after the local file hash table is generated, whether a data synchronization instruction is triggered is further determined based on the local file hash table and the file hash table in the data synchronization request, and only when the data synchronization instruction is triggered, the target data file to be synchronized is determined based on the file hash table and the local file hash table, so that the number of data synchronization times can be reduced, and the data synchronization accuracy can be improved.
Referring to fig. 3, fig. 3 is a schematic flow chart of another data synchronization method according to an embodiment of the present disclosure.
as shown in fig. 3, the data synchronization method includes steps S301 to 307.
Step S301, when receiving a data synchronization request, judging whether the data synchronization request is a shared data synchronization request or an authorized data synchronization request.
And each gateway node in the alliance chain randomly selects the gossip address of one gateway node under the shared file directory at regular time, generates a corresponding file hash table according to the data file under the shared file directory, then generates a shared data synchronization request carrying the gossip address and the file hash table, and sends the shared data synchronization request to the corresponding other gateway nodes. Or, some gateway nodes in the federation chain need to synchronize authorization data, and for this reason, the gateway node locally storing the authorization data table regularly acquires the latest authorization private key ID and the gossip address of the gateway node from the authorization data table, generates an authorization data synchronization request carrying the authorization private key ID and the gossip address, and then sends the authorization data synchronization request to the corresponding gateway node.
when one gateway node receives a data synchronization request sent by another gateway node, the gateway node receiving the data synchronization request judges whether the data synchronization request is a shared data synchronization request or an authorized data synchronization request, namely, a request type label is obtained from the data synchronization request, if the request type label is a first preset label, the data synchronization request can be determined to be the shared data synchronization request, and if the request type label is a second preset label, the data synchronization request can be determined to be the authorized data synchronization request. It should be noted that the preset tag may be set based on actual situations, and this is not specifically limited in this application.
Step S302, if the data synchronization request is a shared data synchronization request, generating a corresponding local file hash table according to the data files in the local shared file directory.
if the data synchronization request is a shared data synchronization request, generating a corresponding local file hash table according to the data files in the local shared file directory, namely acquiring the file content, the file updating time, the file name and the storage path of each data file from the local shared file directory, calculating the file hash value corresponding to the file content of each data file, and then generating the corresponding local file hash table according to the file name, the storage path, the file updating time of each data file and the file hash value corresponding to the file content of each data file.
Step S303, determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
After the local file hash table is generated, the gateway node receiving the data synchronization request determines a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request, namely, the file hash table is obtained from the data synchronization request, a target file name and a target storage path of the data file which is not located in the file hash table are obtained from the local file hash table, then, according to the target storage path, the data file corresponding to the target file name is read from the local shared file directory, and the read data file is determined as the target data file.
step S304, the target data file is sent to the corresponding gateway node according to the data synchronization request, so that the gateway node executes data synchronization operation based on the target data file.
After determining a target data file to be synchronized, the gateway node receiving the data synchronization request sends the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs data synchronization operation based on the target data file, that is, a data communication protocol address, namely a gossip address, is obtained from the data synchronization request, the target data file is sent to the gateway node corresponding to the gossip address, and the gateway node receiving the target data file stores the target data file in a local shared file directory to achieve data synchronization.
step S305, if the data synchronization request is an authorized data synchronization request, determining whether a first authorized private key ID in a local data authorization table is the same as a second authorized private key ID in the authorized data synchronization request.
if the data synchronization request is an authorized data synchronization request, the gateway node receiving the data synchronization request judges whether a first authorized private key ID in a local data authorization table is the same as a second authorized private key ID in the authorized data synchronization request, if the first authorized private key ID is the same as the second authorized private key ID, it can be determined that the data is not required to be synchronized, and if the first authorized private key ID is different from the second authorized private key ID, the data is required to be synchronized.
the data authorization table stores an authorization private key ID, a self-increment ID, an authorization private key, an authorization data ID, an authorization gateway node and an authorized gateway node, and the fields of the data authorization table are shown in table 3.
Table 3 is a data authorization table
Self-increment ID Authorization private Key ID Authorized private key Authorization data ID Authorized gateway node Authorized gateway node
The field "self-increment ID" stores the database self-increment ID, the field "authorization private key ID" stores the authorization gateway node identifier-authorized gateway node identifier-self-increment ID, the field "data private key" stores the private key for data encryption, the field "authorization data ID" stores the linked key of the authorization data, the field "authorization gateway node" stores the identifier and gossip address of the authorization gateway node, and the field "authorized gateway node" stores the identifier and gossip address of the authorized gateway node.
Step S306, if the first authorization private key ID is different from the second authorization private key ID, determining a target authorization record to be synchronized according to the first authorization private key ID and the second authorization private key ID.
if the first authorization private key ID is different from the second authorization private key ID, the gateway node determines a target authorization record to be synchronized according to the first authorization private key ID and the second authorization private key ID, specifically, calculates a difference value between the first authorization private key ID and the second authorization private key ID, and determines the target authorization private key ID according to the difference value, namely, sequentially increasing 1 on the basis of the second authorization private key ID until the difference value between the target authorization private key ID and the first authorization private key ID is 1; after the target authorization private key ID is determined, the authorization record corresponding to the target authorization private key ID in the local data authorization table is determined as the target authorization record. Wherein the authorization record includes, but is not limited to, an authorization private key ID, a self-increment ID, an authorization private key, an authorization data ID, an authorization gateway node, and an authorized gateway node.
step S307, the target authorization record is sent to a corresponding gateway node, so that the gateway node performs data synchronization operation based on the target authorization record.
After the target authorization record is determined, the target authorization record is sent to the corresponding gateway node, so that the gateway node performs data synchronization operation based on the target authorization record, that is, a gossip address is obtained from the data synchronization request, the target authorization record is sent to the gateway node corresponding to the gossip address, the gateway node receiving the target authorization record writes the target authorization record into a local data authorization table, and authorization data synchronization is achieved.
The data synchronization method provided by the embodiment can quickly determine whether the authorization data needs to be synchronized or not by comparing the self-added authorization private key ID, so that the synchronization efficiency of the authorization data is improved.
Referring to fig. 4, fig. 4 is a schematic block diagram of a data synchronization apparatus according to an embodiment of the present disclosure.
As shown in fig. 4, the data synchronization apparatus 400 includes: a generation module 401, a determination module 402 and a sending module 403.
The generating module 401 is configured to generate a corresponding local file hash table according to the data file in the local shared file directory when the data synchronization request is received.
In an embodiment, the generating module 401 is further configured to obtain the file content, the file update time, the file name, and the storage path of each data file from the local shared file directory, and calculate a file hash value corresponding to the file content of each data file; and generating a corresponding local file hash table according to the file name, the storage path, the file updating time and the file hash value corresponding to the file content of each data file.
A determining module 402, configured to determine a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
In an embodiment, the determining module 402 is further configured to obtain a file hash table from the data synchronization request, and obtain a target file name and a target storage path of a data file that is not located in the file hash table from the local file hash table; and reading the data file corresponding to the target file name from the local shared file directory according to the target storage path, and determining the read data file as a target data file.
a sending module 403, configured to send the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs a data synchronization operation based on the target data file.
referring to fig. 5, fig. 5 is a schematic block diagram of another data synchronization apparatus according to an embodiment of the present disclosure.
As shown in fig. 5, the data synchronization apparatus 500 includes: a generation module 501, an instruction triggering module 502, a determination module 503 and a sending module 504.
The generating module 501 is configured to generate a corresponding local file hash table according to the data file in the local shared file directory when receiving the data synchronization request.
An instruction triggering module 502, configured to determine whether to trigger a data synchronization instruction according to the local file hash table and the file hash table in the data synchronization request;
In an embodiment, the instruction triggering module 502 is further configured to obtain a file hash table from the data synchronization request, and determine whether the file hash table is the same as the local file hash table; if the file hash table is the same as the local file hash table, not triggering a data synchronization instruction; and if the file hash table is different from the local file hash table, triggering a data synchronization instruction.
A determining module 503, configured to determine, if a data synchronization instruction is triggered, a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
A sending module 504, configured to send the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs data synchronization operation based on the target data file.
Referring to fig. 6, fig. 6 is a schematic block diagram of another data synchronization apparatus according to an embodiment of the present disclosure.
as shown in fig. 6, the data synchronization apparatus 600 includes: a first judging module 601, a generating module 602, a first determining module 603, a sending module 604, a second judging module 605 and a second determining module 606.
The first determining module 601 is configured to determine, when a data synchronization request is received, whether the data synchronization request is a shared data synchronization request or an authorized data synchronization request.
A generating module 602, configured to generate a corresponding local file hash table according to the data file in the local shared file directory if the data synchronization request is a shared data synchronization request.
A first determining module 603, configured to determine a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
A sending module 604, configured to send the target data file to a corresponding gateway node according to the data synchronization request, so that the gateway node performs a data synchronization operation based on the target data file.
A second determining module 605, configured to determine whether the first authorization private key ID in the local data authorization table is the same as the second authorization private key ID in the authorization data synchronization request if the data synchronization request is an authorization data synchronization request.
A second determining module 606, configured to determine, if the first authorized private key ID is different from the second authorized private key ID, a target authorization record to be synchronized according to the first authorized private key ID and the second authorized private key ID.
in an embodiment, the second determining module 606 is further configured to calculate a difference between the first authorized private key ID and the second authorized private key ID; and determining a target authorization private key ID according to the difference, and determining an authorization record corresponding to the target authorization private key ID in the local data authorization table as a target authorization record.
The sending module 603 is further configured to send the target authorization record to a corresponding gateway node, so that the gateway node performs data synchronization operation based on the target authorization record.
It should be noted that, as will be clear to those skilled in the art, for convenience and brevity of description, the specific working processes of the apparatus and each module and unit described above may refer to the corresponding processes in the foregoing data synchronization method embodiment, and are not described herein again.
The apparatus provided by the above embodiments may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 7.
Referring to fig. 7, fig. 7 is a schematic block diagram illustrating a structure of a computer device according to an embodiment of the present disclosure. The computer device may be a server.
as shown in fig. 7, the computer device includes a processor, a memory, and a network interface connected by a system bus, wherein the memory may include a nonvolatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any of the data synchronization methods.
The processor is used for providing calculation and control capability and supporting the operation of the whole computer equipment.
The internal memory provides an environment for the execution of a computer program on a non-volatile storage medium, which when executed by a processor, causes the processor to perform any of a variety of data synchronization methods.
the network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It should be understood that the Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein, in one embodiment, the processor is configured to execute a computer program stored in the memory to implement the steps of:
When a data synchronization request is received, generating a corresponding local file hash table according to the data files in the local shared file directory;
determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
and sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
In one embodiment, when the processor generates the corresponding local file hash table according to the data file in the local shared file directory, the processor is configured to:
Acquiring the file content, the file updating time, the file name and the storage path of each data file from a local shared file directory, and calculating the file hash value corresponding to the file content of each data file;
And generating a corresponding local file hash table according to the file name, the storage path, the file updating time and the file hash value corresponding to the file content of each data file.
In one embodiment, when determining the target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request, the processor is configured to:
Acquiring a file hash table from the data synchronization request, and acquiring a target file name and a target storage path of a data file which is not located in the file hash table from the local file hash table;
And reading the data file corresponding to the target file name from the local shared file directory according to the target storage path, and determining the read data file as a target data file.
Wherein in another embodiment the processor is adapted to run a computer program stored in the memory to implement the steps of:
When a data synchronization request is received, generating a corresponding local file hash table according to the data files in the local shared file directory;
Determining whether to trigger a data synchronization instruction according to the local file hash table and a file hash table in the data synchronization request;
If a data synchronization instruction is triggered, determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
and sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
In one embodiment, when determining whether to trigger a data synchronization instruction according to the local file hash table and the file hash table in the data synchronization request, the processor is configured to:
Acquiring a file hash table from the data synchronization request, and judging whether the file hash table is the same as the local file hash table;
If the file hash table is the same as the local file hash table, not triggering a data synchronization instruction;
And if the file hash table is different from the local file hash table, triggering a data synchronization instruction.
Wherein in another embodiment the processor is adapted to run a computer program stored in the memory to implement the steps of:
When a data synchronization request is received, judging whether the data synchronization request is a shared data synchronization request or an authorized data synchronization request;
If the data synchronization request is a shared data synchronization request, generating a corresponding local file hash table according to the data files in the local shared file directory;
Determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
Sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node performs data synchronization operation based on the target data file;
If the data synchronization request is an authorized data synchronization request, judging whether a first authorized private key ID in a local data authorization table is the same as a second authorized private key ID in the authorized data synchronization request;
If the first authorization private key ID is different from the second authorization private key ID, determining a target authorization record to be synchronized according to the first authorization private key ID and the second authorization private key ID;
And sending the target authorization record to a corresponding gateway node so that the gateway node performs data synchronization operation based on the target authorization record.
In one embodiment, the processor, when implementing determining a target authorization record to be synchronized according to the first authorization private key ID and the second authorization private key ID, is configured to implement:
Calculating a difference between the first authorization private key ID and the second authorization private key ID;
And determining a target authorization private key ID according to the difference, and determining an authorization record corresponding to the target authorization private key ID in the local data authorization table as a target authorization record.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, where the computer program includes program instructions, and a method implemented when the program instructions are executed may refer to the embodiments of the data synchronization method in the present application.
The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
it should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
the above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method of data synchronization, comprising:
when a data synchronization request is received, generating a corresponding local file hash table according to the data files in the local shared file directory;
Determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
And sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
2. the data synchronization method according to claim 1, wherein the step of generating the corresponding local file hash table according to the data files in the local shared file directory comprises:
acquiring the file content, the file updating time, the file name and the storage path of each data file from a local shared file directory, and calculating the file hash value corresponding to the file content of each data file;
and generating a corresponding local file hash table according to the file name, the storage path, the file updating time and the file hash value corresponding to the file content of each data file.
3. The data synchronization method of claim 1, wherein the step of determining the target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request comprises:
acquiring a file hash table from the data synchronization request, and acquiring a target file name and a target storage path of a data file which is not located in the file hash table from the local file hash table;
And reading the data file corresponding to the target file name from the local shared file directory according to the target storage path, and determining the read data file as a target data file.
4. The data synchronization method of claim 1, wherein before the step of determining the target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request, the method further comprises:
determining whether to trigger a data synchronization instruction according to the local file hash table and a file hash table in the data synchronization request;
And if the data synchronization instruction is triggered, determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request.
5. The data synchronization method of claim 4, wherein the step of determining whether to trigger a data synchronization instruction based on the local file hash table and the file hash table in the data synchronization request comprises:
Acquiring a file hash table from the data synchronization request, and judging whether the file hash table is the same as the local file hash table;
if the file hash table is the same as the local file hash table, not triggering a data synchronization instruction;
and if the file hash table is different from the local file hash table, triggering a data synchronization instruction.
6. the data synchronization method according to any one of claims 1 to 5, wherein before the step of generating the corresponding local file hash table according to the data files in the local shared file directory, the method further comprises:
When a data synchronization request is received, judging whether the data synchronization request is a shared data synchronization request or an authorized data synchronization request;
if the data synchronization request is a shared data synchronization request, generating a corresponding local file hash table according to the data files in the local shared file directory;
If the data synchronization request is an authorized data synchronization request, judging whether a first authorized private key ID in a local data authorization table is the same as a second authorized private key ID in the authorized data synchronization request;
If the first authorization private key ID is different from the second authorization private key ID, determining a target authorization record to be synchronized according to the first authorization private key ID and the second authorization private key ID;
And sending the target authorization record to a corresponding gateway node so that the gateway node performs data synchronization operation based on the target authorization record.
7. the data synchronization method of claim 6, wherein the step of determining the target authorization record to be synchronized based on the first authorization private key ID and the second authorization private key ID comprises:
Calculating a difference between the first authorization private key ID and the second authorization private key ID;
and determining a target authorization private key ID according to the difference, and determining an authorization record corresponding to the target authorization private key ID in the local data authorization table as a target authorization record.
8. A data synchronization apparatus, characterized in that the data synchronization apparatus comprises:
The generating module is used for generating a corresponding local file hash table according to the data files in the local shared file directory when receiving the data synchronization request;
the determining module is used for determining a target data file to be synchronized according to the local file hash table and the file hash table in the data synchronization request;
And the sending module is used for sending the target data file to a corresponding gateway node according to the data synchronization request so that the gateway node executes data synchronization operation based on the target data file.
9. a computer arrangement, characterized in that the computer arrangement comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the data synchronization method according to any of claims 1 to 7.
10. a computer-readable storage medium, having stored thereon a computer program, wherein the computer program, when being executed by a processor, carries out the steps of the data synchronization method according to any one of claims 1 to 7.
CN201910662462.2A 2019-07-22 2019-07-22 data synchronization method, device, equipment and computer readable storage medium Pending CN110543448A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910662462.2A CN110543448A (en) 2019-07-22 2019-07-22 data synchronization method, device, equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910662462.2A CN110543448A (en) 2019-07-22 2019-07-22 data synchronization method, device, equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN110543448A true CN110543448A (en) 2019-12-06

Family

ID=68709972

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910662462.2A Pending CN110543448A (en) 2019-07-22 2019-07-22 data synchronization method, device, equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN110543448A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241115A (en) * 2020-01-07 2020-06-05 腾讯科技(深圳)有限公司 Data synchronization method, device, equipment and storage medium
CN111294391A (en) * 2020-01-17 2020-06-16 深信服科技股份有限公司 Configuration synchronization method, device, equipment and readable storage medium
CN111654522A (en) * 2020-04-27 2020-09-11 平安证券股份有限公司 File synchronization method, file synchronization server and storage medium
CN112527756A (en) * 2020-12-17 2021-03-19 厦门市美亚柏科信息股份有限公司 File synchronization operation method, terminal equipment and storage medium
CN112905553A (en) * 2021-02-07 2021-06-04 全链通有限公司 Data sharing method, device and system based on block chain
WO2021196463A1 (en) * 2020-03-30 2021-10-07 深圳壹账通智能科技有限公司 Blockchain data synchronization method and apparatus, and electronic device and storage medium
WO2022088807A1 (en) * 2020-10-30 2022-05-05 深圳壹账通智能科技有限公司 Distributed file storage method and system based on blockchain, and server and client
CN114547195A (en) * 2022-02-14 2022-05-27 平安科技(深圳)有限公司 Batch data synchronization task processing method, device, equipment and storage medium
CN115361399A (en) * 2022-10-24 2022-11-18 中国水利水电第七工程局有限公司 Multi-terminal data synchronization method, device and system
CN115630341A (en) * 2022-12-22 2023-01-20 湖南国科亿存信息科技有限公司 Software license authorization management and control method and system in high-availability storage equipment
CN117112508A (en) * 2023-10-20 2023-11-24 杭州美创科技股份有限公司 File synchronization method and device based on serial numbers, computer equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090041249A1 (en) * 2007-04-09 2009-02-12 Tomonari Tanaka Method and system for managing a key for encryption or decryption of data
US20140304840A1 (en) * 2011-10-12 2014-10-09 International Business Machines Corporation Deleting Information to Maintain Security Level
CN107948283A (en) * 2017-11-24 2018-04-20 中钞信用卡产业发展有限公司杭州区块链技术研究院 A kind of big file of alliance's chain stores and the method and system of verification
CN108810894A (en) * 2018-05-31 2018-11-13 康键信息技术(深圳)有限公司 Authorization terminal method, apparatus, computer equipment and storage medium
WO2018208546A1 (en) * 2017-05-08 2018-11-15 Amazon Technologies, Inc. Generation of shared secrets using pairwise implicit certificates
CN109062935A (en) * 2018-06-14 2018-12-21 新华三技术有限公司 A kind of method and apparatus of file operation
US20190068615A1 (en) * 2017-08-31 2019-02-28 Topia Technology, Inc. Secure decentralized file sharing systems and methods
CN109714170A (en) * 2018-12-20 2019-05-03 弗洛格(武汉)信息科技有限公司 Data isolation method and corresponding alliance's catenary system in a kind of alliance's chain
CN109739810A (en) * 2018-12-07 2019-05-10 中山市江波龙电子有限公司 File synchronisation method, server, client and the device with store function

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090041249A1 (en) * 2007-04-09 2009-02-12 Tomonari Tanaka Method and system for managing a key for encryption or decryption of data
US20140304840A1 (en) * 2011-10-12 2014-10-09 International Business Machines Corporation Deleting Information to Maintain Security Level
WO2018208546A1 (en) * 2017-05-08 2018-11-15 Amazon Technologies, Inc. Generation of shared secrets using pairwise implicit certificates
US20190068615A1 (en) * 2017-08-31 2019-02-28 Topia Technology, Inc. Secure decentralized file sharing systems and methods
CN107948283A (en) * 2017-11-24 2018-04-20 中钞信用卡产业发展有限公司杭州区块链技术研究院 A kind of big file of alliance's chain stores and the method and system of verification
CN108810894A (en) * 2018-05-31 2018-11-13 康键信息技术(深圳)有限公司 Authorization terminal method, apparatus, computer equipment and storage medium
CN109062935A (en) * 2018-06-14 2018-12-21 新华三技术有限公司 A kind of method and apparatus of file operation
CN109739810A (en) * 2018-12-07 2019-05-10 中山市江波龙电子有限公司 File synchronisation method, server, client and the device with store function
CN109714170A (en) * 2018-12-20 2019-05-03 弗洛格(武汉)信息科技有限公司 Data isolation method and corresponding alliance's catenary system in a kind of alliance's chain

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241115A (en) * 2020-01-07 2020-06-05 腾讯科技(深圳)有限公司 Data synchronization method, device, equipment and storage medium
CN111241115B (en) * 2020-01-07 2022-02-22 腾讯科技(深圳)有限公司 Data synchronization method, device, equipment and storage medium
CN111294391A (en) * 2020-01-17 2020-06-16 深信服科技股份有限公司 Configuration synchronization method, device, equipment and readable storage medium
WO2021196463A1 (en) * 2020-03-30 2021-10-07 深圳壹账通智能科技有限公司 Blockchain data synchronization method and apparatus, and electronic device and storage medium
CN111654522A (en) * 2020-04-27 2020-09-11 平安证券股份有限公司 File synchronization method, file synchronization server and storage medium
CN111654522B (en) * 2020-04-27 2023-02-03 平安证券股份有限公司 File synchronization method, file synchronization server and storage medium
WO2022088807A1 (en) * 2020-10-30 2022-05-05 深圳壹账通智能科技有限公司 Distributed file storage method and system based on blockchain, and server and client
CN112527756A (en) * 2020-12-17 2021-03-19 厦门市美亚柏科信息股份有限公司 File synchronization operation method, terminal equipment and storage medium
CN112905553A (en) * 2021-02-07 2021-06-04 全链通有限公司 Data sharing method, device and system based on block chain
CN114547195A (en) * 2022-02-14 2022-05-27 平安科技(深圳)有限公司 Batch data synchronization task processing method, device, equipment and storage medium
CN115361399A (en) * 2022-10-24 2022-11-18 中国水利水电第七工程局有限公司 Multi-terminal data synchronization method, device and system
CN115630341A (en) * 2022-12-22 2023-01-20 湖南国科亿存信息科技有限公司 Software license authorization management and control method and system in high-availability storage equipment
CN115630341B (en) * 2022-12-22 2023-03-10 湖南国科亿存信息科技有限公司 Software license authorization control method and system in high-availability storage equipment
CN117112508A (en) * 2023-10-20 2023-11-24 杭州美创科技股份有限公司 File synchronization method and device based on serial numbers, computer equipment and storage medium
CN117112508B (en) * 2023-10-20 2024-02-06 杭州美创科技股份有限公司 File synchronization method and device based on serial numbers, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN110543448A (en) data synchronization method, device, equipment and computer readable storage medium
CN109831487B (en) Fragmented file verification method and terminal equipment
CN109889505B (en) Data consistency verification method and terminal equipment
CN111163182B (en) Block chain-based device registration method and apparatus, electronic device, and storage medium
CN109492013B (en) Data processing method, device and system applied to database cluster
CN109032803B (en) Data processing method and device and client
CN109522314B (en) Block chain-based data archiving method and terminal equipment
CN110222535B (en) Processing device, method and storage medium for block chain configuration file
CN110597918A (en) Account management method and device and computer readable storage medium
CN109145053B (en) Data processing method and device, client and server
CN111597567A (en) Data processing method, data processing device, node equipment and storage medium
CN112214519A (en) Data query method, device, equipment and readable medium
CN111338834B (en) Data storage method and device
CN111343267B (en) Configuration management method and system
WO2016101759A1 (en) Data routing method, data management device and distributed storage system
CN112702167A (en) Cluster feature code obtaining method and device and electronic equipment
CN112385178B (en) Lightweight certificate status checking system for large number of certificates
CN113542405A (en) Block chain-based network communication system, method, device and storage medium
CN110209347B (en) Traceable data storage method
CN110445765B (en) Data sharing method based on block chain, terminal device and medium
CN109670956B (en) Block chain transaction method and device
CN116578746A (en) Object de-duplication method and device
CN111464312B (en) Method and device for processing account addresses in blockchain and electronic equipment
CN113760876A (en) Data filtering method and device
CN113469815A (en) Data management method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination