CN113556371B - Data synchronization method, system, equipment and computer readable storage medium - Google Patents
Data synchronization method, system, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113556371B CN113556371B CN202010337596.XA CN202010337596A CN113556371B CN 113556371 B CN113556371 B CN 113556371B CN 202010337596 A CN202010337596 A CN 202010337596A CN 113556371 B CN113556371 B CN 113556371B
- Authority
- CN
- China
- Prior art keywords
- data
- data storage
- synchronized
- identifier
- storage center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Abstract
The application discloses a data synchronization method, a system, equipment and a computer readable storage medium, which are applied to a target data storage center and used for determining self-acquired data to be synchronized; acquiring a first data identifier of data to be synchronized, wherein the first data identifier is an equipment identifier for operating new data; judging whether the first data identifier is an identifier corresponding to the target data storage center; if so, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if not, storing the data to be synchronized. In the application, other data storage centers can only store the data to be synchronized, so that the data consistency of multiple data centers is realized, and the repeated transmission of the data among the data storage centers is avoided; in addition, the target data storage center synchronizes the data to be synchronized to other data storage centers, so that the data can be read and written to a plurality of data storage centers at one time, and the data synchronization efficiency is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data synchronization method, system, device, and computer-readable storage medium.
Background
The multi-data center refers to a service server storage center established in different areas, is used for solving the problem of disaster tolerance of a machine room or the access performance of different areas, and plays a plurality of roles in the field of computer storage. In the application process of multiple data centers, in order to ensure that the data of each data storage center are consistent, the data of each data storage center needs to be synchronized.
However, in the existing multi-data center, a read function is separated from a write function, the read function is placed in one data storage center, the write function is placed in another data storage center, only one data storage center for writing data sources is provided each time, and only one data storage center for reading data sources is provided, so that when data reading is needed, two modes are provided, the first mode is reading from a source back to the storage center where the data sources are located, and at the moment, the data storage center needs to be crossed; the second is directly from another data storage center, and the data synchronization of the data writing centers needs to be waited to reach consistency; all resulting in slower data consistency efficiency for multiple data centers.
In summary, how to improve the data consistency efficiency of multiple data centers is an urgent problem to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a data synchronization method which can solve the technical problem of improving the data consistency efficiency of multiple data centers to a certain extent. The application also provides a data synchronization system, a device and a computer readable storage medium.
In order to achieve the above purpose, the present application provides the following technical solutions:
a data synchronization method is applied to a target data storage center and comprises the following steps:
determining self-acquired data to be synchronized;
acquiring a first data identifier of the data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized;
judging whether the first data identifier is an identifier corresponding to the target data storage center;
if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized;
and if the first data identification is not the identification corresponding to the target data storage center, storing the data to be synchronized.
Preferably, the synchronizing the data to be synchronized to other data storage centers includes:
and sending the data to be synchronized to the other data storage centers based on a data synchronization mechanism.
Preferably, the synchronizing the data to be synchronized to other data storage centers includes:
and sending the data to be synchronized to the other data storage centers based on a master-slave copying mechanism.
Preferably, the method further comprises the following steps:
receiving a data processing request;
determining a response data storage center responding to the data processing request;
and sending the data processing request to the response data storage center so that the response data storage center responds to the data processing request.
Preferably, the data processing request carries a timestamp;
after receiving the data processing request, the method further includes:
and judging whether the difference value between the timestamp of the stored data and the timestamp of the data processing request is greater than a preset difference value, if so, pulling and storing the data from other data storage centers.
Preferably, the determining a response data storage center responding to the data processing request includes:
and determining the response data storage center based on the slicing principle.
Preferably, the determining the response data storage center based on the fragmentation principle includes:
acquiring a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request;
and determining the response data storage center based on the slicing principle, the second data identifier and the number of the data storage centers.
Preferably, the determining the response data storage center based on the fragmentation principle, the second data identifier and the number of data storage centers includes:
if the second data identification is of a digital type, acquiring the remainder of the second data identification and the number of the data storage centers;
and determining the data storage center corresponding to the remainder as the response data storage center.
Preferably, the determining the response data storage center based on the fragmentation principle, the second data identifier and the number of data storage centers includes:
if the second data identifier is a character type, converting the second data identifier into a corresponding ASCII value, and acquiring an accumulated value of the ASCII value;
obtaining the remainder of the accumulated value and the number of the data storage centers;
and determining the data storage center corresponding to the remainder as the response data storage center.
A data synchronization system is applied to a target data storage center and comprises:
the first determining module is used for determining the data to be synchronized acquired by the first determining module;
a first obtaining module, configured to obtain a first data identifier of the data to be synchronized, where the first data identifier is an equipment identifier for operating the data to be synchronized;
the first judging module is used for judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if the first data identification is not the identification corresponding to the target data storage center, storing the data to be synchronized.
A data synchronization apparatus comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data synchronization method as described in any one of the above when executing the computer program.
A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the data synchronization method as set forth in any one of the preceding claims.
The data synchronization method is applied to a target data storage center and used for determining data to be synchronized acquired by the target data storage center; acquiring a first data identifier of data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized; judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if the first data identifier is not the identifier corresponding to the target data storage center, storing the data to be synchronized. In the application, after the target data storage center acquires the data to be synchronized, a first data identifier of the data to be synchronized is acquired, wherein the first data identifier is an equipment identifier for operating the data to be synchronized; judging whether the first data identification is an identification corresponding to a target data storage center, namely whether the first data identification is obtained by self modification, if so, storing data to be synchronized, and synchronizing the data to be synchronized to other data storage centers, so that the other data storage centers can store the data to be synchronized, thereby realizing the data consistency of multiple data centers, and if not, storing the data to be synchronized, namely if the data to be synchronized is not modified by the target data storage center, the target data storage center only stores the data to be synchronized, and the data to be synchronized cannot be synchronized to other data storage centers, thereby avoiding the repeated transmission of the data to be synchronized among the data storage centers; in addition, the target data storage center synchronizes the data to be synchronized to other data storage centers, so that the data can be read and written to a plurality of data storage centers at one time, and the data synchronization efficiency is improved. The data synchronization system, the data synchronization equipment and the computer readable storage medium solve the corresponding technical problems.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a first flowchart of a data synchronization method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of data synchronization of a machine room;
FIG. 3 is a schematic diagram of master-slave replication of data in a machine room;
fig. 4 is a second flowchart of a data synchronization method according to an embodiment of the present application;
FIG. 5 is a flow chart of a data synchronization method provided in the present application;
fig. 6 is a schematic structural diagram of a data synchronization system according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application;
fig. 8 is another schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The multi-data center refers to a service server storage center established in different areas, and at least comprises two data storage centers, so that the problem of disaster tolerance of a machine room or access performance of different areas is solved, and the multi-data center plays a lot of roles in the field of computer storage. In the application process of multiple data centers, in order to ensure that the data of each data storage center are consistent, the data of each data storage center needs to be synchronized. However, in the existing multi-data center, a read function is separated from a write function, the read function is placed in one data storage center, the write function is placed in another data storage center, only one data storage center for writing data sources is provided each time, and only one data storage center for reading data sources is provided, so that when data reading is needed, two modes are provided, the first mode is reading from a source back to the storage center where the data sources are located, and at the moment, the data storage center needs to be crossed; the second is directly from another data storage center, and the data synchronization of the data writing centers needs to be waited to reach consistency; all resulting in slower data consistency efficiency for multiple data centers. The data synchronization method provided by the application can improve the data consistency efficiency of multiple data centers.
Referring to fig. 1, fig. 1 is a first flowchart of a data synchronization method according to an embodiment of the present disclosure.
The data processing method under the multiple data centers, provided by the embodiment of the application, is applied to a target data storage center of the multiple data centers, and can comprise the following steps:
step S101: and determining the self-acquired data to be synchronized.
In practical application, since the situation of data inconsistency in multiple data centers occurs after the data to be synchronized is generated, that is, when the data storage center does not update itself, in order to achieve data consistency in multiple data centers, the data to be synchronized acquired by itself may be determined first. In a specific application scene, the data to be synchronized can be acquired by analyzing the time content monitored by the user; or the data to be synchronized is acquired by analyzing the log information of the database.
It should be noted that the application is not limited to the type of the target data storage center, the target data storage center is any data storage center of multiple data centers, and the type of the data storage center may be a machine room, a storage medium, a storage server, or the like.
Step S102: and acquiring a first data identifier of the data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized.
In practical application, in order to determine whether the data to be synchronized is data modified by a certain data storage center, a first data identifier may be added to the data to be synchronized, and since the first data identifier is an equipment identifier for operating the data to be synchronized, the attribution of the data to be synchronized is determined by the first data identifier of the data to be synchronized. It should be noted that the device identifier refers to an identifier that distinguishes one device from other devices and belongs to the device, in a specific application scenario, a factory number of the device may be used as the device identifier, or the device identifier may be generated based on an IP address, a factory number, and the like of the device, which is not limited herein.
Step S103: judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, executing step S104; if the first data identifier is not the identifier corresponding to the target data storage center, step S105 is executed.
Step S104: and storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized.
Step S105: and storing the data to be synchronized.
In practical application, after the data identifier of the data to be synchronized is obtained, in order to avoid repeated copying of the data to be synchronized among the data storage centers, whether the first data identifier is an identifier corresponding to a target data storage center or not can be judged first; if yes, the data to be synchronized can be stored, and the data to be synchronized is synchronized to other data storage centers, so that the other data storage centers can store the data to be synchronized, data updating of the other data storage centers is completed, and data consistency under multiple data centers is finally achieved; if not, only the data to be synchronized is stored, and at this time, the target data storage center does not synchronize the data to be synchronized to other data storage centers any more, so that the situation that the target data storage center which has received the data to be synchronized repeatedly receives the data to be synchronized is avoided. In addition, in the application, any data storage center under multiple data centers supports simultaneous reading and writing functions.
The data processing method is applied to a target data storage center and used for determining the data to be synchronized acquired by the data processing method; acquiring a first data identifier of data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized; judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if the first data identifier is not the identifier corresponding to the target data storage center, storing the data to be synchronized. In the application, after the target data storage center acquires the data to be synchronized, a first data identifier of the data to be synchronized is acquired, wherein the first data identifier is an equipment identifier for operating the data to be synchronized; judging whether the first data identification is an identification corresponding to a target data storage center, namely whether the first data identification is obtained by self modification, if so, storing data to be synchronized, and synchronizing the data to be synchronized to other data storage centers, so that the other data storage centers can store the data to be synchronized, thereby realizing the data consistency of multiple data centers, and if not, storing the data to be synchronized, namely if the data to be synchronized is not modified by the target data storage center, the target data storage center only stores the data to be synchronized, and the data to be synchronized cannot be synchronized to other data storage centers, thereby avoiding the repeated transmission of the data to be synchronized among the data storage centers; in addition, the target data storage center synchronizes the data to be synchronized to other data storage centers, so that the data can be read and written to a plurality of data storage centers at one time, and the data synchronization efficiency is improved.
In the data synchronization method provided in the embodiment of the present application, the process of synchronizing the data to be synchronized to other data storage centers may specifically be: and sending the data to be synchronized to other data storage centers based on a data synchronization mechanism.
For convenience of understanding, taking a data storage center as a machine room as an example, a data synchronization mechanism is described with reference to fig. 2, and assuming that a target data storage center is the machine room a in fig. 2 and other data storage centers are the machine rooms B in fig. 2, based on the data synchronization mechanism, a process of sending data to be synchronized to the other data storage centers may be: after monitoring that the data to be synchronized is generated by the machine room A, the machine room A directly sends the data to be synchronized to the machine room B.
In a data synchronization method provided in an embodiment of the present application, in a multiple data center, in order to facilitate data management, data to be synchronized may be deployed in a cross deployment manner, and data is synchronized by using a master-slave replication mechanism, which is still taken as an example, and the master-slave replication is described with reference to fig. 3, assuming that 2000 data are present, the first 1000 data are stored in a machine room a, and the second 1000 data are stored in a machine room B, in order to achieve data consistency, a backup of the last 1000 data needs to be created in the machine room a, and a backup of the first 1000 data is created in the machine room B, at this time, the machine room a and the machine room B form a master-slave replication relationship, and in the multiple data center, a process of synchronizing the data to be synchronized to other data storage centers may specifically be: and sending the data to be synchronized to other data storage centers based on a master-slave copying mechanism.
Referring to fig. 4, fig. 4 is a second flowchart of a data synchronization method according to an embodiment of the present application.
The data synchronization method provided by the embodiment of the application is applied to a target data storage center of multiple data centers, and comprises the following steps:
step S201: and determining the data to be synchronized acquired by the user.
Step S202: and acquiring a first data identifier of the data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized.
Step S203: judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, executing step S204; if the first data identifier is not the identifier corresponding to the target data storage center, step S205 is executed.
Step S204: and storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized.
Step S205: and storing the data to be synchronized.
Step S206: a data processing request is received.
Step S207: a responsive data storage center is determined responsive to the data processing request.
Step S208: and sending the data processing request to the response data storage center so that the response data storage center responds to the data processing request.
In practical application, under multiple data centers, stored data may need to be operated, however, since the generation time of data to be synchronized is not controllable, when a data processing request is responded, the data storage center does not necessarily have the data to be synchronized, so that a data processing request response fails, or a data storage center receiving a data request fails, so that a data request response also fails, in order to ensure that the data storage center can normally respond to the data processing request, after the data processing request is received, a response data storage center responding to the data processing request may be determined, and then the data processing request is sent to the response data storage center, so that the response data storage center can successfully respond to the data processing request.
In practical application, as the data to be synchronized flows under multiple data centers, the time attributes of the data to be synchronized processed by each data storage center are different, so that the generation sequence of the data to be synchronized processed by each data storage center can be determined by means of the time attributes of each data storage center, and the like, that is, the time stamps can be added to the data to be synchronized, the data to be synchronized is managed by means of the time stamps and the first data identifiers, and for example, the source of the data to be synchronized is determined based on the time stamps and the first data identifiers of the data to be synchronized in each data storage center. In addition, the data consistency under multiple data centers can be ensured by means of the timestamps, for example, corresponding timestamps are also added to the data processing requests; after receiving the data processing request, judging whether the difference value between the timestamp of the stored data and the timestamp of the data processing request is greater than a preset difference value, if so, pulling and storing the data from other data storage centers; that is, when the difference between the timestamp of the data stored in the data storage center and the timestamp of the data processing request is large, the data in the data storage center is considered to be the data that is not the most to be synchronized, and at this time, the data needs to be pulled from other data storage centers and stored, so that the data of the data storage center is updated, and the data consistency between the data storage center and the other data storage centers is realized.
In the data synchronization method provided in the embodiment of the present application, the data to be synchronized and the data processing request may be allocated according to the same principle, so that the data to be synchronized and the data processing request may be preferentially allocated to the same data storage center, for example, a data storage center to which the data to be synchronized and the data processing request belong may be determined based on a fragmentation principle, that is, a process of determining a data storage center to which the data processing request is responded may specifically be: based on the sharding principle, a response data storage center is determined.
In practical application, the data processing request may be fragmented according to the device identifier initiating the data processing request, and the process of determining to respond to the data storage center based on the fragmentation principle may specifically be: acquiring a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request; and determining the response data storage center based on the fragmentation principle, the second data identification and the number of the data storage centers.
In practical application, the process of determining the response data storage center based on the fragmentation principle, the second data identifier and the number of the data storage centers may specifically be: if the second data identification is of a digital type, acquiring the remainder of the second data identification and the number of the data storage centers; and determining the data storage center corresponding to the remainder as the response data storage center. That is, when the second data identifier is of a digital type, a remainder between the second data identifier and the number of data storage centers may be directly obtained, and the data storage center corresponding to the remainder may be determined as a response data storage center or the like, specifically, the remainder of the number of the second data identifier and the data storage center may be calculated by the formula a ═ b mod c, where b denotes the second data identifier, c denotes the number of data storage centers, a denotes the calculated remainder, and the data storage center having a sequence number equal to the remainder may be determined as a response data storage center or the like. For convenience of understanding, it is assumed that each data storage center is composed of one server, and of course, in practical applications, each data storage center may also be composed of a plurality of servers, and the like, and assuming that the number of the data storage centers is 3, and assuming that the device identifiers of the 3 data storage centers are 1, 2, and 3, respectively, when the second data identifier is 1, the remainder of the obtained second data identifier and the number of the data storage centers is 1, and at this time, the data storage center with the device identifier of 1 may be used as the response data storage center; when the second data identifier is 2, the remainder of the obtained second data identifier and the number of the data storage centers is 2, and at this time, the data storage center with the device identifier of 2 may be used as the response data storage center; when the second data identifier is 3, the remainder of the number of the obtained second data identifier and the data storage center is 0, at this time, the data storage center with the device identifier of 3 may be used as the response data storage center, and of course, other ways of corresponding the remainder to the data storage center may also be available.
In practical application, the process of determining the response data storage center based on the fragmentation principle, the second data identifier and the number of the data storage centers may specifically be: if the second data identifier is of a character type, converting the second data identifier into a corresponding ASCII value, and acquiring an accumulated value of the ASCII value; calculating the remainder of the accumulated value and the number of the data storage centers; and determining the data storage center corresponding to the remainder as the response data storage center. That is, when the second data identifier is a character type, the second data identifier may be converted into a corresponding ASCII value, an accumulated value of the ASCII value is obtained, a remainder of the accumulated value and the number of the data storage centers is calculated, and finally the data storage center corresponding to the remainder is determined as a response data storage center or the like. For convenience of understanding, assuming that each data storage center is composed of one server, assuming that the number of the data storage centers is 3, and assuming that the device identifications of the 3 data storage centers are A, B, C respectively, when the second data identification is a, the accumulated value of the ASCII value corresponding to the second data identification is 65, and the remainder of the obtained accumulated value and the number of the data storage centers is 2, at this time, the data storage center with the device identification B may be used as a response data storage center; when the second data identifier is B, the accumulated value of the ASCII value corresponding to the second data identifier is 66, and the remainder of the number of the acquired accumulated value and the number of the data storage centers is 0, at this time, the data storage center with the device identifier C may be used as the response data storage center; when the second data identifier is C, the accumulated value of the ASCII value corresponding to the second data identifier is 67, and the remainder of the obtained accumulated value and the number of data storage centers is 1, at this time, the data storage center with device identifier a may be used as the response data storage center; of course, there may be other ways of associating the remainder with the data storage center, which is not specifically limited herein, for example, when the remainder is 1, the data storage center with the device identifier C may be used as the response data storage center, when the remainder is 2, the data storage center with the device identifier a may be used as the response data storage center, and when the remainder is 0, the data storage center with the device identifier B may be used as the response data storage center.
It should be noted that, when the data to be synchronized is fragmented, the data to be synchronized may be fragmented based on the data identifier of the data to be synchronized according to the fragmentation principle of the data processing request.
Referring to fig. 5, fig. 5 is a flowchart illustrating a data synchronization method provided in the present application.
To facilitate understanding of the data synchronization method provided in the present application, a process in which a user terminal controls multiple data centers through a network terminal is described, and the process may include the following steps:
the multi-data center acquires data to be synchronized;
the method comprises the steps that a multi-data center obtains a first data identification of data to be synchronized, and first equipment is represented as an equipment identification for operating the data to be synchronized;
the multi-data center judges whether the first data identification is the own equipment identification; if so, storing the data to be synchronized, and synchronizing the synchronized data to other data storage centers so that the other data storage centers store new data to be synchronized; if not, storing the data to be synchronized;
a user terminal generates a data processing request under user operation;
the user terminal transmits a data processing request to the multi-data center through the network terminal;
the multi-data center determines a response data storage center responding to the data processing request;
the multi-data center sends the data processing request to the response data storage center so that the response data storage center responds to the data processing request;
and the multi-data center transmits the response result of the data processing request to the user terminal through the network terminal.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a data synchronization system according to an embodiment of the present disclosure.
The data synchronization system provided by the embodiment of the application is applied to a target data storage center, and may include:
the first determining module 101 is configured to determine data to be synchronized acquired by the first determining module;
a first obtaining module 102, configured to obtain a first data identifier of data to be synchronized, where the first data identifier is an apparatus identifier for operating the data to be synchronized;
the first judging module 103 is configured to judge whether the first data identifier is an identifier corresponding to a target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if the first data identifier is not the identifier corresponding to the target data storage center, storing the data to be synchronized.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the first determining module may include:
and the first sending unit is used for sending the data to be synchronized to other data storage centers based on the data synchronization mechanism.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the first determining module may include:
and the second sending unit is used for sending the data to be synchronized to other data storage centers based on a master-slave copying mechanism.
The data synchronization system provided by the embodiment of the application is applied to a target data storage center, and may further include:
the first receiving module is used for receiving a data processing request;
a second determining module for determining a response data storage center responding to the data processing request;
and the first sending module is used for sending the data processing request to the response data storage center so as to enable the response data storage center to respond to the data processing request.
The data synchronization system provided by the embodiment of the application is applied to a target data storage center, and a data processing request can carry a timestamp;
the method can also comprise the following steps:
and the second judgment module is used for judging whether the difference value between the timestamp of the self stored data and the timestamp of the data processing request is greater than the preset difference value or not after the first receiving module receives the data processing request, and if so, pulling and storing the data from other data storage centers.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the second determining module may include:
and the first determining submodule is used for determining the response data storage center based on the slicing principle.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the first determining submodule may include:
the second obtaining unit is used for obtaining a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request;
and the second determining submodule is used for determining the response data storage center based on the fragmentation principle, the second data identifier and the number of the data storage centers.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the second determining submodule may include:
the first calculating unit is used for acquiring the remainder of the number of the second data identification and the data storage center if the second data identification is of a digital type;
and the first determining unit is used for determining the data storage center corresponding to the remainder as a response data storage center.
The data synchronization system provided in the embodiment of the present application is applied to a target data storage center, and the second determining sub-module may include:
a first conversion unit, configured to convert the second data identifier into a corresponding ASCII value if the second data identifier is of a character type, and obtain an accumulated value of the ASCII values;
the second calculating unit is used for acquiring the remainder of the accumulated value and the number of the data storage centers;
and the second determining unit is used for determining the data storage center corresponding to the remainder as a response data storage center.
The application also provides data synchronization equipment and a computer readable storage medium, which have corresponding effects of the data synchronization method provided by the embodiment of the application. Referring to fig. 7, fig. 7 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present disclosure.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program:
determining self-acquired data to be synchronized;
acquiring a first data identifier of data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized;
judging whether the first data identification is an identification corresponding to a target data storage center or not;
if the first data identification is the identification corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized;
and if the first data identifier is not the identifier corresponding to the target data storage center, storing the data to be synchronized.
The data synchronization device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: and sending the data to be synchronized to other data storage centers based on a data synchronization mechanism.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: and sending the data to be synchronized to other data storage centers based on a master-slave copying mechanism.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: receiving a data processing request; determining a response data storage center responding to the data processing request; and sending the data processing request to the response data storage center so that the response data storage center responds to the data processing request.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: the data processing request carries a timestamp; after the data processing request is received, whether the difference value between the timestamp of the stored data and the timestamp of the data processing request is larger than a preset difference value or not is judged, and if yes, the data are pulled from other data storage centers and stored.
The data synchronization device provided in the embodiment of the present application includes a memory 201 and a processor 202, where the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: based on the sharding principle, a responsive data storage center is determined.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: acquiring a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request; and determining the response data storage center based on the fragmentation principle, the second data identification and the number of the data storage centers.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: if the second data identification is of a digital type, acquiring a remainder of the second data identification and the number of the data storage centers; and determining the data storage center corresponding to the remainder as a response data storage center.
The data synchronization device provided by the embodiment of the application comprises a memory 201 and a processor 202, wherein a computer program is stored in the memory 201, and the processor 202 implements the following steps when executing the computer program: if the second data identifier is character type, converting the second data identifier into corresponding ASCII value, and acquiring an accumulated value of the ASCII value; obtaining the remainder of the accumulated value and the number of the data storage centers; and determining the data storage center corresponding to the remainder as a response data storage center.
Referring to fig. 8, another data synchronization device provided in the embodiment of the present application may further include: an input port 203 connected to the processor 202, for transmitting an externally input command to the processor 202; a display unit 204 connected to the processor 202, for displaying the processing result of the processor 202 to the outside; and the communication module 205 is connected with the processor 202 and is used for realizing the communication between the data synchronization device and the outside world. The display unit 204 may be a display panel, a laser scanning display, or the like; the communication method adopted by the communication module 205 includes, but is not limited to, mobile high definition link technology (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), and wireless connection: wireless fidelity technology (WiFi), bluetooth communication technology, bluetooth low energy communication technology, ieee802.11s based communication technology.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps:
determining self-acquired data to be synchronized;
acquiring a first data identifier of data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized;
judging whether the first data identification is an identification corresponding to a target data storage center or not;
if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized;
and if the first data identifier is not the identifier corresponding to the target data storage center, storing the data to be synchronized.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: and sending the data to be synchronized to other data storage centers based on a data synchronization mechanism.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: and sending the data to be synchronized to other data storage centers based on a master-slave copying mechanism.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: receiving a data processing request; determining a response data storage center responding to the data processing request; and sending the data processing request to the response data storage center so that the response data storage center responds to the data processing request.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: the data processing request carries a timestamp; after the data processing request is received, whether the difference value between the timestamp of the stored data and the timestamp of the data processing request is larger than a preset difference value or not is judged, and if yes, the data are pulled from other data storage centers and stored.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: based on the sharding principle, a responsive data storage center is determined.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: acquiring a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request; and determining the response data storage center based on the fragmentation principle, the second data identification and the number of the data storage centers.
A computer-readable storage medium provided in an embodiment of the present application stores a computer program, and when executed by a processor, the computer program implements the following steps: if the second data identification is of a digital type, acquiring a remainder of the second data identification and the number of the data storage centers; and determining the data storage center corresponding to the remainder as the response data storage center.
A computer-readable storage medium is provided in an embodiment of the present application, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps: if the second data identifier is of a character type, converting the second data identifier into a corresponding ASCII value, and acquiring an accumulated value of the ASCII value; obtaining the remainder of the accumulated value and the number of the data storage centers; and determining the data storage center corresponding to the remainder as the response data storage center.
The computer-readable storage media to which this application relates include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage media known in the art.
For a description of a relevant part in the data synchronization system, the data synchronization device, and the computer-readable storage medium provided in the embodiments of the present application, reference is made to detailed descriptions of a corresponding part in the data synchronization method provided in the embodiments of the present application, and details are not repeated here. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (12)
1. A data synchronization method is applied to a target data storage center and comprises the following steps:
determining self-acquired data to be synchronized;
acquiring a first data identifier of the data to be synchronized, wherein the first data identifier is an equipment identifier for operating the data to be synchronized;
judging whether the first data identifier is an identifier corresponding to the target data storage center;
if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized;
and if the first data identification is not the identification corresponding to the target data storage center, storing the data to be synchronized.
2. The method of claim 1, wherein synchronizing the data to be synchronized to other data storage centers comprises:
and sending the data to be synchronized to the other data storage centers based on a data synchronization mechanism.
3. The method of claim 1, wherein synchronizing the data to be synchronized to other data storage centers comprises:
and sending the data to be synchronized to the other data storage centers based on a master-slave copying mechanism.
4. The method of claim 1, further comprising:
receiving a data processing request;
determining a response data storage center responding to the data processing request;
and sending the data processing request to the response data storage center so that the response data storage center responds to the data processing request.
5. The method of claim 4, wherein the data processing request carries a timestamp;
after receiving the data processing request, the method further includes:
and judging whether the difference value between the timestamp of the self stored data and the timestamp of the data processing request is greater than a preset difference value, if so, pulling and storing the data from the other data storage centers.
6. The method of claim 4, wherein determining a responsive data storage center responsive to the data processing request comprises:
and determining the response data storage center based on the slicing principle.
7. The method of claim 6, wherein determining the responsive data storage center based on sharding comprises:
acquiring a second data identifier of the data processing request, wherein the second data identifier is an equipment identifier for initiating the data processing request;
and determining the response data storage center based on the fragmentation principle, the second data identification and the number of the data storage centers.
8. The method of claim 7, wherein determining the responsive data storage center based on the sharding principle, the second data identity, and a number of data storage centers comprises:
if the second data identification is of a digital type, acquiring a remainder of the second data identification and the number of the data storage centers;
and determining the data storage center corresponding to the remainder as the response data storage center.
9. The method of claim 7, wherein determining the responsive data storage center based on the sharding principle, the second data identity, and a number of data storage centers comprises:
if the second data identifier is a character type, converting the second data identifier into a corresponding ASCII value, and acquiring an accumulated value of the ASCII value;
obtaining the remainder of the accumulated value and the number of the data storage centers;
and determining the data storage center corresponding to the remainder as the response data storage center.
10. A data synchronization system, applied to a target data storage center, includes:
the first determining module is used for determining the data to be synchronized acquired by the first determining module;
a first obtaining module, configured to obtain a first data identifier of the data to be synchronized, where the first data identifier is an equipment identifier for operating the data to be synchronized;
the first judging module is used for judging whether the first data identifier is an identifier corresponding to the target data storage center; if the first data identifier is the identifier corresponding to the target data storage center, storing the data to be synchronized, and synchronizing the data to be synchronized to other data storage centers so that the other data storage centers store the data to be synchronized; and if the first data identification is not the identification corresponding to the target data storage center, storing the data to be synchronized.
11. A data synchronization apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data synchronization method according to any one of claims 1 to 9 when executing said computer program.
12. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a data synchronization method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010337596.XA CN113556371B (en) | 2020-04-26 | 2020-04-26 | Data synchronization method, system, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010337596.XA CN113556371B (en) | 2020-04-26 | 2020-04-26 | Data synchronization method, system, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113556371A CN113556371A (en) | 2021-10-26 |
| CN113556371B true CN113556371B (en) | 2022-09-30 |
Family
ID=78129838
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010337596.XA Active CN113556371B (en) | 2020-04-26 | 2020-04-26 | Data synchronization method, system, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113556371B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101500208A (en) * | 2008-01-31 | 2009-08-05 | 三星电子株式会社 | Data synchronization method and system between devices |
| CN102063494A (en) * | 2010-12-31 | 2011-05-18 | 青岛海信网络科技股份有限公司 | Database synchronizing method and device |
| CN104639613A (en) * | 2015-01-06 | 2015-05-20 | 中国农业大学 | Realizing method for mobile data synchronization middleware based on improved network protocol |
| CN105306585A (en) * | 2015-11-12 | 2016-02-03 | 焦点科技股份有限公司 | Data synchronization method for plurality of data centers |
| CN105701116A (en) * | 2014-11-27 | 2016-06-22 | 英业达科技有限公司 | Data synchronization system |
| CN106487884A (en) * | 2016-09-29 | 2017-03-08 | 乐视控股（北京）有限公司 | Data transmission method and client |
| CN106980625A (en) * | 2016-01-18 | 2017-07-25 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization, apparatus and system |
| CN108319619A (en) * | 2017-01-18 | 2018-07-24 | 腾讯科技（北京）有限公司 | Data processing method and device |
| CN110198335A (en) * | 2018-08-30 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Message synchronization method and device, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10129364B2 (en) * | 2015-09-30 | 2018-11-13 | International Business Machines Corporation | Data consistency maintenance for sequential requests |
-
2020
- 2020-04-26 CN CN202010337596.XA patent/CN113556371B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101500208A (en) * | 2008-01-31 | 2009-08-05 | 三星电子株式会社 | Data synchronization method and system between devices |
| CN102063494A (en) * | 2010-12-31 | 2011-05-18 | 青岛海信网络科技股份有限公司 | Database synchronizing method and device |
| CN105701116A (en) * | 2014-11-27 | 2016-06-22 | 英业达科技有限公司 | Data synchronization system |
| CN104639613A (en) * | 2015-01-06 | 2015-05-20 | 中国农业大学 | Realizing method for mobile data synchronization middleware based on improved network protocol |
| CN105306585A (en) * | 2015-11-12 | 2016-02-03 | 焦点科技股份有限公司 | Data synchronization method for plurality of data centers |
| CN106980625A (en) * | 2016-01-18 | 2017-07-25 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization, apparatus and system |
| CN106487884A (en) * | 2016-09-29 | 2017-03-08 | 乐视控股（北京）有限公司 | Data transmission method and client |
| CN108319619A (en) * | 2017-01-18 | 2018-07-24 | 腾讯科技（北京）有限公司 | Data processing method and device |
| CN110198335A (en) * | 2018-08-30 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Message synchronization method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113556371A (en) | 2021-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107508795B (en) | Cross-container cluster access processing device and method | |
| CN111399764B (en) | Data storage method, data reading device, data storage equipment and data storage medium | |
| KR20080068110A (en) | A method for processing data synchronization and client terminal, server and data synchronization system thereof | |
| CN105338078A (en) | Data storage method and device used for storing system | |
| CN103312544A (en) | Method, equipment and system for controlling terminals during log file reporting | |
| CN102752294B (en) | Method and system for synchronizing data of multiple terminals on basis of equipment capacity | |
| CN108900572B (en) | Main database domain name switching control method, computer equipment and storage medium | |
| US20160241635A1 (en) | M2m data querying and invoking methods, querying and invoking devices, and system | |
| CN113051110A (en) | Cluster switching method, device and equipment | |
| CN112632029B (en) | Data management method, device and equipment of distributed storage system | |
| CN113835369A (en) | Edge terminal scene management method, device and system | |
| US20160308871A1 (en) | Network element data access method and apparatus, and network management system | |
| US11128622B2 (en) | Method for processing data request and system therefor, access device, and storage device | |
| CN113556371B (en) | Data synchronization method, system, equipment and computer readable storage medium | |
| CN113766027A (en) | Method and equipment for forwarding data by flow replication cluster node | |
| CN107623705B (en) | Storage mode upgrading method, device and system based on video cloud storage system | |
| CN109257403B (en) | Data storage method and device and distributed storage system | |
| CN110018875B (en) | Desktop synchronization implementation method based on IDV | |
| CN111367761A (en) | Information management method, system and related components of universal server | |
| CN112685486B (en) | Data management method and device for database cluster, electronic equipment and storage medium | |
| CN115629909A (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN112698926B (en) | Data processing method, device, equipment, storage medium and system | |
| CN111752245B (en) | Data access and processing method, device, equipment and readable storage medium | |
| CN114625566A (en) | Data disaster tolerance method and device, electronic equipment and storage medium | |
| CN107332679B (en) | Centerless information synchronization method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |