CN113779153A - Data synchronization method and device, electronic equipment and storage medium - Google Patents
Data synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113779153A CN113779153A CN202111080658.4A CN202111080658A CN113779153A CN 113779153 A CN113779153 A CN 113779153A CN 202111080658 A CN202111080658 A CN 202111080658A CN 113779153 A CN113779153 A CN 113779153A
- Authority
- CN
- China
- Prior art keywords
- data
- synchronization
- last
- identification information
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000001360 synchronised effect Effects 0.000 claims abstract description 120
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioethics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data synchronization method, a data synchronization device, an electronic device and a storage medium, wherein the method comprises the following steps: receiving a data synchronization request sent by a client; inquiring data table identification information and data position information corresponding to the identity from the data synchronization configuration table; comparing the last synchronization time with the last synchronization time corresponding to the data table identification information, and determining a data acquisition mode of the data table to be synchronized; acquiring data to be synchronized based on a data acquisition mode and data position information; and encapsulating all the acquired data to be synchronized, and synchronizing the encapsulated data to the client. By adopting the data synchronization method, the data synchronization device, the electronic equipment and the storage medium, the problem that different synchronous data cannot be provided for different clients during data synchronization is solved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data synchronization method and apparatus, an electronic device, and a storage medium.
Background
The block chain is a distributed shared account book and a database, and has the characteristics of decentralization, no tampering, trace retaining in the whole process, traceability, collective maintenance, openness and transparency and the like. With the continuous development of the technology of the blockchain alliance, most blockchain Baas platforms are constructed and managed operation and maintenance platforms based on a centralized application and service resource mode.
In the prior art, the data synchronization technology on the blockchain platform is usually synchronization from a server to a client, and each synchronization synchronizes all data or modified data on the server to all clients. However, when synchronizing data to a plurality of clients, not every client needs to synchronize all the data on the server, but only needs to keep synchronization of part of the data.
Disclosure of Invention
In view of this, the present application provides a data synchronization method, apparatus, electronic device and storage medium, and aims to implement a fine-grained controllable data synchronization manner by setting different synchronization conditions for different clients.
In a first aspect, an embodiment of the present application provides a data synchronization method, including:
receiving a data synchronization request sent by a client, wherein the data synchronization request comprises an identity of the client and last synchronization time;
inquiring data table identification information and data position information corresponding to the identity from a data synchronization configuration table, wherein the data synchronization configuration table stores the corresponding relation of the identity, the data table identification information and the data position information;
comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine a data acquisition mode;
acquiring data to be synchronized based on a data acquisition mode and data position information;
and encapsulating all the acquired data to be synchronized, and synchronizing the encapsulated data to the client.
Optionally, comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine the data obtaining manner may include: inquiring whether the last synchronization time corresponding to the data table identification information exists in a data synchronization record table, wherein the last synchronization time corresponding to the data table identification information is stored in the data synchronization record table; if the last synchronization time corresponding to the data table identification information exists, determining that the data acquisition mode is incremental acquisition; and if the last synchronization time corresponding to the data table identification information does not exist, determining that the data acquisition mode is full acquisition.
Optionally, the data synchronization method may further include: judging whether synchronous data loss exists or not based on the last synchronous time; and if the synchronous data are lost, inquiring a historical synchronous record which corresponds to the identification information of the data table and has the data synchronous time between the last synchronous time and the last synchronous time in a data synchronous historical table, and sending the inquired historical synchronous record to the client, wherein the data synchronous historical table is used for storing the historical record of each time of synchronization, and the data synchronous lost record table is used for storing the historical record corresponding to the lost synchronous data.
Optionally, the determining whether there is synchronization data loss based on the last synchronization time may include: judging whether the last synchronization time is earlier than the last synchronization time; if the last synchronization time is earlier than the last synchronization time, determining that the synchronization data is lost; if the last synchronization time is not earlier than the last synchronization time, it is determined that there is no loss of synchronization data.
Optionally, acquiring the data to be synchronized based on the data acquisition mode and the data location information may include: based on the data acquisition mode, acquiring data to be synchronized corresponding to the data position information and meeting synchronization conditions from a database in which the data to be synchronized is stored, wherein a data table corresponding to the data table identification information is stored in the database, the data to be synchronized corresponding to the data position information is stored in the data table, and the synchronization conditions are stored in a data synchronization configuration table and used for determining the data to be synchronized meeting the conditions.
Optionally, after receiving the data synchronization request sent by the client, the method may further include: matching the identity with a preset identity white list, and if the matching is successful, determining that the identity of the client is valid; and if the matching is unsuccessful, rejecting the data synchronization request sent by the client.
In a second aspect, an embodiment of the present application provides a data synchronization apparatus, including:
the receiving module is used for receiving a data synchronization request sent by a client, wherein the data synchronization request comprises an identity of the client and last synchronization time;
the query module is used for querying data table identification information and data position information corresponding to the identity identifier from a data synchronization configuration table, and the data synchronization configuration table stores the corresponding relation among the identity identifier, the data table identification information and the data position information;
the determining module is used for comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine a data acquisition mode;
the acquisition module is used for acquiring data to be synchronized based on a data acquisition mode and data position information;
and the synchronization module is used for encapsulating all the acquired data to be synchronized and synchronizing the encapsulated data to the client.
Optionally, the determining module may be further configured to: inquiring whether the last synchronization time corresponding to the data table identification information exists in a data synchronization record table, wherein the last synchronization time corresponding to the data table identification information is stored in the data synchronization record table; if the last synchronization time corresponding to the data table identification information exists, determining that the data acquisition mode is incremental acquisition; and if the last synchronization time corresponding to the data table identification information does not exist, determining that the data acquisition mode is full acquisition.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the data synchronization method as described above.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the data synchronization method.
The embodiment of the application brings the following beneficial effects:
the embodiment of the application provides a data synchronization method, a data synchronization device, an electronic device and a storage medium, and the method comprises the following steps: receiving a data synchronization request sent by a client, wherein the data synchronization request comprises an identity of the client and last synchronization time; inquiring data table identification information and data position information corresponding to the identity from a data synchronization configuration table, wherein the data synchronization configuration table stores the corresponding relation of the identity, the data table identification information and the data position information; comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine a data acquisition mode; acquiring data to be synchronized based on a data acquisition mode and data position information; the method and the device have the advantages that all the obtained data to be synchronized are packaged, the packaged data are synchronized to the client, and the problems that different synchronous data cannot be provided for different clients during data synchronization, and synchronization conditions and synchronization fields cannot be controlled are solved.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments are briefly described below, and it is obvious that the following drawings are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other related drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data synchronization method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the prior art, data synchronization is usually performed from a server to a client, and all data on the server or part of modified data is synchronized to all clients each time synchronization is performed. However, when data is synchronized among a plurality of clients, not all the data on the server need to be synchronized for each client, and some clients need only keep synchronization of partial data. Meanwhile, when the client is in an intranet environment (a secret-related network or a high-security level network environment) and the server is in an extranet environment, the server cannot acquire synchronous data from the client, and cannot perform data synchronization in a one-way reachable network environment,
based on this, embodiments of the present application provide a data synchronization method, apparatus, electronic device, and storage medium, where a client actively sends a data synchronization request to a server, and sets different synchronization conditions and synchronization fields for different clients, so as to solve the problem that different synchronization data cannot be provided for different clients during data synchronization, and the synchronization conditions and the synchronization fields are not controllable, and also solve the problem that data synchronization cannot be performed in a one-way reachable network environment.
To facilitate understanding of the present embodiment, first, a data synchronization method disclosed in the embodiments of the present application is described in detail, and fig. 1 is a schematic flow chart of the data synchronization method provided in the embodiments of the present application, as shown in fig. 1, the method includes the following steps:
Specifically, a plurality of data synchronization tables need to be created, the data synchronization tables can be created at both the server and the client, and the purpose of creating the data synchronization tables at the client is as follows: the method can not only realize that the client side initiates a data synchronization request for synchronizing data from the server to the client side, but also realize that the client side initiates a data synchronization request for synchronizing data from the client side to the server, which is equivalent to the client side realizing the function of the server, so that the indirect data synchronization between the client side and the client side can be realized. These data synchronization tables include: the data synchronization device comprises a data synchronization configuration table, a data synchronization record table, a data synchronization history table and a data synchronization loss record table. Fields in the data synchronization configuration table include, but are not limited to, a table name, a field name, a primary key, synchronization conditions and an identity, and the table name field is used for configuring the table name of a data table to be synchronized where data to be synchronized is located; the field name field is used for configuring the field name of the field of the data to be synchronized in the data table to be synchronized; the main key field is used for configuring main key data corresponding to a row where the data to be synchronized is located in the main key field of the data table to be synchronized, and the data position information of the data to be synchronized can be determined according to the field name and the main key; the identity field is used for configuring the data table to be synchronized in the table name field to be synchronized by the client corresponding to which identity. The data synchronization record table records a timestamp, a synchronization number, and a synchronization identifier corresponding to the last synchronization time of each data table synchronized by each client, and also records the content of each data table corresponding to the data synchronization configuration table, for example: the synchronization field, the synchronization condition, the primary key and the like, the synchronization number is used for recording how many pieces of data are synchronized in the data table, and the synchronization identifier is used for recording whether the data table is successfully synchronized. Here, the data synchronization table in which the content corresponding to each data table in the data synchronization arrangement table is recorded is referred to as a full table. The history record of each synchronization of each client is recorded in the data synchronization history table, and the data synchronization history table is also a full table. The data synchronization loss record table records the history corresponding to the lost synchronization data, and is also a full table.
Based on the created data synchronization table, the server may perform data synchronization to the client that sent the data synchronization request. The information in the data synchronization request includes, but is not limited to: the identity of the client, the last synchronization time, the data type to be synchronized and the equipment information. Here, the identity is a unique identifier corresponding to each client, for example: and (4) APPID. The last synchronization time refers to a timestamp generated by the client when the client performs data synchronization with the server last time, and a special table is used for recording the time of each data synchronization at the client.
In one example, after a data synchronization request sent by a client is received, matching an identity with a preset identity white list, and if the matching is successful, determining that the identity of the client is valid; and if the matching is unsuccessful, rejecting the data synchronization request sent by the client. It can be understood that after receiving the data synchronization request sent by the client, the validity of the client identity needs to be checked to ensure that the data synchronization request is acceptable.
And 102, inquiring data table identification information and data position information corresponding to the identity from the data synchronization configuration table. The data synchronization configuration table stores the corresponding relation of the identity, the data table identification information and the data position information.
Specifically, after determining that the client identity is valid, it is necessary to determine which data to synchronize to the client. Firstly, the data synchronization configuration table includes a table name field, a field name field, a primary key field, and an identity field, that is, the data synchronization configuration table stores a corresponding relationship between an identity, data table identification information, and data location information. Therefore, the data table identification information and the data position information corresponding to the identification can be inquired from the data configuration table.
Taking the above example as an example, the data table identification information is the table name of the data table to be synchronized, and the data location information can be determined by the field name and the content configured in the primary key field.
And 103, comparing the last synchronization time with the last synchronization time corresponding to the data table identification information, and determining a data acquisition mode.
Specifically, whether the last synchronization time corresponding to the data table identification information exists is inquired in a data synchronization record table, and the last synchronization time corresponding to the data table identification information is stored in the data synchronization record table; if the last synchronization time corresponding to the data table identification information exists, determining that the data acquisition mode is incremental acquisition; and if the last synchronization time corresponding to the data table identification information does not exist, determining that the data acquisition mode is full acquisition.
In an example, if the data table to be synchronized corresponding to the data table identification information is multiple data tables, sequentially inquiring whether each data table to be synchronized has the last synchronization time corresponding to the data table identification information in the data synchronization record table according to the data table identification information, and determining a data acquisition mode corresponding to the data table to be synchronized for each data table to be synchronized. And if the last synchronization time corresponding to the data table identification information does not exist, the data acquisition mode is determined to be full acquisition, and if the last synchronization time corresponding to the data table identification information exists, the data acquisition mode is determined to be incremental acquisition, and if the last synchronization time corresponding to the data table identification information does not exist, the data acquisition mode is determined to be incremental acquisition.
In the data synchronization process, a situation of synchronous data loss may occur due to reasons such as network instability, power failure or abnormal downtime, and therefore, it is necessary to determine whether synchronous data loss exists in a time interval from last synchronous time to last synchronous time, and if synchronous data loss exists, a specific synchronous record with the synchronous data loss may be sent to a client, so that the client can accurately locate the reason of synchronous data loss or reacquire data to be synchronized corresponding to the synchronous data loss time point. The above process can be realized by the following steps: judging whether synchronous data loss exists or not based on the last synchronous time; and if the synchronous data are lost, inquiring a historical synchronous record which corresponds to the identification information of the data table and has the data synchronous time between the last synchronous time and the last synchronous time in the data synchronous historical table, and sending the inquired historical synchronous record to the client. Here, the data synchronization history table is used to store a history of each synchronization, and the data synchronization loss record table is used to store a history corresponding to the lost synchronization data.
Wherein, whether the synchronous data loss exists can be judged through the following procedures: judging whether the last synchronization time is earlier than the last synchronization time; if the last synchronization time is earlier than the last synchronization time, determining that the synchronization data is lost; if the last synchronization time is not earlier than the last synchronization time, it is determined that there is no loss of synchronization data.
Taking the above example as an example, if the last synchronization time recorded by the client is earlier than the last synchronization time recorded in the server data synchronization record table, it indicates that the last synchronization time recorded by the client is inconsistent with the last synchronization time recorded by the server, because one or several data synchronizations within the period between the last synchronization time and the last synchronization time are not normally synchronized by the client, and there is a case that the synchronization data is lost. On the contrary, if the last synchronization time recorded by the client is not earlier than the last synchronization time recorded in the server data synchronization recording table, it indicates that the last synchronization time recorded by the client is consistent with the last synchronization time recorded by the server, and there is no case of synchronization data loss. Here, since the history record of each synchronization of each client is recorded in the data synchronization history table, and the data synchronization history table is a full table, according to the data table identification information, all the history synchronization records corresponding to the data table identification information can be searched from the data synchronization history table, and the history synchronization records with the data synchronization time between the last synchronization time and the last synchronization time are selected, and the selected history synchronization records are sent to the client.
And 104, acquiring the data to be synchronized based on the data acquisition mode and the data position information.
Specifically, the data to be synchronized can be acquired according to the determined data acquisition mode and the data position information. The data to be synchronized can be acquired through the following specific steps: and based on the data acquisition mode, acquiring the data to be synchronized corresponding to the data position information and meeting the synchronization condition from a database in which the data to be synchronized is stored, wherein a data table corresponding to the data table identification information is stored in the database, the data to be synchronized corresponding to the data position information is stored in the data table, and the synchronization condition is stored in the data synchronization configuration table and is used for determining the data to be synchronized meeting the condition.
Taking the above example as an example, according to the data table identification information and the data location information, it can be determined in which data table the data to be synchronized is stored, and the specific field name and the primary key corresponding to the data to be synchronized in the data table, so that the data to be synchronized without being screened by the synchronization condition can be determined, and then, according to the synchronization condition corresponding to the data table identification information configured in the data synchronization configuration table, the data to be synchronized screened by the synchronization condition can be determined. For example: the data to be synchronized which is not screened by the synchronization condition is all data in the telephone number field of the user data table, if the synchronization condition is that the telephone number beginning with 130 can be synchronized to the client, the data of the telephone number beginning with 130 in the telephone number field of the user data table is determined as the data to be synchronized based on the synchronization condition.
Here, the SQL script for data query may be constructed according to the data table identification information, the data location information, and the synchronization condition, and then the server calls and runs the SQL script to obtain the data to be synchronized corresponding to the data obtaining manner from the database. It can be understood that the step of acquiring the data to be synchronized corresponding to the data acquisition mode from the database means that, if the data acquisition mode is full-scale acquisition, the data to be synchronized corresponding to the data table identification information and the data position information and having the data synchronization time before the last synchronization time is acquired from the database. And if the data acquisition mode is incremental acquisition, acquiring the data to be synchronized corresponding to the data table identification information and the data position information and with the data synchronization time within the time from the last synchronization time to the last synchronization time from the database.
It should be noted that, if the data table identification information corresponds to multiple data tables to be synchronized, an SQL script for data query is constructed for each data table to be synchronized, and then the server calls and runs the SQL script to acquire data to be synchronized corresponding to the data acquisition mode from the database.
And 105, encapsulating all the acquired data to be synchronized, and synchronizing the encapsulated data to the client.
Specifically, after the data to be synchronized is obtained, the data to be synchronized needs to be encapsulated, and the encapsulating process includes serialization and asymmetric encryption of the data to be processed. Here, the serialization is a data exchange format, including but not limited to JSON, XML, YAML, and those skilled in the art can select the serialization according to the actual situation, and the present application does not limit the serialization. The asymmetric encryption is realized by an asymmetric encryption algorithm, which includes but is not limited to RSA, Elgamal, knapsack algorithm, Rabin, and D-H, ECC, and those skilled in the art can select the algorithm according to the actual situation, which is not limited in this application. And finally, synchronizing the packaged data to the client, and updating the data synchronization record table and the data synchronization history table. After the client completes data synchronization, the last synchronization time is also updated.
Therefore, the data synchronization method can not only provide different synchronous data for different clients and realize fine-grained controllable data synchronization, but also realize data synchronization in a one-way reachable network environment in a manner that the clients actively send data synchronization requests to the server and the data synchronization scheme is configured.
Fig. 2 is a schematic structural diagram of a data synchronization apparatus according to an embodiment of the present application, and as shown in fig. 2, the apparatus includes the following modules:
a receiving module 201, configured to receive a data synchronization request sent by a client, where the data synchronization request includes an identity of the client and a last synchronization time.
The query module 202 is configured to query data table identification information and data location information corresponding to the identity identifier from a data synchronization configuration table, where the data synchronization configuration table stores a corresponding relationship between the identity identifier, the data table identification information, and the data location information.
The determining module 203 is configured to compare the last synchronization time with the last synchronization time corresponding to the data table identification information, and determine a data obtaining manner.
An obtaining module 204, configured to obtain data to be synchronized based on the data obtaining manner and the data location information.
The synchronization module 205 is configured to encapsulate all acquired data to be synchronized, and synchronize the encapsulated data to the client.
Corresponding to the data synchronization method in fig. 1, an embodiment of the present application further provides a schematic structural diagram of an electronic device 300, as shown in fig. 3, where the electronic device 300 includes a processor 310, a memory 320, and a bus 330. The memory 320 stores machine-readable instructions executable by the processor 310, when the electronic device 300 runs, the processor 310 and the memory 320 communicate through the bus 330, and when the machine-readable instructions are executed by the processor 310, the data synchronization method can be executed, and by means of the manner that a client actively sends a data synchronization request to a server and a data synchronization scheme is configured, different synchronization data can be provided for different clients, fine-grained controllable data synchronization can be achieved, and data synchronization in a one-way reachable network environment can also be achieved.
Corresponding to the data synchronization method in fig. 1, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the steps of the data synchronization method.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, when a computer program on the storage medium is run, the data synchronization method can be executed, and different synchronization data can be provided for different clients, fine-grained controllable data synchronization can be realized, and data synchronization in a one-way reachable network environment can also be realized by actively sending a data synchronization request to a server by a client and configuring a data synchronization scheme.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the foregoing systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of a unit is merely a division of one logic function, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing an electronic device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A data synchronization method is applied to a server and is characterized by comprising the following steps:
receiving a data synchronization request sent by a client, wherein the data synchronization request comprises an identity of the client and last synchronization time;
inquiring data table identification information and data position information corresponding to the identity from a data synchronization configuration table, wherein the data synchronization configuration table stores the corresponding relation of the identity, the data table identification information and the data position information;
comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine a data acquisition mode;
acquiring data to be synchronized based on the data acquisition mode and the data position information;
and encapsulating all the acquired data to be synchronized, and synchronizing the encapsulated data to the client.
2. The data synchronization method of claim 1, wherein the comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine the data acquisition mode comprises:
inquiring whether the last synchronization time corresponding to the data table identification information exists in a data synchronization record table, wherein the last synchronization time corresponding to the data table identification information is stored in the data synchronization record table;
if the last synchronization time corresponding to the data table identification information exists, determining that the data acquisition mode is incremental acquisition;
and if the last synchronization time corresponding to the data table identification information does not exist, determining that the data acquisition mode is full acquisition.
3. The data synchronization method of claim 2, further comprising:
judging whether synchronous data loss exists or not based on the last synchronous time;
and if the synchronous data are lost, inquiring a historical synchronous record which corresponds to the identification information of the data table and has the data synchronous time between the last synchronous time and the last synchronous time in a data synchronous historical table, and sending the inquired historical synchronous record to the client, wherein the data synchronous historical table is used for storing the historical record of each time of synchronization, and the data synchronous lost record table is used for storing the historical record corresponding to the lost synchronous data.
4. The data synchronization method of claim 3, wherein the determining whether there is a loss of synchronization data based on the last synchronization time comprises:
judging whether the last synchronization time is earlier than the last synchronization time;
determining that there is a loss of synchronization data if the last synchronization time is earlier than the last synchronization time;
determining that there is no loss of synchronization data if the last synchronization time is not earlier than the last synchronization time.
5. The data synchronization method of claim 1, wherein the acquiring data to be synchronized based on the data acquisition manner and the data location information comprises:
and acquiring the data to be synchronized corresponding to the data position information and meeting synchronization conditions from a database storing the data to be synchronized based on the data acquisition mode, wherein a data table corresponding to the data table identification information is stored in the database, the data table stores the data to be synchronized corresponding to the data position information, and the synchronization conditions are stored in the data synchronization configuration table and are used for determining the data to be synchronized meeting the conditions.
6. The data synchronization method according to claim 1, wherein after receiving the data synchronization request sent by the client, the method further comprises:
matching the identity with a preset identity white list, and if the identity is successfully matched, determining that the identity of the client is valid;
and if the matching is unsuccessful, rejecting the data synchronization request sent by the client.
7. A data synchronization apparatus, comprising:
the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for receiving a data synchronization request sent by a client, and the data synchronization request comprises an identity of the client and last synchronization time;
the query module is used for querying data table identification information and data position information corresponding to the identity from a data synchronization configuration table, and the data synchronization configuration table stores the corresponding relation of the identity, the data table identification information and the data position information;
the determining module is used for comparing the last synchronization time with the last synchronization time corresponding to the data table identification information to determine a data acquisition mode;
the acquisition module is used for acquiring data to be synchronized based on the data acquisition mode and the data position information;
and the synchronization module is used for encapsulating all the acquired data to be synchronized and synchronizing the encapsulated data to the client.
8. The data synchronization apparatus of claim 7, wherein the determination module is further configured to:
inquiring whether the last synchronization time corresponding to the data table identification information exists in a data synchronization record table, wherein the last synchronization time corresponding to the data table identification information is stored in the data synchronization record table;
if the last synchronization time corresponding to the data table identification information exists, determining that the data acquisition mode is incremental acquisition;
and if the last synchronization time corresponding to the data table identification information does not exist, determining that the data acquisition mode is full acquisition.
9. An electronic device comprising a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is run, the machine-readable instructions when executed by the processor performing the steps of the data synchronization method of any of claims 1-6.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the data synchronization method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111080658.4A CN113779153A (en) | 2021-09-15 | 2021-09-15 | Data synchronization method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111080658.4A CN113779153A (en) | 2021-09-15 | 2021-09-15 | Data synchronization method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113779153A true CN113779153A (en) | 2021-12-10 |
Family
ID=78844006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111080658.4A Pending CN113779153A (en) | 2021-09-15 | 2021-09-15 | Data synchronization method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113779153A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115080311A (en) * | 2022-07-01 | 2022-09-20 | 深圳美克拉网络技术有限公司 | Informatization remote control method and device for big data |
| US11526491B1 (en) * | 2022-02-21 | 2022-12-13 | Clari Inc. | Method and system for collecting data from data sources with commit lag to maintain data consistency in a data store |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787077A (en) * | 2016-03-02 | 2016-07-20 | 百度在线网络技术（北京）有限公司 | Data synchronizing method and device |
| CN106101256A (en) * | 2016-07-07 | 2016-11-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for synchrodata |
| CN107040610A (en) * | 2017-05-27 | 2017-08-11 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, storage medium, terminal and server |
| CN110321387A (en) * | 2019-07-10 | 2019-10-11 | 中国联合网络通信集团有限公司 | Method of data synchronization, equipment and terminal device |
| CN111831755A (en) * | 2020-07-23 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronization method, system, medium and equipment |
-
2021
- 2021-09-15 CN CN202111080658.4A patent/CN113779153A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787077A (en) * | 2016-03-02 | 2016-07-20 | 百度在线网络技术（北京）有限公司 | Data synchronizing method and device |
| CN106101256A (en) * | 2016-07-07 | 2016-11-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for synchrodata |
| CN107040610A (en) * | 2017-05-27 | 2017-08-11 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, storage medium, terminal and server |
| CN110321387A (en) * | 2019-07-10 | 2019-10-11 | 中国联合网络通信集团有限公司 | Method of data synchronization, equipment and terminal device |
| CN111831755A (en) * | 2020-07-23 | 2020-10-27 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronization method, system, medium and equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11526491B1 (en) * | 2022-02-21 | 2022-12-13 | Clari Inc. | Method and system for collecting data from data sources with commit lag to maintain data consistency in a data store |
| CN115080311A (en) * | 2022-07-01 | 2022-09-20 | 深圳美克拉网络技术有限公司 | Informatization remote control method and device for big data |
| CN115080311B (en) * | 2022-07-01 | 2023-01-24 | 深圳美克拉网络技术有限公司 | Informatization remote control method and device for big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108228814B (en) | Data synchronization method and device | |
| CN108769212B (en) | Data synchronization method and device, computer equipment and storage medium | |
| CN110619114B (en) | Serial number generation method and system | |
| CN113779153A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN111327586A (en) | Time stamping of data in offline nodes | |
| CN107592199B (en) | Data synchronization method and system | |
| CN112199447A (en) | Database synchronization method, device and system and electronic equipment | |
| CN113407635B (en) | Multi-terminal data synchronization method and system for intelligent manufacturing | |
| CN113407555B (en) | Operation data updating method and system of equipment handheld terminal | |
| CN115114303A (en) | Service data auditing method, platform and system | |
| US20050246395A1 (en) | Databases synchronization | |
| CN116389385A (en) | System resource processing method, device, storage medium and equipment | |
| CN115632815A (en) | Data updating method and device, electronic equipment and storage medium | |
| US11360785B2 (en) | Execution path determination in a distributed environment | |
| CN111090648B (en) | Relational database data synchronization conflict resolution method | |
| CN113221151A (en) | Data processing method and device based on block chain and storage medium | |
| CN114116889A (en) | Method and device for synchronizing material data, computer equipment and storage medium | |
| CN113064950A (en) | Data synchronization method, device, equipment and storage medium | |
| CN113315875B (en) | Call information synchronization method and device, call information synchronization equipment and storage medium | |
| CN115033647B (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN115987785A (en) | Firewall data synchronization method and device and electronic equipment | |
| CN112559548B (en) | Data synchronization system and method for message middleware | |
| CN117118757B (en) | Terminal login method, device, equipment and medium in industrial control environment | |
| CN113938371B (en) | Node monitoring method, system and equipment based on blockchain | |
| CN117234672A (en) | Transaction processing method, apparatus, product, device, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Liu Renhao Inventor after: Lin Sen Inventor after: Qiao Dengjian Inventor after: Xie Pan Inventor after: Zhang Qihang Inventor before: Liu Renhao |