CN112347103B - Data synchronization method, device, electronic equipment and storage medium - Google Patents
Data synchronization method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112347103B CN112347103B CN202011226846.9A CN202011226846A CN112347103B CN 112347103 B CN112347103 B CN 112347103B CN 202011226846 A CN202011226846 A CN 202011226846A CN 112347103 B CN112347103 B CN 112347103B
- Authority
- CN
- China
- Prior art keywords
- data
- change
- column
- record
- information system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 83
- 230000008859 change Effects 0.000 claims abstract description 185
- 230000005540 biological transmission Effects 0.000 claims abstract description 63
- 230000001360 synchronised effect Effects 0.000 claims abstract description 27
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 abstract description 19
- 230000005856 abnormality Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 9
- 230000004048 modification Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 239000000243 solution Substances 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000002347 injection Methods 0.000 description 3
- 239000007924 injection Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data synchronization method, a device, electronic equipment and a storage medium, wherein the data synchronization method comprises the following steps: storing change record data of the first information system into a data stack table, wherein the change record data corresponds to the change data of the first information system; acquiring change record data to be synchronized from a data stack table, and storing the change record data into a hash table capable of serialization; creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data; and sending the Json data to the associated equipment so that the associated equipment automatically synchronizes all the associated data of the second information system according to the Json data. The method and the device can well synchronize data among different information systems, do not need to process reference data, are relatively suitable for the situation of data mixing of different information systems, are not easy to cause repetition or abnormality of data unit numbers, and can well match other associated data of a target information system.
Description
Technical Field
The present invention relates to the field of data synchronization technologies, and in particular, to a data synchronization method, a data synchronization device, an electronic device, and a storage medium.
Background
At present, a queue technology is generally used for data synchronization among different information systems, however, the technology generally uses a single data synchronization mode, when a plurality of related data are synchronized, all related data need to be stored in a stack and are synchronized according to sequence, and under general conditions, all data values are unchanged or only automatic column data values are changed during data synchronization, so that data of different information systems before and after data synchronization are easy to be completely consistent.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a data synchronization method, apparatus, electronic device, and storage medium, which can perform data synchronization between different information systems well, without processing reference data, and is more suitable for situations or complex application scenarios where different information systems data are mixed, and is not easy to cause repetition or abnormality of a data unit number, and can better match other associated data of a target information system.
In a first aspect, an embodiment of the present application provides a data synchronization method, including:
storing change record data of a first information system into a data stack table, wherein the change record data corresponds to the change data of the first information system;
acquiring the change record data to be synchronized from the data stack table, and storing the change record data into a serializable hash table;
creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data;
and sending the Json data to the associated equipment so that the associated equipment automatically synchronizes all associated data of the second information system according to the Json data.
In the implementation process, the data synchronization method of the embodiment of the application acquires the stored change record data to be synchronized from the data stack table, the change record data corresponds to the change data of the first information system, and the change record data is stored in the hash table capable of serialization; then, creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data; and then, the Json data is sent to the association equipment, so that the association equipment can automatically synchronize all the association data of the second information system according to the Json data, the data synchronization among different information systems can be better performed in the mode, the data for data synchronization is minimally generated, the reference data is not required to be processed, the method is more suitable for the situation of data mixing of different information systems or complex application scenes, the repetition or abnormality of the data single number is not easy to cause, and other association data of a target information system can be better matched.
Further, the obtaining the change record data to be synchronized from the data stack table and storing the change record data in a hash table capable of serialization includes:
acquiring the change record data to be synchronized from the data stack table;
according to the change record data, acquiring the structure of a change data table corresponding to the change data from a preset table list structure table, and judging the attribute of each column of the change data table;
when the column of the change data table is a conventional numerical Value type column, a Key with a column name of a hash table is taken, and a data Value is taken as a Value of the hash table;
when the column of the change data table is a reference data column, acquiring a reference data table ID corresponding to the reference data column, a data ID corresponding to the reference data column and a GUID data Value corresponding to the data ID according to the preset table column structure table, taking a column name as a Key of the hash table, and taking the GUID data Value of the reference data as a Value of the hash table.
In the implementation process, the method can be beneficial to the data synchronization among different information systems by storing the change record data into the serializable hash table so as to better perform the data synchronization among different information systems.
Further, after the sending the Json data to the association device, so that the association device performs automatic synchronization of all association data on the second information system according to the Json data, the method further includes:
receiving notification information sent by the associated equipment, wherein the notification information comprises a record ID of the change record data in the data stack table;
and deleting the change record data in the data stack table according to the notification information.
In the implementation process, according to the notification information sent by the receiving and associating device, the method deletes the change record data in the data stack table, so that repeated data synchronization of the same change data can be avoided, the data synchronization of the next change data can be performed more quickly, and the data synchronization efficiency is improved.
In a second aspect, an embodiment of the present application provides a data synchronization method, including:
receiving Json data sent by data change equipment;
performing deserialization on the Json data to obtain a data transmission class;
and according to the data transmission class, carrying out automatic synchronization on all the associated data of the second information system.
In the implementation process, the data synchronization method of the embodiment of the application deserializes the Json data sent by the received data change device to obtain a data transmission class, and further, according to the data transmission class, the second information system can be automatically synchronized with all associated data, so that the data synchronization among different information systems can be better performed, the data used for data synchronization is minimally generated, reference data is not required to be processed, the method is more suitable for the situation of data mixing of different information systems or complex application scenarios, repetition or abnormality of a data single number is not easy to cause, and other associated data of a target information system can be better matched.
Further, according to the data transmission class, the automatic synchronization of all the associated data of the second information system includes:
acquiring a change data table ID and a GUID value of a change data record according to the data transmission class;
acquiring the table name of the associated data table of the second information system according to the changed data table ID and a preset table name management table, and judging whether a corresponding associated data record exists in the associated data table according to the GUID value of the changed data record;
acquiring a corresponding data change type according to the data transmission class;
and correspondingly carrying out automatic synchronization on all the associated data of the associated data table according to the table name of the associated data table, whether the associated data table has the corresponding associated data record and the data change type.
In the implementation process, the method can better perform automatic synchronization of all the associated data on the second information system according to the mode of performing automatic synchronization of all the associated data on the second information system by the data transmission class, so that data synchronization among different information systems can be better completed.
Further, after the automatic synchronization of all associated data for the second information system according to the data transmission class, the method further comprises:
And sending notification information to the data changing device so that the data changing device deletes corresponding change record data in a data stack table according to the notification information, wherein the notification information comprises record IDs of the change record data in the data stack table.
In the implementation process, the method sends the notification information to the data changing device, so that the data changing device deletes the corresponding change record data in the data stack table according to the notification information, repeated data synchronization of the same change data can be avoided, data synchronization of next change data can be faster, and the data synchronization efficiency is improved.
In a third aspect, an embodiment of the present application provides a data synchronization device, including:
the first storage module is used for storing change record data of a first information system into a data stack table, wherein the change record data corresponds to the change data of the first information system;
the second storage module is used for acquiring the change record data to be synchronized from the data stack table and storing the change record data into a serializable hash table;
the processing module is used for creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data;
And the sending module is used for sending the Json data to the associated equipment so that the associated equipment can automatically synchronize all associated data of the second information system according to the Json data.
In the above implementation process, the data synchronization device of the embodiment of the present application obtains, from the data stack table, stored change record data to be synchronized, where the change record data corresponds to change data of the first information system, and stores the change record data in a hash table capable of serialization; then, creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data; and then, the Json data is sent to the association equipment, so that the association equipment can automatically synchronize all the association data of the second information system according to the Json data, the data synchronization among different information systems can be better performed in the mode, the data for data synchronization is minimally generated, the reference data is not required to be processed, the method is more suitable for the situation of data mixing of different information systems or complex application scenes, the repetition or abnormality of the data single number is not easy to cause, and other association data of a target information system can be better matched.
In a fourth aspect, an embodiment of the present application provides a data synchronization device, including:
the receiving module is used for receiving the Json data sent by the data changing equipment;
the deserialization module is used for deserializing the Json data to obtain a data transmission class;
and the data synchronization module is used for automatically synchronizing all the associated data of the second information system according to the data transmission class.
In the implementation process, the data synchronization device of the embodiment of the application performs deserialization on the Json data sent by the received data change device to obtain a data transmission class, and further, according to the data transmission class, performs automatic synchronization on all associated data of the second information system, so that the data synchronization between different information systems can be performed well, data for data synchronization is generated minimally, reference data is not required to be processed, the data synchronization device is more suitable for the situation of data mixing of different information systems or complex application scenarios, repetition or abnormality of a data single number is not easy to cause, and other associated data of a target information system can be matched well.
In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to execute the computer program to cause the electronic device to perform the data synchronization method described above.
In a sixth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program that when executed by a processor implements the above-described data synchronization method.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a data synchronization method according to an embodiment of the present application;
fig. 2 is a flowchart of step S120 provided in the first embodiment of the present application;
fig. 3 is a flow chart of a data synchronization method according to a second embodiment of the present application;
fig. 4 is a flowchart of step S230 provided in the second embodiment of the present application;
fig. 5 is a block diagram of a data synchronization device according to a third embodiment of the present application;
fig. 6 is a block diagram of a data synchronization device according to a fourth embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
At present, a queue technology is generally used for data synchronization among different information systems, however, the technology generally uses a single data synchronization mode, when a plurality of related data are synchronized, all related data need to be stored in a stack and synchronized according to sequence, and generally, a data table can only use an unchangeable column and column combination as a main key of the table, so that data of different information systems before and after data synchronization are easy to be completely consistent.
Aiming at the problems in the prior art, the application provides a data synchronization method, a device, electronic equipment and a storage medium, which can well perform data synchronization among different information systems without processing reference data, are more suitable for the situation of data mixing of different information systems or complex application scenes, are not easy to cause repetition or abnormality of data single numbers, and can well match other associated data of a target information system.
Example 1
Referring to fig. 1, fig. 1 is a flow chart of a data synchronization method according to an embodiment of the present application. The data synchronization method described below in the embodiments of the present application may be applied to a data changing device, and the data changing device may be a server.
In this embodiment, the data change device is in communication connection with the associated device, where the data change device is a device that generates data change in the information system, and the associated device is a device that needs to perform data synchronization in the information system, where the data change device corresponds to the first information system, the associated device corresponds to the second information system, and the associated device may be single or multiple.
The data synchronization method of the embodiment of the application comprises the following steps:
step S110, the change record data of the first information system is stored in the data stack table, and the change record data corresponds to the change data of the first information system.
In this embodiment, the first information system may be a parking information system or the like, and in this embodiment, the parking information system is taken as an example.
The change log data may include a yard code, a data table ID, a record ID, a data change type (add, modify, or delete).
In this embodiment, each data table primary key is a system self-increasing column, and the column name is ID; the unique record identification list is named GUID, the field attribute is named GUID type (global unique identifier Globally Unique Identifier), and the global unique data value is automatically generated when the record is newly added.
Each data table structure and column attribute information is defined in the table name management table jzmetadata_dataentity and the table column structure table jzmetadata_dataentitycol, and each data table has a unique identification EntityID in the table name management table jzmetadata_dataentity.
In this embodiment, the data table primary key adopts an ID manner, and the globally unique GUID data column is used as an identification column, so that data coverage possibly caused by synchronizing and integrating multiple subsystems can be prevented.
Step S120, the change record data to be synchronized is obtained from the data stack table, and the change record data is stored in the serializable hash table.
In this embodiment, when the change record data is acquired in the data stack table, the first change record data to be synchronized is acquired sequentially.
Step S130, a data transmission class is newly established according to the change record data and the change data, and the data transmission class is serialized to generate corresponding Json data.
In this embodiment, the new data transmission class DataSyncMsg includes the following attributes:
ID: record ID in the data stack table;
organozationid: organization structure ID;
ParkCode: yard coding;
EntityID: a data table ID;
refguide: GUID value of the data record;
UpdateType: the uploading type is 1-3, and the uploading type respectively represents the new adding type, the modification type and the deletion type;
DetailEntities: the data detail, the hash table generated is taken as the value.
And step S140, transmitting the Json data to the associated equipment so that the associated equipment automatically synchronizes all the associated data of the second information system according to the Json data.
In this embodiment, the second information system may be a property information system, which is taken as an example in this embodiment.
According to the data synchronization method, stored change record data to be synchronized is obtained from a data stack table, the change record data corresponds to the change data of the first information system, and the change record data is stored in a hash table capable of being serialized; then, creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data; and then, the Json data is sent to the association equipment, so that the association equipment can automatically synchronize all the association data of the second information system according to the Json data, the data synchronization among different information systems can be better performed in the mode, the data for data synchronization is minimally generated, the reference data is not required to be processed, the method is more suitable for the situation of data mixing of different information systems or complex application scenes, the repetition or abnormality of the data single number is not easy to cause, and other association data of a target information system can be better matched.
Referring to fig. 2, fig. 2 is a schematic flow chart of step S120 provided in the embodiment of the present application.
As an optional implementation manner, in the data synchronization method of the embodiment of the present application, step S120, the obtaining the change record data to be synchronized from the data stack table, and storing the change record data in the hash table capable of serialization may include the following steps:
Step S121, obtaining the change record data to be synchronized from the data stack table;
step S122, according to the change record data, acquiring the structure of a change data table corresponding to the change data from a preset table structure table, and judging the attribute of each column of the change data table;
step S123, when the column of the change data table is a conventional Value type column, the Key with the column name of the hash table is taken, and the data Value is taken as the Value of the hash table;
in step S124, when the column of the change data table is the reference data column, the reference data table ID corresponding to the reference data column, the data ID corresponding to the reference data column, and the GUID data Value corresponding to the data ID are obtained according to the preset table column structure table, the Key with the column name of the hash table is obtained, and the GUID data Value of the reference data is the Value of the hash table.
Wherein the conventional numeric value type column may be a literal type column, a numeric type column, a date type column, a boolean type column, etc.
When the data ID corresponding to the reference data column is ValueID and the GUID data value corresponding to the data ID is obtained, the name of the data table (for example, table name is tableaa) is obtained according to Select tableName From JzMetaData _ DataEntity where EntityID =data table ID, and the GUID data value corresponding to the data ID is obtained according to Select Guid From TableA Where ID =valueid.
In the process, the method can be beneficial to the data synchronization among different information systems by storing the change record data into the serializable hash table so as to better synchronize the data among different information systems.
As an optional implementation manner, after the data synchronization method in the embodiment of the present application sends Json data to the association device in step S140, so that the association device performs automatic synchronization of all associated data on the second information system according to the Json data, the method may include the following steps:
receiving notification information sent by the associated equipment, wherein the notification information comprises a record ID of change record data in a data stack table;
and deleting the change record data in the data stack table according to the notification information.
In the process, according to the notification information sent by the receiving and associating device, the method deletes the change record data in the data stack table, so that repeated data synchronization of the same change data can be avoided, the data synchronization of the next change data can be performed more quickly, and the data synchronization efficiency is improved.
Example two
Referring to fig. 3, fig. 3 is a flow chart of a data synchronization method according to an embodiment of the present application. The data synchronization method described below in the embodiments of the present application may be applied to an associated device, which may be a server.
In this embodiment, the association device is in communication connection with the data change device, that is, the device in which the information system changes data, and the association device, that is, the device in which the information system needs to perform data synchronization, corresponds to the first information system, and corresponds to the second information system.
The data synchronization method of the embodiment of the application comprises the following steps:
step S210, json data sent by the data changing device is received.
Step S220, deserializing the Json data to obtain a data transmission class.
Step S230, according to the data transmission class, carrying out automatic synchronization of all the associated data on the second information system.
According to the data synchronization method, the Json data sent by the received data change equipment is deserialized to obtain one data transmission class, and then automatic synchronization of all associated data can be carried out on the second information system according to the data transmission class.
Referring to fig. 4, fig. 4 is a flowchart of step S230 provided in the embodiment of the present application.
As an optional implementation manner, the data synchronization method in the embodiment of the present application, step S230, according to the data transmission class, performs automatic synchronization of all associated data on the second information system, and may include the following steps:
step S231, according to the data transmission class, acquiring a change data table ID and a GUID value of a change data record;
step S232, according to the changed data table ID and a preset table name management table, acquiring the table name of the associated data table of the second information system, and judging whether a corresponding associated data record exists in the associated data table according to the GUID value of the changed data record;
step S233, obtaining a corresponding data change type according to the data transmission class;
step S234, according to the table name of the associated data table, whether the associated data table has the corresponding associated data record and the data change type, the associated data table is automatically synchronized correspondingly.
Wherein, the change data table ID corresponds to EntityID of the data transmission class DataSyncMsg, and the GUID value of the change data record corresponds to RefGuid value of the data transmission class DataSyncMsg.
When the table name of the associated data table of the second information system is acquired, select tableName From JzMetaData _ DataEntity where EntityID = @ table id is used to acquire the table name of the associated data table @ table name.
When judging whether the corresponding associated data record exists in the associated data table, judging whether the corresponding associated data record exists or not by using the selection from@tablename window guide= @value.
The data change type corresponds to the UpdateType of the data transfer class DataSyncMsg: uploading type.
If the data change type is a new addition or modification type and the current record exists, using Update logic, otherwise using Insert new addition logic; if the data change type is a Delete type, the current record already exists, and Delete logic is used.
Specifically, insert newly added logic, the steps are as follows:
a) Acquiring a value of a DetailEntities attribute and storing the value as a Hashtable;
b) Acquiring an associated data table structure from a table column structure table jzmetadata_dataencycol, and judging column attributes one by one from the acquired table structure;
i. when the column is a conventional attribute column, judging whether the Hashtable exists in the column or not;
if the Hashtable exists in the column, directly taking the value as the data value of the column; if not, generating a default value; ( And (3) injection: this is done in order to accommodate possible structural inconsistencies in the data tables of the two information systems. )
if the column is an automatic code column, calling a record code of a system code generation rule generation platform as a data value of the column; ( And (3) injection: the processing mode can achieve that the data of the synchronized automatic code column accords with the coding rule of the second information system in the second information system. )
if the column is the reference data column, judging whether the Hashtable exists in the column or not;
v. when there is no assignment of 0 (0 is a default value); otherwise, acquiring a value of a Key corresponding to the Hashtable, namely a data unique identifier GUID value, and inquiring whether the GUID of an associated data table of the associated data table exists or not according to the reference column attribute;
if the GUID data corresponding to the reference table exists, acquiring the ID of the record as the value of the current column and storing the ID; when the GUID data corresponding to the reference table does not exist, informing the parking information system to upload the record and newly store the reference record, and acquiring the ID of the newly generated reference record as the value of the current column to store;
when uploading the reference data, the same checking process is executed, all the reference data and the reference data of the reference data are uploaded in a recursion mode, and all the data are ensured to exist;
c) All columns of Key and data Value are spliced into an Insert statement to be inserted into a database.
Update logic, steps are as follows:
a) Acquiring a value of a DetailEntities attribute and storing the value as a Hashtable;
b) Acquiring an associated data table structure from a table column structure table jzmetadata_dataencycol, and judging column attributes one by one from the acquired table structure;
i. if the column is a conventional attribute column, judging whether the Hashtable exists in the column or not;
if the Hashtable exists in the column, directly taking the value as the data value of the column; if not, not updating;
such columns are auto-coded columns, and no processing is performed; ( And (3) injection: the encoded values that have been generated in accordance with the second information system encoding rules are not modified. )
if the column is the reference data column, judging whether the Hashtable exists in the column or not;
v. when no value is assigned to 0; otherwise, acquiring a value of a Key corresponding to the Hashtable, namely a data unique identifier GUID value, and inquiring whether the GUID of an associated data table of the associated data table exists or not according to the reference column attribute;
if the GUID data corresponding to the reference table exists, acquiring the ID of the record as the value of the current column and storing the ID; when the GUID data corresponding to the reference table does not exist, informing the parking information system to upload the record and newly store the reference record, and acquiring the ID of the newly generated reference record as the value of the current column to store;
When uploading the reference data, the same checking process is executed, all the reference data and the reference data of the reference data are uploaded in a recursion mode, and all the data are ensured to exist;
c) And splicing all the column keys and the data Value into an Update statement Update database.
Delete logic, steps as follows:
judging whether the corresponding record exists or not, if so, deleting the record by using Delete from@tablename white guide= @value.
In the process, the method can better perform automatic synchronization of all the associated data on the second information system according to the mode of performing automatic synchronization of all the associated data on the second information system by the data transmission class, so that data synchronization among different information systems can be better completed;
meanwhile, the data uploading automatically judges the reference data columns, and all the reference data are automatically uploaded in a recursion mode, so that the service operation is simple, and the integrity of the data is ensured; the data uploading type is automatic compatibility judgment of new addition and modification, so that the complexity of service judgment is reduced; in addition, the difference of the data structures of the two information systems is allowed, and the difference column automatically uses a default value, so that the maintainability of different information systems is improved; the data synchronization process aims at automatic coding and automatic processing, and each information system uses own coding rules, so that the consistency of the data rules is ensured.
As an optional implementation manner, the data synchronization method of the embodiment of the present application may include, after performing automatic synchronization of all associated data on the second information system according to the data transmission class in step S230, the following steps:
and sending notification information to the data changing device so that the data changing device deletes the corresponding change record data in the data stack table according to the notification information, wherein the notification information comprises the record ID of the change record data in the data stack table.
In the process, the method sends the notification information to the data changing device, so that the data changing device deletes the corresponding change record data in the data stack table according to the notification information, repeated data synchronization of the same change data can be avoided, data synchronization of next change data can be faster, and the data synchronization efficiency is improved.
Example III
In order to perform a corresponding method of the above embodiment to achieve the corresponding functions and technical effects, a data synchronization device is provided below.
Referring to fig. 5, fig. 5 is a block diagram of a data synchronization device according to an embodiment of the present application.
The data synchronization device of the embodiment of the application comprises:
A first storage module 310, configured to store change record data of the first information system in a data stack table, where the change record data corresponds to change data of the first information system;
a second storage module 320, configured to obtain the change record data to be synchronized from the data stack table, and store the change record data into a hash table capable of serialization;
the processing module 330 is configured to create a data transmission class according to the change record data and the change data, and sequence the data transmission class to generate corresponding Json data;
and the sending module 340 is configured to send the Json data to the association device, so that the association device performs automatic synchronization on all association data of the second information system according to the Json data.
The data synchronizing device acquires stored change record data to be synchronized from the data stack table, wherein the change record data corresponds to the change data of the first information system, and the change record data is stored in a serializable hash table; then, creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data; and then, the Json data is sent to the association equipment, so that the association equipment can automatically synchronize all the association data of the second information system according to the Json data, the data synchronization among different information systems can be better performed in the mode, the data for data synchronization is minimally generated, the reference data is not required to be processed, the method is more suitable for the situation of data mixing of different information systems or complex application scenes, the repetition or abnormality of the data single number is not easy to cause, and other association data of a target information system can be better matched.
As an alternative embodiment, the second storage module 320 may be specifically used for:
acquiring the change record data to be synchronized from a data stack table;
according to the change record data, acquiring the structure of a change data table corresponding to the change data from a preset table list structure table, and judging the attribute of each column of the change data table;
when the column of the change data table is a conventional numerical Value type column, the column name is taken as the Key of the hash table, and the data Value is taken as the Value of the hash table;
when the column of the change data table is a reference data column, acquiring a reference data table ID corresponding to the reference data column, a data ID corresponding to the reference data column and a GUID data Value corresponding to the data ID according to a preset table column structure table, taking a column name as a Key of the hash table, and taking the GUID data Value of the reference data as a Value of the hash table.
As an optional implementation manner, the data synchronization device of the embodiment of the present application may further include:
the receiving module is used for receiving notification information sent by the associated equipment, wherein the notification information comprises record IDs of change record data in the data stack table;
and the deleting module is used for deleting the change record data in the data stack table according to the notification information.
The data synchronization device can implement the data synchronization method of the first embodiment. The options in the first embodiment described above also apply to this embodiment, and are not described in detail here.
The rest of the embodiments of the present application may refer to the content of the first embodiment, and in this embodiment, no further description is given.
Example IV
In order to execute the method corresponding to the second embodiment to achieve the corresponding functions and technical effects, a data synchronization device is provided below.
Referring to fig. 6, fig. 6 is a block diagram of a data synchronization device according to an embodiment of the present application.
The data synchronization device of the embodiment of the application comprises:
a receiving module 410, configured to receive Json data sent by the data change device;
the deserializing module 420 is configured to deserialize the Json data to obtain a data transmission class;
the data synchronization module 430 is configured to automatically synchronize all associated data of the second information system according to the data transmission class.
According to the data synchronization device, the Json data sent by the received data change equipment is deserialized to obtain a data transmission class, and then automatic synchronization of all associated data can be carried out on the second information system according to the data transmission class.
As an alternative embodiment, the data synchronization module 430 may be specifically configured to:
acquiring a change data table ID and a GUID value of a change data record according to the data transmission class;
acquiring the table name of the associated data table of the second information system according to the ID of the changed data table and a preset table name management table, and judging whether a corresponding associated data record exists in the associated data table according to the GUID value of the changed data record;
acquiring a corresponding data change type according to the data transmission type;
and according to the table name of the associated data table, whether the associated data table has corresponding associated data records and data change types, carrying out automatic synchronization on all associated data of the associated data table correspondingly.
As an optional implementation manner, the data synchronization device of the embodiment of the present application may further include:
and the sending module is used for sending notification information to the data changing equipment so that the data changing equipment deletes corresponding change record data in the data stack table according to the notification information, wherein the notification information comprises record IDs of the change record data in the data stack table.
The data synchronization device can implement the data synchronization method of the second embodiment. The options in the second embodiment described above are also applicable to this embodiment, and will not be described in detail here.
The rest of the embodiments of the present application may refer to the content of the first embodiment and the second embodiment, and in this embodiment, a detailed description is omitted.
Example five
An embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor runs the computer program to enable the electronic device to execute the data synchronization method of the first embodiment or the second embodiment.
Alternatively, the electronic device may be a data modification device or an association device, and the data modification device or the association device may be a server.
In addition, the embodiment of the present application further provides a computer readable storage medium storing a computer program, where the computer program implements the data synchronization method of the first embodiment or the second embodiment when executed by a processor.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners as well. The apparatus embodiments described above are merely illustrative, for example, flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (8)
1. A method of data synchronization, comprising:
storing change record data of a first information system into a data stack table, wherein the change record data corresponds to the change data of the first information system;
acquiring the change record data to be synchronized from the data stack table, and storing the change record data into a serializable hash table;
creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data;
the Json data is sent to the association equipment, so that the association equipment automatically synchronizes all association data of a second information system according to the Json data;
the obtaining the change record data to be synchronized from the data stack table and storing the change record data in a serializable hash table includes:
acquiring the change record data to be synchronized from the data stack table;
according to the change record data, acquiring the structure of a change data table corresponding to the change data from a preset table list structure table, and judging the attribute of each column of the change data table;
When the column of the change data table is a conventional numerical Value type column, a Key with a column name of a hash table is taken, and a data Value is taken as a Value of the hash table;
when the column of the change data table is a reference data column, acquiring a reference data table ID corresponding to the reference data column, a data ID corresponding to the reference data column and a GUID data Value corresponding to the data ID according to the preset table column structure table, taking a column name as a Key of the hash table, and taking the GUID data Value of the reference data as a Value of the hash table.
2. The data synchronization method of claim 1, wherein after said sending the Json data to the associated device to cause the associated device to perform automatic synchronization of all associated data for the second information system based on the Json data, the method further comprises:
receiving notification information sent by the associated equipment, wherein the notification information comprises a record ID of the change record data in the data stack table;
and deleting the change record data in the data stack table according to the notification information.
3. A method of data synchronization, comprising:
receiving Json data sent by data change equipment;
performing deserialization on the Json data to obtain a data transmission class;
According to the data transmission class, carrying out automatic synchronization on all associated data of the second information system;
and according to the data transmission class, automatically synchronizing all associated data of the second information system, wherein the automatic synchronization comprises the following steps:
acquiring a change data table ID and a GUID value of a change data record according to the data transmission class;
acquiring the table name of the associated data table of the second information system according to the changed data table ID and a preset table name management table, and judging whether a corresponding associated data record exists in the associated data table according to the GUID value of the changed data record;
acquiring a corresponding data change type according to the data transmission class;
and correspondingly carrying out automatic synchronization on all the associated data of the associated data table according to the table name of the associated data table, whether the associated data table has the corresponding associated data record and the data change type.
4. A data synchronization method according to claim 3, characterized in that after said automatic synchronization of all associated data of the second information system according to said data transmission class, the method further comprises:
and sending notification information to the data changing device so that the data changing device deletes corresponding change record data in a data stack table according to the notification information, wherein the notification information comprises record IDs of the change record data in the data stack table.
5. A data synchronization device, comprising:
the first storage module is used for storing change record data of a first information system into a data stack table, wherein the change record data corresponds to the change data of the first information system;
the second storage module is used for acquiring the change record data to be synchronized from the data stack table and storing the change record data into a serializable hash table;
the processing module is used for creating a data transmission class according to the change record data and the change data, and serializing the data transmission class to generate corresponding Json data;
the sending module is used for sending the Json data to the associated equipment so that the associated equipment can automatically synchronize all associated data of the second information system according to the Json data;
the second storage module may be specifically configured to: acquiring the change record data to be synchronized from a data stack table;
according to the change record data, acquiring the structure of a change data table corresponding to the change data from a preset table list structure table, and judging the attribute of each column of the change data table;
when the column of the change data table is a conventional numerical Value type column, the column name is taken as the Key of the hash table, and the data Value is taken as the Value of the hash table;
When the column of the change data table is a reference data column, acquiring a reference data table ID corresponding to the reference data column, a data ID corresponding to the reference data column and a GUID data Value corresponding to the data ID according to a preset table column structure table, taking a column name as a Key of the hash table, and taking the GUID data Value of the reference data as a Value of the hash table.
6. A data synchronization device, comprising:
the receiving module is used for receiving the Json data sent by the data changing equipment;
the deserialization module is used for deserializing the Json data to obtain a data transmission class;
the data synchronization module is used for automatically synchronizing all the associated data of the second information system according to the data transmission class;
the data synchronization module may be specifically configured to:
acquiring a change data table ID and a GUID value of a change data record according to the data transmission class;
acquiring the table name of the associated data table of the second information system according to the ID of the changed data table and a preset table name management table, and judging whether a corresponding associated data record exists in the associated data table according to the GUID value of the changed data record;
acquiring a corresponding data change type according to the data transmission type;
And according to the table name of the associated data table, whether the associated data table has corresponding associated data records and data change types, carrying out automatic synchronization on all associated data of the associated data table correspondingly.
7. An electronic device comprising a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the data synchronization method according to any one of claims 1 to 4.
8. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the data synchronization method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011226846.9A CN112347103B (en) | 2020-11-05 | 2020-11-05 | Data synchronization method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011226846.9A CN112347103B (en) | 2020-11-05 | 2020-11-05 | Data synchronization method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112347103A CN112347103A (en) | 2021-02-09 |
| CN112347103B true CN112347103B (en) | 2024-04-12 |
Family
ID=74429914
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011226846.9A Active CN112347103B (en) | 2020-11-05 | 2020-11-05 | Data synchronization method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347103B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107766572A (en) * | 2017-11-13 | 2018-03-06 | 北京国信宏数科技有限责任公司 | Distributed extraction and visual analysis method and system based on economic field data |
| CN107885804A (en) * | 2017-11-02 | 2018-04-06 | 平安科技（深圳）有限公司 | Database synchronization method, application server and computer-readable recording medium |
| CN108287886A (en) * | 2018-01-16 | 2018-07-17 | 阿里巴巴集团控股有限公司 | The method and device of synchrodata modification information |
| CN108874644A (en) * | 2018-06-06 | 2018-11-23 | 平安科技（深圳）有限公司 | Data monitoring method, device, computer equipment and storage medium |
| CN109121436A (en) * | 2015-11-25 | 2019-01-01 | 蒂米菲尔股份有限公司 | Augmentation, exploration and Maintenance Significant Items method layered |
| CN109284312A (en) * | 2018-08-27 | 2019-01-29 | 山东威尔数据股份有限公司 | A kind of heterogeneous database change real-time informing method |
| CN110032571A (en) * | 2019-04-18 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Business flow processing method, apparatus, storage medium and calculating equipment |
| CN110753916A (en) * | 2017-10-16 | 2020-02-04 | 因特比有限公司 | Method and system for virtualizing blockchains |
| CN110909006A (en) * | 2019-10-15 | 2020-03-24 | 中国平安人寿保险股份有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN111400407A (en) * | 2020-04-10 | 2020-07-10 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070005734A1 (en) * | 2005-06-30 | 2007-01-04 | Microsoft Corporation | Generically extensible client application |
| WO2018052726A1 (en) * | 2016-09-15 | 2018-03-22 | Nuts Holdings, Llc | Encrypted userdata transit and storage |
-
2020
- 2020-11-05 CN CN202011226846.9A patent/CN112347103B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109121436A (en) * | 2015-11-25 | 2019-01-01 | 蒂米菲尔股份有限公司 | Augmentation, exploration and Maintenance Significant Items method layered |
| CN110753916A (en) * | 2017-10-16 | 2020-02-04 | 因特比有限公司 | Method and system for virtualizing blockchains |
| CN107885804A (en) * | 2017-11-02 | 2018-04-06 | 平安科技（深圳）有限公司 | Database synchronization method, application server and computer-readable recording medium |
| CN107766572A (en) * | 2017-11-13 | 2018-03-06 | 北京国信宏数科技有限责任公司 | Distributed extraction and visual analysis method and system based on economic field data |
| CN108287886A (en) * | 2018-01-16 | 2018-07-17 | 阿里巴巴集团控股有限公司 | The method and device of synchrodata modification information |
| CN108874644A (en) * | 2018-06-06 | 2018-11-23 | 平安科技（深圳）有限公司 | Data monitoring method, device, computer equipment and storage medium |
| CN109284312A (en) * | 2018-08-27 | 2019-01-29 | 山东威尔数据股份有限公司 | A kind of heterogeneous database change real-time informing method |
| CN110032571A (en) * | 2019-04-18 | 2019-07-19 | 腾讯科技（深圳）有限公司 | Business flow processing method, apparatus, storage medium and calculating equipment |
| CN110909006A (en) * | 2019-10-15 | 2020-03-24 | 中国平安人寿保险股份有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN111400407A (en) * | 2020-04-10 | 2020-07-10 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
Non-Patent Citations (3)
| Title |
|---|
| A survey on attack vectors in stack cache memory;Navid Khoshavi 等;Integration;134-147 * |
| 基于可变时间窗口的增量数据抽取模型;刘杰 等;计算机科学;204-209 * |
| 输电线路外破管控系统的设计与实现;尹俊强;信息科技;1-67 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112347103A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104796326B (en) | A kind of group creating method and device of enterprise IM | |
| US8700560B2 (en) | Populating a multi-relational enterprise social network with disparate source data | |
| CN111008521B (en) | Method, device and computer storage medium for generating wide table | |
| CN108304251B (en) | Thread synchronization method and server | |
| CN112783760B (en) | Processing method and device for test cases and storage medium | |
| CN112256318A (en) | Construction method and equipment for dependent product | |
| CN114281253B (en) | Storage volume management method | |
| CN105868196A (en) | Method for generating industrial data report in server | |
| CN112347103B (en) | Data synchronization method, device, electronic equipment and storage medium | |
| CN110555600B (en) | Method, device, equipment and storage medium for preventing forbidden components in modular BOM from being misused | |
| CN109614442B (en) | Data table maintenance method and device for data synchronization, storage medium and electronic equipment | |
| CN112785248A (en) | Human resource data cross-organization interaction method, device, equipment and storage medium | |
| CN113641761B (en) | Data synchronization method and device | |
| CN115437703A (en) | Business process arranging method, device, equipment and storage medium | |
| CN115858686B (en) | Tree data synchronization method, synchronization circulation model thereof, electronic equipment and medium | |
| CN113852919B (en) | Method and device for generating early warning message, storage medium and electronic device | |
| CN111382060A (en) | File distribution record processing method, electronic device and computer storage medium | |
| CN116739397B (en) | Dynamic management method for new energy indexes | |
| CN112948406B (en) | Method, system and device for storing and synchronizing configuration change data | |
| CN112148725B (en) | Method and device for realizing global unique index | |
| CN109583231B (en) | Automatic generation method for multiple identifiers of data object of scientific data service platform | |
| CN114300153A (en) | Medical data transmission method, device, computer equipment and storage medium | |
| CN117527895A (en) | HTTP request processing method and system based on AOP | |
| CN105868198A (en) | Generating method of server-side industrial data report | |
| CN116954474A (en) | Method, apparatus, device, storage medium and program product for data reading |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |