CN112187916B - Cross-system data synchronization method and device - Google Patents
Cross-system data synchronization method and device Download PDFInfo
- Publication number
- CN112187916B CN112187916B CN202011031792.0A CN202011031792A CN112187916B CN 112187916 B CN112187916 B CN 112187916B CN 202011031792 A CN202011031792 A CN 202011031792A CN 112187916 B CN112187916 B CN 112187916B
- Authority
- CN
- China
- Prior art keywords
- data
- synchronous
- server
- synchronization
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 230000001360 synchronised effect Effects 0.000 claims abstract description 229
- 238000012545 processing Methods 0.000 claims abstract description 71
- 238000004458 analytical method Methods 0.000 claims description 4
- 239000003999 initiator Substances 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 230000003139 buffering effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 3
- 238000010295 mobile communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000001364 causal effect Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The embodiment of the application discloses a cross-system data synchronization method and device, and belongs to the technical field of data processing. The method comprises the following steps: the method comprises the steps that a first server obtains a first synchronous message from a message queue of a second server, the first server and the second server are servers in different systems, the first synchronous message comprises first synchronous data, and the second server stores first processing result data of the first synchronous data; the first server analyzes the first synchronous message and acquires first synchronous data; the first server takes the target object including first synchronous data of the first server as target synchronous data; and the first server processes the target synchronous data and determines and stores first processing result data of the target synchronous data. The embodiment of the application meets the consistency of data processing among business layers of different systems.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a cross-system data synchronization method and device.
Background
Today, the rapid development of the internet has become more and more demanding with respect to system stability and high availability, and especially in the field of financial services, it has become important to be able to provide stable service capabilities to the outside. Therefore, each system realizes a multi-center deployment architecture so as to improve the stability and disaster tolerance of the system and ensure high availability.
In the prior art, when each center system receives a transaction request, the center system firstly performs data processing, after the data processing is completed, the result data is synchronized to other centers, and the other centers store the result data. In the prior art, the data synchronization process only performs data synchronization on a database, so that the synchronization requirement on a service layer cannot be met, and the real-time consistency of data among the centers cannot be ensured.
Disclosure of Invention
The embodiment of the application provides a cross-system data synchronization method and device, which are used for meeting the consistency of data processing among business layers of different systems.
In one aspect, an embodiment of the present application provides a method for synchronizing data across systems, where the method includes:
the method comprises the steps that a first server obtains a first synchronous message from a message queue of a second server, the first server and the second server are servers in different systems, the first synchronous message comprises first synchronous data, and the second server stores first processing result data of the first synchronous data;
the first server analyzes the first synchronous message and acquires first synchronous data;
the first server takes the target object including first synchronous data of the first server as target synchronous data;
and the first server processes the target synchronous data and determines and stores first processing result data of the target synchronous data.
Optionally, the first server obtains a first synchronization packet from a message queue of the second server, including:
the first server receives synchronous broadcast sent by the second server;
the first server sends a synchronous data acquisition request to the second server according to the synchronous broadcast;
and the first server receives the first synchronous message sent by the second server.
Optionally, after the first server parses the target synchronization packet, the method further includes:
if the first server fails to analyze the target synchronous message, caching the target synchronous message;
after the first server processes the target synchronous data, the method further comprises:
and if the first server fails to process the target synchronous data, caching the target synchronous data.
Optionally, the processing, by the first server, the target synchronization data includes:
and adding a synchronization identifier into the target synchronization data by the first server.
Optionally, the method further comprises:
the first server receives a data synchronization request, wherein the data synchronization request comprises second synchronization data;
the first server processes the second synchronous data, and determines and stores second processing result data of the second synchronous data;
the first server generates a second synchronous message according to the second synchronous data;
and the first server adds the second synchronous message into a message queue of the first server.
In another aspect, an embodiment of the present application further provides a cross-system data synchronization device, where the device includes:
the receiving and transmitting unit is used for acquiring a first synchronous message from a message queue of a second server, wherein the first server and the second server are servers in different systems, the first synchronous message comprises first synchronous data, and the second server stores first processing result data of the first synchronous data;
the analyzing unit is used for analyzing the first synchronous message and acquiring first synchronous data;
a judging unit, configured to take target object including first synchronization data of the first server as target synchronization data;
and the processing unit is used for processing the target synchronous data and determining and storing first processing result data of the target synchronous data.
Optionally, the transceiver unit is specifically configured to:
receiving synchronous broadcast sent by the second server;
according to the synchronous broadcast, sending a synchronous data acquisition request to the second server;
and receiving the first synchronous message sent by the second server.
Optionally, the device further comprises a buffer unit for:
if the analysis of the target synchronous message fails, caching the target synchronous message;
and if the processing of the target synchronous data fails, caching the target synchronous data.
Optionally, the processing unit is specifically configured to:
and adding a synchronous identifier into the target synchronous data.
Optionally, the transceiver unit is further configured to receive a data synchronization request, where the data synchronization request includes second synchronization data;
the processing unit is further used for processing the second synchronous data, and determining and storing second processing result data of the second synchronous data; generating a second synchronous message according to the second synchronous data; and adding the second synchronous message into a message queue of the first server.
In another aspect, an embodiment of the present application provides a computing device, including at least one processor, and at least one memory, where the memory stores a computer program, and when the program is executed by the processor, causes the processor to execute the steps of the cross-system data synchronization method provided by the embodiment of the present application.
In another aspect, an embodiment of the present application provides a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the steps of the cross-system data synchronization method provided by the embodiment of the present application.
Embodiments of the present application relate to two or more systems, each including at least one server. The first server and the second server are servers in different systems. After the second server receives the first synchronous data, the first synchronous data is processed to obtain first processing result data, the second server stores the first processing result data, and a first synchronous message is generated according to the first synchronous data and is cached in a message queue. The first server acquires a first synchronous message from a message queue of the second server, and analyzes the first synchronous message to obtain first synchronous data. And taking the target object including the first synchronous data of the first server as the target synchronous data of the first server. And processing the target synchronous data, and determining and storing first processing result data of the target synchronous data. In the embodiment of the application, the second server directly synchronizes unprocessed original data, namely the first synchronization data, and directly transmits the first synchronization data to the service layer of the first server for data processing by means of the message middleware, so that the data synchronization requirement between the service layer of the first server and the service layer of the second server is ensured, and the real-time consistency of the data processing is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 shows a system architecture diagram of a cross-system data synchronization system provided by an embodiment of the present application;
fig. 2 is a schematic structural diagram of a three-layer architecture of a server in a central system according to an embodiment of the present application;
FIG. 3 is a flowchart of a cross-system data synchronization method according to an embodiment of the present application;
fig. 4 is a schematic diagram showing a format of a synchronization packet specified by a data synchronization protocol according to an embodiment of the present application;
FIG. 5 is a block diagram showing the structure of a cross-system data synchronization apparatus according to an embodiment of the present application;
fig. 6 is a block diagram showing the structure of a server according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the technical solutions of the present application, but not all embodiments. All other embodiments, based on the embodiments described in the present document, which can be obtained by a person skilled in the art without any inventive effort, are within the scope of protection of the technical solution of the present application.
The terms first and second in the description and claims of the application and in the above-mentioned figures are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
The following description is made for a few simple descriptions of the architecture of the technical solution of the embodiments of the present application, and it should be noted that the architecture described below is only used for illustrating the embodiments of the present application and is not limiting. In the specific implementation, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.
In order to further explain the technical solution provided by the embodiments of the present application, the following details are described with reference to the accompanying drawings and the detailed description. Although embodiments of the present application provide the method operational steps shown in the following embodiments or figures, more or fewer operational steps may be included in the method, either on a routine or non-inventive basis. In steps where there is logically no necessary causal relationship, the execution order of the steps is not limited to the execution order provided by the embodiments of the present application.
Referring to fig. 1, a system architecture diagram of a cross-system data synchronization method according to an embodiment of the present application is shown. The embodiment of the application provides a data processing method. As shown in fig. 1, a system architecture applicable to the embodiment of the present application includes a plurality of central systems, where each central system includes a plurality of servers, and the servers may be network devices such as a computer, and the servers may be an independent device or a server cluster formed by the plurality of servers. Preferably, the server can process information by adopting cloud computing technology. The database and the center can communicate through an INTERNET network, and can also communicate through a global system for mobile communication (Global System for Mobile Communications, abbreviated as GSM), a long term evolution (long term evolution, abbreviated as LTE) system and other mobile communication systems.
Specifically, the first server in the embodiment of the present application may be a server in any central system in fig. 1, and the corresponding second server is a server in any central system other than the central system in which the first server is located. Each central system in fig. 1 can synchronize the data of the other central systems by using the data synchronization method in the embodiment of the present application, so as to satisfy the data consistency between the service layers and the data layers of different central systems.
In an alternative embodiment, the server of each central system in fig. 1 is a three-layer architecture, that is, each functional module is divided into an interface layer (UI), a service logic layer (BLL) and a Data Access Layer (DAL), the layers are mutually accessed by adopting interfaces, and entity classes (Model) of object models are used as carriers for data transmission, the entity classes of different object models generally correspond to different tables of a database, and the attribute of the entity classes is consistent with the field names of the tables of the database.
Fig. 2 shows a schematic structural diagram of a three-layer architecture. Wherein, the main functions of the system and the business logic are processed in the business logic layer. The data access layer is mainly an operation layer for non-original data, namely, the data access layer is used for operating the form of data storage such as a database or a text file, and particularly, a business logic layer or an interface layer is used for providing data service. The business logic layer is mainly used for operating specific problems, and can also be understood as operating the data layer to process the data business logic. The interface layer mainly represents a WEB mode or a WINFORM mode, and mainly receives a request of a user and data feedback, provides access of an application program for a client and provides an interactive operation interface for the user.
Specifically, in the embodiment of the application, a label (also referred to as a Topic) is newly built in the message middleware of the server of each central system, and the message middleware is specially used for storing data that needs to be synchronized across centers in the present center. When the center has data to be synchronized across centers, the data only needs to be cached to the cross-center synchronization Topic of the message middleware. Other hub systems may subscribe to the hub system's synchronous Topic consumption synchronization data.
Each central system is newly added with one or more data release services for packaging and releasing the data to be synchronized according to the synchronization protocol, and if the release fails, the original data needs to be locally recorded and the secondary processing capability of the local original data is provided. When the data is released, the source of the data to be synchronized needs to be judged, if the data is sourced from other centers, the data is not processed, and the formation of data cycle synchronization is avoided.
Each center is newly added with one (or more) synchronous data subscription service for subscribing and consuming the cross-center synchronous Topic data of other centers. When the synchronous Topic of other centers has data, the current center can directly acquire the synchronous data through the subscription service and process the synchronous data to realize the data cross-center circulation.
When the current center processes synchronous data, a synchronous mark needs to be added in the data to identify the data source side as other centers, so that the data cycle synchronization is avoided; if the processing fails, the consumption progress of the cross-center synchronous data is still ensured not to be affected so as not to affect the operation of the center for issuing the data. For this reason, data that fails to be processed requires local caching and has the ability to initiate the process again.
Of course, the architecture provided by the embodiment of the present application is not limited to the structures shown in fig. 1 and 2, and the embodiment of the present application is not limited thereto. In order to further explain the technical solution provided by the embodiments of the present application, the following details are described with reference to the accompanying drawings and the detailed description. Although embodiments of the present application provide the method operational steps shown in the following embodiments or figures, more or fewer operational steps may be included in the method, either on a routine or non-inventive basis. In steps where there is logically no necessary causal relationship, the execution order of the steps is not limited to the execution order provided by the embodiments of the present application.
The prior art cross-center synchronization method performs synchronization only at the database level. However, because the synchronous data is not only stored in the database, but also needs to refresh the service layer, the traditional scheme of only carrying out data synchronization on the database cannot solve the problem of refreshing the service layer of other centers, if the database and the service layer are synchronous at the same time, the problem of center synchronization can be solved, but the resource cost such as bandwidth and the like of data crossing the center is overlarge due to two synchronization rules, and once one synchronization has a problem, the problem of inconsistent data between the database and the service layer in the center is easily caused.
In order to solve the above problems, the embodiment of the present application provides a data synchronization method across systems. Referring to fig. 3, a flow chart of a cross-system data synchronization method according to an embodiment of the present application is shown, and the method may be applied to the scenario shown in fig. 1, and the flow chart of the method is described as follows.
Step 301, a first server acquires a first synchronization message from a message queue of a second server.
The first server and the second server are servers in different systems, the first synchronization message comprises first synchronization data, and the second server stores first processing result data of the first synchronization data.
Step 302, the first server parses the first synchronization message to obtain first synchronization data.
Step 303, the first server takes the target object including the first synchronization data of the first server as target synchronization data.
Step 304, the first server processes the target synchronous data, and determines and stores first processing result data of the target synchronous data.
Embodiments of the present application relate to two or more systems, each including at least one server. The first server and the second server are servers in different systems. After the second server receives the first synchronous data, the first synchronous data is processed to obtain first processing result data, the second server stores the first processing result data, and a first synchronous message is generated according to the first synchronous data and is cached in a message queue. The first server acquires a first synchronous message from a message queue of the second server, and analyzes the first synchronous message to obtain first synchronous data. And taking the target object including the first synchronous data of the first server as the target synchronous data of the first server. And processing the target synchronous data, and determining and storing first processing result data of the target synchronous data. In the embodiment of the application, the second server directly synchronizes unprocessed original data, namely the first synchronization data, and directly transmits the first synchronization data to the service layer of the first server for data processing by means of the message middleware, so that the data synchronization requirement between the service layer of the first server and the service layer of the second server is ensured, and the real-time consistency of the data processing is ensured.
In the embodiment of the application, the server of each central system can be used as a first server or a second server. That is, the server of each central system can be used as the issuer of the synchronous data and also can be used as the receiver of the synchronous data.
When the server is used as the publisher of the synchronous data, the data synchronization method in the embodiment of the application further comprises the following steps:
the method comprises the steps that a first server receives a data synchronization request, wherein the data synchronization request comprises second synchronization data;
the first server processes the second synchronous data, and determines and stores second processing result data of the second synchronous data;
the first server generates a second synchronous message according to the second synchronous data;
and the first server adds the second synchronous message into a message queue of the first server.
In the implementation process, after the first server receives the data synchronization request, the first server processes second synchronization data in the data synchronization request, and after the second synchronization data is processed, second processing result data of the second synchronization data is stored in a database. The first server generates a second synchronous message from the original data, namely the second synchronous data.
The specific synchronization message is generated according to a data synchronization protocol, and the format of the synchronization message is specified by the data synchronization protocol as shown in fig. 4. In addition to the need of synchronous data, a synchronous message has contents such as synchronous protocol version number, target service name, synchronous initiator and target, data type, data length, etc. The first server generates a target processing business service name according to business scene requirements, selects all center systems or servers in a designated center system as target parties, and encapsulates second synchronous data according to a data synchronous protocol to output a second synchronous message.
And after the first server generates a second synchronous message according to the second synchronous data, adding the second synchronous message into the message middleware of the first server. The servers of the other centers may subscribe to the second synchronization data in the message middleware of the first server.
Further, the first server obtains a first synchronization message from a message queue of the second server, including:
the first server receives synchronous broadcast sent by the second server;
the first server sends a synchronous data acquisition request to the second server according to the synchronous broadcast;
and the first server receives the first synchronous message sent by the second server.
In a specific implementation process, in the embodiment of the present application, the first server may acquire the first synchronization message by broadcasting the first synchronization message to all the servers in all other central systems. The servers of the other central systems serve as the first server to receive the synchronous broadcast sent by the second server. And the first server sends a synchronous data acquisition request to the second server according to synchronous broadcasting, and receives a first synchronous message sent by the second server.
Another way to obtain the first synchronization packet may be that the first server actively sends a synchronization data obtaining request to the second server. Specifically, the first server transmits a synchronous data acquisition request to the second server at a certain frequency. And after receiving the synchronous data acquisition request, the second server sends a first synchronous message in the message middleware to the first server. The manner in which the first server obtains the first synchronization message in the embodiment of the present application is merely exemplary, and is not limited.
Further, after the first server parses the target synchronization packet, the method further includes:
if the first server fails to analyze the target synchronous message, caching the target synchronous message;
after the first server processes the target synchronous data, the method further comprises:
and if the first server fails to process the target synchronous data, caching the target synchronous data.
In the implementation process, after receiving the first synchronization message, the first server analyzes the first synchronization message, and if the analysis is successful, the first synchronization data in the first synchronization message is processed. And judging whether the target object of the first synchronous message comprises the first server according to the content of the first synchronous message. If yes, the first synchronous data is processed as target synchronous data, if not, the first server is not required to synchronize the synchronous data in the first synchronous message, and the first server deletes the first synchronous data.
On the other hand, if the first server does not analyze the first synchronization message successfully, in order to prevent data loss, the first synchronization message is cached first for subsequent unified processing. In addition, if the first server fails to process the target synchronous data, the target synchronous data is cached and waits for subsequent unified processing, so that the consumption progress of the cross-center synchronous data is not influenced, and the center operation of the second server for issuing the synchronous data is not influenced.
Further, the first server processes the target synchronous data, including:
and adding a synchronization identifier into the target synchronization data by the first server.
In the implementation process, when the first server processes the target synchronous data, a synchronous identifier needs to be added in the target synchronous data to identify the synchronous data source party as other central heartbeats, so that data cycle synchronization is avoided.
The above-described flow is described in detail below with specific examples. The scenario in the specific embodiment uses this scheme for a merchant online query service system (hereinafter referred to as MOQS), specifically: the upstream will transmit the merchant data to the MOQS by way of a real-time RPC call, which needs to be binned and reloaded to redis. Because the MOQS is a multi-center deployment, in order to make the deployment of the calling party in the multi-center not concerned, the MOQS adopts the data synchronization method in the embodiment of the application. When a service processing request is received, data processing of the central system is performed firstly, after the data is successfully processed (warehouse entry+Redis loading), the data is synchronized to other central systems through a message middleware, and the data processing is performed by the other central systems. The specific process comprises the following steps:
the message queue of each central system adds a cross-center Topic, and a new synchronization subscription service subscribes to the cross-center Topic.
After receiving the real-time synchronization request, the single-center system processes the data of the center system; after the processing is completed, the data and the synchronization parameters are transmitted to the synchronous release service, and the synchronous data in the scene is the original real-time synchronous request data.
The synchronous data release service checks synchronous data, packs the data which is not from the synchronization of other central systems according to a trans-central protocol contract format, and releases the data to a message queue of the central system.
The synchronous subscription service of the rest center systems acquires the synchronous data of the current center system, unpacks the message according to the protocol requirement, and caches the synchronous message into a local file if the unpacking fails. To avoid increasing the network overhead across the central system, synchronization data is acquired in bulk each time.
And after unpacking, judging the source and the target, processing synchronous data of which the source is not the central system and the target is the central system, and discarding other data.
And taking out the synchronous data in the message, calling according to the target service name in the message, and obtaining a calling result.
If the call fails, the synchronous message is cached in the local file, and unified processing is carried out after the abnormal situation is processed; if the call is successful, the current synchronous message is successful, and the synchronization among the data centers is completed.
And aiming at the transaction with processing failure, providing a manual redoing mode, and carrying out secondary processing after the inter-center upgrading window period is closed or the abnormal problem is repaired, so as to finish failure data compensation.
The following is an embodiment of the device according to the present application, and for details of the device embodiment that are not described in detail, reference may be made to the above-described one-to-one embodiment of the method.
Referring to fig. 5, a block diagram of a cross-system data synchronization device according to an embodiment of the application is shown. The cross-system data synchronization device is implemented as all or part of a server in the central system of fig. 1 by hardware or a combination of hardware and software. The device comprises:
a transceiver 501, configured to obtain a first synchronization packet from a message queue of a second server, where the first server and the second server are servers in different systems, the first synchronization packet includes first synchronization data, and the second server stores first processing result data of the first synchronization data;
an parsing unit 502, configured to parse the first synchronization message and obtain first synchronization data;
a judging unit 503, configured to take the target object including the first synchronization data of the first server as target synchronization data;
and the processing unit 504 is configured to process the target synchronization data, and determine and store first processing result data of the target synchronization data.
Optionally, the transceiver unit 501 is specifically configured to:
receiving synchronous broadcast sent by the second server;
according to the synchronous broadcast, sending a synchronous data acquisition request to the second server;
and receiving the first synchronous message sent by the second server.
Optionally, the apparatus further includes a buffer unit 505, configured to:
if the analysis of the target synchronous message fails, caching the target synchronous message;
and if the processing of the target synchronous data fails, caching the target synchronous data.
Optionally, the processing unit 504 is specifically configured to:
and adding a synchronous identifier into the target synchronous data.
Optionally, the transceiver unit 501 is further configured to receive a data synchronization request, where the data synchronization request includes second synchronization data;
the processing unit 504 is further configured to process the second synchronization data, determine and store second processing result data of the second synchronization data; generating a second synchronous message according to the second synchronous data; and adding the second synchronous message into a message queue of the first server.
Referring to fig. 6, a block diagram of a server according to an embodiment of the present application is shown. The server 1000 is implemented as a server in the central system of fig. 1. Specifically, the present application relates to a method for manufacturing a semiconductor device.
The server 1000 includes a Central Processing Unit (CPU) 1001, a system memory 1004 including a Random Access Memory (RAM) 1002 and a Read Only Memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the central processing unit 1001. The server 1000 also includes a basic input/output system (I/O system) 1006 for aiding in the transfer of information between the various devices within the computer, and a mass storage device 1007 for storing an operating system 1013, application programs 1014, and other program modules 1015.
The basic input/output system 1006 includes a display 1008 for displaying information and an input device 1009, such as a mouse, keyboard, etc., for a user to input information. Wherein the display 1008 and the input device 1009 are connected to the central processing unit 1001 via an input output controller 1010 connected to a system bus 1005. The basic input/output system 1006 may also include an input/output controller 1010 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 1010 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 1007 is connected to the central processing unit 1001 through a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1007 and its associated computer-readable media provide non-volatile storage for the server 1000. That is, the mass storage device 1007 may include a computer readable medium (not shown) such as a hard disk or CD-ROM drive.
The computer readable medium may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the one described above. The system memory 1004 and mass storage devices 1007 described above may be collectively referred to as memory.
The server 1000 may also operate in accordance with various embodiments of the present application, through a network, such as the internet, to remote computers connected to the network. I.e. the server 1000 may be connected to the network 1012 via a network interface unit 1011 connected to said system bus 1005, or alternatively, the network interface unit 1011 may be used to connect to other types of networks or remote computer systems (not shown).
The memory also includes one or more programs stored in the memory, the one or more programs containing instructions for performing the cross-system data synchronization method provided by the embodiments of the present application.
Those skilled in the art will appreciate that all or part of the steps in the check-in method of the above embodiments may be implemented by a program for instructing relevant hardware, and the program may be stored in a computer readable storage medium, where the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps in the check-in method of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.
Claims (8)
1. A method of cross-system data synchronization, the method comprising:
the method comprises the steps that a first server obtains a first synchronous message from a message queue of a second server, the first server and the second server are servers in different systems, the first server subscribes to data in the message queue of the second server, and the first synchronous message comprises first synchronous data; the first synchronous message is encapsulated according to a synchronous protocol; the first synchronization message includes a synchronization target, and further includes one or more of the following: the method comprises the steps of first synchronous data, a synchronous protocol version number, a target service name, a synchronous initiator, a data type and a data length;
the first server analyzes the first synchronous message and acquires first synchronous data;
the first server takes the target object including first synchronous data of the first server as target synchronous data;
the first server processes the target synchronous data on a service layer, and determines and stores first processing result data of the target synchronous data;
wherein,
the first server processes the target synchronous data, and the method comprises the following steps:
the first server adds a synchronization identifier in the target synchronization data to identify a source side of the target synchronization data as other centers;
after the first server parses the first synchronization message, the method further includes:
if the first server fails to analyze the first synchronous message, caching the first synchronous message;
after the first server processes the target synchronous data, the method further comprises:
and if the first server fails to process the target synchronous data, caching the target synchronous data.
2. The method of claim 1, wherein the first server obtaining the first synchronization message from the message queue of the second server comprises:
the first server receives synchronous broadcast sent by the second server;
the first server sends a synchronous data acquisition request to the second server according to the synchronous broadcast;
and the first server receives the first synchronous message sent by the second server.
3. The method according to any one of claims 1 to 2, further comprising:
the first server receives a data synchronization request, wherein the data synchronization request comprises second synchronization data;
the first server processes the second synchronous data, and determines and stores second processing result data of the second synchronous data;
the first server generates a second synchronous message according to the second synchronous data;
and the first server adds the second synchronous message into a message queue of the first server.
4. A cross-system data synchronization apparatus, the apparatus comprising:
the receiving and transmitting unit is used for acquiring a first synchronous message from a message queue of a second server, wherein the first server and the second server are servers in different systems, the first server subscribes to data in the message queue of the second server, and the first synchronous message comprises first synchronous data; the first synchronous message is encapsulated according to a synchronous protocol; the first synchronization message includes a synchronization target, and further includes one or more of the following: the method comprises the steps of first synchronous data, a synchronous protocol version number, a target service name, a synchronous initiator, a data type and a data length;
the analyzing unit is used for analyzing the first synchronous message and acquiring first synchronous data;
a judging unit, configured to take target object including first synchronization data of the first server as target synchronization data;
the processing unit is used for processing the target synchronous data on a service layer and determining and storing first processing result data of the target synchronous data;
wherein,
the processing unit is specifically configured to: adding a synchronization identifier into the target synchronization data to identify a source side of the target synchronization data as other centers;
the buffer unit is used for buffering the first synchronous message if the analysis of the first synchronous message fails;
the caching unit is further configured to cache the target synchronous data if the target synchronous data fails to be processed.
5. The device according to claim 4, wherein the transceiver unit is specifically configured to:
receiving synchronous broadcast sent by the second server;
according to the synchronous broadcast, sending a synchronous data acquisition request to the second server;
and receiving the first synchronous message sent by the second server.
6. The apparatus according to any one of claim 4 to 5, wherein,
the receiving and transmitting unit is further configured to receive a data synchronization request, where the data synchronization request includes second synchronization data;
the processing unit is further used for processing the second synchronous data, and determining and storing second processing result data of the second synchronous data; generating a second synchronous message according to the second synchronous data; and adding the second synchronous message into a message queue of the first server.
7. A computing device comprising at least one processor, and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the cross-system data synchronization method of any one of claims 1 to 3.
8. A storage medium storing computer instructions which, when run on a computer, cause the computer to perform the cross-system data synchronization method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011031792.0A CN112187916B (en) | 2020-09-27 | 2020-09-27 | Cross-system data synchronization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011031792.0A CN112187916B (en) | 2020-09-27 | 2020-09-27 | Cross-system data synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112187916A CN112187916A (en) | 2021-01-05 |
| CN112187916B true CN112187916B (en) | 2023-12-05 |
Family
ID=73945298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011031792.0A Active CN112187916B (en) | 2020-09-27 | 2020-09-27 | Cross-system data synchronization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112187916B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116723204B (en) * | 2023-08-07 | 2024-02-09 | 南京国电南自轨道交通工程有限公司 | Method for synchronizing large-data-volume real-time data among layered systems |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932472A (en) * | 2012-11-13 | 2013-02-13 | 深圳市路友网络科技有限公司 | Data processing method and communication system |
| CN103646353A (en) * | 2013-11-28 | 2014-03-19 | 国家电网公司 | Database synchronization method for substation automated monitoring system |
| CN106254456A (en) * | 2016-08-03 | 2016-12-21 | 深圳市商汤科技有限公司 | A kind of method of data synchronization and data synchronous system |
| CN107612947A (en) * | 2017-11-03 | 2018-01-19 | 北京微视酷科技有限责任公司 | Synchronous communication means and system between a kind of virtual reality device |
| CN107819795A (en) * | 2016-09-12 | 2018-03-20 | 北京金山云网络技术有限公司 | A kind of major-minor server data synchronous method of DNS, apparatus and system |
| CN108256002A (en) * | 2017-12-31 | 2018-07-06 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device, system and server |
| CN108347476A (en) * | 2018-01-22 | 2018-07-31 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device and server |
| CN109101627A (en) * | 2018-08-14 | 2018-12-28 | 交通银行股份有限公司 | heterogeneous database synchronization method and device |
| CN109299056A (en) * | 2018-09-19 | 2019-02-01 | 潍坊工程职业学院 | A kind of method of data synchronization and device based on distributed file system |
| CN109788053A (en) * | 2019-01-04 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Method of data synchronization and system |
| CN110502572A (en) * | 2019-06-26 | 2019-11-26 | 中电万维信息技术有限责任公司 | A kind of method and server that data are synchronous |
| CN110727736A (en) * | 2019-10-24 | 2020-01-24 | 中国建设银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN110795443A (en) * | 2019-10-08 | 2020-02-14 | 中国建设银行股份有限公司 | Method, device, equipment and computer readable medium for data synchronization |
| CN111309693A (en) * | 2020-02-07 | 2020-06-19 | 北京奇艺世纪科技有限公司 | Data synchronization method, device and system, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9264414B2 (en) * | 2013-03-15 | 2016-02-16 | Microsoft Technology Licensing, Llc | Retry and snapshot enabled cross-platform synchronized communication queue |
-
2020
- 2020-09-27 CN CN202011031792.0A patent/CN112187916B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932472A (en) * | 2012-11-13 | 2013-02-13 | 深圳市路友网络科技有限公司 | Data processing method and communication system |
| CN103646353A (en) * | 2013-11-28 | 2014-03-19 | 国家电网公司 | Database synchronization method for substation automated monitoring system |
| CN106254456A (en) * | 2016-08-03 | 2016-12-21 | 深圳市商汤科技有限公司 | A kind of method of data synchronization and data synchronous system |
| CN107819795A (en) * | 2016-09-12 | 2018-03-20 | 北京金山云网络技术有限公司 | A kind of major-minor server data synchronous method of DNS, apparatus and system |
| CN107612947A (en) * | 2017-11-03 | 2018-01-19 | 北京微视酷科技有限责任公司 | Synchronous communication means and system between a kind of virtual reality device |
| CN108256002A (en) * | 2017-12-31 | 2018-07-06 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device, system and server |
| CN108347476A (en) * | 2018-01-22 | 2018-07-31 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device and server |
| CN109101627A (en) * | 2018-08-14 | 2018-12-28 | 交通银行股份有限公司 | heterogeneous database synchronization method and device |
| CN109299056A (en) * | 2018-09-19 | 2019-02-01 | 潍坊工程职业学院 | A kind of method of data synchronization and device based on distributed file system |
| CN109788053A (en) * | 2019-01-04 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Method of data synchronization and system |
| WO2020140661A1 (en) * | 2019-01-04 | 2020-07-09 | 深圳壹账通智能科技有限公司 | Data synchronization method and system |
| CN110502572A (en) * | 2019-06-26 | 2019-11-26 | 中电万维信息技术有限责任公司 | A kind of method and server that data are synchronous |
| CN110795443A (en) * | 2019-10-08 | 2020-02-14 | 中国建设银行股份有限公司 | Method, device, equipment and computer readable medium for data synchronization |
| CN110727736A (en) * | 2019-10-24 | 2020-01-24 | 中国建设银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN111309693A (en) * | 2020-02-07 | 2020-06-19 | 北京奇艺世纪科技有限公司 | Data synchronization method, device and system, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Kun Qian ; Zhongzhi Luan ; Zuowei Zhang,et.al..Data Analysis and Synchronization on Inter-Continent Data Placement Laboratory.《2015 International Conference on Cloud Computing and Big Data (CCBD)》.2015,全文. * |
| 异构网络的时间同步技术研究;莫皓颖;户江民;张伟;;中国新通信(第20期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112187916A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101863398B1 (en) | Method and system for synchronization mechanism on multi-server reservation system | |
| CN109976667B (en) | Mirror image management method, device and system | |
| EP2180663B1 (en) | Trickle sync protocol | |
| CN109783151B (en) | Method and device for rule change | |
| EP3364631A1 (en) | Dynamic orchestration of microservices | |
| US20200213385A1 (en) | Offline mobile data storage system and method | |
| US20160241441A1 (en) | Method and apparatus for changing configurations | |
| CN110351366A (en) | A kind of service scheduling method of Internet application, system and computer readable storage medium | |
| CN111427701A (en) | Workflow engine system and business processing method | |
| CN104468299A (en) | Enterprise service bus system based on user rule | |
| CN115114044A (en) | Message pushing method, device, equipment and medium | |
| CN115268949A (en) | Mirror preheating method, device, equipment and storage medium | |
| CN112187916B (en) | Cross-system data synchronization method and device | |
| CN110798495B (en) | Method and server for end-to-end message push in cluster architecture mode | |
| CN113556387A (en) | Edge gateway control method, system, device, electronic equipment and storage medium | |
| CN113079098A (en) | Method, device, equipment and computer readable medium for updating route | |
| CN115629909A (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN113553206B (en) | Data event execution method and device, electronic equipment and computer readable medium | |
| CN110324425B (en) | Hybrid cloud transaction route processing method and device | |
| US9749426B2 (en) | Method, system, and apparatus for agent-based architecture for integrated mobile applications | |
| CN110661857B (en) | Data synchronization method and device | |
| CN113472638A (en) | Edge gateway control method, system, device, electronic equipment and storage medium | |
| CN113722115A (en) | Method, device, equipment and computer readable medium for calling interface | |
| CN112052104A (en) | Message queue management method based on multi-computer-room realization and electronic equipment | |
| CN114884956B (en) | Method and device for realizing multi-cluster architecture and multi-cluster architecture system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |