CN111708787A - Multi-center service data management system - Google Patents
Multi-center service data management system Download PDFInfo
- Publication number
- CN111708787A CN111708787A CN202010376839.0A CN202010376839A CN111708787A CN 111708787 A CN111708787 A CN 111708787A CN 202010376839 A CN202010376839 A CN 202010376839A CN 111708787 A CN111708787 A CN 111708787A
- Authority
- CN
- China
- Prior art keywords
- data
- service data
- data center
- center
- update log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013523 data management Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims description 30
- 230000001360 synchronised effect Effects 0.000 claims description 14
- 230000010076 replication Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
- G06F16/2386—Bulk updating operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention discloses a multi-center service data management system. The system comprises: the system comprises a first front-end server cluster, a first data center, a second front-end server cluster and a second data center, wherein the first front-end server cluster is used for receiving a first service data writing request of a target application and sending the first service data writing request to the first data center; the first data center is used for performing first write operation according to the first service data write-in request, writing a first data updating log and synchronizing the first service data to the second data center according to the first data updating log; the second front-end server cluster is used for receiving a second service data writing request of the target application and sending the second service data writing request to the second data center; and the second data center is used for performing second write operation according to the second service data write-in request, writing a second data updating log and synchronizing the second service data to the first data center according to the second data updating log.
Description
Technical Field
The application relates to the technical field of computers, in particular to a multi-center service data management system.
Background
At present, with the development of computer technology, the application of distributed databases is more and more extensive. The distributed database not only solves the problem that the traditional database is difficult to expand on the storage space and the bottleneck problem of highly concurrent access, but also well ensures the integrity and the safety of data through a data redundancy method, provides strong disaster tolerance measures and can quickly recover the data after major problems occur.
In the related art, the distributed database may implement data redundancy in a "one-master-multi-standby" manner, and the "one-master-multi-standby" may be a master node and at least one backup node. The master node may have data read-write capability, and the backup node may synchronize data of the master node and may have data read capability. In order to facilitate uniform management of business data of each area by an enterprise, a distributed database is usually deployed based on a single server cluster and multiple data nodes, that is, a single server cluster includes a main node and multiple backup nodes.
However, when the database is accessed across areas, the deployment mode may cause problems of excessive access delay, low access efficiency and the like.
Disclosure of Invention
The embodiment of the specification provides a multi-center management system to solve the problems that in the prior art, when a database is accessed in a cross-region mode, access delay is too large and access efficiency is low.
The embodiment of the specification adopts the following technical scheme:
a multi-center business data management system comprising:
a first cluster of front-end servers, a first data center, a second cluster of front-end servers, and a second data center, wherein,
the first front-end server cluster is used for receiving a first service data writing request of a target application and sending the first service data writing request to the first data center;
the first data center is used for performing first write operation according to the first service data write-in request, writing a first data update log, and synchronizing first service data to the second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
the second front-end server cluster is used for receiving a second service data writing request of the target application and sending the second service data writing request to the second data center;
the second data center is used for performing second write operation according to the second service data write-in request, writing a second data update log, and synchronizing second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
Optionally, in an implementation, the first data center issues the first data update log, so that the second data center subscribes to the first data update log and writes the first service data into the second data center based on the first data update log, so as to synchronize the first service data;
and the second data center issues the second data update log so that the first data center subscribes to the second data update log and writes the first service data into the first data center based on the second data update log to synchronize the second service data.
Optionally, in an embodiment, the first data center includes a first main node and a first backup node, and the second data center includes a second main node and a second backup node, wherein,
the first master node is configured to perform a first write operation and write a first data update log according to the first service data write request, issue a first write operation in the first data update log, subscribe to a second write operation in the second data update log issued by the second data center, and execute the subscribed second write operation to synchronize the second service data;
the first backup node is used for synchronizing a first data update log in the first main node;
the second master node is configured to perform a second write operation and write a second data update log according to the second service data write request, issue a second write operation in the second data update log, subscribe to a first write operation in the first data update log issued by the first data center, and execute the subscribed first write operation to synchronize the first service data;
and the second backup node is used for synchronizing a second data update log in the second main node.
Optionally, in an implementation manner, the first data center further includes a first identifier adding module, configured to add a subscribed identifier to a second write operation that is subscribed after subscribing to the second write operation in the second data update log that is published by the second data center;
the second data center further comprises a second identifier adding module, which is used for adding a subscribed identifier to the subscribed first write operation after subscribing the first write operation in the first data update log issued by the first data center.
Optionally, in an embodiment, the first backup node is configured to synchronize the first data update log in the first master node based on a synchronous stream replication manner;
and the second backup node is used for synchronizing a second data update log in the second main node based on a synchronous stream replication mode.
Optionally, in an embodiment, the system further includes:
the first master node fault processing module is used for switching the first backup node into a master node and switching an IP address of the first master node of the first front-end server cluster accessing the first data center to be the IP address of the first backup node when the first master node fails;
and the second main node fault processing module is used for switching the second backup node to the main node and switching the IP address of the second main node of the second front-end server cluster accessing the second data center to be the IP address of the second backup node when the second main node fails. A
Optionally, in an embodiment, the system further includes:
the first data center fault processing module is used for redirecting the domain name address of a first service data write-in request sent by the target application to the domain name address of the second data center under the condition that the first data center has a fault;
and the second data center fault processing module is used for redirecting the domain name address of the second service data write-in request sent by the target application to the domain name address of the first data center under the condition that the second data center has a fault.
Optionally, in an implementation manner, the first front-end server cluster is further configured to receive a third service data reading request of a target application, and send the third service data reading request to the first data center;
the first data center is further configured to perform a first reading operation according to the third service data reading request, and return third service data read based on the first reading operation to the target application.
Optionally, in an implementation manner, the second front-end server cluster is further configured to receive a fourth service data reading request of the target application, and send the fourth service data reading request to the second data center;
the second data center is further configured to perform a second reading operation according to the fourth service data reading request, and return fourth service data read based on the second reading operation to the target application.
A multi-center service data management method based on the multi-center service data management system comprises the following steps:
receiving a first service data writing request of a target application through a first front-end server cluster, and sending the first service data writing request to a first data center;
performing a first write operation and writing a first data update log according to the first service data write request through a first data center;
synchronizing, by the first data center, first business data to a second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
receiving a second service data writing request of the target application through a second front-end server cluster, and sending the second service data writing request to a second data center;
performing a second write operation and writing a second data update log according to the second service data write request through the second data center;
synchronizing, by the second data center, second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
An electronic device, comprising: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of any one of the multi-center service data management methods when being executed by the processor.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the multicenter business data management methods.
The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects:
the first front-end server cluster and the second front-end server cluster are used for receiving a first service data writing request and a second service data writing request sent by a target application, and then the first service data writing request and the second service data writing request are respectively sent to the first data center and the second data center to write data.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic structural diagram of a multi-center service data management system provided in an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of another multi-center service data management system provided in an embodiment of the present specification;
fig. 3 is a schematic flowchart of a method for managing multi-center service data according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
In the prior art, in order to facilitate uniform management of service data of each area by an enterprise, a distributed database is generally deployed based on a single server cluster and multiple data nodes, that is, the single server cluster includes a main node and multiple backup nodes. However, when the database is accessed across areas, the deployment mode may cause problems of excessive access delay, low access efficiency and the like.
In order to solve the above technical problem, an embodiment of the present specification provides a multi-center service data management system, which is used for managing service data of multiple data centers. The schematic structural diagram of the multi-center service data management system is shown in fig. 1, and includes: a first front-end server cluster 101, a first data center 102, a second front-end server cluster 103, and a second data center 104.
The first front-end server cluster 101 is configured to receive a first service data write request of a target application, and send the first service data write request to a first data center 102;
the first data center 102 is configured to perform a first write operation according to the first service data write request, write a first data update log, and synchronize the first service data to the second data center 104 according to the first data update log;
the second front-end server cluster 103 is configured to receive a second service data write request of a target application, and send the second service data write request to the second data center 104;
the second data center 104 is configured to perform a second write operation according to the second service data write request and write a second data update log, and synchronize the second service data to the first data center 102 according to the second data update log.
In practical applications, the first service data write request and the second service data write request may be requests issued by users through target applications to write service data into the data center. In this embodiment, the service data write request may include: add service data requests, delete service data requests, and modify service data requests. The first write operation may include: adding, deleting and modifying the service data, the second write operation may also include: adding service data, deleting service data and modifying service data.
In this description embodiment, the first data update log may include recording a first write operation and recording first traffic data written according to the first write operation, and the second data update log may include recording a second write operation and recording second traffic data written according to the second write operation.
To facilitate understanding of how to record the write operation, for implementing the write operation of the data by using a Structured Query Language (SQL) statement, one SQL statement of a record in the data update log may be one write operation of the record, for example, SQL statement "insert into XS (num, name) values ('032', 'wang bi')" indicating that the service data "032" "wang bi" is added to the table XS, and it is understood that "adding the service data" 032 "" wang bi "into the table XS" may be a write operation, "032" "wang bi" may be service data.
By recording the first write operation and the first service data in the first data update log and recording the second write operation and the second service data in the second service data update log, data synchronization between the first data center 102 and the second data center 104 can be facilitated.
In one or more embodiments of the present description, to implement data synchronization between the first data center 102 and the second data center 104, the first data center 102 and the second data center 104 may synchronize data in a publish-subscribe mode, which may specifically include:
the first data center 102 issues a first data update log so that the second data center 104 subscribes to the first data update log and writes the first service data into the second data center 104 based on the first data update log to synchronize the first service data;
the second data center 104 issues the second data update log so that the first data center 102 subscribes to the second data update log and writes the first service data to the first data center 102 based on the second data update log to synchronize the second service data.
Through data synchronization, the first data center 102 and the second data center 104 can have equal total amount of service data, and when a target application sends a service data write request through any front-end server cluster, service data can be written according to the service data write request, so that data access efficiency can be improved.
In one or more embodiments of the present description, the user may issue a service data read request through the target application in addition to a service data write request, where the service data read request may be understood as a query service data request, and then,
the first front-end server cluster 101 is further configured to receive a third service data reading request of the target application, and send the third service data reading request to the first data center 102;
the first data center 102 is further configured to perform a first reading operation according to the third service data reading request, and return third service data read based on the first reading operation to the target application;
the second front-end server cluster 103 is further configured to receive a fourth service data reading request of the target application, and send the fourth service data reading request to the second data center 104;
the second data center 104 is further configured to perform a second reading operation according to the fourth service data reading request, and return fourth service data read based on the second reading operation to the target application.
The first reading operation may include querying the service data, and the second reading operation may also include querying the service data.
In the embodiment of the present specification, a first front-end server cluster 101 and a second front-end server cluster 103 receive a first service data write-in request and a second service data write-in request sent by a target application, and then send the first service data write-in request and the second service data write-in request to a first data center 102 and a second data center 104, respectively, to write data, and can write data into a database for the same target application, so that data can be written into the database through multiple receiving channels and multiple data centers.
In practical applications, as described above, the distributed database may include a plurality of data nodes to implement data redundancy, and a data redundancy method may well ensure data integrity and security, and provide a strong disaster tolerance measure, then in one or more embodiments of the present disclosure, the first data center 102 and the second data center 104 may include a plurality of data nodes, respectively, as shown in the schematic structural diagram of the multi-center service data management system shown in fig. 2, the first data center 102 may include a first master node and a first backup node, and the second data center 104 may include a second master node and a second backup node, where,
the first master node is used for performing a first write operation according to the first service data write-in request, writing a first data update log, issuing the first write operation in the first data update log, subscribing to a second write operation in a second data update log issued by the second data center 104, and executing the subscribed second write operation to synchronize the second service data;
the first backup node is used for synchronizing a first data update log in the first main node;
the second host node is configured to perform a second write operation and write a second data update log according to the second service data write request, issue the second write operation in the second data update log, subscribe to a first write operation in the first data update log issued by the first data center 102, and execute the subscribed first write operation to synchronize the first service data;
and the second backup node is used for synchronizing a second data update log in the second main node.
In actual practice, the first write operation that may be recorded in the first data update log, as described above. Then, when data synchronization is implemented, the first data center 102 may synchronize only the first write operation to the second data center 104, and correspondingly, the second data center 104 may synchronize only the second write operation to the first data center 102, and after the first data center 102 performs the synchronized second write operation, synchronization of the second service data may be implemented, and similarly, after the second data center 104 performs the synchronized first write operation, synchronization of the first service data may be implemented.
In one or more embodiments of the present disclosure, in order to avoid that the first data center and the second data center publish synchronized service data or publish a subscribed read operation when performing data synchronization, the first data center 102 may further include a first identifier adding module, configured to add a subscribed identifier to a subscribed second write operation after subscribing to a second write operation in a second data update log published by the second data center 104. Correspondingly, the second data center 104 may further include a second identifier adding module, configured to add the subscribed identifier to the subscribed first write operation after subscribing to the first write operation in the first data update log published by the first data center.
In one or more embodiments of the present disclosure, when the backup node synchronizes the data update log of the primary node, the synchronization may be implemented based on a synchronous stream replication, and specifically, the first backup node may be configured to synchronize a first data update log in the first primary node based on the synchronous stream replication, and the second backup node may be configured to synchronize a second data update log in the second primary node based on the synchronous stream replication.
In practical applications, as described above, a user may send a service data write request and a service data read request through a target application, in this embodiment of this specification, the first master node may be further configured to perform a read operation according to a first service data read request, the first backup node may be also configured to perform a read operation according to the first service data read request, the second master node may be further configured to perform a read operation according to a second service data read request, and the second backup node may be also configured to perform a read operation according to the second service data read request.
In the embodiment of the present specification, the master node may implement a write operation and a read operation, and implement a read operation by using the backup node, so as to share the pressure of the master node and improve the utilization rate of resources.
In practical application, because the first backup node and the second backup node can respectively synchronize the first data update log of the first main node and the second data update log of the second main node, when the first main node and the second main node fail and cannot provide services to the outside, the first backup node and the second backup node can respectively replace the first main node and the second main node, thereby avoiding influence on use of a user.
In one or more embodiments of the present specification, in order to perform switching between the first backup node and the second backup node when the first master node and the second master node fail, the multi-center service data management system may further include:
the first master node failure processing module is configured to switch the first backup node to a master node when the first master node fails, and switch an IP address of the first master node, where the first front-end server cluster 101 accesses the first data center 102, to be the IP address of the first backup node;
and a second main node failure processing module, configured to switch the second backup node to the main node when the second main node fails, and switch the IP address of the second main node, where the second front-end server cluster 103 accesses the second data center 104, to be the IP address of the second backup node.
By switching the IP address of the main node of the front-end server cluster accessing data center to the IP address of the backup node, the writing operation of the service data and the like can be realized through the backup node when the main node fails. It can be understood that the backup node is switched to the main node, that is, the backup node has the functions of the main node, so that the influence on the use of the user caused by the failure of the main node is avoided.
In an actual situation, in addition to a possible failure of a master node of a data center, a failure of the entire data center may also occur, and in order to not affect the use of a user when the data center fails, that is, to read and write service data through a target application, in one or more embodiments of the present specification, the multi-center service data management system may further include:
a first data center fault processing module, configured to redirect a domain name address of a first service data write request sent by a target application to a domain name address of a second data center 104 when a fault occurs in the first data center 102;
and the second data center fault processing module is configured to redirect the domain name address of the second service data write request sent by the target application to the domain name address of the first data center 102 when the second data center 104 fails.
It is understood that each data center may correspond to a different domain name address, and in case of a failure of the first data center 102, the domain name address of the first service data write request is redirected to the domain name address of the second data center 104, so that a write operation and a second data update log can be performed in the second data center 104 based on the first service data write request, and likewise, in case of a failure of the second data center 104, the domain name address of the second service data write request is redirected to the domain name address of the first data center 102, so that a write operation and a first data update log can be performed in the second data center 104 based on the first service data write request, and so on.
In practical application, after the first data center with the fault is repaired, the service data written into the second data center can be synchronized to the first data center after the fault occurs and the fault is repaired, and then the domain name address of the first service data writing request is cancelled to be redirected to the domain name address of the second data center 104, so that the function of the first data center is restored, and the operation of the whole multi-center service data management is ensured.
The embodiment of the present specification further provides a method for managing multi-center service data based on the multi-center service data management system, based on the same inventive concept. As shown in fig. 3, a flow diagram of a method for managing multi-center service data specifically includes:
In one embodiment, the first data update log is published through the first data center, so that the second data center subscribes to the first data update log and writes the first service data to the second data center based on the first data update log to synchronize the first service data;
and the second data center issues the second data update log so that the first data center subscribes to the second data update log and writes the first service data into the first data center based on the second data update log to synchronize the second service data.
In one embodiment, the first data center includes a primary master node and a first backup node, and the second data center includes a secondary master node and a second backup node, wherein,
performing, by a first host node, a first write operation and writing a first data update log according to the first service data write request, issuing a first write operation in the first data update log, subscribing to a second write operation in the second data update log issued by the second data center, and executing the subscribed second write operation to synchronize the second service data;
synchronizing a first data update log in the first main node through a first backup node;
performing, by a second host node, a second write operation and writing a second data update log according to the second service data write request, issuing a second write operation in the second data update log, subscribing to a first write operation in the first data update log issued by the first data center, and executing the subscribed first write operation to synchronize the first service data;
synchronizing, by a second backup node, a second data update log in the second master node.
In one embodiment, the method further comprises:
after subscribing a second write operation in the second data update log issued by the second data center, adding a subscribed identifier to the subscribed second write operation;
and after subscribing the first write operation in the first data update log published by the first data center, adding a subscribed identifier to the subscribed first write operation.
In one embodiment, a first data update log in a first main node is synchronized by a first backup node based on a synchronous stream replication mode;
and synchronizing a second data update log in the second main node through a second backup node based on a synchronous stream replication mode.
In one embodiment, the method further comprises:
under the condition that the first main node fails, switching the first backup node into a main node, and switching an IP address of the first main node of the first front-end server cluster accessing the first data center to be the IP address of the first backup node;
and under the condition that the second main node fails, switching the second backup node to the main node, and switching the IP address of the second main node in the second data accessed by the second front-end server cluster to be the IP address of the second backup node.
In one embodiment, the method further comprises:
under the condition that the first data center has a fault, redirecting a domain name address of a first service data write request sent by the target application to a domain name address of the second data center;
and under the condition that the second data center has a fault, redirecting the domain name address of the second service data write request sent by the target application to the domain name address of the first data center.
In one embodiment, a third service data reading request of a target application is received through a first front-end server cluster and sent to a first data center; and performing a first reading operation according to the third service data reading request through a first data center, and returning third service data read based on the first reading operation to the target application.
In one embodiment, a second front-end server cluster receives a fourth service data reading request of the target application, and sends the fourth service data reading request to the second data center; and performing, by a second data center, a second reading operation according to the fourth service data reading request, and returning fourth service data read based on the second reading operation to the target application.
In the embodiment of the specification, a first business data writing request and a second business data writing request sent by a target application are received by a first front-end server cluster and a second front-end server cluster, and then are respectively sent to a first data center and a second data center to write data, so that the business data writing requests sent by the same target application can be written, data can be written into a database through multiple receiving channels and multiple data centers, and the access speed and the access efficiency of data in the database can be improved compared with the database deployment mode adopting a single-service cluster in the prior art.
An embodiment of this specification further provides an electronic device, and referring to fig. 4, in a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (peripheral component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
And the processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the application multi-center service data management system on a logic level. A processor executing the program stored in the memory and configured to perform at least the following:
receiving a first service data writing request of a target application through a first front-end server cluster, and sending the first service data writing request to a first data center;
performing a first write operation and writing a first data update log according to the first service data write request through a first data center;
synchronizing, by the first data center, first business data to a second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
receiving a second service data writing request of the target application through a second front-end server cluster, and sending the second service data writing request to a second data center;
performing a second write operation and writing a second data update log according to the second service data write request through the second data center;
synchronizing, by the second data center, second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
The method performed by the multi-center service data management system according to the embodiment shown in fig. 1 of the present application may be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a network Processor (FP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The electronic device may further execute the method executed by the multi-center service data management system in fig. 1, and implement the functions of the multi-center service data management system in the embodiment shown in fig. 1, which are not described herein again in this specification.
This specification embodiment also proposes a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by an electronic device including a plurality of application programs, enable the electronic device to perform the method performed by the multi-center business data management system in the embodiment shown in fig. 1, and at least to perform:
receiving a first service data writing request of a target application through a first front-end server cluster, and sending the first service data writing request to a first data center;
performing a first write operation and writing a first data update log according to the first service data write request through a first data center;
synchronizing, by the first data center, first business data to a second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
receiving a second service data writing request of the target application through a second front-end server cluster, and sending the second service data writing request to a second data center;
performing a second write operation and writing a second data update log according to the second service data write request through the second data center;
synchronizing, by the second data center, second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A multi-center business data management system, comprising:
a first cluster of front-end servers, a first data center, a second cluster of front-end servers, and a second data center, wherein,
the first front-end server cluster is used for receiving a first service data writing request of a target application and sending the first service data writing request to the first data center;
the first data center is used for performing first write operation according to the first service data write-in request, writing a first data update log, and synchronizing first service data to the second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
the second front-end server cluster is used for receiving a second service data writing request of the target application and sending the second service data writing request to the second data center;
the second data center is used for performing second write operation according to the second service data write-in request, writing a second data update log, and synchronizing second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
2. The system of claim 1,
the first data center issues the first data updating log so that the second data center subscribes to the first data updating log and writes the first service data into the second data center based on the first data updating log so as to synchronize the first service data;
and the second data center issues the second data update log so that the first data center subscribes to the second data update log and writes the first service data into the first data center based on the second data update log to synchronize the second service data.
3. The system of claim 1,
the first data center comprises a first main node and a first backup node, the second data center comprises a second main node and a second backup node, wherein,
the first master node is configured to perform a first write operation and write a first data update log according to the first service data write request, issue a first write operation in the first data update log, subscribe to a second write operation in the second data update log issued by the second data center, and execute the subscribed second write operation to synchronize the second service data;
the first backup node is used for synchronizing a first data update log in the first main node;
the second master node is configured to perform a second write operation and write a second data update log according to the second service data write request, issue a second write operation in the second data update log, subscribe to a first write operation in the first data update log issued by the first data center, and execute the subscribed first write operation to synchronize the first service data;
and the second backup node is used for synchronizing a second data update log in the second main node.
4. The system of claim 3,
the first data center further comprises a first identifier adding module, which is used for adding a subscribed identifier to the subscribed second write operation after subscribing the second write operation in the second data update log issued by the second data center;
the second data center further comprises a second identifier adding module, which is used for adding a subscribed identifier to the subscribed first write operation after subscribing the first write operation in the first data update log issued by the first data center.
5. The system of claim 3,
the first backup node is used for synchronizing a first data update log in the first main node based on a synchronous stream replication mode;
and the second backup node is used for synchronizing a second data update log in the second main node based on a synchronous stream replication mode.
6. The system of claim 3, wherein the system further comprises:
the first master node fault processing module is used for switching the first backup node into a master node and switching an IP address of the first master node of the first front-end server cluster accessing the first data center to be the IP address of the first backup node when the first master node fails;
and the second main node fault processing module is used for switching the second backup node to the main node and switching the IP address of the second main node of the second front-end server cluster accessing the second data center to be the IP address of the second backup node when the second main node fails.
7. The system of claim 1, wherein the system further comprises:
the first data center fault processing module is used for redirecting the domain name address of a first service data write-in request sent by the target application to the domain name address of the second data center under the condition that the first data center has a fault;
and the second data center fault processing module is used for redirecting the domain name address of the second service data write-in request sent by the target application to the domain name address of the first data center under the condition that the second data center has a fault.
8. The system of claim 1,
the first front-end server cluster is also used for receiving a third service data reading request of a target application and sending the third service data reading request to the first data center;
the first data center is further configured to perform a first reading operation according to the third service data reading request, and return third service data read based on the first reading operation to the target application.
9. The system of claim 1,
the second front-end server cluster is further configured to receive a fourth service data reading request of the target application, and send the fourth service data reading request to the second data center;
the second data center is further configured to perform a second reading operation according to the fourth service data reading request, and return fourth service data read based on the second reading operation to the target application.
10. A multi-center service data management method based on the multi-center service data management system of claim 1, wherein the multi-center service data management method comprises:
receiving a first service data writing request of a target application through a first front-end server cluster, and sending the first service data writing request to a first data center;
performing a first write operation and writing a first data update log according to the first service data write request through a first data center;
synchronizing, by the first data center, first business data to a second data center according to the first data update log; the first data update log comprises the first write operation and the first service data written according to the first write operation;
receiving a second service data writing request of the target application through a second front-end server cluster, and sending the second service data writing request to a second data center;
performing a second write operation and writing a second data update log according to the second service data write request through the second data center;
synchronizing, by the second data center, second service data to the first data center according to the second data update log; the second data update log comprises the second writing operation and the second service data written according to the second writing operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010376839.0A CN111708787A (en) | 2020-05-07 | 2020-05-07 | Multi-center service data management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010376839.0A CN111708787A (en) | 2020-05-07 | 2020-05-07 | Multi-center service data management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111708787A true CN111708787A (en) | 2020-09-25 |
Family
ID=72536662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010376839.0A Pending CN111708787A (en) | 2020-05-07 | 2020-05-07 | Multi-center service data management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708787A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112350931A (en) * | 2020-10-29 | 2021-02-09 | 中国移动通信集团内蒙古有限公司 | Service processing method, device, equipment and computer storage medium |
| CN113268540A (en) * | 2021-03-26 | 2021-08-17 | 北京视博云信息技术有限公司 | Data synchronization method and device |
| CN113836221A (en) * | 2021-08-21 | 2021-12-24 | 济南浪潮数据技术有限公司 | Data synchronization method and device of distributed storage system |
| CN114448781A (en) * | 2021-12-22 | 2022-05-06 | 天翼云科技有限公司 | Data processing system |
| CN114567646A (en) * | 2022-03-08 | 2022-05-31 | 京东科技信息技术有限公司 | Data processing method, data processing system, electronic device, and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8145838B1 (en) * | 2009-03-10 | 2012-03-27 | Netapp, Inc. | Processing and distributing write logs of nodes of a cluster storage system |
| CN105187464A (en) * | 2014-06-19 | 2015-12-23 | 中兴通讯股份有限公司 | Data synchronization method, device and system in distributed storage system |
| US20190095293A1 (en) * | 2016-07-27 | 2019-03-28 | Tencent Technology (Shenzhen) Company Limited | Data disaster recovery method, device and system |
| CN109901949A (en) * | 2019-02-25 | 2019-06-18 | 中国工商银行股份有限公司 | The application disaster recovery and backup systems and method of dual-active data center |
| CN110674105A (en) * | 2019-08-30 | 2020-01-10 | 中国人民财产保险股份有限公司 | Data backup method, system and server |
| US20200026616A1 (en) * | 2018-07-17 | 2020-01-23 | EMC IP Holding Company LLC | Storage system with multiple write journals supporting synchronous replication failure recovery |
| CN110750278A (en) * | 2019-09-24 | 2020-02-04 | 苏宁云计算有限公司 | Gray scale distribution method and device, routing equipment and storage medium |
-
2020
- 2020-05-07 CN CN202010376839.0A patent/CN111708787A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8145838B1 (en) * | 2009-03-10 | 2012-03-27 | Netapp, Inc. | Processing and distributing write logs of nodes of a cluster storage system |
| CN105187464A (en) * | 2014-06-19 | 2015-12-23 | 中兴通讯股份有限公司 | Data synchronization method, device and system in distributed storage system |
| US20190095293A1 (en) * | 2016-07-27 | 2019-03-28 | Tencent Technology (Shenzhen) Company Limited | Data disaster recovery method, device and system |
| US20200026616A1 (en) * | 2018-07-17 | 2020-01-23 | EMC IP Holding Company LLC | Storage system with multiple write journals supporting synchronous replication failure recovery |
| CN109901949A (en) * | 2019-02-25 | 2019-06-18 | 中国工商银行股份有限公司 | The application disaster recovery and backup systems and method of dual-active data center |
| CN110674105A (en) * | 2019-08-30 | 2020-01-10 | 中国人民财产保险股份有限公司 | Data backup method, system and server |
| CN110750278A (en) * | 2019-09-24 | 2020-02-04 | 苏宁云计算有限公司 | Gray scale distribution method and device, routing equipment and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112350931A (en) * | 2020-10-29 | 2021-02-09 | 中国移动通信集团内蒙古有限公司 | Service processing method, device, equipment and computer storage medium |
| CN113268540A (en) * | 2021-03-26 | 2021-08-17 | 北京视博云信息技术有限公司 | Data synchronization method and device |
| CN113836221A (en) * | 2021-08-21 | 2021-12-24 | 济南浪潮数据技术有限公司 | Data synchronization method and device of distributed storage system |
| CN113836221B (en) * | 2021-08-21 | 2024-07-26 | 济南浪潮数据技术有限公司 | Data synchronization method and device for distributed storage system |
| CN114448781A (en) * | 2021-12-22 | 2022-05-06 | 天翼云科技有限公司 | Data processing system |
| CN114448781B (en) * | 2021-12-22 | 2024-06-07 | 天翼云科技有限公司 | Data processing system |
| CN114567646A (en) * | 2022-03-08 | 2022-05-31 | 京东科技信息技术有限公司 | Data processing method, data processing system, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111708787A (en) | Multi-center service data management system | |
| US20210056074A1 (en) | File System Data Access Method and File System | |
| US10719407B1 (en) | Backing up availability group databases configured on multi-node virtual servers | |
| US11093387B1 (en) | Garbage collection based on transmission object models | |
| CN108599973B (en) | Log association method, device and equipment | |
| CN110162512B (en) | Log retrieval method, device and storage medium | |
| CN108776670B (en) | Remote disaster recovery method, system and electronic equipment | |
| CN106897345B (en) | Data storage method and device | |
| CN110674105A (en) | Data backup method, system and server | |
| CN107609011B (en) | Database record maintenance method and device | |
| US20210303413A1 (en) | Tagging application resources for snapshot capability-aware discovery | |
| CN110895545B (en) | Shared data synchronization method and device | |
| CN111078127B (en) | Data migration method, system and device | |
| US12001450B2 (en) | Distributed table storage processing method, device and system | |
| US11243777B2 (en) | Process stream replication for content management system synchronization | |
| US10592530B2 (en) | System and method for managing transactions for multiple data store nodes without a central log | |
| US12026056B2 (en) | Snapshot capability-aware discovery of tagged application resources | |
| US11341159B2 (en) | In-stream data load in a replication environment | |
| US11288112B1 (en) | Enforcing data loss thresholds for performing updates to mirrored data sets | |
| CN115391337A (en) | Database partitioning method and device, storage medium and electronic equipment | |
| CN111522688A (en) | Data backup method and device for distributed system | |
| CN115827590A (en) | Data migration and cloud method, computing equipment and computer storage medium | |
| CN114090528A (en) | Cross-center file synchronization method, system and computer readable storage medium | |
| US10880388B1 (en) | Automatic redirection in scale-out cluster environments that perform distributed deduplication | |
| WO2024174894A1 (en) | Physical volume access method, apparatus, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200925 |