CN116244040B - Main and standby container cluster system, data synchronization method thereof and electronic equipment - Google Patents
Main and standby container cluster system, data synchronization method thereof and electronic equipment Download PDFInfo
- Publication number
- CN116244040B CN116244040B CN202310232124.1A CN202310232124A CN116244040B CN 116244040 B CN116244040 B CN 116244040B CN 202310232124 A CN202310232124 A CN 202310232124A CN 116244040 B CN116244040 B CN 116244040B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- file synchronization
- container cluster
- backup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012508 change request Methods 0.000 claims abstract description 38
- 230000015654 memory Effects 0.000 claims abstract description 25
- 238000004590 computer program Methods 0.000 claims abstract description 6
- 230000009471 action Effects 0.000 claims description 30
- 238000004891 communication Methods 0.000 claims description 23
- 238000013500 data storage Methods 0.000 claims description 17
- 238000012795 verification Methods 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 101100322583 Caenorhabditis elegans add-2 gene Proteins 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The specification discloses a master-slave container cluster system, a data synchronization method thereof and electronic equipment. The system includes a primary container cluster, a backup container cluster, and a remote data warehouse, the method comprising: a first database in a main container cluster receives a data change request and changes original data in the first database according to the data change request; the first file synchronization program acquires the changed original data and copies the original data to a remote data warehouse to serve as backup data; a second file synchronization program in the standby container cluster acquires backup data from the remote data warehouse and synchronizes the backup data to a second database to be used as synchronization data; the second database is a spare database of the first database. The electronic device comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the data synchronization method when executing the program.
Description
Technical Field
The invention relates to the technical field of container clusters, in particular to a data synchronization method, a system and electronic equipment of a main container cluster and a standby container cluster.
Background
With the widespread use of container clusters in production environments, applications for a variety of different services have begun to run within container clusters. Correspondingly, different business applications will use appropriate database technology in combination with the respective business scenarios. For example, the redis database is suitable for quick authentication of user identity, the Mysql database is suitable for access operation of order data, and the clickhouse database is suitable for data analysis in big data scene, etc. Meanwhile, in order to prevent disaster situations such as downtime of the power-off and network-off system, a standby container cluster is generally prepared, and service applications and databases similar to those of the main container cluster are deployed. And the data synchronization operation is carried out by using the data backup tools of the databases of different types in the main and standby container clusters and adopting different backup methods.
With the increasing types of production environment databases, a great technical challenge is brought to the realization of data synchronization between the main and standby container clusters. In order to realize data synchronization among multiple types of databases in a main and standby container cluster, a worker needs to know and master database backup tools and backup methods matched with various database types in depth, and high requirements are put on professional quality of the worker. In order to maintain the standby container clusters, the data synchronization between the main container clusters and the standby container clusters is ensured to require larger manual intervention, the data synchronization efficiency is low, and the realization difficulty is high.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a master/slave container cluster system, a data synchronization method thereof, and an electronic device, which can automatically and efficiently implement data synchronization between multiple databases in a master/slave container cluster.
In a first aspect, embodiments of the present disclosure provide a data synchronization method of a primary and backup container cluster system, where the method is applied to the primary and backup container cluster system, and the primary and backup container cluster system includes a primary container cluster, a backup container cluster, and a remote data repository, and the method includes:
in the main container cluster, a first database receives a data change request and changes original data in the first database according to the data change request;
The first file synchronization program obtains the changed original data and copies the original data to the remote data warehouse to serve as backup data;
in the backup container cluster, a second file synchronization program acquires the backup data from the remote data warehouse and synchronizes the backup data into the second database as synchronous data;
The second database is a standby database of the first database.
Optionally, before the original data is changed according to the data change request, the first database performs validity and consistency verification on the data change request;
And after the verification is passed, the first database modifies the data value corresponding to the data change request in the original data.
Optionally, in the main container cluster, the first database and the first file synchronization program are deployed as containers, and the first database and the first file synchronization program are set in the same container group;
in the standby container cluster, the second database and the second file synchronization program are deployed as containers, and the second database and the second file synchronization program are arranged in the same container group.
Optionally, the operation parameters of the first file synchronization program and the second file synchronization program include an action parameter, a target parameter and an address parameter;
the action parameters are used for designating the execution action of the file synchronization program;
The target parameter is used for pointing to an address of the remote data warehouse;
the address parameter is used to point to a data storage address in the remote data warehouse.
Optionally, when the first file synchronization program copies the original data to the remote data warehouse, the action parameter is set to out, and the action parameter out represents transmission of data to the outside;
The first file synchronization program copying the raw data to the remote data repository, comprising:
the first file synchronization program establishes communication connection with the remote data warehouse according to the target parameter;
The original data is transferred in a file copy to a data storage address in the remote data repository to which the address parameter points.
Optionally, when the second file synchronization program obtains the backup data from the remote data repository, the action parameter is set to in, where the action parameter in indicates reading data from the outside;
the second file synchronization program obtaining the backup data from the remote data repository, comprising:
The second file synchronization program establishes communication connection with the remote data warehouse according to the target parameter;
And acquiring the backup data from a data storage address pointed by the address parameter in the remote data warehouse.
Optionally, the second file synchronization program periodically obtains the backup data from the remote data repository.
Optionally, after the second file synchronization program obtains the backup data from the remote data repository, the method further includes:
Comparing the backup data with the local data of the second database to determine whether the backup data is changed;
and in response to determining that the backup data is changed, synchronizing the backup data to the second database.
In a second aspect, embodiments of the present disclosure also provide a primary and backup container cluster system, the system comprising a primary container cluster, a backup container cluster, and a remote data warehouse;
The main container cluster comprises a first database and a first file synchronization program;
The first database is used for receiving a data change request and changing the original data in the first database according to the data change request;
the first file synchronization program is used for acquiring the changed original data and copying the original data into the remote data warehouse to serve as backup data;
the standby container cluster comprises a second database and a second file synchronization program;
The first file synchronization program is configured to acquire the backup data from the remote data repository, and synchronize the backup data to the second database as synchronization data;
The second database is a standby database of the second database.
In a third aspect, embodiments of the present disclosure further provide a data synchronization electronic device of a primary and backup container cluster system, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the data synchronization method according to the first aspect when executing the program.
From the above, it can be seen that the master and slave container cluster system, the data synchronization method thereof and the electronic device provided by the embodiments of the present disclosure have the following beneficial technical effects:
a remote data warehouse is arranged on the basis of a main container cluster and a standby container cluster to store all data backups in a plurality of databases of different types in the container cluster. When the data in the main container cluster changes, the backup data in the remote data warehouse also changes, and then the changed backup data is resynchronized to the corresponding database in the backup container cluster by adopting an asynchronous synchronization mode. In such a way, the method can uniformly manage and control the databases of different types in the main container cluster, a special synchronization tool is not required to be set for the databases of different types, and the difference of the databases of different types in the synchronization tool and the synchronization method is shielded. The file synchronization program is arranged in the main and standby container clusters to copy, transmit and synchronize the database data, so that the automatic synchronization of the database data in the main and standby container clusters can be realized, the manual access degree is reduced, the data synchronization efficiency is improved, the pressure of staff is reduced, and the implementation cost of the scheme is reduced.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and should not be construed as limiting the invention in any way, in which:
FIG. 1 is a schematic diagram of a method for synchronizing data in a primary and backup container cluster system according to one or more embodiments of the present disclosure;
FIG. 2 is a schematic diagram illustrating a method for copying original data to a remote data repository by a first file synchronization program in a data synchronization method of a primary and backup container cluster system according to one or more embodiments of the present disclosure;
FIG. 3 is a schematic diagram illustrating a method for a two-file synchronization program to obtain backup data from a remote data repository in a data synchronization method of a primary and backup container cluster system according to one or more embodiments of the present disclosure;
FIG. 4 is a schematic diagram of a primary and secondary container cluster system according to one or more alternative embodiments of the present disclosure;
FIG. 5 illustrates yet another architecture diagram of a master-slave container cluster system provided in one or more alternative embodiments of the present disclosure;
fig. 6 is a schematic diagram of a data synchronization electronic device of a master-slave container cluster system according to one or more alternative embodiments of the present disclosure.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The container cloud mainly uses the container as a minimum unit to realize the support of system resources of the cloud platform. The management and scheduling of containers is achieved through container scheduling tools such as Rancher, kubernetes or Swarm, and the management of services of applications in the container cloud is achieved through service discovery and service routing. The containers are managed by virtualizing the operating system, each container contains an exclusive complete user environment space, and changes in one container do not affect the running environment of the other containers. A container cloud typically includes multiple containers, which may be referred to as a cluster of containers.
With the widespread use of container clusters in production environments, applications for a variety of different services have begun to run within container clusters. Correspondingly, different business applications will use appropriate database technology in combination with the respective business scenarios. For example, the redis database is suitable for quick authentication of user identity, the Mysql database is suitable for access operation of order data, and the clickhouse database is suitable for data analysis in big data scene, etc. Meanwhile, in order to prevent disaster situations such as downtime of the power-off and network-off system, a standby container cluster is generally prepared, and service applications and databases similar to those of the main container cluster are deployed. And the data synchronization operation is carried out by using the data backup tools of the databases of different types in the main and standby container clusters and adopting different backup methods.
With the increasing types of production environment databases, a great technical challenge is brought to the realization of data synchronization between the main and standby container clusters. In order to realize data synchronization among multiple types of databases in the main and standby container clusters, operators are required to know and master database backup tools and backup methods matched with the various database types in depth, high requirements are provided for professional quality of the operators, large manual intervention is required for data backup synchronization among the main and standby container clusters, data synchronization efficiency is low, and realization difficulty is high.
In view of the above problems, an object of the embodiments of the present disclosure is to provide a system for a primary container cluster and a secondary container cluster, a data synchronization method thereof, and an electronic device, in which a remote data warehouse is disposed in the system and connected to the primary container cluster and the secondary container cluster, backup data of a plurality of databases in the primary container cluster and the secondary container cluster are stored in the remote data warehouse, and automatic and efficient synchronization of data of a plurality of different databases from the primary container cluster to the secondary container cluster is achieved by using the remote data warehouse, so that differences of a plurality of synchronization tool technologies between the different databases can be shielded.
In view of the above objective, in one aspect, an embodiment of the present disclosure provides a data synchronization method of a primary and backup container cluster system.
As shown in fig. 1, one or more alternative embodiments of the present disclosure provide a data synchronization method for a primary and secondary container cluster system, which is applied to the primary and secondary container cluster system. The master-slave container cluster system comprises a master container cluster, a slave container cluster and a remote data warehouse. The method comprises the following steps:
S1: in the main container cluster, a first database receives a data change request and changes original data in the first database according to the data change request.
The data change request may be generated by an application corresponding to the first database. After receiving an operation instruction from a user, the application program can generate the data change request according to the operation instruction. Wherein the operation instruction from the user may be an identity information modification instruction, an order generation instruction, a log modification instruction, or the like.
Different user operation instructions can be responded by different application programs in the main container cluster, and different data change requests are correspondingly generated; the different data change requests may be received by correspondingly adapted different kinds of first databases.
And after the first database receives the data change request, performing operations such as adding, deleting and modifying data items corresponding to the data change request in the original data.
In some optional embodiments, the first database may further perform validity and consistency verification for the data change request before receiving the data change request and changing the original data according to the data change request. And after the verification is passed, the first database modifies the data value corresponding to the data change request in the original data.
The validity verification means that the first database verifies the validity of the user identity corresponding to the data change request; and the consistency verification means that the first database verifies the corresponding data format of the data change request.
Taking the MySQL database as an example for explanation, the data change request aims at modifying the data of a certain row in the MySQL database, and the MySQL database firstly checks whether the data format in the data change request meets the specification requirement of the row in the database. If so, the data line is locked using a line level lock and the data value is modified.
In the main container cluster, the first database is deployed and set in the form of a container, and data in the database is mounted in the container in the form of a persistent volume.
S2: and the first file synchronization program acquires the changed original data and copies the original data into the remote data warehouse to serve as backup data.
The first file synchronization program (Sync) is also deployed in the primary container cluster in the form of a container. After the first database changes the original data according to the data change request, the first file synchronization program Sync corresponding to the first database can read a data directory of the first database, and copy and transmit the data in the first database to the remote data warehouse in a file copy mode to serve as backup data corresponding to the original data in the remote data warehouse.
The operation parameters of the first file synchronization program comprise action parameters, target parameters and address parameters. Wherein the action parameter (out/in) is used for specifying the execution action of the file synchronization program; -said target parameter (repo) is used to point to an address of said remote data warehouse; the address parameter (path) is used to point to a data storage address in the remote data warehouse.
The first file synchronization program sets the operation parameter to out when copying the original data to the remote data warehouse, and the operation parameter out indicates transmission of data to the outside.
As shown in fig. 2, in some alternative embodiments, the first file synchronization program copies the original data to the remote data repository, may include the method steps of:
s201: the first file synchronization program establishes a communication connection with the remote data warehouse according to the target parameter.
The target parameter repo points to the remote data repository, with which the first file synchronization program Sync1 can establish a communication connection in accordance with the target parameter repo at run-time.
S202: the original data is transferred in a file copy to a data storage address in the remote data repository to which the address parameter path points.
The first file synchronization program Sync1 may first perform a connectivity check on the remote data repository pointed to by the target parameter based on the target parameter repo to determine whether the remote data repository is accessible. If the data cannot be accessed, the subsequent data copying operation is not performed, and the checking is continued until the data can be accessed. If so, the first file synchronization program Sync1 transfers the data read from the first database to the data storage address pointed by the address parameter path under the remote data repository by means of a file copy.
It should be noted that a plurality of databases of different types may be included in the master container cluster. Different databases are provided with corresponding file synchronization programs, and the address parameters corresponding to the different file synchronization programs can point to different storage addresses in the remote data warehouse.
The specific values of the address parameters in different file synchronization programs can be set according to the definition of actual situations when the file synchronization programs are deployed. For example, for MySql database, the address parameter path in its corresponding file synchronization program may be set to add_0; for the redis database, the address parameter path in the corresponding file synchronization program can be set as add_2; for clickhouse databases, the address parameter path in the corresponding file synchronization program may be set to add_4, and so on. The data in the MySql database, the redis database, clickhouse database would be replicated and transferred to the remote data warehouse at repo/add_0, repo/add_2, repo/add_4, respectively.
It should be further noted that the types of databases in the master container cluster include, but are not limited to, the MySql database, the redis database, and the clickhouse database, and the corresponding data that may be stored in the remote data warehouse also includes, but is not limited to, the three database data. In the main container cluster, the number and the types of the databases can be flexibly set according to the actual application requirements.
In view of the limitation of data transmission and sharing between different containers in the container cluster, in some alternative embodiments, the first database and the corresponding first file synchronization program Sync1 may be disposed in the same container group Pod. The first file synchronization program is used as sidecar containers to be deployed in the same container group Pod with the first database in a non-invasive way, so that the data catalog of the first database can be shared, and in such a way, the first file synchronization program can directly acquire the original data in the first database, and can conveniently and rapidly realize copy transmission of the original data.
S3: in the backup container cluster, a second file synchronization program obtains the backup data from the remote data warehouse and synchronizes the backup data into the second database as synchronization data.
And the backup container cluster is provided with the same business application and database as the main container cluster. The second database is a standby database corresponding to the first database in the main container cluster. The second file synchronization program corresponding to the second database can acquire the backup data from the remote data warehouse and synchronize the backup data to the second database, so that data synchronization between the first database and the second database in the main and standby container clusters is realized.
The second database and the second file synchronization program Sync2 are deployed in the standby container cluster in the form of a container.
The operation parameters of the second file synchronization program comprise action parameters, target parameters and address parameters. Wherein the action parameter (out/in) is used for specifying the execution action of the file synchronization program; -said target parameter (repo) is used to point to an address of said remote data warehouse; the address parameter (path) is used to point to a data storage address in the remote data warehouse.
When the second file synchronization program acquires the backup data from the remote data warehouse, the action parameter is set to in, and the action parameter in represents reading data from the outside.
As shown in fig. 3, the second file synchronization program obtains the backup data from the remote data repository, including:
S301: the second file synchronization program establishes a communication connection with the remote data warehouse according to the target parameter.
The target parameter repo points to the remote data repository, with which the second file synchronization program Sync2 can establish a communication connection in accordance with the target parameter repo at run-time.
S302: and acquiring the backup data from a data storage address pointed by the address parameter in the remote data warehouse.
After the second file synchronization program Sync2 establishes a communication connection with the remote data repository, the backup data may be obtained from the data storage address pointed by the address parameter path in the remote data repository.
Further, when the action parameter is in, the second file synchronization program adopts a read-only mode when reading data from the outside. In this way the second file synchronization program will only read data from the remote data repository and will not modify the data in the remote data repository.
The second database is a standby database corresponding to the first database in the main container cluster. The address parameter in the second file synchronization program corresponding to the second database is the same as the address parameter in the first file synchronization program corresponding to the first database.
Taking the address parameter in the first file synchronization program as an example, the address parameter in the second file synchronization program is also set to be add_x. In the primary container cluster, the first file synchronization program may copy and transfer the primary data in the first database to the remote data warehouse as backup data at a data storage address specified by repo/add_x. Correspondingly, the second file synchronization program in the standby container cluster acquires the backup data from the data storage address designated by repo/add_x in the remote data warehouse according to the address parameter add_x, and synchronizes the backup data to the local second database.
In some alternative embodiments, the second file synchronization program is disposed in the same container group Pod as the second file database. And after the second file synchronization program acquires the backup data, the backup data can be more conveniently and rapidly synchronized to the second database to serve as the synchronization data.
According to the data synchronization method of the master and slave container cluster system, a remote data warehouse is arranged on the basis of the master and slave container clusters to store all data backups in a plurality of databases of different types in the container cluster. When the data in the main container cluster changes, the backup data in the remote data warehouse also changes, and then the changed backup data is resynchronized to the corresponding database in the backup container cluster by adopting an asynchronous synchronization mode. In such a way, the method can uniformly manage and control the databases of different types in the main container cluster, a special synchronization tool is not required to be set for the databases of different types, and the difference of the databases of different types in the synchronization tool and the synchronization method is shielded. The file synchronization program is arranged in the main and standby container clusters to copy, transmit and synchronize the database data, so that the automatic synchronization of the database data in the main and standby container clusters can be realized, the manual access degree is reduced, the data synchronization efficiency is improved, the pressure of staff is reduced, and the implementation cost of the scheme is reduced.
In some alternative embodiments, the second file synchronization program in the backup container cluster periodically obtains the backup data from the remote data repository, and after the second file synchronization program obtains the backup data from the remote data repository, queries are performed on the backup data to determine whether a change has occurred to the backup data. Specifically, the second file synchronization program may compare the backup data with local data of the second database to determine whether the backup data is changed; and in response to determining that the backup data is changed, synchronizing the backup data to the second database.
It should be noted that, when the original data in the first database in the primary container cluster is changed, the backup data in the remote data repository is changed accordingly, and in this case, the data in the second database in the backup container cluster is required to be changed accordingly. The second file synchronization program periodically acquires backup data and inquires whether the backup data is changed, and the changed backup data is synchronized to the second database only when the backup data is changed, so that the data synchronization workload of the second database in the backup container cluster can be greatly reduced.
It should be noted that the methods of one or more embodiments of the present description may be performed by a single device, such as a computer or server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of one or more embodiments of the present description, the devices interacting with each other to accomplish the methods.
It should be noted that the foregoing describes specific embodiments of the present invention. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Based on the same inventive concept, the embodiment of the present disclosure further provides a master/slave container cluster system corresponding to the method of any embodiment.
Referring to fig. 4, the primary and backup container cluster system includes a primary container cluster, a backup container cluster, and a remote data warehouse.
The primary container cluster includes a first database and a first file synchronization program.
The first database is used for receiving a data change request and changing the original data in the first database according to the data change request;
the first file synchronization program is used for acquiring the changed original data and copying the original data into the remote data warehouse to serve as backup data;
the standby container cluster comprises a second database and a second file synchronization program;
The first file synchronization program is configured to acquire the backup data from the remote data repository, and synchronize the backup data to the second database as synchronization data;
The second database is a standby database of the second database.
In one or more alternative embodiments of the present disclosure, in a primary and backup container cluster system, the first database is further configured to perform validity and consistency verification on the data change request before the original data is changed according to the data change request; after the verification is passed, the first database modifies the data value corresponding to the data change request in the original data.
As shown in fig. 5, in a primary and backup container cluster system provided in one or more alternative embodiments of the present disclosure, the first database and the first file synchronization program in the primary container cluster are deployed as containers, and the first database and the first file synchronization program are disposed in the same container group; in the standby container cluster, the second database and the second file synchronization program are deployed as containers, and the second database and the second file synchronization program are arranged in the same container group.
In one or more embodiments of the present disclosure, in a primary and standby container cluster system, the operation parameters of the first file synchronization program and the second file synchronization program include an action parameter, a target parameter, and an address parameter; the action parameters are used for designating the execution action of the file synchronization program; the target parameter is used for pointing to an address of the remote data warehouse; the address parameter is used to point to a data storage address in the remote data warehouse.
In one or more alternative embodiments of the present disclosure, in a primary and backup container cluster system, the first file synchronization program sets the action parameter to out when copying the original data to the remote data repository, where out represents transmitting data to the outside.
The first file synchronization program is further configured to establish a communication connection with the remote data repository according to the target parameter; the original data is transferred in a file copy to a data storage address in the remote data repository to which the address parameter points.
In one or more alternative embodiments of the present disclosure, in a primary and backup container cluster system, when the second file synchronization program obtains the backup data from the remote data repository, the action parameter is set to in, where the action parameter in indicates reading data from outside.
The second file synchronization program is further configured to establish a communication connection with the remote data repository according to the target parameter; and acquiring the backup data from a data storage address pointed by the address parameter in the remote data warehouse.
In a primary and backup container cluster system provided in one or more alternative embodiments of the present disclosure, the second file synchronization program is further configured to periodically obtain the backup data from the remote data repository.
In one or more alternative embodiments of the present disclosure, after the second file synchronization program obtains the backup data from the remote data repository, the second file synchronization program is further configured to compare the backup data with local data of the second database to determine whether the backup data is changed; and synchronizing the backup data to the second database when the backup data is changed.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in one or more pieces of software and/or hardware when implementing one or more embodiments of the present description.
The device of the foregoing embodiment is configured to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Fig. 6 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage, dynamic storage, etc. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.
The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The electronic device of the foregoing embodiment is configured to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Based on the same inventive concept, corresponding to any of the above embodiments of the method, the present disclosure further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the data synchronization method as described in any of the above embodiments.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The storage medium of the foregoing embodiments stores computer instructions for causing the computer to perform the data synchronization method according to any one of the foregoing embodiments, and has the advantages of the corresponding method embodiments, which are not described herein.
It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. Wherein the storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (RandomAccessMemory, RAM), a Flash Memory (Flash Memory), a hard disk (HARD DISK DRIVE, abbreviated as HDD), a Solid state disk (Solid-STATE DRIVE, SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the disclosure, including the claims, is limited to these examples; combinations of features of the above embodiments or in different embodiments are also possible within the spirit of the present disclosure, steps may be implemented in any order, and there are many other variations of the different aspects of one or more embodiments described above which are not provided in detail for the sake of brevity.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The present disclosure is intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Any omissions, modifications, equivalents, improvements, and the like, which are within the spirit and principles of the one or more embodiments of the disclosure, are therefore intended to be included within the scope of the disclosure.
Claims (10)
1. A method for synchronizing data of a primary and a secondary container cluster, wherein the method is applied to a primary and a secondary container cluster system, the primary and the secondary container cluster system comprising a primary container cluster, a secondary container cluster and a remote data warehouse, the method comprising:
In the main container cluster, a first database receives a data change request and changes original data in the first database according to the data change request; the master container cluster includes a plurality of databases of different types; the databases of different types are provided with corresponding file synchronization programs, and corresponding address parameters of different file synchronization programs point to different storage addresses in the remote data warehouse;
The first file synchronization program obtains the changed original data and copies the original data to the remote data warehouse to serve as backup data;
in the backup container cluster, a second file synchronization program acquires the backup data from the remote data warehouse and synchronizes the backup data into a second database to serve as synchronous data;
The second database is a standby database of the first database.
2. The method of claim 1, wherein the first database further performs validity and consistency verification for the data change request before changing the original data according to the data change request;
And after the verification is passed, the first database modifies the data value corresponding to the data change request in the original data.
3. The method of claim 1, wherein in the master container cluster, the first database and the first file synchronization program are deployed as containers, and the first database and the first file synchronization program are disposed in the same container group;
in the standby container cluster, the second database and the second file synchronization program are deployed as containers, and the second database and the second file synchronization program are arranged in the same container group.
4. The method of claim 1, wherein the operating parameters of the first file synchronization program and the second file synchronization program include an action parameter, a target parameter, and an address parameter;
the action parameters are used for designating the execution action of the file synchronization program;
The target parameter is used for pointing to an address of the remote data warehouse;
the address parameter is used to point to a data storage address in the remote data warehouse.
5. The method of claim 4, wherein the first file synchronization program sets the action parameter to out when copying the original data to the remote data repository, the action parameter out representing transmission of data to the outside;
The first file synchronization program copying the raw data to the remote data repository, comprising:
the first file synchronization program establishes communication connection with the remote data warehouse according to the target parameter;
The original data is transferred in a file copy to a data storage address in the remote data repository to which the address parameter points.
6. The method of claim 4, wherein the action parameter is set to in when the second file synchronization program obtains the backup data from the remote data repository, the action parameter in indicating reading data from the outside;
the second file synchronization program obtaining the backup data from the remote data repository, comprising:
The second file synchronization program establishes communication connection with the remote data warehouse according to the target parameter;
And acquiring the backup data from a data storage address pointed by the address parameter in the remote data warehouse.
7. The method of claim 1, wherein the second file synchronization program periodically obtains the backup data from the remote data repository.
8. The method of claim 7, wherein after the second file synchronization program obtains the backup data from the remote data repository, further comprising:
Comparing the backup data with the local data of the second database to determine whether the backup data is changed;
and in response to determining that the backup data is changed, synchronizing the backup data to the second database.
9. A master-slave container cluster system, wherein the system comprises a master container cluster, a slave container cluster and a remote data warehouse;
the main container cluster comprises a first database and a first file synchronization program; the master container cluster includes a plurality of databases of different types; the databases of different types are provided with corresponding file synchronization programs, and corresponding address parameters of different file synchronization programs point to different storage addresses in the remote data warehouse;
The first database is used for receiving a data change request and changing the original data in the first database according to the data change request;
the first file synchronization program is used for acquiring the changed original data and copying the original data into the remote data warehouse to serve as backup data;
the standby container cluster comprises a second database and a second file synchronization program;
The first file synchronization program is configured to acquire the backup data from the remote data repository, and synchronize the backup data to the second database as synchronization data;
The second database is a standby database of the second database.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 8 when the program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310232124.1A CN116244040B (en) | 2023-03-10 | 2023-03-10 | Main and standby container cluster system, data synchronization method thereof and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310232124.1A CN116244040B (en) | 2023-03-10 | 2023-03-10 | Main and standby container cluster system, data synchronization method thereof and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116244040A CN116244040A (en) | 2023-06-09 |
| CN116244040B true CN116244040B (en) | 2024-05-03 |
Family
ID=86623957
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310232124.1A Active CN116244040B (en) | 2023-03-10 | 2023-03-10 | Main and standby container cluster system, data synchronization method thereof and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116244040B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301250A (en) * | 2017-07-27 | 2017-10-27 | 南京南瑞集团公司 | A kind of Various database cooperates with backup method |
| CN109086170A (en) * | 2018-08-15 | 2018-12-25 | 郑州云海信息技术有限公司 | A kind of data base backup recovery method and device |
| CN111078463A (en) * | 2018-10-19 | 2020-04-28 | 阿里巴巴集团控股有限公司 | Data backup method, device and system |
| CN113010498A (en) * | 2021-03-25 | 2021-06-22 | 腾讯科技（深圳）有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN114138902A (en) * | 2021-11-26 | 2022-03-04 | 杭州安恒信息技术股份有限公司 | Method, system, computer and storage medium for master-slave synchronization and switching of two databases |
| CN114168380A (en) * | 2021-11-23 | 2022-03-11 | 阿里巴巴（中国）有限公司 | Database configuration method, device, system and storage medium |
| CN114238255A (en) * | 2021-11-30 | 2022-03-25 | 中国工商银行股份有限公司 | Method and device for synchronizing files, electronic equipment and computer readable storage medium |
| CN114466027A (en) * | 2022-01-26 | 2022-05-10 | 苏州浪潮智能科技有限公司 | Cloud native database service providing method, system, equipment and medium |
| CN114625566A (en) * | 2021-08-19 | 2022-06-14 | 湖南亚信软件有限公司 | Data disaster tolerance method and device, electronic equipment and storage medium |
| CN115510156A (en) * | 2022-09-22 | 2022-12-23 | 新华三技术有限公司 | Cloud native high-availability database service providing system and method |
| CN115576655A (en) * | 2022-12-09 | 2023-01-06 | 浪潮电子信息产业股份有限公司 | Container data protection system, method, device, equipment and readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10911536B2 (en) * | 2015-10-27 | 2021-02-02 | Talkcycle Llc | Real-time synchronization of data between disparate cloud data sources |
-
2023
- 2023-03-10 CN CN202310232124.1A patent/CN116244040B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301250A (en) * | 2017-07-27 | 2017-10-27 | 南京南瑞集团公司 | A kind of Various database cooperates with backup method |
| CN109086170A (en) * | 2018-08-15 | 2018-12-25 | 郑州云海信息技术有限公司 | A kind of data base backup recovery method and device |
| CN111078463A (en) * | 2018-10-19 | 2020-04-28 | 阿里巴巴集团控股有限公司 | Data backup method, device and system |
| CN113010498A (en) * | 2021-03-25 | 2021-06-22 | 腾讯科技（深圳）有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN114625566A (en) * | 2021-08-19 | 2022-06-14 | 湖南亚信软件有限公司 | Data disaster tolerance method and device, electronic equipment and storage medium |
| CN114168380A (en) * | 2021-11-23 | 2022-03-11 | 阿里巴巴（中国）有限公司 | Database configuration method, device, system and storage medium |
| CN114138902A (en) * | 2021-11-26 | 2022-03-04 | 杭州安恒信息技术股份有限公司 | Method, system, computer and storage medium for master-slave synchronization and switching of two databases |
| CN114238255A (en) * | 2021-11-30 | 2022-03-25 | 中国工商银行股份有限公司 | Method and device for synchronizing files, electronic equipment and computer readable storage medium |
| CN114466027A (en) * | 2022-01-26 | 2022-05-10 | 苏州浪潮智能科技有限公司 | Cloud native database service providing method, system, equipment and medium |
| CN115510156A (en) * | 2022-09-22 | 2022-12-23 | 新华三技术有限公司 | Cloud native high-availability database service providing system and method |
| CN115576655A (en) * | 2022-12-09 | 2023-01-06 | 浪潮电子信息产业股份有限公司 | Container data protection system, method, device, equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116244040A (en) | 2023-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10291696B2 (en) | Peer-to-peer architecture for processing big data | |
| US20220253458A1 (en) | Method and device for synchronizing node data | |
| KR101503202B1 (en) | Data synchronization | |
| US10291704B2 (en) | Networked solutions integration using a cloud business object broker | |
| JP5624479B2 (en) | Sync server process | |
| CN107172214B (en) | Service node discovery method and device with load balancing function | |
| CN111124589B (en) | Service discovery system, method, device and equipment | |
| CN111800468A (en) | Cloud-based multi-cluster management method, device, medium and electronic equipment | |
| TW202008763A (en) | Data processing method and apparatus, and client | |
| EP3779692A1 (en) | Data processing method and device, client, and server | |
| CN113553178A (en) | Task processing method and device and electronic equipment | |
| US20160352813A1 (en) | Cloud computing infrastructure | |
| CN116244040B (en) | Main and standby container cluster system, data synchronization method thereof and electronic equipment | |
| CN110489483B (en) | Data synchronization method, device, computer equipment and storage medium | |
| CN114422537B (en) | Multi-cloud storage system, multi-cloud data reading and writing method and electronic equipment | |
| CN112000850A (en) | Method, device, system and equipment for data processing | |
| CN115098537B (en) | Transaction execution method and device, computing equipment and storage medium | |
| CN113297168B (en) | Data migration method and device in distributed system | |
| CN111147226A (en) | Data storage method, device and storage medium | |
| CN114710350A (en) | Allocation method and device for callable resources | |
| US11455318B2 (en) | Data replication using probabilistic replication filters | |
| CN113282617A (en) | Data query method and business system page turning method | |
| CN113761075A (en) | Method, device, equipment and computer readable medium for switching databases | |
| CN111880895A (en) | Data reading and writing method and device based on Kubernetes platform | |
| JPWO2016067370A1 (en) | Information processing apparatus, method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |