CN115292408A - Master-slave synchronization method, device, equipment and medium for MySQL database - Google Patents

Master-slave synchronization method, device, equipment and medium for MySQL database Download PDF

Info

Publication number
CN115292408A
CN115292408A CN202210849391.9A CN202210849391A CN115292408A CN 115292408 A CN115292408 A CN 115292408A CN 202210849391 A CN202210849391 A CN 202210849391A CN 115292408 A CN115292408 A CN 115292408A
Authority
CN
China
Prior art keywords
database
slave
master
data
log
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210849391.9A
Other languages
Chinese (zh)
Inventor
丁鹏
卢雨
彭程
范景新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
Tianyi Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Cloud Technology Co Ltd filed Critical Tianyi Cloud Technology Co Ltd
Priority to CN202210849391.9A priority Critical patent/CN115292408A/en
Publication of CN115292408A publication Critical patent/CN115292408A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2358Change logging, detection, and notification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Abstract

The application discloses a master-slave synchronization method, a master-slave synchronization device, master-slave synchronization equipment and a master-slave synchronization medium for a MySQL database. The method comprises the following steps: the data to be updated is persisted to the shared storage system by controlling the first database, so that a log file corresponding to the data to be updated is generated in the shared storage system. Further, the first database is controlled to send the log point locations of the log files to the second database, so that the second database caches the corresponding log files according to the log point locations. In the above process, the second database stores the log point locations of all log files generated in the system before the master-slave switching, rather than the complete log file. After master-slave switching, corresponding log files can be found in the shared storage system only by inquiring prestored log point positions. Therefore, the slave database can support the master-slave synchronization function only by a very small resource amount, and the resource utilization rate is improved.

Description

Master-slave synchronization method, device, equipment and medium for MySQL database
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a master-slave synchronization method, device, equipment and medium for a MySQL database.
Background
With the development of science and technology, the coverage business of internet manufacturers is increasingly widespread. In the related art, a database with a master-slave synchronization relationship is provided for a service system to ensure the consistency and security of service data. The master-slave synchronization relationship of the database mainly refers to that the master database provides a read-write function to the outside to support external services. When the data in the master database is changed, the corresponding log files are synchronously transmitted to the slave database for backup. Therefore, when the master database is abnormal, the slave database can be upgraded to the master database to ensure the integrity of the data and take over the abnormal master database to provide external services.
In practical application, the same or similar computing resources and storage resources are required to be allocated to the master database and the slave database. The reason is that when the difference between the resources held by the master database and the slave database is large, a large time delay is generated in the data synchronization process, and further data loss is caused when the master database and the slave database are switched.
Disclosure of Invention
The embodiment of the application provides a master-slave synchronization method, device, equipment and medium of a MySQL database, which are used for reducing the held resource quantity of a second database and further improving the resource utilization rate of a server.
In a first aspect, an embodiment of the present application provides a master-slave synchronization method for a MySQL database, where the method includes:
responding to a data updating request, and acquiring data to be updated;
controlling a first database to persist the data to be updated to a shared storage system so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
controlling the first database to send the log point locations of the log files to a second database so that the second database performs cache operation on the log files based on the log point locations; wherein the second database is a slave database in the MySQL database; the number of the memory occupied by the master database and the number of the cores of the central processing unit are both higher than that of the slave database.
According to the embodiment of the application, the first database is controlled to persist the data to be updated into the shared storage system, so that a log file corresponding to the data to be updated is generated in the shared storage system. Further, the first database is controlled to send the log point location of the log file to the second database, so that the second database caches the corresponding log file according to the log point location. In the above process, the second database stores the log point locations of all log files generated in the system before the master-slave switching, rather than the complete log file. After master-slave switching, corresponding log files can be found in the shared storage system only by inquiring prestored log point positions. Therefore, the slave database can support the master-slave synchronization function only by a very small resource amount, and the resource utilization rate is improved.
In some possible embodiments, the method further comprises:
responding to a disaster tolerance switching instruction, and updating the current occupied resource of the second database by taking the occupied resource of the main database as a standard; wherein the occupied resources comprise the number of the memory and the central processor cores;
updating the current occupied resource of the first database by taking the occupied resource of the slave database as a standard, and executing restarting operation on the updated first database;
and taking the updated second database as the master database, and taking the restarted first database as the slave database.
In some possible embodiments, the method further comprises:
if the first database fails to restart, dividing equal amount of resources to be processed from the cloud system by taking the occupied resources of the slave database as a standard;
and generating a third database according to the resource to be processed, and taking the third database as the slave database.
In some possible embodiments, after updating the currently occupied resource of the second database based on the occupied resource of the master database, the method further includes:
and executing log playback and transaction rollback operations on the currently cached log file of the second database.
In some possible embodiments, after the second database performs the caching operation on the log file, the method further includes:
if a new data updating request is monitored, newly added data to be updated is obtained;
and after controlling the first database to send the log point location of the newly added data to be updated to the second database, controlling the second database to clear the current cache and then executing cache operation on the log file of the newly added data to be updated.
In some possible embodiments, the master database is provided with permission to perform read and write operations on the shared storage system;
the slave database has a right to perform read-only operations on the shared storage system.
In a second aspect, an embodiment of the present application provides a master-slave synchronization apparatus for a MySQL database, where the apparatus includes:
the data acquisition module is configured to respond to the data updating request and acquire the data to be updated;
the log generation module is configured to execute controlling a first database to persist the data to be updated to a shared storage system so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
the data synchronization module is configured to execute control of the first database to send the log point locations of the log file to a second database, so that the second database executes caching operation on the log file based on the log point locations; wherein the second database is a slave database in the MySQL database; the number of the memory occupied by the master database and the number of the cores of the central processing unit are both higher than that of the slave database.
In some possible embodiments, the apparatus further comprises:
a master-slave switching module configured to update currently occupied resources of the second database with occupied resources of the master database as a standard in response to a disaster recovery switching instruction; wherein the occupied resources comprise the number of the memory and the central processor cores;
updating the current occupied resource of the first database by taking the occupied resource of the slave database as a standard, and executing restarting operation on the updated first database;
and taking the updated second database as the master database, and taking the restarted first database as the slave database.
In some possible embodiments, the master-slave switching module is further configured to:
if the first database fails to restart, dividing equal amount of resources to be processed from the cloud system by taking the occupied resources of the slave database as a standard;
and generating a third database according to the resource to be processed, and taking the third database as the slave database.
In some possible embodiments, after performing the updating of the currently occupied resource of the second database based on the occupied resource of the master database, the master-slave switching module is further configured to:
and executing log playback and transaction rollback operations on the currently cached log file of the second database.
In some possible embodiments, after performing the caching operation on the log file by the second database, the data synchronization module is further configured to:
if a new data updating request is monitored, acquiring newly added data to be updated;
and after controlling the first database to send the log point location of the newly added data to be updated to the second database, controlling the second database to clear the current cache and then executing cache operation on the log file of the newly added data to be updated.
In some possible embodiments, the master database has permission to perform read-write operations on the shared storage system;
the slave database has a right to perform read-only operations on the shared storage system.
In a third aspect, an embodiment of the present application provides an electronic device, including:
a memory for storing program instructions;
a processor for calling the program instructions stored in the memory and executing the steps comprised in the method of any one of the first aspect according to the obtained program instructions.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program comprising program instructions that, when executed by a computer, cause the computer to perform the method of any one of the first aspects.
In a fifth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes: computer program code which, when run on a computer, causes the computer to perform the method of any one of the first aspects.
Drawings
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;
fig. 2 is a flowchart of an overall master-slave synchronization method of the MySQL database according to the embodiment of the present application;
fig. 3 is a schematic processing flow diagram of data to be updated according to an embodiment of the present application;
fig. 4 is a schematic diagram of master-slave switching according to an embodiment of the present application;
FIG. 5 is a system architecture diagram provided in accordance with an embodiment of the present application;
fig. 6 is a structure diagram of a master-slave synchronization apparatus 600 of a MySQL database according to an embodiment of the present application;
fig. 7 is a schematic view of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
The terms "first" and "second" in the description and claims of the present application and the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the term "comprises" and any variations thereof, which are intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. The term "a plurality" in the present application may mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
Currently, the highly available schemes of the MySQL database provided by the public cloud manufacturer mainly include the following two schemes: one is a data redundancy scheme based on a database master-slave replication technology, which mainly realizes the consistency of logs and data through Two-phase commit (2 PC), and then realizes the synchronization of remote data and local data through a data synchronization thread on the basis. The scheme at least needs two sets of databases configured with the same specification, and meanwhile, when the write operation requests are frequent, data delay between the synchronous databases is generated at a high probability, data loss is easily caused, and high availability is influenced. The other is a solution based on Paxos distributed consistency protocol. The scheme realizes the final consistency of the distributed database by combining the Paxos algorithm with the database so as to ensure high availability. However, this solution requires at least three databases with the same specification configuration to avoid the above data loss problem, and the performance of the distributed consistency protocol still needs to be improved and optimized due to the network overhead.
Therefore, in the design of the high-availability architecture in the related technology, at least two sets of database services with the same specification need to be redundantly deployed to form a cluster so as to realize high availability. The computing and storage resources for deploying the database service are at least two, which is high in cost for cloud computing manufacturers.
In order to solve the above problems, the inventive concept of the present application is: the data to be updated is persisted to the shared storage system by controlling the first database, so that a log file corresponding to the data to be updated is generated in the shared storage system. And controlling the first database to send the log point locations of the log files to the second database so that the second database caches the corresponding log files according to the log point locations. In the above process, the second database stores the log point locations of all log files generated in the system before the master-slave switching, rather than the complete log file. After master-slave switching, a corresponding log file can be found in the shared storage system only by inquiring prestored log point positions. Therefore, the slave database can support the master-slave synchronization function only by a very small resource amount, and the resource utilization rate is improved.
Referring to fig. 1, a schematic diagram of an application scenario according to an embodiment of the present application is shown.
As shown in fig. 1, the application scenario may include, for example, a network 10, a cloud server 20, and a business system 60 that includes a first database 30, a second database 40, and a shared storage system 50. Wherein the first database 30 holds much higher computing resources and storage resources than the second database 40.
Before the master-slave switching, the first database 30 is a master database in the system, and the second database 40 is a slave database in the system.
When a user sends a service request to the service system 60 through the front-end interface, data to be updated corresponding to the service request is sent to the shared storage system 50 by the first database 30 through the network 10, so as to store a log file corresponding to the data to be updated in the shared storage system 50. Further, the first database 30 sends the log point of the log file to the second database 40, so that the second database 40 loads the corresponding log file from the shared storage system 50 for caching based on the log point.
When the master database is abnormal, the master database and the slave database need to be switched. And the original slave database is upgraded into the master database to replace the abnormal master database to provide external services. Specifically, when the first database 30 is abnormal, the cloud server 20 allocates the same resource as the second database 40 to the currently abnormal first database 30 to restart the first database 30. Meanwhile, the configuration of the second database 40 is upgraded, and the held resources of the second database 40 are upgraded to be the same as the first database 30 before the abnormality. Finally, the master-slave switching is completed by using the restarted first database 30 as a slave database and using the upgraded second database 40 as a master database.
After introducing the application scenario of the present application, a master-slave synchronization method of the MySQL database provided in the embodiment of the present application is introduced below. As shown in fig. 2, the method comprises the following steps:
step 201: responding to a data updating request, and acquiring data to be updated;
the data to be updated can be understood as the data to be updated generated by the service system according to the request command after receiving the request command for representing the data update.
Step 202: controlling a first database to persist the data to be updated to a shared storage system so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
after the business system receives the data to be updated, the main database needs to update the database form of the main database aiming at the data to be updated, and meanwhile, a log file representing the data record of the data to be updated is generated. In the related art, independent storage spaces are mostly set for a master database and a slave database, and the master database and the slave database respectively store log files of data to be updated. The method is equivalent to that the log file of the master database is always backed up in the slave database so as to ensure the consistency of data after master-slave switching.
In the embodiment of the application, the shared storage system is arranged to store the log file of the data to be updated, so that the complete log file does not need to be transmitted between the master database and the slave database, and the resource requirement of the slave database is further reduced. In particular, shared storage refers to a parallel architecture in which two or more processors share a main memory. Each processor may store information in or retrieve information from the main memory. The shared storage system in the embodiment of the application is used for storing a log file of all data update records generated in the service system. That is, the master server and the slave server share the same storage area. Therefore, when data to be updated is generated in the business system, the first database needs to be controlled to persist the data to be updated into the shared storage system to generate a corresponding log file. And then sending the log point of the log file to a second database to implement data synchronization, which is described in detail in the following steps.
Step 203: controlling the first database to send the log point locations of the log files to a second database so that the second database performs cache operation on the log files based on the log point locations; wherein the second database is a slave database in the MySQL database; the number of the memory and the central processor core of the master database is higher than that of the slave database.
As mentioned above, the master-slave synchronization mechanism in the related art needs to control the master database to send the complete log file to the slave database. The main reason is that the master database and the slave database are respectively provided with independent storage spaces. In the embodiment of the application, the shared storage system is arranged to provide a piece of common storage area for the master database and the slave database. Therefore, the master database sends the log point (corresponding to the storage address of the corresponding log file in the shared storage system) of the record (namely, the log file) generated by each data change to the second database, and the second database is controlled to query the shared storage system according to the stored log point to find out all corresponding log files after master-slave switching. Compared with the transmission of a complete log file, the log point location requires few resources, so the process can greatly reduce the requirement of the database on the resources.
In practical application, a processing flow of a service system to data to be updated is shown in fig. 3, when a service request is received, a log file requesting for corresponding update data is generated, then the log file is flushed into a bottom-layer disk, and post-processing such as loading a configuration page is performed by calling the log file in the bottom-layer disk to realize external service of the update data. After the second database performs cache operation on the log file, if it is monitored that a new data update request exists in a service system, new update data is obtained; and after controlling the first database to send the log point location of the newly added updated data to the second database, controlling the second database to clear the current cache and then executing cache operation on the log file of the newly added updated data. Therefore, the latest generated log file in the service system can be cached in the second database all the time, so that the log file can be directly taken from the cache and subjected to post-processing during master-slave switching, and the time for reading the latest log file from the bottom-layer disk is saved.
In addition, the master database in the embodiment of the application has the authority to execute read-write operation on the shared storage system, and the slave database has the authority to execute read-only operation on the shared storage system, and the read-only operation is not open to the outside. By the above arrangement, the amount of resources required from the database can be further reduced. Specifically, in practical application, the log file (redo log) size of the MySQL database master library available for the public cloud height generally defaults to 2G. The computing resources allocated by the master database and the slave database in the related art are generally not lower than 2 cores and 4G memory. In the above flow of the application, the shared storage system is arranged to realize that the log point location is transmitted between the master database and the slave database to replace the complete log file, and the slave database is arranged to provide only the read-only service in the pair to further reduce the resource amount required by the slave database. Through tests, the slave database in the embodiment of the application can meet the requirement of data synchronization only by adopting low configuration of a 1-core 2G memory. And the 2G memory of the slave database is mainly used for caching the latest generated log file in the service system.
Furthermore, when a disaster recovery mechanism in the service system is triggered, that is, the master database has problems such as downtime and failure, the master-slave switching is required to be started to upgrade the second database from the original slave database to the master database, so as to replace the first database to continue providing service to the outside.
When the method is implemented, the current occupied resource of the second database is updated by taking the occupied resource of the main database as a standard; the occupied resources at least comprise the number of the internal memory and the central processing unit core. Meanwhile, updating the current occupied resources of the first database by taking the occupied resources of the slave database as a standard, and executing restart operation on the updated first database. And taking the updated second database as a master database, and taking the restarted first database as a slave database. In addition, if the first database fails to be restarted, the same amount of resources to be processed are divided from the cloud system by taking the occupied resources of the database as a standard. And after generating a third database according to the resource to be processed, taking the third database as a slave database.
Specifically, as shown in fig. 4, it is assumed that the configuration before the first database is down is 8 cores 32G, and the second database is 1 core 2G. And after the first database is down, upgrading the second database from 1 core 2G to 8 cores 32G. Meanwhile, the configuration of the first database is reduced from 8 cores 32G to 1 core 2G, and the first database is restarted. It is not shown in the figure that, assuming that the first database is restarted successfully within the specified time, the updated second database is taken as the master database, and the restarted first database is taken as the slave database. If the first database fails to be restarted within the specified time, in order to ensure the normal operation of the service system, the 1 core 2G computing resource needs to be divided from the system resource to construct a third database, and the newly constructed third database replaces the first database which fails to be restarted to execute the functions of the slave database.
After the master-slave switching is completed through the process, log playback and transaction rollback operations need to be executed on the currently cached log file of the second database. Namely, after master-slave switching, the second database directly takes the log file from the cache and carries out post-processing. Therefore, the time for reading the latest log file from the bottom disk can be saved.
To facilitate understanding of the technical solution provided by the present application, a system architecture of a master-slave synchronization method of a MySQL database provided in the embodiment of the present application is described below, and as shown in fig. 5, a service system 500 mainly includes: the system comprises a shared storage system 501, a database Primary computing node 502 (master database), a database replay computing node 503 (slave database), a data synchronization channel 504 and a data operation and maintenance center 505.
The points 502 and 503 are MySQL databases set by the master-slave synchronization method of the MySQL database provided by the present application.
Before master-slave switching, the Replica computing node 503 only caches the latest WAL log in the system, does not provide service for users, only supports part of master-slave computing node management commands, and only reads the database files on the shared storage system. When the main database is abnormal, the Replica computing node 503 is upgraded into a Primary computing node 502 for external service through online hot upgrade switching.
The data synchronization channel 504 synchronizes only the WAL (redo log) log site information of the Primary computation node 502 to the replay computation node 503. In contrast to traditional MySQL replication, there is no need to synchronize write operation logs (binlogs), i.e. complete log files. In addition, even if a log site is lost due to a network anomaly during synchronization, the WAL (redo log) log file stored on the shared storage system 501 is not affected. Therefore, the synchronous data volume of the master database and the slave database can be greatly reduced through the process, and the problem of master-slave communication data loss caused by communication network abnormality and the like is solved.
The data operation and maintenance center 505 is configured to monitor the survival status of the Primary and backup database nodes, and execute a Primary-secondary switching operation when the Primary computing node 502 is abnormal.
After detecting the abnormality of the master database, the data operation and maintenance center 505 notifies the shared storage system 501 to perform master-slave switching preparation, sets the original master database to provide read-only service, and performs backup-up slave operation on the Replica computing nodes 503.
Specifically, the Replica computing node 503 is upgraded to the same computing resource configuration as the abnormal master database by performing a line hot upgrade computing resource configuration. That is, the upgraded Replica computing node 503 is changed to the Primary computing node 502 before the abnormality. At this time, the upgraded Replica computing node 503 is set to read and write the database file in the shared storage system 501.
Further, the control upgraded Replica computing node 503 restores the memory data to be consistent with the memory data before the exception of the Primary database, so as to provide a service outside the exception of the abnormal Primary computing node 502. After the computing resources are allocated to the abnormal Primary computing node 502 again, the abnormal Primary computing node 502 is restarted, and the restarted Primary computing node 502 is used as a slave database. When the abnormal Primary computing node 502 is not restarted successfully within the specified time, in order to ensure the data security, resources required by the secondary database need to be divided from the business system to construct a new database. And establishes a data synchronization channel 504 between the new database and the master database.
Based on the same inventive concept, an embodiment of the present application provides a data synchronization apparatus 600 for MySQL database, as shown in fig. 6, including:
a data obtaining module 601 configured to perform obtaining data to be updated in response to a data update request;
a log generating module 602 configured to execute controlling a first database to persist the data to be updated to a shared storage system, so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
the data synchronization module 603 is configured to execute control of the first database to send the log point of the log file to a second database, so that the second database performs a caching operation on the log file based on the log point; wherein the second database is a slave database in the MySQL database; the number of the memory occupied by the master database and the number of the cores of the central processing unit are both higher than that of the slave database.
In some possible embodiments, the apparatus 600 further comprises:
the master-slave switching module is configured to execute updating of the current occupied resource of the second database by taking the occupied resource of the master database as a standard in response to a disaster recovery switching instruction; wherein the occupied resources comprise the number of the memory and the central processor cores;
updating the current occupied resource of the first database by taking the occupied resource of the slave database as a standard, and executing restarting operation on the updated first database;
and taking the updated second database as the master database, and taking the restarted first database as the slave database.
In some possible embodiments, the master-slave switching module is further configured to:
if the first database fails to be restarted, dividing equivalent resources to be processed from the cloud system by taking the occupied resources of the slave database as a standard;
and generating a third database according to the resource to be processed, and taking the third database as the slave database.
In some possible embodiments, after performing the updating of the currently occupied resource of the second database based on the occupied resource of the master database, the master-slave switching module is further configured to:
and executing log playback and transaction rollback operations on the currently cached log file of the second database.
In some possible embodiments, after performing the caching operation on the log file by the second database, the data synchronization module 603 is further configured to:
if a new data updating request is monitored, acquiring newly added data to be updated;
and after controlling the first database to send the log point locations of the newly added data to be updated to the second database, controlling the second database to clear the current cache and then executing cache operation on the log files of the newly added data to be updated.
In some possible embodiments, the master database has permission to perform read-write operations on the shared storage system;
the slave database has a right to perform read-only operations on the shared storage system.
The electronic device 130 according to this embodiment of the present application is described below with reference to fig. 7. The electronic device 130 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the electronic device 130 is represented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 that connects the various system components (including the memory 132 and the processor 131).
Bus 133 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
The memory 132 may include readable media in the form of volatile memory, such as Random Access Memory (RAM) 1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.
Memory 132 may also include a program/utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 130, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 130 to communicate with one or more other electronic devices. Such communication may occur via input/output (I/O) interfaces 135. Also, the electronic device 130 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 136. As shown, network adapter 136 communicates with other modules for electronic device 130 over bus 133. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, to name a few.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 132 comprising instructions, executable by the processor 131 of the apparatus 400 to perform the above-described method is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product comprising computer programs/instructions which, when executed by the processor 131, implement any of the master-slave synchronization methods of a MySQL database as provided herein.
In an exemplary embodiment, the aspects of the master-slave synchronization method for MySQL database provided by the present application may also be implemented in the form of a program product, which includes program code for causing a computer device to execute the steps in the master-slave synchronization method for MySQL database according to various exemplary embodiments of the present application described above in this specification when the program product is run on the computer device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The program product for master-slave synchronization of MySQL database of the embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code, and may run on an electronic device. However, the program product of the present application is not limited thereto, and in the present application, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the PowerPC programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on the consumer electronic device, as a stand-alone software package, partly on the consumer electronic device and partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., through the internet using an internet service provider).
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, according to embodiments of the application. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.
Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable image scaling apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable image scaling apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable image scaling apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable image scaling device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A master-slave synchronization method of a MySQL database is characterized by comprising the following steps:
responding to a data updating request, and acquiring data to be updated;
controlling a first database to persist the data to be updated to a shared storage system so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
controlling the first database to send the log point locations of the log files to a second database so that the second database performs caching operation on the log files based on the log point locations; wherein the second database is a slave database in the MySQL database; the number of the memory occupied by the master database and the number of the cores of the central processing unit are both higher than that of the slave database.
2. The method of claim 1, further comprising:
responding to a disaster recovery switching instruction, and updating the current occupied resource of the second database by taking the occupied resource of the main database as a standard; wherein the occupied resource comprises the memory and the number of the central processor cores;
updating the current occupied resource of the first database by taking the occupied resource of the slave database as a standard, and executing restarting operation on the updated first database;
and taking the updated second database as the master database, and taking the restarted first database as the slave database.
3. The method of claim 2, further comprising:
if the first database fails to restart, dividing equal amount of resources to be processed from the cloud system by taking the occupied resources of the slave database as a standard;
and generating a third database according to the resource to be processed, and taking the third database as the slave database.
4. The method according to claim 2, wherein after updating the currently occupied resource of the second database based on the occupied resource of the primary database, the method further comprises:
and executing log playback and transaction rollback operations on the currently cached log file of the second database.
5. The method of claim 1, wherein after the second database performs the caching operation on the log file, the method further comprises:
if a new data updating request is monitored, acquiring newly added data to be updated;
and after controlling the first database to send the log point location of the newly added data to be updated to the second database, controlling the second database to clear the current cache and then executing cache operation on the log file of the newly added data to be updated.
6. The method according to any one of claims 1-5, wherein the master database has permission to perform read and write operations on the shared storage system;
the slave database has a right to perform read-only operations on the shared storage system.
7. A master-slave synchronization apparatus for MySQL databases, the apparatus comprising:
the data acquisition module is configured to respond to the data updating request and acquire data to be updated;
the log generation module is configured to execute control of the first database to persist the data to be updated to a shared storage system so as to generate a log file of the data to be updated in the shared storage system; wherein the first database is a main database in the MySQL database;
the data synchronization module is configured to execute control of the first database to send the log point location of the log file to a second database so that the second database executes a caching operation on the log file based on the log point location; wherein the second database is a slave database in the MySQL database; the number of the memory occupied by the master database and the number of the cores of the central processing unit are both higher than that of the slave database.
8. An electronic device, comprising:
a memory for storing program instructions;
a processor for calling program instructions stored in said memory and for executing the steps comprised in the method of any one of claims 1 to 6 in accordance with the obtained program instructions.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the method according to any of claims 1-6 when executed by a processor.
CN202210849391.9A 2022-07-19 2022-07-19 Master-slave synchronization method, device, equipment and medium for MySQL database Pending CN115292408A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210849391.9A CN115292408A (en) 2022-07-19 2022-07-19 Master-slave synchronization method, device, equipment and medium for MySQL database

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210849391.9A CN115292408A (en) 2022-07-19 2022-07-19 Master-slave synchronization method, device, equipment and medium for MySQL database

Publications (1)

Publication Number Publication Date
CN115292408A true CN115292408A (en) 2022-11-04

Family

ID=83824882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210849391.9A Pending CN115292408A (en) 2022-07-19 2022-07-19 Master-slave synchronization method, device, equipment and medium for MySQL database

Country Status (1)

Country Link
CN (1) CN115292408A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515574A (en) * 2021-05-17 2021-10-19 聚好看科技股份有限公司 Data synchronization method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515574A (en) * 2021-05-17 2021-10-19 聚好看科技股份有限公司 Data synchronization method and device
CN113515574B (en) * 2021-05-17 2023-05-19 聚好看科技股份有限公司 Data synchronization method and device

Similar Documents

Publication Publication Date Title
EP3694148B1 (en) Configuration modification method for storage cluster, storage cluster and computer system
US6823474B2 (en) Method and system for providing cluster replicated checkpoint services
US11262918B1 (en) Data storage system with uneven drive wear reduction
KR20110044858A (en) Maintain data indetermination in data servers across data centers
EP3745269B1 (en) Hierarchical fault tolerance in system storage
US20140258659A1 (en) Protection of former primary volumes in a synchronous replication relationship
US20110078396A1 (en) Remote copy control method and system in storage cluster environment
CN111177161A (en) Data processing method and device, computing equipment and storage medium
US20120278429A1 (en) Cluster system, synchronization controlling method, server, and synchronization controlling program
CN115858236A (en) Data backup method and database cluster
CN112434008A (en) Distributed database upgrading method, device and medium
CN115599747A (en) Metadata synchronization method, system and equipment of distributed storage system
CN114003350B (en) Data distribution method and system of super-fusion system
CN110377664B (en) Data synchronization method, device, server and storage medium
CN115292408A (en) Master-slave synchronization method, device, equipment and medium for MySQL database
WO2020135889A1 (en) Method for dynamic loading of disk and cloud storage system
WO2023019953A1 (en) Data synchronization method and system, server, and storage medium
CN115587141A (en) Database synchronization method and device
CN115658390A (en) Container disaster tolerance method, system, device, equipment and computer readable storage medium
US9710298B2 (en) Information processing system, storage apparatus, and program
CN113934575A (en) Big data backup system and method based on distributed copy
JPH09293001A (en) Non-stop maintenance system
CN115687250A (en) Storage method, equipment, system and computer storage medium
JP5956364B2 (en) Cluster system
WO2016046951A1 (en) Computer system and file management method therefor

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination