CN118312363A - Data processing method, device, storage medium and program product - Google Patents

Abstract

The embodiment of the invention discloses a data processing method, data processing equipment, a storage medium and a program product. The method is performed by a master node in a database cluster, the database cluster comprising one of the master nodes and a plurality of slave nodes, comprising: receiving a database operation request and generating a first type log based on the database operation request; sending the first type log to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log; writing the first type log to a physical storage medium; and when the message of the first type log which is sent by the nodes of the plurality of groups and is written into the physical storage medium is received, executing the database operation request. According to the data processing method provided by the embodiment of the invention, the main node executes the database operation request when receiving the information that the first type log sent by the plurality of groups of nodes is written into the physical storage medium, so that the consistency of data among all nodes in the database cluster can be ensured.

Description

Data processing method, device, storage medium and program product

Technical Field

Embodiments of the present invention relate to the field of database technologies, and in particular, to a data processing method, device, storage medium, and program product.

Background

Database group replication (MySQL Group Replication, MGR) employs Paxos multiple-dispatch protocol to ensure data consistency between at least multiple-dispatch replica nodes. However, because the MGR achieves the most common request to save in the memory, if there is a relatively large delay between the master node and the slave nodes, when all the nodes are down, there may be more data submitted on the master node than the slave nodes before the down, if the slave nodes are selected as new master nodes to reconstruct the cluster after the down, there is a case of losing data, and if the old master nodes rejoin the cluster after the old master nodes, there may be a problem that they are inconsistent with the data of the other nodes.

Disclosure of Invention

The embodiment of the invention provides a data processing method, data processing equipment, a storage medium and a program product, which can ensure the consistency of data among nodes in a database cluster.

In a first aspect, an embodiment of the present invention provides a data processing method, where the method is performed by a master node in a database cluster, where the database cluster includes one master node and a plurality of slave nodes, and includes:

receiving a database operation request and generating a first type log based on the database operation request;

Sending the first type log to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log;

Writing the first type log to a physical storage medium;

And when the message of the first type log which is sent by the nodes of the plurality of groups and is written into the physical storage medium is received, executing the database operation request.

In a second aspect, an embodiment of the present invention further provides a data processing method, where the method is performed by a slave node in a database cluster, where the database cluster further includes a master node and other slave nodes, and includes:

Receiving a first type log sent by the master node so as to achieve majority consensus with the master node and the rest slave nodes on the first type log;

Writing the first type log into a physical storage medium, and sending a message that the first type log is completely written into the physical storage medium to the master node;

And executing the database operation request when receiving the information which is sent by at least one of the master node and the rest slave nodes and is executed by the database operation request.

In a third aspect, an embodiment of the present invention further provides a data processing method, where the method is applied to a database cluster, where the database cluster includes a master node and a plurality of slave nodes; comprising the following steps:

The master node receives a database operation request and generates a first type log based on the database operation request;

the master node sends the first type log to the plurality of slave nodes so that the master node and the plurality of slave nodes achieve majority consensus on the first type log;

the master node and the plurality of slave nodes writing the first type log to a physical storage medium;

When the master node receives a message that the first type log sent by a plurality of groups of nodes is written into a physical storage medium, the master node executes the database operation;

And when the slave node receives the information which is sent by at least one of the master node and the rest slave nodes and is used for executing the database operation request, executing the database operation request.

In a fourth aspect, an embodiment of the present invention further provides an electronic device, including:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data processing method according to the embodiment of the present invention.

In a fifth aspect, an embodiment of the present invention further provides a computer readable storage medium, where computer instructions are stored, where the computer instructions are configured to cause a processor to execute the data processing method according to the embodiment of the present invention.

In a sixth aspect, the embodiment of the present invention further provides a computer program product, including a computer program, which when executed by a processor implements the data processing method according to the embodiment of the present invention.

The embodiment of the invention discloses a data processing method, data processing equipment, a storage medium and a program product. Receiving a database operation request and generating a first type log based on the database operation request; transmitting the first type log to a plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log; writing a first type of log to a physical storage medium; and when the message of the first type log which is sent by the nodes of the plurality of groups and is written into the physical storage medium is received, executing a database operation request. According to the data processing method provided by the embodiment of the invention, the main node executes the database operation request when receiving the information that the first type log sent by the plurality of groups of nodes is written into the physical storage medium, so that the consistency of data among all nodes in the database cluster can be ensured.

Drawings

FIG. 1 is a flow chart of a data processing method in accordance with a first embodiment of the present invention;

FIG. 2 is a flow chart of a data processing method in a second embodiment of the invention;

FIG. 3 is a flow chart of a data processing method in a third embodiment of the invention;

Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.

Example 1

Fig. 1 is a flowchart of a data processing method according to a first embodiment of the present invention, where the method may be applied to processing a database operation request, and the method may be performed by a data processing apparatus, where the apparatus may be implemented in software and/or hardware, and optionally, implemented by an electronic device, where the electronic device may be a mobile terminal, a PC side, a server, or the like. In this embodiment, the data processing apparatus is disposed in a master node in a database cluster, where the database cluster includes a master node and a plurality of slave nodes, as shown in fig. 1, and the method specifically includes the following steps:

s110, receiving a database operation request and generating a first type log based on the database operation request.

The database operation request may include a query data operation, an insert data operation, an update data operation, and a delete data operation. The first type log may be a database oplog constructed based on a first type protocol. In this embodiment, the first type of protocol may be a distributed coherence (Paxos) protocol, which may also be referred to as a multi-dispatch protocol, i.e., the first type of log may be referred to as a Paxos log.

In this embodiment, an application (also referred to as an application layer) connected to the database sends a database operation request to a master node in the database cluster, and the master node generates a Paxos log based on the database operation request after receiving the database operation request.

S120, the first type log is sent to the plurality of slave nodes so as to achieve majority consensus with the plurality of slave nodes on the first type log.

Wherein sending the first type of log to the plurality of slave nodes may be understood as the master node sending the first type of log to all slave nodes in the cluster. The manner in which the master node and the plurality of slave nodes perform multiple consensus on the first type log may be: the Paxos algorithm is used for majority consensus. The specific principle can be as follows: the master node sends the first type log to the plurality of slave nodes, so that the master node and the plurality of slave nodes vote on the first type log, the master node collects voting results of all nodes, and if more than half of the voting results of the nodes agree with the first type log, the master node and the plurality of slave nodes achieve majority consensus on the first type log.

S130, writing the first type log into a physical storage medium.

Wherein the physical storage medium may be a storage disk. In this embodiment, after the master node and the plurality of slave nodes have reached a majority consensus on the first type log, both the master node and the slave nodes perform an operation of writing the first type log to the physical storage medium. After each slave node finishes the operation of writing the first type log into the physical storage medium, the slave node also sends the message of writing the first type log into the physical storage medium to the master node and other slave nodes, namely the slave node sends the message of writing the first type log into the physical storage medium to all other nodes in the cluster, including the master node and all other slave nodes.

And S140, when receiving the message that the first type log sent by the nodes of the plurality of groups completes writing the physical storage medium, executing a database operation request.

Wherein a plurality of groups of nodes may be understood as more than half of the nodes. In this embodiment, the master node is responsible for collecting the information that each node completes writing the first type log into the physical storage medium, and when more than half of the nodes complete writing the first type log into the physical storage medium, the master node executes the database operation request. If the database operation request is the data query operation, the master node executes the data query operation; if the database operation request is the data insertion operation, the master node executes the data insertion operation; if the database operation request is the data updating operation, the master node executes the data updating operation; and if the database operation request is a data deleting operation, the master node executes the data deleting operation.

In this embodiment, a database operation request is executed at a master node, and information of the executed database operation request is sent to a plurality of slave nodes.

Optionally, after generating the first type log based on the database operation request, the method further includes the steps of: the first type log is assigned a number. Accordingly, the process of sending the first type log to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log may be: transmitting the first type log and the number to the plurality of slave nodes to achieve majority consensus with the plurality of slave nodes on the first type log and the number; the process of writing the first type of log to the physical storage medium may be: the first type log and number are written to the physical storage medium.

The number may be understood as a unique identifier of the first type of log, and each first type of log corresponds to a unique number. In this embodiment, the manner of assigning numbers to the first type log may be: and generating the number of the current first type log according to a preset coding mode (such as sequence number).

Wherein sending the first type log and number to the plurality of slave nodes can be understood as: the first type log and number are sent to all slave nodes in the cluster.

The manner in which the master node and the plurality of slave nodes achieve multiple consensus on the first type log and the number may be: the Paxos algorithm is used to perform majority consensus on the first type of log and number. The specific principle can be as follows: the master node sends the first type log and the number to the plurality of slave nodes so that the master node and the plurality of slave nodes vote on the first type log and the number, the master node collects voting results of all the nodes, and if more than half of the voting results of the nodes agree with the first type log and the number, the master node and the plurality of slave nodes achieve majority consensus on the first type log and the number.

In this embodiment, after the master node and the plurality of slave nodes have reached a majority consensus on the first type log and the number, the master node and the slave nodes each perform an operation of writing the first type log and the number to the physical storage medium. After each slave node completes the operation of writing the first type log and the number into the physical storage medium, the slave node also sends a message that the first type log and the number are completely written into the physical storage medium to the master node. In this embodiment, a number is allocated to the first type log, which is beneficial to searching the first type log later.

According to the technical scheme, a database operation request is received, and a first type log is generated based on the database operation request; transmitting the first type log to a plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log; writing a first type of log to a physical storage medium; and when the message of the first type log which is sent by the nodes of the plurality of groups and is written into the physical storage medium is received, executing a database operation request. According to the data processing method provided by the embodiment of the invention, the main node executes the database operation request when receiving the information that the first type log sent by the plurality of groups of nodes is written into the physical storage medium, so that the consistency of data among all nodes in the database cluster can be ensured.

Example two

Fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention, where on the basis of the foregoing embodiment, the present embodiment may be adapted to process a database operation request, and the method may be performed by a data processing apparatus, where the apparatus may be implemented in a form of software and/or hardware, and optionally, implemented by an electronic device, where the electronic device may be a mobile terminal, a PC side, a server, or the like. In this embodiment, the data processing apparatus is disposed in a slave node in a database cluster, where the database cluster includes a master node and a plurality of slave nodes, as shown in fig. 2, and the method specifically includes the following steps:

s210, receiving the first type log sent by the master node, so as to achieve majority consensus with the master node and other slave nodes on the first type log.

Wherein the first type log may be generated by the master node according to a database operation request. The first type log may be a database oplog constructed based on a first type protocol. In this embodiment, the first type of protocol may be a distributed coherence (Paxos) protocol, which may also be referred to as a multi-dispatch protocol, i.e., the first type of log may be referred to as a Paxos log. The manner in which the slave node performs majority consensus with the master node and the remaining slave nodes on the first type log may be referred to the above embodiments, and will not be described herein.

S220, the first type log is written into the physical storage medium, and a message that the first type log is written into the physical storage medium is sent to the master node.

Wherein the physical storage medium may be a storage disk. In this embodiment, the slave node sends the message that the first type log completes writing into the physical storage medium to the master node and also sends the message to the rest of the slave nodes, that is, the current slave node sends the message that the first type log completes writing into the physical storage medium to all the nodes in the cluster, including the master node and the rest of the slave nodes. In this embodiment, after the slave node and the master node and the other slave nodes have a majority consensus on the first type log, the slave node writes the first type log into the physical storage medium, and sends a message that the first type log is completely written into the physical storage medium to the master node.

And S230, when receiving the information of the executed database operation request sent by at least one of the master node and the rest of slave nodes, executing the database operation request.

Specifically, the slave node executes the database operation request when receiving the information of the executed database operation request sent by at least one of the master node and the other slave nodes, so that the data consistency among the nodes can be ensured.

Optionally, before writing the first type log to the physical storage medium, the method further comprises the steps of: receiving the serial number of a first type log sent by a master node; accordingly, the process of achieving majority consensus with the master node and the remaining slave nodes for the first type log may be: multiple consensus is achieved for the first type log and the numbers with the master node and other slave nodes; the process of writing the first type of log to the physical storage medium may be: the first type log and number are written to the physical storage medium.

Wherein the number of the first type log is assigned by the master node. The numbers may be understood as unique identifications of the first type of logs, each first type of log corresponding to a unique one of the numbers. The manner in which the slave node and the master node and the remaining slave nodes achieve multiple consensus on the first type log and the number can be referred to the above embodiments, and will not be described herein again.

Optionally, after writing the first type log and the number to the physical storage medium, further comprising: and sending the first type log and the information of the number which is written into the physical storage medium to the master node, so that the master node executes the database operation request when receiving the information of the first type log which is sent by the plurality of nodes and is written into the physical storage medium.

According to the technical scheme, a first type log sent by a master node is received, so that a plurality of common identities are achieved for the first type log with the master node and other slave nodes; writing the first type log into a physical storage medium, and sending a message that the first type log is completely written into the physical storage medium to a master node; and executing the database operation request when receiving the information of the executed database operation request sent by at least one of the master node and the rest of the slave nodes. According to the data processing method provided by the embodiment of the invention, the slave node executes the database operation request when receiving the information of the executed database operation request sent by at least one of the master node and the rest of the slave nodes, so that the consistency of data among all nodes in the database cluster can be ensured.

Example III

Fig. 3 is a flowchart of a data processing method according to a third embodiment of the present invention, where on the basis of the foregoing embodiment, the present embodiment may be adapted to process a database operation request, and the method may be performed by a data processing apparatus, where the apparatus may be implemented in a form of software and/or hardware, and optionally, implemented by an electronic device, where the electronic device may be a mobile terminal, a PC side, a server, or the like. In this embodiment, the data processing device is disposed in a database cluster, where the database cluster includes a master node and a plurality of slave nodes, as shown in fig. 3, and the method specifically includes the following steps:

S310, the master node receives the database operation request and generates a first type log based on the database operation request.

S320, the master node sends the first type log to the plurality of slave nodes, so that the master node and the plurality of slave nodes achieve majority consensus on the first type log.

Wherein the master node sending the first type log to the plurality of slave nodes may be understood as the master node sending the first type log to all slave nodes in the cluster.

S330, the master node and the plurality of slave nodes write the first type log into the physical storage medium.

S340, when the master node receives the message that the first type log sent by the plurality of nodes completes writing into the physical storage medium, the master node executes database operation.

And S350, when the slave node receives the information of the executed database operation request sent by at least one of the master node and the rest slave nodes, executing the database operation request.

The specific implementation of S310-S350 may be referred to the above embodiments, and will not be described herein.

Optionally, after generating the first type log based on the database operation request, the method further includes the steps of: the master node allocates numbers to the first type logs; accordingly, the process in which the master node sends the first type log to the plurality of slave nodes so that the master node and the plurality of slave nodes achieve the majority consensus on the first type log may be: the master node sends the first type log and the number to the plurality of slave nodes, so that the master node and the plurality of slave nodes achieve majority consensus on the first type log and the number; the process by which the master node and the plurality of slave nodes write the first type log to the physical storage medium may be: the master node and the plurality of slave nodes write the first type log and the number to the physical storage medium.

Optionally, before the master node receives the database operation request, the method further includes the following steps: starting a plurality of dispatch instance nodes; respectively acquiring target numbers from a plurality of started instance nodes to acquire a plurality of target numbers; the target number is the number of the largest first type log in each instance node; and taking the instance node corresponding to the maximum target number as a master node to construct a database cluster.

In this embodiment, when a cluster needs to be built or re-built, a plurality of instance nodes need to be restarted. And storing a plurality of first type logs and corresponding numbers thereof in physical storage media of each started instance node, and respectively acquiring the number of the largest first type log from the started instance node as a target number. The number of the largest first type log may characterize the number of first type logs stored by the instance node. And finally, taking the instance node corresponding to the largest target number as a master node and the rest instance nodes as slave nodes to construct a database cluster, so that the maximum database operation executed by the master node is ensured, and the condition of losing data of the constructed database cluster can be prevented.

Optionally, after taking the instance node corresponding to the largest target number as the master node, the method further comprises the following steps: each instance node acquires a first type log stored by the instance node and a second type log corresponding to the first type log; and if the second type log corresponding to the first type log is not obtained, generating the second type log based on the first type log, and writing the second type log into a physical storage medium.

The second type log may be a database operation log generated based on a second type protocol, the second type protocol may be a binlog protocol, and the second type log may be referred to as a binlog log. The binlog log may be used for incremental synchronization by third party components.

In this embodiment, the second type log is also generated based on the database operation request, and the second type log is asynchronously written into the physical storage medium, however, when the node is down, the second type log may be lost, so when the second type log corresponding to the first type log is not obtained, the second type log is generated based on the first type log, and the second type log is written into the physical storage medium, that is, the binlog log needs to be compensated based on the Paxos log.

The solution of this embodiment has the following advantages: the data consistency is ensured by using the Paxos log, the binlog is changed into asynchronous writing, and the binlog which is lost due to downtime is compensated from the Paxos log when the node is restarted, so that the influence of the persistence log on the system performance is reduced to the greatest extent. After the Paxos logs of the most groups are persisted to the physical storage medium, the numbers are persisted together, so that the most groups of nodes are selected according to the size of the message numbers after being started, and the main selection logic of the high-availability modules is greatly simplified. The binlog is not modified and the third party component that depends on the binlog is not affected.

According to the technical scheme of the embodiment, a master node receives a database operation request and generates a first type log based on the database operation request; the master node sends the first type log to a plurality of slave nodes, so that the master node and the plurality of slave nodes achieve majority consensus on the first type log; the master node and the plurality of slave nodes write the first type log into the physical storage medium; when the master node receives a message that the first type log sent by a plurality of groups of nodes is written into a physical storage medium, the master node executes database operation; and when the slave node receives the information of the executed database operation request sent by at least one of the master node and the rest of slave nodes, executing the database operation request. The consistency of data among nodes in the database cluster can be ensured.

Example 4

Fig. 4 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 4, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with the rest of the device via a computer network, such as the internet, and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as data processing methods.

In some embodiments, the data processing method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. One or more of the steps of the data processing method described above may be performed when the computer program is loaded into RAM 13 and executed by processor 11. Alternatively, in other embodiments, the processor 11 may be configured to perform the data processing method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

Embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements a data processing method as provided by any of the embodiments of the present application.

Computer program product in the implementation, the computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on the design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (11)

1. A data processing method, the method being performed by a master node in a database cluster, the database cluster comprising one of the master nodes and a plurality of slave nodes, comprising:

receiving a database operation request and generating a first type log based on the database operation request;

Sending the first type log to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log;

Writing the first type log to a physical storage medium;

And when the message of the first type log which is sent by the nodes of the plurality of groups and is written into the physical storage medium is received, executing the database operation request.

2. The method of claim 1, further comprising, after generating a first type log based on the database operation request:

assigning a number to the first type log;

accordingly, sending the first type log to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log includes:

Sending the first type log and the number to the plurality of slave nodes to achieve a majority consensus with the plurality of slave nodes for the first type log and the number;

Writing the first type log to a physical storage medium, comprising:

The first type log and the number are written to a physical storage medium.

3. A data processing method, the method being performed by a slave node in a database cluster, the database cluster further comprising a master node and remaining slave nodes, comprising:

Receiving a first type log sent by the master node so as to achieve majority consensus with the master node and the rest slave nodes on the first type log;

Writing the first type log into a physical storage medium, and sending a message that the first type log is completely written into the physical storage medium to the master node;

And executing the database operation request when receiving the information which is sent by at least one of the master node and the rest slave nodes and is executed by the database operation request.

4. The method of claim 3, further comprising, prior to writing the first type of log to a physical storage medium:

receiving the serial number of the first type log sent by the master node;

accordingly, the majority consensus with the master node and the remaining slave nodes for the first type log is achieved, including:

a majority consensus is achieved with the master node and the remaining slave nodes for the first type log and the number;

Writing the first type log to a physical storage medium, comprising:

The first type log and the number are written to a physical storage medium.

5. A data processing method, wherein the method is applied to a database cluster, and the database cluster comprises a master node and a plurality of slave nodes; comprising the following steps:

The master node receives a database operation request and generates a first type log based on the database operation request;

the master node sends the first type log to the plurality of slave nodes so that the master node and the plurality of slave nodes achieve majority consensus on the first type log;

the master node and the plurality of slave nodes writing the first type log to a physical storage medium;

When the master node receives a message that the first type log sent by a plurality of groups of nodes is written into a physical storage medium, the master node executes the database operation;

And when the slave node receives the information which is sent by at least one of the master node and the rest slave nodes and is used for executing the database operation request, executing the database operation request.

6. The method of claim 5, further comprising, after generating a first type log based on the database operation request:

the master node assigns numbers to the first type logs;

accordingly, the master node sending the first type log to the plurality of slave nodes such that the master node and the plurality of slave nodes agree on a majority consensus for the first type log, comprising:

The master node sends the first type log and the number to the plurality of slave nodes, so that the master node and the plurality of slave nodes achieve majority consensus on the first type log and the number;

the master node and the plurality of slave nodes writing the first type log to a physical storage medium, comprising:

The master node and the plurality of slave nodes write the first type log and the number to a physical storage medium.

7. The method of claim 5, further comprising, prior to the master node receiving a database operation request:

Starting a plurality of dispatch instance nodes;

Respectively acquiring target numbers from the started multiple groups of instance nodes to acquire multiple target numbers; the target number is the number of the largest first type log in each instance node;

and taking the instance node corresponding to the maximum target number as a master node to construct a database cluster.

8. The method according to claim 7, further comprising, after taking the instance node corresponding to the largest target number as the master node:

each instance node acquires a first type log stored by the instance node and a second type log corresponding to the first type log;

And if the second type log corresponding to the first type log is not obtained, generating the second type log based on the first type log, and writing the second type log into the physical storage medium.

9. An electronic device, the electronic device comprising:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for causing a processor to implement the data processing method of any one of claims 1-8 when executed.

11. A computer program product comprising a computer program which, when executed by a processor, implements the data processing method of any of claims 1-8.