WO2022036901A1

WO2022036901A1 - Implementation method and apparatus for redis replica set

Info

Publication number: WO2022036901A1
Application number: PCT/CN2020/130220
Authority: WO
Inventors: 于占水
Original assignee: 紫光云(南京)数字技术有限公司
Priority date: 2020-08-20
Filing date: 2020-11-19
Publication date: 2022-02-24
Also published as: CN112100005B; CN112100005A

Abstract

An implementation method and apparatus for a Redis replica set, the method comprising the following steps: S1. using a preset method to detect the activity of a proxy, and achieving high availability for the proxy; S2. achieving automatic failover, manual master-slave node switching, automatic sentinel switching, self-rescue of failed nodes, and split-brain multi-master detection by means of preset rules; and S3. the proxy obtains the latest structure information of a database Redis by using a preset rule, and corrects a routing request. The beneficial effects are: by means of collaboratively using Proxy, KeepAlived, Redis-server, Redis-sentinel and Check-sentinel scripts, the lowest configuration Redis can be used to achieve a high-availability solution, which not only effectively solves the problem in which split brain and virtual machine downtime cannot be switched, but also effectively solves the multi-master problem after a virtual machine is started. At the same time, the addition of the proxy can provide protection for Redis, such as the shielding of performance loss commands and the client connection full filling of Redis causing Redis to exit abnormally and so on.

Description

A method and device for implementing a Redis replica set

technical field

The present invention relates to the technical field of databases, and in particular, to a method and device for implementing a Redis replica set.

Background technique

At present, in online blogs or many project applications, many Redis replica sets are built with one master, two slaves, three sentinels, multiple sentinels, or one master, one slave, and two sentinels. The minimum configuration method of one master, one slave, and one sentinel is rarely mentioned. The common schemes are as follows: Scheme 1 (as shown in Figure 1) is one master, one slave and two sentinels, M1 and R1 are the master and slave Redis respectively, and S1 and S2 are high-availability sentinels; Scheme 2 (as shown in Figure 2) is one Master-slave three sentinels, if deployed separately, three sentinels will take up more resources; if deployed together with Redis, it will also encounter the problem of failover; solution 3 (as shown in Figure 3) is one master, two slaves, and three sentinels , is still not the minimum configuration, and consumes more resources; in addition, without an agent, users need to perceive all node information such as IP, PORT, etc., and all administrator operations such as FLUSH can be performed through the client, which brings security risks.

However, for the above scheme 1, it has the following defects: 1) it cannot solve the problem of split brain after a network failure; 2) if one of the sentinel processes is down or a certain virtual machine is down, the failover cannot be carried out smoothly; 3 ) The client needs to clear the underlying deployment architecture information such as IP, PORT, etc., and can perform all administrator operations such as FLUSH.

For the above scheme 2, add a sentinel; if three sentinels are deployed separately, it will cause additional resource consumption, and the sentinel process is just a failover and will not occupy too many resources; if the sentinel process is deployed together with Redis, there will also be Two obvious shortcomings: 1) If the virtual machine with many sentinels goes down, the failover cannot be carried out smoothly; 2) The client needs to clarify the underlying deployment architecture information such as IP, PORT, etc., and can perform all administrator operations such as FLUSH et al.

For the above scheme 3, one master, two slaves and three sentinels, adding an additional physical resource is not the minimum configuration.

For the problems in the related technologies, no effective solutions have been proposed so far.

SUMMARY OF THE INVENTION

Aiming at the problems in the related art, the present invention proposes a method and device for implementing a Redis replica set, so as to overcome the above-mentioned technical problems existing in the related art.

For this reason, the concrete technical scheme that the present invention adopts is as follows:

According to an aspect of the present invention, a method for implementing a Redis replica set is provided, comprising the following steps:

S1. Use a preset method to detect the activity of the agent, so as to realize the high availability of the agent;

S2. Implement automatic failover, manual master-slave node switching, sentinel automatic switching, fault node self-rescue and split-brain multi-master detection through preset rules;

S3. The agent obtains the latest structure information of the Redis database by using a preset rule, and routes the request correctly.

Further, in the S1, a preset method is used to detect the activity of the agent, and the realization of the high availability of the agent specifically includes the following steps: using the preset software KeepAlived to control the VIP offset by detecting the agent activity, so as to realize the high availability of the agent.

Further, the agent in the S1 is the open source agent Predixy, the open source agent Predixy can shield some password reset and parameter modification commands, and the user does not need to perceive the underlying architecture deployment information, and connecting the agent is like connecting to a single database. Redis, which can implement subsequent replica set expansion.

Further, implementing automatic failover, manual master-slave node switching, sentinel automatic switching, faulty node self-rescue and split-brain multi-master detection through preset rules in S2 specifically includes the following steps:

S21. After the replica set is initialized, the sentinel runs on the slave node, and periodically obtains the status of the database Redis and is responsible for the automatic failover of the database Redis and manual master-slave server switching;

S22. After the master-slave switch is completed, the periodic triggering of the Check-sentinel script will detect that the sentinel running node does not match the database Redis role, and then the sentinel will be correspondingly switched to the current slave node server by the Check-sentinel script.

Further, periodically triggering Check-sentinel script in described S22 specifically comprises the following steps:

When the server is the main node server and there is a sentinel, the sentinel process is stopped, the agent process configuration file soft connection points to the local, the modified soft connection points to the remote agent configuration file, and the agent is restarted;

When the server is a slave node server and there is no sentinel, start the sentinel process, the agent process configuration file soft connection points to the remote end, modify the soft connection to point to the local agent configuration file, and restart the agent;

When the PING command returns an error, it is regarded as a slave node server, and it is accumulated once every cycle. If it still fails 5 times, the soft connection of the proxy configuration file is modified to point to the local configuration file.

Further, the preset rule for the agent to obtain the latest structure information of the database Redis in the S3 refers to that the agent periodically obtains from the sentinel.

According to another aspect of the present invention, an electronic device is also provided, the electronic device includes a memory and a processor, the memory stores an implementation program of a Redis replica set that can run on the processor, the When the implementation program of the Redis replica set is executed by the processor, the following steps are implemented:

S22. When the master-slave switch is completed, the periodic triggering of the Check-sentinel script will detect that the sentinel running node does not match the database Redis role, and then the sentinel will be correspondingly switched to the current slave node server by the Check-sentinel script.

Further, periodically triggering the Check-sentinel script in S22 specifically includes the following steps:

The beneficial effects of the present invention are as follows: through the coordinated use of the components Proxy, KeepAlived, Redis-server, Redis-sentinel and Check-sentinel scripts, the present invention can use the lowest-configured Redis to realize the high-availability scheme, which not only effectively solves the problem of brain It also effectively solves the multi-master problem after the virtual machine is started. At the same time, the addition of the proxy can provide a kind of protection for Redis, such as the shielding of performance loss commands and the client connection full of Redis. Cause Redis to exit abnormally, etc.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is the schematic diagram of the construction method of Redis replica set scheme 1 in the prior art;

Fig. 2 is a schematic diagram of the construction method of Redis replica set solution 2 in the prior art;

FIG. 3 is a schematic diagram of the construction method of Redis replica set solution 3 in the prior art;

4 is a schematic flowchart of a method for implementing a Redis replica set according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of a principle framework in a method for implementing a Redis replica set according to an embodiment of the present invention.

detailed description

In order to further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention, and are mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operation principles of the embodiments. For these, those of ordinary skill in the art will understand other possible implementations and the advantages of the present invention. Components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

According to the embodiments of the present invention, a method and apparatus for implementing a Redis replica set are provided.

The present invention will now be further described with reference to the accompanying drawings and specific embodiments. As shown in Figures 4-5, according to an embodiment of the present invention, a method for implementing a Redis replica set is provided, including the following steps:

The S1 specifically includes the following steps: using the preset software KeepAlived to control the VIP offset by detecting the activity of the proxy (Proxy), so as to realize the high availability of the proxy.

Specifically, the agent in the S1 is the open source agent Predixy, which can shield some password reset, parameter modification and loss performance commands such as SCAN, KEYS and other commands, and the user does not need to perceive the underlying architecture deployment information, connect The agent is like connecting to a single database Redis, which can realize subsequent replica set expansion.

Wherein, the S2 specifically includes the following steps:

Specifically, the failover in S21 includes the following steps:

Select a slave node server from all the slave node servers of the offline master node server, and convert it to the master node server; preferably, the election principle for converting the slave node server to the master node server is as follows: first, delete the nodes in the list. The slave node server in offline or disconnected state ensures that all servers are online; then delete the server that has not responded to the leading sentinel in the last 5 seconds to ensure that all servers can communicate normally; finally delete the offline server. The master node server disconnects the slave node server that exceeds the preset time, and is used to filter out the slave database of newer data. After deletion, the slave node server with the highest priority and the largest replication offset will be set first. If it cannot be selected, the slave node server with the smallest running ID will be selected;

When a new master node server is elected, the sentinel will send the INFO command at a frequency of once per second. When the upgraded role attribute is upgraded from slave to master, it is considered to have been upgraded to the master node server. After detecting that the selected slave node server has been upgraded to the master node server, it will continue to send the SLAVEOF command to the other slave node servers to change the settings of the master node server.

Let the other slave node servers of the offline master node server replicate the new master node server;

Set the offline master node server as the slave node server of the new master node server. When the offline master node server comes back online, it will become the slave node server of the new master node server.

Specifically, the old master node server has been offline, so the settings are configured in the old master node server object of the leading sentinel. When the old node master server goes online, the sentinel will reissue the SLAVEOF command.

Specifically, the Check-sentinel script in S22 is triggered periodically, such as every 3 seconds, as follows:

When the PING command returns an error (Error), it is regarded as a slave node server, and it is accumulated once every cycle. If it still fails 5 times, then modify the proxy configuration file soft connection to point to the local configuration file.

In addition, the S22 also includes the implementation of automatic switching of sentinels (because the faulty sentinel will be responsible for switching, after the switch, the Redis role and the sentinel only run on the slave node, which is inconsistent, so the sentinel needs to be switched; the sentinel runs on the slave node, because the master node. If the service fails to start after 3 attempts for self-rescue of the faulty node, an alarm email will be sent, and if a split-brain multi-master appears, the Redis process with a long running time will be killed accordingly.

Wherein, the preset rule for the agent to obtain the latest structure information of the database Redis in S3 refers to the agent periodically obtains from the sentinel, and the information includes information such as status/role.

According to another embodiment of the present invention, an electronic device is also provided.

In this embodiment, the electronic device may be a computer or a server. The electronic device includes at least a memory, a processor, a communication bus, and a network interface.

Wherein, the memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. The memory may in some embodiments be an internal storage unit of an electronic device, such as a hard disk of the electronic device. In other embodiments, the memory can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash memory equipped on the electronic device. Card (Flash Card), etc. Further, the memory may also include both an internal storage unit of the electronic device and an external storage device. The memory can not only be used to store application software installed in the electronic device and various types of data, such as the code of the implementation program of the Redis replica set, etc., but also can be used to temporarily store the data that has been output or will be output.

In some embodiments, the processor may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip for executing program codes or processing data stored in the memory.

The communication bus is used to realize the connection communication between these components.

Optionally, the network interface may include a standard wired interface and a wireless interface (such as a WI-FI interface), which is generally used to establish a communication connection between the electronic device and other electronic devices.

Optionally, the electronic device may further include a user interface, and the user interface may include a display (Display), an input unit such as a keyboard (Keyboard), and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like. The display may also be appropriately called a display screen or a display unit, which is used for displaying information processed in the electronic device and for displaying a visual user interface.

The electronic device includes a memory and a processor, the memory stores an implementation program of a Redis replica set that can run on the processor, and the implementation program of the Redis replica set is executed by the processor. The following steps are implemented :

S1 , using a preset method to detect the activity of the agent to achieve high availability of the agent; wherein, the S1 specifically includes the following steps: using the preset software KeepAlived to detect the agent activity to control the VIP offset to achieve the high availability of the agent.

S2. Implement automatic failover, manual master-slave node switching, automatic sentinel switching, faulty node self-rescue and split-brain multi-master detection through preset rules; wherein, the S2 specifically includes the following steps: S21. After the replica set is initialized, Sentinel runs on the slave node, and periodically obtains the status of database Redis and is responsible for automatic failover of database Redis and manual master-slave server switching; S22. When master-slave switching is completed, the periodic triggering of the Check-sentinel script will detect the sentinel running The node does not match the database Redis role, and the sentinel will be correspondingly switched to the current slave node server by the Check-sentinel script.

Specifically, periodically triggering the Check-sentinel script in S22 specifically includes the following steps:

To sum up, with the help of the above technical solutions of the present invention, through the components Proxy (proxy), KeepAlived (highly available and reliable software), Redis-server (node), Redis-sentinel (sentinel) and Check-sentinel scripts The combined use of Redis enables the present invention to use the lowest configuration of Redis to implement a high-availability solution, which not only effectively solves the problem of split-brain and virtual machine downtime that cannot be switched, but also effectively solves the multi-master problem after the virtual machine is started. At the same time, the addition of the proxy can provide a kind of protection for Redis, such as the shielding of performance loss commands and the abnormal exit of Redis caused by the client connection full of Redis.

It should be noted that the above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprising", "comprising" or any other variation thereof herein are intended to encompass a non-exclusive inclusion such that a process, device, article or method comprising a list of elements includes not only those elements, but also includes no explicit Other elements listed, or those inherent to such a process, apparatus, article, or method are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article, or method that includes the element.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disc), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims

A method for implementing a Redis replica set, which includes the following steps:

S1. Use a preset method to detect the activity of the agent, so as to realize the high availability of the agent;

S2. Implement automatic failover, manual master-slave node switching, sentinel automatic switching, fault node self-rescue and split-brain multi-master detection through preset rules;

S3. The agent obtains the latest structure information of the database Redis by using a preset rule, and routes the request correctly.
The method for implementing a Redis replica set according to claim 1, wherein in the S1, a preset method is used to detect the activity of the agent, and the realization of the high availability of the agent specifically includes the following steps: using the preset software KeepAlived to detect the agent by using the preset method. Liveness to control VIP offset and achieve high availability of the proxy.
A method for implementing a Redis replica set according to claim 1, wherein the agent in the S1 is an open source agent Predixy, the open source agent Predixy can block some password reset and parameter modification commands, and the user There is no need to perceive the underlying architecture deployment information, and connecting the agent is like connecting to a single database Redis, which can realize subsequent replica set expansion.
The method for implementing a Redis replica set according to claim 1, wherein in S2, automatic failover, manual master-slave node switching, sentinel automatic switching, fault node self-rescue, and split-brain multi-master detection are implemented through preset rules. Specifically include the following steps:

S21. After the replica set is initialized, the sentinel runs on the slave node, and periodically obtains the status of the database Redis and is responsible for the automatic failover of the database Redis and manual master-slave server switching;

S22. After the master-slave switch is completed, the periodic triggering of the Check-sentinel script will detect that the sentinel running node does not match the database Redis role, and then the sentinel will be correspondingly switched to the current slave node server by the Check-sentinel script.
The method for realizing a Redis replica set according to claim 4, wherein the periodic triggering of the Check-sentinel script in the S22 specifically includes the following steps:

When the server is the main node server and there is a sentinel, the sentinel process is stopped, the agent process configuration file soft connection points to the local, the modified soft connection points to the remote agent configuration file, and the agent is restarted;

When the server is a slave node server and there is no sentinel, start the sentinel process, the agent process configuration file soft connection points to the remote end, modify the soft connection to point to the local agent configuration file, and restart the agent;

When the PING command returns an error, it is regarded as a slave node server, and it is accumulated once every cycle. If it still fails 5 times, the soft connection of the proxy configuration file is modified to point to the local configuration file.
The method for implementing a Redis replica set according to claim 1, wherein the preset rule for the agent to obtain the latest Redis structure information of the database in the S3 means that the agent periodically obtains from the sentinel.
An electronic device, wherein the electronic device includes a memory and a processor, the memory stores an implementation program of a Redis replica set that can run on the processor, and the implementation program of the Redis replica set is stored by the The processor implements the following steps when executing:

S1. Use a preset method to detect the activity of the agent, so as to realize the high availability of the agent;

S2. Implement automatic failover, manual master-slave node switching, sentinel automatic switching, fault node self-rescue and split-brain multi-master detection through preset rules;

S3. The agent obtains the latest structure information of the database Redis by using a preset rule, and routes the request correctly.
The electronic device according to claim 7, wherein in the S1, a preset method is used to detect the activity of the agent, and realizing the high availability of the agent specifically includes the following steps: using the preset software KeepAlived to control the VIP by detecting the activity of the agent offset to achieve high availability of the proxy.
The electronic device according to claim 7, wherein, in said S2, implementing automatic failover, manual master-slave node switching, sentinel automatic switching, fault node self-rescue and split-brain multi-master detection through a preset rule specifically includes the following steps :

S21. After the replica set is initialized, the sentinel runs on the slave node, and periodically obtains the status of the database Redis and is responsible for the automatic failover of the database Redis and manual master-slave server switching;

S22. After the master-slave switch is completed, the periodic triggering of the Check-sentinel script will detect that the sentinel running node does not match the database Redis role, and then the sentinel will be correspondingly switched to the current slave node server by the Check-sentinel script.
A kind of electronic device according to claim 9, wherein, periodically triggering Check-sentinel script in described S22 specifically comprises the following steps:

When the server is the master node server and there is a sentinel, the sentinel process is stopped, the agent process configuration file soft connection points to the local, the modified soft connection points to the remote agent configuration file, and the agent is restarted;

When the server is a slave node server and there is no sentinel, start the sentinel process, the agent process configuration file soft connection points to the remote end, modify the soft connection to point to the local agent configuration file, and restart the agent;

When the PING command returns an error, it is regarded as a slave node server, and it is accumulated once every cycle. If it still fails 5 times, the soft connection of the proxy configuration file is modified to point to the local configuration file.