CN112416847B

CN112416847B - Redis cluster creation method, system and device

Info

Publication number: CN112416847B
Application number: CN202010293491.9A
Authority: CN
Inventors: 赵雪松
Original assignee: Shanghai Bilibili Technology Co Ltd
Current assignee: Shanghai Bilibili Technology Co Ltd
Priority date: 2020-04-15
Filing date: 2020-04-15
Publication date: 2023-05-02
Anticipated expiration: 2040-04-15
Also published as: CN112416847A

Abstract

The application discloses a Redis cluster creation method, which comprises the following steps: calculating master-slave and slot position relations of each node in the Redis cluster in advance; generating a Redis cluster configuration file according to the calculation result of the pre-calculation; releasing the Redis cluster configuration file and starting each node so that each node obtains corresponding role and slot position information according to the Redis cluster configuration file and completes role setting; and acquiring node routing information of each node, and determining that the node routing information of each node is consistent so as to complete the creation of the Redis cluster. The application also discloses a Redis cluster creation system and a Redis cluster creation device. Therefore, each Redis node can simulate downtime and recover to obtain a complete cluster, and the establishment process of the Redis cluster is completed rapidly.

Description

Redis cluster creation method, system and device

Technical Field

The present disclosure relates to the field of Redis cluster technologies, and in particular, to a method, a system, and an apparatus for creating a Redis cluster.

Background

Currently, many companies use remote dictionary service (Remote Dictionary Server, redis) caching components to act as a caching layer. Redis is an open-source log-type Key-Value database written and supported by ANSI C language and capable of being based on memory and being persistent. However, single-node Redis is not sufficient to carry the data volume of most companies, and single-point risk is too high. It is therefore common to employ the clustering functionality provided after dis 3.0 to guarantee its high availability. Redis clusters are used for providing the same service by adding the number of Redis nodes, so that a stable and efficient state is achieved.

The existing Redis cluster creation process is essentially a sequential election process. Firstly, N nodes are prepared, then, one node is selected randomly, and a slot list responsible for the node is declared. After more than half of the master nodes reply agreements, the node promotes itself to be the master node. Repeating the steps until all slots are allocated, and connecting the slave nodes prepared in advance with the own master node in turn.

However, the inventors found that the prior art has at least the following problems: redis cluster creation is currently very slow. Typical clusters of 400 nodes, traditional creation methods take up to even more than 30 minutes. When the network conditions are poor, it performs even worse.

The greatest difficulty in creating the dis cluster is that in fact, in the election process, the whole process is sequentially executed one by one, and cannot be executed concurrently. Moreover, because the Gossip protocol is adopted during the Redis election, the protocol itself needs convergence time, and is greatly interfered by network fluctuation, and the election failure retry often occurs. In addition, to perform election back-off, the election will sleep randomly for a period of time when the election fails, thus creating a cluster more slowly.

It should be noted that the foregoing is not intended to limit the scope of the application.

Disclosure of Invention

The main purpose of the application is to provide a Redis cluster creation method, a system and a device, which aim to solve the problem of how to quickly complete the Redis cluster creation process.

To achieve the above object, an embodiment of the present application provides a method for creating a dis cluster, where the method includes:

calculating master-slave and slot position relations of each node in the Redis cluster in advance;

generating a Redis cluster configuration file according to the calculation result of the pre-calculation;

releasing the Redis cluster configuration file and starting each node so that each node obtains corresponding role and slot position information according to the Redis cluster configuration file and completes role setting; a kind of electronic device with high-pressure air-conditioning system

And acquiring node routing information of each node, and determining that the node routing information of each node is consistent so as to complete the creation of the Redis cluster.

Optionally, after determining that the node routing information of each node is consistent, the method further includes:

checking whether the master-slave relation of each node in the created Redis cluster accords with the calculation result;

when nodes which do not accord with the calculation result exist, correction is carried out in an active election mode, so that the master-slave relation of each node accords with the calculation result.

Optionally, the dis cluster configuration file includes topology, roles, slots, and epoch information of the dis cluster.

Optionally, the node routing information is master-slave relationship and slot position information recorded in each node.

Optionally, the correcting by the active election method includes:

and sending a failover command to the node which does not accord with the calculation result, so that the node actively performs an election process of claiming itself as a master node, and the node is lifted to be the master node, thereby correcting the deviation.

In addition, to achieve the above object, an embodiment of the present application further proposes a Redis cluster creation system, where the system includes:

the calculation module is used for calculating the master-slave and slot position relation of each node in the Redis cluster in advance;

the generation module is used for generating a Redis cluster configuration file according to the calculation result of the pre-calculation;

the issuing module is used for issuing the Redis cluster configuration file and starting each node so that each node can acquire corresponding role and slot position information according to the Redis cluster configuration file and complete role setting; a kind of electronic device with high-pressure air-conditioning system

The determining module is used for acquiring the node routing information of each node and determining that the node routing information of each node is consistent so as to complete the creation of the Redis cluster.

In order to achieve the above object, an embodiment of the present application further provides a Redis cluster creation apparatus, where the apparatus includes: the system comprises a memory, a processor and a Redis cluster creation program which is stored in the memory and can run on the processor, wherein the Redis cluster creation program realizes the Redis cluster creation method when being executed by the processor.

To achieve the above object, another method for creating a dis cluster is further provided in an embodiment of the present application, where the method includes:

receiving a starting command sent by a Redis cluster creating device, and acquiring a Redis cluster configuration file issued by the Redis cluster creating device, wherein the Redis cluster configuration file is generated by the Redis cluster creating device according to a master-slave and slot position relation of each node in the Redis cluster calculated in advance;

reading the role and slot position information corresponding to the current node from the Redis cluster configuration file and completing role setting; a kind of electronic device with high-pressure air-conditioning system

And sending node routing information to the Redis cluster creation device so that the Redis cluster creation device completes the creation of the Redis cluster after determining that the node routing information of each node is consistent.

Optionally, reading the role and slot information corresponding to the current node from the dis cluster configuration file and completing role setting includes:

Reading role information of a current node through myself field identification in the Redis cluster configuration file so as to judge whether the current node is a master node or a slave node;

if the current node is the master node, reading the served slot information in the Redis cluster configuration file through the slots field identification;

if the current node is a slave node, reading corresponding master node information in the Redis cluster configuration file through a slave field identification;

the role information of the current node is announced to other nodes through the Gossip protocol.

To achieve the above object, an embodiment of the present application further provides an electronic device, including: the system comprises a memory, a processor and a Redis cluster creation program which is stored in the memory and can run on the processor, wherein the Redis cluster creation program realizes the Redis cluster creation method when being executed by the processor.

According to the Redis cluster creation method, system and device, all the Redis nodes can be started after the Redis cluster configuration file is generated in advance, each Redis node is enabled to simulate a state of being in a down state and then to be restored, a complete cluster is obtained through parallel restoration according to the master-slave and slot position relation in the Redis cluster configuration file, and therefore the Redis cluster creation is completed, and the effect of accelerating the speed of creating the whole cluster is achieved. In addition, the creation method can basically disregard the number of nodes and the network condition, and the performance is more stable.

Drawings

FIG. 1 is a diagram of an application environment architecture for implementing various embodiments of the present application;

fig. 2 is a flowchart of a method for creating a dis cluster according to the first embodiment of the present application;

fig. 3 is a flowchart of a method for creating a dis cluster according to a second embodiment of the present application;

fig. 4 is a flowchart of a method for creating a dis cluster according to a third embodiment of the present application;

FIG. 5 is a schematic diagram of the refinement procedure of step S402 in FIG. 4;

fig. 6 is a schematic hardware architecture diagram of a dis cluster creating device according to a fourth embodiment of the present application;

fig. 7 is a schematic block diagram of a dis cluster creation system according to a fifth embodiment of the present application;

fig. 8 is a schematic block diagram of a dis cluster creation system according to a sixth embodiment of the present application;

fig. 9 is a schematic hardware architecture of an electronic device according to a seventh embodiment of the present application;

fig. 10 is a schematic block diagram of a dis cluster creation system according to an eighth embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

It should be noted that the descriptions of "first," "second," etc. in the embodiments of the present application are for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.

Referring to fig. 1, fig. 1 is a schematic diagram of an application environment for implementing various embodiments of the present application. The present application is applicable to application environments including, but not limited to, redis cluster creation device 2, node 4, network 6. Moreover, the application platform for creating the Redis cluster is not limited, and can be, for example, a kubernet platform, a virtualization platform and the like.

Wherein the Redis cluster creation means 2 is configured to control the node 4 to complete role setting by generating and issuing a Redis cluster configuration file (nodes. Conf), thereby creating the Redis cluster. The Redis cluster creation device 2 may be a server or other computer devices, where the server may be a rack-mounted server, a blade server, a tower server, or a cabinet server, and may be an independent server, or may be a server cluster formed by multiple servers.

The node 4 is node equipment contained in the Redis cluster to be created, and is configured to obtain and set a role corresponding to the node equipment according to a Redis cluster configuration file issued by the Redis cluster creation device 2, so as to complete creation of the Redis cluster. The node 4 may also be a server or other computer device. One of the Redis clusters includes a plurality of nodes 4, and each node 4 may have a role in the Redis cluster as a master node or a slave node.

The network 6 may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, GSM), wideband code division multiple access (Wideband Code Division Multiple Access, WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc. The Redis cluster creation device 2 and a plurality of the nodes 4 are in communication connection through the network 6 so as to perform data transmission and interaction. Notably, the network 6 described in this application requires interworking that can be either through or network address translation (Network Address Translation, NAT), particularly for use in an intranet.

Example 1

Fig. 2 is a flowchart of a method for creating a dis cluster according to the first embodiment of the present application. It will be appreciated that the flow charts in the method embodiments are not intended to limit the order in which the steps are performed. The following exemplarily describes with the Redis cluster creation apparatus 2 as an execution subject.

The method comprises the following steps:

s200, calculating the master-slave and slot relation of each node 4 in the Redis cluster in advance.

Redis clusters hold key-value pairs in the database in a sliced manner, the entire database in the cluster is divided into 16384 slots (slots), and each key in the database belongs to one of the 16384 slots. Each node 4 in the cluster is responsible for apportioning a portion of the 16384 slots. That is, each slot is responsible for processing for one node 4.

To increase the accessibility of the cluster, the node 4 is generally configured as a master-slave structure, and a master node (master node) downloads a plurality of slave nodes (slave nodes), where the master node is a readable and writable node and the slave node is a read-only node.

In this embodiment, before formally creating the dis cluster, master-slave and slot relationships of each node 4 in the dis cluster are calculated in advance, that is, roles of each node 4 (including which are master nodes and which are slave nodes, and which master node each slave node corresponds to) are allocated, and corresponding slots are assigned to each node 4. The pre-calculation of this embodiment does not limit the calculation scheme or strategy, and even a fully random scheme (i.e. randomly allocating master-slave nodes and corresponding slots) can be directly adopted. All that is required is to calculate in advance the master-slave relationship between the individual nodes 4 and the physical machine in which the node 4 is located.

S202, generating a Redis cluster configuration file according to the calculation result.

After the master-slave and slot position relation of each node 4 is calculated in advance, a Redis cluster configuration file is generated according to the calculation result. The Redis cluster configuration file generally comprises topology, roles, slots, epoch (epoch) information and the like of the Redis cluster, and is enabled when the Redis is operated in the Redis cluster mode.

For example, the main fields contained in the Redis cluster configuration file and their meanings:

runid: the run ID of each Redis node, which is unique throughout the cluster, is used to identify and distinguish the Redis nodes, 40 bytes long. Examples: 8a3b8b7bc1e0858b46b9554b1aa505b441524c7f.

address: the location of the Redis node, which consists of: the IP of the node is the service port of the node @ the Gossip port of the node. Examples: the node in the field of '10.23.59.143:3229@13229' is located at an IP of 10.23.59.143, the service port of the node is 3229, and the Gossip port of the node is 13229.

state: the state of the Redis node, wherein slave represents a slave node, master represents a master node, fail represents a node downtime, myself represents a current node. Examples: "master, myself" means the master node and is the present node; "slave, fail" means a slave node and has been down.

slave of: only on the slave node, the omission (-) is given on the master node, which indicates what the runid of the master node corresponding to the node is. Examples: 6f7923d5ffe4269c0f6ca0c9b89d2f8bae9117dd.

slots: enabled only on the master node, declaring the slot information it serves. Only the node allocated with slot information provides services to the outside, and the number of services is positively correlated with the number of allocated slots (16384 slots in total). Examples: "1-20" means slot number 1-20; "50" means a single slot number 50.

S204, the Redis cluster configuration file is issued, each node 4 is started, and accordingly the node 4 obtains corresponding role and slot position information according to the Redis cluster configuration file, and role setting is completed.

If the conventional Redis cluster creation flow is adopted, the Redis cluster configuration file is generated after the clusters are created through a complex election process. And once a certain node is down in the running process, recovering according to the Redis cluster configuration file.

In this embodiment, in order to bypass the election process in the conventional manner, the speed of creating the dis cluster is increased, and the master-slave relationship between all nodes 4 is pre-declared in the form of the dis cluster configuration file directly before each node 4 is started. Therefore, when each node 4 is started, the state of recovering after the shutdown can be simulated.

For a Redis cluster, after a certain node 4 is down, in order to resume startup, an attempt is made to read own role and slot information from the Redis cluster configuration file, and to rejoin the whole cluster with the read role information. And when a plurality of nodes are down and recovery is attempted, the whole recovery process can be performed in parallel. That is, by generating the dis cluster configuration file in advance, all nodes 4 consider themselves to be in a state that is desired to be restored after downtime, and start each node 4 to restore the entire cluster in parallel. In this way, the process of creating the Redis cluster changes from a process of sequentially electing to a process of simulating all nodes in the cluster to recover simultaneously, and the self-healing function of the cluster is relied on to repair, so that a complete cluster is finally obtained, and the Redis cluster is created.

After generating the Redis cluster configuration file according to the calculation result, the Redis cluster creating device 2 issues the Redis cluster configuration file to all nodes 4 and simultaneously starts all nodes 4. Each node 4 acquires the Redis cluster configuration file, and reads own role information through myself field identification in the Redis cluster configuration file so as to judge whether the node is a master node or a slave node. If the node is the master node, the slot information served by the node is read through the slots field identification in the Redis cluster configuration file. If the node is a slave node, the corresponding master node information is further read through a slave field identification in the Redis cluster configuration file. Each node 4 then announces its own role information to the other nodes 4 via the Gossip protocol. The Gossip protocol is a protocol for information exchange between nodes or processes based on epidemic propagation modes, is widely used in distributed systems, propagates information into the whole network in a random mode, and makes all node data in the network consistent within a certain time. The Gossip protocol is a distributed protocol of a decentralization idea, and solves two problems of state propagation in a cluster and state consistency assurance.

S206, acquiring node routing information of each node 4, and determining that the node routing information of all the nodes 4 is consistent so as to complete creation of the Redis cluster.

The node routing information is the master-slave and slot relation recorded in each node 4. Because the Redis cluster adopts the Gossip protocol, which is a non-strongly consistent protocol, multiple pieces of different role information, namely inconsistent node routing information, may exist on each node 4 of the cluster at the same time. For example, a certain node a is the master node, but a node B farther away in the Gossip protocol may not have synchronized to this information, and thus may have erroneous node routing information.

The Redis cluster creating device 2 acquires node routing information of each node 4 in real time, and judges whether the node routing information of all the nodes 4 is consistent. If not, continuing waiting. When it is determined that the node routing information of all the nodes 4 is consistent, it means that the process of creating the dis cluster is completed. The convergence process is completely automatic, wherein node routing information of all nodes 4 is waited to be consistent, namely, master-slave and slot position relations of all nodes 4 are consistent.

According to the Redis cluster creation method, all the Redis nodes can be restarted by generating the Redis cluster configuration file in advance, so that each Redis node can simulate a state of recovering after being in downtime, and a complete cluster is obtained by parallel recovery according to the master-slave and slot position relation in the Redis cluster configuration file, so that the Redis cluster is created, and the speed of creating the whole cluster is increased. In addition, the creation method can basically disregard the number of nodes and the network condition, and the performance is more stable.

It is proved by actual measurement that the creation method provided by the embodiment can generally create a typical Redis cluster of 400 nodes within 5 seconds, and the time spent by using the traditional method is different from 1 minute to 30 minutes, so that the creation method has obvious effect in acceleration.

Example two

As shown in fig. 3, a flowchart of a method for creating a dis cluster according to a second embodiment of the present application is provided. In the second embodiment, the Redis cluster creation method further includes steps S308-S310 on the basis of the first embodiment. It will be appreciated that the flow charts in the method embodiments are not intended to limit the order in which the steps are performed.

The method comprises the following steps:

s300, calculating the master-slave and slot relation of each node 4 in the Redis cluster in advance.

Redis clusters hold key-value pairs in the database in a sliced manner, the entire database in the cluster is divided into 16384 slots, and each key in the database belongs to one of the 16384 slots. Each node 4 in the cluster is responsible for apportioning a portion of the 16384 slots. That is, each slot is responsible for processing for one node 4.

To increase the accessibility of the cluster, the nodes 4 are typically configured as a master-slave structure, with one master node downloading multiple slave nodes.

S302, generating a Redis cluster configuration file according to the calculation result.

After the master-slave and slot position relation of each node 4 is calculated in advance, a Redis cluster configuration file is generated according to the calculation result. The Redis cluster configuration file generally comprises topology, roles, slots, epoch information and the like of the Redis cluster.

S304, the Redis cluster configuration file is issued, and each node 4 is started, so that the node 4 obtains corresponding role and slot position information according to the Redis cluster configuration file, and role setting is completed.

After generating the Redis cluster configuration file according to the calculation result, the Redis cluster creating device 2 issues the Redis cluster configuration file to all nodes 4 and simultaneously starts all nodes 4. Each node 4 acquires the Redis cluster configuration file, and reads own role information through myself field identification in the Redis cluster configuration file so as to judge whether the node is a master node or a slave node. If the node is the master node, the slot information served by the node is read through the slots field identification in the Redis cluster configuration file. If the node is a slave node, the corresponding master node information is further read through a slave field identification in the Redis cluster configuration file. Each node 4 then announces its own role information to the other nodes 4 via the Gossip protocol.

S306, acquiring node routing information of each node 4, and determining that the node routing information of all the nodes 4 is consistent so as to complete the initial creation of the Redis cluster.

The Redis cluster creating device 2 acquires node routing information of each node 4 in real time, and judges whether the node routing information of all the nodes 4 is consistent. If not, continuing waiting. When it is determined that the node routing information of all the nodes 4 is consistent, it means that the first creation process of the dis cluster is completed. The convergence process is completely automatic, wherein node routing information of all nodes 4 is waited to be consistent, namely, master-slave and slot position relations of all nodes 4 are consistent.

S308, checking whether the master-slave relation of each node 4 in the created Redis cluster accords with the calculation result.

When the primary creation is completed, because each node 4 in the cluster is started simultaneously, but the starting speed is high or low, if the slave node finishes starting before the corresponding master node, the slave node may be mistakenly shut down for the master node connected with the slave node, and the slave node automatically tries to enter the election process of claiming the slave node as the master node, so that the distribution of the master node and the calculation result actually obtained finally have deviation. Therefore, it is necessary to check whether the master-slave relationship meets expectations after the cluster is first created.

And S310, when nodes which do not accord with the calculation result exist, correcting in an active election mode, so that the master-slave relationship of each node 4 accords with the calculation result.

For the node 4 that does not conform to the calculation result, the Redis CLUSTER creating apparatus 2 sends a cluse failure command to the node 4, so that the node 4 (those nodes that should be master but are currently slave in the calculation result) actively performs an election process of claiming itself as a master node to promote the node 4 as a master node, thereby correcting the deviation. The CLUSTER FAILOVER command can only be executed in the slave nodes of the CLUSTER for a manual FAILOVER of the slave nodes in order to switch roles between the current master node (the master node to which the slave node executing the command corresponds) and one of the slave nodes (the slave node executing the command) in a secure (data-free) manner.

It should be noted that having node 4 perform an active election does not necessarily succeed in promoting it to the master node, which involves the problem of convergence of the Gossip protocol, but can be solved by retry.

According to the Redis cluster creation method, the master-slave relation of the Redis cluster created for the first time can be further checked according to the calculation result, if the situation does not accord with the calculation result exists, the node is corrected in an active election mode, so that the created Redis cluster accords with the expectation, the creation effect of the Redis cluster is further improved, and the user experience is improved.

Of course, the above steps S308-S310 are just checking measures taken to adjust the distribution position of the master-slave nodes as expected, and do not affect the actual creation of the cluster. When the node routing information of all nodes 4 is consistent, the Redis cluster has been created and can be serviced normally, this checking and correction does not affect the core functionality of the cluster. And, because the Redis has larger forgiving capability for reconnecting the reconstruction cluster (which is shown by larger timeout time setting), the normal start of all nodes 4 within 10 seconds can be ensured.

Example III

Fig. 4 is a flowchart of a method for creating a dis cluster according to a third embodiment of the present application. It will be appreciated that the flow charts in the method embodiments are not intended to limit the order in which the steps are performed. An exemplary description will be made below with the node 4 as an execution body.

The method comprises the following steps:

s400, receiving a starting command sent by the Redis cluster creating device 2, and acquiring a Redis cluster configuration file issued by the Redis cluster creating device 2.

In this embodiment, in order to bypass the election process in the conventional creation manner, to accelerate the speed of creating the Redis cluster, the Redis cluster creation apparatus 2 first calculates the master-slave and slot relationship of each node 4 in the Redis cluster in advance, that is, allocates the role of each node 4 (including which is a master node, which is a slave node, and which master node each slave node corresponds to) and assigns a corresponding slot to each node 4 before formally creating the Redis cluster. The master-slave relationship between all nodes 4 is then pre-declared by the form of the dis cluster configuration file immediately before each node 4 is started. Therefore, when each node 4 is started, the state of recovering after the shutdown can be simulated.

After generating the dis cluster configuration file according to the calculation result, the dis cluster creating device 2 issues the dis cluster configuration file to each node 4, and simultaneously starts all the nodes 4 (issues a start command). Each node 4 receives the starting instruction sent by the Redis cluster creating device 2, and obtains the Redis cluster configuration file when starting.

S402, reading the corresponding role and slot information from the Redis cluster configuration file and completing role setting.

Each node 4 reads its own role information through myself field identification in the dis cluster configuration file to determine whether it is a master node or a slave node. If the node is the master node, the slot information served by the node is read through the slots field identification in the Redis cluster configuration file. If the node is a slave node, the corresponding master node information is further read through a slave field identification in the Redis cluster configuration file. Then, each node 4 announces its own role information to other nodes 4 through the Gossip protocol, and completes the role setting.

S404, node routing information is sent to the Redis cluster creation device 2.

After completing the role setting, each node 4 sends its own node routing information to the Redis cluster creating device 2, so that the Redis cluster creating device 2 determines whether the node routing information of all the nodes 4 is consistent. If not, continuing waiting. When it is determined that the node routing information of all the nodes 4 is consistent, it means that the first creation process of the dis cluster is completed.

Further referring to fig. 5, a detailed flowchart of the above step S402 is shown. In this embodiment, the step S402 specifically includes:

s4020, judging whether the node is a master node according to the Redis cluster configuration file. If yes, step S4022 is executed. If not the master node (i.e., the slave node), step S4024 is performed.

And each node 4 reads own role information through myself field identification in the Redis cluster configuration file so as to judge whether the node is a master node or a slave node.

S4022, setting the node as a master node, and declaring the node as the master node to other nodes.

If the current node 4 is the master node, the role information of the current node is declared to the other nodes 4 as the master node through the Gossip protocol. In addition, the master node also needs to read the slot information served by the master node through the slots field identification in the Redis cluster configuration file.

S4024, obtaining the corresponding main node information, and judging whether the main node can be normally connected. If the connection is possible, step S4028 is performed. If the normal connection is not possible, step S4026 is executed.

If the current node 4 is a slave node, the corresponding master node information is further read from the Redis cluster configuration file through the slave field identification. Then, the connection of the Gossip port of the master node is tried, and if the connection is successful, the corresponding master node can be judged to be connected normally.

S4026, the election of the host node is actively initiated, and after the election is successful, the step S4022 is executed.

Because each node 4 is started simultaneously, but the speed of starting is high or low, after the current node 4 (slave node) starts, the corresponding master node may not be normally connected (for example, the slave node completes starting before the corresponding master node), and then the current node 4 automatically enters the election process of claiming itself to be the master node. If the election is successful (after more than half of the master nodes reply with consent), the current node 4 promotes itself to the master node.

S4028, judging whether the master node is declared to be the master node. If yes, go to step S4030. If not, returning to the step S4024.

Similarly, because each node 4 is started simultaneously, but the speed of the start is slow, after the current node 4 (slave node) starts, the corresponding master node may not have declared itself as a master node (e.g., slave node completed the start before its corresponding master node). At this time, it may return to step S4024 to continue waiting.

S4030, setting itself as the slave node, and declaring itself as the slave node to the other nodes.

If the master node corresponding to the current node 4 (slave node) has declared itself as the master node, that is, the master-slave relationship between the two nodes is clearly allocated, the current node 4 sets itself as the slave node, and declares itself as the role information of the slave node to other nodes 4 through the Gossip protocol.

According to the Redis cluster creation method, each Redis node can obtain the Redis cluster creation device to generate the Redis cluster configuration file in advance when being started, simulate the state of recovering after downtime, and recover in parallel according to the master-slave and slot position relations in the Redis cluster configuration file, so that a complete cluster is obtained, the creation of the Redis cluster is completed, and the effect of accelerating the speed of creating the whole cluster is achieved. In addition, the creation method can basically disregard the number of nodes and the network condition, and the performance is more stable.

It should be noted that, when it is checked that the master-slave relationship of the node 4 does not conform to the calculation result based on steps S308 to S310 of the third embodiment, the node 4 needs to correct the master-slave relationship by actively electing. The process of the active election is similar to the conventional method, and will not be described again.

Example IV

As shown in fig. 6, a hardware architecture diagram of a dis cluster creating apparatus 2 is provided in a fourth embodiment of the present application. In this embodiment, the Redis cluster creation device 2 may include, but is not limited to, a memory 21, a processor 22, and a network interface 23 that may be communicatively connected to each other through a system bus. It is noted that fig. 6 only shows the dis cluster creation device 2 with components 21-23, but it is understood that not all shown components are required to be implemented, and that more or fewer components may alternatively be implemented.

The memory 21 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the Redis cluster creation device 2, for example, a hard disk or a memory of the Redis cluster creation device 2. In other embodiments, the memory 21 may also be an external storage device of the dis cluster creation device 2, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the dis cluster creation device 2. Of course, the memory 21 may also comprise both the internal storage unit of the dis cluster creation device 2 and its external storage device. In this embodiment, the memory 21 is generally configured to store an operating system and various application software installed in the Redis cluster creation device 2, for example, program codes of the Redis cluster creation system 20, and the like. Further, the memory 21 may be used to temporarily store various types of data that have been output or are to be output.

The processor 22 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 22 is typically used to control the overall operation of the Redis cluster creation device 2. In this embodiment, the processor 22 is configured to execute the program code or process data stored in the memory 21, for example, execute the Redis cluster creation system 20.

The network interface 23 may comprise a wireless network interface or a wired network interface, which network interface 23 is typically used for establishing a communication connection between the Redis cluster creating device 2 and other electronic equipment.

Example five

As shown in fig. 7, a schematic block diagram of a Redis cluster creation system 20 according to a fifth embodiment of the present application is provided. The Redis cluster creation system 20 may be partitioned into one or more program modules that are stored in a storage medium and executed by one or more processors to accomplish the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of implementing specific functions, and the following description specifically describes the functions of each program module in the embodiments.

In this embodiment, the Redis cluster creation system 20 includes:

the calculation module 200 is configured to calculate in advance a master-slave and slot relationship of each node 4 in the Redis cluster.

In this embodiment, before formally creating the dis cluster, the calculation module 200 calculates the master-slave and slot relationship of each node 4 in the dis cluster in advance, that is, assigns the role of each node 4 (including which are master nodes and which are slave nodes, and which master node each slave node corresponds to), and assigns a corresponding slot to each node 4. The pre-calculation of this embodiment does not limit the calculation scheme or strategy, and even a fully random scheme (i.e. randomly allocating master-slave nodes and corresponding slots) can be directly adopted. All that is required is to calculate in advance the master-slave relationship between the individual nodes 4 and the physical machine in which the node 4 is located.

A generating module 202, configured to generate a Redis cluster configuration file according to the calculation result.

After the master-slave and slot relation of each node 4 is calculated in advance, the generating module 202 generates a Redis cluster configuration file according to the calculation result. The Redis cluster configuration file generally comprises topology, roles, slots, epoch information and the like of the Redis cluster.

And the publishing module 204 is used for publishing the Redis cluster configuration file and starting each node 4 so that the node 4 obtains the corresponding role and slot position information according to the Redis cluster configuration file and completes role setting.

After the generating module 202 generates the dis cluster configuration file according to the calculation result, the publishing module 204 publishes the dis cluster configuration file to all the nodes 4 and simultaneously starts all the nodes 4. Each node 4 acquires the Redis cluster configuration file, and reads own role information through myself field identification in the Redis cluster configuration file so as to judge whether the node is a master node or a slave node. If the node is the master node, the slot information served by the node is read through the slots field identification in the Redis cluster configuration file. If the node is a slave node, the corresponding master node information is further read through a slave field identification in the Redis cluster configuration file. Each node 4 then announces its own role information to the other nodes 4 via the Gossip protocol.

A determining module 206, configured to obtain node routing information of each node 4, and determine that node routing information of all nodes 4 is consistent, so as to complete creation of the dis cluster.

The determining module 206 obtains the node routing information of each node 4 in real time, and determines whether the node routing information of all the nodes 4 is consistent. If not, continuing waiting. When it is determined that the node routing information of all the nodes 4 is consistent, it means that the process of creating the dis cluster is completed. The convergence process is completely automatic, wherein node routing information of all nodes 4 is waited to be consistent, namely, master-slave and slot position relations of all nodes 4 are consistent.

Example six

As shown in fig. 8, a schematic block diagram of a dis cluster creation system 20 is provided in a sixth embodiment of the present application. In this embodiment, the Redis cluster creation system 20 includes a checking module 208 and a correction module 210 in addition to the calculation module 200, the generation module 202, the publication module 204, and the determination module 206 in the fifth embodiment.

The checking module 208 is configured to check whether the master-slave relationship of each node 4 in the created Redis cluster conforms to the calculation result.

When the first creation is completed according to the sixth embodiment, because each node 4 in the cluster is started simultaneously, but the starting speed is high or low, if the slave node finishes starting before its corresponding master node, the slave node may be mistakenly shut down for the master node connected to the slave node, and the slave node automatically tries to enter the election process of claiming itself as the master node, so that the distribution of the master node and the slave node actually obtained finally deviate from the calculation result. Thus, the checking module 208 needs to check whether the master-slave relationship is in line with expectations after the initial creation of the cluster is completed.

The correction module 210 is configured to correct, when there are nodes that do not conform to the calculation result, by means of active election, the master-slave relationship of each node 4 conforms to the calculation result.

For the node 4 that does not conform to the calculation result (those nodes that should be master but are currently slaves in the calculation result), the correction module 210 sends a CLUSTER FAILOVER command to the node 4 to cause the node 4 to actively perform an election process that declares itself as master to promote the node 4 as master, thereby correcting the bias.

Example seven

As shown in fig. 9, a hardware architecture diagram of an electronic device 44 according to a seventh embodiment of the present application is provided. The electronic device 44 may be a Redis node (i.e., the node 4) that makes up the Redis cluster.

In this embodiment, the electronic device 44 may include, but is not limited to, a memory 41, a processor 42, and a network interface 43, which may be communicatively connected to each other through a system bus. It should be noted that fig. 9 only shows an electronic device 44 having components 41-43, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may alternatively be implemented.

The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the electronic device 44, such as a hard disk or a memory of the electronic device 44. In other embodiments, the memory 41 may also be an external storage device of the electronic apparatus 44, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic apparatus 44. Of course, the memory 41 may also include both an internal memory unit and an external memory device of the electronic device 44. In this embodiment, the memory 41 is typically used to store an operating system and various application software installed on the electronic device 44, such as program codes of the Redis cluster creation system 40. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is generally operative to control the overall operation of the electronic device 44. In this embodiment, the processor 42 is configured to execute the program code or process data stored in the memory 41, for example, execute the Redis cluster creation system 40.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used to establish a communication connection between the electronic apparatus 44 and other electronic devices.

Example eight

As shown in fig. 10, a schematic block diagram of a Redis cluster creation system 40 according to an eighth embodiment of the present application is provided. The Redis cluster creation system 40 may be partitioned into one or more program modules that are stored in a storage medium and executed by one or more processors to accomplish the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of implementing specific functions, and the following description specifically describes the functions of each program module in the embodiments.

In this embodiment, the Redis cluster creation system 40 includes:

an obtaining module 400, configured to receive a start command sent by the Redis cluster creating device 2, and obtain a Redis cluster configuration file issued by the Redis cluster creating device 2.

After generating the dis cluster configuration file according to the calculation result, the dis cluster creating device 2 issues the dis cluster configuration file to each node 4, and simultaneously starts all the nodes 4 (issues a start command). The acquiring module 400 of each node 4 (electronic device 44) receives the start-up instruction sent by the Redis cluster creating device 2, and acquires the Redis cluster configuration file when starting up.

And a reading module 402, configured to read the corresponding role and slot information from the dis cluster configuration file and complete role setting.

The reading module 402 of each node 4 reads its role information through myself field identification in the Redis cluster configuration file to determine whether it is a master node or a slave node. If the node is the master node, the slot information served by the node is read through the slots field identification in the Redis cluster configuration file. If the node is a slave node, the corresponding master node information is further read through a slave field identification in the Redis cluster configuration file. Then, the role information of the other nodes 4 is announced by the Gossip protocol, and the role setting is completed.

A sending module 404, configured to send node routing information to the Redis cluster creating apparatus 2.

After each node 4 completes the role setting, the sending module 404 sends its own node routing information to the Redis cluster creating device 2, so that the Redis cluster creating device 2 determines whether the node routing information of all the nodes 4 is consistent. If not, continuing waiting. When it is determined that the node routing information of all the nodes 4 is consistent, it means that the first creation process of the dis cluster is completed.

Example nine

The present application also provides another embodiment, namely, a computer-readable storage medium storing a Redis cluster creation program executable by at least one processor to cause the at least one processor to perform the steps of the Redis cluster creation method as described above.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

It will be apparent to those skilled in the art that the modules or steps of the embodiments of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

The foregoing is only the preferred embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application, and all equivalent structures or equivalent processes using the descriptions of the embodiments of the present application and the contents of the drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the embodiments of the present application.

Claims

1. A method for creating a dis cluster, the method comprising:

before formally creating the Redis cluster, calculating the master-slave and slot relation of each node in the Redis cluster in advance;

Acquiring node routing information of each node, and determining that the node routing information of each node is consistent so as to complete creation of the Redis cluster;

the method further comprises the following steps after determining that the node routing information of each node is consistent:

when nodes which do not accord with the calculation result exist, and the nodes which do not accord with the calculation result are the nodes which are the master nodes and the current slave nodes, correcting the nodes in an active election mode to ensure that the master-slave relationship of each node accords with the calculation result.

2. The method for creating the Redis cluster according to claim 1, wherein the Redis cluster configuration file comprises topology, roles, slots and epoch information of the Redis cluster.

3. The method for creating a dis cluster according to claim 1, wherein the node routing information is master-slave relationship and slot information recorded in each node.

4. The method for creating a dis cluster according to claim 1, wherein the correcting by means of active election includes:

and sending a failover command to a node which does not accord with the calculation result and is a slave node at present, so that the node which receives the failover command actively performs an election process of claiming itself as the master node so as to promote the node as the master node, thereby correcting the deviation.

5. A dis cluster creation system, the system comprising:

the calculation module is used for calculating the master-slave and slot relation of each node in the Redis cluster in advance before formally creating the Redis cluster;

The determining module is used for acquiring the node routing information of each node and determining that the node routing information of each node is consistent so as to complete the creation of the Redis cluster;

the checking module is used for checking whether the master-slave relation of each node in the created Redis cluster accords with the calculation result;

and the correction module is used for correcting the nodes which do not accord with the calculation result through an active election mode when the nodes which do not accord with the calculation result are the nodes which are the master nodes and the current slave nodes, so that the master-slave relationship of each node accords with the calculation result.

6. A dis cluster creation apparatus, the apparatus comprising: memory, a processor and a Redis cluster creation program stored on the memory and executable on the processor, which when executed by the processor implements the Redis cluster creation method of any of claims 1-4.

7. A method for creating a dis cluster, the method comprising:

receiving a starting command sent by a Redis cluster creating device, and acquiring a Redis cluster configuration file issued by the Redis cluster creating device, wherein the Redis cluster configuration file is generated according to calculation results corresponding to master-slave and slot position relations of each node in the Redis cluster calculated in advance before formally creating the Redis cluster by the Redis cluster creating device;

Node routing information is sent to the Redis cluster creation device, so that the Redis cluster creation device completes creation of the Redis cluster after determining that the node routing information of each node is consistent;

the Redis cluster creating device further comprises, after determining that the node route information of each node is consistent:

8. The method for creating the dis cluster according to claim 7, wherein the reading the role and slot information corresponding to the current node from the dis cluster configuration file and completing the role setting includes:

9. An electronic device, the electronic device comprising: memory, a processor and a Redis cluster creation program stored on the memory and executable on the processor, which when executed by the processor implements the Redis cluster creation method of any of claims 7-8.