CN117493458A - Method for automatically constructing chain type replication of redis multi-slave nodes - Google Patents
Method for automatically constructing chain type replication of redis multi-slave nodes Download PDFInfo
- Publication number
- CN117493458A CN117493458A CN202311483567.4A CN202311483567A CN117493458A CN 117493458 A CN117493458 A CN 117493458A CN 202311483567 A CN202311483567 A CN 202311483567A CN 117493458 A CN117493458 A CN 117493458A
- Authority
- CN
- China
- Prior art keywords
- node
- redis
- slave
- nodes
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000010076 replication Effects 0.000 title abstract description 17
- 238000007689 inspection Methods 0.000 claims abstract description 9
- 238000013515 script Methods 0.000 claims description 11
- 238000011144 upstream manufacturing Methods 0.000 claims description 9
- 230000000977 initiatory effect Effects 0.000 claims description 4
- 230000005055 memory storage Effects 0.000 claims description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of cloud computing, in particular to a method for automatically constructing a redis multi-slave node chained copy, which comprises the following steps: checking the running environment; node executable program preparation; starting a service; service chaining; post-chain inspection; the beneficial effects are as follows: the method for automatically constructing the chain replication of the redis multi-slave node improves the performance of the redis service by utilizing the read-only characteristic of the slave node, and the multi-slave node can better embody stronger performance on the basis of the chain replication. The method can automatically deploy and build the chain type replication of the redis multi-slave node, and simply and efficiently realize the rapid and automatic deployment and build of the chain type replication of the redis multi-slave node.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a method for automatically constructing a redis multi-slave node chained copy.
Background
Under the background of cloud computing and big data, more and more users are added into a cloud computing platform, redis is used as a memory database, and the method has the advantages of extremely high performance, abundant data types, data atomicity execution, support of a publish-subscribe mode and the like, and more users start to know and use redis as a cache database.
In the prior art, although single service system deployment is simple and easy to use, single service deployment redis only has one node to provide reading and writing capability to the outside, which prevents the architecture of a redis single-thread mode, can not further promote the redis performance, multiple slave nodes can rely on the read-only characteristic of the slave nodes, the slave nodes can provide reading capability to the outside, and the multiple slave nodes can increase the reading capability of the redis by times. At present, the redis multi-slave node sees that data is copied in two copy modes, namely a star mode and a chain mode.
However, star replication keeps all the replicas directly synchronous with the master, each replicas are independent of each other, any node abnormality does not affect other nodes, and meanwhile, because the replication chain is relatively short, the replication delay on the replicas is relatively small. Redis is a single-process single-thread model, data replication between a master thread and a slave thread is also processed in the master thread, the more the number of replicas is, the more serious the CPU consumption of the master by data synchronization is, and the writing performance of the cluster is reduced along with the increase of the replicas. In addition, star architecture allows the master's egress bandwidth to multiply as replica increases. The higher CPU and network load on the Master can offset the advantage of lower star replication delay, so that the star replication architecture can bring about a serious expansion problem, and the performance of the whole cluster can be limited by the Master; the method realizes that the rediss multiple slave nodes automatically find the copying upstream and downstream after starting, automatically forms a chain, and solves the problem of manual chain forming.
Disclosure of Invention
The invention aims to provide a method for automatically constructing a redis multi-slave node chained copy, which aims to solve the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: a method for automatically building a redis multi-slave node chained copy, the method comprising the steps of:
checking the running environment;
node executable program preparation;
starting a service;
service chaining;
post-chain inspection.
Preferably, the specific operation steps of the operation environment inspection include:
selecting one deployment node machine, and configuring ssh information of each node machine;
a checking step in the running method, wherein a scp command is used for copying a checking script to each node machine;
and invoking the checking script node by using the ssh command to check the version of the operating system, the dependence of the runtime environment and the information of the memory storage disk in the cpu.
Preferably, the specific operation steps of the node executable program preparation include:
compressing the package to each node by the scp copy executable program;
ssh decompressing the compressed packets;
and installing the runtime environment at each node in turn.
Preferably, the specific operation steps of service initiation include:
sequentially connecting node machines through ssh, and executing a redis starting command;
detecting a result of starting redis by each node;
after the redis is successfully started, recording the starting state of each node, including connecting ip, recording the password into a cache, and distributing the password into the cache of each node;
and generating serial numbers according to the starting sequence, and updating the caches of the records of all nodes.
Preferably, the specific operation steps of service chaining include:
the method comprises the steps of obtaining the generation numbers of cache records of each node, and sequentially connecting the generation numbers to each node;
each node executes a slave port, ip is the ip of the node upstream of the current node of the cache record, and the port is the service port of redis.
Preferably, the specific operation steps of the post-chain inspection include:
checking the script to each node machine using the scp copy chain;
according to the actual service, designating a timing task execution interval;
registering a system timing task, wherein the working content of the timing task is to check the connectivity among the nodes, then checking the state of master-slave configuration and the data synchronization state through executing an initiation command by the nodes, analyzing whether the nodes have problems according to the acquired results, if so, updating the cache data of the nodes in time, removing the problem nodes from the chain, and directing a command to an upstream node of the problem nodes by a downstream node of the problem nodes.
Compared with the prior art, the invention has the beneficial effects that:
the method for automatically constructing the chain replication of the redis multi-slave node improves the performance of the redis service by utilizing the read-only characteristic of the slave node, and the multi-slave node can better embody stronger performance on the basis of the chain replication. The method can automatically deploy and build the chain type replication of the redis multi-slave node, and simply and efficiently realize the rapid and automatic deployment and build of the chain type replication of the redis multi-slave node.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions, and advantages of the present invention more apparent, the embodiments of the present invention will be further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are some, but not all, embodiments of the present invention, are intended to be illustrative only and not limiting of the embodiments of the present invention, and that all other embodiments obtained by persons of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
Example 1
Referring to fig. 1, the present invention provides a technical solution: a method for automatically building a redis multi-slave node chained copy, the method comprising the steps of:
run-time environment inspection
(1) Selecting a deployment node machine, and configuring ssh information of each node machine, such as login ip user name passwords or login keys;
(2) A checking step in the running method, wherein a scp command is used for copying a checking script to each node machine;
(3) Invoking a checking script node by using the ssh command to check information such as an operating system version, a runtime environment dependence, a cpu memory storage disk and the like;
(two) node executable program preparation
(1) Compressing the package to each node by the scp copy executable program;
(2) ssh decompressing the compressed packets;
(3) Installing the runtime environment on each node in turn;
(III) service initiation
(1) Sequentially connecting node machines through ssh, and executing a redis starting command;
(2) Detecting a result of starting redis by each node;
(3) After the redis is successfully started, recording the starting state of each node, including connecting ip, recording the password into a cache, and distributing the password into the cache of each node;
(4) Generating serial numbers according to the starting sequence, and updating the cache of each node record;
(IV) service chaining
(1) The method comprises the steps of obtaining the generation numbers of cache records of each node, and sequentially connecting the generation numbers to each node;
(2) Each node executes a slave of ip port, ip is the ip of the node upstream of the current node of the cache record, and the port is the service port of redis
(fifth) post-chain inspection
(1) Checking scripts to node machines using scp replication chains
(2) Specifying timed task execution intervals based on actual traffic
(3) Registering a system timing task, wherein the working content of the timing task is to check the connectivity among the nodes, then checking the state of master-slave configuration and the data synchronization state through each node executing an info reply command, analyzing whether the node has a problem according to the acquired result, if so, updating the cache data of each node in time, removing the problem node from the chain, and directing a downstream node of the problem node to an upstream node of the problem node by a command of directly slave.
Example two
On the basis of the first embodiment, the method comprises seven steps, namely: machine running environment checking, redis executable preparation, starting service, obtaining service states, encoding the started redis service, chaining the redis service, and post-chaining checking, each step must meet the requirements and must return success.
The method provides a chain state caching function, caches upstream and downstream information of each node in the whole multi-slave node chain copy, and ensures that the chain state caching function can be checked after the chain state is formed. The link forming information is synchronized to a machine cache where each slave node on the link is located, so that all the information of the same node is synchronized every time the adjustment of the nodes on the link is performed, and the problem that the whole link cannot be effectively formed due to the fact that a single node records the link information is avoided. The cache information can also be provided for operators to check and rebuild the chain on-chain nodes when a certain node on the chain is down to exit service.
The method provides access modes of a plurality of machines needing to be deployed with the slave nodes before operation, and the method performs state check and step command execution among the machines through ssh, so that one machine is required to be used as a method execution machine, and the machine can be any one of the slave node machines or the machine where the redis master node is located. Because the ssh is used for accessing each node, each node needs to ensure that the ssh is smooth, and the ssh access parameters of each node are provided correctly.
The method is suitable for full-version redis chained copy construction, automatically identifies the redis version, and performs differential operation aiming at the functional differences among different versions.
In the environment detection process, virtual machines of all slave nodes are actively connected through ssh, then an environment detection script is executed, the script respectively checks information such as an operating system version, a running environment dependence, a CPU memory storage disk and the like of the environment, and compares specification information of a redismaster node, so that the performance of a machine where the slave node is located cannot be lower than that of the master node, otherwise, the master-slave node is synchronized slowly due to the performance problem, and further the problem of full synchronization is caused. Checking the machine connectivity between the slave nodes and the master node ensures that the normal master-slave synchronization and external service can be provided.
In the method, in the preparation process of a redis executable program, a redis running environment is copied and built on a machine where each slave node is located through ssh automatically, redis running dependence and a redis running program binary file are copied, whether retry or manual maintenance is determined through a copying result returned by each node, and if continuous copying fails, the problem that a machine network is not enabled is likely to be caused.
The method comprises the steps of starting service, state checking and service coding in the process of starting service, starting the redis services in each machine one by one, checking whether the redis process is normally serviced, starting one redis service one by one in the process of starting the redis services one by one, automatically synchronizing the cache on each machine by the method, informing the access address and the starting state of newly added redis of the cache on each machine, recording if the starting fails, maintaining the redis service normally by manual maintenance, and updating the cache on each machine. The service codes are digital codes for providing a sequence for each redis service according to the starting sequence, so that the definition of the upper node and the lower node of the chain can be performed according to the sequence in the subsequent chaining process.
In the chaining process, the method executes the slave command one by one according to the sequential codes of the redis service of each machine buffer, and the slave command parameter takes the ip and the port of the last serial number of the local machine from each node buffer, so that the service on each node can be chained by executing the command one by one.
In the method, a timing task is registered in a post-chain checking process, the connectivity among all nodes is checked, then an info reply command is executed by all nodes to check the state of master-slave configuration and the data synchronization state, whether the nodes have problems is analyzed according to the acquired result, if the nodes have the problems of communication, the cache data of all the nodes are updated in time, the problem nodes are removed from the chain, and a downstream node of the problem nodes directs a slave command to an upstream node of the problem nodes.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (6)
1. A method for automatically constructing a redis multi-slave node chained copy is characterized by comprising the following steps of: the method comprises the following steps:
checking the running environment;
node executable program preparation;
starting a service;
service chaining;
post-chain inspection.
2. The method for automatically constructing the redis multi-slave node chained copy according to claim 1, wherein the method comprises the following steps of: the specific operation steps of the operation environment inspection comprise:
selecting one deployment node machine, and configuring ssh information of each node machine;
a checking step in the running method, wherein a scp command is used for copying a checking script to each node machine;
and invoking the checking script node by using the ssh command to check the version of the operating system, the dependence of the runtime environment and the information of the memory storage disk in the cpu.
3. The method for automatically constructing the redis multi-slave node chained copy according to claim 1, wherein the method comprises the following steps of: the specific operation steps of the node executable program preparation include:
compressing the package to each node by the scp copy executable program;
ssh decompressing the compressed packets;
and installing the runtime environment at each node in turn.
4. The method for automatically constructing the redis multi-slave node chained copy according to claim 1, wherein the method comprises the following steps of: the specific operation steps of service starting include:
sequentially connecting node machines through ssh, and executing a redis starting command;
detecting a result of starting redis by each node;
after the redis is successfully started, recording the starting state of each node, including connecting ip, recording the password into a cache, and distributing the password into the cache of each node;
and generating serial numbers according to the starting sequence, and updating the caches of the records of all nodes.
5. The method for automatically constructing the redis multi-slave node chained copy according to claim 1, wherein the method comprises the following steps of: the specific operation steps of service chaining include:
the method comprises the steps of obtaining the generation numbers of cache records of each node, and sequentially connecting the generation numbers to each node;
each node executes a slave port, ip is the ip of the node upstream of the current node of the cache record, and the port is the service port of redis.
6. The method for automatically constructing the redis multi-slave node chained copy according to claim 1, wherein the method comprises the following steps of: the specific operation steps of the post-chain inspection include:
checking the script to each node machine using the scp copy chain;
according to the actual service, designating a timing task execution interval;
registering a system timing task, wherein the working content of the timing task is to check the connectivity among the nodes, then checking the state of master-slave configuration and the data synchronization state through executing an initiation command by the nodes, analyzing whether the nodes have problems according to the acquired results, if so, updating the cache data of the nodes in time, removing the problem nodes from the chain, and directing a command to an upstream node of the problem nodes by a downstream node of the problem nodes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311483567.4A CN117493458A (en) | 2023-11-09 | 2023-11-09 | Method for automatically constructing chain type replication of redis multi-slave nodes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311483567.4A CN117493458A (en) | 2023-11-09 | 2023-11-09 | Method for automatically constructing chain type replication of redis multi-slave nodes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117493458A true CN117493458A (en) | 2024-02-02 |
Family
ID=89684456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311483567.4A Pending CN117493458A (en) | 2023-11-09 | 2023-11-09 | Method for automatically constructing chain type replication of redis multi-slave nodes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117493458A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118101428A (en) * | 2024-04-24 | 2024-05-28 | 浪潮云信息技术股份公司 | Redis chain type replication fault detection and repair method and device |
-
2023
- 2023-11-09 CN CN202311483567.4A patent/CN117493458A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118101428A (en) * | 2024-04-24 | 2024-05-28 | 浪潮云信息技术股份公司 | Redis chain type replication fault detection and repair method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110389900B (en) | Distributed database cluster testing method and device and storage medium | |
| CN109831500B (en) | Synchronization method for configuration file and Pod in Kubernetes cluster | |
| CN108076098B (en) | Service processing method and system | |
| CN107797767B (en) | One kind is based on container technique deployment distributed memory system and its storage method | |
| CN117493458A (en) | Method for automatically constructing chain type replication of redis multi-slave nodes | |
| CN104615598B (en) | The emigration processing method and device of meta data server | |
| CN112199178A (en) | Cloud service dynamic scheduling method and system based on lightweight container | |
| CN111371599A (en) | Cluster disaster recovery management system based on ETCD | |
| CN113946559A (en) | Data processing method, target database system and data processing system | |
| CN111064626A (en) | Configuration updating method, device, server and readable storage medium | |
| CN114185558A (en) | Native application master selection method and device based on K8s and storage medium | |
| CN115658166A (en) | System, method and medium for centralized management and easy-to-use application configuration | |
| Gu et al. | Compositional model checking of consensus protocols via interaction-preserving abstraction | |
| CN112559525B (en) | Data checking system, method, device and server | |
| CN101751292B (en) | Method for realizing consistency function of multimachine core data in ATC (automatic timing corrector) system | |
| CN116633766A (en) | Fault processing method and device, electronic equipment and storage medium | |
| CN114500289B (en) | Control plane recovery method, device, control node and storage medium | |
| CN113179332B (en) | Method, electronic device and storage medium for acquiring configuration information | |
| CN115238006A (en) | Retrieval data synchronization method, device, equipment and computer storage medium | |
| CN115248751A (en) | Redis cache disaster recovery switching method, device, platform, equipment and storage medium | |
| CN113672591A (en) | Data migration method, system, storage medium and electronic device | |
| CN110007934B (en) | Distributed database deployment method and device | |
| CN113901047A (en) | Simple cluster master-slave election method based on memory database | |
| CN113238950A (en) | System and method for testing distributed system, storage medium and electronic equipment | |
| CN112269803A (en) | Data set filling method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |