CN114756527A - Method and device for expanding Redis cluster, electronic equipment and storage medium - Google Patents

Method and device for expanding Redis cluster, electronic equipment and storage medium Download PDF

Info

Publication number
CN114756527A
CN114756527A CN202210376717.0A CN202210376717A CN114756527A CN 114756527 A CN114756527 A CN 114756527A CN 202210376717 A CN202210376717 A CN 202210376717A CN 114756527 A CN114756527 A CN 114756527A
Authority
CN
China
Prior art keywords
migration
slot
node
migrated
slot position
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210376717.0A
Other languages
Chinese (zh)
Inventor
应宏辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Tuhu Information Technology Co ltd
Original Assignee
Shanghai Tuhu Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Tuhu Information Technology Co ltd filed Critical Shanghai Tuhu Information Technology Co ltd
Priority to CN202210376717.0A priority Critical patent/CN114756527A/en
Publication of CN114756527A publication Critical patent/CN114756527A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/214Database migration support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a method, a device, electronic equipment and a storage medium for expanding a Redis cluster, wherein the method comprises the following steps: adding the newly added nodes into the cluster, distributing the slot positions to all the nodes evenly, and determining a slot position migration plan; acquiring the configuration of a slot position needing to be migrated; executing a slot position migration plan, and migrating slot positions needing to be migrated from a source node to a target node; and judging whether the migration process is abnormal or not, if so, judging whether the migration process is abnormal, if so, performing compatible processing, and if not, sending alarm information.

Description

Method and device for expanding Redis cluster, electronic equipment and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for expanding a Redis cluster, an electronic device, and a storage medium.
Background
Redis refers to an open source journaled, keyed database written in ANSIC (American national standards institute for standards, C), supporting networks, based on memory, and persistent, and providing multiple language application interfaces. The Redis cluster is a decentralized cluster, and each node is connected with other nodes to exchange information with each other.
In the prior art, when performing capacity expansion of a Redis cluster, a manual processing method is usually adopted, that is, after receiving a capacity expansion notification for the Redis cluster notified by a service person, an administrator inputs a relevant command on the Redis cluster to operate the Redis cluster.
The prior art mainly has the following three problems:
firstly, the Redis cluster expansion process is complex and is easy to make mistakes.
Secondly, the data migration process of the Redis cluster takes a long time, and one time of capacity expansion migration may take more than ten hours, for example, operation and maintenance service is performed at night, which makes operation and maintenance difficult.
Thirdly, the conditions of large keys and the like in the Redis cluster easily cause capacity expansion failure, abnormal conditions are not correspondingly processed, and great hidden dangers exist.
Disclosure of Invention
The following presents a summary of various exemplary aspects. Some simplifications and omissions may be made in the following summary, which is intended to highlight and introduce some aspects of the various exemplary embodiments, but not to limit the scope of the invention. Detailed descriptions of exemplary embodiments sufficient to enable those of ordinary skill in the art to make and use the inventive concepts will be presented in the following sections.
The technical scheme of the invention provides a method for expanding a Redis cluster, which does not overcome the problems in the prior art and comprises the following steps: adding the newly added nodes into the cluster, distributing the slot positions to all the nodes evenly, and determining a slot position migration plan; acquiring the configuration of a slot position needing to be migrated; executing a slot position migration plan, and migrating slot positions needing to be migrated from a source node to a target node; and judging whether the migration process is abnormal or not, if so, judging whether the migration process is abnormal, if so, performing compatible processing, and if not, sending alarm information.
Optionally, the allocating the slot evenly to all nodes comprises: and dividing the total slot number by the node number to obtain the average slot number of each node, and if the remainder exists, randomly distributing the remainder to each node.
Optionally, the configuration of the slot to be migrated includes a slot migration speed.
Optionally, the executing the slot migration plan includes: and executing the slot position migration plan in a multithreading mode.
Optionally, the executing the slot migration plan includes: in one thread, a single slot position is used as a dimension, and after one slot position is migrated, whether the next slot position is migrated or not is judged according to the migration condition of the previous slot position.
Optionally, the method further comprises: and after the migration is finished, setting a home node, and adopting a locking mode to disallow concurrent setting of node home at the same time.
Optionally, after the determining the slot migration plan, further comprising; and manually adjusting the slot position migration plan.
Optionally, the method further comprises: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
The technical solution of the present invention further provides a capacity expansion device for a Redis cluster, including: the computing module is configured to add the newly added nodes into the cluster, evenly distribute the slot positions to all the nodes and determine a slot position migration plan; the configuration module is configured to obtain the configuration of the slot position needing to be migrated; the migration module is configured to execute a slot position migration plan and migrate a slot position needing to be migrated from a source node to a target node; and the processing module is configured to judge whether the migration process is abnormal or not, judge whether the migration process is abnormal or not if the migration process is abnormal, perform compatible processing if the migration process is abnormal, and send alarm information if the migration process is not abnormal.
Optionally, the apparatus further comprises: and the locking module is configured to set the home node after the migration is completed, adopt a locking mode and not allow the concurrent setting of the node home at the same time.
Optionally, the apparatus further comprises: a notification module: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
Optionally, the migration module is further configured to: in one thread, a single slot position is taken as a dimension, and after one slot position is migrated, whether the next slot position is migrated is judged according to the migration condition of the previous slot position.
The technical scheme of the invention also provides a terminal, which comprises the following steps: processor, memory and computer program stored and run on the memory, characterized in that the processor implements the steps of any of the above-mentioned technical solutions method when executing the computer program.
The technical solution of the present invention also provides a computer-readable storage medium, and the computer program realizes the steps of any one of the above technical solutions when executed by a processor.
The invention has the following three beneficial effects:
Firstly, an algorithm is adopted, slot position node attribution can be automatically calculated, and slots are evenly distributed to all nodes.
Secondly, a multi-thread and multi-concurrency mode is adopted, the migration speed can be increased, the concurrency can be controlled through the thread quantity, and the quantity of batch migration keys can be changed, so that the speed is dynamically controlled.
Thirdly, in one thread, a single slot position is used as a dimension, condition judgment is carried out before the slot position is migrated every time, configuration is pulled again, migration can be suspended, migration can be resumed, and migration speed can be adjusted in the migration process.
Drawings
For a better understanding of the various exemplary embodiments, reference may be made to the accompanying drawings in which:
fig. 1 shows a flowchart of an embodiment of an extension method for a Redis cluster;
fig. 2 is a flowchart illustrating an embodiment of an extension method for a Redis cluster; and is provided with
Fig. 3 shows a schematic structural diagram of an extension apparatus of a Redis cluster according to an embodiment.
To facilitate understanding, the same reference numerals have been used to designate elements having substantially the same or similar structure and/or substantially the same or similar function.
Detailed Description
The description and drawings illustrate the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are included within its scope. Moreover, all examples cited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically cited examples and conditions. Further, as used herein, the term "or" refers to a non-exclusive or (i.e., and/or) unless otherwise indicated (e.g., "or otherwise" or in the alternative). Moreover, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments.
Fig. 1 shows an embodiment of an expansion method for a Redis cluster, including the following steps:
and S101, adding the newly added nodes into the cluster, distributing the slot positions to all the nodes evenly, and determining a slot position migration plan.
For example, 16384 hash slots (slots) may be built in a Redis cluster, and when a Redis cluster capacity expansion instruction is received, a migration plan is calculated according to an algorithm, where the calculation has two principles: firstly, the source node and the target node average as much as possible to occupy the slot position, and secondly, the slot position is migrated from the source node as little as possible.
Alternatively, the following algorithm may be used: and dividing the total slot number by the node number to obtain the average slot number of each node, and if the remainder exists, randomly distributing the remainder to each node to finally obtain the slot number of each node. For example, when a key-value needs to be placed in a Redis cluster, Redis calculates a result for the key by using the crc16 algorithm, and then calculates the remainder for the result pair 16384, so that each key corresponds to a hash slot with a number between 0 and 16383, and Redis maps the hash slot to different nodes according to the approximately equal number of nodes. For example, all the 16384 original hash slots all correspond to 1 node (node 1), and then the Redis cluster receives a capacity expansion command, and adds 2 nodes (node 2 and node 3), which are totally 3 nodes, and the total number of slots divided by the number of nodes is 16384/3 ═ 5461 remaining 1, and the average number of each node is 5461 slots calculated, and the remaining 1 slots are randomly allocated to 3 nodes, wherein 2 nodes are divided into 5461 slots, and 1 machine is divided into 5462 slots, for example, node 1 (i.e., the original node) may correspond to slots 0-5461, node 2 may correspond to slots 5462-.
Optionally, the source node is analyzed, and the Slot to be left is reserved, so that the Slot to be migrated out of each source node can be calculated. Next, the above example is analyzed, and data migration is required, namely, the slot 5462 and the slot 10922 are migrated from the source node (node 1) to the target node (node 2), and the slot 10923 and the slot 16383 are migrated from the source node (node 1) to another target node (node 3).
Optionally, the target node is analyzed, and the Slot to be migrated is obtained from the Slot migrated from the source node, so as to ensure that the Slot is continuous and whole-segment as much as possible.
After the automatic calculation is finished, a corresponding migration plan, key data, a source node and a target node are generated.
Optionally, the method further comprises: and manually adjusting the generated migration to form a better migration plan, and if the slot positions corresponding to the target node are scattered, manually adjusting the slot positions to ensure that the slot positions are continuous and reasonable as much as possible.
Step S102, obtaining the configuration of the slot position needing to be migrated.
Optionally, the configuration comprises a Slot migration velocity. And acquiring the Slot migration corresponding configuration from the cache before the Slot migration. The configuration includes whether migration is suspended, the number of Slot batch migration keys are migrated.
Step S103, executing the slot position migration plan, and migrating the slot position needing to be migrated from the source node to the target node.
Alternatively, the slot migration plan may be executed in a multi-threaded manner. The number of threads may be used to adjust the migration speed. And (4) executing the migration plan in a multithread manner, wherein each plan task can be participated in by a single node only once, and judging whether a Slot needs to be migrated or not in the plan. Multiple client services, which can be started in a jobmode (distributed nodes), each client service executes 1 generated plan by using a thread pool.
Optionally, in one thread, a single slot is used as a dimension, and after one slot is migrated, whether the next slot is migrated is determined according to the migration condition of the previous slot. And after the single Slot migration is finished, judging whether the single Slot migration has abnormity again. There may be two cases, one is normal migration, and if there are also migrated slots, then migration continues. Another case is migration failure. The migration failure is classified into an abnormal operation migration failure, a suspended service migration failure, and the like. And finding corresponding programs for processing different abnormal failures.
A single Slot is migrated with two results, an abnormal result and a successful result. And carrying out data migration on a single Slot, firstly setting a Slot state, migrating a target node in and migrating a source node out, then carrying out batch migration on the Slot key of the source node, and when a large key is encountered, temporarily skipping and migrating the key capable of being migrated. And when the key is migrated, locking to set the Slot attribution. Successful results: and successfully migrating the data to the target node, and setting the home relationship of the Slot. The abnormal conditions include: suspend migration, large key, migration process other unknown cases.
And step S104, judging whether the migration process is abnormal or not, if so, performing compatible processing, and if not, sending alarm information.
Optionally, if there is no exception, the data is successfully migrated to the target node, and the home relationship of the Slot is set. If there is an exception, the migration is halted.
For abnormal results, a compensation procedure for the abnormality is triggered. The pause case is skipped directly and the node homing case attempts to restore cluster node homing. In the first case: the Slot belongs to the node, and the cluster node is informed to explain the Slot attribution at the moment; and in the second situation, the Slot does not belong to the source node, whether the node recorded by the source node is used for the node is inquired at the moment, if the node belongs to the node, the whole cluster is informed, otherwise, an alarm is sent, and the operation and maintenance person carries out processing.
And step S105, setting a home node after the migration is finished, and setting node home in a locking mode at the same time without allowing concurrent setting.
Optionally, the method further comprises: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
Optionally, the method further comprises: and checking the cluster state. And if the migration is successful, verifying the cluster state and sending the migration task progress. After the smooth migration, whether the Slot condition of each node is consistent or not and whether the cluster information of each node is consistent or not are checked, so that the correctness of the cluster is ensured.
Fig. 2 is a schematic flowchart illustrating an embodiment of an extension method for a Redis cluster. Fig. 2 specifically shows a specific flow of slot migration. Here, one thread is taken as an example.
Start of
Slot migration is generated 201, and slots are evenly distributed to each node according to an algorithm.
202 whether to perform the next slot migration; if so, 203 is executed, whether to execute the next slot migration may be determined according to whether the previous slot migration is successful, and if so, the next slot migration is continuously executed.
203, obtaining migration configuration parameters including the conditions of migration speed, whether to suspend migration and batch migration key.
205, migrating a single slot, returning to the step 202 after the execution is finished, and executing in a loop. And carrying out data migration on a single Slot, firstly setting a Slot state, migrating a target node in and migrating a source node out, then carrying out batch migration on the Slot key of the source node, and when a large key is encountered, temporarily skipping and migrating the key capable of being migrated. And when the key is migrated, locking to set the Slot attribution. And (4) migrating a single slot position, wherein two results are provided, namely an abnormal result and a successful result. Successful results: and successfully migrating the data to the target node, and setting the home relationship of the Slot. The abnormal conditions include: suspend migration, large key, migration process other unknown cases.
After step 202 is executed, step 204 is executed to determine whether a single slot has an exception again, if yes, step 206 is executed, and if no, step 208 is executed. There are two cases, one is that the last slot is normally migrated, then the next slot is continuously migrated. Another case is migration failure. The migration failure is classified into an abnormal operation migration failure, a suspended service migration failure, and the like. And finding corresponding programs for processing different abnormal failures.
206 trigger the compensation mechanism; the pause case is skipped directly and the node homing case attempts to restore cluster node homing. In the first case: the Slot belongs to the node, and the cluster node is informed to explain the Slot attribution at the moment; and in the second situation, the Slot does not belong to the source node, whether the node recorded by the source node is used for the node is inquired at the moment, if the node belongs to the node, the whole cluster is informed, otherwise, an alarm is sent, and the operation and maintenance person carries out processing.
207 sends an abnormal alarm to inform the operation and maintenance person.
208 migrate smoothly, check, and send notifications.
And (6) ending.
Fig. 3 is a schematic structural diagram of an expansion apparatus of a Redis cluster according to an embodiment.
A capacity expansion device for a Redis cluster, comprising: a calculation module 301, a configuration module 302, a migration module 303, a processing module 304, and a lock module 305.
The calculation module 301 is configured to add the newly added node into the cluster, equally allocate slots to all nodes, and determine a slot migration plan.
Alternatively, the following algorithm may be used: and if the remainder exists, the remainder is randomly distributed to each node, and finally the slot number of each node is obtained. For example, when a key-value needs to be placed in a Redis cluster, Redis calculates a result by using a crc16 algorithm on the key, and then calculates the remainder of the result on 16384, so that each key corresponds to a hash slot with the number of 0-16383, and Redis maps the hash slot to different nodes according to the roughly equal number of the nodes
Optionally, the calculating module 301 analyzes the source nodes, and reserves the slots to be left, so that the Slot to be migrated out of each source node can be calculated. Next, the above example is analyzed, and data migration is required, namely, the slot with numbers 5462 and 10922 is migrated from the source node (node 1) to the target node (node 2), and the slot with numbers 10923 and 16383 is migrated from the source node (node 1) to another target node (node 3).
Optionally, the calculating module 301 analyzes the target node, and obtains the Slot to be migrated from the Slot migrated from the source node, so as to ensure that the Slot is continuous and whole-segment as much as possible.
After the calculation module 301 completes the automatic calculation, the corresponding migration plan, the key data, the source node, and the target node are generated.
Optionally, the calculation module 301 further adjusts the generated migration manually to form a better migration plan, and if the slot positions corresponding to the target node are scattered, the slot positions can be adjusted manually to ensure that the slot positions are continuous and reasonable as much as possible.
A configuration module 302 configured to obtain a configuration of slots that need to be migrated.
Optionally, the configuration comprises a Slot migration velocity. And acquiring the Slot migration corresponding configuration from the cache before the Slot migration. The configuration includes whether migration is suspended, the number of Slot batch migration keys are migrated.
The migration module 303 is configured to execute the slot migration plan, and migrate the slot to be migrated from the source node to the target node.
Alternatively, the slot migration plan may be executed in a multi-threaded manner. The number of threads may be used to adjust the migration speed. And (4) multithreading is carried out to execute the migration plan, each plan task can be participated in by a single node only once, and whether the Slot needs to be migrated or not is judged for the plan. Multiple client services, which can be started in a joba manner (distributed nodes), each client service executing 1 generated plan using a thread pool.
Optionally, in one thread, a single slot is used as a dimension, and after one slot is migrated, whether the next slot is migrated is determined according to the migration condition of the previous slot. And after the single Slot migration is finished, judging whether the single Slot migration has abnormity again. There may be two cases, one is normal migration, and if there are also migrated slots, then migration continues. Another case is migration failure. The migration failure is classified into an abnormal operation migration failure, a suspended service migration failure, and the like. And finding corresponding programs for processing different abnormal failures.
A single Slot is migrated with two results, an abnormal result and a successful result. And carrying out data migration on a single Slot, firstly setting a Slot state, migrating a target node in and migrating a source node out, then carrying out batch migration on the Slot key of the source node, and when a large key is encountered, temporarily skipping and migrating the key capable of being migrated. And when the key is migrated, locking to set the Slot attribution. Successful results: and successfully migrating the data to the target node, and setting the home relationship of the Slot. The abnormal conditions include: suspend migration, large key, migration process other unknown cases.
The processing module 304 is configured to determine whether the migration process has an abnormality, determine whether the migration process is a known abnormality if the migration process has an abnormality, perform compatible processing if the migration process has a known abnormality, and send alarm information if the migration process does not have a known abnormality.
Optionally, if there is no exception, the data is successfully migrated to the target node, and the home relationship of the Slot is set. If there is an exception, the migration is halted.
For exception results, a compensation for exception procedure is triggered. The pause case is skipped directly and the node homing case attempts to restore cluster node homing. In the first case: the Slot belongs to the node, and the cluster node is informed to explain the Slot attribution at the moment; and in the second situation, the Slot does not belong to the source node, whether the node recorded by the source node is used for the node is inquired at the moment, if the node belongs to the node, the whole cluster is informed, otherwise, an alarm is sent, and the operation and maintenance person carries out processing.
Optionally, the apparatus further comprises: and the locking module 305 is configured to set the home node after the migration is completed, and adopt a locking mode to not allow the node home to be set concurrently at the same time. Optionally, the method further comprises: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
Optionally, the apparatus further comprises: a notification module: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
Optionally, the method further comprises: and checking the cluster state. And if the migration is successful, verifying the cluster state and sending the migration task progress. After the smooth migration, whether the Slot condition of each node is consistent or not and whether the cluster information of each node is consistent or not are checked, so that the correctness of the cluster is ensured.
An embodiment further provides a terminal, including: processor, memory and computer program stored in the memory for execution, wherein the processor implements the steps of the method of any of the above embodiments, such as steps S101 to S105, when executing the computer program, or implements the functions of the modules/units in the above embodiments, such as the functions of units 201 to 205 shown in fig. 1, when executing the computer program. The computer program may be divided into one or more modules/units, which are stored in the memory and executed by the processor. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program in the terminal.
The terminal can be a mobile terminal such as a smart phone or a computing device such as a desktop computer, a notebook, a palm computer and a cloud server. The terminal may include, but is not limited to, a processor, memory, may include more or fewer components, or may combine certain components, e.g., the terminal may also include an input-output device, a network access device, a bus, etc. The processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory may also be an external storage device of the terminal, such as a plug-in hard disk provided on the terminal, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card (FlashCard), and the like. Further, the memory may also include both an internal storage unit and an external storage device of the terminal.
Embodiments also provide a computer-readable storage medium, which when executed by a processor implements the steps of any of the above-described embodiment methods.
Each functional unit in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit. The integrated module/unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, Read-only memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the above-mentioned apparatus may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment. Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus, terminal and method may be implemented in other ways. For example, the above-described device and terminal embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (14)

1. A method for expanding Redis cluster is characterized by comprising the following steps:
adding the newly added nodes into the cluster, distributing the slot positions to all the nodes evenly, and determining a slot position migration plan;
acquiring the configuration of a slot position needing to be migrated;
executing a slot position migration plan, and migrating slot positions needing to be migrated from a source node to a target node;
and judging whether the migration process is abnormal or not, if so, judging whether the migration process is abnormal, if so, performing compatible processing, and if not, sending alarm information.
2. The method of claim 1, wherein the evenly allocating slots to all nodes comprises: and dividing the total slot number by the node number to obtain the average slot number of each node, and if the remainder exists, randomly distributing the remainder to each node.
3. The method of claim 1, wherein the configuration of the slots to be migrated comprises a slot migration speed.
4. The method of claim 1, wherein the executing the slot migration plan comprises: and executing the slot position migration plan in a multithreading mode.
5. The method for capacity expansion of a Redis cluster according to claim 1, wherein the executing the slot migration plan comprises: in one thread, a single slot position is used as a dimension, and after one slot position is migrated, whether the next slot position is migrated or not is judged according to the migration condition of the previous slot position.
6. The method for accommodating Redis clusters according to claim 1, further comprising: and after the migration is finished, setting a home node, and adopting a locking mode to set the node home at the same time without allowing concurrency.
7. The method for accommodating Redis clusters according to claim 1, further comprising, after said determining a slot migration plan; and manually adjusting the slot position migration plan.
8. The method for the capacity expansion of Redis clusters according to claim 1, further comprising: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, success of migration, migration time and migration speed.
9. An apparatus for capacity expansion of a Redis cluster, comprising:
the computing module is configured to add the newly added nodes into the cluster, evenly distribute the slot positions to all the nodes and determine a slot position migration plan;
The configuration module is configured to acquire the configuration of the slot position needing to be migrated;
the migration module is configured to execute a slot position migration plan and migrate a slot position needing to be migrated from a source node to a target node;
and the processing module is configured to judge whether the migration process has an abnormality, judge whether the migration process is a known abnormality if the migration process has the abnormality, perform compatible processing if the migration process has the known abnormality, and send alarm information if the migration process does not have the known abnormality.
10. The apparatus for capacity extension of a Redis cluster according to claim 9, further comprising:
and the locking module is configured to set the home node after the migration is completed, and set the node home in a locking mode at the same time without allowing concurrency.
11. The apparatus for capacity extension of a Redis cluster according to claim 9, further comprising:
a notification module: sending a notification to a migrated important event node, wherein the important event comprises: starting migration, successful migration, migration time and migration speed.
12. The apparatus for capacity expansion of a Redis cluster of claim 9, wherein the migration module is further configured to: in one thread, a single slot position is taken as a dimension, and after one slot position is migrated, whether the next slot position is migrated is judged according to the migration condition of the previous slot position.
13. An electronic device, comprising: processor, memory and a computer program stored for execution on the memory, characterized in that the processor, when executing the computer program, implements the method of extensions of a Redis cluster according to any of claims 1-8.
14. A computer-readable storage medium, characterized in that the computer program, when being executed by a processor, implements the method of extensions to Redis clusters of any of the claims 1-8.
CN202210376717.0A 2022-04-12 2022-04-12 Method and device for expanding Redis cluster, electronic equipment and storage medium Pending CN114756527A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210376717.0A CN114756527A (en) 2022-04-12 2022-04-12 Method and device for expanding Redis cluster, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210376717.0A CN114756527A (en) 2022-04-12 2022-04-12 Method and device for expanding Redis cluster, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114756527A true CN114756527A (en) 2022-07-15

Family

ID=82329266

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210376717.0A Pending CN114756527A (en) 2022-04-12 2022-04-12 Method and device for expanding Redis cluster, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114756527A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117439993A (en) * 2023-12-22 2024-01-23 中电云计算技术有限公司 Redis cluster load balancing method, device, equipment and storage medium
CN118377435A (en) * 2024-06-24 2024-07-23 杭州宇信数字科技有限公司 Cluster capacity expansion and contraction method, device, medium and equipment based on terminal agent

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117439993A (en) * 2023-12-22 2024-01-23 中电云计算技术有限公司 Redis cluster load balancing method, device, equipment and storage medium
CN117439993B (en) * 2023-12-22 2024-04-16 中电云计算技术有限公司 Redis cluster load balancing method, device, equipment and storage medium
CN118377435A (en) * 2024-06-24 2024-07-23 杭州宇信数字科技有限公司 Cluster capacity expansion and contraction method, device, medium and equipment based on terminal agent

Similar Documents

Publication Publication Date Title
CN108549580B (en) Method for automatically deploying Kubernets slave nodes and terminal equipment
CN109683826B (en) Capacity expansion method and device for distributed storage system
CN107666525B (en) Method and device for IP allocation of cluster container
US11194569B2 (en) Method, electronic device and medium for upgrading a hyper-converged infrastructure node
CN114756527A (en) Method and device for expanding Redis cluster, electronic equipment and storage medium
US9594801B2 (en) Systems and methods for allocating work for various types of services among nodes in a distributed computing system
CN109032796B (en) Data processing method and device
CN110225078B (en) Application service updating method, system and terminal equipment
CN111694638A (en) Rule package loading method, rule package executing method and terminal equipment
CN111209110A (en) Task scheduling management method, system and storage medium for realizing load balance
CN106940671B (en) Method, device and system for monitoring running of task threads in cluster
CN112579550B (en) Metadata information synchronization method and system of distributed file system
WO2021114025A1 (en) Incremental data determination method, incremental data determination apparatus, server and terminal device
CN106407395A (en) A processing method and device for data query
CN112650812A (en) Data fragment storage method and device, computer equipment and storage medium
CN111538585A (en) Js-based server process scheduling method, system and device
CN114185558A (en) Native application master selection method and device based on K8s and storage medium
US9832137B1 (en) Provisioning system and method for a distributed computing environment using a map reduce process
CN115033551A (en) Database migration method and device, electronic equipment and storage medium
CN112631994A (en) Data migration method and system
CN108920936B (en) Method and device for achieving vmware authentication based on Purley platform
CN111431951B (en) Data processing method, node equipment, system and storage medium
CN112579247A (en) Method and device for determining task state
US20230019037A1 (en) Reactive non-blocking input and output for target device communication
CN114070889B (en) Configuration method, traffic forwarding device, storage medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination