CN114116216A - Method and device for realizing high availability of distributed block storage based on vip - Google Patents
Method and device for realizing high availability of distributed block storage based on vip Download PDFInfo
- Publication number
- CN114116216A CN114116216A CN202111403414.5A CN202111403414A CN114116216A CN 114116216 A CN114116216 A CN 114116216A CN 202111403414 A CN202111403414 A CN 202111403414A CN 114116216 A CN114116216 A CN 114116216A
- Authority
- CN
- China
- Prior art keywords
- vip
- high availability
- block storage
- session
- distributed block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0617—Improving the reliability of storage systems in relation to availability
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Hardware Redundancy (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to the technical field of storage, in particular to a method and a device for realizing distributed block storage high availability based on VIP. The invention realizes the high-availability network protocol by drifting the IP mode on the unified view, the unified view ensures that the floating IP can be bound to the available target service in the cluster, and the VIP can be ensured to be available under the condition that the cluster is available due to the high availability of the distributed storage service, thereby ensuring the connection reliability of the storage protocol client. The invention supports general hardware and a general iSCSI client, and does not increase hardware cost and client configuration management work. The usability and the universality of the software defined storage product are optimized.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a method and a device for realizing high availability of distributed block storage based on vip.
Background
With the development of storage technology, the iSCSI as a general interface for software defined storage has the advantages of low construction price, rich and general client and the like, and greatly promotes the popularization and application of a Storage Area Network (SAN). However, the redundant access mechanism provided in high availability has the disadvantages of cost, client support, complex configuration management and the like. In general, the iSCSI protocol provides 3 mechanisms, i.e., network card binding, multiple sessions, and multiple links.
The bottom layer of the iSCSI is IP and Ethernet, theoretically, the method can be directly implemented from a network card, a plurality of network cards on a host are bound under 1 IP by using a Port traversing/learning/Link Aggregation mode, then the network cards are connected to the iSCSI storage equipment, and corresponding settings on a transmission Port of the iSCSI storage equipment are matched, so that entity multipath connection is realized. See fig. 1.
Multiple Sessions may allow 1 iSCSI Initiator to connect to the same 1 iSCSI Target device through Multiple Sessions, so as to enable a failover mechanism, also known as a Multiple Sessions per Initiator, using Multiple network cards or iSCSI HBAs. See fig. 2.
Multiple links may allow multiple TCP/IP connections to be established between an iSCSI Initiator and an iSCSI Target in the same Session, and may also allow a user to enable a failover mechanism using multiple network cards or iSCSI HBAs. See fig. 3.
In short, the multi-session operates on a higher network stack layer, and the load balancing mechanism among multiple access paths operates for 1 designated independent logical driver (LUN); and the multi-link operates on the iSCSI layer, so that the transmission verification capability is better.
The problem of the physical network card binding mode is that a plurality of network cards or network ports are required to be connected to the Target end. If no more than 2 network ports are available, the host does not naturally talk about multipath access and requires additional network card management tools, as shown in fig. 2, the binding method lacks inter-node synchronization in a distributed environment. And therefore can only be supplemented by other solutions.
The multi-session and multi-link require iSCSI Initiator software and provide iSCSI equipment software support to establish multi-path access, the iSCSI Initiator software provided by each main operating system manufacturer has difference and defects in universality, and the multi-session and multi-link require additional configuration management of a client.
Disclosure of Invention
Aiming at the defects of the prior art, the invention discloses a method and a device for realizing distributed block storage high availability based on VIP, which are used for distributing a plurality of VIPs among nodes in a distributed block storage cluster to realize iSCSI high availability and load balancing.
The invention is realized by the following technical scheme:
in a first aspect, the present invention provides a method for realizing high availability of distributed block storage based on vip, which includes the following steps:
s1, acquiring all nodes of the cluster, electing a master node through a raft algorithm, and establishing the master to the heartbeat of all target services;
s2, establishing a uniform hash view according to the targetID, and uniformly distributing the multiple VIPs into the view;
s3, each node VIP service obtains the distribution result from the master and active or inactive VIP; and active/inactive local VIP broadcasts to the service network through ARP after the application records are updated;
s4 sends reset command to service after iSCSI initiator receives ARP, and processes new and old links through session;
s5 repeating S1-S4 to reapply floating IP after the master adjusts target view through heartbeat, ensuring service IP availability and further realizing iSCSI protocol high availability.
Furthermore, in the method, a master node is selected through a raft algorithm, and an external component etcd tool is adopted to select and drift the master.
Furthermore, in the method, the master uniformly distributes the multiple VIPs into the views, and a consistent hash algorithm is adopted in the process, so that the multiple VIPs are balanced.
Furthermore, in the method, the application record is subscribed through the VIP service, and after the updated active/inactive local VIP is received, the updated active/inactive local VIP is broadcasted to the service network through the ARP.
Furthermore, in the method, when the session processes the new link and the old link, the session is reestablished, and the session manager switches the new link and the old link.
Furthermore, in the method, when the session processes the new link and the old link, the onfly data on the old session is all retried.
Furthermore, the method ensures that the floating IP can be bound to the available target service in the cluster by the unified view.
Further, in the method, the VIP is used to provide a fixed "virtual" access address to the client.
Further, in the method, the VIP is one or more and when one or more clients link to the storage, the links are evenly distributed to all nodes of the cluster.
In a second aspect, the present invention provides a device for realizing distributed block storage high availability based on vip, where the device is used to realize the method for realizing distributed block storage high availability based on vip in the first aspect, and the method includes
The master node is used for controlling the distribution and the drift of the VIP through a control node selected among the storage nodes by a raft algorithm;
the slave node is used for receiving the master synchronous unified view and externally publishing the available VIP;
the vip service is used for running on all nodes, is used for active or inactive vip, and carries out ARP broadcast to a service network;
and the session manager is used for recording a new session and an old session and processing the onfly data on the old session in the VIP switching process.
The invention has the beneficial effects that:
the invention realizes the high-availability network protocol by drifting the IP mode on the unified view, the unified view ensures that the floating IP can be bound to the available target service in the cluster, and the VIP can be ensured to be available under the condition that the cluster is available due to the high availability of the distributed storage service, thereby ensuring the connection reliability of the storage protocol client.
The invention supports general hardware and a general iSCSI client, and does not increase hardware cost and client configuration management work. The usability and the universality of the software defined storage product are optimized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a diagram of a network card binding high availability mode in the background art of the present invention;
FIG. 2 is a multi-session high availability graph of the background of the invention;
FIG. 3 is a background art multilink high availability graph of the present invention;
figure 4 is a flow diagram of VIP allocation according to an embodiment of the present invention;
FIG. 5 is a VIP service flow diagram according to an embodiment of the invention;
fig. 6 is a flowchart illustrating session manager switching according to an embodiment of the present invention;
FIG. 7 is a VIP drift flow diagram according to an embodiment of the invention;
fig. 8 is a diagram illustrating the high availability of VIP according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
The embodiment provides a method for realizing high availability of distributed block storage based on vip, which comprises the following steps:
s1, acquiring all nodes of the cluster, electing a master node through a raft algorithm, and establishing the master to the heartbeat of all target services;
s2, establishing a uniform hash view according to the targetID, and uniformly distributing the multiple VIPs into the view;
s3, each node VIP service obtains the distribution result from the master and active or inactive VIP; and active/inactive local VIP broadcasts to the service network through ARP after the application records are updated;
s4 sends reset command to service after iSCSI initiator receives ARP, and processes new and old links through session;
s5 repeating S1-S4 to reapply floating IP after the master adjusts target view through heartbeat, ensuring service IP availability and further realizing iSCSI protocol high availability.
The present embodiment allocates multiple VIPs among nodes in a distributed block storage cluster to achieve iSCSI high availability and load balancing.
In this embodiment, the VIP is a Virtual IP Address, and the VIP is used to provide a fixed "Virtual" access Address to the client, so as to avoid an influence on the client when the backend server is switched.
The multiple VIPs in the embodiment ensure that when a plurality of clients are linked to the storage, the links can be uniformly distributed to all nodes of the cluster, so that the balance of the number of the links and the flow is achieved.
Example 2
In a specific implementation level, this embodiment provides a specific implementation of a method for implementing distributed block storage high availability based on vip, which is specifically as follows:
1) all nodes of the cluster select the master node through a raft algorithm, and an external component etcd tool is adopted to select and drift the master in the invention.
2) The master establishes heartbeats to all target services and establishes a unified hash view according to the target ID.
3) The master distributes the multiple VIPs into the views uniformly, and the process adopts a consistent hash algorithm to realize the multiple VIPs balance.
4) And each node VIP service acquires the distribution result from the master and activates or inactivates VIP. The specific flow is shown in figure 4.
5) The VIP service subscribes to the application record, receives the updated active/inactive local VIP and broadcasts the updated active/inactive local VIP to the service network through the ARP. See in particular fig. 5.
6) The session processes the new link and the old link, the iSCSI initiator sends a reset instruction to the service after receiving the ARP, the session is reestablished, the session manager switches the new session and the old session, and the onfly data on the old session is retried. See in particular fig. 6.
7) After the master adjusts the target view by heartbeat, the steps 1-6 are repeated to reapply the floating IP. Service IP is guaranteed to be available, and high availability of an iSCSI protocol is achieved. See in particular fig. 7.
In the embodiment, a high-availability network protocol is realized in a mode of drifting an IP on a unified view, the unified view ensures that a floating IP can be bound to a target service available in a cluster, and due to the high availability of the distributed storage service, VIP can be ensured to be available under the condition that the cluster is available, so that the connection reliability of a storage protocol client is ensured.
The embodiment supports universal hardware and a universal iSCSI client, and does not increase hardware cost and client configuration management work. The usability and the universality of the software defined storage product are optimized, and the effect of the software defined storage product is shown in a figure 8.
Example 3
The embodiment provides a device for realizing distributed block storage high availability based on vip, which comprises
The master node is used for controlling the distribution and the drift of the VIP through a control node selected among the storage nodes by a raft algorithm;
the slave node is used for receiving the master synchronous unified view and externally publishing the available VIP;
the vip service is used for running on all nodes, is used for active or inactive vip, and carries out ARP broadcast to a service network;
and the session manager is used for recording a new session and an old session and processing the onfly data on the old session in the VIP switching process.
In summary, the present invention implements a high-availability network protocol by drifting an IP on a unified view, the unified view ensures that a floating IP can be bound to a target service available in a cluster, and due to the high availability of the distributed storage service, it can be ensured that VIP is available when the cluster is available, thereby ensuring the connection reliability of the storage protocol client.
The invention supports general hardware and a general iSCSI client, and does not increase hardware cost and client configuration management work. The usability and the universality of the software defined storage product are optimized.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for realizing high availability of distributed block storage based on vip is characterized by comprising the following steps:
s1, acquiring all nodes of the cluster, electing a master node through a raft algorithm, and establishing the master to the heartbeat of all target services;
s2, establishing a uniform hash view according to the targetID, and uniformly distributing the multiple VIPs into the view;
s3, each node VIP service obtains the distribution result from the master and active or inactive VIP; and active/inactive local VIP broadcasts to the service network through ARP after the application records are updated;
s4 sends reset command to service after iSCSI initiator receives ARP, and processes new and old links through session;
s5 repeating S1-S4 to reapply floating IP after the master adjusts target view through heartbeat, ensuring service IP availability and further realizing iSCSI protocol high availability.
2. The vip-based method for achieving high availability of distributed block storage, according to claim 1, wherein in the method, a master node is selected through a raft algorithm, and an external component etcd tool is adopted to select and drift the master.
3. The VIP-based method for achieving the high availability of the distributed block storage, according to claim 1, wherein in the method, a master evenly distributes multiple VIPs to views, and a consistent hash algorithm is adopted in the process, so that the multiple VIP balancing is achieved.
4. The VIP-based implementation distributed block storage high availability method of claim 1, wherein the application is subscribed to record through VIP service, and active/inactive local VIP is received after update and broadcasted to a service network through ARP.
5. The vip-based method for achieving high availability of distributed block storage, as claimed in claim 1, wherein in the method, when a session processes a new link and an old link, the session is reestablished, and a session manager switches the new link and the old link.
6. The vip-based implementation distributed block storage high availability method of claim 5, wherein in the method, when a session processes a new link and an old link, an onfly data on the old session is retried in its entirety.
7. The vip-based implementation distributed block storage high availability method of claim 1, wherein the method guarantees that a floating IP can be bound to a target service available in a cluster by a unified view.
8. The VIP-based implementation distributed block storage high availability method of claim 1, wherein a VIP is used to provide a fixed "virtual" access address to a client.
9. The VIP-based implementation distributed block storage high availability method of claim 8, wherein the VIP is one or more and when one or more clients are linked to storage, the links are evenly distributed to all nodes of a cluster.
10. A vip based implementation distributed block storage high availability apparatus for implementing a vip based implementation distributed block storage high availability method according to any one of claims 1 to 9, comprising
The master node is used for controlling the distribution and the drift of the VIP through a control node selected among the storage nodes by a raft algorithm;
the slave node is used for receiving the master synchronous unified view and externally publishing the available VIP;
the vip service is used for running on all nodes, is used for active or inactive vip, and carries out ARP broadcast to a service network;
and the session manager is used for recording a new session and an old session and processing the onfly data on the old session in the VIP switching process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111403414.5A CN114116216A (en) | 2021-11-24 | 2021-11-24 | Method and device for realizing high availability of distributed block storage based on vip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111403414.5A CN114116216A (en) | 2021-11-24 | 2021-11-24 | Method and device for realizing high availability of distributed block storage based on vip |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114116216A true CN114116216A (en) | 2022-03-01 |
Family
ID=80371870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111403414.5A Pending CN114116216A (en) | 2021-11-24 | 2021-11-24 | Method and device for realizing high availability of distributed block storage based on vip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114116216A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103004A (en) * | 2022-06-30 | 2022-09-23 | 浪潮电子信息产业股份有限公司 | Session establishing method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453360A (en) * | 2016-10-26 | 2017-02-22 | 上海爱数信息技术股份有限公司 | Distributed block storage data access method and system based on iSCSI (Internet Small Computer System Interface) protocol |
| CN109032830A (en) * | 2018-07-25 | 2018-12-18 | 广东浪潮大数据研究有限公司 | A kind of fault recovery method of distributed memory system, system and associated component |
| CN113301005A (en) * | 2020-09-29 | 2021-08-24 | 阿里云计算有限公司 | Data acquisition method and device |
-
2021
- 2021-11-24 CN CN202111403414.5A patent/CN114116216A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453360A (en) * | 2016-10-26 | 2017-02-22 | 上海爱数信息技术股份有限公司 | Distributed block storage data access method and system based on iSCSI (Internet Small Computer System Interface) protocol |
| CN109032830A (en) * | 2018-07-25 | 2018-12-18 | 广东浪潮大数据研究有限公司 | A kind of fault recovery method of distributed memory system, system and associated component |
| CN113301005A (en) * | 2020-09-29 | 2021-08-24 | 阿里云计算有限公司 | Data acquisition method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103004A (en) * | 2022-06-30 | 2022-09-23 | 浪潮电子信息产业股份有限公司 | Session establishing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110912780B (en) | High-availability cluster detection method, system and controlled terminal | |
| US10389824B2 (en) | Service management modes of operation in distributed node service management | |
| US20190028538A1 (en) | Method, apparatus, and system for controlling service traffic between data centers | |
| US7911939B2 (en) | Fault tolerant symmetric multi-computing system | |
| US8880935B2 (en) | Redundancy and load balancing in remote direct memory access communications | |
| US8842518B2 (en) | System and method for supporting management network interface card port failover in a middleware machine environment | |
| US7089281B1 (en) | Load balancing in a dynamic session redirector | |
| US9762669B2 (en) | Service management roles of processor nodes in distributed node service management | |
| US9037719B2 (en) | Hypervisor level distributed load-balancing | |
| US11734138B2 (en) | Hot standby method, apparatus, and system | |
| WO2017114017A1 (en) | Computer device, system and method for implementing load balancing | |
| US6112249A (en) | Non-disruptively rerouting network communications from a secondary network path to a primary path | |
| US7401114B1 (en) | Method and apparatus for making a computational service highly available | |
| CN104468151B (en) | A kind of system and method for keeping TCP sessions during cluster switching | |
| US7805520B2 (en) | Storage system, program and method | |
| US10771402B2 (en) | Link aggregated fibre channel over ethernet system | |
| CN110928637A (en) | Load balancing method and system | |
| CN109525411B (en) | Network function component cluster, system, control method, device and storage medium | |
| CN114116216A (en) | Method and device for realizing high availability of distributed block storage based on vip | |
| JPH1027146A (en) | Communication processor and its method | |
| CN117527802A (en) | Implementation method for improving load balancing forwarding efficiency | |
| CN116346581A (en) | Communication method, device and system | |
| KR20100111443A (en) | Financial business treatment system and controlling method thereof | |
| JPH04252537A (en) | Communication processing unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220301 |