CN113489784A - Distributed storage asymmetric logic unit access multipath implementation method and system - Google Patents
Distributed storage asymmetric logic unit access multipath implementation method and system Download PDFInfo
- Publication number
- CN113489784A CN113489784A CN202110755683.1A CN202110755683A CN113489784A CN 113489784 A CN113489784 A CN 113489784A CN 202110755683 A CN202110755683 A CN 202110755683A CN 113489784 A CN113489784 A CN 113489784A
- Authority
- CN
- China
- Prior art keywords
- scsi
- gateway
- path
- logical volume
- metadata cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000002159 abnormal effect Effects 0.000 claims abstract description 13
- 230000001360 synchronised effect Effects 0.000 claims abstract description 8
- 230000008859 change Effects 0.000 claims abstract description 5
- 239000003999 initiator Substances 0.000 claims description 44
- 238000013507 mapping Methods 0.000 claims description 14
- 239000007858 starting material Substances 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000012423 maintenance Methods 0.000 claims description 4
- 230000005856 abnormality Effects 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 101100371648 Caenorhabditis elegans usp-14 gene Proteins 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007717 exclusion Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 101150090882 tgt-1 gene Proteins 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 101150041549 tgt-2 gene Proteins 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a distributed storage asymmetric logic unit access multipath realization method and a distributed storage asymmetric logic unit access multipath realization system, wherein the method comprises the following steps: the data cluster saves the SCSI gateway, the target device and the logical volume relation view; starting the SCSI gateway, and when the target device is associated with the SCSI gateway or the logical volume is associated with the target device or the SCSI gateway detects that the path is abnormal, informing the metadata cluster; the metadata cluster updates corresponding metadata information and synchronizes the updated information to the SCSI gateway; heartbeat is sent among the SCSI gateways, when the path abnormality is detected and reported to the metadata cluster, the metadata cluster updates metadata information after deciding that the SCSI gateway has a fault, and the updated information is synchronized to the SCSI gateway; and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logical volume if the path state sends a change. The invention realizes the path selection of the asymmetric logic unit by managing the SCSI gateway, the target device and the logical volume relation view through metadata clusters.
Description
Technical Field
The invention relates to the field of distributed storage, in particular to a distributed storage Asymmetric Logic Unit Access (ALUA) multi-path implementation method and system.
Background
A distributed storage system uses a plurality of independent servers to form a cluster to store data. Each server in the cluster acts as a node on which multiple processes run to manage multiple physical disks on the server. In order to ensure high reliability of data, a plurality of identical copies of the data are copied, and each copy is stored on a different node, so that the data loss condition caused by single-node failure is avoided. In order to ensure that the data can be quickly recovered under the condition of node failure, the data of the storage system is uniformly distributed according to a certain rule, and continuous data seen by a client can be dispersed in the storage system. Compared with the traditional storage, the distributed storage has remarkable improvement in the aspects of high performance, high reliability, easiness in expansion, low cost and the like.
SCSI (Small Computer System Interface) is a complete set of data transfer protocols whose primary function is to transfer request, status, and block data between a host and a storage device. SCSI is a specification for interface transfer between a storage device and a server. The architecture of iSCSI (Internet Small Computer System Interface) defines a provider and a user of storage service as iSCSI Target for converting SCSI requests and data in TCP/IP packets, i.e. a storage service end, called iSCSI Target, abbreviated as Target; and an iSCSI Initiator, namely a storage client, called an iSCSI Initiator, which is abbreviated as Initiator, for generating an SCSI request, and encapsulating the SCSI request and data into a TCP/IP packet and sending the TCP/IP packet to an IP network. The storage server side and the storage client side communicate with each other by using TCP, and the storage server side maps the block storage device into a volume and provides the volume to the storage client side, so that remote block storage service is realized.
An iSCSI Initiator (iSCSI Initiator) is a client device that connects and initiates requests to a server (iSCSI target). An iSCSI target (iSCSI target) is a server component of an iSCSI network that responds to requests from iSCSI initiators. The SCSI gateway (also referred to as tgt for short herein) is a bridge for communication between the client and the storage cluster, supports the iSCSI protocol, and can provide a method for accessing various storage devices; the SCSI gateway is a process that is responsible for parsing SCSI requests from an Initiator and then sending them to the storage cluster.
LUN (logical Unit number) is a name space resource described in SCSI target, one target may include multiple LUNs, and the attributes of each LUN may be different, for example, LUN #0 may be a disk, and LUN # 1 may be other devices. In a distributed storage system, a LUN is a created logical volume. Relationship between Initiator, SCSI gateway, target and LUN, as in FIG. 1. The SCSI gateway maintains a Target list associated with the SCSI gateway, and one SCSI gateway can be associated with a plurality of targets; multiple SCSI gateways and multiple LUNs may be associated under one Target. The Initiator accesses the Target-associated LUN through the SCSI gateway.
The multipath is characterized in that a plurality of paths are used for connecting one host and the storage array end, so that the number of visible paths from the host to the array is larger than one, and the formation of single-point faults is avoided. AA (Active-Active) mode: all paths have the same priority, and the IO is issued through any path. After the Target and the Initiator end establish connection, the path state is not reported, so that the Initiator end considers that the established path states are consistent, that is, each path has the same priority. As in FIG. 2, paths 1-5 and paths 2-7 have the same priority, and paths 3-6 and paths 4-8 have the same priority.
Alua (asymmetric Logical Unit access) asymmetric Logical Unit access: of all paths, only one path is in active/optimized (AO) state, and the other paths are in active/non-optimized (AN) state. The IO will be issued preferentially from the AO path. unavailable state: when the SCSI gateway fails, the path state associated with the SCSI gateway needs to be set as unavailable. If the SCSI gateway supports ALUA, the LUN has a logical home. Referring to FIG. 3, LUN1 belongs to tgt1, then the connection established by Initiator and tgt1 is the optimized (Active/optimized) path, and the connection established by tgt2 is the non-optimized (Active/non-optimized) path. The Initiator end accessing the LUN will take the optimized path first.
In the prior art, a distributed block storage performance optimization method (CN111190546A) based on ALUA and local cache adopts an ALUA multi-path mode to preferentially complete an IO request through an AO path, and specifically includes the following steps, step S1: adding a local cache layer below a Target layer, adding a cluster cache equalizer below the local cache layer, wherein the local cache layer is used for single-machine storage, and the cluster cache equalizer is responsible for collecting cache use details of each node in real time and equalizing cache use of each node as much as possible; step S2: defining the cache utilization rate of each node as Hcr, the cache occupancy of each Target on the node as Tcr, and defining the cache occupancy rate of a single node as T1cr + T2cr + T3cr + … + Tncr; step S3: when a new Target needs to receive client connection each time, the device with the lowest Hcr is calculated to be Hn, the connection attribute of the Target on the Hn is set to be AO, the connection attribute of the Target on other nodes is AN, and local cache service is provided for the Target by preferentially using the node with the most abundant cache. The first prior art has the following defects: (1) the load balance is adjusted by taking Target as a unit, and the adjusted granularity is thicker; when the number of LUNs associated with Target is greatly different, the load balancing effect may be poor; in extreme cases (e.g., only 1 Target, and many LUNs are associated with the Target), load balancing cannot even be achieved; (2) without a fault detection mechanism, when a storage node fails, a path cannot be switched quickly.
In the second prior art, ALUA multipath is implemented by using a distributed mutual exclusion lock (exclusive-lock): mutual exclusion lock taking the logical volume as unit; only one client can have exclusive-lock at the same time; referring to FIG. 4, tgt refers to the SCSI gateway, and try _ lock attempts to acquire a distributed mutually exclusive lock, which fails if it is owned by other tgt. Exclusive-lock principle: and the locker _ owner in the storage cluster records the owner of the current exclusive-lock, acquires the exclusive-lock by a certain tgt, acquires the lock if the locker _ owner is empty or can acquire the lock by the locker, and otherwise, fails to acquire the lock. Meanwhile, the logical volume and the storage cluster have heartbeats, and if a certain logical volume is the owner of the locker owner and the heartbeat times out, the corresponding locker owner is removed. The second prior art has the following defects: according to the scheme, the ALUA path selection is realized by acquiring exclusive-lock, namely the path state of successful acquisition of exclusive-lock is AO, and the path states of failure acquisition of other distributed locks are AN; however, the path selection of ALUA depends on who first preempts to exclusive-lock, and the path state is random, and load balancing cannot be achieved.
Disclosure of Invention
In order to make up for the defects that path selection is not universal and load balancing cannot be realized in the prior art, the invention provides a distributed storage asymmetric logic unit access multipath realization method and system.
The technical problem of the invention is solved by the following technical scheme:
the invention provides a distributed storage asymmetric logic unit access multipath implementation method, which comprises the following steps: s1: the metadata cluster saves the relation views of the SCSI gateway, the target device and the logical volume; s2: the command starts the SCSI gateway, when the target device is associated with the SCSI gateway, or the logical volume is associated with the target device, or the SCSI gateway detects the abnormal path, the SCSI gateway notifies the metadata cluster; instructing the metadata cluster to update corresponding metadata information, and synchronizing the updated information to the SCSI gateway; s3: heartbeat is sent among the SCSI gateways, when the path abnormity is detected, the heartbeat is reported to the metadata cluster, the metadata cluster is instructed to update metadata information after the SCSI gateway is judged to be in fault, and the updated information is synchronized to the SCSI gateway; s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logical volume if the path state sends a change.
In some embodiments, the relational view maintained by the instruction metadata cluster comprises: mapping relation between SCSI gateway and target device, mapping relation between target device and logical volume, and asymmetric logical unit access path state of logical volume.
In some embodiments, the step S2 includes: s21: when the SCSI gateway process is started, informing the metadata cluster to update the corresponding SCSI gateway state in the relation view; s22: when the target device and the logic volume are associated, the metadata cluster is informed to update the corresponding target device and logic volume state and the mapping relation thereof in the relation view, and the metadata cluster is instructed to modify corresponding information; s23: the command initiator queries the state information of the access path of the asymmetric logic unit through a request, and the SCSI gateway returns the state of the access path of the asymmetric logic unit according to the content of the relational view.
In some embodiments, the detecting, by the SCSI gateway, the path failure comprises: a1: selecting an associated SCSI gateway set by the SCSI gateway according to the relation view broadcasted by the metadata cluster; a2: the SCSI gateway sends heartbeat messages to the SCSI gateways in the set at regular time; a3: the SCSI gateway regularly checks whether other SCSI gateways reply heartbeat messages within a time threshold range, and if the SCSI gateways do not reply within a specified time, the SCSI gateway is reported to the metadata cluster to be abnormal; a4: the command metadata cluster receives and records the abnormal information reported by the SCSI gateway; if a plurality of SCSI gateways report the same SCSI gateway, the metadata cluster modifies the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
In some embodiments, when the path accessed by the asymmetric logical unit changes, the processing flow of SCSI gateway path switching includes the following steps: b1: the SCSI gateway analyzes the relation view of metadata cluster broadcasting; b2: if the path state of the logical volume changes, the SCSI gateway modifies the path state of the corresponding logical volume; b3: the SCSI gateway informs the initiator to inquire the path state again by replying the request issued by the initiator.
In some embodiments, the SCSI gateway sends load information of all the logical volumes to the metadata cluster at regular time, and instructs the metadata cluster to adjust the path state of the corresponding logical volume according to the load information of the logical volume, so as to implement load balancing; the method for realizing load balancing comprises the following steps: c1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logical volume; c2: selecting an optimized path by the instruction metadata cluster according to the load information, updating the relation view and pushing the relation view to the SCSI gateway; c3: if the path state of the logical volume changes, the SCSI gateway modifies the path state of the corresponding logical volume; c4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter; c5: and the instruction initiator inquires the path state again and continuously issues the request from the optimized path so as to realize load balancing.
In some embodiments, in the case of the load information, comparing the load amount on the SCSI gateways, and if the difference between the maximum value and the minimum value exceeds a threshold, transferring the attribution of the partial logical volume on the SCSI gateway with the largest load to the SCSI gateway with the smallest load, that is, the path of the SCSI gateway with the largest load becomes a non-optimized path, and the path of the SCSI gateway with the smallest load becomes an optimized path.
The invention also provides a distributed storage asymmetric logic unit access multipath implementation system, which is characterized by comprising the following steps: metadata cluster, SCSI gateway, starter; the metadata cluster is used for maintaining and transmitting information of the storage cluster, processing information reported by the SCSI gateway, modifying a relation view and pushing the relation view to the SCSI gateway; the SCSI gateway is used for supporting the SCSI target device framework of the creation and maintenance of the SCSI target device driving protocol and providing a method for accessing various storage devices; the initiator is used for acquiring a target device list associated with the SCSI gateway and accessing the logical volume associated with the target device by adding access control authority to the target device; the SCSI gateway interacts with the metadata cluster, informs the metadata cluster of changing the relation view, and receives the latest relation view broadcasted by the metadata cluster; the SCSI gateway interacts with the initiator, the SCSI processes the request of the initiator, and the instruction initiator inquires the path state from the SCSI gateway.
In some embodiments, the metadata cluster is constructed based on a distributed consistency algorithm, has distributed strong consistency, and ensures that data of all metadata nodes are always correct and consistent.
In some embodiments, the SCSI gateways detect whether a path failure occurs through heartbeat; if a path fault is detected, reporting update information, fault information and load information to a metadata cluster; and analyzing the relation view pushed by the metadata cluster, and judging whether the path state sends changes.
In some embodiments, the same client may authorize access to multiple targets; the client can access the same logical volume through different SCSI gateways; each SCSI gateway corresponds to an IO access path.
Compared with the prior art, the invention has the advantages that: through metadata cluster management, the mapping relation among SCSI gateways, Target devices (Target) and logical volumes (LUN) is realized, the SCSI can be associated with a plurality of Target devices, one Target device can be associated with a plurality of logical volumes (LUN), one logical volume can be mapped to only one Target device, and one Target device can be mapped to a plurality of SCSI gateways; realizing general path selection of asymmetric logic unit access; the state of the path can be changed artificially as required. The beneficial effects of the invention in some embodiments further include: by using a heartbeat mechanism between SCSI gateways, the rapid detection of path faults is realized; the state of the path is set according to the load information of the logical volume (LUN), and load balancing is effectively realized.
Drawings
FIG. 1 is a diagram illustrating the relationship between an initiator, a SCSI gateway, a target, and a logical volume in the prior art;
FIG. 2 is a diagram of an AA (Active-Active) mode in the prior art;
FIG. 3 is a diagram illustrating an asymmetric logic cell access pattern in the prior art;
FIG. 4 is a schematic diagram of an exclusive-lock principle in the prior art;
FIG. 5 is a schematic diagram of a consistent hashing algorithm according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating an asymmetric logic unit access multipath implementation method according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a fault detection process according to an embodiment of the present invention;
FIG. 8 is a flow chart illustrating a path state switching according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a load balancing process according to an embodiment of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and preferred embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms of orientation such as left, right, up, down, top and bottom in the present embodiment are only relative concepts to each other or are referred to the normal use state of the product, and should not be considered as limiting.
The invention provides a method and a system for realizing distributed storage Asymmetric Logic Unit Access (ALUA) multipath, which solve the problems of asymmetric logic unit access path selection, path fault rapid detection, path switching, load balancing and the like in a distributed storage system.
The invention provides a method for maintaining information (including mapping relation of SCSI gateway and Target, mapping relation of Target and logical volume, and asymmetric logical unit access path state of all logical volumes, etc., which are collectively called relationship view clientmap) of SCSI gateway, Target (Target) and logical volume (LUN) in metadata cluster, so as to realize distributed storage asymmetric logical unit access multi-path state control and switching.
The embodiment of the invention provides a distributed storage asynchronous asymmetric logic unit access multipath implementation method, which comprises the following steps: s1: the instruction metadata cluster saves the relation views of the SCSI gateway, the target device and the logical volume; s2: the command starts the SCSI gateway, when the target device is associated with the SCSI gateway, or the logical volume is associated with the target device, or the SCSI gateway detects that the path has abnormal fault, the SCSI gateway notifies the metadata cluster; instructing the metadata cluster to update corresponding metadata information, and synchronizing the updated information to the SCSI gateway; s3: heartbeat is sent among the SCSI gateways, when abnormal path faults are detected, the heartbeat is reported to a metadata cluster, the metadata cluster is instructed to update metadata information after the SCSI gateway faults are determined to be abnormal, and the updated information is synchronized to the SCSI gateway; s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logical volume if the path state sends a change.
When the SCSI gateway is started, the target device is associated with the SCSI gateway, the logical volume is associated with the target device t and the SCSI gateway detects path faults, the SCSI gateway reports corresponding information to the metadata cluster. The metadata cluster will make corresponding modifications to the relationship view (clientap) and push the modified relationship view to all SCSI gateways. When the Initiator (Initiator) is in inquiry, the SCSI gateway returns a corresponding path state according to the relation view, thereby realizing the distributed storage asymmetric logic unit access multipath.
And under the condition of no load information, adopting a consistent hash algorithm. As shown in fig. 5, all SCSI gateways and logical volumes are mapped onto a hash ring, and the SCSI gateways hash according to the IP and the logical volumes according to uuid. The logical volume is determined to belong to a SCSI gateway by means of clockwise finding. In the case of a large number of logical volumes, the number of logical volumes to which each SCSI gateway belongs will be relatively uniform. Under the condition of load information, firstly selecting a path in a mode of 1; then, comparing the load capacity on the SCSI gateway, if the difference between the maximum value and the minimum value exceeds a threshold value, transferring the attribution of the partial logical volume on the SCSI gateway with the maximum load to the SCSI gateway with the minimum load, namely, the SCSI gateway path with the maximum load becomes a non-optimized path, and the SCSI gateway path with the minimum load becomes an optimized path.
Meanwhile, the SCSI gateway can send load information of all the logical volumes to the metadata cluster at regular time, and the metadata cluster can adjust the path state of the corresponding logical volumes according to the load information of the logical volumes, so that load balancing is achieved. The granularity of the adjustment is at the logical volume level. The granularity of the adjustment of prior art 1 is the target, and there may be multiple logical volumes below the target.
The metadata cluster is a cluster with distributed strong consistency and constructed based on a Paxos algorithm and is mainly responsible for maintaining and transmitting information of the storage cluster; a distributed consistency algorithm based on Paxos and the like can ensure that all metadata nodes are correct and consistent all the time; the metadata cluster is responsible for processing the reported information of the SCSI gateway, modifying the relation view and pushing the relation view to all the SCSI gateways; by utilizing the existing structure of the metadata cluster, the function of newly adding a maintenance relation view in the metadata cluster is added.
The SCSI gateway is a bridge for communication between a client and a storage cluster, supports the creation and maintenance of various SCSI target device driving protocols, and can provide a method for accessing various storage devices. And detecting whether the SCSI gateway fails or not through the heartbeat between the SCSI gateway and the SCSI gateway. If a path fault is detected, reporting fault information to a metadata cluster; and the SCSI gateway reports the update information, the fault information and the load information to the metadata cluster. And analyzing the relation view pushed by the metadata cluster, and judging whether the path state sends changes. The SCSI gateway and the initiator complete interaction through an iSCSI protocol, so that the SCSI gateway has the capability of analyzing and processing the iSCSI protocol. When the SCSI gateway interacts with the initiator, the mapping relation and the asymmetric logic unit access path state in the relation view are returned. The SCSI gateway interacts with the metadata cluster, informs the metadata cluster of changing the relation view, and receives the latest relation view broadcasted by the metadata cluster; the SCSI gateway interacts with the initiator, the SCSI processes the request of the initiator, and the instruction initiator inquires the path state from the SCSI gateway.
The Initiator (Initiator) is a user of distributed storage and interacts with the SCSI gateway through the iSCSI protocol. The list of the target devices associated with the SCSI gateway can be obtained, the logical volume associated with the target devices is accessed by adding access control authority to the target devices, and the same client can authorize access to a plurality of target devices. The client can access the same logical volume through different SCSI gateways, and each SCSI gateway corresponds to an IO access path.
The specific steps of multipath implementation principle, fault detection and load balancing will be described in detail below.
The asymmetric logic unit accesses a multi-path implementation method. The metadata cluster maintains the information of the SCSI gateway, the target and the logical volume (including the mapping relation of the SCSI gateway and the target, the mapping relation of the target and the logical volume, the asymmetric logical unit access path states of all the logical volumes and the like). The SCSI gateway will report the updated information, and the metadata cluster will modify the relation view correspondingly when receiving the updated information. The SCSI gateway returns the state of the asymmetric logic unit accessing the multipath to the initiator all the time according to the information of the relation view. The specific steps are shown in FIG. 6.
The interaction between the SCSI gateway and the metadata cluster is unique to the present invention. In the prior art, the starter and the tgt can be interactive, the starter can initiate inquiry and return all corresponding path information, and the path information of the invention is based on a relational view. The method comprises the following steps: step 1: when the SCSI gateway process is started, the metadata cluster is informed to update and report a message for updating the SCSI gateway state, and the corresponding SCSI gateway state in the relationship view is modified by the metadata cluster; step 2: when the target device and the logic volume are associated, the metadata cluster is informed to update the corresponding target device and logic volume state and the mapping relation thereof in the relation view, and the metadata cluster modifies the corresponding information; and step 3: the initiator queries the state information of the access path of the asynchronous asymmetric logic unit through the inquiry request, and the SCSI gateway returns the state request of the access path of the asynchronous asymmetric logic unit according to the content of the relational view.
And (4) fault detection. The SCSI gateway may fail (network exception, downtime, etc.). Therefore, the SCSI gateway needs to be detected to detect the failure and perform the path switching in time. The fault detection, as shown in fig. 7, includes the following steps: step 1: selecting an associated SCSI gateway set by the SCSI gateway according to the relation view broadcasted by the metadata cluster; step 2: the SCSI gateway sends heartbeat messages to the SCSI gateways in the set at regular time; and step 3: the SCSI gateway regularly checks whether other SCSI gateways reply the heartbeat message within the time threshold range, if the SCSI gateway does not reply within the specified time, the SCSI gateway is reported to the metadata cluster to be abnormal; and 4, step 4: the metadata cluster records the abnormal information reported by the SCSI gateway when receiving the abnormal information; if a plurality of SCSI gateways report the same SCSI gateway, the metadata cluster modifies the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
And switching the path state. Due to a fault or other reasons, the path accessed by the asymmetric logical unit changes, and the path accessed by the asymmetric logical unit may change from three states, namely AO, AN and unavailable. The processing flow of SCSI gateway path switching, as shown in fig. 8; here, only the flow of switching the path is described, and the correspondence relationship between the paths is not described. That is, tgt can sense whether the path state of its associated logical volume changes by analyzing the relationship view. Only if the path state of the logical volume changes from AO to other state is the Unit Attention returned to the initiator, otherwise nothing is done. The method comprises the following steps: step 1: the SCSI gateway analyzes the relation view of metadata cluster broadcasting; step 2: if the path is CHANGED from the AO STATE to other STATEs, a Unit attachment with the type of ASYMMETRIC ACCESS _ STATE _ CHANGED is created, namely if the path STATE of the logical volume is CHANGED, the SCSI gateway modifies the path STATE of the corresponding logical volume; and step 3: instructing the SCSI gateway to inform the starter to inquire the path state again by replying a request issued by the starter; namely, the initiator has IO to the SCSI gateway and returns the created Unit Attention. And the initiator receives the Unit Attention and re-initiates inquiry.
And (4) load balancing. The load balancing is to enable the SCSI gateway to evenly distribute the requests of the clients and avoid the concentration of the client requests on part of the SCSI gateway. The client will preferably issue the request from the AO path, so the state of the A asymmetric logic unit access path can be adjusted to realize load balancing. As in FIG. 9; the method comprises the following steps: step 1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logical volume; step 2: the metadata cluster selects an optimized path according to the load information, updates the relation view and pushes the relation view to the SCSI gateway; and step 3: if the path state of the logical volume changes, the SCSI gateway modifies the path state of the corresponding logical volume; and 4, step 4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter; and 5: and the starter inquires the path state again and continuously issues the request from the optimized path AO to realize load balancing.
The invention realizes the path selection of the access of the asymmetric logic unit through the mapping relation among the SCSI gateway, the target device and the logic volume managed by the metadata cluster; the state of the path can be artificially changed according to the requirement; by using a heartbeat mechanism between SCSI gateways, the rapid detection of path faults is realized; the state of the path is set according to the load information of the logical volume, and load balancing is effectively achieved. Load balancing is realized based on path selection and is a supplement to the path selection; if there is no load balancing, the path selected through the relationship view may be faulty, and the load balancing functions to avoid using the faulty path.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.
Claims (11)
1. A distributed storage asymmetric logic unit access multipath implementation method is characterized by comprising the following steps:
s1: the instruction metadata cluster saves the relation views of the SCSI gateway, the target device and the logical volume;
s2: the command starts the SCSI gateway, when the target device is associated with the SCSI gateway, or the logical volume is associated with the target device, or the SCSI gateway detects the abnormal path, the SCSI gateway notifies the metadata cluster; instructing the metadata cluster to update corresponding metadata information, and synchronizing the updated information to the SCSI gateway;
s3: heartbeat is sent among the SCSI gateways, when the path abnormity is detected, the heartbeat is reported to the metadata cluster, the metadata cluster is instructed to update metadata information after the SCSI gateway is judged to be in fault, and the updated information is synchronized to the SCSI gateway;
s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logical volume if the path state sends a change.
2. The method of claim 1 for implementing distributed storage asymmetric logic unit access multipath, wherein:
the relationship view saved by the instruction metadata cluster comprises: mapping relation between SCSI gateway and target device, mapping relation between target device and logical volume, and asymmetric logical unit access path state of logical volume.
3. The method for asymmetric logic cell access multipath implementation of distributed storage according to claim 1, wherein said step S2 includes:
s21: when the SCSI gateway process is started, informing the metadata cluster to update the corresponding SCSI gateway state in the instruction relation view;
s22: when the target device and the logic volume are associated, the metadata cluster is informed to update the corresponding target device and logic volume state and the mapping relation thereof in the relation view, and the metadata cluster is instructed to modify corresponding information;
s23: the command initiator queries the state information of the access path of the asymmetric logic unit through a request, and the SCSI gateway returns the state of the access path of the asymmetric logic unit according to the content of the relational view.
4. The method of claim 1, wherein the detecting of the path failure by the SCSI gateway comprises:
a1: selecting an associated SCSI gateway set by the SCSI gateway according to the relation view broadcasted by the metadata cluster;
a2: the SCSI gateway sends heartbeat messages to the SCSI gateways in the set at regular time;
a3: the SCSI gateway regularly checks whether other SCSI gateways reply heartbeat messages within a time threshold range, and if the SCSI gateways do not reply within a specified time, the SCSI gateway is reported to the metadata cluster to be abnormal;
a4: the command metadata cluster receives and records the abnormal information reported by the SCSI gateway; and if a plurality of SCSI gateways report the same SCSI gateway, instructing the metadata cluster to modify the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
5. The method for implementing multipath access to asymmetric logic unit for distributed storage according to claim 1, wherein when the path accessed by the asymmetric logic unit changes, the processing flow of SCSI gateway path switching comprises the following steps:
b1: the SCSI gateway analyzes the relation view of metadata cluster broadcasting;
b2: if the path state of the logical volume changes, the SCSI gateway modifies the path state of the corresponding logical volume;
b3: the SCSI gateway informs the initiator to inquire the path state again by replying the request issued by the initiator.
6. The method as claimed in claim 1, wherein the SCSI gateway periodically sends the load information of all the logical volumes to the metadata cluster, and instructs the metadata cluster to adjust the path state of the corresponding logical volume according to the load information of the logical volume, so as to implement load balancing; the method for realizing load balancing comprises the following steps:
c1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logical volume;
c2: selecting an optimized path by the instruction metadata cluster according to the load information, updating the relation view and pushing the relation view to the SCSI gateway;
c3: if the path state of the logical volume changes, the SCSI gateway modifies the path state of the corresponding logical volume;
c4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter;
c5: and the instruction initiator inquires the path state again and continuously issues the request from the optimized path so as to realize load balancing.
7. The method of claim 1 for implementing distributed storage asymmetric logic unit access multipath, wherein: and under the condition of load information, comparing the load capacity on the SCSI gateway, and if the difference value between the maximum value and the minimum value exceeds a threshold value, transferring the attribution of the partial logical volume on the SCSI gateway with the maximum load to the SCSI gateway with the minimum load, namely, the SCSI gateway path with the maximum load becomes a non-optimized path, and the SCSI gateway path with the minimum load becomes an optimized path.
8. A distributed storage asymmetric logic unit access multipath implementation system, comprising: metadata cluster, SCSI gateway, starter;
the metadata cluster is used for maintaining and transmitting information of the storage cluster, processing information reported by the SCSI gateway, modifying a relation view and pushing the relation view to the SCSI gateway;
the SCSI gateway is used for supporting the SCSI target device framework of the creation and maintenance of the SCSI target device driving protocol and providing a method for accessing various storage devices;
the initiator is used for acquiring a target device list associated with the SCSI gateway and accessing the logical volume associated with the target device by adding access control authority to the target device;
the SCSI gateway interacts with the metadata cluster, informs the metadata cluster of changing the relation view, and receives the latest relation view broadcasted by the metadata cluster;
the SCSI gateway interacts with the initiator, the SCSI processes the request of the initiator, and the instruction initiator inquires the path state from the SCSI gateway.
9. The distributed storage asymmetric logic unit access multipath implementation system of claim 8, wherein: the metadata cluster is constructed based on a distributed consistency algorithm, has distributed strong consistency, and ensures that data of all metadata nodes are always correct and consistent.
10. The distributed storage asymmetric logic unit access multipath implementation system of claim 9, wherein: detecting whether path failure occurs between the SCSI gateways through heartbeat; if a path fault is detected, reporting update information, fault information and load information to a metadata cluster; and analyzing the relation view pushed by the metadata cluster, and judging whether the path state sends changes.
11. The distributed storage asymmetric logic unit access multipath implementation system of claim 9, wherein: the same client can authorize to access a plurality of targets; the client can access the same logical volume through different SCSI gateways; each SCSI gateway corresponds to an IO access path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755683.1A CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755683.1A CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113489784A true CN113489784A (en) | 2021-10-08 |
| CN113489784B CN113489784B (en) | 2024-05-17 |
Family
ID=77940856
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110755683.1A Active CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113489784B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114024834A (en) * | 2021-10-21 | 2022-02-08 | 济南浪潮数据技术有限公司 | Fault positioning method and device, electronic equipment and readable storage medium |
| CN114500577A (en) * | 2021-12-27 | 2022-05-13 | 天翼云科技有限公司 | Data access system and data access method |
| CN115794499A (en) * | 2023-02-03 | 2023-03-14 | 创云融达信息技术(天津)股份有限公司 | Method and system for double-live replication of data among distributed block storage clusters |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110145499A1 (en) * | 2009-12-16 | 2011-06-16 | International Business Machines Corporation | Asynchronous file operations in a scalable multi-node file system cache for a remote cluster file system |
| KR20150061316A (en) * | 2013-11-27 | 2015-06-04 | 주식회사 유투앤 | Method and System for load balancing of iSCSI storage system used network distributed file system and method |
| CN109981768A (en) * | 2019-03-21 | 2019-07-05 | 上海霄云信息科技有限公司 | I/o multipath planning method and equipment in distributed network storage system |
| CN110489388A (en) * | 2019-08-16 | 2019-11-22 | 上海霄云信息科技有限公司 | A kind of implementation method and equipment locked for SCSI in distributed network storage system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
| CN112306380A (en) * | 2019-08-01 | 2021-02-02 | 中移(苏州)软件技术有限公司 | Storage management method and device and computer storage medium |
| CN112395264A (en) * | 2020-11-26 | 2021-02-23 | 深圳市杉岩数据技术有限公司 | Processing method for mapping between logical target and volume in distributed storage system |
-
2021
- 2021-07-05 CN CN202110755683.1A patent/CN113489784B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110145499A1 (en) * | 2009-12-16 | 2011-06-16 | International Business Machines Corporation | Asynchronous file operations in a scalable multi-node file system cache for a remote cluster file system |
| KR20150061316A (en) * | 2013-11-27 | 2015-06-04 | 주식회사 유투앤 | Method and System for load balancing of iSCSI storage system used network distributed file system and method |
| CN109981768A (en) * | 2019-03-21 | 2019-07-05 | 上海霄云信息科技有限公司 | I/o multipath planning method and equipment in distributed network storage system |
| CN112306380A (en) * | 2019-08-01 | 2021-02-02 | 中移(苏州)软件技术有限公司 | Storage management method and device and computer storage medium |
| CN110489388A (en) * | 2019-08-16 | 2019-11-22 | 上海霄云信息科技有限公司 | A kind of implementation method and equipment locked for SCSI in distributed network storage system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
| CN112395264A (en) * | 2020-11-26 | 2021-02-23 | 深圳市杉岩数据技术有限公司 | Processing method for mapping between logical target and volume in distributed storage system |
Non-Patent Citations (1)
| Title |
|---|
| 余胜生, 王国华, 周敬利: "Linux下iSCSI存储系统中逻辑卷管理的设计与实现", 计算机工程, no. 11, 5 November 2004 (2004-11-05) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114024834A (en) * | 2021-10-21 | 2022-02-08 | 济南浪潮数据技术有限公司 | Fault positioning method and device, electronic equipment and readable storage medium |
| CN114500577A (en) * | 2021-12-27 | 2022-05-13 | 天翼云科技有限公司 | Data access system and data access method |
| CN115794499A (en) * | 2023-02-03 | 2023-03-14 | 创云融达信息技术(天津)股份有限公司 | Method and system for double-live replication of data among distributed block storage clusters |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113489784B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11445019B2 (en) | Methods, systems, and media for providing distributed database access during a network split | |
| CN113489784B (en) | Method and system for realizing multipath access of asymmetric logic unit of distributed storage | |
| US10185497B2 (en) | Cluster federation and trust in a cloud environment | |
| US9560093B2 (en) | Higher efficiency storage replication using compression | |
| CA2938768C (en) | Geographically-distributed file system using coordinated namespace replication | |
| US10530855B2 (en) | Lock state synchronization for non-disruptive persistent operation | |
| KR100491541B1 (en) | A contents synchronization system in network environment and a method therefor | |
| US9734199B1 (en) | Data replication framework | |
| KR101544480B1 (en) | Distribution storage system having plural proxy servers, distributive management method thereof, and computer-readable recording medium | |
| WO2016070375A1 (en) | Distributed storage replication system and method | |
| CN111124301A (en) | Data consistency storage method and system of object storage device | |
| WO2014201982A1 (en) | Data synchronization method and device, and distributed system | |
| CA2922665A1 (en) | Distributed file system using consensus nodes | |
| WO2013046352A1 (en) | Computer system, data management method and data management program | |
| US10198492B1 (en) | Data replication framework | |
| CN108055333A (en) | A kind of NAS-CIFS cluster load balancing methods based on UFS | |
| EP3570169B1 (en) | Method and system for processing device failure | |
| US20230205638A1 (en) | Active-active storage system and data processing method thereof | |
| CN115396302B (en) | Multi-node high-availability configuration distribution system and working method thereof | |
| Shi et al. | Sandstone: A dht based carrier grade distributed storage system | |
| CN116707951A (en) | Network security method, system, equipment and medium based on multi-access edge computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |