CN113489784B - Method and system for realizing multipath access of asymmetric logic unit of distributed storage - Google Patents
Method and system for realizing multipath access of asymmetric logic unit of distributed storage Download PDFInfo
- Publication number
- CN113489784B CN113489784B CN202110755683.1A CN202110755683A CN113489784B CN 113489784 B CN113489784 B CN 113489784B CN 202110755683 A CN202110755683 A CN 202110755683A CN 113489784 B CN113489784 B CN 113489784B
- Authority
- CN
- China
- Prior art keywords
- scsi
- gateway
- path
- information
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000001360 synchronised effect Effects 0.000 claims abstract description 8
- 230000005856 abnormality Effects 0.000 claims abstract description 4
- 239000003999 initiator Substances 0.000 claims description 40
- 238000013507 mapping Methods 0.000 claims description 14
- 230000002159 abnormal effect Effects 0.000 claims description 13
- 239000007858 starting material Substances 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 101100371648 Caenorhabditis elegans usp-14 gene Proteins 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000007480 spreading Effects 0.000 description 2
- 101150090882 tgt-1 gene Proteins 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 101150041549 tgt-2 gene Proteins 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a system for realizing multipath access of asymmetric logic units in distributed storage, wherein the method comprises the following steps: the data cluster stores the relationship view of the SCSI gateway, the target and the logical volume; starting a SCSI gateway, and notifying the metadata cluster when abnormality occurs to a target device associated SCSI gateway or a logic volume associated target device or a SCSI gateway detection path; the metadata cluster updates corresponding metadata information and synchronizes the updated information to the SCSI gateway; heartbeat is mutually sent between SCSI gateways, when path abnormality is detected and reported to a metadata cluster, the metadata cluster updates metadata information after deciding the SCSI gateway fault, and the updated information is synchronized to the SCSI gateway; and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logic volume if the path state is changed. The invention realizes the path selection of the asymmetric logic unit by managing the relationship view of the SCSI gateway, the target and the logic volume through the metadata cluster.
Description
Technical Field
The present invention relates to the field of distributed storage, and in particular, to a method and system for implementing Asymmetric Logical Unit Access (ALUA) multipath in distributed storage.
Background
Distributed storage systems use multiple independent servers to cluster to store data. Each server in the cluster acts as a node on which processes run to manage the physical disks on that server. In order to ensure high reliability of data, the data is copied into a plurality of identical copies, each copy is stored on different nodes, and the condition of data loss caused by single node faults is avoided. In order to ensure that the data of the storage system can be quickly recovered under the condition of node faults, the data of the storage system are uniformly distributed according to a certain rule, and continuous data seen by a client are dispersed in the storage system. Distributed storage has significantly improved performance, reliability, ease of expansion, low cost, and the like over conventional storage.
SCSI (Small Computer SYSTEM INTERFACE) is a complete set of data transfer protocols whose primary function is to transfer requests, status and block data between a host and a storage device. SCSI is a specification of the interface transfer between a storage device and a server. The architecture of iSCSI (INTERNET SMALL Computer SYSTEM INTERFACE ) defines a provider and a consumer of storage services as ISCSI TARGET for converting SCSI requests and data in TCP/IP packets, respectively, i.e., a storage server, referred to as an iSCSI target, abbreviated as target; and an iSCSI Initiator, i.e., a storage client, called an iSCSI Initiator, abbreviated as Initiator, for generating the SCSI request and encapsulating the SCSI request and data into TCP/IP packets for transmission to the IP network. And the storage server side and the storage client side communicate by using TCP, and the storage server side maps the block storage equipment into volumes to be provided for the storage client side, so that the remote block storage service is realized.
An iSCSI Initiator (iSCSI Initiator) is a client device that is used to connect and initiate requests to a server (iSCSI target). The iSCSI target (ISCSI TARGET) is a server component of the iSCSI network for responding to requests from iSCSI initiators. SCSI gateway (also referred to herein as tgt) is a bridge for communication between clients and storage clusters, supports iSCSI protocol, and can provide a method for accessing various storage devices; the SCSI gateway is a process responsible for resolving the SCSI request of the Initiator and then sending to the storage cluster.
LUN (Logical Unit Number) is a namespace resource described in SCSI target, one target may include multiple LUNs, and the attributes of each LUN may be differentiated, e.g., LUN #0 may be a disk, and LUN #1 may be another device. In a distributed storage system, a LUN is a logical volume created. The relationship between Initiator, SCSI gateway, target, and LUN, as in FIG. 1. The SCSI gateway maintains a Target list associated with the SCSI gateway, and one SCSI gateway can associate a plurality of targets; multiple SCSI gateways and multiple LUNs may be associated at one Target. The Initiator accesses the Target-associated LUN through the SCSI gateway.
Multiple paths are used to connect a host and a storage array, so that the number of visible paths from the host to the array is greater than one, and single-point faults are avoided. AA (Active-Active) mode: all paths have the same priority, and IOs are issued through any path. After the Target and Initiator terminals establish connection, the path state will not be reported, so that the Initiator terminal can consider that the established path states are consistent, i.e. each path has the same priority. As in FIG. 2, paths 1-5 and paths 2-7 have the same priority, and paths 3-6 and paths 4-8 have the same priority.
ALUA (Asymmetric Logical Unit Access) asymmetric logical unit access: of all paths, only one path is in AN active/optimized (AO) state and the other paths are in AN active/non-optimized (AN) state. The IO will be issued preferentially from the AO path. unavialable state: when the SCSI gateway fails, all associated path states of the SCSI gateway need to be set to unavailable. If the SCSI gateway supports ALUA, the LUN will have a logical home. As in FIG. 3, LUN1 is attributed to tgt1, then the connection established by the Initiator and tgt1 is an optimized (Active/optimized) path, and the connection established by tgt2 is a non-optimized (Active/non-optimized) path. An Initiator side accesses a LUN to take priority over an optimized path.
In the first prior art, a distributed block storage performance optimization method (CN 111190546 a) based on ALUA and a local cache adopts an ALUA multipath mode, and preferentially completes an IO request through an AO path, specifically including the following steps, step S1: adding a local buffer layer below a Target layer, and adding a cluster buffer equalizer below the local buffer layer, wherein the local buffer layer is used for single-machine storage, and the cluster buffer equalizer is responsible for collecting the buffer use details of each node in real time and balancing the buffer utilization of each node as much as possible; step S2: defining the buffer utilization rate of each node as Hcr, and the buffer occupation rate of each Target on the node as Tcr, wherein the buffer occupation rate of a single node is hcr=t1cr+t2cr+t3cr+ … + Tncr; step S3: and when a new Target needs to accept the connection of the client, calculating that the device with the lowest Hcr is Hn, setting the connection attribute of the Target on the Hn as AO, setting the connection attribute of the Target on other nodes as AN, and realizing that the node with the most abundant cache is preferentially used to provide local cache service for the Target. Drawbacks of the first prior art: (1) The load balancing is regulated by taking a Target as a unit, and the granularity of regulation is thicker; when the number of Target-associated LUNs varies widely, the load balancing effect may be poor; in extreme cases (e.g., only 1 Target, and the Target is associated with many LUNs), load balancing may not even be achieved; (2) Without a failure detection mechanism, paths cannot be switched quickly when a storage node fails.
In the second prior art, ALUA multipath is realized by using a distributed mutual exclusion lock (exclusive-lock): mutex lock in logical volume units; only one client can have an exclusive-lock at the same time; referring to FIG. 4, tgt refers to a SCSI gateway, try_lock is an attempt to acquire a distributed exclusive lock, and if the distributed lock is occupied by other tgt at this time, the acquisition of the lock fails. Principle of exclusive-lock: locker _wner in the storage cluster records the owner of the current exclusive-lock, a certain tgt obtains the exclusive-lock, if locker _wner is empty or is self, the lock can be obtained, otherwise, the lock obtaining fails. Meanwhile, the logical volumes and storage clusters have heartbeats, and if a logical volume is the owner of locker _wner and the heartbeats time out, the corresponding locker _wner is removed. Drawbacks of the second prior art: ALUA path selection is realized by acquiring AN exclusive-lock, namely, the successful path state of the exclusive-lock is acquired as AO, and the failed path state of other acquired distributed locks is acquired as AN; however, path selection by ALUA depends on who preempts to the exclusive-lock, and the path state has randomness, so that load balancing cannot be realized.
Disclosure of Invention
In order to overcome the defects that the prior art is not universal in path selection and cannot realize load balancing, the invention provides a method and a system for realizing multi-path access of a distributed storage asymmetric logic unit.
The technical problems of the invention are solved by the following technical scheme:
The invention provides a distributed storage asymmetric logic unit access multipath implementation method, which comprises the following steps: s1: the metadata cluster stores the relationship view of the SCSI gateway, the target and the logical volume; s2: the instruction starts a SCSI gateway, and when the target is associated with the SCSI gateway or the logic volume is associated with the target or the SCSI gateway detects that the path is abnormal, the SCSI gateway informs the metadata cluster; the instruction metadata cluster updates corresponding metadata information and synchronizes the updated information to the SCSI gateway; s3: heartbeat is sent between SCSI gateways, when path abnormality is detected, the heartbeat is reported to a metadata cluster, the metadata cluster is instructed to update metadata information after deciding the SCSI gateway fault, and the updated information is synchronized to the SCSI gateway; s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logic volume if the path state is changed.
In some embodiments, the relational view of the instruction metadata cluster save includes: mapping relation between SCSI gateway and target, mapping relation between target and logic volume, and asymmetric logic unit access path state of logic volume.
In some embodiments, the step S2 includes: s21: when the SCSI gateway process is started, notifying the corresponding SCSI gateway state in the metadata cluster update relation view; s22: when the target device and the associated logical volume are associated, the state of the corresponding target device and the logical volume and the mapping relation thereof in the metadata cluster update relation view are notified, and the metadata cluster is instructed to modify corresponding information; s23: the instruction initiator inquires the access path state information of the asymmetric logic unit by requesting, and the SCSI gateway returns the access path state of the asymmetric logic unit according to the content of the relation view.
In some embodiments, the detecting a path failure by the SCSI gateway comprises: a1: the SCSI gateway selects an associated SCSI gateway set according to the relation view of the metadata cluster broadcast; a2: the SCSI gateway sends heartbeat information to the SCSI gateway in the collection at regular time; a3: the SCSI gateway periodically checks whether other SCSI gateways reply heartbeat messages within a time threshold range, and if the SCSI gateway does not reply within a specified time, the SCSI gateway is reported to the metadata cluster to be abnormal; a4: the instruction metadata cluster receives and records the abnormal information reported by the SCSI gateway; if there are multiple SCSI gateways reporting the same SCSI gateway, the metadata cluster modifies the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
In some embodiments, when the path accessed by the asymmetric logical unit changes, the processing flow of the SCSI gateway path switching includes the following steps: b1: the SCSI gateway analyzes the relation view of the metadata cluster broadcast; b2: if the path state of the logic volume changes, the SCSI gateway modifies the path state of the corresponding logic volume; b3: the SCSI gateway informs the initiator to re-inquire the path state by replying to the request issued by the initiator.
In some embodiments, the SCSI gateway periodically sends load information of all logical volumes to the metadata cluster, and instructs the metadata cluster to adjust path states of corresponding logical volumes according to the load information of the logical volumes so as to realize load balancing; the load balancing method comprises the following steps: c1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logic volume; c2: the instruction metadata cluster selects an optimized path according to the load information, updates the relation view and pushes the relation view to the SCSI gateway; and C3: if the path state of the logic volume changes, the SCSI gateway modifies the path state of the corresponding logic volume; and C4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter; c5: the instruction starter inquires the path state again, and continuously issues the request from the optimized path so as to realize load balancing.
In some embodiments, under the condition of load information, the load capacity on the SCSI gateway is compared, if the difference value between the maximum value and the minimum value exceeds a threshold value, the attribution of part of logic volumes on the SCSI gateway with the maximum load is transferred to the SCSI gateway with the minimum load, namely, the SCSI gateway path with the maximum load becomes a non-optimized path, and the SCSI gateway path with the minimum load becomes an optimized path.
The invention also provides an asymmetric logic unit access multipath implementation system for distributed storage, which is characterized by comprising the following steps: metadata clusters, SCSI gateways, and initiators; the metadata cluster is used for maintaining and spreading information of the storage cluster, processing information reported by a SCSI gateway, modifying a relation view and pushing the relation view to the SCSI gateway; the SCSI gateway is used for supporting the creation and maintenance of a SCSI target device driving protocol and providing a method for accessing various storage devices; the initiator is used for acquiring a target list associated with the SCSI gateway and accessing the associated logical volume under the target by adding access control authority to the target; the SCSI gateway interacts with the metadata clusters, and informs the metadata clusters to change the relation view and receives the latest relation view broadcasted by the metadata clusters; the SCSI gateway interacts with the initiator, and the SCSI processes the initiator request, and instructs the initiator to inquire the path state from the SCSI gateway.
In some embodiments, the metadata clusters are constructed based on a distributed consistency algorithm, have distributed strong consistency, and ensure that the data of all metadata nodes are always correct and consistent.
In some embodiments, the SCSI gateways detect whether path faults occur through heartbeats; if a path fault is detected, reporting update information, fault information and load information to the metadata cluster; and analyzing the relation view pushed by the metadata cluster, and judging whether the path state changes or not.
In some embodiments, the same client may authorize access to multiple targets; the client can access the same logical volume through different SCSI gateways; each SCSI gateway corresponds to an IO access path.
Compared with the prior art, the invention has the beneficial effects that: through the mapping relation among the metadata cluster management SCSI gateway, the Target and the logical volume (LUN), the SCSI can be associated with a plurality of targets, one Target can be associated with a plurality of logical volumes (LUNs), one logical volume can be mapped to only one Target, and one Target can be mapped to a plurality of SCSI gateways; realizing general path selection of asymmetric logic unit access; the state of the path can be manually changed according to the requirement. The beneficial effects of the invention in some embodiments also include: the heartbeat mechanism between SCSI gateways is used to realize the rapid detection of path faults; the state of the path is set according to the load information of the logical volume (LUN), so that load balancing is effectively realized.
Drawings
FIG. 1 is a schematic diagram of the relationship between initiator, SCSI gateway, target and logical volumes in the prior art;
FIG. 2 is a schematic diagram of an AA (Active-Active) mode in the prior art;
FIG. 3 is a schematic diagram of an asymmetric logical unit access pattern in the prior art;
FIG. 4 is a schematic diagram of the prior art exclusive-lock principle;
FIG. 5 is a schematic diagram of a consistent hashing algorithm in accordance with an embodiment of the present invention;
FIG. 6 is a flow chart of an implementation method of an asymmetric logical unit access multipath according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a fault detection flow in accordance with an embodiment of the present invention;
FIG. 8 is a schematic diagram of a path state switching flow according to an embodiment of the present invention;
Fig. 9 is a schematic diagram of a load balancing flow according to an embodiment of the present invention.
Detailed Description
The application will be further described with reference to the following drawings in conjunction with the preferred embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
It should be noted that, in this embodiment, the terms of left, right, upper, lower, top, bottom, etc. are merely relative terms, or refer to the normal use state of the product, and should not be considered as limiting.
The embodiment of the invention discloses a method and a system for realizing multiple paths of Asymmetric Logical Unit Access (ALUA) of distributed storage, which solve the problems of asymmetric logical unit access path selection, path fault rapid detection, path switching, load balancing and the like in a distributed storage system.
The invention provides a method for maintaining information of a SCSI gateway, a Target and a logical volume (LUN) (comprising mapping relation of the SCSI gateway and the Target, mapping relation of the Target and the logical volume, asymmetric logical unit access path states of all the logical volumes and the like, which are collectively called as a relation view clientmap) in a metadata cluster so as to realize control and switching of asymmetric logical unit access multipath path states of distributed storage.
The invention discloses a multipath implementation method for asynchronous asymmetric logic unit access of distributed storage, which comprises the following steps: s1: the instruction metadata cluster stores a relationship view of the SCSI gateway, the target and the logical volume; s2: the instruction starts the SCSI gateway, and when the target device is related to the SCSI gateway or the logic volume is related to the target device or the SCSI gateway detects that the path has abnormal faults, the SCSI gateway informs the metadata cluster; the instruction metadata cluster updates corresponding metadata information and synchronizes the updated information to the SCSI gateway; s3: heartbeat is sent between SCSI gateways, when path abnormal faults are detected, the heartbeat is reported to a metadata cluster, the metadata cluster is instructed to update metadata information after the SCSI gateway faults are abnormal, and the updated information is synchronized to the SCSI gateway; s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logic volume if the path state is changed.
When the SCSI gateway is started, the target device is associated with the SCSI gateway, the logic volume is associated with the target device t and the SCSI gateway detects the path fault, the SCSI gateway can report corresponding information to the metadata cluster. The metadata cluster will make corresponding modifications to the relationship view (clientmap) and push the modified relationship view to all SCSI gateways. And when the Initiator (Initiator) is in the requirement, the SCSI gateway returns a corresponding path state according to the relation view, so that the distributed storage asymmetric logic unit access multipath is realized.
And under the condition of no load information, adopting a consistent hash algorithm. As shown in fig. 5, all SCSI gateways and logical volumes are mapped onto one hash ring, and the SCSI gateway hashes according to IP and logical volumes according to uuid. The logical volume is determined to be attached to a certain SCSI gateway by looking in a clockwise direction. In the case of a relatively large number of logical volumes, the number of logical volumes to which each SCSI gateway belongs will be relatively uniform. Under the condition of load information, selecting a path according to a mode of 1; and then, comparing the load capacity on the SCSI gateway, if the difference value between the maximum value and the minimum value exceeds a threshold value, transferring the attribution of part of logic volumes on the SCSI gateway with the maximum load to the SCSI gateway with the minimum load, namely changing the SCSI gateway path with the maximum load into a non-optimized path, and changing the SCSI gateway path with the minimum load into an optimized path.
Meanwhile, the SCSI gateway can send the load information of all the logic volumes to the metadata clusters at regular time, and the metadata clusters can adjust the path states of the corresponding logic volumes according to the load information of the logic volumes, so that load balancing is realized. The granularity of adjustment is at the logical volume level. The granularity of adjustment of prior art 1 is a target under which there may be multiple logical volumes.
The metadata cluster is a cluster with distributed strong consistency constructed based on a Paxos algorithm and is mainly responsible for maintaining and spreading information of a storage cluster; distributed consistency algorithms based on, for example, paxos, etc., can ensure that all metadata nodes are always correct and consistent; the metadata cluster is responsible for processing the reported information of the SCSI gateway, modifying the relation view and pushing the relation view to all the SCSI gateways; the existing architecture of the metadata cluster is utilized, and the function of maintaining the relation view is newly added to the metadata cluster.
The SCSI gateway is a bridge for communication between the client and the storage cluster, supports the creation and maintenance of various SCSI target drive protocols, and can provide a method for accessing various storage devices. And detecting whether the SCSI gateway has faults or not through heartbeat between the SCSI gateway and the SCSI gateway. If a path fault is detected, fault information is reported to the metadata cluster; the SCSI gateway reports the update information, fault information and load information to the metadata cluster. And analyzing the relation view pushed by the metadata cluster, and judging whether the path state changes or not. Interaction is completed between the SCSI gateway and the initiator through the iSCSI protocol, so that the SCSI gateway has the capability of analyzing and processing the iSCSI protocol. When the SCSI gateway interacts with the initiator, the mapping relation and the asymmetric logic unit access path state in the relation view are returned. The SCSI gateway interacts with the metadata clusters, and informs the metadata clusters to change the relation view and receives the latest relation view broadcasted by the metadata clusters; the SCSI gateway interacts with the initiator, and the SCSI processes the initiator request, and instructs the initiator to inquire the path state from the SCSI gateway.
An Initiator (Initiator), which is a consumer of distributed storage, interacts with the SCSI gateway through the iSCSI protocol. A list of targets associated with the SCSI gateway may be obtained, access to the logical volumes associated under the targets may be performed by adding access control rights to the targets, and the same client may grant access to multiple targets. The client can access the same logical volume through different SCSI gateways, and each SCSI gateway corresponds to one IO access path.
The specific steps of the multipath implementation principle, fault detection and load balancing will be described in detail below.
An asymmetric logic unit access multipath implementation method. The metadata cluster maintains information of the SCSI gateway, the target, and the logical volumes (including mapping relationship of the SCSI gateway and the target, mapping relationship of the target and the logical volumes, and asymmetric logical unit access path states of all the logical volumes, etc.). The SCSI gateway reports the updated information, and the metadata cluster receives the updated information and modifies the relational view accordingly. And the SCSI gateway always returns the state of the asymmetric logic unit access multipath to the initiator according to the information of the relation view. The specific steps are shown in fig. 6.
Interactions between the SCSI gateway and the metadata cluster are unique to the present invention. In the prior art, the initiator and tgt can interact, the initiator can initiate the query and return all corresponding path information, and the path information of the invention is according to the relation view. The method comprises the following steps: step 1: when the SCSI gateway process is started, notifying a message for updating the SCSI gateway state to be updated to the metadata cluster, and instructing the metadata cluster to modify the corresponding SCSI gateway state in the relational view; step 2: when the target device and the associated logical volume are associated, the state and the mapping relation of the corresponding target device and the logical volume in the metadata cluster update relation view are notified, and the metadata cluster modifies corresponding information; step 3: the initiator inquires the access path state information of the asynchronous asymmetric logic unit through the requirement request, and the SCSI gateway returns the access path state requirement of the asynchronous asymmetric logic unit according to the content of the relation view.
And (5) fault detection. The SCSI gateway may fail (network anomalies, downtime, etc.). Therefore, it is necessary to detect the SCSI gateway in order to discover the failure and switch paths in time. Fault detection, as in fig. 7, includes the steps of: step 1: the SCSI gateway selects an associated SCSI gateway set according to the relation view of the metadata cluster broadcast; step 2: the SCSI gateway sends heartbeat information to the SCSI gateway in the collection at regular time; step 3: the SCSI gateway periodically checks whether other SCSI gateways reply heartbeat messages within a time threshold range, and if the SCSI gateway does not reply within a specified time, the SCSI gateway is reported to the metadata cluster to be abnormal; step 4: the metadata cluster receives the abnormal information reported by the SCSI gateway and records the abnormal information; if there are multiple SCSI gateways reporting the same SCSI gateway, the metadata cluster modifies the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
And switching the path state. The path of asymmetric logical unit access may change from among the three states AO, AN and unavialable due to a failure or other cause. The process flow of SCSI gateway path switching is shown in FIG. 8; here, the flow of switching paths is merely a flow, and the correspondence relationship between paths is not involved. That is, tgt can sense whether the path state of the relevant logical volume changes or not by analyzing the relation view. Only the path state of the logical volume changes from AO to other state returns the Unit Attention to the initiator, otherwise it is not processed. The method comprises the following steps: step 1: the SCSI gateway analyzes the relation view of the metadata cluster broadcast; step 2: if the path changes from the AO STATE to other STATEs, creating a Unit Attention of ASYMMETRIC ACCESS _STATE_CHANGED type, i.e. if the path STATE of the logical volume changes, the SCSI gateway modifies the path STATE of the corresponding logical volume; step 3: the instruction SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter; i.e. the initiator has IO to SCSI gateway, return the created Unit Attention. The starter receives the Unit Attention and reinitiates the requirement.
Load balancing. The load balancing is to uniformly distribute the requests of clients by the SCSI gateway, so that the client requests are prevented from being concentrated on part of the SCSI gateway. The client may issue requests from the AO path preferentially, so the state of the a asymmetric logical unit access path may be adjusted to achieve load balancing. As in fig. 9; the method comprises the following steps: step 1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logic volume; step 2: the metadata cluster selects an optimized path according to the load information, updates the relation view and pushes the relation view to the SCSI gateway; step 3: if the path state of the logic volume changes, the SCSI gateway modifies the path state of the corresponding logic volume; step 4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter; step 5: the starter inquires the path state again, and continuously issues a request from the optimized path AO to realize load balancing.
The invention realizes the path selection of the asymmetric logical unit access by managing the mapping relation among the SCSI gateway, the target and the logical volume through the metadata cluster; the state of the path can be manually changed according to the requirement; the heartbeat mechanism between SCSI gateways is used to realize the rapid detection of path faults; the state of the path is set according to the load information of the logic volume, so that load balancing is effectively realized. The load balancing is realized based on path selection, and is a supplement to the path selection; if there is no load balancing, the paths selected through the relationship view may be faulty, and the load balancing serves to avoid using faulty paths.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several equivalent substitutions and obvious modifications can be made without departing from the spirit of the invention, and the same should be considered to be within the scope of the invention.
Claims (7)
1. The method for realizing the multipath access of the asymmetric logic unit of the distributed storage is characterized by comprising the following steps:
s1: the instruction metadata cluster stores a relationship view of the SCSI gateway, the target and the logical volume;
S2: the instruction starts a SCSI gateway, and when the target is associated with the SCSI gateway or the logic volume is associated with the target or the SCSI gateway detects that the path is abnormal, the SCSI gateway informs the metadata cluster; the instruction metadata cluster updates corresponding metadata information and synchronizes the updated information to the SCSI gateway;
s3: heartbeat is sent between SCSI gateways, when path abnormality is detected, the heartbeat is reported to a metadata cluster, the metadata cluster is instructed to update metadata information after deciding the SCSI gateway fault, and the updated information is synchronized to the SCSI gateway;
s4: and the SCSI gateway receives the synchronous information, and modifies the path state of the corresponding logic volume if the path state is changed.
2. The asymmetric logical unit access multipath implementation method of distributed storage as claimed in claim 1, wherein:
The relational view stored by the instruction metadata cluster comprises: mapping relation between SCSI gateway and target, mapping relation between target and logic volume, and asymmetric logic unit access path state of logic volume.
3. The method for implementing multiple paths of asymmetric logical unit access for distributed storage as claimed in claim 1, wherein said step S2 comprises:
s21: when the SCSI gateway process is started, notifying the corresponding SCSI gateway state in the metadata cluster update instruction relationship view;
s22: when the target device and the associated logical volume are associated, the state of the corresponding target device and the logical volume and the mapping relation thereof in the metadata cluster update relation view are notified, and the metadata cluster is instructed to modify corresponding information;
S23: the instruction initiator inquires the access path state information of the asymmetric logic unit by requesting, and the SCSI gateway returns the access path state of the asymmetric logic unit according to the content of the relation view.
4. The method of claim 1, wherein the detecting, by the SCSI gateway, that the path is abnormal comprises:
a1: the SCSI gateway selects an associated SCSI gateway set according to the relation view of the metadata cluster broadcast;
a2: the SCSI gateway sends heartbeat information to the SCSI gateway in the collection at regular time;
A3: the SCSI gateway periodically checks whether other SCSI gateways reply heartbeat messages within a time threshold range, and if the SCSI gateway does not reply within a specified time, the SCSI gateway is reported to the metadata cluster to be abnormal;
A4: the instruction metadata cluster receives and records the abnormal information reported by the SCSI gateway; if there are multiple SCSI gateways reporting the same SCSI gateway, then the instruction metadata cluster modifies the state of the SCSI gateway and the path state of the associated logical volume in the relational view.
5. The method for implementing multiple paths of asymmetric logical unit access in distributed storage as claimed in claim 1, wherein when the paths of asymmetric logical unit access change, the process flow of path switching of SCSI gateway includes the steps of:
B1: the SCSI gateway analyzes the relation view of the metadata cluster broadcast;
B2: if the path state of the logic volume changes, the SCSI gateway modifies the path state of the corresponding logic volume;
b3: the SCSI gateway informs the initiator to re-inquire the path state by replying to the request issued by the initiator.
6. The method for implementing multiple paths of asymmetric logical unit access of distributed storage according to claim 1, wherein the SCSI gateway periodically transmits load information of all logical volumes to the metadata cluster, and instructs the metadata cluster to adjust path states of corresponding logical volumes according to the load information of the logical volumes so as to implement load balancing; the load balancing method comprises the following steps:
c1: the SCSI gateway reports the load information of each associated logical volume at regular time; the load information comprises the IO number, the data flow and the IO processing time delay of the logic volume;
C2: the instruction metadata cluster selects an optimized path according to the load information, updates the relation view and pushes the relation view to the SCSI gateway;
and C3: if the path state of the logic volume changes, the SCSI gateway modifies the path state of the corresponding logic volume;
And C4: the SCSI gateway informs the starter to inquire the path state again by replying the request issued by the starter;
C5: the instruction starter inquires the path state again, and continuously issues the request from the optimized path so as to realize load balancing.
7. The asymmetric logical unit access multipath implementation method of distributed storage as claimed in claim 1, wherein: under the condition of load information, the load capacity on the SCSI gateway is compared, if the difference value between the maximum value and the minimum value exceeds a threshold value, the attribution of partial logic volumes on the SCSI gateway with the maximum load is transferred to the SCSI gateway with the minimum load, namely, the SCSI gateway path with the maximum load becomes a non-optimized path, and the SCSI gateway path with the minimum load becomes an optimized path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755683.1A CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110755683.1A CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113489784A CN113489784A (en) | 2021-10-08 |
| CN113489784B true CN113489784B (en) | 2024-05-17 |
Family
ID=77940856
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110755683.1A Active CN113489784B (en) | 2021-07-05 | 2021-07-05 | Method and system for realizing multipath access of asymmetric logic unit of distributed storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113489784B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114024834A (en) * | 2021-10-21 | 2022-02-08 | 济南浪潮数据技术有限公司 | Fault positioning method and device, electronic equipment and readable storage medium |
| CN114500577A (en) * | 2021-12-27 | 2022-05-13 | 天翼云科技有限公司 | Data access system and data access method |
| CN115794499B (en) * | 2023-02-03 | 2023-05-16 | 创云融达信息技术(天津)股份有限公司 | Method and system for dual-activity replication data among distributed block storage clusters |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20150061316A (en) * | 2013-11-27 | 2015-06-04 | 주식회사 유투앤 | Method and System for load balancing of iSCSI storage system used network distributed file system and method |
| CN109981768A (en) * | 2019-03-21 | 2019-07-05 | 上海霄云信息科技有限公司 | I/o multipath planning method and equipment in distributed network storage system |
| CN110489388A (en) * | 2019-08-16 | 2019-11-22 | 上海霄云信息科技有限公司 | A kind of implementation method and equipment locked for SCSI in distributed network storage system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
| CN112306380A (en) * | 2019-08-01 | 2021-02-02 | 中移(苏州)软件技术有限公司 | Storage management method and device and computer storage medium |
| CN112395264A (en) * | 2020-11-26 | 2021-02-23 | 深圳市杉岩数据技术有限公司 | Processing method for mapping between logical target and volume in distributed storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8495250B2 (en) * | 2009-12-16 | 2013-07-23 | International Business Machines Corporation | Asynchronous file operations in a scalable multi-node file system cache for a remote cluster file system |
-
2021
- 2021-07-05 CN CN202110755683.1A patent/CN113489784B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20150061316A (en) * | 2013-11-27 | 2015-06-04 | 주식회사 유투앤 | Method and System for load balancing of iSCSI storage system used network distributed file system and method |
| CN109981768A (en) * | 2019-03-21 | 2019-07-05 | 上海霄云信息科技有限公司 | I/o multipath planning method and equipment in distributed network storage system |
| CN112306380A (en) * | 2019-08-01 | 2021-02-02 | 中移(苏州)软件技术有限公司 | Storage management method and device and computer storage medium |
| CN110489388A (en) * | 2019-08-16 | 2019-11-22 | 上海霄云信息科技有限公司 | A kind of implementation method and equipment locked for SCSI in distributed network storage system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
| CN112395264A (en) * | 2020-11-26 | 2021-02-23 | 深圳市杉岩数据技术有限公司 | Processing method for mapping between logical target and volume in distributed storage system |
Non-Patent Citations (1)
| Title |
|---|
| Linux下iSCSI存储系统中逻辑卷管理的设计与实现;余胜生, 王国华, 周敬利;计算机工程;20041105(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113489784A (en) | 2021-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113489784B (en) | Method and system for realizing multipath access of asymmetric logic unit of distributed storage | |
| US11853263B2 (en) | Geographically-distributed file system using coordinated namespace replication over a wide area network | |
| US11445019B2 (en) | Methods, systems, and media for providing distributed database access during a network split | |
| US10185497B2 (en) | Cluster federation and trust in a cloud environment | |
| JP6628730B2 (en) | Geographically distributed file system using equivalent namespace replicas over wide area networks | |
| US9560093B2 (en) | Higher efficiency storage replication using compression | |
| JP4068473B2 (en) | Storage device, assignment range determination method and program | |
| US9495381B2 (en) | Geographically-distributed file system using coordinated namespace replication over a wide area network | |
| WO2016070375A1 (en) | Distributed storage replication system and method | |
| CN111124301A (en) | Data consistency storage method and system of object storage device | |
| US20070061379A1 (en) | Method and apparatus for sequencing transactions globally in a distributed database cluster | |
| JP6491210B2 (en) | System and method for supporting persistent partition recovery in a distributed data grid | |
| US9262323B1 (en) | Replication in distributed caching cluster | |
| EP2605151A1 (en) | Methods and apparatus for implementing a distributed database | |
| EP2904763A1 (en) | Load balancing access to replicated databases | |
| CN108055333A (en) | A kind of NAS-CIFS cluster load balancing methods based on UFS | |
| CN114385561A (en) | File management method and device and HDFS system | |
| US11755541B2 (en) | Distributed file system and method for accessing a file in such a system | |
| US9544371B1 (en) | Method to discover multiple paths to disk devices cluster wide | |
| CN112104752B (en) | Hot spot balancing method and system for cache nodes of content distribution network | |
| US20230205638A1 (en) | Active-active storage system and data processing method thereof | |
| Vardhan et al. | A demand based fault tolerant file replication model for clouds | |
| CN116069860A (en) | Data synchronization method and multi-activity system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |