CN1633131A - A method for implementing iSCSI memory system - Google Patents
A method for implementing iSCSI memory system Download PDFInfo
- Publication number
- CN1633131A CN1633131A CN 200510011179 CN200510011179A CN1633131A CN 1633131 A CN1633131 A CN 1633131A CN 200510011179 CN200510011179 CN 200510011179 CN 200510011179 A CN200510011179 A CN 200510011179A CN 1633131 A CN1633131 A CN 1633131A
- Authority
- CN
- China
- Prior art keywords
- scsi
- module
- memory device
- iscsi
- order
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
A realization method for ISCSI storage system characterizes in realizing ISCSI protocol on ISCSI controller including a net card binding module realizing different load balances, a storage device management module providing fixed virtual device identifier and the heat plug function, an adaptive IQ load control program controlling parallel IQ load of low end storage device and a multi-path module for IQ requiring controlling different storage array channels. Storage device image module controlling the access limitation of a starter to the storage device, a block device support module realizing the transformation of SCSI order and soft device IQ and a software RAID module are added if necessary.
Description
Technical field
The implementation method of iSCSI storage system belongs to field of storage area networks, relates in particular to iSCSI technical field of memory wherein.
Background technology
ISCSI (Internet Small Computer Systems Interface) agreement is to utilize IP network to carry out the host-host protocol end to end of I/O blocks of data level storage, has realized in the transfer of data between application server (starter) and memory device (iSCSI controller) on the ethernet device of standard.The iSCSI agreement is worked out by the Internet engineering service groups (IETF), and for fiber channel storage network (SAN), iSCSI is a kind of replenishing, or alternate standard.One of advantage of iSCSI is to make IT user can utilize the advantage and the professional technique of existing framework (TCP/IP network).Except the scheme of substituted for optical fibers channel that a kind of cheapness is provided, iSCSI also brings many advantages in application facet, as: storage merge, professional continuation, remote backup and recovery, and remote data access.
Yet because the restriction of Ethernet bandwidth, compared with the optical-fibre channel storage, the iSCSI storage system exists the not high enough shortcoming of throughput.And because the iSCSI storage system is the own characteristic of cheap substitute, the iSCSI systems face the challenge that memory device efficient is lower and failure rate is higher.And storage and network in conjunction with produced as: equipment dynamically increases and decreases, the new management role of memory device mapping, feasible path.
The present invention has realized a kind of novel high-performance and the iSCSI storage system of high availability, has effectively solved above-described problem.
Summary of the invention
The object of the present invention is to provide a kind of implementation method with iSCSI storage system of high-performance and high availability.The hardware configuration of iSCSI storage system comprises application server, storage networking, iSCSI controller and memory device and supervisor console.The iSCSI controller that focuses on the iSCSI storage system of the present invention.The iSCSI controller manages storage resources, and application server provides stores service, is the key of iSCSI storage system.On the iSCSI controller, we have taked a series of methods with innovation level, have improved the IO throughput of iSCSI storage system, have improved its inefficacy fault-tolerance (FaultTolerance), have improved its availability (Availability).The complete formation of this serial of methods a implementation method with iSCSI storage system of high-performance, high availability.
Thinking of the present invention is:
(1), realizes the load balancing of network link level and lost efficacy redundant respectively by a plurality of network interface cards on iSCSI storage system applications server and the iSCSI controller are bound.
(2) the virtual unit identifier algorithm in the storage device management module has guaranteed that same memory device has fixing device identifier, helps continuing of stores service.More original random device identifier algorithm, this method has remarkable advantages.Memory device dynamically increases and decreases algorithm and allows to increase new memory device under the non-stop prerequisite of iSCSI stores service, perhaps reduces memory device, effectively raises the availability of iSCSI storage system.
(3) the memory device mapping block is controlled the access request of separate sources, purpose, guarantees the fail safe of data access in the network storage.IP address-based memory device mapping (Storage Device Mapping) method allows two-layer mapping is carried out in the IP address of different network interface cards on starter address and the iSCSI controller, supports the mask mode, and the configuration interface based on mapping ruler is provided.Compared with traditional mapping mode, this method is more flexible, disposes also comparatively simple.
(4) self adaptation IO load control can be according to the ability of memory device, and load is controlled to IO automatically, guarantees that effectively the iSCSI storage system still can even running when the lower and IO load of storage device performance is very high.
(5) introducing of many IO routing algorithm has improved the concurrent IO ability of iSCSI storage system iSCSI controller on the one hand, has realized that on the other hand channel failure is fault-tolerant.
(6) the iSCSI controller of iSCSI storage system has been realized RAID5 and RAID6 algorithm with software mode, and the integrality of data when guaranteeing disk failure, this method realize as the replacement of hardware RAID card, characteristics such as has and configurability is better cheap.
(7) the block device support module has been realized scsi interface on general block device, make the dummy block will equipment of different non-scsi interface equipment or even Linux volume management software (LVM) and many disk managements module (MD) class can both to be adapted in this system, expanded the memory device scope that the iSCSI storage system supported and the function of iSCSI controller system memory.
The invention is characterized in that it comprises following steps successively:
1. the 1st go on foot: at following two network small computer system interfaces, also claim on any one iSCSI controller in the storage system of iSCSI, start stores service, the iSCSI agreement promptly this controller software be loaded in its operating system as the kernel module of this system:
First kind of iSCSI storage system comprises: a plurality of network application servers also claim starter; A supervisor console; By the TCP/IP procotol respectively with a kilomega network switch of above-mentioned each starter and a supervisor console communication; ISCSI controller according to iSCSI agreement and above-mentioned kilomega network exchange communication; Under (SuSE) Linux OS and according to many memory devices of iSCSI agreement and the communication of above-mentioned iSCSI controller;
Second kind of iSCSI storage system comprises: a plurality of network application servers also claim starter; A supervisor console; A kilomega network switch; An iSCSI controller; Many memory devices; Straight line connects between each starter and the iSCSI controller, through the communication of TCP/IP procotol, move under (SuSE) Linux OS according to the iSCSI agreement between iSCSI control desk and each memory device between kilomega network switch and each starter, supervisor console, the iSCSI controller;
Above-mentioned iSCSI agreement comprises:
The upper strata: by gateway binding module and front end target drives is that the FETD module once is formed by connecting;
Intermediate layer: by forming with IO load control module and many IO path module that above-mentioned front end object machine driver module links to each other successively;
Lower floor, i.e. physical layer: a device management module that links to each other with many IO path module is arranged;
Described network card binding module, it is a kind of link aggregation formula module, promptly it makes the polylith network interface card virtual to be a network interface card, all to have identical IP address; This module is under (SuSE) Linux OS, and at different needs, being equipped with the many network interface card wheels that meet IEEE 802.3ad standard changes and two kinds of patterns of Hot Spare;
Described front end object machine driver module, it is known;
Described storage device management module, when starter started iSCSI controller stores service, its scanning was connected each the deposit equipment on the iSCSI controller, and fixing virtual unit identifier is provided; When hot plug storage equipment, start the operation that memory device is increased and decreased;
It is set a linear combination form and in order to the virtual target identifier array of sign memory device numbering, includes successively on the iSCSI controller: the host adapter of this equipment number, i.e. host; Channel number, i.e. channel; Order label, i.e. target LUN, i.e. lun; This module is determined the position of a memory device with above-mentioned four variablees; Simultaneously, safeguarding on the iSCSI controller of iSCSI storage system that a memory device tabulation and expression memory device are scanned the array of number of times; It provides a write command so that the PROC interface that this module scans this memory device again for the iSCSI control desk; It also provides another PROC interface of dynamic increase and decrease memory device for the iSCSI controller;
The iSCSI controller starts the operation of scanning memory device in the following cases: during the iSCSI controller end Loading Control software of iSCSI storage system; The iSCSI system is after reducing or having increased memory device, and the storage administrator sends the order of rescanning to the iSCSI controller;
Described memory device module is implemented memory device scanning flow process successively according to the following steps:
Initialization module;
There is more host number not,, just return if do not have; Otherwise, carry out next step;
Also have more equipment not among the Host,, just return the step below the initialization module if do not have; Otherwise, carry out next step;
Whether the type of equipment is memory disk, if not, just return the step below the initialization module; Otherwise, carry out next step;
Obtain the information of equipment;
The structure facility information, insertion equipment is in table;
Available devices increases 1, returns the step below the initialization module;
Described storage device management module is implemented memory device deletion flow process successively according to the following steps:
The keeper specifies the identifier of wanting sweep equipment;
Obtain the information of this equipment in the slave unit tabulation;
The deleted marker of this equipment is set;
Discharge all not fill orders of this equipment;
The reference count that this equipment is set is 0, quotes the number of times that is scanned of this equipment that number of times is meant that storage administration is set;
Return;
Described memory management module is implemented memory device according to the following steps increases flow process:
The keeper specifies the equipment mark symbol that will increase;
The increase sign of this equipment is set;
This equipment mark symbol is joined in the memory device tabulation;
The reference count that this equipment is set is 0;
Self-adapting load control algolithm program, it adapts to the ability of memory device automatically, the IO load on the control low side memory device;
It is in the data set of a memory device position of above-mentioned expression, be provided with respectively this equipment of expression simultaneously the territory of length of treatable scsi command formation, the territory of the number of the scsi command in the execution formation that expression is in SCSI at present is set simultaneously, and at any time the latter is less than the former; Simultaneously, in the intermediate layer, a ST_HOLD_ON state is set, it represent all arrive intermediate layers and ready so as to carry out immediately but because of the number of the scsi command in the execution formation that is in SCSI at present arrived this equipment while the fault value of treatable scsi command queue length fail the state of the order that obtains carrying out;
The program of described enforcement loaded self-adaptive algorithm contains following program successively:
Module obtains the scsi command that need are handled from command queue;
Differentiate this order and whether be in the ST_HOLD_ON state, if not, then carry out next step;
Judge whether this order is write order, if write order and do not receive the storage data as yet, if, then this order is arranged to be in the state that SCSI carries out a just pending order in the formation at present, be finished, then return, if not, then be arranged to the ST_HOLD_ON state, when judging target storage device then treatable scsi command queue length whether greater than the number of the scsi command in the execution formation that is in SCSI at present, if not, then return; If, then to carry out scsi command and the number that is in the scsi command in the SCSI execution formation at present of memory device is increased 1, the number of scsi command to be processed subtracts 1, returns;
If the scsi command to be processed that obtains from command queue is in the ST_HOLD_ON state, the attitude of then remaining stationary is waited for the next round processing, carries out next step;
This module obtains next step scsi command to be processed again by above step process from command queue; If it is overtime that order does not have, then circulate repeatedly having till the scsi command to be processed in handling command queue;
Described many IO path module, it carries out IO request control to different storage queue passages, and the realization wheel changes or the Hot Spare configuration, promptly makes full use of the binary channels function with two control function disk arrays, realizes the redundant and raising between a plurality of passages; The asynchronous IO ability of SCSI storage system;
Set: the largest passages number of each host adapter, write down the pairing host adapter of scsi adapter number in many IO path module; For two control function disk arrays, the relevant information records of the memory device on adaptive number represented scsi adapter of a main frame is in the memory device tabulation, another then is kept in the structure of many IO path module definition, the status field of writing down two passages simultaneously; This module provides corresponding PROC command interface;
Express a scsi command in the intermediate layer with state ST_PATH_FAIL and on present activate channel, carry out failure, in case this module detects this state, just the state of this order is arranged to ST_PATH_FAIL, and puts into formation once more, select the another one passage to carry out simultaneously;
Many IO path module is as follows to the handling process of scsi command:
Judge earlier whether the scsi command of receiving is write order;
If not, then directly the distribution command formation is put in this order; Otherwise, put into waiting list for write order distribute data buffering area and this order, wait data and arrive; If data arrive, then from waiting list, take out this write order, put into the distribution command formation;
Under the configuration mode of determining, judged whether selectable IO path for the order in the distribution command formation, if do not have, then release; If have, then select an available IO path, and order is sent to this path wait for execution result again according to setting strategy, correctly then return if carry out, otherwise, the retry formation is put in this order, carry out the Path selection of next round.
2. in physical layer, also have a software RAID module to link to each other with described many IO path module:
Described software RAID module, its is being converted into the physics scsi command at virtual scsi command of RAID storage queue according to different RAID algorithms, and the physics scsi command on each independent physical disk the execution result merging or to the response results of virtual scsi command;
Make the global variable of RAID module safeguard following two command queues:
Virtual scsi command formation, it must comprise the sign of the state and the virtual scsi command of virtual scsi command at least;
The formation of physics scsi command, it must comprise following field at least: the state of physics scsi command, the pairing dummy command of physics scsi command, and the sequence number of this physics scsi command;
Described RAID module contains following steps successively:
According to logical address and other parameters of virtual scsi command, a virtual scsi command is resolved into the physics scsi command and the physics scsi command is formed formation, safeguard relation and state between virtual SCSI and the physics scsi command simultaneously;
By the SCSI layer each physics scsi command is operated;
After having a physics scsi command to return at every turn, detect the original directive formation by this order, dispose if find all corresponding physics scsi commands, form a response results virtual scsi command by the response results of analyzing each physics scsi command.
3. in physical layer, also have a block device support module to link to each other with above-mentioned many IO path module:
Described block device support module, it makes the starter of iSCSI storage system can connect multiple non-scsi device, comprises dummy block will equipment, and it asks to realize by scsi command being transformed into piece IO;
Described block device module contains following execution in step successively:
After receiving a scsi command from the intermediate layer, whether the target of differentiating this order is scsi device, if, then this order is sent to scsi device, return; If not, then carry out next procedure;
Whether differentiate this order is the order of inquiry class, if, then return response according to configuration information, return again; If not, just carry out next step;
Judge whether this order is read command, if, then analyze LBA and data length, calculate offset address corresponding in the virtual unit according to the LBA address again; Again offset address, data length and data buffer address are sent to the block device reading of data; After waiting for that block device writes data into the data buffer zone, the block device support module is provided with the status field of SCSI request, returns the SCSI intermediate layer; If write order, then analyze LBA and data length after, the deposit data that the starter end is sended over is in the data buffer zone again; Calculate offset address corresponding in the virtual unit according to the LBA address again; Offset address, data length and data buffer address are sent to data in block device writes SCSI request to block device the data buffer zone; Wait for the success of block device layer operation, the block device support module is provided with the status field of SCSI request and it is returned the SCSI intermediate layer.
4. in the intermediate layer, a memory device mapping block is arranged between described front end destination drive module and IO load control module:
It provides memory device position, iSCSI controller IP address and starter IP address three's mapping relations, the control starter is to the authority of storage device access in the storage system, in view of the above, access rights are divided into three kinds of ranks: visit fully, read-only access and disable access;
It changes the authority planning that is set as follows form to the storage administrator:
[source IP address] [source address mask] [purpose IP address] [Target ID] [access rights]; The memory device mapping block provides the interface that writes or check planning, starts or stops the memory device mapping block to the storage administrator with the PROC form; The planning that memory device mapping block handle writes to the PROC interface resolves to the planning with above-mentioned chained list form tissue, and this module determines the authority of a starter to certain storage device access by inquiring about this chained list;
Described memory device has mapping block to realize the mapping flow process by following three phases successively:
In the device discovery stage, it contains following steps successively:
Receive the order of starter, judge whether to be the device discovery order, if not, then return; If then carry out next step;
Judge whether memory device in addition, if not, then return; If have, then carry out next step;
According to above rule judgment starter to authority that memory device identified;
Whether as seen to judge memory device according to authority,, then device identifier is inserted the back and return the memory device tabulation to starter if as seen to starter; Otherwise, return " whether also having memory device " step;
In the parameter negotiation stage, it contains following steps successively:
The memory device mapping block is obtained the address of the client and server of connection from starter;
The memory device mapping block judges whether this memory device is a memory device that starter is forbidden, if the memory device of forbidding, then this starter of active break connects to the network of iSCSI controller; Otherwise, carry out next stage IO operation;
The IO request stage, it contains following steps successively:
Judge whether scsi command is write order.Forward " for read command " if not to;
For write order, first allocation buffer, and the authority of this order is set according to rule, at last waiting list is put in order;
After receiving data, corresponding scsi command is taken out from waiting list;
The authority of sense command, differentiating this order could be performed.If then the distribution formation is put in this order; If not, abandon this order and structure SCSI response to the starter reporting errors;
For read command, the authority of this order is set according to rule directly;
The authority of sense command, differentiating this order could be performed.If then the distribution formation is put in this order; If not, abandon this order and structure SCSI response to the starter reporting errors.
The present invention carried out testing and being applied in actual production environment in department of computer science, Tsinghua university high-performance calculation technical research institute.The result shows that novel high-performance high availability iSCSI storage system effectively raises the IO throughput of iSCSI system, and possesses bigger advantage on the availability of system, inefficacy fault-tolerance.
Description of drawings:
Fig. 1 iSCSI schematic diagram.
The hardware structure diagram of Fig. 2 iSCSI storage system, 2a. switch connected mode, the direct-connected line connected mode of 2b..
The software configuration of Fig. 3 iSCSI storage system and submodule graph of a relation.
Many network card bindings of Fig. 4 schematic diagram, 4a. switch connected mode, it is high available to utilize the redundant network link to realize, the direct line mode of 4b., two paths provide＞1.5 times of bandwidth, the direct line mode of 4c., it is high available to utilize the redundant network link to realize.
Fig. 5 memory device mapping schematic diagram.
Many IO of Fig. 6 path module schematic diagram.
Fig. 7 device management module flow chart, 7a. device scan flow process, 7b. sweep equipment flow process.
Fig. 8 device map module flow chart, 8a. are in the device discovery stage, and 8b. is in the parameter negotiation stage, and 8c. is in the IO request stage.
Fig. 9 IO load-balancing algorithm flow chart.
Many IO of Figure 10 path module flow chart.
Figure 11 block device support module flow chart.
Embodiment:
The iSCSI agreement is the host-host protocol end to end that carries out the storage of I/O blocks of data level on the ethernet device of standard, as shown in Figure 1.
The present invention is promptly to the implementation method of iSCSI network store system.In this iSCSI network store system, front end application server is connected on the iSCSI storage system iSCSI controller by the kilomega network switch or by the direct-connected line of gigabit, carry out transfer of data by ICP/IP protocol between them, constitute the iSCSI storage networking.Its hardware architecture as shown in Figure 2.
Wherein, the iSCSI controller of iSCSI storage system is equipped with 6 kilomega network network interface cards, and the parallel data stream of 8 gigabit Ethernet mouths outwards is provided, and total bandwidth reaches 6 * 1Gb.Can connect a plurality of memory disk arrays in addition on the iSCSI controller, by IP network to the storage resources service is provided with application server.
On the iSCSI of iSCSI storage system controller, the iSCSI controller software is loaded in the operating system nucleus when starting stores service as the operating system nucleus module.The iSCSI controller module is responsible for the protocol conversion work between iSCSI agreement and the SCSI agreement, and has realized the control and management of memory device and the request of application server such as are managed at task.Wherein, the iSCSI controller software can be divided into two parts on software architecture: front-end driven (FETD) and SCSI intermediate layer (STML), taking it by and large the former has realized the conversion between iSCSI agreement and responsible iSCSI agreement and the SCSI agreement, the latter has realized a simulation scsi target device, is responsible for scsi command is responded.Serial of methods proposed by the invention is all based on the iSCSI controller software.
Serial of methods proposed by the invention can be divided into independent sub-function module in realization, the relation between each functional module is seen Fig. 3.The classification of each sub-function module and concise and to the point function see the following form:
Table 1 module list
| The module title | Functions of modules |
| The network card binding module | Is different network card bindings an IP address.Realize different load-balancing algorithms, make being distributed on each network interface card that network traffics can be balanced, and when one of them link failure, the network card binding module can be adjusted the network link path automatically, realizes high availability. |
| The storage device management module | In the device start module initialization stage, device management module is responsible for scanning the memory device that is connected on the controller, and fixing virtual unit identifier is provided.When hot plug storage equipment, provide the function of dynamic increase and decrease memory device. |
| The memory device mapping block | Memory device, iSCSI controller IP address and starter IP address three's mapping relations are provided.The control starter is to the access rights of memory device in the storage networking, the safety of protected data. |
| Self adaptation IO load control algolithm | Automatically the ability that adapts to memory device, the I load on the control low side memory device guarantees the safety and steady operation of memory device. |
| Many IO path module | Different storage array passages is carried out IO request control.Realize that wheel changes (Round-Robin) and two kinds of configuration modes of Hot Spare (Active-Standby), the former realizes load balancing to improve the IO throughput of storage, and the latter has realized the passage redundancy. |
| Software RAID module | RAID5 and RAID6 have been realized with software mode. |
| The block device support module | Realize the mutual conversion of scsi command and block device IO, make the starter of iSCSI storage system can connect multiple non-scsi device, comprise dummy block will equipment. |
Below each module is described in detail.
The storage device management module:
In the iSCSI storage system, starter is connected to certain specific memory equipment on the iSCS controller by the designated equipment identifier.Device identifier i.e. the sign of certain memory device on storage control, each memory device all corresponding a device identifier.In common controller module, device identifier is according to the discovery of equipment order assignment, as: the identifier of first memory device is 0, by that analogy.The shortcoming of doing like this is, the discovery order of equipment may be different when each iSCSI controller started, and after a new memory device adds the iSCSI controller to, device identifier originally will be upset, therefore same device identifier of different time may be corresponding different memory devices.
In Linux, the SCSI floor drives the position of determining a scsi device with host (host adapter number), channel (channel number), target (order label) and four variablees of lun (LUN), and this position is determined by hardware connection mode.
We have realized virtual object identifier on the controller of iSCSI storage system: the virtual target identifier is by one 13 integer representation.Wherein, the host of bit 12～bit 10 indication equipments, the channel of bit 9～bit 8 indication equipments, the target of bit 7～bit 4 indication equipments, the lun of last 4 bit indication equipments.
Host, channel, target and lun formed virtual device identifier by linear combination and guaranteed equipment (target is lun) to the one-to-one relationship of device identifier for host, channel.The benefit of doing like this has been to safeguard the continuation of stores service, after an iSCSI controller node restarts, this algorithm can guarantee the consistency of the virtual unit identifier of this disk, and application server need not just to reconfigure again and can come into operation immediately.
Safeguarding the tabulation ext_scsi_device of a memory device and the array ext_target_valid_flag that the expression memory device is scanned number of times on the controller node of iSCSI storage system.As:
stml_device
*ext_scsi_device[MAX_HOSTS][MAX_CHANNELS][MAX_TARGETS][MAX_LUNS]；
int
ext_target_valid_flag[MAX_HOSTS][MAX_CHANNELS][MAX_TARGETS][MAX_LUNS]；
int?ext_scan_count；
Wherein, Max_HOSTS, MAX_CHANNELS, MAX_TARGETS, MAX_LUNS represent host respectively, channel, target, the maximum of lun, they are respectively: 8,4,16,16.Stml_device is the structure that shows a memory device that defines in the iSCSI storage system, and ext_scsi_device is the array of stml_device pointer.Ext_scsi_device[i then] [j] [k] [l] expression be to be positioned at (i, j, k, the memory device of l) locating, ext_target_valid_flag[i] [j] [k] [l] then point out the number of times that this equipment is scanned accordingly, ext_scan_count is the number of times of iSCSI controller scanning memory device.
When the initialization of iSCSI storage system, ext_scsi_device is every, and every and ext_scan_count is 0 for sky and ext_target_valid_flag.The device management module of iSCSI storage system obtains the information of memory device by the scsi_hostlist tabulation of scanning SCSI.When each scanning memory device is tabulated, ext_scan_count is from increasing 1, and just construct a stml_device when whenever scanning an effective memory device and corresponding of ext_scsi_device is pointed to it, at last the corresponding item of an ext_target_valid_flag assignment is become ext_scan_count.Like this, by relatively corresponding of ext_target_valid_flag and ext_scan_count just can judge whether a memory device is effective.If ext_target_valid_flag is not equal to ext_scan_count, then explanation is not found this equipment in the scanning the last time, and promptly this equipment is invalid.
The iSCSI controller module provides PROC interface, by allowing device management module again memory device be scanned to the write command of PROC interface.Utilize above-mentioned method, the iSCSI controller module does not influence the operate as normal of existing effective equipment when equipment is scanned again, this means that the iSCSI controller can scan equipment when stores service is provided.
Be the opportunity of iSCSI storage system scanning memory device: when the iSCSI controller of iSCSI storage system is loaded; The iSCSI storage system has increased memory device, and the storage administrator sends the order of rescaning to the iSCSI controller; The iSCSI storage system has reduced memory device, and the storage administrator sends the order of rescaning to the iSCSI controller.
And the scsi system of Linux provides the PROC interface of dynamic increase and decrease scsi device, deletes a scsi device:
echo″scsi?remove-single-device?a?b?c?d″＞/proc/scsi/scsi
Similar, increase a scsi device:
echo″scsi?add-single-device?a?b?c?d″＞/proc/scsi/scsi
Wherein:
A is host adapter number (since 0)
B is channel number (since 0)
C is order label (since 0)
D is LUN (since 0)
By this PROC interface, the storage administrator can dynamically update the list of devices of Linux SCSI layer.The device management module of iSCSI storage system and the Dynamic Updating Mechanism of Linux are used the function of the dynamic increase and decrease of memory device that just can realize the iSCSI storage system.
When the storage administrator after the iSCSI controller system memory adds new memory disk, the order that the storage administrator at first increases equipment and rescans memory device to the controller initiation to SCSI with add-single-device, the iSCSI controller will scan and find new disk to connecting superincumbent memory device, add in the memory device tabulation.Otherwise, when the storage administrator need delete a memory device, earlier with remove-single-deivce sweep equipment from scsi system, allow the iSCSI controller rescan memory device then, in the list of devices of iSCSI controller system memory, deleted memory device is with no longer valid.
Obviously, the memory device that is using is from logic cannot be deleted.In linux system, Scsi_Device is used to describe a scsi device, and Scsi_Device has the access_count territory of expression memory device reference count.Like this, when a disk was used by front end, we increased 1 to the access_count of this disk, otherwise access_count subtracts 1.Scsi system can judge at first whether the access_count of a memory device is 0, and for the memory device of access_count non-0, scsi system can not deleted them in remove-single-device.Reference count has guaranteed that the memory device that is using can not cause the unusual of stores service because mistake is deleted.
Thus, in fact having realized finding dynamically that at the memory device that does not stop to serve under the prerequisite memory device dynamically increases and decreases, is that whole iSCSI storage system satisfies the online access demand of 7 * 24 hours high speeds and the important leverage of the sustainability of business.
The scanning of the memory device of storage device management module and the flow process of deleting a memory device are as shown in Figure 7.
The network card binding module:
Become a network interface card with the polylith network interface card is virtual, make it have identical IP address, realize promoting the network throughput of main frame or improving availability, this technology is known as link aggregation (Channel bonding).We have realized independently that under Linux the many network interface card wheels that meet IEEE 802.3ad standard change (Round-Robin) and two kinds of sending modes of Hot Spare (Active-Standby), thereby have improved the availability of application server and iSCSI controller and provide the load balancing of network interface card to improve efficient.As shown in Figure 4.
ISCSI storage system binding module is supported following two kinds of transmission modes under IEEE 802.3ad standard card cage:
1. take turns revolving die formula (the autonomous realization): based on the order of network interface card, wheel changes by different network interface cards transmission data successively; Thereby realize transmission performance greater than a single link.
2. hot-standby mode: guarantee only to send data by a network interface card at synchronization, another one activates the Hot Spare of card as this, and data flow switches to the Hot Spare card at once when activating card and break down.
At different application demands, we use Ethernet switch in fairly large storage networking, to reach high availability and extensibility preferably, and because the function of Ethernet switch restriction (most mainstream switches are incomplete to the support of IEEE 802.3ad agreement), can only realize the load balancing (Transmit Load Balancing) that sends can't realizing balancing received load (Receive Load Balancing).And in needing the small-scale application of superior performance, we have adopted the direct-connected structure of gigabit cross spider, utilize the binding module of wheel revolving die formula to realize the load balancing that receives and send, thereby reached very high network performance: in the test, under the direct-connected topology that two gigabits connect, the tcp data newspaper sends and acceptance reaches 172Mbyte/s, and the UDP datagram agreement more reaches 192Mbyte/s.
The memory device mapping block:
In actual applications, a problem being concerned about of network store system is exactly the consideration for the data access fail safe.IP address-based memory device mapping has realized the control of starter to virtual unit identifier access rights.As shown in Figure 5, the memory device mapping block allows the double medium filtration of iSCSI controller network interface card IP address and starter IP address.In brief, the memory device mapping block carries out the authority judgement to the device identifier of source IP address, purpose IP address and the destination disc of an IO request, and the foundation of judgement is the rules of competence that the storage administrator writes.Access rights are divided into three ranks: visit fully, read-only access and disable access.
The device map module is to exist as an independent operating system nucleus module (Kernel Module) mode, and the storage administrator can determine whether load as required.
Owing to taked IP on the iSCSI controller and the two-layer coupling of starter IP, and to the more traditional mapping method of the Permission Levels of equipment many read-only access, the storage administrator is more flexible to the distribution of memory device.Storage administrator's operation has then been simplified in the introducing of the rules of competence.
For the storage administrator, the form of a rule as:
[source IP address] [source address mask] [purpose IP address] [TargetID] [access rights]
And in the memory device mapping block, the organization definition of rule is as follows:
struct?target_mapping_rule{
struct?list_head?????list；
struct?list_head?????hash_list；
＿u32??target_id；
＿u32??src_addr；
＿u32??net_mask；
＿u32??dest_addr；
＿u8???permission；
}；
Wherein, target_mapping_rule is a structure of describing mapping ruler, and the list_head structure is used for strictly all rules is connected to become chained list, and list is general doubly linked list, and hash_list is the hash chained list, is used for the high speed rule searching.Target_id indicates this device identifier that mates, if the value of this field is 0xffffffff then mates all memory devices.Src_addr and dest_addr formulate the IP address of starter and the IP address of iSCSI controller respectively, and be same, if the value of these two fields is 0xffffffff then mates all IP addresses that the net_mask field is the subnet mask of src_addr.Field permission indicates authority, comprises visit fully, read-only access and disable access.
The memory device mapping block provides the interface that writes rule or check rule with the form of PROC to the storage administrator, and by this interface, the storage administrator can enable the memory device mapping block or stop the memory device mapping block in addition.Mapping block becomes the rule parsing that writes the list of rules of target_mapping_rule structure and insert module.
The memory device mapping block becomes the target_mapping_rule structure to the rule parsing that writes to the PROC interface, and with the form tissue of chained list.The memory device mapping determines the access rights of an application server to certain memory device by inquiring about this chained list.
Memory device is mapped on three links the visit of starter is controlled: memory device finds that memory device connects and the IO read-write operation.The process that application server (starter) is obtained the memory device tabulation from the iSCSI controller is the memory device discovery procedure.Application server returns the memory device tabulation by sending order from the request of iSCSI controller, the memory device mapping block is intercepted and captured this order and the store list of iSCSI controller is filtered, and the final only visible memory device of this application server the tabulation returned; Application server is after the tabulation that obtains memory device, and application server can be connected to this locality to the memory device of certain appointment, and this process is the memory device connection procedure.When application server was attempted one connected it memory device of forbidding, the memory device mapping block interrupted this application server immediately and connects to the network of iSCSI controller; Application server just can carry out read-write operation to memory device after successfully connecting memory device.The memory device mapping block is monitored all IO streams, in case find that certain IO request is (as read-only memory device is carried out write operation) that is not allowed to, the memory device mapping block abandons this IO request, and returns error message to application server.
In realization, increased the process of scope check in the response function to device discovery of memory device mapping block in FETD (front-end driven).At first, all possible virtual memory facilities identifier is traveled through, if this virtual memory facilities identifier effectively and according to list of rules judges that the IP address of this storage device identification symbol application server and the IP address of iSCSI controller are visible (can visit fully or read-only access), then this storage device identification symbol is put into the storage device identification of preparing to return and accord with tabulation.
When handling the request of application server connection memory device, whether as seen to judge this memory device application server.In the process that memory device connects, the iSCSI parameter of application server and iSCSI controller need be held consultation, to determine the transmission parameter of application server and iSCSI controller.The storage device identification that we attempt application server to connect accords with also as both sides' negotiation parameter.When consulting, for application server whether as seen the memory device mapping block judges this storage device identification symbol, if invisible then interrupt negotiations process, returns the negotiation failure, and the network that finally allows the iSCSI controller be disconnected to application server connects.
The memory device mapping block increases in SCSI intermediate layer (STML) filtration treatment is carried out in IO operation.In order to show access rights, in the structure of expression order, increase the territory authority of an expression authority to memory device.The step that the scsi command authority is set is as follows, at first obtain the IP address of the starter that sends this order, the IP address of iSCSI controller and the destination device identifier of this order, in list of rules, inquire about the authority territory that corresponding authority is given this order then.
In the IO operational phase, the memory device mapping block is at first judged the command type of scsi command.For write order, need the allocation buffer, and the authority of this order is set according to rule, at last waiting list is put in order, wait for the required gathering data arrival that writes.After receiving data, corresponding scsi command is taken out from waiting list, and the authority of sense command, differentiating this order could be performed.If allow to carry out, then the distribution formation is put in this order, otherwise abandon this order and construct the SCSI response to the starter reporting errors.For read command, the authority of this order directly is set according to rule, the authority of sense command again, differentiating this order could be performed.If allow to carry out, then the distribution formation is put in this order, otherwise abandon this order and construct the SCSI response to the starter reporting errors.
The program flow diagram of this module as shown in Figure 8.
Self adaptation IO load control algolithm:
For the lower memory device of some performances, too high IO request can cause the unusual of device drives, thereby causes the collapse of whole system, therefore is necessary to introduce on the iSCSI controller control to the IO load.And general iSCSI storage system generally lacks the monitoring mechanism to the IO load, and the present invention has realized the IO load controlled function based on scsi command.
In the stml_device structure of a memory device of expression, can_queue and cmd_pending represent respectively in two territories this equipment simultaneously treatable scsi command queue length and be in the number of the scsi command in the execution formation of SCSI at present, should guarantee all that at any time cmd_pending is less than can_queue.
In SCSI intermediate layer (STML), we identify the residing flow process of a scsi command with different states, and one of them state is ST_HOLD_ON.The ST_HOLD_ON state is the key that realizes load control, and all arrive SCSI intermediate layers (STML) and the ready order that can carry out immediately and fail the state of the order that obtains carrying out because cmd_pending arrives the threshold value of can_queue and all be configured to the ST_HOLD_ON state.The ST_PENDING state shows that then order is in wait state.
When SCSI intermediate layer (STML) handles a scsi command, if order is write order and does not receive as yet when storing data, the state of order is set to the ST_PENDING state and returns, otherwise the state of order is set to the ST_HOLD_ON state, whether the cmd_pending that judges target storage device then is less than can_queue, carry out scsi command and the cmd_pending of memory device increased 1 if then call scsi_do_req, after scsi_do_req is finished with call back function te_cmnd_processed.Wherein, scsi_do_req is the interface function that linux system SCSI layer provides, be used to carry out scsi command, and te_cmnd_processed is the function that defines in the intermediate layer, its address is imported into the SCSI layer and always be called back execution when scsi command is finished.Cmd_pending subtracts 1 in te_cmnd_processed, and like this, the value of cmd_pending is exactly the number that is in the scsi command in the execution formation of SCSI.If cmd_pending has arrived thresholding can_queue, then scsi command maintains the state of ST_HOLD_ON, waits for the next round processing.In the handle_cmd function, be ST_HOLD_ON if find the state of an order equally, whether the cmd_pending that judges target storage device if then call scsi_do_req carries out this scsi command less than can_queue.So just can remain on the cmd_pending number below the can_queue, guarantee that memory device normally works stably.The program flow diagram of this module as shown in Figure 9.
Many IO path module:
Many IO path module makes full use of the binary channels function with two control function disk arrays, has realized the redundancy between a plurality of SCSI passages and has improved the concurrent IO ability of iSCSI storage system.As shown in Figure 6.
In many IO path module, write down the relevant information of many IO routing algorithm with following data structure:
#define?MAX_PATH??????4；
int?multipath_host0[MAX_PATH]，multipath_host1[MAX_PATH]；
struct?multipath_private_data{
stml_device*scsi_device；
int?host；
unsigned?int?ext_device_state；
unsigned?int?multi_device_state；
}
multipath_conf[MAX_HOSTS][MAX_CHANNELS][MAX_TARGETS][MAX_LUNS]；
Wherein, the value representation largest passages number of MAX_PATH, multipath_host0, multipath_host1 write down the pairing host adapter of scsi adapter number in many IO path module.Many IO path module relevant information of depositing multipath_host0 and multipath_host1 respectively with two arrays, this two numbers group is respectively ext_scsi_device and multipath_conf.The relevant information records that is connected the memory device on the represented scsi adapter of multipath_host0 is in the ext_scsi_device array, and the relevant information records of memory device that is connected the represented scsi adapter of multipath_host1 is in the multipath_conf array.For example, host adapter in many IO path number is respectively host1 and host2, multipath_host0[0 then]=1, multipath_host1[0]=2, the relevant information that is connected the memory device of host1 is kept at ext_scsi_device[1] in [] [] [], and the relevant information that is connected the memory device of host2 is kept at multipath_conf[2] in [] [] [], ext_device_state and multi_device_state write down the state of two passages respectively.
Many IO path module has realized following two kinds of main modes:
(1) Hot Spare (Active-Standby) mode: IO request is all the time to two one of them transmissions of passage, and this passage promptly activates (Active) passage, and another one is as standby redundancy.Find the IO request after activate channel is carried out failure when many IO path module, activate alternate channel immediately and forward request to alternate channel transmission IO.
(2) wheel commentaries on classics (Round-Robin) mode: IO request sends to two passages in turn with round robin algorithm.
In order to realize ordering the repeating transmission after the failure, in SCSI intermediate layer (STML), represent that with state ST_PATH_FAIL a scsi command carries out failure on present activate channel, after many IO path module detects a scsi command execution failure, just the state of this order is arranged to ST_PATH_FAIL and is put into formation once more.If the state of an order is ST_PATH_FAIL, then need to select the another one passage to carry out.
Many IO path module is as follows to the scsi command handling process:
(1) after many IO path module receives a scsi command, judges whether this order is write order earlier.
(2), then directly the distribution command formation is put in this order if not write order.Otherwise, put into waiting list for write order distribute data buffering area and this order, wait pending data to arrive.If data arrive, then from waiting list, take out this order and put into the distribution command formation.
(3) judged whether selectable IO path for the order in the distribution command formation, if not then withdraw from, if have, then select an available IO path according to strategy, and order sent to this path and wait for execution result, if carry out correctly then return, otherwise the retry formation is put in this order, carry out the Path selection of next round.
The program circuit of many IO path module as shown in figure 10.
The PROC interface that the storage administrator provides by many IO path module is provided with the mode of many IO path module, is implemented in the concurrent ability of IO of the multi-channel redundant or the raising iSCSI controller in IO path on the iSCSI controller.
Software RAID module:
In traditional application, the function of the RAID of storage is generally finished by hardware RAID card, and the software RAID module of iSCSI storage system can become RAID5 or RAID6 redundant storage array to a plurality of disks with scsi interface with the software mode tissue.The iSCSI controller is made into storage array to independent disk groups and offers application server, the scsi command that application server sends these storage arrays becomes virtual scsi command, virtual scsi command must be broken down into the SCSI physics scsi command at the single physical disk after arriving the iSCSI controller.On the other hand, the result of physics scsi command execution also need be combined into the SCSI response of application server on the iSCSI controller.What software RAID module was done is exactly that the virtual scsi command at the RAID storage array is converted into the physics scsi command according to different RAID algorithms, and the execution result of physics scsi command on each independent physical disk merged the response results that becomes virtual scsi command.
According to the above, the processing procedure of scsi command mainly comprises three steps:
1. virtual scsi command is to the fractionation of physics scsi command
2. the processing of physics scsi command
3. physics scsi command result is to the feedback of virtual scsi command
The global variable target_data of software RAID module safeguards two command queues: one is virtual scsi command formation, and another is the formation of physics scsi command.Wherein virtual scsi command formation is responsible for the virtual scsi command that passes over from front-end driven (FETD) is handled.Analyze by logical address and other parameters to virtual scsi command, a virtual scsi command can split into one or more physics scsi commands, and each physics scsi command can carry out executable operations by the SCSI layer.After all physics scsi commands of a virtual scsi command correspondence are all handled, by analyzing the response results of each physics scsi command, thereby form a response results to virtual scsi command.And the processing to the physics scsi command is responsible in the formation of physics scsi command.
In order to realize above-mentioned functions, virtual scsi command must comprise the sign of the state and the virtual scsi command of virtual scsi command at least; And the physics scsi command must comprise following field: the virtual scsi command under the state of physics scsi command, physics scsi command, and the sequence number of this physics scsi command.
In the function of handling virtual scsi command, the virtual scsi command that receives from front-end driven (FETD) is packaged into a Target_Virtual_Scsi_Cmnd structure, and this dummy command is inserted the dummy command formation.
Following several steps is the key step of virtual scsi command processing procedure.
Be that virtual scsi command is resolved into the physics scsi command, and the physics scsi command is inserted respective queue, safeguard relation and state between virtual scsi command and the physics scsi command simultaneously.
After having a physics scsi command to return at every turn, detect the original directive formation by this order, if the virtual scsi command that discovery has all corresponding physics scsi commands to finish dealing with, then change the state of this virtual scsi command, carry out simulation then virtual scsi command return results.
As long as we do following change like this, just can realize expansion, thereby make it to be fit to software RAID module STML.
On the described in the above basis, we can carry out order fractionation, execution and result to a scsi command at RAID5/RAID6 and synthesize.
Software RAID need simulate a series of read write command, comprising: READ (6)/WRITE (6), READ (10)/WRITE (10), READ (12)/WRITE (12) and READ (16)/WRITE (16).The difference of these orders is that order length is different, so the length of their addressing range and once request is all little the same.Resolving to the different length order also is not quite similar.
The 0th byte as READ (10) order is a command code, and it also comprises information such as piece logical address (LBA, Logical Block Address) and transmission length.Wherein the piece logical address is positioned at the 2nd to the 5th byte of this order, so the logical address LBA of READ (10) order can calculate (wherein cmnd is a scsi command) with following formula:
lba＝(cmnd[2]＜＜24)+(cmnd[3]＜＜16)+(cmnd[4]＜＜8)+cmnd[5]；
Obtain the piece logical address of order according to computing formula after, we just can calculate the deviation post of read request in virtual unit:
seek＝lba*BLOCKSIZE
The size of BLOCKSIZE is generally 512 bytes.
And the piece number that the READ_10 order is asked can be calculated as follows:
num＝cmd[8]+(cmd[7]＜＜8)
After resolving to LBA address and request transmission length, software RAID module draws read-write skew at each disk, reads and writes data and length according to the configuration information of different RAID algorithms and RAID, and re-constructs the physics scsi command at actual disk.
RAID module invokes scsi_do_req carries out and obtains its result to each physics order afterwards, again the synthetic result of all results.
Except read-write operation, software RAID module also needs some querying commands that starter is initiated are made response, orders as INQUIRY.For these orders, we can directly return its result according to configuration information.
The block device support module:
The block device support module realizes that scsi command transforms to piece IO request, and the request results of piece IO is packaged into SCSI result.In fact this be equivalent to increase one deck scsi interface layer on general block device.The block device support module not only can support not possess the physical equipment of scsi interface, as the IDE disk, also can support to comprise LVM, MD is at interior dummy block will equipment, and LVM has realized virtual storage, MD has realized software RAID function, and obviously the block device support has enlarged the support scope of iSCSI storage system to memory device, has expanded the function of iSCSI storage system.
The block device support module is realized focuses on to the parsing of SCSI read write command with to the simulation of SCSI read write command return results.The block device support module at first needs to become the block device read-write requests from the SCSI read write command parsing that front end sends over, and mainly is the parsing of device address and reading and writing data length.The result that the block device request the is carried out request results that transforms into scsi command is used for respective front ends then.
With regard to read request and the write request of SCSI this processing procedure is done concise and to the point introduction respectively below.
For read request, the data length that the block device support module at first parses the LBA address and read from the SCSI request, calculate offset address corresponding in the virtual unit according to the LBA address then, again offset address, data length and data buffer address are passed to the block device reading of data.After waiting for that block device is read into the data buffer zone to data, the block device support module is provided with the status field of SCSI request and it is returned iSCSI controller SCSI intermediate layer (STML).
For write request.The block device support module from SCSI request, parse LBA address, data length equally and deposit data such as data buffering area that the starter end is sended in, calculate in the virtual unit corresponding offset address according to the LBA address then, again offset address, data length and data buffer address are passed to the block device layer and block device is write data in the data buffer zone of SCSI request.After waiting for the success of block device layer operation, the block device support module is provided with the status field of SCSI request and it is returned SCSI intermediate layer (STML).
The SCSI read write command of block device support module support comprises: READ (6)/WRITE (6), READ (10)/WRITE (10), READ (12)/WRITE (12) and READ (16)/WRITE (16).The resolving of these orders and the split process in the RAID module are similar.Different is that its execution is different with the process that the result synthesizes.In realization of the present invention, to opening of block device close with read-write operation be the filp_open/filp_close that calls kernel, f_op-〉llseek, f_op-〉read and f_op-write finishes.
The flow chart of block device support module as shown in figure 11.
Test result:
Performance test
Respectively from IO throughput (throughput), carry out average reaction time (average responsetime) two aspects to the test of iSCSI storage system.The two is an important indicator of weighing the iSCSI performance of storage system.Test is carried out on an iSCSI controller and three front-end servers.I/O node machine adopts the two cpu servers of 32 Anthem II .4GHZ, internal memory 2GB, and operating system is Linux (Kernel 2.4.26).Storage subsystem adopts the QLA2200 optical channel card of Qlogic company, and by the disk chassis of 2 Odysys companies.We are equipped with 6 gigabit ethernet cards on the iSCSI controller, and bind an IP for per two, every of front-end server is equipped with 2 network interface cards, all binds an IP, front-end server links to each other with direct-connected line with the iSCSI controller, and the Bonding algorithm is selected the Active-Standby mode.
In test, we have tested throughput and the average reaction time of reading and writing fully in proper order under the different masses size, and testing tool is the iometer of intel, and the result is as shown in the table.
Table 2IO throughput
| The data block size | IO throughput (Mbytes/s) | |
| Read | Write | |
| ?64KB | ?126.10 | ?120.30 |
| ?128KB | ?126.77 | ?120.70 |
| ?512KB | ?126.23 | ?120.96 |
| ?1024KB | ?125.93 | ?120.94 |
| ?2048KB | ?126.13 | ?120.48 |
| ?4096KB | ?126.38 | ?120.97 |
Table 3 average response time
| The data block size | Average response time (ms) |
| Read | Write | |
| ?64KB | ?2.95 | ?3.10 |
| ?128KB | ?11.87 | ?12.38 |
| ?512KB | ?23.81 | ?24.79 |
| ?1024KB | ?47.55 | ?49.55 |
| ?2048KB | ?94.91 | ?99.11 |
| ?4096KB | ?190.21 | ?198.74 |
The above results shows that the iSCSI storage system has higher IO throughput and short average response time.
The network link redundancy testing:
The purpose of this test is that test I is carried out many network card bindings and is configured as link redundancy function when transmitting with the Hot Spare algorithm.
In test, we become an IP to two network card bindings on the IO controller, equally two network card bindings of application server are become an IP, and the transmission algorithm of setting network card binding module is the Hot Spare mode.Two network interface cards are connected by two network interface cards of direct-connected line and application server respectively on the IO controller, test according to following steps:
(1) the iSCSI stores service of startup IO controller starts application server iSCSI launcher software the last memory device of iSCSI system is mapped to this locality, and the startup application program is read and write the iSCSI memory device.On the IO controller, start the traffic instrument flow of two network interface cards is monitored, can see that at this moment data flow passes through to activate (Active) link and transmit.
(2) people is the netting twine that pulls out active link.At this moment can see that data flow switches to original backup link immediately.Application program on the application server is read and write not discover and tangible influence.
(3) netting twine is inserted again, pulled out original backup netting twine again.Can see that data flow switches to original link immediately.Application program on the application server is read and write not discover and tangible influence.
Above test shows, the iSCSI storage system provides the redundancy of network link level by network card binding, has improved the availability of system.
The test of many IO path module:
The purpose of this test is the passage redundancy feature of many IO path module of test I.
In test, we install two SCSI cards at the IO controller, and are connected by two passages of two scsi data lines and a binary channels disk array, test according to following steps:
(1) the iSCSI stores service of startup IO controller starts application server iSCSI launcher software the binary channels Disk Array in the iSCSI system is mapped to this locality, and the startup application program is read and write array.
(2) the scsi data passage that artificially breaks and wherein using.The application program read-write is not subjected to tangible influence.
(3) the scsi data line is inserted again the another one of breaking again scsi data passage.Application program on the application server is read and write not discover and tangible influence.
Top test shows, the iSCSI storage system has improved the availability of system by the redundancy of scsi data channel level is provided.
The device management module test:
This tests the online situation that subtracts memory device of adding of main testing equipment administration module.Test in the following several ways respectively:
1. delete the memory device of reading and writing.This situation simulation this disk generation physical fault when application server being is is being read and write certain piece disk, the situation that needs deletion to change.
Test according to following steps:
(1) the iSCSI stores service of startup IO controller starts application server iSCSI launcher software the last memory disk of iSCSI system is mapped to this locality, starts on application server and with application program this disk is carried out read-write operation.
(2) on the IO controller, call the iscsi_remove script, the order that the PROC interface that provides by iSCSI storage system device management module sends this disk of deletion.
Can see that (3) read-write operation of application program stops on the application server.After a period of time, application server interrupts to the network link of IO controller, and the memory disk that mapping is come on the application server disappears, and application program is interrupted read-write.
From above test as can be seen, occur at memory device under the situation of physical fault, the iSCSI system can safe this equipment of deletion.
2. delete other memory devices when reading and writing.This situation simulation is when application server is is read and write, and certain piece needs the situation of deletion replacing not in the disk generation physical fault of read-write.
Test according to following steps:
(1) the iSCSI stores service of startup IO controller starts application server iSCSI launcher software the last memory disk of iSCSI system is mapped to this locality, starts on application server and with application program this disk is carried out read-write operation.
(2) on the IO controller, call the iscsi_remove script, the order that the PROC interface that provides by iSCSI storage system device management module sends another one disk on the deletion IO controller.
Can see that (3) read-write of application program is not affected on the application server.And can't see deleted disk in the tabulation of the memory device of the iSCSI starter on the application server.
From above test as can be seen, the iSCSI storage system provides the function of deletion memory device under the situation that does not stop stores service.
3. increase a memory device when reading and writing.This situation simulation needs to increase the situation of storage resources when application server is is read and write.
Test according to following steps:
(1) the iSCSI stores service of startup IO controller starts application server iSCSI launcher software the last memory disk of iSCSI system is mapped to this locality, starts on application server and with application program this disk is carried out read-write operation.
(2) in storage resources, increase a physical disk, on the IO controller, call the scsi_add script, make SCSI updated stored list of devices, the PROC interface that calling the iscsi_rescan script again provides by iSCSI storage system device management module send and rescan the order that memory device is tabulated.
Can see that (3) read-write of application program is not affected on the application server.And increased the disk that just adds in the tabulation of the memory device of the iSCSI starter on the application server.
From above test as can be seen, the iSCSI storage system provides the function that increases memory device under the situation that does not stop stores service.
The test of block device support module:
In this test, we create an internal memory virtual disk (all read-write operations are all in internal memory) on the IO controller, a hardware RAID0 array, a MD module RAID0 virtual unit.Virtual unit is a block device, does not possess scsi interface, needs the support of block device support module.We use IOMeter test memory dish respectively on application server, the virtual disk of scsi disk and block device support module draws IO throughput and IO time-delay.In this test, application server has only one.At block size is 1MB, situation under, test result is as follows:
Table 4IO throughput (MB/s)
| Ram disk | Hardware RAID0 | Virtual RAID0 | |
| Read operation | 66.47 | ?24.66 | ?50.23 |
| Write operation | 69.41 | ?28.35 | ?51.61 |
Table 5IO time-delay (ms)
| Ram disk | Hardware RAID0 | Virtual RAID0 | |
| Read operation | 0.23 | ?0.63 | ?0.31 |
| Write operation | 0.22 | ?0.55 | ?0.29 |
Can see from above two tables:
For read request, the IO throughput of virtual RAID0 volume is 2.04 times of IO throughput of hardware RAID0 volume.And for write request, this value is 1.82 times.
For read request, the IO time-delay of virtual RAID0 volume be hardware RAID0 volume the IO time-delay 20%.And for write request, this value is 23%.
Test shows, piece support module can well the virtual support block devices, and high performance protocol conversion mechanism is provided.The performance of virtual block device even be higher than hardware device in the test, this mainly is because the effect of block device floor height speed buffer memory (Cache).
Claims (4)
1. the implementation method of an iSCSI storage system is characterized in that, it comprises following steps successively:
The 1st step: at following two network small computer system interfaces, also claim on any one iSCSI controller in the storage system of iSCSI, start stores service, the iSCSI agreement promptly this controller software be loaded in its operating system as the kernel module of this system:
First kind of iSCSI storage system comprises: a plurality of network application servers also claim starter; A supervisor console; By the TCP/IP procotol respectively with a kilomega network switch of above-mentioned each starter and a supervisor console communication; ISCSI controller according to iSCSI agreement and above-mentioned kilomega network exchange communication; Under (SuSE) Linux OS and according to many memory devices of iSCSI agreement and the communication of above-mentioned iSCSI controller;
Second kind of iSCSI storage system comprises: a plurality of network application servers also claim starter; A supervisor console; A kilomega network switch; An iSCSI controller; Many memory devices; Straight line connects between each starter and the iSCSI controller, through the communication of TCP/IP procotol, move under (SuSE) Linux OS according to the iSCSI agreement between iSCSI control desk and each memory device between kilomega network switch and each starter, supervisor console, the iSCSI controller;
Above-mentioned iSCSI agreement comprises:
The upper strata: by gateway binding module and front end target drives is that the FETD module once is formed by connecting;
Intermediate layer: by forming with IO load control module and many IO path module that above-mentioned front end object machine driver module links to each other successively;
Lower floor, i.e. physical layer: a device management module that links to each other with many IO path module is arranged;
Described network card binding module, it is a kind of link aggregation formula module, promptly it makes the polylith network interface card virtual to be a network interface card, all to have identical IP address; This module is under (SuSE) Linux OS, and at different needs, being equipped with the many network interface card wheels that meet IEEE 802.3ad standard changes and two kinds of patterns of Hot Spare;
Described front end object machine driver module, it is known;
Described storage device management module, when starter started iSCSI controller stores service, its scanning was connected each the deposit equipment on the iSCSI controller, and fixing virtual unit identifier is provided; When hot plug storage equipment, start the operation that memory device is increased and decreased;
It is set a linear combination form and in order to the virtual target identifier array of sign memory device numbering, includes successively on the iSCSI controller: the host adapter of this equipment number, i.e. host; Channel number, i.e. channel; Order label, i.e. target; LUN, i.e. lun; This module is determined the position of a memory device with above-mentioned four variablees; Simultaneously, safeguarding on the iSCSI controller of iSCSI storage system that a memory device tabulation and expression memory device are scanned the array of number of times; It provides a write command so that the PROC interface that this module scans this memory device again for the iSCSI control desk; It also provides another PROC interface of dynamic increase and decrease memory device for the iSCSI controller;
The iSCSI controller starts the operation of scanning memory device in the following cases: during the iSCSI controller end Loading Control software of iSCSI storage system; The iSCSI system is after reducing or having increased memory device, and the storage administrator sends the order of rescanning to the iSCSI controller;
Described memory device module is implemented memory device scanning flow process successively according to the following steps:
Initialization module;
There is more host number not,, just return if do not have; Otherwise, carry out next step;
Also have more equipment not among the Host,, just return the step below the initialization module if do not have; Otherwise, carry out next step;
Whether the type of equipment is memory disk, if not, just return the step below the initialization module; Otherwise, carry out next step;
Obtain the information of equipment;
The structure facility information, insertion equipment is in table;
Available devices increases 1, returns the step below the initialization module;
Described storage device management module is implemented memory device deletion flow process successively according to the following steps:
The keeper specifies the identifier of wanting sweep equipment;
Obtain the information of this equipment in the slave unit tabulation;
The deleted marker of this equipment is set;
Discharge all not fill orders of this equipment;
The reference count that this equipment is set is 0, quotes the number of times that is scanned of this equipment that number of times is meant that storage administration is set;
Return;
Described memory management module is implemented memory device according to the following steps increases flow process:
The keeper specifies the equipment mark symbol that will increase;
The increase sign of this equipment is set;
This equipment mark symbol is joined in the memory device tabulation;
The reference count that this equipment is set is 0;
Self-adapting load control algolithm program, it adapts to the ability of memory device automatically, the IO load on the control low side memory device;
It is in the data set of a memory device position of above-mentioned expression, be provided with respectively this equipment of expression simultaneously the territory of length of treatable scsi command formation, the territory of the number of the scsi command in the execution formation that expression is in SCSI at present is set simultaneously, and at any time the latter is less than the former; Simultaneously, in the intermediate layer, a ST_HOLD_ON state is set, it represent all arrive intermediate layers and ready so as to carry out immediately but because of the number of the scsi command in the execution formation that is in SCSI at present arrived this equipment while the fault value of treatable scsi command queue length fail the state of the order that obtains carrying out;
The program of described enforcement loaded self-adaptive algorithm contains following program successively:
Module obtains the scsi command that need are handled from command queue;
Differentiate this order and whether be in the ST_HOLD_ON state, if not, then carry out next step;
Judge whether this order is write order, if write order and do not receive the storage data as yet, if, then this order is arranged to be in the state that SCSI carries out a just pending order in the formation at present, be finished, then return, if not, then be arranged to the ST_HOLD_ON state, when judging target storage device then treatable scsi command queue length whether greater than the number of the scsi command in the execution formation that is in SCSI at present, if not, then return; If, then to carry out scsi command and the number that is in the scsi command in the SCSI execution formation at present of memory device is increased 1, the number of scsi command to be processed subtracts 1, returns;
If the scsi command to be processed that obtains from command queue is in the ST_HOLD_ON state, the attitude of then remaining stationary is waited for the next round processing, carries out next step;
This module obtains next step scsi command to be processed again by above step process from command queue; If it is overtime that order does not have, then circulate repeatedly having till the scsi command to be processed in handling command queue;
Described many IO path module, it carries out IO request control to different storage queue passages, and the realization wheel changes or the Hot Spare configuration, promptly makes full use of the binary channels function with two control function disk arrays, realizes the redundant and raising between a plurality of passages; The asynchronous IO ability of SCSI storage system;
Set: the largest passages number of each host adapter, write down the pairing host adapter of scsi adapter number in many IO path module; For two control function disk arrays, the relevant information records of the memory device on adaptive number represented scsi adapter of a main frame is in the memory device tabulation, another then is kept in the structure of many IO path module definition, the status field of writing down two passages simultaneously; This module provides corresponding PROC command interface;
Express a scsi command in the intermediate layer with state ST_PATH_FAIL and on present activate channel, carry out failure, in case this module detects this state, just the state of this order is arranged to ST_PATH_FAIL, and puts into formation once more, select the another one passage to carry out simultaneously;
Many IO path module is as follows to the handling process of scsi command:
Judge earlier whether the scsi command of receiving is write order;
If not, then directly the distribution command formation is put in this order; Otherwise, put into waiting list for write order distribute data buffering area and this order, wait data and arrive; If data arrive, then from waiting list, take out this write order, put into the distribution command formation;
Under the configuration mode of determining, judged whether selectable IO path for the order in the distribution command formation, if do not have, then release; If have, then select an available IO path, and order is sent to this path wait for execution result again according to setting strategy, correctly then return if carry out, otherwise, the retry formation is put in this order, carry out the Path selection of next round.
2. the implementation method of a kind of iSCSI storage system according to claim 1 is characterized in that: in physical layer, also have a software RAID module to link to each other with described many IO path module:
Described software RAID module, its is being converted into the physics scsi command at virtual scsi command of RAID storage queue according to different RAID algorithms, and the physics scsi command on each independent physical disk the execution result merging or to the response results of virtual scsi command;
Make the global variable of RAID module safeguard following two command queues:
Virtual scsi command formation, it must comprise the sign of the state and the virtual scsi command of virtual scsi command at least;
The formation of physics scsi command, it must comprise following field at least: the state of physics scsi command, the pairing dummy command of physics scsi command, and the sequence number of this physics scsi command;
Described RAID module contains following steps successively:
According to logical address and other parameters of virtual scsi command, a virtual scsi command is resolved into the physics scsi command and the physics scsi command is formed formation, safeguard relation and state between virtual SCSI and the physics scsi command simultaneously;
By the SCSI layer each physics scsi command is operated;
After having a physics scsi command to return at every turn, detect the original directive formation by this order, dispose if find all corresponding physics scsi commands, form a response results virtual scsi command by the response results of analyzing each physics scsi command.
3. the implementation method of a kind of iSCSI storage system according to claim 1 is characterized in that: in physical layer, also have a block device support module to link to each other with above-mentioned many IO path module:
Described block device support module, it makes the starter of iSCSI storage system can connect multiple non-scsi device, comprises dummy block will equipment, and it asks to realize by scsi command being transformed into piece IO;
Described block device module contains following execution in step successively:
After receiving a scsi command from the intermediate layer, whether the target of differentiating this order is scsi device, if, then this order is sent to scsi device, return; If not, then carry out next procedure;
Whether differentiate this order is the order of inquiry class, if, then return response according to configuration information, return again; If not, just carry out next step;
Judge whether this order is read command, if, then analyze LBA and data length, calculate offset address corresponding in the virtual unit according to the LBA address again; Again offset address, data length and data buffer address are sent to the block device reading of data; After waiting for that block device writes data into the data buffer zone, the block device support module is provided with the status field of SCSI request, returns the SCSI intermediate layer; If write order, then analyze LBA and data length after, the deposit data that the starter end is sended over is in the data buffer zone again; Calculate offset address corresponding in the virtual unit according to the LBA address again; Offset address, data length and data buffer address are sent to data in block device writes SCSI request to block device the data buffer zone; Wait for the success of block device layer operation, the block device support module is provided with the status field of SCSI request and it is returned the SCSI intermediate layer.
4. the implementation method of a kind of iSCSI storage system according to claim 1 is characterized in that: in the intermediate layer, a memory device mapping block is arranged between described front end destination drive module and IO load control module:
It provides memory device position, iSCSI controller IP address and starter IP address three's mapping relations, the control starter is to the authority of storage device access in the storage system, in view of the above, access rights are divided into three kinds of ranks: visit fully, read-only access and disable access;
It changes the authority planning that is set as follows form to the storage administrator:
[source IP address] [source address mask] [purpose IP address] [Target ID] [access rights]; The memory device mapping block provides the interface that writes or check planning, starts or stops the memory device mapping block to the storage administrator with the PROC form; The planning that memory device mapping block handle writes to the PROC interface resolves to the planning with above-mentioned chained list form tissue, and this module determines the authority of a starter to certain storage device access by inquiring about this chained list;
Described memory device has mapping block to realize the mapping flow process by following three phases successively:
In the device discovery stage, it contains following steps successively:
Receive the order of starter, judge whether to be the device discovery order, if not, then return; If then carry out next step;
Judge whether memory device in addition, if not, then return; If have, then carry out next step;
According to above rule judgment starter to authority that memory device identified;
Whether as seen to judge memory device according to authority,, then device identifier is inserted the back and return the memory device tabulation to starter if as seen to starter; Otherwise, return " whether also having memory device " step;
In the parameter negotiation stage, it contains following steps successively:
The memory device mapping block is obtained the address of the client and server of connection from starter;
The memory device mapping block judges whether this memory device is a memory device that starter is forbidden, if the memory device of forbidding, then this starter of active break connects to the network of iSCSI controller; Otherwise, carry out next stage IO operation;
The IO request stage, it contains following steps successively:
Judge whether scsi command is write order.Forward " for read command " if not to;
For write order, first allocation buffer, and the authority of this order is set according to rule, at last waiting list is put in order;
After receiving data, corresponding scsi command is taken out from waiting list;
The authority of sense command, differentiating this order could be performed.If then the distribution formation is put in this order; If not, abandon this order and structure SCSI response to the starter reporting errors;
For read command, the authority of this order is set according to rule directly;
The authority of sense command, differentiating this order could be performed.If then the distribution formation is put in this order; If not, abandon this order and structure SCSI response to the starter reporting errors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005100111791A CN100396065C (en) | 2005-01-14 | 2005-01-14 | A method for implementing iSCSI memory system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005100111791A CN100396065C (en) | 2005-01-14 | 2005-01-14 | A method for implementing iSCSI memory system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1633131A true CN1633131A (en) | 2005-06-29 |
| CN100396065C CN100396065C (en) | 2008-06-18 |
Family
ID=34853118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2005100111791A Expired - Fee Related CN100396065C (en) | 2005-01-14 | 2005-01-14 | A method for implementing iSCSI memory system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100396065C (en) |
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100442780C (en) * | 2006-01-27 | 2008-12-10 | 杭州华三通信技术有限公司 | iSCSI memory network, CD array and capacity extending method thereof |
| CN100454227C (en) * | 2005-11-21 | 2009-01-21 | 普安科技股份有限公司 | Data access methods and storage subsystems thereof |
| CN100456724C (en) * | 2005-12-28 | 2009-01-28 | 国际商业机器公司 | Data input/output request processing method and system |
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | IP storage control method based on iSCSI protocol and apparatus thereof |
| CN101778050A (en) * | 2010-03-11 | 2010-07-14 | 浪潮(北京)电子信息产业有限公司 | Load balancing method, device and system |
| CN101132418B (en) * | 2007-09-26 | 2010-11-24 | 杭州华三通信技术有限公司 | Method and apparatus for client logoff in iSCSI system |
| CN101938516A (en) * | 2010-09-02 | 2011-01-05 | 华中科技大学 | User-oriented dynamic storage resource distribution method |
| CN101419534B (en) * | 2007-10-22 | 2011-01-12 | 京瓷美达株式会社 | Information processing device and cirtual disk management method |
| CN101964812A (en) * | 2010-09-14 | 2011-02-02 | 浪潮(北京)电子信息产业有限公司 | Method, device and system for accessing resources |
| CN101227378B (en) * | 2008-01-24 | 2011-05-11 | 创新科存储技术（深圳）有限公司 | Communication service establishing method based on network storage and storage device thereof |
| CN101471947B (en) * | 2007-12-29 | 2012-07-04 | 英业达股份有限公司 | Storage server implemented through iSCSI in Linux system |
| CN101320332B (en) * | 2007-06-05 | 2012-07-04 | 英特尔公司 | Dynamic linking and loading of post-processing kernels |
| CN102546347A (en) * | 2010-12-17 | 2012-07-04 | 中兴通讯股份有限公司 | Implementation method and implementation system for hot plugging of virtual serial port based on proc technology |
| CN101557419B (en) * | 2009-05-06 | 2012-07-11 | 成都市华为赛门铁克科技有限公司 | Data reading and writing system and data management method |
| CN101626403B (en) * | 2009-07-31 | 2012-10-17 | 瑞红科技股份有限公司 | Magnetic disc virtualization method of iSCSI service |
| WO2012146034A1 (en) * | 2011-04-25 | 2012-11-01 | 中兴通讯股份有限公司 | Method and system for identifying storage device |
| CN101741878B (en) * | 2008-11-18 | 2012-12-12 | 英业达股份有限公司 | Network equipment capable of accelerating processing of iSCSI data packets |
| CN102831014A (en) * | 2012-07-30 | 2012-12-19 | 华中科技大学 | Method for prompting IO (input/output) parallelism and reducing small IO delay by utilizing multiple request queues |
| CN103023723A (en) * | 2012-12-14 | 2013-04-03 | 创新科软件技术（深圳）有限公司 | Method for performing iSCSI (internet small computer system interface) multipath test in linux environment |
| CN103257941A (en) * | 2013-04-17 | 2013-08-21 | 浪潮(北京)电子信息产业有限公司 | Multi-protocol storage controller and system |
| CN103795753A (en) * | 2012-10-31 | 2014-05-14 | 中国移动通信集团四川有限公司 | Method and system for achieving SAN network data equalized transmission |
| CN104077083A (en) * | 2014-07-01 | 2014-10-01 | 浪潮(北京)电子信息产业有限公司 | New added storage method, device and system |
| CN104144171A (en) * | 2013-05-06 | 2014-11-12 | 中国科学院声学研究所 | Implementation method for SCSI and SCSI subsystem |
| US8893160B2 (en) | 2008-06-09 | 2014-11-18 | International Business Machines Corporation | Block storage interface for virtual memory |
| CN104185972A (en) * | 2012-03-05 | 2014-12-03 | 日本电气株式会社 | Network system, switch, and network building method |
| CN104394163A (en) * | 2014-12-05 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | Security detection method based on Web application |
| CN104503709A (en) * | 2015-01-14 | 2015-04-08 | 浪潮(北京)电子信息产业有限公司 | Contention arbitrating method and system for shared disk of double-control storage array |
| CN104536845A (en) * | 2014-12-10 | 2015-04-22 | 深圳市锐明视讯技术有限公司 | Write cache data synchronization method and device |
| US9081912B2 (en) | 2011-11-24 | 2015-07-14 | Huawei Technologies Co., Ltd. | Method and apparatus for node hot-swapping |
| CN105279010A (en) * | 2015-10-20 | 2016-01-27 | 华为技术有限公司 | Virtual machine-based equipment heat replacing method and device |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN106201341A (en) * | 2016-06-30 | 2016-12-07 | 浪潮集团有限公司 | A kind of method replacing disk array based on server generation |
| CN106302598A (en) * | 2015-06-03 | 2017-01-04 | 南宁富桂精密工业有限公司 | Transmission method for optimizing route and system |
| CN106293502A (en) * | 2015-06-29 | 2017-01-04 | 联想(北京)有限公司 | A kind of collocation method, method for interchanging data and server system |
| CN106341488A (en) * | 2016-10-08 | 2017-01-18 | 深圳市云舒网络技术有限公司 | TCMU virtual block device-based network storage back-end timeout processing method |
| CN106648450A (en) * | 2015-11-04 | 2017-05-10 | 南京壹进制信息技术股份有限公司 | Method for realizing secure storage of data |
| CN106775992A (en) * | 2017-01-18 | 2017-05-31 | 清华大学 | A kind of redundancy computer system task compensative dispatching method |
| CN107317848A (en) * | 2017-05-25 | 2017-11-03 | 北京华云网际科技有限公司 | Reading/writing method, device and the system of I/O data |
| CN107704209A (en) * | 2017-10-26 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of disk scanning method, system, equipment and computer-readable storage medium |
| CN109144412A (en) * | 2018-07-26 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of iSCSI adapter batch scanning method and system |
| CN110515536A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Data-storage system |
| CN110515535A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Disk read-write control method, device, electronic equipment and storage medium |
| CN111625193A (en) * | 2020-05-22 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Configuration method, device, equipment and medium of multipath alias |
| CN111901399A (en) * | 2020-07-08 | 2020-11-06 | 苏州浪潮智能科技有限公司 | Cloud platform block equipment exception auditing method, device, equipment and storage medium |
| CN112015527A (en) * | 2019-05-29 | 2020-12-01 | 闪迪技术有限公司 | Managing retrieval and execution of commands from a submission queue |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571991B (en) * | 2012-03-27 | 2014-06-11 | 电子科技大学 | Multistage-mapping-based large-scale multi-copy distributed storage system and application method thereof |
| US10439927B2 (en) | 2018-01-31 | 2019-10-08 | International Business Machines Corporation | Distributed storage path configuration |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2377137B (en) * | 2001-06-27 | 2004-10-20 | Hewlett Packard Co | Network appliances |
| US7779428B2 (en) * | 2002-06-18 | 2010-08-17 | Symantec Operating Corporation | Storage resource integration layer interfaces |
| US20040049603A1 (en) * | 2002-09-05 | 2004-03-11 | International Business Machines Corporation | iSCSI driver to adapter interface protocol |
| US20040078521A1 (en) * | 2002-10-17 | 2004-04-22 | International Business Machines Corporation | Method, apparatus and computer program product for emulating an iSCSI device on a logical volume manager |
| US7581007B2 (en) * | 2003-03-11 | 2009-08-25 | Hitachi, Ltd. | Method, apparatus and services for leasing volumes |
-
2005
- 2005-01-14 CN CNB2005100111791A patent/CN100396065C/en not_active Expired - Fee Related
Cited By (68)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | IP storage control method based on iSCSI protocol and apparatus thereof |
| CN100454227C (en) * | 2005-11-21 | 2009-01-21 | 普安科技股份有限公司 | Data access methods and storage subsystems thereof |
| CN100456724C (en) * | 2005-12-28 | 2009-01-28 | 国际商业机器公司 | Data input/output request processing method and system |
| CN100442780C (en) * | 2006-01-27 | 2008-12-10 | 杭州华三通信技术有限公司 | iSCSI memory network, CD array and capacity extending method thereof |
| CN101320332B (en) * | 2007-06-05 | 2012-07-04 | 英特尔公司 | Dynamic linking and loading of post-processing kernels |
| CN101132418B (en) * | 2007-09-26 | 2010-11-24 | 杭州华三通信技术有限公司 | Method and apparatus for client logoff in iSCSI system |
| CN101419534B (en) * | 2007-10-22 | 2011-01-12 | 京瓷美达株式会社 | Information processing device and cirtual disk management method |
| CN101471947B (en) * | 2007-12-29 | 2012-07-04 | 英业达股份有限公司 | Storage server implemented through iSCSI in Linux system |
| CN101227378B (en) * | 2008-01-24 | 2011-05-11 | 创新科存储技术（深圳）有限公司 | Communication service establishing method based on network storage and storage device thereof |
| US8893160B2 (en) | 2008-06-09 | 2014-11-18 | International Business Machines Corporation | Block storage interface for virtual memory |
| CN101741878B (en) * | 2008-11-18 | 2012-12-12 | 英业达股份有限公司 | Network equipment capable of accelerating processing of iSCSI data packets |
| CN101557419B (en) * | 2009-05-06 | 2012-07-11 | 成都市华为赛门铁克科技有限公司 | Data reading and writing system and data management method |
| CN101626403B (en) * | 2009-07-31 | 2012-10-17 | 瑞红科技股份有限公司 | Magnetic disc virtualization method of iSCSI service |
| CN101778050A (en) * | 2010-03-11 | 2010-07-14 | 浪潮(北京)电子信息产业有限公司 | Load balancing method, device and system |
| CN101938516B (en) * | 2010-09-02 | 2012-11-14 | 华中科技大学 | User-oriented dynamic storage resource distribution method |
| CN101938516A (en) * | 2010-09-02 | 2011-01-05 | 华中科技大学 | User-oriented dynamic storage resource distribution method |
| CN101964812A (en) * | 2010-09-14 | 2011-02-02 | 浪潮(北京)电子信息产业有限公司 | Method, device and system for accessing resources |
| CN102546347A (en) * | 2010-12-17 | 2012-07-04 | 中兴通讯股份有限公司 | Implementation method and implementation system for hot plugging of virtual serial port based on proc technology |
| WO2012146034A1 (en) * | 2011-04-25 | 2012-11-01 | 中兴通讯股份有限公司 | Method and system for identifying storage device |
| US9281992B2 (en) | 2011-04-25 | 2016-03-08 | Zte Corporation | Method and system for identifying storage device |
| US9081912B2 (en) | 2011-11-24 | 2015-07-14 | Huawei Technologies Co., Ltd. | Method and apparatus for node hot-swapping |
| US9548933B2 (en) | 2012-03-05 | 2017-01-17 | Nec Corporation | Network system, switch, and methods of network configuration |
| CN104185972A (en) * | 2012-03-05 | 2014-12-03 | 日本电气株式会社 | Network system, switch, and network building method |
| CN102831014B (en) * | 2012-07-30 | 2016-05-25 | 华中科技大学 | A kind of method of utilizing many request queues to promote IO concurrency and reducing little IO delay |
| CN102831014A (en) * | 2012-07-30 | 2012-12-19 | 华中科技大学 | Method for prompting IO (input/output) parallelism and reducing small IO delay by utilizing multiple request queues |
| CN103795753A (en) * | 2012-10-31 | 2014-05-14 | 中国移动通信集团四川有限公司 | Method and system for achieving SAN network data equalized transmission |
| CN103023723B (en) * | 2012-12-14 | 2015-11-18 | 创新科软件技术（深圳）有限公司 | The method of iSCSI multipath test is carried out under a kind of linux environment |
| CN103023723A (en) * | 2012-12-14 | 2013-04-03 | 创新科软件技术（深圳）有限公司 | Method for performing iSCSI (internet small computer system interface) multipath test in linux environment |
| CN103257941A (en) * | 2013-04-17 | 2013-08-21 | 浪潮(北京)电子信息产业有限公司 | Multi-protocol storage controller and system |
| CN103257941B (en) * | 2013-04-17 | 2015-09-23 | 浪潮(北京)电子信息产业有限公司 | Multi-protocol storage controller and system |
| CN104144171A (en) * | 2013-05-06 | 2014-11-12 | 中国科学院声学研究所 | Implementation method for SCSI and SCSI subsystem |
| CN104144171B (en) * | 2013-05-06 | 2017-11-21 | 中国科学院声学研究所 | SCSI and iSCSI subsystems a kind of implementation method |
| CN104077083A (en) * | 2014-07-01 | 2014-10-01 | 浪潮(北京)电子信息产业有限公司 | New added storage method, device and system |
| CN104394163A (en) * | 2014-12-05 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | Security detection method based on Web application |
| CN104536845B (en) * | 2014-12-10 | 2018-02-23 | 深圳市锐明技术股份有限公司 | The method of data synchronization and device of write buffer |
| CN104536845A (en) * | 2014-12-10 | 2015-04-22 | 深圳市锐明视讯技术有限公司 | Write cache data synchronization method and device |
| CN104503709B (en) * | 2015-01-14 | 2018-05-22 | 浪潮(北京)电子信息产业有限公司 | The shared disk contention referee method and system of a kind of dual control storage array |
| CN104503709A (en) * | 2015-01-14 | 2015-04-08 | 浪潮(北京)电子信息产业有限公司 | Contention arbitrating method and system for shared disk of double-control storage array |
| CN106302598B (en) * | 2015-06-03 | 2019-07-23 | 南宁富桂精密工业有限公司 | Transmission path optimization method and system |
| CN106302598A (en) * | 2015-06-03 | 2017-01-04 | 南宁富桂精密工业有限公司 | Transmission method for optimizing route and system |
| CN106293502B (en) * | 2015-06-29 | 2019-09-24 | 联想(北京)有限公司 | A kind of configuration method, method for interchanging data and server system |
| CN106293502A (en) * | 2015-06-29 | 2017-01-04 | 联想(北京)有限公司 | A kind of collocation method, method for interchanging data and server system |
| US11102322B2 (en) | 2015-09-29 | 2021-08-24 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| US10708378B2 (en) | 2015-09-29 | 2020-07-07 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| CN105282244B (en) * | 2015-09-29 | 2018-10-02 | 华为技术有限公司 | a kind of data processing method, device, server and controller |
| CN105279010B (en) * | 2015-10-20 | 2018-09-21 | 华为技术有限公司 | The hot replacement method of equipment based on virtual machine and device |
| CN105279010A (en) * | 2015-10-20 | 2016-01-27 | 华为技术有限公司 | Virtual machine-based equipment heat replacing method and device |
| CN106648450B (en) * | 2015-11-04 | 2019-02-19 | 南京壹进制信息技术股份有限公司 | A kind of implementation method of data safety storage |
| CN106648450A (en) * | 2015-11-04 | 2017-05-10 | 南京壹进制信息技术股份有限公司 | Method for realizing secure storage of data |
| CN106201341A (en) * | 2016-06-30 | 2016-12-07 | 浪潮集团有限公司 | A kind of method replacing disk array based on server generation |
| CN106341488A (en) * | 2016-10-08 | 2017-01-18 | 深圳市云舒网络技术有限公司 | TCMU virtual block device-based network storage back-end timeout processing method |
| CN106775992A (en) * | 2017-01-18 | 2017-05-31 | 清华大学 | A kind of redundancy computer system task compensative dispatching method |
| CN106775992B (en) * | 2017-01-18 | 2020-03-27 | 清华大学 | Redundant computer system task compensation scheduling method |
| CN107317848A (en) * | 2017-05-25 | 2017-11-03 | 北京华云网际科技有限公司 | Reading/writing method, device and the system of I/O data |
| CN107317848B (en) * | 2017-05-25 | 2018-12-21 | 北京华云网际科技有限公司 | Reading/writing method, device and the system of I/O data |
| CN107704209A (en) * | 2017-10-26 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of disk scanning method, system, equipment and computer-readable storage medium |
| CN110515535B (en) * | 2018-05-22 | 2021-01-01 | 杭州海康威视数字技术股份有限公司 | Hard disk read-write control method and device, electronic equipment and storage medium |
| CN110515536A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Data-storage system |
| CN110515535A (en) * | 2018-05-22 | 2019-11-29 | 杭州海康威视数字技术股份有限公司 | Disk read-write control method, device, electronic equipment and storage medium |
| CN110515536B (en) * | 2018-05-22 | 2020-10-27 | 杭州海康威视数字技术股份有限公司 | Data storage system |
| CN109144412A (en) * | 2018-07-26 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of iSCSI adapter batch scanning method and system |
| CN112015527A (en) * | 2019-05-29 | 2020-12-01 | 闪迪技术有限公司 | Managing retrieval and execution of commands from a submission queue |
| CN112015527B (en) * | 2019-05-29 | 2024-03-22 | 闪迪技术有限公司 | Managing fetching and executing commands from a commit queue |
| CN111625193A (en) * | 2020-05-22 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Configuration method, device, equipment and medium of multipath alias |
| CN111901399A (en) * | 2020-07-08 | 2020-11-06 | 苏州浪潮智能科技有限公司 | Cloud platform block equipment exception auditing method, device, equipment and storage medium |
| CN111901399B (en) * | 2020-07-08 | 2022-12-09 | 苏州浪潮智能科技有限公司 | Cloud platform block equipment exception auditing method, device, equipment and storage medium |
| US11782609B2 (en) | 2020-07-08 | 2023-10-10 | Inspur Suzhou Intelligent Technology Co., Ltd. | Method and apparatus for auditing abnormality of block device in cloud platform, device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100396065C (en) | 2008-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1633131A (en) | A method for implementing iSCSI memory system | |
| CN1241134C (en) | Storage field management system | |
| CN1959618A (en) | Storage system | |
| CN1684029A (en) | Storage system | |
| CN1869914A (en) | Storage system and operation method of storage system | |
| CN1581877A (en) | Storage system | |
| CN1311328C (en) | Storage device | |
| CN101042676A (en) | Storage system, storage extent release method and storage apparatus | |
| CN1272946C (en) | Expansible network processor and method and device for operating the network processor | |
| CN1204509C (en) | Network processor, memory organization and methods | |
| CN1453715A (en) | Storing system having vitual source | |
| CN101064656A (en) | Storage cluster system, data processing method, and program | |
| CN1794729A (en) | Data arrangement management method, data arrangement management system, data arrangement management device, and data arrangement management program | |
| CN1269056C (en) | Virtual centralized method and apparatus for multi-network memory | |
| CN1204503C (en) | Network processor processing complex and methods | |
| CN1239984C (en) | VLSI network processor and method | |
| CN1299207C (en) | Large scale resource memory managing method based on network under SAN environment | |
| CN1387125A (en) | Memory system | |
| CN1265274C (en) | Storage device control device and control method of storage device control device | |
| CN101061476A (en) | Online initial mirror synchronization and mirror synchronization verification in storage area networks | |
| CN1637714A (en) | Remote copy network | |
| CN1291747A (en) | Cache device and its application | |
| CN1773510A (en) | Memory management system,method and program | |
| CN1945520A (en) | Data management method in storage pool and virtual volume in dkc | |
| CN101075177A (en) | Computer systeme |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080618 Termination date: 20110114 |