CN1543092A - SCSI command optimization method based on FC-SAN storage device - Google Patents
SCSI command optimization method based on FC-SAN storage device Download PDFInfo
- Publication number
- CN1543092A CN1543092A CNA2003101031969A CN200310103196A CN1543092A CN 1543092 A CN1543092 A CN 1543092A CN A2003101031969 A CNA2003101031969 A CN A2003101031969A CN 200310103196 A CN200310103196 A CN 200310103196A CN 1543092 A CN1543092 A CN 1543092A
- Authority
- CN
- China
- Prior art keywords
- command
- scsi
- function
- commands
- scsi command
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000005457 optimization Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims description 32
- 239000013307 optical fiber Substances 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 2
- 230000037430 deletion Effects 0.000 claims description 2
- 239000000835 fiber Substances 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
基于FC－SAN存储设备的SCSI命令优化方法属于FC－SAN即光纤通道——存储区域网络技术领域，其特征在于：它是针对专用I/O处理机的，它针对队列中的操作设备、操作命令、操作地址和等待时间，对每一个SCSI队列命令中的SCSI命令进行判断：是否存在由不同用户发送的或者同一用户发送的重复命令、是否存在SCSI命令的等待时间大于程序规定的最大时间、是否存在这些SCSI命令发往同一设备且地址又近邻的情况，分别进行重复命令删除处理、超时命令延迟处理近邻命令合并处理的操作，以解决处理SCSI命令时的延缓繁杂问题。经过优化后，吞吐量可以得到明显增长。
The SCSI command optimization method based on FC-SAN storage device belongs to FC-SAN namely fiber channel--storage area network technology field, it is characterized in that: it is aimed at special-purpose I/O processing machine, and it is aimed at the operation equipment in the queue, operation Command, operation address and waiting time, judge the SCSI commands in each SCSI queue command: whether there are repeated commands sent by different users or the same user, whether there are SCSI commands whose waiting time is greater than the maximum time specified by the program, Whether these SCSI commands are sent to the same device and the addresses are adjacent to each other, the operation of deleting duplicate commands, delaying processing of timeout commands, and merging adjacent commands to solve the complicated problem of delaying SCSI commands is solved. After optimization, the throughput can be significantly increased.
Description
Technical field
Belonging to FC-SAN based on the scsi command optimization method of FC-SAN memory device is optical-fibre channel---the storage area network technical field.
Background technology
Because the high transfer rate of optical-fibre channel, support multiple transport protocols, based on characteristics such as the low expense of block I/O access, remote accesses, become the main flow of network store system based on storage area network (SAN) storage system of optical-fibre channel.Characteristics such as particularly multi-user's parallel access, equipment are shared, data sharing, optical-fibre channel SAN (FC-SAN) storage system need realize features such as enhanced scalability, high-performance, and guarantees multi-user's access performance demands such as I/O request response time.
Under multi-user's parallel environment, the user can share FC-SAN memory device arbitrarily, and wherein each kind of data.Not only exist because the problem that the increase of client causes operating load stream to increase, and may exist a plurality of users in a period of time, to concentrate same disk of access or same blocks of data etc. and the disk hot spot phenomenon that occurs.
Therefore the I/O access performance that how to guarantee the FC-SAN storage system is the major issue under multi-user's parallel environment.According to our The performance test results, the performance bottle of present FC-SAN storage system footpath mainly is present in the SAN memory device.But yet there are no the fine granularity optimization method that flows feature under the concurrent user environment according to scsi command based on the FC-SAN memory device, the method that we propose has taken into full account the scsi command stream feature under the parallel environment, has improved the I/O performance of system effectively.
Summary of the invention
The object of the present invention is to provide a kind of small computer system interface that is conceived to solve memory device one side is the scsi command optimization method based on the FC-SAN memory device that scsi command is handled complicated inefficiency problem.
The invention is characterized in:
It is by being stored in optical-fibre channel---storage area equipment is that the small computer system interface in the memory node controller among the FC-SAN is that the scsi target simulator is carried out at the special I processor.It contains successively and has the following steps:
(1) set:
The data structure of queue command, it contains successively:
Operational order int Top
Operating equipment int Tlun
Operation address int Taddr
Stand-by period int Ttime
Executing state int Tcmdstatus
Follow-up command pointer struct TSCThread Node*next
Wherein, TCT is a TSC transmitter start code,
Node is a data structure
Interface function:
The optimal conditions discriminant function,
Iterated command judgement and processing function,
Overtime judgement and processing function,
Neighbour's command determination and processing function,
(2) carry out as follows successively
(2.1) judge whether and can optimize with containing queue command Parameter Optimization condition judgment function,, carry out according to the following steps successively if can optimize,
(2.2) judge whether to exist the scsi command of the repetition that a plurality of users that sent by different user or same send with the iterated command discriminant function, if exist, then processing threads is only dispatched this scsi command with corresponding pooled function processing once, and the order of other repetition is deleted; Otherwise, change next step over to.
(2.3) judge whether the situation whether stand-by period in the queue command parameter requires greater than the processing time with overtime discriminant function, if exist, then processing threads postpones the back that these orders are put into this scsi command to handle with the timeout treatment function according to its start address and skew, otherwise, change next step over to.
(2.4) judge with closing on the command determination function whether the scsi command in each scsi command formation exists correlation on the access address, promptly visiting same device address closes on, if exist, processing threads is merged into a scsi command that uses continuation address with handling the scsi command of function with a plurality of address correlations accordingly, and other order deletion.
If with the throughput data before optimizing is that standard compares, still be medium-term and long-term weather forecast treatment system no matter for the video system of coaching, the throughput after its optimization all has phenomenal growth.
Description of drawings
The topological diagram of Fig. 1 optical fiber storage networking
The location drawing of Fig. 2 memory node controller THMSNSc102 in memory device
The complete I/O path of Fig. 3 SAN storage system
Fig. 4 scsi target simulator structural representation
The program flow diagram of Fig. 5 optimization method of the present invention
Queue structure's schematic diagram of Fig. 6 scsi command
Embodiment
Fig. 1 ~ Fig. 4 is the special I/position of O processor THMSNSc102 in the FC-SAN storage area network.
This method is implemented in special I/O processor THMSNSc102 promptly on the memory node controller.Its hardware environment and operating system are as follows:
| CPU | 2xXeon2GHz |
| Internal memory | 1GB |
| The fiber optic network interface card | Qlogic?qla2310F |
| The SCSI-RAID card | Adapter3410s |
| Operating system | Linux?2.4.18-3smp |
| The RAID disk array | 14xSeagatex73G(10Krpm)SCSI?Disks |
The queue structure of scsi command as shown in Figure 6
This method is used following operation and function (parameter is illustrated with definition in Fig. 6 structure):
The optimal conditions discriminant function:
int?scsi_target_cmd_switch(TSCThreadQueue*ttg，TSCThreadLength?tl)，
The iterated command discriminant function:
Tcompare(ttg-＞Top，ttg-＞Tlun，ttg-＞Taddr，i)，
The iterated command pooled function
scsi_target_cmd_combin(ttg，i)
Overtime judgement
Ttg-＞Ttime＞Tmaxtime Tmaxtime is the maximum processing time of program setting
The timeout treatment function
scsi_target_cmd_timelg(ttg，i)
Neighbour's command determination function
Tnearby(ttg-＞Top，ttg-＞Tlun)
Neighbour's command processing function:
scsi_target_cmd_nearby(ttg，i)
The scsi target simulator is a concentrated I/O request control and handles the position, by each scsi command in the scsi command formation, can clearly discern details such as the position of data block of all clients that send scsi commands, access and block size.It goes to carry out method of the present invention by program circuit shown in Figure 5.
Major advantage based on the scsi command optimization method of FC-SAN memory device is as follows:
1, consider the characteristics of a plurality of user side access adjacent data blocks, this method is carried out the access address and is merged processing, and a plurality of scsi commands are merged into a scsi command, has reduced the physical treatment times such as disk tracking effectively, has improved the I/O handling property.
2, this method has been considered the I/O access performance and the fair principle of scsi command, and all scsi commands can effectively be handled.
3, this method has taken into full account the characteristics of FC-SAN memory device buffer memory CACHE, effectively utilizes CACHE mechanism on the one hand, has solved the read-write problem of big data block on the one hand again effectively.
4, this method can be discerned in the same command queue scsi command from the same characteristic features of different clients, and carries out disposable processing, has reduced by N-1 operation.
5, this method has been considered the feature of disk access, has reduced physical operations number of times such as disk tracking as much as possible.Also adopt related method thereof to solve HOT SPOT problem simultaneously, improved the I/O access performance effectively.
Under identical test environment, employing is based on the special I processor THMSNSc102 of the scsi command optimization method of FC-SAN memory device, can improve the performance of the FC-SAN storage system about 10-20% effectively, and and client user's number and operating load have bigger relation: 1) if under the environment of large-scale storage system, this method can more effectively improve the I/O access performance of system; 2) if there is bigger correlation in the operating load of user side, then systematic function is improved more obvious.Another outstanding feature of this method is irrelevant with architecture and specific implementation, can be applied in the different scsi target simulator architectures.
The present invention (special I processor THMSNSc102) has passed through test on No. hundred billion parallel computers of being researched and developed voluntarily by Tsing-Hua University, and carrying out application such as video on-demand system, medium-term and long-term data of weather forecast treatment system on this basis, performance all is significantly improved behind the employing said method.
With the throughput data before optimizing is that standard value 100% compares
| Throughput before optimizing | Optimize the back throughput | |
| Video on-demand system | 100％ | 125％ |
| Medium-term and long-term data of weather forecast treatment system | 100％ | 115％ |
Claims (1)
- The 1 scsi command optimization method based on the FC-SAN memory device is characterized in that: it is by being stored in optical-fibre channel---storage area equipment is that the small computer system interface in the memory node controller among the FC-SAN is that the scsi target simulator is carried out at the special I processor.It contains successively and has the following steps:(1) set:The data structure of queue command, it contains successively:Operational order int TopOperating equipment int TlunOperation address int TaddrStand-by period int TtimeExecuting state int TcmdstatusFollow-up command pointer struct TSCThread Node*nextWherein, TCT is a TSC transmitter start code,Node is a data structureInterface function:The optimal conditions discriminant function,Iterated command judgement and processing function,Overtime judgement and processing function,Neighbour's command determination and processing function,(2) carry out as follows successively(2.1) judge whether and can optimize with containing queue command Parameter Optimization condition judgment function,, carry out according to the following steps successively if can optimize,(2.2) judge whether to exist the scsi command of the repetition that a plurality of users that sent by different user or same send with the iterated command discriminant function, if exist, then processing threads is only dispatched this scsi command with corresponding pooled function processing once, and the order of other repetition is deleted; Otherwise, change next step over to;(2.3) judge whether the situation whether stand-by period in the queue command parameter requires greater than the processing time with overtime discriminant function, if exist, then processing threads postpones the back that these orders are put into this scsi command to handle with the timeout treatment function according to its start address and skew, otherwise, change next step over to;(2.4) judge with closing on the command determination function whether the scsi command in each scsi command formation exists correlation on the access address, promptly visiting same device address closes on, if exist, processing threads is merged into a scsi command that uses continuation address with handling the scsi command of function with a plurality of address correlations accordingly, and other order deletion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2003101031969A CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI command optimization method based on FC-SAN storage device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2003101031969A CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI command optimization method based on FC-SAN storage device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1543092A true CN1543092A (en) | 2004-11-03 |
Family
ID=34333236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2003101031969A Pending CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI command optimization method based on FC-SAN storage device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1543092A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100362790C (en) * | 2005-08-15 | 2008-01-16 | 华中科技大学 | A core storage exchange platform system for mass storage system |
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | An IP storage control method and device based on iSCSI protocol |
| CN102739440A (en) * | 2012-05-24 | 2012-10-17 | 大唐移动通信设备有限公司 | Method and device for accessing hardware device |
| CN103617136A (en) * | 2013-12-04 | 2014-03-05 | 华为技术有限公司 | SCSI drive side and I/O request control method |
| WO2014059781A1 (en) * | 2012-10-15 | 2014-04-24 | 华为技术有限公司 | Address compression and decompression method, compressor and decompressor |
| CN104035923A (en) * | 2013-03-04 | 2014-09-10 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| WO2014146497A1 (en) * | 2013-03-21 | 2014-09-25 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| CN104572376A (en) * | 2015-01-27 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Method for verifying server in compliance with vSAN standard |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN106155586A (en) * | 2016-05-31 | 2016-11-23 | 华为技术有限公司 | A kind of storage method, server and storage control |
| CN107181636A (en) * | 2016-03-10 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Health examination method and device in a kind of SiteServer LBS |
| CN116401236A (en) * | 2023-06-07 | 2023-07-07 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
-
2003
- 2003-11-07 CN CNA2003101031969A patent/CN1543092A/en active Pending
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100362790C (en) * | 2005-08-15 | 2008-01-16 | 华中科技大学 | A core storage exchange platform system for mass storage system |
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | An IP storage control method and device based on iSCSI protocol |
| CN102739440A (en) * | 2012-05-24 | 2012-10-17 | 大唐移动通信设备有限公司 | Method and device for accessing hardware device |
| US9495304B2 (en) | 2012-10-15 | 2016-11-15 | Huawei Technologies Co., Ltd. | Address compression method, address decompression method, compressor, and decompressor |
| WO2014059781A1 (en) * | 2012-10-15 | 2014-04-24 | 华为技术有限公司 | Address compression and decompression method, compressor and decompressor |
| CN104035923B (en) * | 2013-03-04 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Inquire about the method and device of data |
| CN104035923A (en) * | 2013-03-04 | 2014-09-10 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| US9378009B2 (en) | 2013-03-21 | 2016-06-28 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| WO2014146497A1 (en) * | 2013-03-21 | 2014-09-25 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| CN103617136A (en) * | 2013-12-04 | 2014-03-05 | 华为技术有限公司 | SCSI drive side and I/O request control method |
| CN104572376A (en) * | 2015-01-27 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Method for verifying server in compliance with vSAN standard |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN105282244B (en) * | 2015-09-29 | 2018-10-02 | 华为技术有限公司 | a kind of data processing method, device, server and controller |
| US10708378B2 (en) | 2015-09-29 | 2020-07-07 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| US11102322B2 (en) | 2015-09-29 | 2021-08-24 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| CN107181636A (en) * | 2016-03-10 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Health examination method and device in a kind of SiteServer LBS |
| CN106155586A (en) * | 2016-05-31 | 2016-11-23 | 华为技术有限公司 | A kind of storage method, server and storage control |
| CN116401236A (en) * | 2023-06-07 | 2023-07-07 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
| CN116401236B (en) * | 2023-06-07 | 2023-08-18 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9304928B2 (en) | Systems and methods for adaptive prefetching | |
| Wang et al. | An efficient design and implementation of LSM-tree based key-value store on open-channel SSD | |
| US20120144099A1 (en) | Device driver deployment of similarity-based delta compression for use in a data storage system | |
| CN111143234A (en) | Storage device, system including such storage device and method of operating the same | |
| CN1543092A (en) | SCSI command optimization method based on FC-SAN storage device | |
| US20200319797A1 (en) | System and method for file processing from a block device | |
| CN105589664A (en) | Virtual storage high-speed transmission method | |
| CN112256599A (en) | Data prefetching method and device and storage device | |
| Park et al. | A lookahead read cache: improving read performance for deduplication backup storage | |
| WO2012109145A2 (en) | Pre-cache similarity-based delta compression for use in a data storage system | |
| Bhimani et al. | FIOS: Feature based I/O stream identification for improving endurance of multi-stream SSDs | |
| CN116910314A (en) | Method and device for optimizing range query in key value storage system based on key value separation | |
| CN103200245B (en) | A kind of distributed network caching method based on Device Mapper | |
| US20190057121A1 (en) | Predictable Allocation Latency in Fragmented Log Structured File Systems | |
| US9830097B2 (en) | Application-specific chunk-aligned prefetch for sequential workloads | |
| CN1848065A (en) | A Heterogeneous Dual System Bus Object Storage Controller | |
| CN1220950C (en) | Controller for outer multi-channel network disc array and its protocol fitting method | |
| Geoffray | Opiom: Off-processor I/O with myrinet | |
| Zhu et al. | Performance analysis and testing of the storage area network | |
| CN111338570B (en) | Parallel file system IO optimization method and system | |
| Nijim et al. | Multi-layer prefetching for hybrid storage systems: algorithms, models, and evaluations | |
| Sun et al. | A storage computing architecture with multiple NDP devices for accelerating compaction performance in LSM-tree based KV stores | |
| CN116069752A (en) | Mixed prefetching method for distributed file system | |
| Shu et al. | A highly efficient FC-SAN based on load stream | |
| CN1225698C (en) | Regenerative evolution method for data on storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |