CN1543092A - SCSI instruction optimization method based on FC-SAN storage device - Google Patents
SCSI instruction optimization method based on FC-SAN storage device Download PDFInfo
- Publication number
- CN1543092A CN1543092A CNA2003101031969A CN200310103196A CN1543092A CN 1543092 A CN1543092 A CN 1543092A CN A2003101031969 A CNA2003101031969 A CN A2003101031969A CN 200310103196 A CN200310103196 A CN 200310103196A CN 1543092 A CN1543092 A CN 1543092A
- Authority
- CN
- China
- Prior art keywords
- command
- scsi
- function
- scsi command
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000005457 optimization Methods 0.000 title claims description 12
- 230000006870 function Effects 0.000 claims description 32
- 238000012545 processing Methods 0.000 claims description 20
- 239000013307 optical fiber Substances 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 2
- 230000037430 deletion Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The SCSI command optimizing method based on PC-SAN storage devices characterizes that it is directed against the special I/O processor and the operation device, commands, operation addresses and waiting time to judge the command in every SCSI queue to see if there are repeat commands sent by different users or a same user, if the waiting time of SCSI commands is longer than the longest time stimulated by the program, if these SCSI commands are sent to a same device and addresses are close then it deletes the repeat commands, prolongs the suppertime commands and merges the adjacent commands to solve the complicated problems in dealing with SCSI commands.
Description
Technical field
Belonging to FC-SAN based on the scsi command optimization method of FC-SAN memory device is optical-fibre channel---the storage area network technical field.
Background technology
Because the high transfer rate of optical-fibre channel, support multiple transport protocols, based on characteristics such as the low expense of block I/O access, remote accesses, become the main flow of network store system based on storage area network (SAN) storage system of optical-fibre channel.Characteristics such as particularly multi-user's parallel access, equipment are shared, data sharing, optical-fibre channel SAN (FC-SAN) storage system need realize features such as enhanced scalability, high-performance, and guarantees multi-user's access performance demands such as I/O request response time.
Under multi-user's parallel environment, the user can share FC-SAN memory device arbitrarily, and wherein each kind of data.Not only exist because the problem that the increase of client causes operating load stream to increase, and may exist a plurality of users in a period of time, to concentrate same disk of access or same blocks of data etc. and the disk hot spot phenomenon that occurs.
Therefore the I/O access performance that how to guarantee the FC-SAN storage system is the major issue under multi-user's parallel environment.According to our The performance test results, the performance bottle of present FC-SAN storage system footpath mainly is present in the SAN memory device.But yet there are no the fine granularity optimization method that flows feature under the concurrent user environment according to scsi command based on the FC-SAN memory device, the method that we propose has taken into full account the scsi command stream feature under the parallel environment, has improved the I/O performance of system effectively.
Summary of the invention
The object of the present invention is to provide a kind of small computer system interface that is conceived to solve memory device one side is the scsi command optimization method based on the FC-SAN memory device that scsi command is handled complicated inefficiency problem.
The invention is characterized in:
It is by being stored in optical-fibre channel---storage area equipment is that the small computer system interface in the memory node controller among the FC-SAN is that the scsi target simulator is carried out at the special I processor.It contains successively and has the following steps:
(1) set:
The data structure of queue command, it contains successively:
Operational order int Top
Operating equipment int Tlun
Operation address int Taddr
Stand-by period int Ttime
Executing state int Tcmdstatus
Follow-up command pointer struct TSCThread Node*next
Wherein, TCT is a TSC transmitter start code,
Node is a data structure
Interface function:
The optimal conditions discriminant function,
Iterated command judgement and processing function,
Overtime judgement and processing function,
Neighbour's command determination and processing function,
(2) carry out as follows successively
(2.1) judge whether and can optimize with containing queue command Parameter Optimization condition judgment function,, carry out according to the following steps successively if can optimize,
(2.2) judge whether to exist the scsi command of the repetition that a plurality of users that sent by different user or same send with the iterated command discriminant function, if exist, then processing threads is only dispatched this scsi command with corresponding pooled function processing once, and the order of other repetition is deleted; Otherwise, change next step over to.
(2.3) judge whether the situation whether stand-by period in the queue command parameter requires greater than the processing time with overtime discriminant function, if exist, then processing threads postpones the back that these orders are put into this scsi command to handle with the timeout treatment function according to its start address and skew, otherwise, change next step over to.
(2.4) judge with closing on the command determination function whether the scsi command in each scsi command formation exists correlation on the access address, promptly visiting same device address closes on, if exist, processing threads is merged into a scsi command that uses continuation address with handling the scsi command of function with a plurality of address correlations accordingly, and other order deletion.
If with the throughput data before optimizing is that standard compares, still be medium-term and long-term weather forecast treatment system no matter for the video system of coaching, the throughput after its optimization all has phenomenal growth.
Description of drawings
The topological diagram of Fig. 1 optical fiber storage networking
The location drawing of Fig. 2 memory node controller THMSNSc102 in memory device
The complete I/O path of Fig. 3 SAN storage system
Fig. 4 scsi target simulator structural representation
The program flow diagram of Fig. 5 optimization method of the present invention
Queue structure's schematic diagram of Fig. 6 scsi command
Embodiment
Fig. 1 ~ Fig. 4 is the special I/position of O processor THMSNSc102 in the FC-SAN storage area network.
This method is implemented in special I/O processor THMSNSc102 promptly on the memory node controller.Its hardware environment and operating system are as follows:
| CPU | 2xXeon2GHz |
| Internal memory | 1GB |
| The fiber optic network interface card | Qlogic?qla2310F |
| The SCSI-RAID card | Adapter3410s |
| Operating system | Linux?2.4.18-3smp |
| The RAID disk array | 14xSeagatex73G(10Krpm)SCSI?Disks |
The queue structure of scsi command as shown in Figure 6
This method is used following operation and function (parameter is illustrated with definition in Fig. 6 structure):
The optimal conditions discriminant function:
int?scsi_target_cmd_switch(TSCThreadQueue*ttg，TSCThreadLength?tl)，
The iterated command discriminant function:
Tcompare(ttg-＞Top，ttg-＞Tlun，ttg-＞Taddr，i)，
The iterated command pooled function
scsi_target_cmd_combin(ttg，i)
Overtime judgement
Ttg-＞Ttime＞Tmaxtime Tmaxtime is the maximum processing time of program setting
The timeout treatment function
scsi_target_cmd_timelg(ttg，i)
Neighbour's command determination function
Tnearby(ttg-＞Top，ttg-＞Tlun)
Neighbour's command processing function:
scsi_target_cmd_nearby(ttg，i)
The scsi target simulator is a concentrated I/O request control and handles the position, by each scsi command in the scsi command formation, can clearly discern details such as the position of data block of all clients that send scsi commands, access and block size.It goes to carry out method of the present invention by program circuit shown in Figure 5.
Major advantage based on the scsi command optimization method of FC-SAN memory device is as follows:
1, consider the characteristics of a plurality of user side access adjacent data blocks, this method is carried out the access address and is merged processing, and a plurality of scsi commands are merged into a scsi command, has reduced the physical treatment times such as disk tracking effectively, has improved the I/O handling property.
2, this method has been considered the I/O access performance and the fair principle of scsi command, and all scsi commands can effectively be handled.
3, this method has taken into full account the characteristics of FC-SAN memory device buffer memory CACHE, effectively utilizes CACHE mechanism on the one hand, has solved the read-write problem of big data block on the one hand again effectively.
4, this method can be discerned in the same command queue scsi command from the same characteristic features of different clients, and carries out disposable processing, has reduced by N-1 operation.
5, this method has been considered the feature of disk access, has reduced physical operations number of times such as disk tracking as much as possible.Also adopt related method thereof to solve HOT SPOT problem simultaneously, improved the I/O access performance effectively.
Under identical test environment, employing is based on the special I processor THMSNSc102 of the scsi command optimization method of FC-SAN memory device, can improve the performance of the FC-SAN storage system about 10-20% effectively, and and client user's number and operating load have bigger relation: 1) if under the environment of large-scale storage system, this method can more effectively improve the I/O access performance of system; 2) if there is bigger correlation in the operating load of user side, then systematic function is improved more obvious.Another outstanding feature of this method is irrelevant with architecture and specific implementation, can be applied in the different scsi target simulator architectures.
The present invention (special I processor THMSNSc102) has passed through test on No. hundred billion parallel computers of being researched and developed voluntarily by Tsing-Hua University, and carrying out application such as video on-demand system, medium-term and long-term data of weather forecast treatment system on this basis, performance all is significantly improved behind the employing said method.
With the throughput data before optimizing is that standard value 100% compares
| Throughput before optimizing | Optimize the back throughput | |
| Video on-demand system | 100％ | 125％ |
| Medium-term and long-term data of weather forecast treatment system | 100％ | 115％ |
Claims (1)
- The 1 scsi command optimization method based on the FC-SAN memory device is characterized in that: it is by being stored in optical-fibre channel---storage area equipment is that the small computer system interface in the memory node controller among the FC-SAN is that the scsi target simulator is carried out at the special I processor.It contains successively and has the following steps:(1) set:The data structure of queue command, it contains successively:Operational order int TopOperating equipment int TlunOperation address int TaddrStand-by period int TtimeExecuting state int TcmdstatusFollow-up command pointer struct TSCThread Node*nextWherein, TCT is a TSC transmitter start code,Node is a data structureInterface function:The optimal conditions discriminant function,Iterated command judgement and processing function,Overtime judgement and processing function,Neighbour's command determination and processing function,(2) carry out as follows successively(2.1) judge whether and can optimize with containing queue command Parameter Optimization condition judgment function,, carry out according to the following steps successively if can optimize,(2.2) judge whether to exist the scsi command of the repetition that a plurality of users that sent by different user or same send with the iterated command discriminant function, if exist, then processing threads is only dispatched this scsi command with corresponding pooled function processing once, and the order of other repetition is deleted; Otherwise, change next step over to;(2.3) judge whether the situation whether stand-by period in the queue command parameter requires greater than the processing time with overtime discriminant function, if exist, then processing threads postpones the back that these orders are put into this scsi command to handle with the timeout treatment function according to its start address and skew, otherwise, change next step over to;(2.4) judge with closing on the command determination function whether the scsi command in each scsi command formation exists correlation on the access address, promptly visiting same device address closes on, if exist, processing threads is merged into a scsi command that uses continuation address with handling the scsi command of function with a plurality of address correlations accordingly, and other order deletion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2003101031969A CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI instruction optimization method based on FC-SAN storage device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2003101031969A CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI instruction optimization method based on FC-SAN storage device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1543092A true CN1543092A (en) | 2004-11-03 |
Family
ID=34333236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2003101031969A Pending CN1543092A (en) | 2003-11-07 | 2003-11-07 | SCSI instruction optimization method based on FC-SAN storage device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1543092A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100362790C (en) * | 2005-08-15 | 2008-01-16 | 华中科技大学 | Kern memory exchange platform system for mass memory system |
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | IP storage control method based on iSCSI protocol and apparatus thereof |
| CN102739440A (en) * | 2012-05-24 | 2012-10-17 | 大唐移动通信设备有限公司 | Method and device for accessing hardware device |
| CN103617136A (en) * | 2013-12-04 | 2014-03-05 | 华为技术有限公司 | SCSI drive side and I/O request control method |
| WO2014059781A1 (en) * | 2012-10-15 | 2014-04-24 | 华为技术有限公司 | Address compression and decompression method, compressor and decompressor |
| CN104035923A (en) * | 2013-03-04 | 2014-09-10 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| WO2014146497A1 (en) * | 2013-03-21 | 2014-09-25 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| CN104572376A (en) * | 2015-01-27 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Method for verifying server in compliance with vSAN standard |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN106155586A (en) * | 2016-05-31 | 2016-11-23 | 华为技术有限公司 | A kind of storage method, server and storage control |
| CN107181636A (en) * | 2016-03-10 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Health examination method and device in a kind of SiteServer LBS |
| CN116401236A (en) * | 2023-06-07 | 2023-07-07 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
-
2003
- 2003-11-07 CN CNA2003101031969A patent/CN1543092A/en active Pending
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100362790C (en) * | 2005-08-15 | 2008-01-16 | 华中科技大学 | Kern memory exchange platform system for mass memory system |
| CN1753406B (en) * | 2005-10-26 | 2010-06-30 | 华中科技大学 | IP storage control method based on iSCSI protocol and apparatus thereof |
| CN102739440A (en) * | 2012-05-24 | 2012-10-17 | 大唐移动通信设备有限公司 | Method and device for accessing hardware device |
| US9495304B2 (en) | 2012-10-15 | 2016-11-15 | Huawei Technologies Co., Ltd. | Address compression method, address decompression method, compressor, and decompressor |
| WO2014059781A1 (en) * | 2012-10-15 | 2014-04-24 | 华为技术有限公司 | Address compression and decompression method, compressor and decompressor |
| CN104035923B (en) * | 2013-03-04 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Inquire about the method and device of data |
| CN104035923A (en) * | 2013-03-04 | 2014-09-10 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| US9378009B2 (en) | 2013-03-21 | 2016-06-28 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| WO2014146497A1 (en) * | 2013-03-21 | 2014-09-25 | Tencent Technology (Shenzhen) Company Limited | Method and device for accelerating running of software |
| CN103617136A (en) * | 2013-12-04 | 2014-03-05 | 华为技术有限公司 | SCSI drive side and I/O request control method |
| CN104572376A (en) * | 2015-01-27 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Method for verifying server in compliance with vSAN standard |
| CN105282244A (en) * | 2015-09-29 | 2016-01-27 | 华为技术有限公司 | Data processing method and device, server and controller |
| CN105282244B (en) * | 2015-09-29 | 2018-10-02 | 华为技术有限公司 | a kind of data processing method, device, server and controller |
| US10708378B2 (en) | 2015-09-29 | 2020-07-07 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| US11102322B2 (en) | 2015-09-29 | 2021-08-24 | Huawei Technologies Co., Ltd. | Data processing method and apparatus, server, and controller |
| CN107181636A (en) * | 2016-03-10 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Health examination method and device in a kind of SiteServer LBS |
| CN106155586A (en) * | 2016-05-31 | 2016-11-23 | 华为技术有限公司 | A kind of storage method, server and storage control |
| CN116401236A (en) * | 2023-06-07 | 2023-07-07 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
| CN116401236B (en) * | 2023-06-07 | 2023-08-18 | 瀚高基础软件股份有限公司 | Method and equipment for adaptively optimizing database parameters |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9304928B2 (en) | Systems and methods for adaptive prefetching | |
| US10970202B1 (en) | Managing input/output (‘I/O’) requests in a storage system that includes multiple types of storage devices | |
| US9582222B2 (en) | Pre-cache similarity-based delta compression for use in a data storage system | |
| CN111143234B (en) | Storage device, system comprising such a storage device and method of operating the same | |
| US9141648B1 (en) | Management of database blocks | |
| US20230409546A1 (en) | Compression Algorithm Based On Resource Availability | |
| CN1543092A (en) | SCSI instruction optimization method based on FC-SAN storage device | |
| CN1545030A (en) | Data distribution dynamic mapping method based on magnetic disc characteristic | |
| US20200319797A1 (en) | System and method for file processing from a block device | |
| Park et al. | A lookahead read cache: improving read performance for deduplication backup storage | |
| WO2012109145A2 (en) | Pre-cache similarity-based delta compression for use in a data storage system | |
| CN115470157A (en) | Prefetching method, electronic device, storage medium, and program product | |
| US9830097B2 (en) | Application-specific chunk-aligned prefetch for sequential workloads | |
| CN1220950C (en) | Controller for outer multi-channel network disc array and its protocol fitting method | |
| CN116910314A (en) | Method and device for optimizing range query in key value storage system based on key value separation | |
| Geoffray | Opiom: Off-processor I/O with myrinet | |
| Zhu et al. | Performance analysis and testing of the storage area network | |
| Liang et al. | Improving read performance of SSDs via balanced redirected read | |
| CN111338570B (en) | Parallel file system IO optimization method and system | |
| Hua et al. | The design and implementations of locality-aware approximate queries in hybrid storage systems | |
| Manjunath et al. | Dynamic data replication on flash SSD assisted video-on-demand servers | |
| Nijim | Modelling speculative prefetching for hybrid storage systems | |
| Shu et al. | A highly efficient FC-SAN based on load stream | |
| Takahashi et al. | Layered memory architecture for high IO intensive information services to achieve timeliness | |
| Nijim et al. | DM-pas: A data mining prefetching algorithm for storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |