WO2016206463A1 - 一种写操作控制方法、装置及系统 - Google Patents

一种写操作控制方法、装置及系统 Download PDF

Info

Publication number
WO2016206463A1
WO2016206463A1 PCT/CN2016/079923 CN2016079923W WO2016206463A1 WO 2016206463 A1 WO2016206463 A1 WO 2016206463A1 CN 2016079923 W CN2016079923 W CN 2016079923W WO 2016206463 A1 WO2016206463 A1 WO 2016206463A1
Authority
WO
WIPO (PCT)
Prior art keywords
write operation
margin
data
file system
data length
Prior art date
Application number
PCT/CN2016/079923
Other languages
English (en)
French (fr)
Inventor
曹学文
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2016206463A1 publication Critical patent/WO2016206463A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • the present application relates to, but is not limited to, the field of storage systems, and in particular, to a write operation control method, apparatus and system.
  • disk caching technology is used to reduce access to the disk.
  • dirty data data written in the disk cache
  • the disk cache technology senses that the amount of dirty data reaches a certain threshold or times out, it will write the dirty data to the disk for curing. Call this process a data brush.
  • Write suppression was first proposed by the file system. Its main purpose is to prevent a large amount of concurrent write data from being poured into the disk cache faster than the dirty data brush, resulting in the service IO writing data waiting or sleeping. As a result, the latency of the write data service IO increases rapidly, and the client is very reluctant to accept such a sudden increase in delay. This control of the service IO of writing data prevents the delay waiting or sleeping of the write data service IO. However, no matter how the policy adjustment is made, the write operation IO will not be affected by the disk brush speed. In the file system, the write operation IO waits or sleeps. In the case of the write operation IO, if there is IO wait or sleep, there is no The method is awakened in time, which causes the file system to process the write IO with a longer delay.
  • This paper provides a write operation control method, device and system, which can solve the problem that the write operation of the related art is affected by the disk brush speed, and the write operation IO waits or sleeps.
  • a write operation control method includes: obtaining a dirty data margin of a disk cache; and controlling a write operation according to a dirty data margin and a data length of a write operation request.
  • controlling the write operation according to the dirty data margin includes: acquiring the reserved traffic of the current write operation, Calculate the difference between the margin of dirty data and the reserved traffic, and control the write operation according to the difference of the margin and the length of the data.
  • obtaining the reserved traffic of the current write operation includes: acquiring data traffic of the write operation that is not responded in the current write operation, and data traffic of the unwritten disk cache that has responded to the write operation, and calculating the sum of the two As reserved traffic.
  • controlling the write operation includes: determining a margin value and a data length, and performing a write operation when the margin difference is greater than the data length When the margin difference is less than or equal to the data length, the write operation is suppressed.
  • the method further includes: obtaining a protocol type of the write operation, and controlling the write operation according to the protocol type of the write operation.
  • a write operation control apparatus includes: an acquisition module configured to acquire a dirty data margin of a disk cache; and a control module configured to control a write operation according to a dirty data margin and a data length of the write operation request.
  • controlling, by the control module, the write operation according to the dirty data margin includes: acquiring a reserved traffic of the current write operation, and calculating a margin difference between the dirty data margin and the reserved traffic, according to the margin difference and the data length. Control write operations.
  • control module obtains the reserved traffic of the current write operation, including: acquiring data traffic of the write operation that is not responded in the current write operation, and data traffic of the unwritten disk cache that has responded to the write operation, and calculating the two. The sum is used as reserved traffic.
  • control module controls the write operation according to the dirty data margin and the data length of the write operation request, including: determining a margin difference and a data length, and when the margin difference is greater than the data length, performing A write operation that suppresses a write operation when the margin difference is less than or equal to the data length.
  • control module is further configured to obtain a protocol type of the write operation, according to a protocol of the write operation The type controls the write operation.
  • a write operation control system includes: a local file system and an access protocol file system.
  • the local file system set to get the dirty data margin of the disk cache.
  • the access protocol file system is set to control the write operation according to the dirty data margin and the data length of the write operation request.
  • system further comprises: a distributed file system.
  • the distributed file system is set to: obtain the reserved traffic of the current write operation, calculate the margin difference between the dirty data margin and the reserved traffic, and control the write operation according to the margin difference and the data length.
  • the distributed file system obtains the reserved traffic of the current write operation, including: acquiring data traffic of the write operation that is not responded in the current write operation, and data traffic of the unwritten disk cache that has responded to the write operation, and calculating The sum of the two is used as reserved traffic.
  • the access protocol file system controls the write operation according to the dirty data margin and the data length of the write operation request, including: determining a margin difference and a length of the data length, when the margin difference is greater than the data length, A write operation is performed to suppress the write operation when the margin difference is less than or equal to the data length.
  • the access protocol file system is further configured to obtain a protocol type of the write operation, and the write operation is controlled according to a protocol type of the write operation.
  • the local file system is set to obtain the dirty data margin of the disk cache and upload to the distributed file system.
  • the distributed file system is set to obtain the data traffic of the write operation that is not responded in the current write operation, and the data traffic of the unwritten disk cache that has responded to the write operation, calculate the sum of the two as the reserved traffic, and calculate the dirty
  • the difference between the data margin and the reserved traffic margin is uploaded to the access protocol file system.
  • the access protocol file system is set to control according to the difference value and the data length of the write operation request. Write operation.
  • a computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the above described write operation control method.
  • the solution of the embodiment of the present invention obtains the dirty data margin of the disk cache, and controls whether to perform the write operation according to the dirty data margin and the data length of the write operation request, and performs judgment before each execution of a write operation, only The write operation is performed when the condition is met, ensuring that each write operation can be performed, and the brush disk does not appear during the execution process, and the write operation of the related art is affected by the disk brush speed.
  • the problem of waiting or sleeping improves the user experience.
  • FIG. 1 is a schematic structural diagram of a write operation control apparatus according to a first embodiment of the present invention
  • FIG. 2 is a flowchart of a write operation control method according to a second embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a write operation control system according to a third embodiment of the present invention.
  • FIG. 4 is a flowchart of a write operation control method according to a third embodiment of the present invention.
  • the write operation control apparatus 1 provided by the embodiment of the present invention includes:
  • the obtaining module 11 is configured to obtain a dirty data margin of the disk cache
  • the control module 12 is configured to control the write operation based on the dirty data margin and the data length of the write operation request.
  • control module 12 in the foregoing embodiment controls the write operation according to the dirty data margin, including: acquiring the reserved traffic of the current write operation, and calculating a margin difference between the dirty data margin and the reserved traffic, according to the remaining The amount difference and the data length control the write operation.
  • control module 12 in the foregoing embodiment acquires the reserved traffic of the current write operation, including: acquiring data traffic of the write operation that is not responding in the current write operation, and unwritten disk cache of the responded write operation. Data traffic, calculate the sum of the two as reserved traffic.
  • control module 12 in the foregoing embodiment controls the write operation according to the dirty data margin and the data length of the write operation request, including: determining a margin difference and a data length, when the margin difference When the data length is greater than the data length, a write operation is performed, and when the margin difference is less than or equal to the data length, the write operation is suppressed.
  • control module 12 in the foregoing embodiment is further configured to acquire a protocol type of the write operation, and control the write operation according to the protocol type of the write operation.
  • Embodiments of the present invention provide a write operation control system, including: a local file system, a distributed file system, and an access protocol file system; wherein, the local file system is set to obtain a dirty data margin of the disk cache, and is uploaded to the distributed a file system; the distributed file system is configured to obtain data traffic of a write operation that is not responded in the current write operation, and data traffic of the unwritten disk cache that has responded to the write operation, and calculate the sum of the two as the reserved traffic. And calculating the difference between the margin of the dirty data and the reserved traffic, and uploading to the access protocol file system; the access protocol file system is set to control the write operation according to the difference value and the data length of the write operation request.
  • the write operation control method provided by the embodiment of the present invention includes the following steps S101 to S102:
  • S202 Control the write operation according to the dirty data margin and the data length of the write operation request.
  • controlling the write operation according to the dirty data margin includes: acquiring the reserved traffic of the current write operation, calculating a margin difference between the dirty data margin and the reserved traffic, and controlling the write operation according to the margin difference and the data length.
  • obtaining the reserved traffic of the current write operation includes: acquiring data traffic of the write operation that is not responded in the current write operation, and data traffic of the unwritten disk cache that has responded to the write operation, and calculating the sum of the two As reserved traffic.
  • the write operation is controlled according to the dirty data margin and the data length of the write operation request: determining a margin difference and a data length, and when the margin difference is greater than the data length, performing a write operation, When the margin difference is less than or equal to the data length, the write operation is suppressed.
  • the method further includes: obtaining a protocol type of the write operation, and controlling the write operation according to the protocol type of the write operation.
  • FIG. 3 is a schematic diagram of a write operation control system according to a third embodiment of the present invention, as can be seen from FIG.
  • the write operation control system of the pump embodiment of the invention includes a local file system 31 and an access protocol file system 33.
  • the local file system 31 is set to obtain the dirty data margin of the disk cache.
  • the access protocol file system 33 is arranged to control the write operation based on the dirty data margin and the data length of the write operation request.
  • system further includes: a distributed file system 32.
  • the distributed file system 32 is configured to: obtain the reserved traffic of the current write operation, calculate the margin difference between the dirty data margin and the reserved traffic, and control the write operation according to the margin difference and the data length.
  • the distributed file system 32 obtains the reserved traffic of the current write operation, including: obtaining the current The data traffic of the write operation that is not responded in the write operation, and the data traffic of the responded write operation that is not written to the disk cache, calculate the sum of the two as the reserved traffic.
  • the access protocol file system 33 controls the write operation according to the dirty data margin and the data length of the write operation request, including: determining the margin difference and the length of the data length, when the margin difference is greater than the data length And performing a write operation to suppress the write operation when the margin difference is less than or equal to the data length.
  • the access protocol file system 33 is further configured to obtain the protocol type of the write operation, and the write operation is controlled according to the protocol type of the write operation.
  • the write operation control system includes: a local file system 31, a distributed file system 32, and an access protocol file system 33; wherein the local file system 31 is configured to acquire dirty data of the disk cache.
  • the balance is uploaded to the distributed file system 32; the distributed file system 32 is set to obtain the data traffic of the write operation that is not responding in the current write operation, and the data traffic of the unwritten disk cache that has responded to the write operation, and calculate The sum of the two is used as reserved traffic, and the difference between the dirty data margin and the reserved traffic is calculated and uploaded to the access protocol file system 33; the access protocol file system 33 is set to be based on the margin difference and the write operation.
  • the requested data length controls the write operation.
  • the write operation control method provided by the embodiment of the present invention includes the following steps S401 to S405:
  • the access protocol file system receives the write operation IO request.
  • the current write IO request has two sources, one is the write operation IO request received by the access protocol file system from the underlying protocol of the storage system; the other is the access protocol file system when the write operation IO request is made. , generated write suppression, a write operation IO request placed in the write operation IO request queue.
  • the write suppression, or the suppression of the write operation refers to a case where a write operation IO request is prohibited.
  • the access protocol file system processes the write operation IO request.
  • the access protocol file system After the write operation IO request enters the access protocol file system, the access protocol file system first needs to determine whether the dirty data margin difference a obtained by the current system is greater than the data length of the current write operation IO request; if it is greater than the current write operation The data length of the IO request indicates that the write operation IO request can be directly performed. After subtracting the length of the write operation IO request, the dirty data margin difference a of the system is updated, and the number of current write write IO requests is counted.
  • Add 1 to send a write operation IO request to the distributed file system if the margin difference a is less than or equal to the data length of the current write operation IO request, and the current write operation IO request number is zero, indicating the existence of the storage system Multi-file access protocol write access (there are both ISCSI protocol write access and FC protocol write access), but can not only write to suppress the transmission of the system access protocol, so it is still possible to count the current write write IO request Add 1 to send a write operation IO request to the distributed file system; if it is greater than the data length of the current write operation IO request, and the current write operation IO please If the number of requests is not zero, it indicates that the system needs to perform write suppression on the write operation IO request, and caches the write operation IO request into the queue of the write request write operation IO.
  • S403 The distributed file system performs a write operation IO request.
  • the distributed file system After the write operation IO requests to enter the entry of the distributed file system, the distributed file system counts the data length of the write operation IO currently being performed; determines whether the timing of detecting the dirty data margin is turned on, and if the timing of detecting the dirty data margin is turned on, Then, the timing of detecting the dirty data margin is turned off. In the normal write operation IO process, the detection of the dirty data margin has become meaningless, and the information of the dirty data margin is carried by the response of the write operation IO; the distributed file system writes the current After processing other information of the IO, it is submitted to the local file system for processing.
  • the write operation IO request is normally performed.
  • the local file system is first submitted to the internal file of the subsystem.
  • the system access layer processing and then submitted to the local file system disk cache ARC module processing; after the disk cache ARC module processing, update the current dirty data margin (as the write data IO increases, the dirty data margin will be less and less ), and the updated dirty data margin is again submitted to the file system access layer of the local file system with the response of the write operation IO request; the internal file system access layer of the local file system uniformly carries the dirty data margin to the write Operation IO request response.
  • the write operation IO request response is submitted to the distributed file system and processed by the distributed file system.
  • the local file system When the dirty data margin reaches a set threshold (for example, the dirty data margin is less than or equal to 100M) or the timing expires, the local file system will flush the dirty data to the disk to perform data solidification. In the process, as more and more More dirty data is solidified, and the dirty data margin will become larger and larger. The change of the dirty data margin will be gradually reflected in the various subsystems of the storage system with the request response of the write operation IO, so that the write operation IO does not have IO wait or sleep.
  • a set threshold for example, the dirty data margin is less than or equal to 100M
  • S405 The local file system feeds back the write operation IO response to the file access protocol system, and carries the dirty data margin.
  • the solution of the embodiment of the present invention obtains the dirty data margin of the disk cache, and controls whether to perform the write operation according to the dirty data margin and the data length of the write operation request, and performs judgment before each execution of a write operation, only The write operation is performed when the condition is met, ensuring that each write operation can be performed, and the brush disk does not appear during the execution process, and the write operation of the related art is affected by the disk brush speed.
  • the problem of waiting or sleeping improves the user experience.
  • all or part of the steps of the foregoing embodiments may also be implemented by using an integrated circuit, and the steps may be separately fabricated into integrated circuit modules, or multiple modules thereof or The steps are made into a single integrated circuit module.
  • the devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.
  • the device/function module/functional unit in the above embodiment When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium.
  • the above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
  • the solution of the embodiment of the present invention obtains the dirty data margin of the disk cache, and controls whether to perform the write operation according to the dirty data margin and the data length of the write operation request, and performs judgment before each execution of a write operation, only The write operation is performed when the condition is met, ensuring that each write operation can be performed, and the brush disk does not appear during the execution process, and the write operation of the related art is affected by the disk brush speed.
  • the problem of waiting or sleeping improves the user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

一种写操作控制方法、装置及系统，方法包括：获取磁盘缓存的脏数据余量（S201）；根据脏数据余量以及写操作请求的数据长度，控制写操作（S202）。通过本方法，获取磁盘缓存的脏数据余量，并根据该脏数据余量与写操作请求的数据长度来控制是否执行写操作，在每执行一个写操作之前都进行判断，仅当满足条件时才执行写操作。

Description

一种写操作控制方法、装置及系统 技术领域
本申请涉及但不限于存储系统领域，尤其涉及一种写操作控制方法、装置及系统。
背景技术
一般的存储系统，都会使用磁盘缓存技术，来减少对磁盘的访问。而对于写入磁盘缓存中的数据(通常称之为脏数据)，磁盘缓存技术一旦感知到这些脏数据量达到一定的阈值或者定时超时，就会把这些脏数据写入磁盘固化，存储系统一般称此过程为数据刷盘。
写抑制最早是由文件系统提出的，其主要目的是为了防止出现大量并发的写数据灌入磁盘缓存中的速度超过了脏数据刷盘的速度，导致写入数据的业务IO产生等待或者休眠，从而使写数据业务IO出现时延骤增，客户非常不愿意接受这种时延骤增，这种对写入数据的业务IO进行控制，防止出现写数据业务IO的时延等待或者休眠。但是，无论怎么进行策略调整，都还是会避免不了写操作IO受磁盘刷盘速度的影响，在文件系统中出现写操作IO等待或者休眠的情况，写操作IO一旦出现IO等待或者休眠，就没有办法及时的得到唤醒，从而导致文件系统处理写IO的时延变长。
因此，如何提供一种避免写操作受磁盘刷盘速度影响的写操作控制方法，是本领域技术人员亟待解决的技术问题。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本文提供了一种写操作控制方法、装置及系统，可以解决相关技术存在的写操作受磁盘刷盘速度影响导致的写操作IO等待或者休眠的问题。
一种写操作控制方法，包括：获取磁盘缓存的脏数据余量；根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，根据脏数据余量控制写操作包括：获取当前写操作的预留流量， 计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断余量差值与数据长度的大小，当余量差值大于数据长度时，执行写操作，当所述余量差值小于或等于所述数据长度时，抑制写操作。
可选地，该方法还包括：获取写操作的协议类型，根据写操作的协议类型控制写操作。
一种写操作控制装置，包括：获取模块，设置为获取磁盘缓存的脏数据余量；控制模块，设置为根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，控制模块根据所述脏数据余量控制写操作包括：获取当前写操作的预留流量，计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，控制模块获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，控制模块根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断余量差值与数据长度的大小，当余量差值大于数据长度时，执行写操作，当所述余量差值小于或等于所述数据长度时，抑制写操作。
可选地，控制模块还设置为获取写操作的协议类型，根据写操作的协议 类型控制写操作。
一种写操作控制系统，该系统包括：本地文件系统以及接入协议文件系统。
本地文件系统，设置为获取磁盘缓存的脏数据余量。
接入协议文件系统，设置为根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，该系统还包括：分布式文件系统。
分布式文件系统设置为：获取当前写操作的预留流量，计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，分布式文件系统获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，接入协议文件系统根据脏数据余量以及写操作请求的数据长度，控制写操作包括：判断余量差值与所述数据长度的大小，当余量差值大于数据长度时，执行写操作，当余量差值小于或等于所述数据长度时，抑制写操作。
可选地，接入协议文件系统还设置为获取写操作的协议类型，根据写操作的协议类型控制所述写操作。
可选地，本地文件系统设置为获取磁盘缓存的脏数据余量，上传至分布式文件系统。
分布式文件系统设置为获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量，并计算脏数据余量与预留流量的余量差值，上传至接入协议文件系统。
接入协议文件系统设置为根据余量差值以及写操作请求的数据长度控制 写操作。
一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令被处理器执行时实现上述的写操作控制方法。
通过本发明实施例的方案，获取磁盘缓存的脏数据余量，并根据该脏数据余量与写操作请求的数据长度来控制是否执行写操作，在每执行一个写操作之前都进行判断，仅当满足条件时才执行写操作，保证了每个写操作都可以被执行，并且在执行过程中不会出现刷盘，解决了相关技术存在的写操作受磁盘刷盘速度影响导致的写操作IO等待或者休眠的问题，提高了用户的使用体验。
附图概述
图1为本发明第一实施例提供的写操作控制装置的结构示意图；
图2为本发明第二实施例提供的写操作控制方法的流程图；
图3为本发明第三实施例提供的写操作控制系统的示意图；
图4为本发明第三实施例提供的写操作控制方法的流程图。
本发明的实施方式
现通过具体实施方式结合附图的方式对本发明实施例进行说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互任意组合。
第一实施例：
图1为本发明第一实施例提供的写操作控制装置的结构示意图，由图1可知，在本实施例中，本发明实施例提供的写操作控制装置1包括：
获取模块11，设置为获取磁盘缓存的脏数据余量；
控制模块12，设置为根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，上述实施例中的控制模块12根据所述脏数据余量控制写操作包括：获取当前写操作的预留流量，计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，上述实施例中的控制模块12获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，上述实施例中的控制模块12根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断余量差值与数据长度的大小，当余量差值大于数据长度时，执行写操作，当所述余量差值小于或等于所述数据长度时，抑制写操作。
可选地，上述实施例中的控制模块12还设置为获取写操作的协议类型，根据写操作的协议类型控制写操作。
本发明实施例提供了一种写操作控制系统，包括：本地文件系统、分布式文件系统以及接入协议文件系统；其中，本地文件系统设置为获取磁盘缓存的脏数据余量，上传至分布式文件系统；分布式文件系统设置为获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量，并计算脏数据余量与预留流量的余量差值，上传至接入协议文件系统；接入协议文件系统设置为根据余量差值以及写操作请求的数据长度控制写操作。
第二实施例：
图2为本发明第二实施例提供的写操作控制方法的流程图，由图2可知，在本实施例中，本发明实施例提供的写操作控制方法包括以下步骤S101～S102：
S201：获取磁盘缓存的脏数据余量；
S202：根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，根据脏数据余量控制写操作包括：获取当前写操作的预留流量，计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作：判断余量差值与数据长度的大小，当余量差值大于数据长度时，执行写操作，当所述余量差值小于或等于所述数据长度时，抑制写操作。
可选地，该方法还包括：获取写操作的协议类型，根据写操作的协议类型控制写操作。
现结合具体应用场景对本发明实施例进行说明。
第三实施例：
图3为本发明第三实施例提供的写操作控制系统的示意图，由图3可知，
泵发明实施例的写操作控制系统包括：本地文件系统31以及接入协议文件系统33。
本地文件系统31，设置为获取磁盘缓存的脏数据余量。
接入协议文件系统33，设置为根据脏数据余量以及写操作请求的数据长度，控制写操作。
可选地，该系统还包括：分布式文件系统32。
分布式文件系统32设置为：获取当前写操作的预留流量，计算脏数据余量与预留流量的余量差值，根据余量差值以及数据长度控制写操作。
可选地，分布式文件系统32获取当前写操作的预留流量包括：获取当前 写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量。
可选地，接入协议文件系统33根据脏数据余量以及写操作请求的数据长度，控制写操作包括：判断余量差值与所述数据长度的大小，当余量差值大于数据长度时，执行写操作，当余量差值小于或等于所述数据长度时，抑制写操作。
可选地，接入协议文件系统33还设置为获取写操作的协议类型，根据写操作的协议类型控制所述写操作。
可选地，
在本实施例中，本发明实施例提供的写操作控制系统包括：本地文件系统31、分布式文件系统32以及接入协议文件系统33；其中，本地文件系统31设置为获取磁盘缓存的脏数据余量，上传至分布式文件系统32；分布式文件系统32设置为获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量，并计算脏数据余量与预留流量的余量差值，上传至接入协议文件系统33；接入协议文件系统33设置为根据余量差值以及写操作请求的数据长度控制写操作。
图4为本发明第三实施例提供的写操作控制方法的流程图，由图4可知，在本实施例中，本发明实施例提供的写操作控制方法包括以下步骤S401～S405：
S401：接入协议文件系统接收写操作IO请求。
当前进行写操作IO请求，其来源有两处，一处是接入协议文件系统从存储系统底层协议收到的写操作IO请求；另一处是接入协议文件系统在进行写操作IO请求时，产生过写抑制，存放到写操作IO请求队列中的写操作IO请求。这里，写抑制，或抑制写操作是指一个写操作IO请求被禁止的情况。
S402：接入协议文件系统处理写操作IO请求。
当写操作IO请求进入到了接入协议文件系统后，接入协议文件系统首先需要判断当前本系统得到的脏数据余量差值a是否大于当前写操作IO请求的数据长度；如果大于当前写操作IO的请求的数据长度，则说明可以直接进行写操作IO请求，减去写操作IO请求的长度后，更新本系统的脏数据余量差值a，并统计当前发送写操作IO请求的个数加1，发送写操作IO请求给分布式文件系统；如果余量差值a小于或等于当前写操作IO的请求的数据长度，并且当前写操作IO请求的个数为零，说明存储系统中存在多文件接入协议写访问的情况(同时存在ISCSI协议的写访问和FC协议的写访问)，但又不能只写抑制本系统接入协议的发送，所以还是可以统计当前发送写操作IO请求个数加1，发送写操作IO请求给分布式文件系统；如果大于当前写操作IO的请求的数据长度，并且当前写操作IO请求的个数不为零，则说明需要本系统对该写操作IO请求进行写抑制，将写操作IO请求缓存到待发送请求写操作IO的队列中。
S403：分布式文件系统执行写操作IO请求。
当写操作IO请求进入分布式文件系统的入口后，分布式文件系统统计当前正在进行写操作IO的数据长度；判断是否开启了定时探测脏数据余量，如果开启了定时探测脏数据余量，则关闭定时探测脏数据余量，在正常的写操作IO过程中，探测脏数据余量已显得意义不大，脏数据余量的信息由写操作IO的应答携带；分布式文件系统对当前写操作IO的其他信息处理后，提交给本地文件系统处理。
S404：本地文件系统将写操作IO请求写入磁盘缓存。
经分布式文件系统对当前写操作IO处理后，正常进行写操作IO请求，当写操作IO请求进入本地文件系统，本地文件系统先提交给子系统内部文件 系统接入层处理；再提交给本地文件系统的磁盘缓存ARC模块处理；磁盘缓存ARC模块处理以后，更新当前脏数据余量(随写入数据IO的增加，脏数据余量会越来越少)，并将更新的脏数据余量随写操作IO请求的应答再次提交给本地文件系统的文件系统接入层；由本地文件系统的内部文件系统接入层统一把脏数据余量携带到写操作IO请求应答中。并将写操作IO请求应答提交给分布式文件系统，由分布式文件系统处理。
脏数据余量达到一个设定的阈值(例如脏数据余量小于等于100M)或者定时超时，本地文件系统都会将脏数据刷到磁盘上，进行数据固化，在此过程中，随着越来越多的脏数据固化，脏数据余量就会越来越大。脏数据余量的变化，将随写操作IO的请求应答逐步体现在存储系统的各个子系统中，从而使写操作IO不会出现IO等待或者休眠的情况。
S405：本地文件系统反馈写操作IO响应至文件接入协议系统，携带脏数据余量。
通过本发明实施例的方案，获取磁盘缓存的脏数据余量，并根据该脏数据余量与写操作请求的数据长度来控制是否执行写操作，在每执行一个写操作之前都进行判断，仅当满足条件时才执行写操作，保证了每个写操作都可以被执行，并且在执行过程中不会出现刷盘，解决了相关技术存在的写操作受磁盘刷盘速度影响导致的写操作IO等待或者休眠的问题，提高了用户的使用体验。
本领域普通技术人员可以理解上述实施例的全部或部分步骤可以使用计算机程序流程来实现，所述计算机程序可以存储于一计算机可读存储介质中，所述计算机程序在相应的硬件平台上(如系统、设备、装置、器件等)执行，在执行时，包括方法实施例的步骤之一或其组合。
可选地，上述实施例的全部或部分步骤也可以使用集成电路来实现，这些步骤可以被分别制作成一个个集成电路模块，或者将它们中的多个模块或 步骤制作成单个集成电路模块来实现。
上述实施例中的装置/功能模块/功能单元可以采用通用的计算装置来实现，它们可以集中在单个的计算装置上，也可以分布在多个计算装置所组成的网络上。
上述实施例中的装置/功能模块/功能单元以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。上述提到的计算机可读取存储介质可以是只读存储器，磁盘或光盘等。
工业实用性
通过本发明实施例的方案，获取磁盘缓存的脏数据余量，并根据该脏数据余量与写操作请求的数据长度来控制是否执行写操作，在每执行一个写操作之前都进行判断，仅当满足条件时才执行写操作，保证了每个写操作都可以被执行，并且在执行过程中不会出现刷盘，解决了相关技术存在的写操作受磁盘刷盘速度影响导致的写操作IO等待或者休眠的问题，提高了用户的使用体验。

Claims (15)

  1. 一种写操作控制方法，包括：
    获取磁盘缓存的脏数据余量；
    根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作。
  2. 如权利要求1所述的写操作控制方法，其中，所述根据所述脏数据余量控制写操作包括：获取当前写操作的预留流量，计算所述脏数据余量与所述预留流量的余量差值，根据所述余量差值以及所述数据长度控制所述写操作。
  3. 如权利要求2所述的写操作控制方法，其中，所述获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为所述预留流量。
  4. 如权利要求2所述的写操作控制方法，其中，所述根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断所述余量差值与所述数据长度的大小，当所述余量差值大于所述数据长度时，执行所述写操作，当所述余量差值小于或等于所述数据长度时，抑制所述写操作。
  5. 如权利要求1至4任一项所述的写操作控制方法，还包括：获取所述写操作的协议类型，根据所述写操作的协议类型控制所述写操作。
  6. 一种写操作控制装置，包括：
    获取模块，设置为获取磁盘缓存的脏数据余量；
    控制模块，设置为根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作。
  7. 如权利要求6所述的写操作控制装置，其中，所述控制模块根据所述脏数据余量控制写操作包括：获取当前写操作的预留流量，计算所述脏数据余量与所述预留流量的余量差值，根据所述余量差值以及所述数据长度控制所述写操作。
  8. 如权利要求7所述的写操作控制装置，其中，所述控制模块获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为所述预留流量。
  9. 如权利要求7所述的写操作控制装置，其中，所述控制模块根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断所述余量差值与所述数据长度的大小，当所述余量差值大于所述数据长度时，执行所述写操作，当所述余量差值小于或等于所述数据长度时，抑制所述写操作。
  10. 如权利要求6至9任一项所述的写操作控制装置，所述控制模块还设置为获取所述写操作的协议类型，根据所述写操作的协议类型控制所述写操作。
  11. 一种写操作控制系统，所述系统包括：本地文件系统以及接入协议文件系统；
    本地文件系统，设置为获取磁盘缓存的脏数据余量；
    接入协议文件系统，设置为根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作。
  12. 如权利要求11所述的写操作控制系统，其中，所述系统还包括：分布式文件系统；
    分布式文件系统设置为：获取当前写操作的预留流量，计算所述脏数据余量与所述预留流量的余量差值，根据所述余量差值以及所述数据长度控制所述写操作。
  13. 如权利要求12所述的写操作控制系统，其中，所述分布式文件系统获取当前写操作的预留流量包括：获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为所述预留流量。
  14. 如权利要求12所述的写操作控制系统，其中，所述接入协议文件系统根据所述脏数据余量以及写操作请求的数据长度，控制所述写操作包括：判断所述余量差值与所述数据长度的大小，当所述余量差值大于所述数据长度时，执行所述写操作，当所述余量差值小于或等于所述数据长度时，抑制所述写操作。
  15. 如权利要求11至14任一项所述的写操作控制系统，其中，
    所述本地文件系统设置为获取磁盘缓存的脏数据余量，上传至所述分布式文件系统；
    所述分布式文件系统设置为获取当前写操作中未被响应的写操作的数据流量、以及已响应的写操作的未写入磁盘缓存的数据流量，计算两者之和作为预留流量，并计算所述脏数据余量与所述预留流量的余量差值，上传至所述接入协议文件系统；
    所述接入协议文件系统设置为根据所述分布式文件系统中的脏数据余量以及写操作请求的数据长度控制所述写操作。
PCT/CN2016/079923 2015-06-23 2016-04-21 一种写操作控制方法、装置及系统 WO2016206463A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510351087.1A CN106293500A (zh) 2015-06-23 2015-06-23 一种写操作控制方法、装置及系统
CN201510351087.1 2015-06-23

Publications (1)

Publication Number Publication Date
WO2016206463A1 true WO2016206463A1 (zh) 2016-12-29

Family

ID=57584727

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079923 WO2016206463A1 (zh) 2015-06-23 2016-04-21 一种写操作控制方法、装置及系统

Country Status (2)

Country Link
CN (1) CN106293500A (zh)
WO (1) WO2016206463A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109062505A (zh) * 2018-07-13 2018-12-21 南瑞集团有限公司 一种缓存策略写入分层硬件架构下的写性能优化方法
CN109343797B (zh) * 2018-09-25 2021-09-17 郑州云海信息技术有限公司 一种缓存数据的传输方法、系统及相关组件
CN113672523A (zh) * 2021-07-09 2021-11-19 济南浪潮数据技术有限公司 一种缓存池数据下刷方法、系统、电子设备及存储介质
CN113986118A (zh) * 2021-09-28 2022-01-28 新华三大数据技术有限公司 一种数据处理方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1704910A (zh) * 2004-06-03 2005-12-07 华为技术有限公司 一种磁盘阵列结构中的写处理方法
US20080016274A1 (en) * 2006-07-13 2008-01-17 Xing-Jia Wang Method of protecting cache memory data in storage system
CN103294574A (zh) * 2013-06-25 2013-09-11 浙江宇视科技有限公司 一种快速实现业务存储切换的装置和方法
CN103473180A (zh) * 2013-08-30 2013-12-25 华为技术有限公司 数据存储方法及系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5754888A (en) * 1996-01-18 1998-05-19 The Board Of Governors For Higher Education, State Of Rhode Island And Providence Plantations System for destaging data during idle time by transferring to destage buffer, marking segment blank , reodering data in buffer, and transferring to beginning of segment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1704910A (zh) * 2004-06-03 2005-12-07 华为技术有限公司 一种磁盘阵列结构中的写处理方法
US20080016274A1 (en) * 2006-07-13 2008-01-17 Xing-Jia Wang Method of protecting cache memory data in storage system
CN103294574A (zh) * 2013-06-25 2013-09-11 浙江宇视科技有限公司 一种快速实现业务存储切换的装置和方法
CN103473180A (zh) * 2013-08-30 2013-12-25 华为技术有限公司 数据存储方法及系统

Also Published As

Publication number Publication date
CN106293500A (zh) 2017-01-04

Similar Documents

Publication Publication Date Title
US10838626B2 (en) Methods, apparatuses, and computer program products for controlling write requests in storage system
KR102380670B1 (ko) 메모리 제어기에서의 세분화된 대역폭 프로비저닝
US10860494B2 (en) Flushing pages from solid-state storage device
WO2016206463A1 (zh) 一种写操作控制方法、装置及系统
US9760392B1 (en) Adaptive throttling in hybrid storage environments
US20150206567A1 (en) Dynamic temperature adjustments in spin transfer torque magnetoresistive random-access memory (stt-mram)
US10078470B2 (en) Signal transfer device that maintains order of a read request and write request in posted write memory access
EP2733617A1 (en) Data buffer device, data storage system and method
US7975090B2 (en) Method for efficient I/O controller processor interconnect coupling supporting push-pull DMA read operations
JP5987498B2 (ja) ストレージ仮想化装置、ストレージ仮想化方法及びストレージ仮想化プログラム
WO2016080953A1 (en) Method and apparatus for data cache in converged system
US11675540B2 (en) In-line data flow for computational storage
CN111177032A (zh) 缓存空间申请方法、系统、装置及计算机可读存储介质
JP6229733B2 (ja) 情報処理装置、方法、プログラム及び記録媒体
US10331563B2 (en) Adaptively enabling and disabling snooping bus commands
US20220092008A1 (en) Managing input/output-critical tasks and background tasks within a computer device
US9342472B2 (en) PRD (physical region descriptor) pre-fetch methods for DMA (direct memory access) units
KR102334473B1 (ko) 적응형 딥러닝 가속 장치 및 방법
JP6046523B2 (ja) インメモリ型分散データベース、データ分散方法及びプログラム
EP2902910A1 (en) Electronic device, and method for accessing data in electronic device
WO2020244498A1 (zh) 服务器系统内消息的处理方法、装置及系统
US20180115495A1 (en) Coordinating Accesses of Shared Resources by Clients in a Computing Device
US11726885B2 (en) Efficient method and system of intelligent deploying file level restore agent in multicloud
US20160162216A1 (en) Storage control device and computer system
JP2022546358A (ja) サービスダーティライン追跡の品質

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16813579

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16813579

Country of ref document: EP

Kind code of ref document: A1