WO2017101263A1 - 管理存储设备的方法及装置 - Google Patents

管理存储设备的方法及装置 Download PDF

Info

Publication number
WO2017101263A1
WO2017101263A1 PCT/CN2016/084479 CN2016084479W WO2017101263A1 WO 2017101263 A1 WO2017101263 A1 WO 2017101263A1 CN 2016084479 W CN2016084479 W CN 2016084479W WO 2017101263 A1 WO2017101263 A1 WO 2017101263A1
Authority
WO
WIPO (PCT)
Prior art keywords
amount
storage device
network service
data
redundant space
Prior art date
Application number
PCT/CN2016/084479
Other languages
English (en)
French (fr)
Inventor
许浩
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2017562727A priority Critical patent/JP6522168B2/ja
Priority to EP16874327.6A priority patent/EP3392754B1/en
Priority to KR1020187000330A priority patent/KR102100571B1/ko
Publication of WO2017101263A1 publication Critical patent/WO2017101263A1/zh
Priority to US15/707,860 priority patent/US10198180B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0873—Mapping of cache memory to specific storage devices or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/22—Employing cache memory using specific memory technology
    • G06F2212/222—Non-volatile memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7202—Allocation control and policies
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7204—Capacity control, e.g. partitioning, end-of-life degradation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7205—Cleaning, compaction, garbage collection, erase control
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7206—Reconfiguration of flash memory system

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for managing a storage device.
  • Solid-state hard disks have become the best choice for network service storage devices due to their fast read speed and low power consumption.
  • a technician sets a corresponding basic space and a redundant space for each network service.
  • the basic space capacity and the redundant space capacity of the SSDs allocated by each network service are certain. For a network service with a relatively large amount of traffic, the storage space of the SSD is easily filled, resulting in a relatively high frequency of invalid data cleaning. The cleaning process will damage the storage blocks of the SSD to a certain extent, which will shorten the service life of the target storage device.
  • embodiments of the present invention provide a method and apparatus for managing a storage device.
  • a method of managing a storage device includes: detecting, in a preset duration, a first unit duration write amount of service data of a target network service in a target storage device; according to a correspondence relationship between a pre-stored unit duration write amount and a redundant space amount, Determining a first amount of redundant space corresponding to the first unit duration write amount, wherein, in the correspondence relationship, the amount of redundant space becomes larger as the unit length write amount increases; a redundant amount of space for the target network The service is set in the redundant space in the target storage device.
  • an apparatus for managing a storage device includes: a detecting module, configured to detect a first unit duration write amount of the service data of the target network service in the target storage device within a preset duration; and a determining module, configured to write according to the pre-stored unit duration And determining, by the correspondence between the quantity and the amount of redundant space, a first amount of redundant space corresponding to the first unit duration write amount, wherein the amount of redundant space becomes larger as the unit length write amount increases; And a module, configured to set, according to the first amount of redundant space, a redundant space of the target network service in the target storage device.
  • a computer readable storage medium having program instructions for executing the method is stored on the computer readable storage medium.
  • the amount of redundant space can be dynamically adjusted based on the service data unit duration write amount of the target network service; therefore, when the unit time write amount is increased, the redundant space amount can be automatically increased, and the data clearing is reduced. Frequency, thereby extending the life of the target storage device.
  • FIG. 1 is a flowchart of a method for managing a storage device according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram showing a configuration of a storage management device and a relationship with a target storage device according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a storage management device according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an apparatus for managing a storage device according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a determining module of an apparatus for managing a storage device according to an embodiment of the present disclosure
  • FIG. 6 is a schematic structural diagram of hardware of a storage management device according to an embodiment of the present invention.
  • the embodiment of the invention provides a method for managing a storage device, and the execution entity of the method is a storage management device.
  • the storage management device may be a server, and the storage management device may be provided with a processor, a memory, a transceiver, and the like.
  • the processor can be used to manage the process of the storage device, the memory can be used to store the data needed for processing and the generated data, and the transceiver can be used to receive and transmit data.
  • the processing flow of the method may include steps 101 to 103:
  • Step 101 Detect a first unit duration write amount of the service data of the target network service in the target storage device within a preset duration.
  • the target storage device can provide storage space for service data of multiple network services, and the target storage device can be a solid state drive; the target network service can be an application service, such as a service of a WeChat application, a service of a Drip taxi application. Etc., they can be installed on the storage management device or on other server devices.
  • the target network service can be an application service, such as a service of a WeChat application, a service of a Drip taxi application. Etc., they can be installed on the storage management device or on other server devices.
  • the storage management device can continuously detect the amount of writes of the traffic data of the target network service in the target storage device.
  • the storage management device determines, in the preset duration, the amount of the service data of the target network service in the target storage device in the preset time period, and obtains the service data of the target network service in the first unit duration of the target storage device. Involvement.
  • the unit length write amount can be the amount of data written per unit time.
  • the data amount of the service data written by the target network service in the target storage device is 200M per minute
  • the data amount of the service data written by the target network service in the target storage device is 1G or the like per minute.
  • the business data of the WeChat application needs to be stored to the target storage device.
  • the storage management device detects that the write amount of the service data in the target storage device is 200M in the past 10 minutes, and can determine the first unit duration of the service data of the WeChat application.
  • the write amount is 20M.
  • the unit time write amount may also be the unit time write number, for example, the target network service writes in the target storage device is 200,000 times per minute, and the target network service is in the target storage device every minute. The number of writes is 500,000 times.
  • Step 102 Determine a first redundant space amount corresponding to the first unit duration write amount according to a correspondence relationship between the unit time write amount and the redundant space amount stored in advance.
  • the amount of redundant space includes: redundant space capacity, or redundant space capacity relative to the target network The ratio of the amount of base space capacity of the business in the target storage device.
  • the amount of redundant space may be a storage space capacity other than a fixed storage space capacity (also referred to as a basic space capacity) provided by the target storage device for the target network service, and may be referred to as a redundant space capacity, such as a target storage device.
  • the fixed storage capacity provided by the network service is 1T, and the storage network capacity of the target network service is additionally provided as the amount of redundant space.
  • the amount of redundant space may be a ratio of the storage space capacity additionally provided by the target storage device to the base space capacity of the target network service in the target storage device.
  • the target storage device provides a base space capacity of 10T for the target network service, and provides 1T storage space capacity for the target network service, and the amount of redundant space is 10%.
  • the storage management device can store the correspondence between the unit time write amount and the redundant space amount.
  • the amount of redundant space can increase as the unit time write amount increases.
  • the amount of redundant space is dynamically increased, and the time for data cleaning can be postponed, so that the proportion of invalid data (also referred to as dirty data) cleaned up during data cleaning is relatively large. , reduced write amplification.
  • Table 1 shows a correspondence between the unit time write amount and the redundancy space amount.
  • the unit time write data amount represents the unit time length write amount
  • the redundant space capacity is represented.
  • the amount of redundant space As shown in Table 1, the redundant space capacity corresponding to the unit length write amount can be determined according to the value range to which the unit duration write amount belongs. It should be understood that the corresponding relationship is only an example, and those skilled in the art can modify the correspondence according to actual needs.
  • Unit length write Redundant amount of space 0 to 300M 200M 300 ⁇ 600M 500M ... ...
  • the storage management device determines the first unit duration write amount of the service data of the target network service, it may be determined that the value range of the unit duration write amount to which the first unit duration belongs is the first value range, and then the unit duration may be In the correspondence between the amount of writing and the amount of redundant space, the first amount of redundant space corresponding to the first range of values is determined.
  • the target network service may be a service of a WeChat application
  • the storage management device determines that the first unit duration write amount is 500M, and may determine that the first unit duration write amount is 500M, and the first value range is 300M to 600M, and then Can find the first value range as The first redundant space corresponding to 300M to 600M is 500M, and the amount of redundant space is set accordingly. In this way, when the write amount of the first unit duration is large, the amount of redundant space can be set larger, so that the time of data cleaning can be postponed, and the proportion of cleaned data to invalid data is relatively high when the data is cleaned.
  • Table 2 shows another correspondence between the amount of writing per unit time and the amount of redundant space.
  • the amount of data written in unit time is expressed as the unit time write amount, and the ratio of the redundant space capacity to the base space capacity of the target network service in the target storage device indicates the amount of redundant space.
  • the predetermined numerical range of the unit time write amount corresponds to the amount of redundant space. It should be understood that the corresponding relationship is only an example, and those skilled in the art can modify the correspondence according to actual needs.
  • Unit length write Redundant amount of space 0 to 300M 10% 300 ⁇ 600M 20% ... ...
  • the first redundant space quantity may also be determined according to the basic space capacity and the unit time length write amount in the target storage device of the target network service, and the corresponding processing may be: detecting the basis of the target network service in the target storage device.
  • the space capacity is determined according to the corresponding relationship between the pre-stored basic space capacity, the unit time length write amount and the redundant space quantity, and the first space capacity of the target network service in the target storage device and the first unit time length write amount are correspondingly determined. The amount of redundant space.
  • the storage management device can store the correspondence between the basic space capacity, the unit time write amount, and the redundant space, as shown in Table 3.
  • Table 3 shows the correspondence between the unit time write amount, the base space capacity and the amount of redundant space.
  • different amount of redundant space can be set according to the difference of the base space capacity.
  • different amounts of redundant space may be set according to different unit time length write amounts for the same base space capacity. It should be understood by those skilled in the art that the corresponding relationship shown in Table 3 is only an example, and those skilled in the art may modify the corresponding relationship according to actual needs.
  • the storage management device detects the first unit duration write amount of the service data of the target network service in the target storage device, and can also determine the base space capacity of the target network service in the target storage device, and then can be from the pre-stored base space. In the correspondence between the capacity, the unit time write amount and the redundant space, the base space capacity of the target network service in the target storage device and the first redundant space amount corresponding to the first unit duration write amount are determined.
  • the target network service may be a service of a WeChat application
  • the storage management device determines that the first unit duration write amount is 500M, and then determines that the target storage device provides a base space capacity of 10T for the target network service, and may find the basic space capacity. For 10T, the amount of redundant space corresponding to the first unit duration of 500M is 1T, so that the first amount of redundant space can be determined to be 1T.
  • Step 103 Set a redundant space of the target network service in the target storage device according to the first redundant space amount.
  • the storage management device may set the amount of redundant space of the target network service in the target storage device to the first amount of redundant space.
  • the data cleaning threshold may be determined according to the unit length writing amount, and the corresponding processing may be: determining the writing amount with the first unit duration according to the corresponding relationship between the pre-stored unit duration writing amount and the data cleaning threshold.
  • the data cleaning threshold is a total data amount of the target network service used to trigger the target storage device to perform invalid data cleaning on the target network service in the target storage device; and the target network service is in the target storage device
  • the data cleanup threshold is set to the first data cleanup threshold.
  • the data cleaning threshold may also be referred to as a garbage collection threshold, which may be the amount of service data of the target network service stored in the target storage device when the target storage device cleans up the invalid data of the target network service.
  • the storage management device can store the correspondence between the unit time write amount and the data cleanup threshold.
  • the capacity is written into the business data of the target network service.
  • Table 4 shows a correspondence between the unit time write amount and the data cleanup threshold.
  • the data is written in unit time.
  • the quantity represents the unit length write amount.
  • the data cleaning threshold corresponding to the unit length write amount can be determined according to the value range to which the unit duration write amount belongs. It should be understood that the corresponding relationship is only an example, and those skilled in the art can modify the correspondence according to actual needs.
  • the storage management device may determine the first data cleanup threshold according to the correspondence between the unit duration write amount and the data cleanup threshold, and then the target network service may be in the target storage device.
  • the data cleanup threshold is set to the first data cleanup threshold.
  • the target storage device provides a base space capacity of 100T for the WeChat application.
  • the storage management device When the storage management device detects that the first unit duration of the WeChat application is 500M, the storage management device can determine that the first unit duration write amount belongs to In the range of 300M to 600M, you can find the garbage collection threshold of 95T corresponding to the value range of 300M to 600M, and then set the data cleaning threshold of the WeChat application to the target storage device to 95T, that is, when the WeChat application is used.
  • the target storage device can be triggered to clean the service data of the WeChat application.
  • Tables 1 to 4 all write the data amount in unit time as the unit time write amount, and those skilled in the art should understand that the space redundancy amount can be determined by the unit time write number as the unit time length write amount.
  • the invalid data threshold may be determined according to the amount of redundant space, and the corresponding processing may be: determining, according to the correspondence between the pre-stored redundant space amount and the invalid data threshold, the first corresponding to the first redundant space amount.
  • the invalid data threshold is used to trigger the target network device to perform invalid data cleaning on the target network service, and the ratio of the invalid data volume to the total data volume of the target network service in the target storage device; the target network service is in the target storage device.
  • the invalid data threshold is set to the first invalid data threshold.
  • the invalid data threshold may also be referred to as a dirty data threshold, which may be a ratio of the amount of invalid data in the target storage device to the total amount of data in the target storage device when the target storage device performs invalid data cleaning on the target network service.
  • the storage management device can store the correspondence between the amount of redundant space and the invalid data threshold.
  • the corresponding invalid data threshold is also relatively large, so that the invalid data can be cleaned up when the invalid data is relatively large.
  • the amount of valid data that needs to be copied is relatively small, reducing write amplification. As shown in Table 5. In Table 5, the amount of redundant space is expressed in terms of redundant space capacity.
  • Redundant amount of space Invalid data threshold 50M 80% 100M 87% ... ...
  • the storage management device may determine, according to the correspondence between the redundancy space quantity and the invalid data threshold, a first invalid data threshold corresponding to the first redundant space quantity, and then The invalid data threshold of the target network service in the target storage device is set to the first invalid data threshold, so that when the target network service stores the service data in the target storage device, when the invalid data of the service data and the target network service are in the target storage device
  • the target storage device is triggered to clean the invalid data of the target network service.
  • the storage space provided by the target storage device for the network service may be divided into multiple storage blocks, and may be simultaneously written to multiple storage blocks when writing data, and the ratio of invalid data is preferentially cleared when invalid data is cleaned. Larger storage blocks.
  • the embodiment of the present invention further provides a process for storing service data of a target network service, and the corresponding processing may be as follows:
  • the target network service can be installed on the storage management device or on other server devices.
  • the storage management device can be configured with a virtual disk, a flash translation layer, and an underlying driver.
  • the virtual disk can be used.
  • Providing a logical storage space for the service data of the target network service, and the flash translation layer can be used for storing the corresponding relationship (such as the correspondence between the unit time write amount and the redundant space amount) required for the target network service to store the service data, and the underlying driver.
  • multiple network services may be installed on one storage management device, or the multiple network services may be set on different server devices, but multiple The network service can share the flash translation layer of the storage management device, and can share a target storage device.
  • Each network service has its own corresponding virtual disk, and the number of virtual disks can be determined by the unit time write amount, wherein the virtual disks are The number can be proportional to the unit length write.
  • Technology The personnel can set the correspondence between the unit length write amount and the corresponding redundant space amount and the data cleanup threshold according to the experience value, and store it to the flash translation layer, and also set the registration interface, the control interface, and the status query in the flash translation layer.
  • the registration interface can be used for the amount of redundant space corresponding to the length of the network service query unit, the data cleaning threshold, etc.
  • the control interface can be used to determine the number of virtual disks according to the unit length write amount, and the status query interface can be used. The amount of time written by the network service detection unit.
  • the execution part may be used to determine an address mapping relationship, that is, a mapping relationship between a logical address of a storage address recorded by the network service and a physical address of the storage address recorded by the target storage device.
  • the concurrent access part can be used to simultaneously access multiple storage blocks of the target storage device, wherein the storage space provided by the target storage device for the network service can be divided into multiple storage blocks, and the data can be simultaneously sent to multiple storage blocks when writing data. Write.
  • the target network service When the target network service stores the service data in the target storage device, the target network service can detect the unit time write amount in the preset duration, and then determine the redundant space corresponding to the unit time write amount according to the unit time write amount. The amount, the data cleaning threshold, and the like, and then send the service data to the virtual disk, the virtual disk can send the service data to the flash translation layer, the execution part can receive the service data, and determine the mapping relationship between the logical address and the physical address of the service data. Then, the concurrent access part can store the business data according to the address mapping relationship to the corresponding storage block in the target storage device through the underlying driver.
  • the storage management device when the storage management device stores the service data to the target storage device, the storage management device may also store the service data according to the priority of the service data, that is, when the current service data has a higher priority, the service may be written first. Data, after the business data is written, other business data is written.
  • the first unit duration write amount of the service data of the target network service in the target storage device is detected within a preset duration; and the correspondence between the unit length write amount and the redundant space amount according to the pre-stored amount Determining a first amount of redundant space corresponding to the first unit duration write amount; and setting a redundant space of the target network service in the target storage device according to the first redundant space amount.
  • an embodiment of the present invention further provides an apparatus for managing a storage device.
  • the device includes:
  • the detecting module 410 is configured to detect, in a preset duration, a first unit duration write amount of the service data of the target network service in the target storage device;
  • a determining module 420 configured to determine, according to a pre-stored correspondence between a unit-time duration write amount and a redundant space amount, a first redundant space amount corresponding to the first unit duration write amount, where the corresponding relationship In the middle, the amount of redundant space becomes larger as the writing amount per unit time increases;
  • the setting module 430 is configured to set a redundant space of the target network service in the target storage device according to the first redundant space amount.
  • the unit time write amount includes: a unit time write count or a unit duration write data amount.
  • the amount of redundant space includes: a ratio of redundant space capacity, or a ratio of redundant space capacity to a base space capacity of the target network service in the target storage device.
  • the determining module 420 is further configured to:
  • the setting module 430 is further configured to:
  • the determining module 420 is further configured to:
  • the setting module 430 is further configured to:
  • the determining module 420 described with reference to FIG. 4 may include a detecting submodule 421 and a determining submodule 422, where:
  • the detecting submodule 421 is configured to detect a basic space capacity of the target network service in the target storage device
  • the determining sub-module 422 is configured to determine, according to a pre-stored basic space capacity, a correspondence between a unit-time duration write amount and a redundant space, a basic space capacity and a location of the target network service in the target storage device.
  • the first redundant space amount corresponding to the first unit duration write amount.
  • the first unit duration write amount of the service data of the target network service in the target storage device is detected within a preset duration; and the correspondence between the unit length write amount and the redundant space amount according to the pre-stored amount Determining a first amount of redundant space corresponding to the first unit duration write amount, wherein, in the corresponding relationship, the amount of redundant space becomes larger as the unit length write amount increases;
  • the first amount of redundant space is set for the redundant space of the target network service in the target storage device.
  • the device for managing the storage device when managing the storage device, the device for managing the storage device is only illustrated by the division of the foregoing functional modules. In actual applications, the functions may be assigned to different functions according to requirements.
  • the module is completed, that is, the internal structure of the device that manages the storage device is divided into different functional modules to complete all or part of the functions described above.
  • the device for managing the storage device provided by the foregoing embodiment is the same as the method for managing the storage device. For the specific implementation process, refer to the method embodiment, and details are not described herein again.
  • FIG. 6 is a schematic diagram of a hardware structure of a storage management device according to an embodiment of the present invention.
  • the storage management device may be used to implement the method for managing a storage device provided in the foregoing embodiment. Specifically:
  • the storage management device may be configured as the task execution server 600 illustrated in FIG. 6.
  • the task execution server 600 can vary considerably depending on configuration or performance, and can include one or more central processors 1922 (eg, one or more processors) and memory 1932, one or more storage applications Storage medium 1930 of 1942 or data 1944 (eg, one or one storage device in Shanghai).
  • the memory 1932 and the storage medium 1930 may be short-term storage or persistent storage.
  • the program stored on the storage medium 1930 may include one or more modules (not shown), each of which may include a series of instruction operations in the task execution server.
  • central processor 1922 can be configured to communicate with storage medium 1930 to perform a series of instruction operations in storage medium 1930 on task execution server 600.
  • Task execution server 600 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and/or one or more operations.
  • System 1941 such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and the like.
  • Task execution server 600 can include a memory, and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs included for Execute the above instructions for managing the method of the storage device.
  • the storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明实施例公开了一种管理存储设备的方法及装置，属于计算机技术领域。所述方法包括：检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，在所述对应关系中，冗余空间量随单位时长写入量的增大而变大；根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。采用本发明，可以延长目标存储设备的使用寿命。

Description

管理存储设备的方法及装置
本申请要求于2015年12月17日提交中国专利局、申请号为201510946978.1、发明名称为“一种管理存储设备的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及计算机技术领域，特别涉及一种管理存储设备的方法及装置。
背景技术
随着计算机技术和互联网技术的发展，大存储量的存储设备在网络侧得到了广泛的应用，固态硬盘以其读取速度快、低功耗等特点逐渐成为网络业务存储设备的最佳选择。
现有技术中，在存储设备中，技术人员会分别为每个网络业务设置相应的基础空间和冗余空间。
在实现本发明的过程中，发明人发现现有技术至少存在以下问题：
每个网络业务分配的固态硬盘的基础空间容量和冗余空间容量是一定的，对于业务量比较大的网络业务，固态硬盘的存储空间很容易被写满，导致无效数据清理的频率比较高，清理过程会一定程度的损伤固态硬盘的存储块，从而会缩短目标存储设备的使用寿命。
发明内容
为了解决现有技术的问题，本发明实施例提供了一种管理存储设备的方法以及装置。
第一方面，提供了一种管理存储设备的方法。所述方法包括：检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，在所述对应关系中，冗余空间量随单位时长写入量的增大而变大；根据所述第一冗余空间量，对所述目标网络 业务在目标存储设备中的冗余空间进行设置。
根据本发明的另一方面，提供了一种管理存储设备的装置。所述装置包括：检测模块，用于检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；确定模块，用于根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，冗余空间量随单位时长写入量的增大而变大；设置模块，用于根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。
根据本发明的另一方面，提供了一种计算机可读存储介质，在所述计算机可读存储介质上存储有用于执行所述的方法的程序指令。
根据本发明实施例，可以基于目标网络业务的业务数据单位时长写入量，动态调整冗余空间量；因此，在单位时长写入量增加时，可以自动增加冗余空间量，降低数据清理的频率，从而延长目标存储设备的使用寿命。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种管理存储设备的方法流程图；
图2是本发明实施例示出了存储管理设备的构成以及与目标存储设备的关系的示意图；
图3是本发明实施例提供的一种存储管理设备的功能结构示意图；
图4是本发明实施例提供的一种管理存储设备的装置的结构示意图；
图5是本发明实施例提供的一种管理存储设备的装置的确定模块的示意图；
图6是本发明实施例提供的一种存储管理设备的硬件结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
实施例一
本发明实施例提供了一种管理存储设备的方法，该方法的执行主体为存储管理设备。其中，该存储管理设备可以是服务器，存储管理设备中可以设置有处理器、存储器，收发器等。处理器可以用于管理存储设备的过程处理，存储器可以用于存储处理过程中需要的数据以及产生的数据，收发器可以用于接收以及发送数据。
如图1所示，该方法的处理流程可以包括步骤101至103：
步骤101，检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量。
其中，目标存储设备可以为多个网络业务的业务数据提供存储空间，目标存储设备可以是固态硬盘；目标网络业务可以是应用程序的业务，如微信应用程序的业务、滴滴打车应用程序的业务等，它们可以安装在存储管理设备上，也可以安装在其他服务端设备上。
作为示例，存储管理设备可以持续检测目标网络业务的业务数据在目标存储设备中的写入量。存储管理设备每隔预设时长就确定在该预设时长内目标网络业务的业务数据在目标存储设备中的写入量，得到目标网络业务的业务数据在目标存储设备中的第一单位时长写入量。
单位时长写入量可以是单位时长写入数据量。例如，每分钟内目标网络业务在目标存储设备中写入的业务数据的数据量为200M，每分钟内目标网络业务在目标存储设备中写入的业务数据的数据量为1G等。
再如，对于微信应用程序，微信应用程序的业务数据需要存储至目标存储设备。这里，假设预设时长为10分钟，存储管理设备检测到在过去的10分钟内，在目标存储设备中业务数据的写入量为200M，则可以确定微信应用程序的业务数据的第一单位时长写入量为20M。
可选地，单位时长写入量也可以是单位时长写入次数，如每分钟内目标网络业务在目标存储设备中的写入次数为20万次、每分钟内目标网络业务在目标存储设备中的写入次数为50万次等。
步骤102，根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与第一单位时长写入量对应的第一冗余空间量。
这里，冗余空间量包括：冗余空间容量，或冗余空间容量相对于目标网络 业务在目标存储设备中的基础空间容量的比值。
这里，冗余空间量可以是目标存储设备为目标网络业务提供的固定存储空间容量(也可以称为基础空间容量)以外的存储空间容量，可以称为冗余空间容量，如目标存储设备为目标网络业务提供的固定存储空间容量为1T，额外给目标网络业务提供100G的存储空间容量作为其冗余空间量。
或者，冗余空间量可以是目标存储设备额外提供的存储空间容量与目标网络业务在目标存储设备中的基础空间容量的比值。比如，目标存储设备为目标网络业务提供的基础空间容量为10T，额外给目标网络业务提供1T的存储空间容量，则冗余空间量为10％。
存储管理设备中可以存储单位时长写入量与冗余空间量的对应关系，在对应关系中，冗余空间量可以随着单位时长写入量的增大而增大。这样，目标网络业务的单位时长写入量增加时，动态地增加冗余空间量，可以推迟进行数据清理的时间，使数据清理时清理到的无效数据(也可以称为脏数据)比例比较多，降低了写放大。
作为示例，表一示出了单位时长写入量与冗余空间量之间的一种对应关系，这里，以单位时长写入数据量表示单位时长写入量，而以冗余空间容量来表示冗余空间量。如表一所示，可根据单位时长写入量所属的数值范围来确定与该单位时长写入量对应的冗余空间容量。应该理解，所给出的对应关系只是一种示例，本领域技术人员可以根据实际需要来对该对应关系进行修改。
表一
单位时长写入量 冗余空间量
0～300M 200M
300～600M 500M
… …
作为示例，存储管理设备确定目标网络业务的业务数据的第一单位时长写入量后，可以确定第一单位时长所属的单位时长写入量的数值范围为第一数值范围，然后可以从单位时长写入量与冗余空间量的对应关系中，确定第一数值范围对应的第一冗余空间量。例如，目标网络业务可以是微信应用程序的业务，存储管理设备确定第一单位时长写入量为500M，可以确定第一单位时长写入量为500M所属的第一数值范围为300M～600M，然后可以查找第一数值范围为 300M～600M对应的第一冗余空间量为500M，据此来设置冗余空间量。这样，当第一单位时长写入量大时，可以将冗余空间量设置的大一些，从而可以推迟数据清理的时间，在清理数据时清理到无效数据的比例比较高。
表二示出了单位时长写入量与冗余空间量的另外一种对应关系。在表二中，还是以单位时长写入数据量表示单位时长写入量，而以冗余空间容量相对于目标网络业务在目标存储设备中的基础空间容量的比值表示冗余空间量。在表二中，单位时长写入量的预定的数值范围与冗余空间量相对应。应该理解，所给出的对应关系只是一种示例，本领域技术人员可以根据实际需要来对该对应关系进行修改。
表二
单位时长写入量 冗余空间量
0～300M 10％
300～600M 20％
… …
可选地，还可以根据目标网络业务在目标存储设备中的基础空间容量与单位时长写入量确定第一冗余空间量，相应的处理可以为：检测目标网络业务在目标存储设备中的基础空间容量；根据预先存储的基础空间容量、单位时长写入量与冗余空间量的对应关系，确定目标网络业务在目标存储设备中的基础空间容量、第一单位时长写入量对应的第一冗余空间量。
作为示例，存储管理设备中可以存储基础空间容量、单位时长写入量与冗余空间的对应关系，如表三所示。表三示出了单位时长写入量、基础空间容量与冗余空间量的对应关系，对于一定数值范围内的单位时长写入量，可以根据基础空间容量的不同而设置不同的冗余空间量。作为另一示例，也可以对于同一基础空间容量，可依据不同的单位时长写入量而设置不同的冗余空间量。本领域技术人员应该明白，表三所示出的仅仅作为示例，本领域技术人员可以根据实际需要对该对应关系进行相应的修改。
表三
Figure PCTCN2016084479-appb-000001
Figure PCTCN2016084479-appb-000002
存储管理设备检测到在目标存储设备中，目标网络业务的业务数据的第一单位时长写入量，还可以确定目标网络业务在目标存储设备中的基础空间容量，然后可以从预先存储的基础空间容量、单位时长写入量与冗余空间的对应关系中，确定目标网络业务在目标存储设备中的基础空间容量、第一单位时长写入量对应的第一冗余空间量。例如，目标网络业务可以是微信应用程序的业务，存储管理设备确定第一单位时长写入量为500M，然后确定目标存储设备为目标网络业务提供的基础空间容量为10T，则可以查找基础空间容量为10T，第一单位时长写入量为500M对应的冗余空间量为1T，这样第一冗余空间量可以确定为1T。
步骤103，根据第一冗余空间量，对目标网络业务在目标存储设备中的冗余空间进行设置。
作为示例，存储管理设备在确定第一冗余空间量后，可以将目标网络业务在目标存储设备中的冗余空间量设置为第一冗余空间量。
本发明实施例，还可以根据单位时长写入量确定数据清理阈值，相应的处理可以为：根据预先存储的单位时长写入量与数据清理阈值的对应关系，确定与第一单位时长写入量对应的第一数据清理阈值，数据清理阈值是用于触发目标存储设备对目标网络业务进行无效数据清理的目标网络业务在目标存储设备中的总数据量；将目标网络业务在目标存储设备中的数据清理阈值，设置为第一数据清理阈值。
作为示例，数据清理阈值也可以称为垃圾回收阈值，它可以是目标存储设备对目标网络业务的无效数据进行清理时，在目标存储设备中存储的目标网络业务的业务数据量。
存储管理设备中可以存储有单位时长写入量与数据清理阈值的对应关系，目标网络业务的业务数据单位时长写入量越大，数据清理阈值可以越小，这样可以保证目标存储设备有足够空间容量写入目标网络业务的业务数据。单位时长写入量与数据清理阈值的对应关系如表四所示。作为示例，表四示出了单位时长写入量与数据清理阈值之间的一种对应关系，这里，以单位时长写入数据 量表示单位时长写入量。如表四所示，可根据单位时长写入量所属的数值范围来确定与该单位时长写入量对应的数据清理阈值。应该理解，所给出的对应关系只是一种示例，本领域技术人员可以根据实际需要来对该对应关系进行修改。
表四
单位时长写入量 数据清理阈值
0～300M 97T
300～600M 95T
… …
作为示例，存储管理设备在确定第一单位时长写入量后，可以根据单位时长写入量与数据清理阈值的对应关系，确定第一数据清理阈值，然后可以将目标网络业务在目标存储设备中的数据清理阈值，设置为第一数据清理阈值。例如，目标存储设备为微信应用程序提供的基础空间容量为100T，当存储管理设备检测到微信应用程序的第一单位时长写入量为500M，存储管理设备可以确定第一单位时长写入量属于300M～600M的数值范围内，则可以查找与数值范围为300M～600M对应的垃圾回收阈值为95T，然后可以将微信应用程序在目标存储设备中的数据清理阈值设置为95T，也就是当微信应用程序的业务数据在目标存储设备中的数据量为95T时，可以触发目标存储设备对微信应用程序的业务数据进行清理。
表一至表四都是以单位时长写入数据量作为单位时长写入量，本领域技术人员应该理解，可以以单位时长写入次数作为单位时长写入量来确定空间冗余量。
本发明实施例，还可以根据冗余空间量确定无效数据阈值，相应的处理可以为：根据预先存储的冗余空间量与无效数据阈值的对应关系，确定第一冗余空间量对应的第一无效数据阈值，无效数据阈值是用于触发目标存储设备对目标网络业务进行无效数据清理的目标网络业务在目标存储设备中无效数据量与总数据量的比值；将目标网络业务在目标存储设备中的无效数据阈值，设置为第一无效数据阈值。
作为示例，无效数据阈值也可以称为脏数据阈值，可以是当目标存储设备对目标网络业务进行无效数据清理时，目标网络业务在目标存储设备中无效数据量与总数据量的比值。
存储管理设备中可以存储冗余空间量与无效数据阈值的对应关系，在对应关系中，冗余空间比较大时，对应的无效数据阈值也比较大，这样，可以等到无效数据比较多时进行清理，需要复制的有效数据比较少，降低了写放大。如表五所示。在表5中，冗余空间量以冗余空间容量来表示。
表5
冗余空间量 无效数据阈值
50M 80％
100M 87％
… …
存储管理设备在确定目标网络业务的第一冗余空间量后，可以从冗余空间量与无效数据阈值的对应关系中，确定第一冗余空间量对应的第一无效数据阈值，然后可以将目标网络业务在目标存储设备中的无效数据阈值设置为第一无效数据阈值，这样，当目标网络业务往目标存储设备中存储业务数据时，当业务数据的无效数据与目标网络业务在目标存储设备中的总数据量的比值为第一无效数据阈值时，触发目标存储设备对目标网络业务的无效数据进行清理。作为一种示例，目标存储设备为网络业务提供的存储空间可以分为多个存储块，写入数据时可以往多个存储块上同时写入，在进行无效数据清理时优先清理无效数据比例相对比较大的存储块。
本发明实施例，还提供了存储目标网络业务的业务数据的过程，相应的处理可以如下：
如图2所示，目标网络业务可以安装在存储管理设备上，也可以设置在其他服务端设备上，存储管理设备中可以设置有虚拟磁盘、闪存转换层、底层驱动等，虚拟磁盘可以用于给目标网络业务的业务数据提供逻辑存储空间，闪存转换层可以用于存储目标网络业务存储业务数据时需要的对应关系(如单位时长写入量与冗余空间量的对应关系)等、底层驱动可以用于驱动目标存储设备。
如图3所示，多个网络业务(如目标网络业务、网络业务A等)可以安装在一个存储管理设备上，或者将所述多个网络业务设置在不同的服务端设备上，但是多个网络业务可以共用存储管理设备的闪存转换层，并且可以共用一个目标存储设备，每个网络业务都有自身对应的虚拟磁盘，虚拟磁盘个数可以由单位时长写入量决定，其中，虚拟磁盘个数可以与单位时长写入量成正比。技术 人员可以按照经验值设置单位时长写入量与对应的冗余空间量、数据清理阈值的对应关系等，并且存储至闪存转换层，还可以在闪存转换层中设置注册接口、控制接口、状态查询接口等，注册接口可以用于网络业务查询单位时长写入量对应的冗余空间量、数据清理阈值等，控制接口可以用于根据单位时长写入量确定虚拟磁盘个数，状态查询接口可以用于网络业务检测单位时长写入量。另外，还可以在闪存转换层中设置执行部分、并发访问部分等。执行部分可以用于确定地址映射关系，也即由网络业务记录的存储地址的逻辑地址与由目标存储设备记录的存储地址的物理地址的映射关系。并发访问部分可以用于同时访问目标存储设备的多个存储块，其中，目标存储设备为该网络业务提供的存储空间可以分为多个存储块，写入数据时可以往多个存储块上同时写入。
当目标网络业务往目标存储设备中存储业务数据时，目标网络业务可以检测预设时长内的单位时长写入量，然后根据单位时长写入量，确定该单位时长写入量对应的冗余空间量、数据清理阈值等，然后将业务数据发送至虚拟磁盘，虚拟磁盘可以将该业务数据发送至闪存转换层，执行部分可以接收业务数据，并且确定业务数据的逻辑地址与物理地址的映射关系，然后并发访问部分可以通过底层驱动，将业务数据按照地址映射关系存储至目标存储设备中相应的存储块。
另外，存储管理设备在控制目标网络业务往目标存储设备存储业务数据时，还可以按照该业务数据的优先级进行存储，也就是当前的业务数据的优先级比较高时，可以先写入该业务数据，该业务数据写入完成后，再写入其他业务数据。
本发明实施例中，检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量；根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。这样，单位时长写入量增加时，可以自动增加冗余空间量，降低数据清理的频率，从而延长目标存储设备的使用寿命。
实施例二
基于相同的技术构思，本发明实施例还提供了一种管理存储设备的装置， 如图4所示，所述装置包括：
检测模块410，用于检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；
确定模块420，用于根据预先存储的单位时长写入量与冗余空间量的对应关系，确定所述第一单位时长写入量对应的第一冗余空间量,其中，在所述对应关系中，冗余空间量随单位时长写入量的增大而变大；
设置模块430，用于根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。
可选地，所述单位时长写入量包括：单位时长写入次数或单位时长写入数据量。
可选地，所述冗余空间量包括：冗余空间容量，或冗余空间容量相对于所述目标网络业务在所述目标存储设备中的基础空间容量的比值。
可选地，所述确定模块420还用于：
根据预先存储的单位时长写入量与数据清理阈值的对应关系，确定与所述第一单位时长写入量对应的第一数据清理阈值，所述数据清理阈值是用于触发所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中的总数据量；
所述设置模块430还用于：
将所述目标网络业务在所述目标存储设备中的数据清理阈值，设置为所述第一数据清理阈值。
可选地，所述确定模块420还用于：
根据预先存储的冗余空间量与无效数据阈值的对应关系，确定所述第一冗余空间量对应的第一无效数据阈值，所述无效数据阈值是用于触发所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中无效数据量与总数据量的比值；
所述设置模块430还用于：
将所述目标网络业务在目标存储设备中的无效数据阈值，设置为所述第一无效数据阈值。
作为一种示例，如图5所示，参考图4所描述的所述确定模块420可包括检测子模块421和确定子模块422，其中：
所述检测子模块421，用于检测所述目标网络业务在所述目标存储设备中的基础空间容量；
所述确定子模块422，用于根据预先存储的基础空间容量、单位时长写入量与冗余空间的对应关系，确定与所述目标网络业务在所述目标存储设备中的基础空间容量、所述第一单位时长写入量对应的第一冗余空间量。
本发明实施例中，检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，在所述对应关系中，冗余空间量随单位时长写入量的增大而变大；根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。这样，单位时长写入量增加时，可以自动增加冗余空间量，降低数据清理的频率，从而延长目标存储设备的使用寿命。
需要说明的是：上述实施例提供的管理存储设备的装置在管理存储设备时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配为由不同的功能模块完成，即将管理存储设备的装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的管理存储设备的装置与管理存储设备的方法实施例属于同一构思，其具体实现过程参见方法实施例，这里不再赘述。
实施例三
请参考图6，其示出了本发明实施例所涉及的存储管理设备的硬件结构的示意图，该存储管理设备可以用于实施上述实施例中提供的管理存储设备的方法。具体来讲：
所述存储管理设备可以被配置为图6中所示出的任务执行服务器600。所述任务执行服务器600可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器1922(例如，一个或一个以上处理器)和存储器1932，一个或一个以上存储应用程序1942或数据1944的存储介质1930(例如一个或一个以上海量存储设备)。其中，存储器1932和存储介质1930可以是短暂存储或持久存储。存储在存储介质1930的程序可以包括一个或一个以上模块(图中没有示出)，每个模块可以包括对任务执行服务器中的一系列指令操作。更进一 步地，中央处理器1922可以设置为与存储介质1930通信，在任务执行服务器600上执行存储介质1930中的一系列指令操作。
任务执行服务器600还可以包括一个或一个以上电源1926，一个或一个以上有线或无线网络接口1950，一个或一个以上输入输出接口1958，一个或一个以上键盘1956，和/或，一个或一个以上操作系统1941，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM等等。任务执行服务器600可以包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于执行上述管理存储设备的方法的指令。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (13)

  1. 一种管理存储设备的方法，其特征在于，所述方法包括：
    检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；
    根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，在所述对应关系中，冗余空间量随单位时长写入量的增大而变大；
    根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。
  2. 根据权利要求1所述的方法，其特征在于，所述单位时长写入量包括：单位时长写入次数或单位时长写入数据量。
  3. 根据权利要求1所述的方法，其特征在于，所述冗余空间量包括：冗余空间容量，或冗余空间容量相对于所述目标网络业务在所述目标存储设备中的基础空间容量的比值。
  4. 根据权利要求1所述的方法，其特征在于，所述根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，包括：
    检测所述目标网络业务在所述目标存储设备中的基础空间容量；
    根据预先存储的基础空间容量、单位时长写入量与冗余空间的对应关系，确定与所述目标网络业务在所述目标存储设备中的基础空间容量和所述第一单位时长写入量对应的第一冗余空间量。
  5. 根据权利要求1-4中的任一项所述的方法，其特征在于，所述方法还包括：
    根据预先存储的单位时长写入量与数据清理阈值的对应关系，确定与所述第一单位时长写入量对应的第一数据清理阈值，所述数据清理阈值是用于触发 所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中的总数据量；
    将所述目标网络业务在目标存储设备中的数据清理阈值，设置为所述第一数据清理阈值。
  6. 根据权利要求1-4中任一项所述的方法，其特征在于，所述方法还包括：
    根据预先存储的冗余空间量与无效数据阈值的对应关系，确定与所述第一冗余空间量对应的第一无效数据阈值，所述无效数据阈值是用于触发所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中无效数据量与总数据量的比值；
    将所述目标网络业务在所述目标存储设备中的无效数据阈值，设置为所述第一无效数据阈值。
  7. 一种管理存储设备的装置，其特征在于，所述装置包括：
    检测模块，用于检测在预设时长内，目标网络业务的业务数据在目标存储设备中的第一单位时长写入量；
    确定模块，用于根据预先存储的单位时长写入量与冗余空间量的对应关系，确定与所述第一单位时长写入量对应的第一冗余空间量，其中，冗余空间量随单位时长写入量的增大而变大；
    设置模块，用于根据所述第一冗余空间量，对所述目标网络业务在目标存储设备中的冗余空间进行设置。
  8. 根据权利要求7所述的装置，其特征在于，所述单位时长写入量包括：单位时长写入次数或单位时长写入数据量。
  9. 根据权利要求7所述的装置，其特征在于，所述冗余空间量包括：冗余空间容量，或冗余空间容量相对于所述目标网络业务在所述目标存储设备中的基础空间容量的比值。
  10. 根据权利要求7所述的装置，其特征在于，所述确定模块包括检测子 模块和确定子模块，其中：
    所述检测子模块，用于检测所述目标网络业务在所述目标存储设备中的基础空间容量；
    所述确定子模块，用于根据预先存储的基础空间容量、单位时长写入量与冗余空间的对应关系，确定与所述目标网络业务在所述目标存储设备中的基础空间容量和所述第一单位时长写入量对应的第一冗余空间量。
  11. 根据权利要求7-10中的任一项所述的装置，其特征在于，所述确定模块还用于：
    根据预先存储的单位时长写入量与数据清理阈值的对应关系，确定与所述第一单位时长写入量对应的第一数据清理阈值，所述数据清理阈值是用于触发所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中的总数据量；
    所述设置模块还用于：
    将所述目标网络业务在所述目标存储设备中的数据清理阈值，设置为所述第一数据清理阈值。
  12. 根据权利要求7-10中的任一项所述的装置，其特征在于，所述确定模块还用于：
    根据预先存储的冗余空间量与无效数据阈值的对应关系，确定与所述第一冗余空间量对应的第一无效数据阈值，所述无效数据阈值是用于触发所述目标存储设备对所述目标网络业务进行无效数据清理的所述目标网络业务在所述目标存储设备中无效数据量与总数据量的比值；
    所述设置模块还用于：
    将所述目标网络业务在目标存储设备中的无效数据阈值，设置为所述第一无效数据阈值。
  13. 一种计算机可读存储介质，在所述计算机可读存储介质上存储有用于执行如权利要求1-6中的任一项所述的方法的程序指令。
PCT/CN2016/084479 2015-12-17 2016-06-02 管理存储设备的方法及装置 WO2017101263A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2017562727A JP6522168B2 (ja) 2015-12-17 2016-06-02 ストレージデバイスを管理するための方法および装置
EP16874327.6A EP3392754B1 (en) 2015-12-17 2016-06-02 Method and apparatus for managing storage device
KR1020187000330A KR102100571B1 (ko) 2015-12-17 2016-06-02 스토리지 디바이스를 관리하는 방법 및 장치
US15/707,860 US10198180B2 (en) 2015-12-17 2017-09-18 Method and apparatus for managing storage device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510946978.1 2015-12-17
CN201510946978.1A CN106897022B (zh) 2015-12-17 2015-12-17 一种管理存储设备的方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/707,860 Continuation US10198180B2 (en) 2015-12-17 2017-09-18 Method and apparatus for managing storage device

Publications (1)

Publication Number Publication Date
WO2017101263A1 true WO2017101263A1 (zh) 2017-06-22

Family

ID=59055650

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/084479 WO2017101263A1 (zh) 2015-12-17 2016-06-02 管理存储设备的方法及装置

Country Status (6)

Country Link
US (1) US10198180B2 (zh)
EP (1) EP3392754B1 (zh)
JP (1) JP6522168B2 (zh)
KR (1) KR102100571B1 (zh)
CN (1) CN106897022B (zh)
WO (1) WO2017101263A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107943605B (zh) * 2017-11-14 2021-03-19 青岛海信移动通信技术股份有限公司 一种存储卡处理方法及装置
CN108733320A (zh) * 2018-04-03 2018-11-02 郑州云海信息技术有限公司 一种ssd性能和容量配置方法
CN110058810B (zh) * 2019-01-17 2022-06-17 创新先进技术有限公司 分布式系统的磁盘清理方法和装置、分布式系统
CN111078588B (zh) * 2019-12-17 2022-03-25 北京三快在线科技有限公司 垃圾回收方法、装置、设备及存储介质
CN111194055B (zh) * 2019-12-30 2022-09-09 广东博智林机器人有限公司 数据存储频率处理方法、装置、电子设备以及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102053802A (zh) * 2010-12-31 2011-05-11 中国科学院计算技术研究所 网络独立冗余磁盘阵列系统
CN103713853A (zh) * 2012-09-28 2014-04-09 国际商业机器公司 用于数据管理的方法和系统

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005242897A (ja) * 2004-02-27 2005-09-08 Oki Electric Ind Co Ltd フラッシュディスク装置
KR101089150B1 (ko) * 2005-08-03 2011-12-02 쌘디스크 코포레이션 스케쥴링된 리클레임 작동들을 갖는 비휘발성 메모리
JP2008171103A (ja) * 2007-01-10 2008-07-24 Seiko Epson Corp 半導体記憶装置管理システム、半導体記憶装置、半導体記憶装置管理システムの制御方法およびプログラム
US8214580B2 (en) * 2009-10-23 2012-07-03 International Business Machines Corporation Solid state drive with adjustable drive life and capacity
US8677203B1 (en) * 2010-01-11 2014-03-18 Apple Inc. Redundant data storage schemes for multi-die memory systems
JP2011192260A (ja) * 2010-02-16 2011-09-29 Toshiba Corp 半導体記憶装置
US9678676B2 (en) 2011-05-24 2017-06-13 Marvell World Trade Ltd. Method for storage devices to achieve low write amplification with low over provision
US8671249B2 (en) * 2011-07-22 2014-03-11 Fusion-Io, Inc. Apparatus, system, and method for managing storage capacity recovery
CN103379156B (zh) * 2012-04-24 2016-03-09 深圳市腾讯计算机系统有限公司 实现存储空间动态均衡的方法、系统和装置
US9645917B2 (en) 2012-05-22 2017-05-09 Netapp, Inc. Specializing I/O access patterns for flash storage
JP5788369B2 (ja) 2012-09-12 2015-09-30 株式会社東芝 メモリシステム、コンピュータシステム、およびメモリ管理方法
WO2014101086A1 (zh) * 2012-12-28 2014-07-03 华为技术有限公司 存储空间的处理方法、装置及非易失性计算机可读存储介质
GB2511325A (en) * 2013-02-28 2014-09-03 Ibm Cache allocation in a computerized system
US9645924B2 (en) * 2013-12-16 2017-05-09 International Business Machines Corporation Garbage collection scaling
CN103838524B (zh) * 2014-03-13 2017-01-18 深圳市迪菲特科技股份有限公司 一种自适应精简配置的实现方法与存储系统
US9632702B2 (en) * 2014-10-15 2017-04-25 International Business Machines Corporation Efficient initialization of a thinly provisioned storage array

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102053802A (zh) * 2010-12-31 2011-05-11 中国科学院计算技术研究所 网络独立冗余磁盘阵列系统
CN103713853A (zh) * 2012-09-28 2014-04-09 国际商业机器公司 用于数据管理的方法和系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3392754A4 *

Also Published As

Publication number Publication date
EP3392754A1 (en) 2018-10-24
JP2018516410A (ja) 2018-06-21
JP6522168B2 (ja) 2019-05-29
KR20180015730A (ko) 2018-02-13
EP3392754A4 (en) 2019-06-26
US20180004409A1 (en) 2018-01-04
KR102100571B1 (ko) 2020-04-13
CN106897022A (zh) 2017-06-27
CN106897022B (zh) 2019-05-31
EP3392754B1 (en) 2022-10-19
US10198180B2 (en) 2019-02-05

Similar Documents

Publication Publication Date Title
WO2017101263A1 (zh) 管理存储设备的方法及装置
US9189410B2 (en) Hypervisor-based flash cache space management in a multi-VM environment
US9477407B1 (en) Intelligent migration of a virtual storage unit to another data storage system
US8909887B1 (en) Selective defragmentation based on IO hot spots
CN103714015B (zh) 减少来自侦听过滤器的后无效事务的方法、装置和系统
US11880579B2 (en) Data migration method and apparatus
EP3552109B1 (en) Systems and methods for caching data
US10860494B2 (en) Flushing pages from solid-state storage device
US9507720B2 (en) Block storage-based data processing methods, apparatus, and systems
US9323682B1 (en) Non-intrusive automated storage tiering using information of front end storage activities
US10241836B2 (en) Resource management in a virtualized computing environment
US20110072225A1 (en) Application and tier configuration management in dynamic page reallocation storage system
US9471495B2 (en) Method and apparatus for constructing memory access model
KR20120050891A (ko) 스토리지 시스템에서 요청에 대한 응답과 연관된 지연의 감소
US11416162B2 (en) Garbage collection method and storage device
US20180067854A1 (en) Aggressive write-back cache cleaning policy optimized for non-volatile memory
US10747668B2 (en) Efficient global cache partition and dynamic sizing for shared storage workloads
KR102513961B1 (ko) 멀티 운영시스템을 지닌 전자장치 및 이의 동적 메모리 관리 방법
JP2016162397A (ja) ストレージ制御装置、制御システム及び制御プログラム
US9104317B2 (en) Computer system and method of controlling I/O with respect to storage apparatus
KR20190063378A (ko) 이종 가상화 클라우드 캐시 환경에서의 동적 캐시분할 관리자
US10176103B1 (en) Systems, devices and methods using a solid state device as a caching medium with a cache replacement algorithm
CN112631520A (zh) 分布式块存储系统、方法、装置、设备和介质
KR20230012440A (ko) 이기종 메모리 시스템에서의 로드 밸런싱을 위한 시스템 및 방법
US20150186401A1 (en) Using file element accesses to select file elements in a file system to defragment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16874327

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017562727

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20187000330

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE