WO2021088423A1 - 一种用于raid io的内存管理方法、系统、终端及存储介质 - Google Patents

一种用于raid io的内存管理方法、系统、终端及存储介质 Download PDF

Info

Publication number
WO2021088423A1
WO2021088423A1 PCT/CN2020/104007 CN2020104007W WO2021088423A1 WO 2021088423 A1 WO2021088423 A1 WO 2021088423A1 CN 2020104007 W CN2020104007 W CN 2020104007W WO 2021088423 A1 WO2021088423 A1 WO 2021088423A1
Authority
WO
WIPO (PCT)
Prior art keywords
raid
memory
execution
request
management
Prior art date
Application number
PCT/CN2020/104007
Other languages
English (en)
French (fr)
Inventor
邸忠辉
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2021088423A1 publication Critical patent/WO2021088423A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory

Definitions

  • the present invention relates to the technical field of data storage, in particular to a memory management method, system, terminal and storage medium for RAID IO.
  • the performance and security of data storage are the core issues that users care about.
  • the memory usage of the storage system can greatly affect the stability and performance of the storage system.
  • the storage system is quite complex, with up to millions or tens of millions of lines of code, which places high demands on the use and management of memory.
  • Memory management is especially important in the IO (Input/Output) of RAID (Redundant Arrays of Independent Disks).
  • IO Input/Output
  • RAID Redundant Arrays of Independent Disks
  • the memory used on the IO path of RAID needs to be applied for multiple times from multiple memory pools. It will be released immediately after the memory is used up many times, although this is for the full and fine-grained management and utilization of the memory. But first, because the data cache on the entire RAID IO path is the main memory used, and the small memory of other management structures has a large amount, but the total size is very small; secondly, it is better to apply and release the memory frequently.
  • the present invention provides a memory management method, system, terminal, and storage medium for RAID IO to solve the above-mentioned technical problems.
  • the present invention provides a memory management method for RAID IO, including:
  • the generating of the memory parameters according to the category to which the RAID IO belongs includes:
  • the applying for memory for a RAID IO operation according to memory parameters and executing the RAID IO operation after the application is completed includes:
  • the RAID IO request is stored in the waiting list.
  • the monitoring the execution result of the RAID IO operation includes:
  • the failed operation stripe still fails to execute after the number of repeated executions exceeds the preset number of times, the failed operation stripe is saved to the corresponding RAID management.
  • the present invention provides a memory management system for RAID IO, including:
  • Operation classification unit configured to classify RAID IO according to operation process
  • Memory computing unit configured to generate memory parameters according to the category of RAID IO;
  • a memory application unit configured to apply for memory for a RAID IO operation according to the memory parameters, and execute the RAID IO operation after the application is completed;
  • the memory release unit is configured to monitor the execution result of the RAID IO operation, and release the requested memory after the execution of the RAID IO operation is completed.
  • the memory calculation unit includes:
  • the structure calculation module is configured to calculate the management structure corresponding to each operation flow category according to the number of strips in the operation flow category;
  • the memory calculation module is configured to calculate the memory size required for each type of RAID IO operation according to the management structure and the corresponding type of RAID IO operation execution data.
  • the memory application unit includes:
  • a memory request module configured to request memory at the beginning of the RAID IO operation process
  • Structural construction module configured to construct and initialize the management structure of RAID IO operation if the memory request is successful
  • the request saving module is configured to store the RAID IO request in the waiting list if the memory request fails.
  • the memory release unit includes:
  • the result collection module is configured to collect the execution results of RAID IO operations
  • the repeated execution module is configured to repeat execution of the failed operation strip if there is a failed operation strip in the execution result;
  • the failure location module is configured to save the failed operation strip to the corresponding RAID management if the failed operation strip still fails to execute after the number of repeated executions exceeds the preset number of times.
  • a terminal including a processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.
  • a computer storage medium stores instructions that, when run on a computer, cause the computer to execute the methods described in the above aspects.
  • the beneficial effect of the present invention is that the memory management method, system, terminal, and storage medium for RAID IO provided by the present invention classify different RAID IO operations to calculate the memory size required for different IO operations. After receiving the IO request, apply for the required memory at one time according to the RAID IO operation type. The memory is used to construct all the management structures and data caches of this RAID IO operation. After the RAID IO operation is completed, the memory is released at one time. Since the required memory for the entire RAID IO process is applied at the beginning of RAID IO, there will be no complicated operations such as waiting or failure due to insufficient memory during the RAID IO process, which simplifies the process.
  • Allocating and releasing memory once is more efficient than allocating and releasing memory multiple times, consumes less time, and has better performance.
  • all the memory is released at one time, which also reduces the chance of problems such as referencing the released memory, repeated memory release, and memory leakage, making the storage system more robust.
  • the operation information is more completely retained, and it is easier to locate and debug after storage problems. So this optimizes the implementation of the system and greatly improves the robustness of the storage system.
  • the design principle of the present invention is reliable, the structure is simple, and it has a very wide application prospect.
  • Fig. 1 is a schematic flowchart of a method according to an embodiment of the present invention
  • Figure 2 is a schematic flowchart of a method according to an embodiment of the present invention.
  • Fig. 3 is a schematic block diagram of a system according to an embodiment of the present invention.
  • Fig. 4 is a schematic structural diagram of a terminal provided by an embodiment of the present invention.
  • RAID IO Disk array input and output operations.
  • FIG. 1 is a schematic flowchart of a method according to an embodiment of the present invention.
  • the execution subject of FIG. 1 may be a memory management system for RAID IO.
  • the method 100 includes:
  • Step 110 Classify RAID IO according to the operation process
  • Step 120 Generate memory parameters according to the category to which the RAID IO belongs;
  • Step 130 Apply for memory for a RAID IO operation according to the memory parameter, and execute the RAID IO operation after the application is completed;
  • Step 140 Monitor the execution result of the RAID IO operation, and release the requested memory after the execution of the RAID IO operation is completed.
  • the generating of the memory parameters according to the category to which the RAID IO belongs includes:
  • the requesting memory for a RAID IO operation according to memory parameters and executing the RAID IO operation after the application is completed includes:
  • the RAID IO request is stored in the waiting list.
  • the monitoring the execution result of the RAID IO operation includes:
  • the failed operation stripe still fails to execute after the number of repeated executions exceeds the preset number of times, the failed operation stripe is saved to the corresponding RAID management.
  • the memory management method for RAID IO includes:
  • RAID IO operations are classified, and the amount of memory used is different due to different RAID IO operation processes. Because RAID data is accessed in accordance with striping, each RAID IO operation is classified according to the number of strips in the process.
  • the required memory management structures are: one TCB is used to manage the data structure of this IO operation, one TCB corresponds to three SIO structures to manage the write process of three data strips, each SIO corresponds to multiple IPKs, and each IPK manages A data block write operation process; it also includes the TCS structure related to this IO operation, the structure of each stripe FSW, and the data structure such as xor required in each block operation; it may also include some pairs of faults Management structure for processing, etc.
  • memory is requested at the beginning of the RAID IO process operation. If the memory request is successful, the management structure TCB for the RAID IO operation is constructed and initialized, and the IO process is continued; if the memory allocation fails, the system memory is already tight. Put the IO request into the waiting list, when the memory is released, remove it from the waiting list, continue to request the memory, if it succeeds, continue the IO process, if it fails, still put it back into the request list.
  • the IO operation is split into three strips of operation, and each strip is further split into operations for each block of data. Since the management structure of each stripe and block operation has already applied for memory, it only needs to be initialized. After a stripe or block operation is completed, the memory of its management structure is not released.
  • the IO operation returns a non-retryable error, or returns a retry error, but the number of retry failures exceeds the set number or all stripes and blocks return success, that is, all the blocks of the RAID IO operation return the final execution result and release all , And return the corresponding operation result.
  • the system 300 includes:
  • the operation classification unit 310 is configured to classify the RAID IO according to the operation process
  • the memory calculation unit 320 is configured to generate memory parameters according to the category to which the RAID IO belongs;
  • the memory application unit 330 is configured to apply for memory for a RAID IO operation according to the memory parameters, and execute the RAID IO operation after the application is completed;
  • the memory release unit 340 is configured to monitor the execution result of the RAID IO operation, and release the requested memory after the execution of the RAID IO operation is completed.
  • the memory calculation unit includes:
  • the structure calculation module is configured to calculate the management structure corresponding to each operation flow category according to the number of strips in the operation flow category;
  • the memory calculation module is configured to calculate the memory size required for each type of RAID IO operation according to the management structure and the corresponding type of RAID IO operation execution data.
  • the memory application unit includes:
  • a memory request module configured to request memory at the beginning of the RAID IO operation process
  • Structural construction module configured to construct and initialize the management structure of RAID IO operation if the memory request is successful
  • the request saving module is configured to save the RAID IO request to the waiting list if the memory request fails.
  • the memory release unit includes:
  • the result collection module is configured to collect the execution results of RAID IO operations
  • the repeated execution module is configured to repeat execution of the failed operation strip if there is a failed operation strip in the execution result;
  • the failure location module is configured to save the failed operation strip to the corresponding RAID management if the failed operation strip still fails to execute after the number of repeated executions exceeds the preset number of times.
  • FIG. 4 is a schematic structural diagram of a terminal system 400 provided by an embodiment of the present invention.
  • the terminal system 400 may be used to execute the memory management method for RAID IO provided by the embodiment of the present invention.
  • the terminal system 400 may include: a processor 410, a memory 420, and a communication unit 430.
  • the structure of the server shown in the figure does not constitute a limitation to the present invention. It can be a bus-shaped structure, a star-shaped structure, or It may include more or fewer components than shown, or combine certain components, or different component arrangements.
  • the memory 420 can be used to store the execution instructions of the processor 410, and the memory 420 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electronic Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Memory, Flash Memory, Disk or Optical Disk .
  • SRAM static random access memory
  • EEPROM electronic Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • ROM Read-Only Memory
  • magnetic Memory Flash Memory
  • Disk Disk
  • Disk Disk
  • Optical Disk Optical Disk
  • the processor 410 is the control center of the storage terminal. It uses various interfaces and lines to connect the various parts of the entire electronic terminal. By running or executing software programs and/or modules stored in the memory 420, and calling data stored in the memory, To perform various functions of the electronic terminal and/or process data.
  • the processor may be composed of an integrated circuit (Integrated Circuit, IC for short), for example, it may be composed of a single packaged IC, or may be composed of connecting multiple packaged ICs with the same function or different functions.
  • the processor 410 may only include a central processing unit (Central Processing Unit, CPU for short).
  • the CPU may be a single computing core, or it may include multiple computing cores.
  • the communication unit 430 is configured to establish a communication channel, so that the storage terminal can communicate with other terminals. Receive user data sent by other terminals or send user data to other terminals.
  • the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments provided by the present invention when the program is executed.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM), etc.
  • the present invention classifies different RAID IO operations, and then calculates the memory size required for different IO operations.
  • the required memory is requested at one time according to the RAID IO operation type, and the memory is used for Construct all management structures and data caches for this RAID IO operation.
  • the memory is released at one time. Since the required memory for the entire RAID IO process is applied at the beginning of RAID IO, there will be no complicated operations such as waiting or failure due to insufficient memory during the RAID IO process, which simplifies the process. Allocating and releasing memory once is more efficient than allocating and releasing memory multiple times, consumes less time, and has better performance.
  • the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention can be embodied in the form of a software product in essence or a part that contributes to the prior art.
  • the computer software product is stored in a storage medium such as a USB flash drive or a mobile device.
  • a computer terminal It can be a personal computer, a server, or a second terminal, a network terminal, etc.
  • the disclosed system and method can be implemented in other ways.
  • the system embodiment described above is only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, systems or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

一种用于RAID IO的内存管理方法、系统、终端及存储介质，包括：将RAID IO按操作流程分类；根据RAID IO所属类别生成内存参数；根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。本发明在RAID IO开始的时候就申请了整个RAID IO流程的所需内存，因此不会存在RAID IO流程的进行中出现内存不足从而进行等待或者失败的各种复杂操作，简化了流程。一次分配释放内存，也比多次分配释放内存效率更高，耗时更少，性能更好。

Description

一种用于RAID IO的内存管理方法、系统、终端及存储介质
本申请要求于2019年11月8日提交中国专利局、申请号为201911089362.1、发明名称为“一种用于raid io的内存管理方法、系统、终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及数据存储技术领域，具体涉及一种用于RAID IO的内存管理方法、系统、终端及存储介质。
背景技术
在云计算数据中心中，数据存储的性能和安全性是用户关心的核心问题。而存储系统得内存使用能够很大程度上影响存储系统得稳定性和性能。存储系统相当复杂，代码多达数百万行或千万行，对内存的使用和管理提出了很高的要求。
在RAID(Redundant Arrays of Independent Disks，磁盘阵列)的IO(Input/Output，输入/输出)方面内存管理尤其重要，通常RAID的IO路径上使用的内存需要分多次从多个内存池申请，并分多次在内存使用完后立即释放，这虽然是为了充分和精细化的对内存管理和利用。但是首先由于整个RAID IO路径上的数据缓存为主要使用的内存，而其他管理结构的小内存数量繁多，总量大小却占比非常小；其次，频繁的对内存进行申请和释放的操作，不如一次申请和释放性能更好；第三，如果系统并发量很大，可能存在大量RAID IO进行到一半时，内存或内存池耗尽，批量IO进行等待，甚至导致系统卡住；第四，由于代码庞大，RAID IO路径上涉及内存分配和使用的地方非常多，经常造成引用已释放内存、内存重复释放、内存泄露等问题发生，这类问题解决起来非常困难，经常耗费大量人力和时间。
发明内容
针对现有技术的上述不足，本发明提供一种用于RAID IO的内存管理方法、系统、终端及存储介质，以解决上述技术问题。
第一方面，本发明提供一种用于RAID IO的内存管理方法，包括：
将RAID IO按操作流程分类；
根据RAID IO所属类别生成内存参数；
根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
进一步的，所述根据RAID IO所属类别生成内存参数，包括：
根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
进一步的，所述根据内存参数为RAID IO操作申请内存并在申请完成后执行所述RAID IO操作，包括：
在所述RAID IO操作流程的开始阶段请求内存；
若内存请求成功，则构造并初始化RAID IO操作的管理结构；
若内存请求失败，则将RAID IO请求存入等待链表。
进一步的，所述监控所述RAID IO操作执行结果，包括：
采集RAID IO操作执行结果；
若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
第二方面，本发明提供一种用于RAID IO的内存管理系统，包括：
操作分类单元，配置用于将RAID IO按操作流程分类；
内存计算单元，配置用于根据RAID IO所属类别生成内存参数；
内存申请单元，配置用于根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
内存释放单元，配置用于监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
进一步的，所述内存计算单元包括：
结构计算模块，配置用于根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
内存计算模块，配置用于根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
进一步的，所述内存申请单元包括：
内存请求模块，配置用于在所述RAID IO操作流程的开始阶段请求内存；
结构构造模块，配置用于若内存请求成功，则构造并初始化RAID IO操作的管理结构；
请求保存模块，配置用于若内存请求失败，则将RAID IO请求存入等待链表。
进一步的，所述内存释放单元包括：
结果采集模块，配置用于采集RAID IO操作执行结果；
重复执行模块，配置用于若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
失败定位模块，配置用于若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
第三方面，提供一种终端，包括：处理器、存储器，其中，该存储器用于存储计算机程序，该处理器用于从存储器中调用并运行该计算机程序，使得终端执行上述的终端的方法。
第四方面，提供了一种计算机存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述各方面所述的方法。
本发明的有益效果在于，本发明提供的用于RAID IO的内存管理方法、系统、终端及存储介质，通过把不同的RAID IO操作进行分类，进而计算 不同IO操作所需要的内存大小，在RAID收到IO请求后根据该RAID IO操作类型一次性申请所需要的内存，该内存用于构造本次RAID IO操作的所有管理结构及数据缓存。等到该RAID IO操作完成之后，一次性将内存释放。由于在RAID IO开始的时候就申请了整个RAID IO流程的所需内存，因此不会存在RAID IO流程的进行中出现内存不足从而进行等待或者失败的各种复杂操作，简化了流程。一次分配释放内存，也比多次分配释放内存效率更高，耗时更少，性能更好。而在RAID IO执行完成后一次释放所有内存，也降低了引用已释放内存、内存重复释放、内存泄露等问题发生的机会，使存储系统更加健壮。同时由于RAID IO过程中所有管理和数据信息都存在，较完整的保留了操作的信息，存储出现问题后也更容易定位和调试。所以这优化了系统的实现，大大提高了存储系统的健壮性。
此外，本发明设计原理可靠，结构简单，具有非常广泛的应用前景。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明一个实施例的方法的示意性流程图；
图2是本发明一个实施例的方法的示意性流程图；
图3是本发明一个实施例的系统的示意性框图；
图4为本发明实施例提供的一种终端的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本发明中的技术方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造 性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
下面对本发明中出现的关键术语进行解释。
RAID IO：磁盘阵列输入输出操作。
图1是本发明一个实施例的方法的示意性流程图。其中，图1执行主体可以为一种用于RAID IO的内存管理系统。
如图1所示，该方法100包括：
步骤110，将RAID IO按操作流程分类；
步骤120，根据RAID IO所属类别生成内存参数；
步骤130，根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
步骤140，监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
可选地，作为本发明一个实施例，所述根据RAID IO所属类别生成内存参数，包括：
根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
可选地，作为本发明一个实施例，所述根据内存参数为RAID IO操作申请内存并在申请完成后执行所述RAID IO操作，包括：
在所述RAID IO操作流程的开始阶段请求内存；
若内存请求成功，则构造并初始化RAID IO操作的管理结构；
若内存请求失败，则将RAID IO请求存入等待链表。
可选地，作为本发明一个实施例，所述监控所述RAID IO操作执行结果，包括：
采集RAID IO操作执行结果；
若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
为了便于对本发明的理解，下面以本发明用于RAID IO的内存管理方法的原理，结合实施例中对用于RAID IO的内存进行管理的过程，对本发明提供的用于RAID IO的内存管理方法做进一步的描述。
具体的，所述用于RAID IO的内存管理方法包括：
S1、将RAID IO按操作流程分类。
对不同的RAID IO操作进行分类，由于各种RAID IO操作流程不同造成对内存的使用量不同。由于RAID数据按照条带化进存取，因此对各RAID IO操作按流程的条带数量进行分类。
S2、根据RAID IO所属类别生成内存参数。
由于RAID IO流程的操作是可预测的，因此可以实现将各种IO操作管理结构及数据缓存的内存使用量计算出来。以RAID的写操作为例，由于RAID数据按照条带化进存取。因此假设请求写三个条带的数据。则需要的内存管理结构有：一个TCB用来管理本次IO操作的数据结构、一个TCB对应三个SIO结构管理三个数据条带的写流程、每个SIO对应多个IPK，每个IPK管理一个数据分块的写操作流程；同时还包含本次IO操作相关的TCS结构等及每个条带FSW等结构、每个分块操作中所需要的xor等数据结构；还可能包含部分对故障处理等的管理结构。
不同条带数量的操作对应不同的管理结构，每种管理结构占用的内存是定值，由此可以计算出不同类别的RAID IO操作所需的内存大小。
S3、根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作。
参考图2，在RAID IO流程操作的开始阶段请求内存，如果内存请求成功，则构造并初始化RAID IO操作的管理结构TCB，并继续IO流程；如果内存分配失败，说明系统的内存已经比较紧张。将IO请求放入等待链表，当有内存释放时，从等待链表中将其取出，继续请求内存，若成功，则继续IO流程，若失败，则仍然放回请求链表中。
S4、监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
将IO操作拆分成三个条带的操作，进一步对每个条带拆分成对每个分 块数据的操作。由于每个条带及分块操作的管理结构已经申请内存，只需要对其初始化。某个条带或分块操作完成后，并不释放其管理结构的内存。
当IO操作的某个条带或分块完成后，返回结果是非成功时，根据其错误码进行相应的错误处理。如果错误为超时或者媒介失败等。需要将该条带或者分块进行重试，此时由于所有IO管理内存结构及数据缓存仍然存在，可以直接使用。如果重试设定次数后仍然失败，或错误为非可重试错误，则返回错误码给上层，并将错误的RAID条带记录到相应的RAID管理中。以便进行坏块管理，磁盘离线等管理操作。
如果IO操作返回非可重试错误，或返回重试错误，但重试失败超过设定次数或所有条带和分块返回成功，即RAID IO操作的所有分块均返回最终执行结果则释放所有的内存，并返回相应的操作结果。
如图3示，该系统300包括：
操作分类单元310，配置用于将RAID IO按操作流程分类；
内存计算单元320，配置用于根据RAID IO所属类别生成内存参数；
内存申请单元330，配置用于根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
内存释放单元340，配置用于监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
可选地，作为本发明一个实施例，所述内存计算单元包括：
结构计算模块，配置用于根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
内存计算模块，配置用于根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
可选地，作为本发明一个实施例，所述内存申请单元包括：
内存请求模块，配置用于在所述RAID IO操作流程的开始阶段请求内存；
结构构造模块，配置用于若内存请求成功，则构造并初始化RAID IO操作的管理结构；
请求保存模块，配置用于若内存请求失败，则将RAID IO请求存入等 待链表。
可选地，作为本发明一个实施例，所述内存释放单元包括：
结果采集模块，配置用于采集RAID IO操作执行结果；
重复执行模块，配置用于若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
失败定位模块，配置用于若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
图4为本发明实施例提供的一种终端系统400的结构示意图，该终端系统400可以用于执行本发明实施例提供的用于RAID IO的内存管理方法。
其中，该终端系统400可以包括：处理器410、存储器420及通信单元430。
这些组件通过一条或多条总线进行通信，本领域技术人员可以理解，图中示出的服务器的结构并不构成对本发明的限定，它既可以是总线形结构，也可以是星型结构，还可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
其中，该存储器420可以用于存储处理器410的执行指令，存储器420可以由任何类型的易失性或非易失性存储终端或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。当存储器420中的执行指令由处理器410执行时，使得终端400能够执行以下上述方法实施例中的部分或全部步骤。
处理器410为存储终端的控制中心，利用各种接口和线路连接整个电子终端的各个部分，通过运行或执行存储在存储器420内的软件程序和/或模块，以及调用存储在存储器内的数据，以执行电子终端的各种功能和/或处理数据。所述处理器可以由集成电路(Integrated Circuit，简称IC)组成，例如可以由单颗封装的IC所组成，也可以由连接多颗相同功能或不同功能的封装IC而组成。举例来说，处理器410可以仅包括中央处理器(Central Processing Unit，简称CPU)。在本发明实施方式中，CPU可以是单运算核心，也可以包括多运算核心。
通信单元430，用于建立通信信道，从而使所述存储终端可以与其它终端进行通信。接收其他终端发送的用户数据或者向其他终端发送用户数据。
本发明还提供一种计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可包括本发明提供的各实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM)等。
因此，本发明通过把不同的RAID IO操作进行分类，进而计算不同IO操作所需要的内存大小，在RAID收到IO请求后根据该RAID IO操作类型一次性申请所需要的内存，该内存用于构造本次RAID IO操作的所有管理结构及数据缓存。等到该RAID IO操作完成之后，一次性将内存释放。由于在RAID IO开始的时候就申请了整个RAID IO流程的所需内存，因此不会存在RAID IO流程的进行中出现内存不足从而进行等待或者失败的各种复杂操作，简化了流程。一次分配释放内存，也比多次分配释放内存效率更高，耗时更少，性能更好。而在RAID IO执行完成后一次释放所有内存，也降低了引用已释放内存、内存重复释放、内存泄露等问题发生的机会，使存储系统更加健壮。同时由于RAID IO过程中所有管理和数据信息都存在，较完整的保留了操作的信息，存储出现问题后也更容易定位和调试。所以这优化了系统的实现，大大提高了存储系统的健壮性，本实施例所能达到的技术效果可以参见上文中的描述，此处不再赘述。
本领域的技术人员可以清楚地了解到本发明实施例中的技术可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本发明实施例中的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中如U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质，包括若干指令用以使得一台计算机终端(可以是个人计算机，服务器，或者第二终端、网络终端等)执行本发明各个实施例所述方法的全部或部分 步骤。
本说明书中各个实施例之间相同相似的部分互相参见即可。尤其，对于终端实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例中的说明即可。
在本发明所提供的几个实施例中，应该理解到，所揭露的系统和方法，可以通过其它的方式实现。例如，以上所描述的系统实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，系统或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
尽管通过参考附图并结合优选实施例的方式对本发明进行了详细描述，但本发明并不限于此。在不脱离本发明的精神和实质的前提下，本领域普通技术人员可以对本发明的实施例进行各种等效的修改或替换，而这些修改或替换都应在本发明的涵盖范围内/任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。

Claims (10)

  1. 一种用于RAID IO的内存管理方法，其特征在于，包括：
    将RAID IO按操作流程分类；
    根据RAID IO所属类别生成内存参数；
    根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
    监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
  2. 根据权利要求1所述的方法，其特征在于，所述根据RAID IO所属类别生成内存参数，包括：
    根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
    根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
  3. 根据权利要求1所述的方法，其特征在于，所述根据内存参数为RAID IO操作申请内存并在申请完成后执行所述RAID IO操作，包括：
    在所述RAID IO操作流程的开始阶段请求内存；
    若内存请求成功，则构造并初始化RAID IO操作的管理结构；
    若内存请求失败，则将RAID IO请求存入等待链表。
  4. 根据权利要求1所述的方法，其特征在于，所述监控所述RAID IO操作执行结果，包括：
    采集RAID IO操作执行结果；
    若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
    若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
  5. 一种用于RAID IO的内存管理系统，其特征在于，包括：
    操作分类单元，配置用于将RAID IO按操作流程分类；
    内存计算单元，配置用于根据RAID IO所属类别生成内存参数；
    内存申请单元，配置用于根据所述内存参数为RAID IO操作申请内存，并在申请完成后执行所述RAID IO操作；
    内存释放单元，配置用于监控所述RAID IO操作执行结果，在所述RAID IO操作执行完成后释放申请的内存。
  6. 根据权利要求5所述的系统，其特征在于，所述内存计算单元包括：
    结构计算模块，配置用于根据操作流程类别的条带数量计算各操作流程类别对应的管理结构；
    内存计算模块，配置用于根据所述管理结构和对应类别RAID IO操作执行数据计算各类别RAID IO操作所需内存大小。
  7. 根据权利要求5所述的系统，其特征在于，所述内存申请单元包括：
    内存请求模块，配置用于在所述RAID IO操作流程的开始阶段请求内存；
    结构构造模块，配置用于若内存请求成功，则构造并初始化RAID IO操作的管理结构；
    请求保存模块，配置用于若内存请求失败，则将RAID IO请求存入等待链表。
  8. 根据权利要求5所述的系统，其特征在于，所述内存释放单元包括：
    结果采集模块，配置用于采集RAID IO操作执行结果；
    重复执行模块，配置用于若执行结果中存在失败操作条带，则对所述失败操作条带重复执行；
    失败定位模块，配置用于若重复执行次数超过预设次数后所述失败操作条带仍执行失败，则将所述失败操作条带保存至相应RAID管理中。
  9. 一种终端，其特征在于，包括：
    处理器；
    用于存储处理器的执行指令的存储器；
    其中，所述处理器被配置为执行权利要求1-4任一项所述的方法。
  10. 一种存储有计算机程序的计算机可读存储介质，其特征在于，该程序被处理器执行时实现如权利要求1-4中任一项所述的方法。
PCT/CN2020/104007 2019-11-08 2020-07-24 一种用于raid io的内存管理方法、系统、终端及存储介质 WO2021088423A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911089362.1A CN111007987A (zh) 2019-11-08 2019-11-08 一种用于raid io的内存管理方法、系统、终端及存储介质
CN201911089362.1 2019-11-08

Publications (1)

Publication Number Publication Date
WO2021088423A1 true WO2021088423A1 (zh) 2021-05-14

Family

ID=70111042

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/104007 WO2021088423A1 (zh) 2019-11-08 2020-07-24 一种用于raid io的内存管理方法、系统、终端及存储介质

Country Status (2)

Country Link
CN (1) CN111007987A (zh)
WO (1) WO2021088423A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111007987A (zh) * 2019-11-08 2020-04-14 苏州浪潮智能科技有限公司 一种用于raid io的内存管理方法、系统、终端及存储介质
CN113608675B (zh) * 2021-06-25 2023-08-04 苏州浪潮智能科技有限公司 一种raid数据io处理方法、装置、计算机设备及介质
CN115129265B (zh) * 2022-09-01 2023-01-24 苏州浪潮智能科技有限公司 独立冗余磁盘阵列分块缓存方法、装置、设备及可读介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014036078A2 (en) * 2012-08-28 2014-03-06 Memory Technologies Llc Dynamic central cache memory
CN104881370A (zh) * 2015-05-11 2015-09-02 中国人民解放军国防科学技术大学 协同使用纠删码和纠错码的可靠闪存存储系统构建方法
CN106528551A (zh) * 2015-09-09 2017-03-22 北京国双科技有限公司 申请内存的方法和装置
CN106648907A (zh) * 2017-02-24 2017-05-10 郑州云海信息技术有限公司 一种提高内存复用的方法和装置
CN108509273A (zh) * 2018-03-22 2018-09-07 杭州迪普科技股份有限公司 一种内存处理方法及装置
CN111007987A (zh) * 2019-11-08 2020-04-14 苏州浪潮智能科技有限公司 一种用于raid io的内存管理方法、系统、终端及存储介质

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8078799B2 (en) * 2009-06-10 2011-12-13 Lsi Corporation Method and system of an adaptive input/output scheduler for storage arrays
US20110029728A1 (en) * 2009-07-28 2011-02-03 Lsi Corporation Methods and apparatus for reducing input/output operations in a raid storage system
US9058243B2 (en) * 2011-06-10 2015-06-16 International Business Machines Corporation Releasing blocks of storage class memory
CN102523410B (zh) * 2011-12-28 2014-12-10 创新科存储技术（深圳）有限公司 一种写入视频数据的方法及视频数据存储设备
US8526234B1 (en) * 2012-11-16 2013-09-03 Avalanche Technology, Inc. Controller management of memory array of storage device using magnetic random access memory (MRAM)
CN103150257A (zh) * 2013-02-28 2013-06-12 天脉聚源(北京)传媒科技有限公司 一种内存管理方法和装置
CN104166597B (zh) * 2013-05-17 2018-07-03 华为技术有限公司 一种分配远程内存的方法及装置
JP2015082313A (ja) * 2013-10-24 2015-04-27 富士通株式会社 Ｒａｉｄ構成管理装置、ｒａｉｄ構成管理プログラムおよびｒａｉｄ構成管理方法
US20160179411A1 (en) * 2014-12-23 2016-06-23 Intel Corporation Techniques to Provide Redundant Array of Independent Disks (RAID) Services Using a Shared Pool of Configurable Computing Resources
CN106776046B (zh) * 2017-01-13 2021-06-29 郑州云海信息技术有限公司 一种scst读写优化方法及系统
CN109062823A (zh) * 2018-07-16 2018-12-21 北京中科网威信息技术有限公司 使用大页内存提升基于申威架构的设备性能的方法及装置
CN109522121A (zh) * 2018-11-15 2019-03-26 郑州云海信息技术有限公司 一种内存申请方法、装置、终端及计算机可读存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014036078A2 (en) * 2012-08-28 2014-03-06 Memory Technologies Llc Dynamic central cache memory
CN104881370A (zh) * 2015-05-11 2015-09-02 中国人民解放军国防科学技术大学 协同使用纠删码和纠错码的可靠闪存存储系统构建方法
CN106528551A (zh) * 2015-09-09 2017-03-22 北京国双科技有限公司 申请内存的方法和装置
CN106648907A (zh) * 2017-02-24 2017-05-10 郑州云海信息技术有限公司 一种提高内存复用的方法和装置
CN108509273A (zh) * 2018-03-22 2018-09-07 杭州迪普科技股份有限公司 一种内存处理方法及装置
CN111007987A (zh) * 2019-11-08 2020-04-14 苏州浪潮智能科技有限公司 一种用于raid io的内存管理方法、系统、终端及存储介质

Also Published As

Publication number Publication date
CN111007987A (zh) 2020-04-14

Similar Documents

Publication Publication Date Title
WO2021088423A1 (zh) 一种用于raid io的内存管理方法、系统、终端及存储介质
US10691187B2 (en) Persistent reservations for virtual disk using multiple targets
US9720757B2 (en) Securing crash dump files
US7882386B1 (en) System and method for recovering a logical volume during failover or reboot of a file server in a data storage environment
US8856472B2 (en) Restore in cascaded copy environment
CN109725831B (zh) 管理存储系统的方法、系统和计算机可读介质
CN107515726B (zh) 用于管理存储设备的方法和系统
US9940152B2 (en) Methods and systems for integrating a volume shadow copy service (VSS) requester and/or a VSS provider with virtual volumes (VVOLS)
US9135262B2 (en) Systems and methods for parallel batch processing of write transactions
US11593000B2 (en) Data processing method and apparatus
US8346735B1 (en) Controlling multi-step storage management operations
WO2021072880A1 (zh) 虚拟机内部快照异步创建方法、装置、系统及存储介质
US11372549B2 (en) Reclaiming free space in a storage system
US8688946B2 (en) Selecting an auxiliary storage medium for writing data of real storage pages
Ilin et al. Performance analysis of software with a variant NoSQL data schemes
WO2023184907A1 (zh) 一种多控存储系统的io处理方法、系统及相关组件
US11983578B2 (en) Provisioning a computing subsystem including disaggregated hardware resources that comply with a power domain requirement for a workload
CN115391106A (zh) 一种备端资源池化的方法、系统及装置
CN109766220A (zh) 应用系统的备份恢复方法、装置及计算机可读存储介质
JP2023044720A (ja) クラッシュしたアプリケーションを修復するためのコンピュータ実装方法、コンピュータプログラム製品、およびリモートコンピュータサーバ（クラッシュしたプロセスのリモート修復）
US9811421B1 (en) Managing multi-step storage management operations by using fault recovery policies
US9641613B2 (en) Volume hierarchy download in a storage area network
US11644997B2 (en) Releasing data storage tracks while maintaining logical corruption protection
US20230152992A1 (en) Force provisioning virtual objects in degraded stretched clusters
CN117873649A (zh) 一种数据迁移处理方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20884920

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20884920

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20884920

Country of ref document: EP

Kind code of ref document: A1