WO2021258749A1 - 一种写请求数据压缩方法、系统、终端及存储介质 - Google Patents

一种写请求数据压缩方法、系统、终端及存储介质 Download PDF

Info

Publication number
WO2021258749A1
WO2021258749A1 PCT/CN2021/076886 CN2021076886W WO2021258749A1 WO 2021258749 A1 WO2021258749 A1 WO 2021258749A1 CN 2021076886 W CN2021076886 W CN 2021076886W WO 2021258749 A1 WO2021258749 A1 WO 2021258749A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
storage
write request
request
linked list
Prior art date
Application number
PCT/CN2021/076886
Other languages
English (en)
French (fr)
Inventor
刘文国
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2021258749A1 publication Critical patent/WO2021258749A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0674—Disk device
    • G06F3/0676—Magnetic disk device

Definitions

  • This application relates to the field of storage technology, and in particular to a method, system, terminal, and storage medium for compressing write request data.
  • the real-time compression storage system compresses the data before it is written to the storage device, which not only reduces the storage space occupation, but also further reduces the access to the storage device (such as a disk), thereby improving the performance of the storage system.
  • the current real-time compression storage system compresses user-written data in the same time window in the memory and writes it to the storage device.
  • These user-written data usually include random write data and sequential write data.
  • real-time compression storage systems often compose compressed user-written data into fixed-length data blocks and write them to the storage device.
  • it is often necessary to add corresponding metadata and padding data so that when there is less sequential write data, the more random write data that constitutes a data block, the more space occupied by the corresponding metadata and padding data.
  • the more random data is written, the more IO times to access the disk, the more compression and decompression times that need to be performed, resulting in performance degradation.
  • this application provides a write request data compression method, system, terminal and storage medium to solve the above-mentioned technical problems.
  • this application provides a write request data compression method, including:
  • the write request information corresponding to the write request data transferred to the second storage is cleared from the sequential linked list.
  • the merging of write requests with consecutive storage addresses according to the sequential linked list includes:
  • the newly received write request is matched with the historical write request stored in the sequential linked list with the storage address.
  • the storage address includes a start address and an end address, and the end address is calculated according to the start address and data length of the write request data ;
  • the newly received write request information is merged into the matching historical write request in the sequential linked list.
  • the compressing and transferring the write request data with the largest data length in the first storage to the second storage includes:
  • the storage address of the target request is updated to the request linked list, and the request linked list saves the information of all write requests.
  • the method further includes:
  • the data accessed by the read request is returned according to the data storage address.
  • this application provides a write request data compression system, including:
  • the initial saving unit is configured to save the write request data to the first storage and save the data storage address and data length of the write request to the sequential linked list;
  • a request merging unit configured to merge write requests with consecutive storage addresses according to the sequential linked list
  • the data compression unit is configured to obtain the free space of the first storage, and if the free space is lower than the preset threshold, compress and transfer the write request data with the largest data length in the first storage to the second storage one by one until the The free space is not lower than the preset threshold;
  • the linked list update unit is configured to clear the write request information corresponding to the write request data transferred to the second storage from the sequential linked list.
  • the request merging unit includes:
  • the address matching module is configured to match the storage address of the newly received write request with the historical write request stored in the sequential linked list.
  • the storage address includes a start address and an end address, and the end address is based on the start of the write request data.
  • the start address and data length are calculated;
  • the information merging module is configured to merge the newly received write request information into the matching historical write request in the sequential linked list if there is a historical write request matching the storage address of the newly received write request.
  • the data compression unit includes:
  • the target screening module is configured to filter out the write request with the largest data length in the sequential linked list as the target request and read the storage address of the target request;
  • the compression transfer module is configured to read the corresponding target write request data from the first storage according to the storage address, compress the target write request data and save it to the second storage, and delete the target write request data from the first storage ;
  • the address update module is configured to update the storage address of the target request to a request linked list, and the request linked list saves information of all write requests.
  • system also includes:
  • the address lookup module is configured to obtain the data storage address of the requested data from the request linked list according to the request information of the read request;
  • the data return module is configured to return the data accessed by the read request according to the data storage address.
  • a terminal including:
  • the memory is used to store computer programs
  • the processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.
  • a computer storage medium stores instructions that, when run on a computer, cause the computer to execute the methods described in the foregoing aspects.
  • the beneficial effect of this application is that the write request data compression method, system, terminal, and storage medium provided in this application are provided with two storage modules to store the data before compression and the data after compression respectively, and connect the data before compression by merging addresses.
  • the data converts part of the random write data into sequential write data, which can reduce the number of IOs to access the disk and reduce the consumption of storage space, thereby improving the performance and space utilization of the real-time compression storage system.
  • Fig. 1 is a schematic principle diagram of a method according to an embodiment of the present application.
  • Fig. 2 is a schematic block diagram of a system according to an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a terminal provided by an embodiment of the application.
  • FIG. 1 is a schematic flowchart of a method according to an embodiment of the present application.
  • the execution subject of FIG. 1 may be a write request data compression system.
  • the write request data compression method includes:
  • the write request sequentialization module When a write request accesses the real-time compression storage system, the write request sequentialization module records the starting address and data length of the request in the request linked list and sequential linked list at the same time, and then writes the write data to the SSD cache and writes it in the request linked list. Mark the storage status as SSD_in in;
  • the write request sequential module For a newly arrived write request, the write request sequential module records its starting address and data length in the request linked list, and then compares it with the write request in the sequential linked list. If there is a write request For historical write requests with consecutive logical addresses, the write request information will be merged into the matching historical write requests. If it does not exist, the start address and data length of the write request will be recorded in the sequential linked list, and then the Write data to the SSD cache, and mark the storage status as SSD_in in the request linked list;
  • SSD cache replacement module performs cache data replacement, and selects the largest data length in the sequential linked list Write request to read the corresponding data from the SSD cache;
  • the write request sequentialization module deletes the write request information selected in step (3) from the sequential linked list, and finds the corresponding one or more write requests in the request linked list, and marks its storage status as SSD_out to indicate The corresponding data has been replaced from the SSD cache;
  • the data compression/decompression module compresses the data read in step (3), and then writes it to the HDD;
  • the write request sequencing module records the storage location of the data compressed in step (5) in the HDD at the corresponding request in the request linked list;
  • the SSD cache replacement module detects that the free space of the SSD is higher than the set threshold (for example, 20%), it stops the cache data replacement.
  • the read process mainly includes the following steps:
  • the read request obtains the storage location of the corresponding data in the HDD in the request linked list, and then reads the corresponding data from the HDD.
  • the data compression/decompression module decompresses the data read in step (2).
  • the read request obtains the requested data from the decompressed data block.
  • the system 200 includes:
  • the initial saving unit 210 is configured to save the write request data to the first storage and save the data storage address and data length of the write request to the sequential linked list;
  • the request merging unit 220 is configured to merge write requests with consecutive storage addresses according to the sequential linked list
  • the data compression unit 230 is configured to obtain the free space of the first storage, and if the free space is lower than the preset threshold, compress and transfer the write request data with the largest data length in the first storage to the second storage one by one until all The free space is not lower than a preset threshold;
  • the linked list update unit 240 is configured to clear the write request information corresponding to the write request data transferred to the second storage from the sequential linked list.
  • the request merging unit includes:
  • the address matching module is configured to match the storage address of the newly received write request with the historical write request stored in the sequential linked list.
  • the storage address includes a start address and an end address, and the end address is based on the start of the write request data.
  • the start address and data length are calculated;
  • the information merging module is configured to merge the newly received write request information into the matching historical write request in the sequential linked list if there is a historical write request matching the storage address of the newly received write request.
  • the data compression unit includes:
  • the target screening module is configured to filter out the write request with the largest data length in the sequential linked list as the target request and read the storage address of the target request;
  • the compression transfer module is configured to read the corresponding target write request data from the first storage according to the storage address, compress the target write request data and save it to the second storage, and delete the target write request data from the first storage ;
  • the address update module is configured to update the storage address of the target request to a request linked list, and the request linked list saves information of all write requests.
  • system further includes:
  • the address lookup module is configured to obtain the data storage address of the requested data from the request linked list according to the request information of the read request;
  • the data return module is configured to return the data accessed by the read request according to the data storage address.
  • FIG. 3 is a schematic structural diagram of a terminal system 300 provided by an embodiment of this application.
  • the terminal system 300 may be used to execute the write request data compression method provided by the embodiment of this application.
  • the terminal system 300 may include: a processor 310, a memory 320, and a communication unit 330. These components communicate through one or more buses.
  • a processor 310 a memory 320
  • a communication unit 330 a communication unit 330.
  • these components communicate through one or more buses.
  • the structure of the server shown in the figure does not constitute a limitation to the application. It can be a bus-shaped structure, a star-shaped structure, or It may include more or fewer components than shown, or combine certain components, or different component arrangements.
  • the memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electronic Erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, secondary storage Or CD.
  • SRAM static random access memory
  • EEPROM electronic Erasable programmable read-only memory
  • EPROM erasable programmable read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory flash memory
  • flash memory secondary storage Or CD.
  • the processor 310 is the control center of the storage terminal. It uses various interfaces and lines to connect the various parts of the entire electronic terminal. By running or executing software programs and/or modules stored in the storage 320, and calling data stored in the storage, To perform various functions of the electronic terminal and/or process data.
  • the processor may be composed of an integrated circuit (Integrated Circuit, IC for short), for example, it may be composed of a single packaged IC, or may be composed of connecting multiple packaged ICs with the same function or different functions.
  • the processor 310 may only include a central processing unit (Central Processing Unit, CPU for short). In the embodiments of the present application, the CPU may be a single computing core, or it may include multiple computing cores.
  • CPU Central Processing Unit
  • the communication unit 330 is configured to establish a communication channel, so that the storage terminal can communicate with other terminals. Receive user data sent by other terminals or send user data to other terminals.
  • the present application also provides a computer storage medium, where the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments provided in the present application when the program is executed.
  • the storage medium may be a magnetic disk, an optical disc, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM), etc.
  • two storage modules are provided to store the pre-compression data and the post-compression data respectively, and some random write data are converted into sequential write data by merging the address-connected pre-compression data, which can reduce the number of IO accesses to the disk.
  • the consumption of storage space is reduced, so that the performance and space utilization of the real-time compression storage system can be improved.
  • the disclosed system and method can be implemented in other ways.
  • the system embodiment described above is merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, systems or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

Abstract

一种写请求数据压缩方法、系统、终端及存储介质，该方法包括：将写请求数据保存至第一存储并将所述写请求的数据存储地址和数据长度保存至顺序化链表；根据所述顺序化链表将存储地址相接的写请求合并；获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。该方法能够减少访问磁盘的IO次数，并减少存储空间的消耗，从而能够提高实时压缩存储系统的性能和空间利用率。

Description

一种写请求数据压缩方法、系统、终端及存储介质
本申请要求于2020年06月24日提交中国专利局、申请号为202010590044.X、发明名称为“一种写请求数据压缩方法、系统、终端及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及存储技术领域，具体涉及一种写请求数据压缩方法、系统、终端及存储介质。
背景技术
当今时代，数据的指数级增长对存储容量的要求越来越高。通常有两种途径来应对存储容量的增长，一个是扩展存储容量，但这会带来成本的增加；另一个是对数据进行压缩，减少存储容量的消耗，节省成本。实时压缩存储系统在数据写入存储设备之前对数据进行压缩，不仅减少了存储空间的占用，更能进一步减少对存储设备(比如磁盘)的访问，从而提高存储系统的性能。
当前的实时压缩存储系统，将内存中同一时间窗口内的用户写数据压缩后写向存储设备，这些用户写数据通常包含随机写数据和顺序写数据两种。为了有效利用存储空间，实时压缩存储系统往往将压缩后的用户写数据组成固定长度的数据块写到存储设备上，而在同一个数据块中为了区别逻辑地址不连续的用户写数据(即随机写数据)，往往需要添加相应的元数据和填充数据，这样当顺序写数据越少时，组成一个数据块的随机写数据越多，相应的元数据和填充数据占用的空间就越多。除此之外，随机写数据越多，访问磁盘的IO次数、需要执行的压缩次数和解压缩次数也越多，造成了性能下降。
发明内容
针对现有技术的上述不足，本申请提供一种写请求数据压缩方法、系 统、终端及存储介质，以解决上述技术问题。
第一方面，本申请提供一种写请求数据压缩方法，包括：
将写请求数据保存至第一存储并将所述写请求的数据存储地址和数据长度保存至顺序化链表；
根据所述顺序化链表将存储地址相接的写请求合并；
获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；
将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。
进一步的，所述根据顺序化链表将存储地址相接的写请求合并，包括：
将新接收的写请求与顺序化链表中保存的历史写请求进行存储地址匹配，所述存储地址包括起始地址和结束地址，所述结束地址根据写请求数据的起始地址和数据长度计算得到；
若存在与新接收的写请求的存储地址匹配的历史写请求，则在所述顺序化链表中将新接收的写请求信息合并至匹配历史写请求。
进一步的，所述将第一存储中数据长度最大的写请求数据压缩转移至第二存储，包括：
在顺序化链表中筛选出数据长度最大的写请求作为目标请求并读取目标请求的存储地址；
根据所述存储地址从第一存储读取相应的目标写请求数据，将目标写请求数据压缩后保存至第二存储，并从第一存储删除所述目标写请求数据；
将所述目标请求的存储地址更新至请求链表，所述请求链表保存所有写请求的信息。
进一步的，所述方法还包括：
根据读请求的请求信息，从请求链表获取请求数据的数据存储地址；
根据所述数据存储地址返回读请求访问的数据。
第二方面，本申请提供一种写请求数据压缩系统，包括：
初始保存单元，配置用于将写请求数据保存至第一存储并将所述写请 求的数据存储地址和数据长度保存至顺序化链表；
请求合并单元，配置用于根据所述顺序化链表将存储地址相接的写请求合并；
数据压缩单元，配置用于获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；
链表更新单元，配置用于将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。
进一步的，所述请求合并单元包括：
地址匹配模块，配置用于将新接收的写请求与顺序化链表中保存的历史写请求进行存储地址匹配，所述存储地址包括起始地址和结束地址，所述结束地址根据写请求数据的起始地址和数据长度计算得到；
信息合并模块，配置用于若存在与新接收的写请求的存储地址匹配的历史写请求，则在所述顺序化链表中将新接收的写请求信息合并至匹配历史写请求。
进一步的，所述数据压缩单元包括：
目标筛选模块，配置用于在顺序化链表中筛选出数据长度最大的写请求作为目标请求并读取目标请求的存储地址；
压缩转移模块，配置用于根据所述存储地址从第一存储读取相应的目标写请求数据，将目标写请求数据压缩后保存至第二存储，并从第一存储删除所述目标写请求数据；
地址更新模块，配置用于将所述目标请求的存储地址更新至请求链表，所述请求链表保存所有写请求的信息。
进一步的，所述系统还包括：
地址查找模块，配置用于根据读请求的请求信息，从请求链表获取请求数据的数据存储地址；
数据返回模块，配置用于根据所述数据存储地址返回读请求访问的数据。
第三方面，提供一种终端，包括：
处理器、存储器，其中，
该存储器用于存储计算机程序，
该处理器用于从存储器中调用并运行该计算机程序，使得终端执行上述的终端的方法。
第四方面，提供了一种计算机存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述各方面所述的方法。
本申请的有益效果在于，本申请提供的写请求数据压缩方法、系统、终端及存储介质，通过设置两个存储模块，分别存储压缩前数据和压缩后数据，并通过合并地址相接的压缩前数据将部分随机写数据转换为顺序写数据，能够减少访问磁盘的IO次数，并减少存储空间的消耗，从而能够提高实时压缩存储系统的性能和空间利用率。
此外，本申请设计原理可靠，结构简单，具有非常广泛的应用前景。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请一个实施例的方法的示意性原理图。
图2是本申请一个实施例的系统的示意性框图。
图3为本申请实施例提供的一种终端的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造 性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
下面对本申请中出现的关键术语进行解释。
SSD solid state drive固态盘
HDD hard disk drive磁盘
图1是本申请一个实施例的方法的示意性流程图。其中，图1执行主体可以为一种写请求数据压缩系统。
具体的，请参考图1，所述写请求数据压缩方法包括：
(1)写请求访问实时压缩存储系统时，写请求顺序化模块同时在请求链表和顺序化链表中记录该请求的起始地址和数据长度，然后将写数据写向SSD缓存，并在请求链表中将存储状态标记为SSD_in；
(2)对于新到来的写请求，写请求顺序化模块先将其起始地址和数据长度记录在请求链表中，然后将其与顺序化链表中的写请求进行比较，若存在一个写请求的逻辑地址连续相接的历史写请求，则将该写请求信息合并匹配的历史写请求中，若不存在，则将该写请求的起始地址和数据长度在顺序化链表中记录下来，然后将写数据写向SSD缓存，并在请求链表中将存储状态标记为SSD_in；
(3)重复步骤(2)，当SSD缓存的空闲空间低于设定的阈值(比如SSD缓存空间的20％)时，SSD缓存替换模块执行缓存数据替换，在顺序化链表中选择数据长度最大的写请求，将对应的数据从SSD缓存读出；
(4)写请求顺序化模块将步骤(3)选择的写请求信息从顺序化链表中删除，并在请求链表中找到对应的一个或多个写请求，将其存储状态标记为SSD_out，以表示其对应的数据已从SSD缓存替换出去；
(5)数据压缩/解压缩模块将步骤(3)读出的数据进行压缩，然后写入HDD；
(6)写请求顺序化模块在请求链表相应请求处记录步骤(5)中压缩的数据在HDD中的存储位置；
(7)当SSD缓存替换模块监测到SSD的空闲空间高于设定的阈值(比如20％)时，停止缓存数据替换。
该固态盘缓存方法实现时，其读流程主要包括以下几个步骤：
(1)读请求访问实时压缩存储系统时，首先在请求链表中找到该请求的存储状态，如果是SSD_in，则去SSD缓存读取数据，读取完毕后读流程结束；如果是SSD_out，则转向步骤(2)；
(2)读请求在请求链表中获取对应数据在HDD中的存储位置，然后将相应数据从HDD读出。
(3)数据压缩/解压缩模块将步骤(2)读取的数据解压缩。
(4)读请求从解压的数据块中获取请求的数据。
如图2所示，该系统200包括：
初始保存单元210，配置用于将写请求数据保存至第一存储并将所述写请求的数据存储地址和数据长度保存至顺序化链表；
请求合并单元220，配置用于根据所述顺序化链表将存储地址相接的写请求合并；
数据压缩单元230，配置用于获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；
链表更新单元240，配置用于将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。
可选地，作为本申请一个实施例，所述请求合并单元包括：
地址匹配模块，配置用于将新接收的写请求与顺序化链表中保存的历史写请求进行存储地址匹配，所述存储地址包括起始地址和结束地址，所述结束地址根据写请求数据的起始地址和数据长度计算得到；
信息合并模块，配置用于若存在与新接收的写请求的存储地址匹配的历史写请求，则在所述顺序化链表中将新接收的写请求信息合并至匹配历史写请求。
可选地，作为本申请一个实施例，所述数据压缩单元包括：
目标筛选模块，配置用于在顺序化链表中筛选出数据长度最大的写请求作为目标请求并读取目标请求的存储地址；
压缩转移模块，配置用于根据所述存储地址从第一存储读取相应的目标写请求数据，将目标写请求数据压缩后保存至第二存储，并从第一存储 删除所述目标写请求数据；
地址更新模块，配置用于将所述目标请求的存储地址更新至请求链表，所述请求链表保存所有写请求的信息。
可选地，作为本申请一个实施例，所述系统还包括：
地址查找模块，配置用于根据读请求的请求信息，从请求链表获取请求数据的数据存储地址；
数据返回模块，配置用于根据所述数据存储地址返回读请求访问的数据。
图3为本申请实施例提供的一种终端系统300的结构示意图，该终端系统300可以用于执行本申请实施例提供的写请求数据压缩方法。
其中，该终端系统300可以包括：处理器310、存储器320及通信单元330。这些组件通过一条或多条总线进行通信，本领域技术人员可以理解，图中示出的服务器的结构并不构成对本申请的限定，它既可以是总线形结构，也可以是星型结构，还可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
其中，该存储器320可以用于存储处理器310的执行指令，存储器320可以由任何类型的易失性或非易失性存储终端或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，第二存储或光盘。当存储器320中的执行指令由处理器310执行时，使得终端300能够执行以下上述方法实施例中的部分或全部步骤。
处理器310为存储终端的控制中心，利用各种接口和线路连接整个电子终端的各个部分，通过运行或执行存储在存储器320内的软件程序和/或模块，以及调用存储在存储器内的数据，以执行电子终端的各种功能和/或处理数据。所述处理器可以由集成电路(Integrated Circuit，简称IC)组成，例如可以由单颗封装的IC所组成，也可以由连接多颗相同功能或不同功能的封装IC而组成。举例来说，处理器310可以仅包括中央处理器(Central Processing Unit，简称CPU)。在本申请实施方式中，CPU可以是单运算核 心，也可以包括多运算核心。
通信单元330，用于建立通信信道，从而使所述存储终端可以与其它终端进行通信。接收其他终端发送的用户数据或者向其他终端发送用户数据。
本申请还提供一种计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可包括本申请提供的各实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM)等。
因此，本申请通过设置两个存储模块，分别存储压缩前数据和压缩后数据，并通过合并地址相接的压缩前数据将部分随机写数据转换为顺序写数据，能够减少访问磁盘的IO次数，并减少存储空间的消耗，从而能够提高实时压缩存储系统的性能和空间利用率，本实施例所能达到的技术效果可以参见上文中的描述，此处不再赘述。
本领域的技术人员可以清楚地了解到本申请实施例中的技术可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本申请实施例中的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中如U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质，包括若干指令用以使得一台计算机终端(可以是个人计算机，服务器，或者第二终端、网络终端等)执行本申请各个实施例所述方法的全部或部分步骤。
本说明书中各个实施例之间相同相似的部分互相参见即可。尤其，对于终端实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例中的说明即可。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统和方法，可以通过其它的方式实现。例如，以上所描述的系统实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以 有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，系统或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
尽管通过参考附图并结合优选实施例的方式对本申请进行了详细描述，但本申请并不限于此。在不脱离本申请的精神和实质的前提下，本领域普通技术人员可以对本申请的实施例进行各种等效的修改或替换，而这些修改或替换都应在本申请的涵盖范围内/任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。

Claims (10)

  1. 一种写请求数据压缩方法，其特征在于，包括：
    将写请求数据保存至第一存储并将所述写请求的数据存储地址和数据长度保存至顺序化链表；
    根据所述顺序化链表将存储地址相接的写请求合并；
    获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；
    将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。
  2. 根据权利要求1所述的方法，其特征在于，所述根据顺序化链表将存储地址相接的写请求合并，包括：
    将新接收的写请求与顺序化链表中保存的历史写请求进行存储地址匹配，所述存储地址包括起始地址和结束地址，所述结束地址根据写请求数据的起始地址和数据长度计算得到；
    若存在与新接收的写请求的存储地址匹配的历史写请求，则在所述顺序化链表中将新接收的写请求信息合并至匹配历史写请求。
  3. 根据权利要求1所述的方法，其特征在于，所述将第一存储中数据长度最大的写请求数据压缩转移至第二存储，包括：
    在顺序化链表中筛选出数据长度最大的写请求作为目标请求并读取目标请求的存储地址；
    根据所述存储地址从第一存储读取相应的目标写请求数据，将目标写请求数据压缩后保存至第二存储，并从第一存储删除所述目标写请求数据；
    将所述目标请求的存储地址更新至请求链表，所述请求链表保存所有写请求的信息。
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    根据读请求的请求信息，从请求链表获取请求数据的数据存储地址；
    根据所述数据存储地址返回读请求访问的数据。
  5. 一种写请求数据压缩系统，其特征在于，包括：
    初始保存单元，配置用于将写请求数据保存至第一存储并将所述写请求的数据存储地址和数据长度保存至顺序化链表；
    请求合并单元，配置用于根据所述顺序化链表将存储地址相接的写请求合并；
    数据压缩单元，配置用于获取第一存储的空闲空间，若所述空闲空间低于预设阈值，则逐条将第一存储中数据长度最大的写请求数据压缩转移至第二存储，直至所述空闲空间不低于预设阈值；
    链表更新单元，配置用于将转移至第二存储的写请求数据对应的写请求信息从顺序化链表清除。
  6. 根据权利要求5所述的系统，其特征在于，所述请求合并单元包括：
    地址匹配模块，配置用于将新接收的写请求与顺序化链表中保存的历史写请求进行存储地址匹配，所述存储地址包括起始地址和结束地址，所述结束地址根据写请求数据的起始地址和数据长度计算得到；
    信息合并模块，配置用于若存在与新接收的写请求的存储地址匹配的历史写请求，则在所述顺序化链表中将新接收的写请求信息合并至匹配历史写请求。
  7. 根据权利要求5所述的系统，其特征在于，所述数据压缩单元包括：
    目标筛选模块，配置用于在顺序化链表中筛选出数据长度最大的写请求作为目标请求并读取目标请求的存储地址；
    压缩转移模块，配置用于根据所述存储地址从第一存储读取相应的目标写请求数据，将目标写请求数据压缩后保存至第二存储，并从第一存储删除所述目标写请求数据；
    地址更新模块，配置用于将所述目标请求的存储地址更新至请求链表，所述请求链表保存所有写请求的信息。
  8. 根据权利要求7所述的系统，其特征在于，所述系统还包括：
    地址查找模块，配置用于根据读请求的请求信息，从请求链表获取请求数据的数据存储地址；
    数据返回模块，配置用于根据所述数据存储地址返回读请求访问的数据。
  9. 一种终端，其特征在于，包括：
    处理器；
    用于存储处理器的执行指令的存储器；
    其中，所述处理器被配置为执行权利要求1-4任一项所述的方法。
  10. 一种存储有计算机程序的计算机可读存储介质，其特征在于，该程序被处理器执行时实现如权利要求1-4中任一项所述的方法。
PCT/CN2021/076886 2020-06-24 2021-02-19 一种写请求数据压缩方法、系统、终端及存储介质 WO2021258749A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010590044.XA CN111857574A (zh) 2020-06-24 2020-06-24 一种写请求数据压缩方法、系统、终端及存储介质
CN202010590044.X 2020-06-24

Publications (1)

Publication Number Publication Date
WO2021258749A1 true WO2021258749A1 (zh) 2021-12-30

Family

ID=72989162

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/076886 WO2021258749A1 (zh) 2020-06-24 2021-02-19 一种写请求数据压缩方法、系统、终端及存储介质

Country Status (2)

Country Link
CN (1) CN111857574A (zh)
WO (1) WO2021258749A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117539636A (zh) * 2023-12-06 2024-02-09 摩尔线程智能科技(北京)有限责任公司 总线模块的内存管理方法、装置、电子设备和存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111857574A (zh) * 2020-06-24 2020-10-30 苏州浪潮智能科技有限公司 一种写请求数据压缩方法、系统、终端及存储介质
CN114827288A (zh) * 2022-03-30 2022-07-29 阿里云计算有限公司 数据转发设备、数据处理方法及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103678169A (zh) * 2013-11-11 2014-03-26 华中科技大学 一种高效利用固态盘缓存的方法和系统
CN105204781A (zh) * 2015-09-28 2015-12-30 华为技术有限公司 压缩方法、装置和设备
CN105243027A (zh) * 2015-09-24 2016-01-13 华为技术有限公司 在存储设备中存储数据的方法和存储控制器
WO2016082524A1 (zh) * 2014-11-27 2016-06-02 华为技术有限公司 一种进行数据存储的方法、装置及系统
CN111857574A (zh) * 2020-06-24 2020-10-30 苏州浪潮智能科技有限公司 一种写请求数据压缩方法、系统、终端及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103678169A (zh) * 2013-11-11 2014-03-26 华中科技大学 一种高效利用固态盘缓存的方法和系统
WO2016082524A1 (zh) * 2014-11-27 2016-06-02 华为技术有限公司 一种进行数据存储的方法、装置及系统
CN105243027A (zh) * 2015-09-24 2016-01-13 华为技术有限公司 在存储设备中存储数据的方法和存储控制器
CN105204781A (zh) * 2015-09-28 2015-12-30 华为技术有限公司 压缩方法、装置和设备
CN111857574A (zh) * 2020-06-24 2020-10-30 苏州浪潮智能科技有限公司 一种写请求数据压缩方法、系统、终端及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117539636A (zh) * 2023-12-06 2024-02-09 摩尔线程智能科技(北京)有限责任公司 总线模块的内存管理方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN111857574A (zh) 2020-10-30

Similar Documents

Publication Publication Date Title
WO2021258749A1 (zh) 一种写请求数据压缩方法、系统、终端及存储介质
CN108268219B (zh) 一种处理io请求的方法及装置
CN102609360B (zh) 一种数据处理方法、装置及系统
US6449689B1 (en) System and method for efficiently storing compressed data on a hard disk drive
US9684665B2 (en) Storage apparatus and data compression method
US10521122B2 (en) Storage apparatus and method of controlling same
CN112684975B (zh) 一种数据存储方法及装置
JP6685334B2 (ja) ストレージ装置
US9183217B2 (en) Method for decompressing data in storage system for write requests that cross compressed data boundaries
CN107423425B (zh) 一种对k/v格式的数据快速存储和查询方法
CN110750211B (zh) 一种存储空间的管理方法及装置
CN111930305A (zh) 数据的存储方法和装置、存储介质、电子装置
US11327929B2 (en) Method and system for reduced data movement compression using in-storage computing and a customized file system
US20240070120A1 (en) Data processing method and apparatus
US20070186041A1 (en) Mass storage device, mass storage controller and methods for use therewith
CN106557277B (zh) 磁盘阵列的读方法及装置
US20140108727A1 (en) Storage apparatus and data processing method
WO2023020136A1 (zh) 存储系统中的数据存储方法及装置
JPH05189157A (ja) ディスク型記憶装置
CN116566396A (zh) 数据压缩方法、装置、存储介质、设备集群及程序产品
CN111597128A (zh) 固态非易失存储设备中数据的管理方法以及管理装置
CN117827079A (zh) 压缩数据读取方法、写入方法及装置
JP6760916B2 (ja) ストレージコントローラ、ストレージアレイ装置、データ格納方法、およびプログラム
US11853582B2 (en) Storage system
WO2022021280A1 (zh) 存储控制器、存储控制方法、固态硬盘及存储系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21828069

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21828069

Country of ref document: EP

Kind code of ref document: A1