WO2023071043A1 - 文件聚合兼容方法、装置、计算机设备和存储介质 - Google Patents

文件聚合兼容方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2023071043A1
WO2023071043A1 PCT/CN2022/083738 CN2022083738W WO2023071043A1 WO 2023071043 A1 WO2023071043 A1 WO 2023071043A1 CN 2022083738 W CN2022083738 W CN 2022083738W WO 2023071043 A1 WO2023071043 A1 WO 2023071043A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
snapshot
client
small
metadata server
Prior art date
Application number
PCT/CN2022/083738
Other languages
English (en)
French (fr)
Inventor
解志阳
肖国栋
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2023071043A1 publication Critical patent/WO2023071043A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/84—Using snapshots, i.e. a logical point-in-time copy of the data

Definitions

  • the present application relates to the field of distributed storage systems, in particular to a snapshot and small file aggregation compatible method, device, computer equipment and storage medium in a distributed file storage system.
  • a snapshot is a mirror image of a data set at a specific moment, also known as an instant copy. It is a complete and available copy of the data set, and the previous state can be viewed or restored based on the snapshot. If a write operation to the original storage system occurs during the use of the snapshot, the original data of the corresponding data unit is stored in the snapshot. The data unit is divided into the data at a certain point in time and the current data in the snapshot. Data is shared between the snapshot and the original storage system. The flexibility of virtual views and the high efficiency of using storage space make it the mainstream of snapshot technology.
  • Snapshots in the file system are implemented based on the object-based COW (copy-on-write) mechanism.
  • object-based COW copy-on-write
  • copy-on-write is triggered to generate a snapshot version and a head version.
  • CephFS is a file storage solution provided by Ceph. It is a file system storage type that supports the POSIX interface.
  • file data is stored in the form of objects. The default size of the object is 4MB. When a small file of 1KB is stored, the data also occupies an object, which is 4MB. If there are a large number of small files in the collective storage, It will cause a lot of waste of resources. Large files are aggregated, and small files (less than or equal to 512KB) are written into a special type of file (aggregate file) in a tightly arranged manner (aligned at 4KB). When reading a file, the object of the source file is no longer read, but the source file data is read from the object of the aggregated file. In this way, for small file scenarios, resource utilization can be greatly improved (as shown in Figure 1).
  • the present application provides a snapshot and small file aggregation compatible method in a distributed file storage system, the method comprising:
  • Step 201 The metadata server receives the operation request sent by the client, and when the operation request is an open request, executes a judging step to judge whether the file has an O_TRUNC identifier, and if so, executes step 202;
  • Step 202 the metadata server judges whether the file is an aggregated small file, and there is a snapshot, if yes, then execute step 203;
  • Step 203 the metadata server returns an error identifier to the client
  • Step 204 After receiving the error identifier, the client converts the aggregated small file into a normal small file;
  • Step 205 After the conversion is completed, the client sends an open request to the metadata server again;
  • Step 206 After receiving the open request again, the metadata server performs a truncate operation to trigger a copy-on-write operation of the small file object to ensure the correctness of the snapshot data.
  • it further includes: judging whether the file is an aggregated small file in step 202 includes: judging whether the size of the original file is smaller than a first threshold.
  • step 204 the conversion method in step 204 includes:
  • it further includes: in step 201, when a file is opened in O_TRUNC mode, after receiving the request, the metadata server performs a truncate operation on the file to clear the data of the file.
  • it further includes: wherein the operation request further includes a delete operation.
  • it also includes:
  • the metadata server After the metadata server receives the deletion request sent by the client, it judges whether the file has the O_TRUNC flag. If it exists, it continues to judge whether it is an aggregated small file and whether there is a snapshot. If it is, it returns the error flag to the client.
  • the client After receiving the error flag, the terminal converts the aggregated small files into normal small files, and sends a deletion request to the metadata server again. After receiving the request again, the metadata server performs a truncate operation to trigger the deletion of the small file object , to ensure the correctness of the snapshot data.
  • the operation request also includes a read operation:
  • step 201 if the judgment result in step 201 is no, or if the judgment result in step 202 is no, then end.
  • the process ends.
  • the present application provides a snapshot and small file aggregation compatible device under a distributed file storage system, including a metadata server and a client, and the device also includes:
  • Judging module after the metadata server receives the read/delete request sent by the client, the judging module judges whether the file has an O_TRUNC mark, if it exists, continues to judge whether it is an aggregated small file, and whether there is a snapshot, if so, The metadata server returns an error identifier to the client;
  • the client After receiving the error identifier, the client converts the aggregated small files into normal small files. After the conversion is completed, the client sends an open/delete request to the metadata server again;
  • the metadata server After the metadata server receives the open/delete request again, it performs the truncate operation to trigger the copy-on-write/delete operation of the small file object to ensure the correctness of the snapshot data.
  • the present application provides a computer device, including a memory, one or more processors, and computer-readable instructions stored on the memory and operable on the processor, the processor executing the computer-readable
  • the instructions are the steps to implement the snapshot and small file aggregation compatible method under the distributed file storage system provided by any one of the above embodiments.
  • the present application provides one or more non-volatile storage media storing computer-readable instructions.
  • the computer-readable instructions When executed by one or more processors, the one or more processors execute The steps of the snapshot and small file aggregation compatible method under the distributed file storage system provided by any one of the above embodiments.
  • Figure 1 is a schematic diagram of small file aggregation
  • Fig. 2 is a schematic diagram of writing and deleting operations of aggregated small files in the prior art
  • Fig. 3 is the application environment diagram of snapshot and small file aggregation compatible method under the distributed file storage system
  • FIG. 4 is a schematic flowchart of a snapshot and small file aggregation compatible method in a distributed file storage system in one or more embodiments
  • Fig. 5 is a structural block diagram of a snapshot and small file aggregation compatible device under a distributed file storage system in one or more embodiments;
  • Figure 6 is an internal block diagram of a computer device in one or more embodiments.
  • the snapshot and small file aggregation compatibility method provided by this application in a distributed file storage system can be applied to the application environment shown in FIG. 3 .
  • the client 102 communicates with the metadata server 104 through the network.
  • the metadata server (MDS) 104 After the metadata server (MDS) 104 receives the request sent by the client 102, it judges whether the file has an O_TRUNC mark, if it exists, then continues to judge whether it is an aggregated small file, and whether there is a snapshot, if so, then returns to the client 102 is a special error code. After receiving the error code, the client converts the aggregated small file into a normal small file, and normally triggers the operation of the object of the small file to ensure the correctness of the snapshot data.
  • the client 102 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices.
  • the metadata server 104 can be realized by an independent server or a server cluster composed of multiple servers.
  • a snapshot and small file aggregation compatible method under a distributed file storage system including the following steps:
  • MDS When the client opens a file in O_TRUNC mode, after receiving the request, MDS will first execute the truncate operation on the file to clear the data in the file.
  • the client calls the open interface, the file has not been opened yet, and the cache in the client cannot be guaranteed to be the latest data. Therefore, whether it is an aggregated small file needs to be judged in MDS.
  • Step 201 The metadata server receives the operation request sent by the client, and when the operation request is an open request, executes a judging step to determine whether the file has an O_TRUNC identifier, and if so, executes step 202; wherein, the metadata server responds to the operation request For opening the request, execute the judging step; step 202: the metadata server judges whether the file is an aggregated small file, and there is a snapshot, if yes, then execute step 203;
  • Step 203 the metadata server returns an error identifier to the client
  • Step 204 After receiving the error identifier, the client converts the aggregated small file into a normal small file;
  • Step 205 After the conversion is completed, the client sends an open request to the metadata server again;
  • Step 206 After receiving the open request again, the metadata server performs a truncate operation to trigger a copy-on-write operation of the small file object to ensure the correctness of the snapshot data.
  • the error identifier may be a special error code.
  • the method will end without performing subsequent steps.
  • the metadata server judges whether the file has an O_TRUNC flag, and if so, continues to judge whether it is an aggregated small file, and Whether there is a snapshot, and if so, return the error ID to the client.
  • the client converts the aggregated small file into a normal small file, and triggers the copy-on-write/delete operation of the small file object, and also That is, when the aggregated small file data changes, first convert the small file into a normal small file, and then manipulate the data of the small file, so that the cow of the small file can be triggered to ensure the correctness of the snapshot data.
  • Judging whether the file is an aggregated small file in step 202 includes: judging whether the size of the original file is smaller than a first threshold.
  • a first threshold In CephFS, file data is stored in the form of objects, and the default size of objects is 4MB.
  • the first threshold can be set in advance.
  • the data in the client cache can be guaranteed to be correct, and can be modified directly in the writing process of the client: if the file is an aggregated small file and there is a snapshot, the aggregated small file, converted to a normal small file.
  • the specific conversion method is as follows:
  • the aggregation attribute includes the inode of the small file and the offset of the small file;
  • the operation request also includes a delete operation.
  • the delete operation is the same as the read operation.
  • the metadata server (MDS) After the metadata server (MDS) receives the delete request sent by the client, it judges whether the file has an O_TRUNC flag. If it exists, it continues to judge whether it is an aggregated small file and whether there is a snapshot. If it is, it returns an error to the client. mark, after receiving the error mark, the client converts the aggregated small files into normal small files, and sends the deletion request to the metadata server (MDS) again, and the metadata server (MDS) performs truncate after receiving the request again During the operation, the deletion operation of the object of the small file is triggered to ensure the correctness of the snapshot data.
  • the method will end without performing subsequent steps.
  • an aggregated file multiple source files share one object. Deleting the source file does not really release the occupied space, resulting in a waste of storage space; at the same time, the reduction in the number of source files leads to a decrease in the cache hit rate when reading the file, affecting read performance.
  • the task of defragmentation is to calculate the ratio of valid data in the aggregated file to the total size of the aggregated file, and clean up the aggregated file when it is lower than the set threshold, so as to improve storage space utilization and improve small file reading performance.
  • Defragmentation will reintegrate valid small files into a new aggregated large file, so that invalid space can be released, and the aggregation attribute in the metadata of the small file needs to be modified. If there is a snapshot cache in the client and defragmentation occurs at the same time, since the snapshots in the system are read-only, the metadata of the snapshot in the client will not be synchronized, so the metadata of the snapshot in the client still points to the old aggregated large file. If you try to get the snapshot data at this time, it will fail because the old aggregated large files have been deleted. Therefore, it is necessary to process the reading process of the snapshot:
  • steps in the flow chart of FIG. 4 are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in FIG. 4 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution of these sub-steps or stages The order is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.
  • a snapshot and small file aggregation compatible device under a distributed file storage system including a metadata server (MDS) and a client (client), and the device also includes:
  • Judging module after the metadata server (MDS) receives the read/delete request sent by the client, the judging module judges whether the file has an O_TRUNC mark, if it exists, continues to judge whether it is an aggregated small file, and whether there is a snapshot, If so, the metadata server (MDS) returns an error identifier to the client (client);
  • the client (client) converts the aggregated small files into normal small files after receiving the error identifier, and after the conversion is completed, the client (client) sends the open/delete request to the metadata server (MDS) again ;
  • the metadata server (MDS) receives the request again, it performs the truncate operation to trigger the copy-on-write/delete operation of the small file object to ensure the correctness of the snapshot data.
  • the judging whether the file is an aggregated small file in the judging module includes: judging whether the size of the original file is smaller than a first threshold.
  • a first threshold In CephFS, file data is stored in the form of objects, and the default size of objects is 4MB.
  • the first threshold can be set in advance.
  • the data in the client cache can be guaranteed to be correct, and can be modified directly in the writing process of the client: if the file is an aggregated small file and there is a snapshot, the aggregated small file, converted to a normal small file.
  • the specific conversion method is as follows:
  • the operation request also includes a delete operation.
  • the delete operation is the same as the read operation.
  • the metadata server (MDS) After the metadata server (MDS) receives the delete request sent by the client, it judges whether the file has an O_TRUNC flag. If it exists, it continues to judge whether it is an aggregated small file and whether there is a snapshot. If it is, it returns an error to the client. mark, after receiving the error mark, the client converts the aggregated small files into normal small files, and sends the deletion request to the metadata server (MDS) again, and the metadata server (MDS) performs truncate after receiving the request again During the operation, the deletion operation of the small file object is triggered to ensure the correctness of the snapshot data.
  • the task of defragmentation is to calculate the ratio of valid data in the aggregated file to the total size of the aggregated file, and clean up the aggregated file when it is lower than the set threshold, so as to improve storage space utilization and improve small file reading performance.
  • Defragmentation will reintegrate valid small files into a new aggregated large file, so that invalid space can be released, and the aggregation attribute in the metadata of the small file needs to be modified. If there is a snapshot cache in the client and defragmentation occurs at the same time, since the snapshots in the system are all read-only, the metadata of the snapshot in the client will not be synchronized, so the metadata of the snapshot in the client still points to the old aggregate large files. If you try to get the snapshot data at this time, it will fail because the old aggregated large files have been deleted. Therefore, it is necessary to process the reading process of the snapshot:
  • each module in the snapshot and small file aggregation compatible device under the above-mentioned distributed file storage system can be implemented in whole or in part by software, hardware and combinations thereof.
  • the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure may be as shown in FIG. 6 .
  • the computer device includes one or more processors, memory, network interfaces, and databases connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium.
  • the computer device's database is used to store aggregated data.
  • the network interface of the computer device is used to communicate with an external terminal via a network connection.
  • FIG. 5-6 are only block diagrams of partial structures related to the solution of this application, and do not constitute a limitation to the computer equipment on which the solution of this application is applied.
  • the specific computer Devices may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device including a memory, one or more processors, and computer-readable instructions stored on the memory and operable on the processor, and the processor implements the above-mentioned The steps of the snapshot and small file aggregation compatible method under the distributed file storage system provided by any one embodiment.
  • the embodiment of the present application also provides one or more non-volatile storage media storing computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, one or more The processor executes the steps of the snapshot and small file aggregation compatible method under the distributed file storage system provided by any one of the above embodiments.
  • Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM random access memory
  • RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请涉及一种分布式文件存储系统下快照和小文件聚合兼容方法、装置、计算机设备和存储介质。所述方法包括：元数据服务器接收到客户端发送的操作请求后，当操作请求是打开请求时，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并触发小文件对象的写时复制/删除操作，保证快照数据的正确。

Description

文件聚合兼容方法、装置、计算机设备和存储介质
相关申请的交叉引用
本申请要求于2021年10月29日提交中国专利局，申请号为202111268961.7，申请名称为“文件聚合兼容方法、装置、计算机设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及分布式存储系统领域，特别是涉及一种分布式文件存储系统下快照和小文件聚合兼容方法、装置、计算机设备和存储介质。
背景技术
快照(Snapshot)是某个数据集在某一特定时刻的镜像，也称为即时拷贝，它是这个数据集的一个完整可用的副本，可以根据快照查看或者恢复之前的状态。若在使用快照的过程中发生对原存储系统的写操作，将对应数据单元的原有数据存入快照，该数据单元分为快照中的某个时间点数据和当前数据，其他未发生更新的数据在快照和原存储系统中共享。虚拟视图的灵活性及使用存储空间的高效性，使得它成为快照技术的主流。
文件系统中的快照是基于对象的COW(copy-on-write)机制实现的，当文件发生变化时，触发写时复制，生成快照版本和head版本。
CephFS是Ceph提供的文件存储解决方案，是一个支持POSIX接口的文件系统存储类型。在CephFS中，文件数据是以对象的形式存储，对象默认大小为4MB，当一个1KB的小文件存放时，数据也要占用一个对象，即为4MB，如果集存是以小文件数量较多时，就会造成大量的资源浪费。大文件聚合，小文件(小于或等于512KB)以紧密排列的方式(以4KB对齐)写入到一类特殊的文件(聚合文件)中。在读文件时也不再读取源文件的对象，而是从聚合文件的对象中读取源文件数据。通过这样的方式，针对小文件场景，可以极大的提高资源利用率(如图1所示)。
然而，小文件聚合后，对小文件的读写删等操作，实际操作的是对应的大文件。如果小文件存在快照，进行写删操作时，osd接收到的为大文件的快照域(为空)，不能正常的触发cow(copy-on-write，写时复制)，造成小文件快照无法正常工作。如图2中，小文件ino1发生修改写时，由于不能触发cow，导致快照和head版本仍然都指向聚合大文件中的数据，这样快照的数据，一直和head版本一致，失去了快照的意义。
发明内容
基于此，有必要针对上述技术问题，提供一种分布式文件存储系统下快照和小文件聚合兼容方法、装置、计算机设备和存储介质。
一方面，本申请提供一种分布式文件存储系统下快照和小文件聚合兼容方法，所述方法包括：
步骤201：元数据服务器接收客户端发送的操作请求，当操作请求为打开请求时，执行判断步骤，判断文件是否存在O_TRUNC标识，如果是，则执行步骤202；
步骤202：元数据服务器判断文件是否为聚合的小文件，并且存在快照，如果是，则执行步骤203；
步骤203：元数据服务器向客户端返回错误标识；
步骤204：客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件；
步骤205：转换完成后，客户端再次发送打开请求到元数据服务器；
步骤206：元数据服务器再次接收到打开请求后，进行truncate操作，触发小文件对象的写时复制操作，保证快照数据的正确。
在其中一个实施例中，还包括：步骤202中的判断文件是否为聚合的小文件包括：判断原始文件的大小是否小于第一阈值。
在其中一个实施例中，还包括：步骤204中的转换方法包括：
a)获取聚合的小文件的聚合属性，根据所述聚合属性找到并打开聚合大文件，从中读取小文件的数据，所述聚合属性包括该小文件的inode以及偏移量；
b)将小文件的数据写入到一个新的对象中；
c)发送setxattr请求，更新元数据服务器中小文件的元数据；
d)清空聚合大文件中对象头中该小文件的inode。
在其中一个实施例中，还包括：步骤201中，当以O_TRUNC的方式打开一个文件时，元数据服务器收到请求后，对文件执行truncate操作，清空文件的数据。
在其中一个实施例中，还包括：其中操作请求还包括删除操作。
在其中一个实施例中，还包括：
元数据服务器接收到客户端发送的删除请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并再次发送删除请求到元数据服务器，元数据服务器再次接收到请求后，进行truncate操作， 触发小文件对象的删除操作，保证快照数据的正确。
在其中一个实施例中，操作请求还包括读操作：
a)如果读的对象为快照，且快照存在聚合属性；
b)获取快照最新版本的元数据，并获取该元数据的聚合属性；如果快照最新版本的聚合属性和元数据的聚合属性不一致，则将元数据的聚合属性修改为快照最新版本的聚合属性；
c)继续后续的读操作。
在其中一个实施例中，如果步骤201中的判断结果为否，或者，步骤202中的判断结果为否，则结束。
在其中一个实施例中，元数据服务器如果判断文件不存在O_TRUNC标识，或者文件不是聚合的小文件，并且存在快照，则结束。
另一方面，本申请提供了一种分布式文件存储系统下快照和小文件聚合兼容装置，包括元数据服务器和客户端，装置还包括：
判断模块，元数据服务器接收到客户端发送的读/删除请求后，所述判断模块判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，元数据服务器向客户端返回错误标识；
执行模块，客户端在接收到错误标识后，将聚合的小文件，转换成正常的小文件，转换完成后，客户端再次发送打开/删除请求到元数据服务器；
操作模块，元数据服务器再次接收到打开/删除请求后，进行truncate操作，触发小文件对象的写时复制/删除操作，保证快照数据的正确。
再一方面，本申请提供了一种计算机设备，包括存储器、一个或多个处理器及存储在存储器上并可在处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现上述任意一个实施例提供的分布式文件存储系统下快照和小文件聚合兼容方法的步骤。
又一方面，本申请提供了一种一个或多个存储有计算机可读指令的非易失性存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述任意一个实施例提供的分布式文件存储系统下快照和小文件聚合兼容方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
图1为小文件聚合示意图；
图2为现有技术中聚合小文件进行写删操作示意图；
图3为分布式文件存储系统下快照和小文件聚合兼容方法的应用环境图；
图4为一个或多个实施例中分布式文件存储系统下快照和小文件聚合兼容方法的流程示意图；
图5为一个或多个实施例中分布式文件存储系统下快照和小文件聚合兼容装置的结构框图；
图6为一个或多个实施例中计算机设备的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请提供的分布式文件存储系统下快照和小文件聚合兼容方法，可以应用于如图3所示的应用环境中。其中，客户端102通过网络与元数据服务器104进行通信。元数据服务器(MDS)104接收到客户端102发送的请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端102特殊的错误码，客户端在接收到错误码后，将聚合的小文件转换成正常的小文件，正常的触发小文件的对象的操作，保证快照数据的正确。其中，客户端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备元数据服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一个实施例中，如图4所示，提供了一种分布式文件存储系统下快照和小文件聚合兼容方法，包括以下步骤：
当客户端以O_TRUNC的方式打开一个文件时，MDS收到请求后，首先会对文件执行truncate操作，清空文件的数据。client调用open接口的时候，文件还未完成打开，不能保证client中的缓存为最新的数据，所以是否为聚合的小文件，需要在MDS进行判断。
步骤201：元数据服务器接收客户端发送的操作请求，当操作请求为打开请求时，执行判断步骤，判断文件是否存在O_TRUNC标识，如果是，则执行步骤202；其中，元数据服务器响应于操作请求为打开请求，执行判断步骤；步骤202：元数据服务器判断文件是否为聚合的小文件，并且存在快照，如果是，则执行步骤203；
步骤203：元数据服务器向客户端返回错误标识；
步骤204：客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件；
步骤205：转换完成后，客户端再次发送打开请求到元数据服务器；
步骤206：元数据服务器再次接收到打开请求后，进行truncate操作，触发小文件对象的写时复制操作，保证快照数据的正确。其中错误标识可以为一种特殊的错误码。
以上，如果文件不存在O_TRUNC标识，或者，文件不是聚合的小文件，并且存在快照，本方法将结束，不执行后续步骤。
上述分布式文件存储系统下快照和小文件聚合兼容方法中，元数据服务器接收到客户端发送的操作请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并触发小文件对象的写时复制/删除操作，也就是当聚合的小文件数据发生变化时，首先将小文件转换成正常的小文件，再操作小文件的数据，这样就可以触发小文件的cow，保证快照数据的正确。
步骤202中的判断文件是否为聚合的小文件包括：判断原始文件的大小是否小于第一阈值。在CephFS中，文件数据是以对象的形式存储，对象默认大小为4MB，该第一阈值可以预先设置。
写操作时，由于文件已经被打开过，可以保证客户端缓存中的数据是正确，可以直接在客户端的写流程进行修改：如果文件是聚合的小文件，并且存在快照，则首先将聚合的小文件，转换成正常的小文件。具体的转换方法如下：
a)获取小文件中的聚合属性，找到并打开聚合大文件，从中读取小文件的数据；所述聚合属性包括该小文件的inode以及该小文件的偏移量；
b)将小文件的数据写入到一个新的对象中；
c)发送setxattr请求，更新MDS中小文件的元数据；
d)清空聚合大文件中对象头中该小文件的inode。
其中操作请求还包括删除操作。其中删除操作与读操作相同。元数据服务器(MDS)接收到客户端发送的删除请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并再次发送删除请求到元数据服务器(MDS)，元数据服务器(MDS)再次接收到请求后，进行truncate操作的时候，触发小文件的对象的删除操作，保证快照数据的正确。
同样，以上，如果文件不存在O_TRUNC标识，或者，文件不是聚合的小文件，并且存在快照，本方法将结束，不执行后续步骤。在聚合文件中，多个源文件共用一个对象， 删除源文件并不能真正释放所占用的空间，导致存储空间的浪费；同时源文件数量的减少，导致在读取文件时缓存命中率降低，影响读性能。
碎片整理的任务就是计算聚合文件中有效数据在聚合文件总大小的占比，低于设定的阈值时对聚合文件进行清理，以提高存储空间利用率、提升小文件读性能。
碎片整理，会将有效的小文件重新整合到一个新的聚合大文件中，这样就可以释放掉无效的空间，需要修改小文件元数据中的聚合属性。如果client中存在快照的缓存，同时发生了碎片整理，由于系统中快照都是只读的，所以不会同步client中的快照元数据，这样client中快照的元数据仍然指向旧的聚合大文件。如果此时去获取快照的数据，会获取失败，因为旧的聚合大文件已经被删除。所以需要处理快照的读流程：
a)如果读的对象为快照，且快照存在聚合属性；
b)获取快照最新版本的元数据，并获取该元数据的聚合属性；如果快照最新版本的聚合属性和元数据的聚合属性不一致，则将元数据的聚合属性修改为快照最新版本的聚合属性；
具体的，获取head版本的元数据，如果head版本不存在聚合属性，说明小文件数据发生过变更，则也清楚快照的聚合属性；如果head版本的聚合属性和元数据的不一致，说明发生过碎片整理，则将元数据的聚合属性修改为head版本的；
c)继续后续的读操作。
应该理解的是，虽然图4的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图4中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图5所示，提供了一种分布式文件存储系统下快照和小文件聚合兼容装置，包括元数据服务器(MDS)和客户端(client)，装置还包括：
判断模块，元数据服务器(MDS)接收到客户端发送的读/删除请求后，所述判断模块判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，元数据服务器(MDS)向客户端(client)返回错误标识；
执行模块，客户端(client)在接收到错误标识后，将聚合的小文件，转换成正常的小文件，转换完成后，客户端(client)再次发送打开/删除请求到元数据服务器(MDS)；
操作模块，元数据服务器(MDS)再次接收到请求后，进行truncate操作，触发小文件对象的写时复制/删除操作，保证快照数据的正确。
判断模块中的判断文件是否为聚合的小文件包括：判断原始文件的大小是否小于第一阈值。在CephFS中，文件数据是以对象的形式存储，对象默认大小为4MB，该第一阈值可以预先设置。
写操作时，由于文件已经被打开过，可以保证客户端缓存中的数据是正确，可以直接在客户端的写流程进行修改：如果文件是聚合的小文件，并且存在快照，则首先将聚合的小文件，转换成正常的小文件。具体的转换方法如下：
a)获取聚合的小文件的聚合属性，根据所述聚合属性找到并打开聚合大文件，从中读取小文件的数据，所述聚合属性包括该小文件的inode以及该小文件的偏移量；
b)将小文件的数据写入到一个新的对象中；
c)发送setxattr请求，更新元数据服务器中小文件的元数据；
d)清空聚合大文件中对象头中该小文件的inode。
其中操作请求还包括删除操作。其中删除操作与读操作相同。元数据服务器(MDS)接收到客户端发送的删除请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并再次发送删除请求到元数据服务器(MDS)，元数据服务器(MDS)再次接收到请求后，进行truncate操作的时候，触发小文件对象的删除操作，保证快照数据的正确。
在聚合文件中，多个源文件共用一个对象，删除源文件并不能真正释放所占用的空间，导致存储空间的浪费；同时源文件数量的减少，导致在读取文件时缓存命中率降低，影响读性能。
碎片整理的任务就是计算聚合文件中有效数据在聚合文件总大小的占比，低于设定的阈值时对聚合文件进行清理，以提高存储空间利用率、提升小文件读性能。
碎片整理，会将有效的小文件重新整合到一个新的聚合大文件中，这样就可以释放掉无效的空间，需要修改小文件元数据中的聚合属性。如果客户端中存在快照的缓存，同时发生了碎片整理，由于系统中快照都是只读的，所以不会同步客户端中的快照元数据，这样客户端中快照的元数据仍然指向旧的聚合大文件。如果此时去获取快照的数据，会获取失败，因为旧的聚合大文件已经被删除。所以需要处理快照的读流程：
a)如果读的对象为快照，且快照存在聚合属性；
b)获取快照最新版本的元数据，并获取该元数据的聚合属性；如果快照最新版本的聚 合属性和元数据的聚合属性不一致，则将元数据的聚合属性修改为快照最新版本的聚合属性；
c)继续后续的读操作。
关于分布式文件存储系统下快照和小文件聚合兼容装置的具体限定可以参见上文中对于分布式文件存储系统下快照和小文件聚合兼容方法的限定，在此不再赘述。上述分布式文件存储系统下快照和小文件聚合兼容装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图6所示。该计算机设备包括通过系统总线连接的一个或多个处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储聚合数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种分布式文件存储系统下快照和小文件聚合兼容的方法。
本领域技术人员可以理解，图5-6中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器、一个或多个处理器及存储在存储器上并可在处理器上运行的计算机可读指令，处理器执行计算机可读指令时实现上述任意一个实施例提供的分布式文件存储系统下快照和小文件聚合兼容方法的步骤。
在一个实施例中，本申请实施例还提供了一个或多个存储有计算机可读指令的非易失性存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述任意一个实施例提供的分布式文件存储系统下快照和小文件聚合兼容方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它 介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (11)

  1. 一种分布式文件存储系统下快照和小文件聚合兼容方法，其特征在于，包括：
    步骤201：元数据服务器接收客户端发送的操作请求，当操作请求为打开请求时，执行判断步骤，判断文件是否存在O_TRUNC标识，如果是，则执行步骤202；
    步骤202：元数据服务器判断文件是否为聚合的小文件，并且存在快照，如果是，则执行步骤203；
    步骤203：元数据服务器向客户端返回错误标识；
    步骤204：客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件；
    步骤205：转换完成后，客户端再次发送打开请求到元数据服务器；及
    步骤206：元数据服务器再次接收到打开请求后，进行truncate操作，触发小文件对象的写时复制操作，保证快照数据的正确。
  2. 根据权利要求1所述的方法，其中，步骤202中的判断文件是否为聚合的小文件包括：判断原始文件的大小是否小于第一阈值。
  3. 根据权利要求1所述的方法，其中，步骤204中的转换方法包括：
    a)获取聚合的小文件的聚合属性，根据所述聚合属性找到并打开聚合大文件，从中读取小文件的数据，所述聚合属性包括该小文件的inode以及偏移量；
    b)将小文件的数据写入到一个新的对象中；
    c)发送setxattr请求，更新元数据服务器中小文件的元数据；及
    d)清空聚合大文件中对象头中该小文件的inode。
  4. 根据权利要求1所述的方法，其中，步骤201中，当以O_TRUNC的方式打开一个文件时，元数据服务器收到请求后，对文件执行truncate操作，清空文件的数据。
  5. 根据权利要求1所述的方法，其中操作请求还包括删除操作。
  6. 根据权利要求5所述的方法，元数据服务器接收到客户端发送的删除请求后，判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，则返回给客户端错误标识，客户端在接收到错误标识后，将聚合的小文件转换成正常的小文件，并再次发送删除请求到元数据服务器，元数据服务器再次接收到请求后，进行truncate操作，触发小文件对象的删除操作，保证快照数据的正确。
  7. 根据权利要求1-6中任一所述的方法，操作请求还包括读操作：
    a)如果读的对象为快照，且快照存在聚合属性；
    b)获取快照最新版本的元数据，并获取该元数据的聚合属性；如果快照最新版本的聚 合属性和元数据的聚合属性不一致，则将元数据的聚合属性修改为快照最新版本的聚合属性；及
    c)继续后续的读操作。
  8. 根据权利要求1所述的方法，如果步骤201中的判断结果为否，或者，步骤202中的判断结果为否，则结束。
  9. 根据权利要求6所述的方法，元数据服务器如果判断文件不存在O_TRUNC标识，或者文件不是聚合的小文件，并且存在快照，则结束。10、一种分布式文件存储系统下快照和小文件聚合兼容装置，包括元数据服务器和客户端，其特征在于，所述装置还包括：
    判断模块，元数据服务器接收到客户端发送的读/删除请求后，所述判断模块判断文件是否存在O_TRUNC标识，如果存在，则继续判断是否是聚合的小文件，并且是否存在快照，如果是，元数据服务器向客户端返回错误标识；
    执行模块，客户端在接收到错误标识后，将聚合的小文件，转换成正常的小文件，转换完成后，客户端再次发送打开/删除请求到元数据服务器；
    操作模块，元数据服务器再次接收到打开/删除请求后，进行truncate操作，触发小文件对象的写时复制/删除操作，保证快照数据的正确。
  10. 一种计算机设备，包括存储器、一个或多个处理器及存储在存储器上并可在处理器上运行的计算机可读指令，其特征在于，所述处理器执行所述计算机可读指令时实现权利要求1至9中任一项所述方法的步骤。
  11. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如权利要求1-9中任一项所述方法的步骤。
PCT/CN2022/083738 2021-10-29 2022-03-29 文件聚合兼容方法、装置、计算机设备和存储介质 WO2023071043A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111268961.7A CN113704027B (zh) 2021-10-29 2021-10-29 文件聚合兼容方法、装置、计算机设备和存储介质
CN202111268961.7 2021-10-29

Publications (1)

Publication Number Publication Date
WO2023071043A1 true WO2023071043A1 (zh) 2023-05-04

Family

ID=78647639

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/083738 WO2023071043A1 (zh) 2021-10-29 2022-03-29 文件聚合兼容方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN113704027B (zh)
WO (1) WO2023071043A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113704027B (zh) * 2021-10-29 2022-02-18 苏州浪潮智能科技有限公司 文件聚合兼容方法、装置、计算机设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060123209A1 (en) * 2004-12-06 2006-06-08 Devin Borland Devices and methods of performing direct input/output operations using information indicative of copy-on-write status
CN110032541A (zh) * 2019-04-12 2019-07-19 苏州浪潮智能科技有限公司 一种可写快照实现方法及系统
CN111966637A (zh) * 2020-08-21 2020-11-20 苏州浪潮智能科技有限公司 一种打开文件的方法、装置、设备及可读介质
CN113110801A (zh) * 2021-04-15 2021-07-13 山东英信计算机技术有限公司 一种加快小文件读取速度的方法、系统、设备和存储介质
CN113704027A (zh) * 2021-10-29 2021-11-26 苏州浪潮智能科技有限公司 文件聚合兼容方法、装置、计算机设备和存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090204650A1 (en) * 2007-11-15 2009-08-13 Attune Systems, Inc. File Deduplication using Copy-on-Write Storage Tiers
CN105069090A (zh) * 2015-08-05 2015-11-18 浪潮电子信息产业股份有限公司 一种实现文件一写多读的方法及装置
CN107872489B (zh) * 2016-09-28 2020-06-05 杭州海康威视数字技术股份有限公司 一种文件切片上传方法、装置及云存储系统
CN107506466B (zh) * 2017-08-30 2020-08-04 郑州云海信息技术有限公司 一种小文件存储方法及系统
CN107704203B (zh) * 2017-09-27 2021-08-31 郑州云海信息技术有限公司 聚合大文件的删除方法、装置、设备及计算机存储介质
CN109101639B (zh) * 2018-08-21 2021-03-23 赛凡信息科技（厦门）有限公司 一种提高文件系统性能的聚合模式
CN111309687A (zh) * 2020-01-21 2020-06-19 苏州浪潮智能科技有限公司 一种对象存储小文件处理方法、装置、设备及存储介质
CN111625515A (zh) * 2020-05-28 2020-09-04 苏州浪潮智能科技有限公司 一种聚合小文件的操作请求的处理方法及装置
CN111984196B (zh) * 2020-08-21 2022-08-19 苏州浪潮智能科技有限公司 一种文件迁移方法、装置、设备及可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060123209A1 (en) * 2004-12-06 2006-06-08 Devin Borland Devices and methods of performing direct input/output operations using information indicative of copy-on-write status
CN110032541A (zh) * 2019-04-12 2019-07-19 苏州浪潮智能科技有限公司 一种可写快照实现方法及系统
CN111966637A (zh) * 2020-08-21 2020-11-20 苏州浪潮智能科技有限公司 一种打开文件的方法、装置、设备及可读介质
CN113110801A (zh) * 2021-04-15 2021-07-13 山东英信计算机技术有限公司 一种加快小文件读取速度的方法、系统、设备和存储介质
CN113704027A (zh) * 2021-10-29 2021-11-26 苏州浪潮智能科技有限公司 文件聚合兼容方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN113704027B (zh) 2022-02-18
CN113704027A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
US10831614B2 (en) Visualizing restoration operation granularity for a database
US9727452B2 (en) Distributing metadata across multiple different disruption regions within an asymmetric memory system
US20190213085A1 (en) Implementing Fault Domain And Latency Requirements In A Virtualized Distributed Storage System
US8280858B2 (en) Storage pool scrubbing with concurrent snapshots
EP2879040B1 (en) Data storage method, data storage apparatus, and storage device
US10678446B2 (en) Bitmap processing for log-structured data store
US9778860B2 (en) Re-TRIM of free space within VHDX
WO2020093501A1 (zh) 文件存储方法、删除方法、服务器及存储介质
US10620871B1 (en) Storage scheme for a distributed storage system
US20190188090A1 (en) Snapshot Deletion In A Distributed Storage System
TW200910187A (en) Logged FAT file system and method of accessing the same
WO2023071043A1 (zh) 文件聚合兼容方法、装置、计算机设备和存储介质
US20200034049A1 (en) Spot coalescing of distributed data concurrent with storage i/o operations
WO2020119709A1 (zh) 数据合并的实现方法、装置、系统及存储介质
CN115470156A (zh) 基于rdma的内存使用方法、系统、电子设备和存储介质
US9619336B2 (en) Managing production data
US10430110B2 (en) Implementing a hybrid storage node in a distributed storage system
US11256434B2 (en) Data de-duplication
US11010408B2 (en) Hydration of a hierarchy of dehydrated files
US11023328B2 (en) Redo log for append only storage scheme
US11249851B2 (en) Creating snapshots of a storage volume in a distributed storage system
US10452308B2 (en) Encoding tags for metadata entries in a storage system
US20240241856A1 (en) Method and apparatus for compatibility of file aggregation, computer device and storage medium
US11748203B2 (en) Multi-role application orchestration in a distributed storage system
WO2023070462A1 (zh) 一种文件去重方法、装置和设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22884960

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18574636

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE