WO2021128893A1 - 一种垃圾数据的识别方法、系统、电子设备及存储介质 - Google Patents

一种垃圾数据的识别方法、系统、电子设备及存储介质 Download PDF

Info

Publication number
WO2021128893A1
WO2021128893A1 PCT/CN2020/110732 CN2020110732W WO2021128893A1 WO 2021128893 A1 WO2021128893 A1 WO 2021128893A1 CN 2020110732 W CN2020110732 W CN 2020110732W WO 2021128893 A1 WO2021128893 A1 WO 2021128893A1
Authority
WO
WIPO (PCT)
Prior art keywords
tail
data
uploaded
storage pool
head
Prior art date
Application number
PCT/CN2020/110732
Other languages
English (en)
French (fr)
Inventor
赵煜
胡永刚
Original Assignee
浪潮电子信息产业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浪潮电子信息产业股份有限公司 filed Critical 浪潮电子信息产业股份有限公司
Priority to US17/789,336 priority Critical patent/US11687489B2/en
Publication of WO2021128893A1 publication Critical patent/WO2021128893A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/122—File system administration, e.g. details of archiving or snapshots using management policies
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/162—Delete operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2219—Large Object storage; Management thereof

Definitions

  • This application relates to the field of computer technology, and in particular to a method and system for identifying junk data, an electronic device, and a storage medium.
  • the object When uploading an object to a distributed object storage system, the object needs to be split into a head object and multiple tail objects less than or equal to 4M, and the tail object and the head object are uploaded to the data storage pool in sequence.
  • the tail object and the head object are uploaded to the data storage pool in sequence.
  • the tail object written into the data storage pool becomes inaccessible garbage data.
  • the purpose of this application is to provide a method and system for identifying junk data, an electronic device and a storage medium, which can identify junk data in a distributed object storage system.
  • the method for identifying junk data includes:
  • the target data group corresponding to the head object identifier in the data storage pool; wherein the target data group includes the data content of the head object and the tail object identifier corresponding to the head object;
  • uploading the object to the distributed object storage system includes:
  • the tail object identifier of the tail object to be uploaded is stored in the data storage pool in a manifest structure.
  • splitting the object to be uploaded into the head object to be uploaded and the tail object to be uploaded includes:
  • the data size of the header object to be uploaded is less than a first preset value
  • the data size of the tail object to be uploaded is less than a second preset value
  • the first preset value is less than a second preset value
  • writing the header object identifier of the header object to be uploaded into the index storage pool includes:
  • obtaining the head object identifier in the index storage pool of the distributed object storage system includes:
  • the method further includes:
  • it also includes:
  • the data content of the head object and the data content of the tail object corresponding to the target head object identifier in the data storage pool are returned as the access result to the access terminal.
  • This application also provides a junk data identification system, which includes:
  • Object upload module used to upload objects to the distributed object storage system
  • the avatar identifier acquisition module is used to acquire the head object identifier in the index storage pool of the distributed object storage system
  • a data group query module for querying a target data group corresponding to the head object identifier in the data storage pool; wherein the target data group includes the data content of the head object and the tail object identifier corresponding to the head object;
  • a marking module for marking the tail object corresponding to the tail object identifier in the target data group as the target tail object
  • the junk data identification module is used to mark tail objects in the data storage pool other than the target tail object as junk data.
  • the application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps performed by the method for identifying junk data are implemented.
  • the present application also provides an electronic device including a memory and a processor, the memory is stored with a computer program, and when the processor invokes the computer program in the memory, the steps performed by the method for identifying junk data are implemented.
  • This application provides a method for identifying junk data, which includes uploading an object to a distributed object storage system; obtaining a header object identifier in an index storage pool of the distributed object storage system; and querying data storage pool and the header
  • the target data group corresponding to the object identifier wherein the target data group includes the data content of the head object and the tail object identifier corresponding to the head object; and the tail object identifier corresponding to the tail object identifier in the target data group
  • the object is marked as a target tail object; the tail objects in the data storage pool other than the target tail object are marked as junk data.
  • This application first obtains the head object identifier from the index storage pool of the distributed object storage system, queries the tail object identifier corresponding to the head object based on the head object identifier, and marks the tail object corresponding to the tail object identifier in the data storage pool as the target Tail objects, and then mark the tail objects that are not marked as target tail objects as garbage data. Since the tail object is stored in the data storage pool, but the head object is not stored in the data storage pool, it will cause garbage data. Therefore, this application determines the tail object corresponding to the head object that has been stored in the data storage pool based on the head object identifier. The tail object corresponding to the head object is marked as junk data, and it can be seen that this application can identify junk data in the distributed object storage system. This application also provides a junk data identification system, an electronic device and a storage medium, which have the above-mentioned beneficial effects, and will not be repeated here.
  • FIG. 1 is a flowchart of a method for identifying junk data provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of the relationship between a head object and a tail object provided by an embodiment of the application;
  • FIG. 3 is a schematic structural diagram of a junk data identification system provided by an embodiment of the application.
  • Figure 1 is a flowchart of a method for identifying junk data provided by an embodiment of the application.
  • this embodiment is applied to a distributed object storage system.
  • this embodiment can upload objects to the distributed storage system by means of PUT instructions.
  • the objects can be split into one first.
  • each head object and tail object will generate a unique identifier (ie oid, object identifier, object identifier) according to the naming rules.
  • the tail objects are written into the data storage pool (that is, the data storage pool) in the order of splitting, and then the manifest structure that records the identifier information of the tail object and the data content of the head object are written into the data storage pool, and finally the head object is written into the data storage pool.
  • the identifier information of the object is written into the index storage pool (that is, the index storage pool). It is understandable that an object can be divided into a head object and a tail object according to the data position.
  • the head object is the head of the object, and the tail object is data other than the head of the object.
  • An object can be divided into a head object and at least one Tail object.
  • a storage pool is a group of disks that store data in a storage system.
  • the operation of storing objects in the distributed object storage system in this embodiment requires uploading target data A to the distributed object storage system.
  • the target data A is first split into head object A1, tail object A2, and Tail object A3, the object identifier of the head object A1 is 001, the object identifier of the tail object A2 is 002, and the object identifier of the tail object A3 is 003.
  • the head object A1, the object identifier 002 and the object identifier 003 are stored in the data storage pool, and finally the object identifier 001 is stored in the index storage pool.
  • this embodiment may also obtain the head object identifier in the index storage pool of the distributed object storage system according to a preset period, so as to realize the The junk data of the distributed object storage system is checked regularly.
  • the index storage pool may store the head object identifier of the head object, and the specific index storage pool may store the object identifier in the form of a bucket.
  • this step is based on the header object identifier that has been obtained, and the target data group corresponding to the header object identifier in the data storage pool can be queried.
  • the aforementioned target data group may include the data content of the header object and the header object.
  • the corresponding tail object identifier That is, the data content of the head object in the data storage pool is stored in the data storage pool together with the tail object identifier of the tail object.
  • the corresponding head object identifier can be queried from the data storage pool.
  • the data content of the head object is obtained, and the tail object identifier corresponding to the data content of the head object is obtained.
  • the header object identifier in this embodiment can be obtained by mapping (such as hash mapping) according to the data content of the header object, so the corresponding header object data content can be determined according to the header object identifier; the same goes for It can be seen that the corresponding tail object data content can be determined according to the tail object identifier.
  • S104 Mark the tail object corresponding to the tail object identifier in the target data group as the target tail object;
  • this embodiment can determine that the object identifier in the target data group is, and then mark the tail object in the storage data pool corresponding to the object identifier in the target data group as the target tail object. That is, in the process of storing the target tail object in the data storage pool, the head object data corresponding to the target tail object is also written into the data storage pool, and the target tail object is not garbage data.
  • S105 Mark tail objects other than the target tail object in the data storage pool as junk data.
  • this step is established on the basis that the target tail object has been determined, and the tail objects in the data storage pool other than the target tail object can be marked as junk data, that is, there are no tail objects other than the target tail object in the storage pool. Head objects corresponding to tail objects, these tail objects cannot be accessed.
  • the head object identifier is first obtained from the index storage pool of the distributed object storage system, the tail object identifier corresponding to the head object is queried based on the head object identifier, and the tail object corresponding to the tail object identifier in the data storage pool is marked as Target tail objects, and then mark the tail objects that are not marked as target tail objects as garbage data. Since the tail object is stored in the data storage pool, but the head object is not stored in the data storage pool, it will cause garbage data. Therefore, this embodiment determines the tail object corresponding to the head object stored in the data storage pool based on the head object identifier, and then No tail object corresponding to the head object is marked as junk data. It can be seen that this embodiment can identify junk data in the distributed object storage system.
  • the junk data in the data storage pool can also be deleted .
  • the target header object identifier corresponding to the object access instruction in the index storage pool is queried;
  • the data content of the head object and the data content of the tail object corresponding to the target head object identifier are returned to the access terminal as the access result.
  • the process of uploading objects in S101 may specifically include the following operations:
  • Step 1 Split the object to be uploaded into the head object to be uploaded and the tail object to be uploaded;
  • the object to be uploaded can be divided into one header object to be uploaded and multiple tail objects to be uploaded according to a preset strategy; wherein the data size of the header object to be uploaded is smaller than a first preset value, The data size of the tail object to be uploaded is smaller than the second preset value, and the first preset value is smaller than the second preset value.
  • Step 2 Assign a header object identifier to the header object to be uploaded, and assign a tail object identifier to the last object to be uploaded;
  • Step 3 Write the data content of the tail object to be uploaded into the data storage pool in the order of splitting
  • Step 4 Use the data content of the head object to be uploaded and the tail object identifier of the tail object to be uploaded corresponding to the head object to be uploaded as the target data group, and write them into the data storage pool;
  • the tail object identifier of the tail object to be uploaded in this step is stored in the data storage pool in a manifest structure.
  • the manifest is a kind of data defined in the storage system, and its role is to manage the correspondence between the application object and the rados object.
  • Step 5 Write the header object identifier of the header object to be uploaded into the index storage pool.
  • the header object identifier of the header object to be uploaded may be written into the bucket of the index storage pool, so that the bucket of the index storage pool of the distributed object storage system Obtain the head object identifier in.
  • the storage system When uploading an object to a distributed object in a distributed object storage system, the storage system will split the object data into a 512k head object and multiple tail objects less than or equal to 4M according to a certain strategy.
  • the process of storing objects in the storage system in the storage system is to first store all tail objects in the data storage pool in sequence and name them their respective object identifiers, and then store the manifest structure of all tail objects in the data storage together with the head object.
  • the pool is named after the object identifier of the head object, and finally the object identifier information of the head object is written into the storage space of the object identifier of the bucket in the index storage pool.
  • this embodiment proposes a method for identifying junk data. After traversing the object identifiers stored in the buckets in the index storage pool, then according to the object identifiers stored in the buckets Traverse the head object identifiers in the data storage pool and mark them. Finally, according to the head object identifiers, find all tail objects corresponding to the head object and mark them. In the end, all unmarked tail objects in the data storage pool are garbage data. Please refer to Figure 2.
  • Figure 2 is a schematic diagram of the relationship between a head object and a tail object provided by an embodiment of the application.
  • h1, h2, and h3 are head object identifiers
  • t1, t2, t3, t4, t5, t6 is the tail object identifier.
  • the bucket stores the object identifiers h1, h2, and h3, and then queries the tail object identifiers t1, t2, t3, t4, and t5 corresponding to h1, h2, and h3 from the data storage pool, so you can
  • the data content of the tail object that does not store the tail object identifier is marked as junk data.
  • the foregoing embodiment proposes a method for identifying junk data in a storage system, which provides a solution for removing junk in the storage system, enriches the functions of the distributed object storage system, improves the utilization of the storage system's disk space, and avoids the use of disk space. Waste, provide accurate positioning for junk data removal.
  • FIG. 3 is a schematic structural diagram of a junk data identification system provided by an embodiment of the application.
  • the system can include:
  • the object upload module 100 is used to upload objects to the distributed object storage system
  • the avatar identifier obtaining module 200 is configured to obtain the head object identifier in the index storage pool of the distributed object storage system
  • the data group query module 300 is used to query the target data group corresponding to the head object identifier in the data storage pool; wherein the target data group includes the data content of the head object and the tail object identifier corresponding to the head object ;
  • the marking module 400 is used to mark the tail object corresponding to the tail object identifier in the target data group as the target tail object;
  • the junk data identification module 500 is configured to mark tail objects in the data storage pool other than the target tail object as junk data.
  • the head object identifier is first obtained from the index storage pool of the distributed object storage system, the tail object identifier corresponding to the head object is queried based on the head object identifier, and the tail object corresponding to the tail object identifier in the data storage pool is marked as Target tail objects, and then mark the tail objects that are not marked as target tail objects as garbage data. Since the tail object is stored in the data storage pool but the head object is not stored in the data storage pool, it will cause garbage data. Therefore, the present embodiment determines the tail object corresponding to the head object stored in the data storage pool based on the head object identifier, and thus will not The tail object corresponding to the head object is marked as junk data. It can be seen that this embodiment can identify junk data in the distributed object storage system.
  • the object upload module 100 includes:
  • the object splitting unit is used to split the object to be uploaded into the head object to be uploaded and the tail object to be uploaded;
  • An identifier allocation unit configured to allocate a header object identifier for the header object to be uploaded, and a tail object identifier for the tail object to be uploaded;
  • the data writing unit is used to write the data content of the end object to be uploaded into the data storage pool in the order of splitting; and is also used to correspond the data content of the head object to be uploaded to the head object to be uploaded.
  • the tail object identifier of the tail object to be uploaded is used as the target data group and written into the data storage pool; it is also used to write the head object identifier of the head object to be uploaded into the index storage pool.
  • tail object identifier of the tail object to be uploaded is stored in the data storage pool in a manifest structure.
  • the object splitting unit is specifically a unit for splitting the object to be uploaded into one head object to be uploaded and multiple tail objects to be uploaded according to a preset strategy; wherein, the data of the head object to be uploaded The size is less than the first preset value, the data size of the tail object to be uploaded is less than the second preset value, and the first preset value is less than the second preset value.
  • the data writing unit is configured to write the header object identifier of the header object to be uploaded into the bucket of the index storage pool;
  • the avatar identifier acquisition module 200 includes:
  • the data removal module is configured to delete the junk data in the data storage pool after marking the tail objects in the data storage pool other than the target tail object as junk data.
  • the object access module is configured to, when an object access instruction is received, query the target header object identifier corresponding to the object access instruction in the index storage pool; it is also used to associate the data storage pool with the target The data content of the head object and the data content of the tail object corresponding to the head object identifier are returned to the access terminal as the access result.
  • the present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above-mentioned embodiments can be implemented.
  • the storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various media that can store program code.
  • the present application also provides an electronic device, which may include a memory and a processor, the memory stores a computer program, and when the processor invokes the computer program in the memory, the steps provided in the foregoing embodiments can be implemented.
  • the electronic device may also include various network interfaces, power supplies and other components.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种垃圾数据的识别方法、系统、电子设备及存储介质，该方法包括：向分布式对象存储系统上传对象（S101）；获取分布式对象存储系统的索引存储池中的头对象标识符（S102）；查询数据存储池中与头对象标识符对应的目标数据组（S103）；将与目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象（S104）；将数据存储池中除目标尾对象之外的尾对象标记为垃圾数据（S105）。所述方法能够识别分布式对象存储系统中的垃圾数据。

Description

一种垃圾数据的识别方法、系统、电子设备及存储介质
本申请要求于2019年12月26日提交中国专利局、申请号为201911367801.0、发明名称为“一种垃圾数据的识别方法、系统、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种垃圾数据的识别方法、系统、一种电子设备及一种存储介质。
背景技术
在向分布式对象存储系统中上传对象时，需要将对象拆分为一个头对象以及多个小于等于4M的尾对象，将尾对象和头对象先后上传至数据存储池。但是在将尾对象写入数据存储池过程中，遇到断电等异常中段进程情况，会造成尾对象写入了数据存储池，记录尾对象信息的manifest结构和头对象数据并未写入数据存储池，写入数据存储池的尾对象就成了无法访问的垃圾数据。
因此，如何识别分布式对象存储系统中的垃圾数据是本领域技术人员目前需要解决的技术问题。
发明内容
本申请的目的是提供一种垃圾数据的识别方法、系统、一种电子设备及一种存储介质，能够识别分布式对象存储系统中的垃圾数据。
为解决上述技术问题，本申请提供一种垃圾数据的识别方法，该垃圾数据的识别方法包括：
向分布式对象存储系统上传对象；
获取所述分布式对象存储系统的索引存储池中的头对象标识符；
查询数据存储池中与所述头对象标识符对应的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；
将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
可选的，所述向分布式对象存储系统上传对象包括：
将待上传对象拆分为待上传头对象和待上传尾对象；
为所述待上传头对象分配头对象标识符，为所述待上传尾对象分配尾对象标识符；
将所述待上传尾对象的数据内容按照拆分顺序写入所述数据存储池；
将所述待上传头对象的数据内容和所述待上传头对象对应的待上传尾对象的尾对象标识符作为所述目标数据组，写入所述数据存储池；
将所述待上传头对象的头对象标识符写入所述索引存储池。
可选的，所述待上传尾对象的尾对象标识符以manifest结构存储于所述数据存储池。
可选的，将待上传对象拆分为待上传头对象和待上传尾对象包括：
按照预设策略将所述待上传对象拆分为1个待上传头对象和多个待上传尾对象；
其中，所述待上传头对象的数据大小小于第一预设值，所述待上传尾对象的数据大小小于第二预设值，第一预设值小于第二预设值。
可选的，将所述待上传头对象的头对象标识符写入所述索引存储池包括：
将所述待上传头对象的头对象标识符写入所述索引存储池的桶中；
相应的，获取所述分布式对象存储系统的索引存储池中的头对象标识符包括：
从所述分布式对象存储系统的索引存储池的桶中获取所述头对象标识符。
可选的，在将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据之后，还包括：
删除所述数据存储池中的所述垃圾数据。
可选的，还包括：
当接收到对象访问指令时，查询所述索引存储池中的与所述对象访问指令对应的目标头对象标识符；
将所述数据存储池中与所述目标头对象标识符对应的头对象的数据内容和尾对象的数据内容作为访问结果返回至访问端。
本申请还提供了一种垃圾数据的识别系统，该垃圾数据的识别系统包括：
对象上传模块，用于向分布式对象存储系统上传对象；
头像标识符获取模块，用于获取所述分布式对象存储系统的索引存储池中的头对象标识符；
数据组查询模块，用于查询数据存储池中与所述头对象标识符对应的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；
标记模块，用于将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
垃圾数据识别模块，用于将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
本申请还提供了一种存储介质，其上存储有计算机程序，所述计算机程序执行时实现上述垃圾数据的识别方法执行的步骤。
本申请还提供了一种电子设备，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器调用所述存储器中的计算机程序时实现上述垃圾数据的识别方法执行的步骤。
本申请提供了一种垃圾数据的识别方法，包括向分布式对象存储系统上传对象；获取所述分布式对象存储系统的索引存储池中的头对象标识符；查询数据存储池中与所述头对象标识符对应的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
本申请首先从分布式对象存储系统的索引存储池中获取头对象标识符，基于头对象标识符查询头对象对应的尾对象标识符，将数据存储池中尾对象标识符对应的尾对象标记为目标尾对象，进而将未被标记为目标尾对象的尾对象标记为垃圾数据。由于尾对象存储至数据存储池，但头对象未存储至数据存储池会导致出现垃圾数据，因此本申请基于头对象标识符确定已存储至数据存储池的头对象对应的尾对象，进而将不存在对应头对象的尾对象标记为垃圾数据，可见本申请能够识别分布式对象存储系统中的垃圾数据。本申请同时还提供了一种垃圾数据的识别系统、一种电子设备和一种存储介质，具有上述有益效果，在此不再赘述。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例所提供的一种垃圾数据的识别方法的流程图；
图2为本申请实施例所提供的一种头对象和尾对象的关系示意图；
图3为本申请实施例所提供的一种垃圾数据的识别系统的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面请参见图1，图1为本申请实施例所提供的一种垃圾数据的识别方 法的流程图。
具体步骤可以包括：
S101：向分布式对象存储系统上传对象；
其中，本实施例应用于分布式对象存储系统中，作为一种可行的实施方式本实施例可以通过PUT指令的方式向分布式存储系统上传对象，在上传过程中可以先将对象拆分为一个头对象以及多个小于或等于预设大小的尾对象，每段头对象和尾对象都会按照命名规则生成一个唯一的标识符(即oid，object identifier,对象标识符)。首先将尾对象按照拆分的顺序依次分别写入数据存储池(即data存储池)，然后再将记录尾对象标识符信息的manifest结构以及头对象数据内容写入数据存储池，最后再将头对象的标识符信息写入索引存储池(即index存储池)。可以理解的是，可以按照数据位置将对象拆分为头对象和尾对象，头对象为对象头部，尾对象为除对象头部的其他数据，一个对象可以拆分为一个头对象和至少一个尾对象。存储池是存储系统中存储数据的一组磁盘。
举例说明，本实施例中向分布式对象存储系统中存储对象的操作，例如需要向分布式对象存储系统上传目标数据A，此时先将目标数据A拆分为头对象A1、尾对象A2和尾对象A3，头对象A1的对象标识符为001，尾对象A2的对象标识符为002，尾对象A3的对象标识符为003，先将尾对象A2和尾对象A3的全部数据内容存储至数据存储池，再将头对象A1与对象标识符002和对象标识符003存储至数据存储池，最后将对象标识符001存储至索引存储池。
S102：获取所述分布式对象存储系统的索引存储池中的头对象标识符；
其中，在本步骤之前可以存在接收到垃圾数据检测指令的操作，当然本实施例也可以按照预设周期获取所述分布式对象存储系统的索引存储池中的头对象标识符，以实现对于分布式对象存储系统的垃圾数据定期检测。索引存储池中可以存储头对象的头对象标识符，具体的索引存储池中可以以桶的形式存储对象标识符。
S103：查询数据存储池中与所述头对象标识符对应的目标数据组；
其中，本步骤建立在已经得到头对象标识符的基础上，可以查询数据 存储池中与头对象标识符对应的目标数据组，具体的，上述目标数据组可以包括头对象的数据内容和头对象对应的尾对象标识符。即在数据存储池中头对象的数据内容与尾对象的尾对象标识符一同存储在数据存储池中，在得到头对象标识符的基础上，可以从数据存储池中查询该头对象标识符对应的头对象的数据内容，进而得到与头对象数据内容对应的尾对象标识符。作为一种可行的实施方式，本实施例中头对象标识符可以根据头对象的数据内容进行映射(如哈希映射)得到，因此能够根据头对象标识符确定对应的头对象数据内容；同理可知，可以根据尾对象标识符确定对应的尾对象数据内容。
S104：将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
其中，本实施例可以确定目标数据组中的为对象标识符，进而将存储数据池中与目标数据组中的为对象标识符对应的尾对象标记为目标尾对象。也就是说，在将目标尾对象存储至数据存储池的过程中，目标尾对象对应的头对象数据也写入了数据存储池，目标尾对象不为垃圾数据。
S105：将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
其中，本步骤建立在已经确定目标尾对象的基础上，可以将数据存储池中除目标尾对象之外的尾对象标记为垃圾数据，即存储池中不存在除所述目标尾对象之外的尾对象对应的头对象，这些尾对象无法进行访问。
本实施例首先从分布式对象存储系统的索引存储池中获取头对象标识符，基于头对象标识符查询头对象对应的尾对象标识符，将数据存储池中尾对象标识符对应的尾对象标记为目标尾对象，进而将未被标记为目标尾对象的尾对象标记为垃圾数据。由于尾对象存储至数据存储池，但头对象未存储至数据存储池会导致出现垃圾数据，因此本实施例基于头对象标识符确定已存储至数据存储池的头对象对应的尾对象，进而将不存在对应头对象的尾对象标记为垃圾数据，可见本实施例能够识别分布式对象存储系统中的垃圾数据。
作为对于图1对应实施例的进一步说明，在将所述数据存储池中除所述 目标尾对象之外的尾对象标记为垃圾数据之后，还可以删除所述数据存储池中的所述垃圾数据。
作为对于图1对应实施例的进一步说明，当接收到对象访问指令时，查询所述索引存储池中的与所述对象访问指令对应的目标头对象标识符；将所述数据存储池中与所述目标头对象标识符对应的头对象的数据内容和尾对象的数据内容作为访问结果返回至访问端。
作为对于图1对应实施例的进一步介绍，S101中上传对象的过程可以具体包括以下操作：
步骤1：将待上传对象拆分为待上传头对象和待上传尾对象；
其中，本步骤可以按照预设策略将所述待上传对象拆分为1个待上传头对象和多个待上传尾对象；其中，所述待上传头对象的数据大小小于第一预设值，所述待上传尾对象的数据大小小于第二预设值，第一预设值小于第二预设值。
步骤2：为所述待上传头对象分配头对象标识符，为所述待上传尾对象分配尾对象标识符；
步骤3：将所述待上传尾对象的数据内容按照拆分顺序写入所述数据存储池；
步骤4：将所述待上传头对象的数据内容和所述待上传头对象对应的待上传尾对象的尾对象标识符作为所述目标数据组，写入所述数据存储池；
其中，本步骤中待上传尾对象的尾对象标识符以manifest结构存储于所述数据存储池。具体的，manifest为存储系统中定义的一种数据，作用是管理应用对象和rados对象的对应关系，下载对象时，通过读取头对象中manifest信息，可下载完整的应用对象。
步骤5：将所述待上传头对象的头对象标识符写入所述索引存储池。
作为一种可行的实施方式，上述实施方式中可以将所述待上传头对象的头对象标识符写入所述索引存储池的桶中，以便所述分布式对象存储系统的索引存储池的桶中获取所述头对象标识符。
下面通过在实际应用中的实施例说明上述实施例描述的流程。
在上传一个对象到分布式对象存储系统中分布式对象，存储系统会按照一定策略将对象数据拆分成一个512k大小的头对象以及多个小于等于4M的尾对象。存储系统将对象存入存储池的过程是，首先依次按顺序将所有尾对象存入数据存储池命名为各自的对象标识符，然后将记录所有尾对象的manifest结构和头对象一起存入数据存储池以头对象的对象标识符命名，最后将头对象的对象标识符信息写入到索引存储池中存储桶的对象标识符的存储空间中。
根据对象的存储策略以及各个对象标识符之间的关联关系，本实施例提出了一种识别垃圾数据的方法，遍历索引存储池中桶存储的对象标识符后，之后根据桶存储的对象标识符遍历数据存储池中头对象标识符并记上标记，最后根据头对象标识符，找出头对象对应的所有尾对象并记上标记。最终数据存储池中所有未做标记的尾对象均为垃圾数据。请参见图2，图2为本申请实施例所提供的一种头对象和尾对象的关系示意图，图2中h1、h2和h3为头对象标识符，t1、t2、t3、t4、t5、t6为尾对象标识符，桶中存储有对象标识符h1、h2和h3，进而从数据存储池中查询h1、h2和h3对应的尾对象标识符t1、t2、t3、t4和t5，因此可以将未存储尾对象标识符的尾对象的数据内容标记为垃圾数据。
上述实施例提出一种识别存储系统中垃圾数据的方法，为清除存储系统垃圾提供了解决办法，丰富了分布式对象存储系统的功能，提高了存储系统磁盘空间的利用率，避免了磁盘空间的浪费，为垃圾数据清除提供准确的定位。
请参见图3，图3为本申请实施例所提供的一种垃圾数据的识别系统的结构示意图；
该系统可以包括：
对象上传模块100，用于向分布式对象存储系统上传对象；
头像标识符获取模块200，用于获取所述分布式对象存储系统的索引存储池中的头对象标识符；
数据组查询模块300，用于查询数据存储池中与所述头对象标识符对应 的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；
标记模块400，用于将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
垃圾数据识别模块500，用于将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
本实施例首先从分布式对象存储系统的索引存储池中获取头对象标识符，基于头对象标识符查询头对象对应的尾对象标识符，将数据存储池中尾对象标识符对应的尾对象标记为目标尾对象，进而将未被标记为目标尾对象的尾对象标记为垃圾数据。由于尾对象存储至数据存储池但头对象未存储至数据存储池会导致出现垃圾数据，因此本实施例基于头对象标识符确定已存储至数据存储池的头对象对应的尾对象，进而将不存在对应头对象的尾对象标记为垃圾数据，可见本实施例能够识别分布式对象存储系统中的垃圾数据。
进一步的，对象上传模块100包括：
对象拆分单元，用于将待上传对象拆分为待上传头对象和待上传尾对象；
标识符分配单元，用于为所述待上传头对象分配头对象标识符，为所述待上传尾对象分配尾对象标识符；
数据写入单元，用于将所述待上传尾对象的数据内容按照拆分顺序写入所述数据存储池；还用于将所述待上传头对象的数据内容和所述待上传头对象对应的待上传尾对象的尾对象标识符作为所述目标数据组，写入所述数据存储池；还用于将所述待上传头对象的头对象标识符写入所述索引存储池。
进一步的，所述待上传尾对象的尾对象标识符以manifest结构存储于所述数据存储池。
进一步的，对象拆分单元具体为用于按照预设策略将所述待上传对象拆分为1个待上传头对象和多个待上传尾对象的单元；其中，所述待上传头对象的数据大小小于第一预设值，所述待上传尾对象的数据大小小于第二 预设值，第一预设值小于第二预设值。
进一步的，数据写入单元用于将所述待上传头对象的头对象标识符写入所述索引存储池的桶中；
相应的，头像标识符获取模块200包括：
从所述分布式对象存储系统的索引存储池的桶中获取所述头对象标识符。
进一步的，还包括：
数据清除模块，用于在将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据之后，删除所述数据存储池中的所述垃圾数据。
进一步的，还包括：
对象访问模块，用于当接收到对象访问指令时，查询所述索引存储池中的与所述对象访问指令对应的目标头对象标识符；还用于将所述数据存储池中与所述目标头对象标识符对应的头对象的数据内容和尾对象的数据内容作为访问结果返回至访问端。
由于系统部分的实施例与方法部分的实施例相互对应，因此系统部分的实施例请参见方法部分的实施例的描述，这里暂不赘述。
本申请还提供了一种存储介质，其上存有计算机程序，该计算机程序被执行时可以实现上述实施例所提供的步骤。该存储介质可以包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本申请还提供了一种电子设备，可以包括存储器和处理器，所述存储器中存有计算机程序，所述处理器调用所述存储器中的计算机程序时，可以实现上述实施例所提供的步骤。当然所述电子设备还可以包括各种网络接口，电源等组件。
说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统而言，由于其与实施例公开的方法相对应，所以描 述的比较简单，相关之处参见方法部分说明即可。应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以对本申请进行若干改进和修饰，这些改进和修饰也落入本申请权利要求的保护范围内。
还需要说明的是，在本说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的状况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。

Claims (10)

  1. 一种垃圾数据的识别方法，其特征在于，包括：
    向分布式对象存储系统上传对象；
    获取所述分布式对象存储系统的索引存储池中的头对象标识符；
    查询数据存储池中与所述头对象标识符对应的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；
    将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
    将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
  2. 根据权利要求1所述识别方法，其特征在于，所述向分布式对象存储系统上传对象包括：
    将待上传对象拆分为待上传头对象和待上传尾对象；
    为所述待上传头对象分配头对象标识符，为所述待上传尾对象分配尾对象标识符；
    将所述待上传尾对象的数据内容按照拆分顺序写入所述数据存储池；
    将所述待上传头对象的数据内容和所述待上传头对象对应的待上传尾对象的尾对象标识符作为所述目标数据组，写入所述数据存储池；
    将所述待上传头对象的头对象标识符写入所述索引存储池。
  3. 根据权利要求2所述识别方法，其特征在于，所述待上传尾对象的尾对象标识符以manifest结构存储于所述数据存储池。
  4. 根据权利要求2所述识别方法，其特征在于，将待上传对象拆分为待上传头对象和待上传尾对象包括：
    按照预设策略将所述待上传对象拆分为1个待上传头对象和多个待上传尾对象；
    其中，所述待上传头对象的数据大小小于第一预设值，所述待上传尾对象的数据大小小于第二预设值，所述第一预设值小于所述第二预设值。
  5. 根据权利要求2所述识别方法，其特征在于，将所述待上传头对象的头对象标识符写入所述索引存储池包括：
    将所述待上传头对象的头对象标识符写入所述索引存储池的桶中；
    相应的，获取所述分布式对象存储系统的索引存储池中的头对象标识符包括：
    从所述分布式对象存储系统的索引存储池的桶中获取所述头对象标识符。
  6. 根据权利要求1所述识别方法，其特征在于，在将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据之后，还包括：
    删除所述数据存储池中的所述垃圾数据。
  7. 根据权利要求1至6任一项所述识别方法，其特征在于，还包括：
    当接收到对象访问指令时，查询所述索引存储池中的与所述对象访问指令对应的目标头对象标识符；
    将所述数据存储池中与所述目标头对象标识符对应的头对象的数据内容和尾对象的数据内容作为访问结果返回至访问端。
  8. 一种垃圾数据的识别系统，其特征在于，包括：
    对象上传模块，用于向分布式对象存储系统上传对象；
    头像标识符获取模块，用于获取所述分布式对象存储系统的索引存储池中的头对象标识符；
    数据组查询模块，用于查询数据存储池中与所述头对象标识符对应的目标数据组；其中，所述目标数据组包括头对象的数据内容和所述头对象对应的尾对象标识符；
    标记模块，用于将与所述目标数据组中的尾对象标识符对应的尾对象标记为目标尾对象；
    垃圾数据识别模块，用于将所述数据存储池中除所述目标尾对象之外的尾对象标记为垃圾数据。
  9. 一种电子设备，其特征在于，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器调用所述存储器中的计算机程序时实现如权利要求1至7任一项所述垃圾数据的识别方法的步骤。
  10. 一种存储介质，其特征在于，所述存储介质中存储有计算机可执行指令，所述计算机可执行指令被处理器加载并执行时，实现如上权利要 求1至7任一项所述垃圾数据的识别方法的步骤。
PCT/CN2020/110732 2019-12-26 2020-08-24 一种垃圾数据的识别方法、系统、电子设备及存储介质 WO2021128893A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/789,336 US11687489B2 (en) 2019-12-26 2020-08-24 Method and system for identifying garbage data, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911367801.0A CN111177075B (zh) 2019-12-26 2019-12-26 一种垃圾数据的识别方法、系统、电子设备及存储介质
CN201911367801.0 2019-12-26

Publications (1)

Publication Number Publication Date
WO2021128893A1 true WO2021128893A1 (zh) 2021-07-01

Family

ID=70652192

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/110732 WO2021128893A1 (zh) 2019-12-26 2020-08-24 一种垃圾数据的识别方法、系统、电子设备及存储介质

Country Status (3)

Country Link
US (1) US11687489B2 (zh)
CN (1) CN111177075B (zh)
WO (1) WO2021128893A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111177075B (zh) * 2019-12-26 2022-04-22 浪潮电子信息产业股份有限公司 一种垃圾数据的识别方法、系统、电子设备及存储介质
CN116136846B (zh) * 2023-04-19 2023-07-11 苏州浪潮智能科技有限公司 一种非结构化对象数据的管理方法、装置及存储系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107066604A (zh) * 2017-04-25 2017-08-18 努比亚技术有限公司 一种垃圾文件清理方法和终端
CN107818136A (zh) * 2017-09-26 2018-03-20 华为技术有限公司 用于回收垃圾对象数据的方法和装置
CN108415986A (zh) * 2018-02-11 2018-08-17 杭州朗和科技有限公司 一种数据处理方法、装置、系统、介质和计算设备
US20190057090A1 (en) * 2017-08-17 2019-02-21 Samsung Electronics Co., Ltd. Method and device of storing data object
CN111177075A (zh) * 2019-12-26 2020-05-19 浪潮电子信息产业股份有限公司 一种垃圾数据的识别方法、系统、电子设备及存储介质

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5687368A (en) * 1994-07-22 1997-11-11 Iowa State University Research Foundation, Inc. CPU-controlled garbage-collecting memory module
US7296180B1 (en) * 2004-06-30 2007-11-13 Sun Microsystems, Inc. Method for recovery of data
US20070044399A1 (en) 2005-08-15 2007-03-01 Sylvester Palko Window insulation apparatus
US7991807B2 (en) * 2007-11-21 2011-08-02 Sap Ag Method and system for garbage collection
US8074014B2 (en) * 2008-03-31 2011-12-06 Microsoft Corporation Storage systems using write off-loading
KR101562794B1 (ko) * 2009-08-04 2015-10-26 삼성전자주식회사 데이터 저장 장치
WO2011073940A1 (en) * 2009-12-17 2011-06-23 International Business Machines Corporation Data management in solid state storage systems
CA2808752C (en) * 2010-08-31 2016-06-28 Nec Corporation Storage system
KR20120049604A (ko) * 2010-11-09 2012-05-17 삼성전자주식회사 모바일 디바이스의 메모리 관리 방법 및 장치
KR101713314B1 (ko) * 2013-05-03 2017-03-07 한국전자통신연구원 가비지 파일의 삭제 방법 및 시스템
WO2015066085A1 (en) * 2013-10-28 2015-05-07 Bawaskar Swapnil Prakash Selecting files for compaction
US9619381B2 (en) * 2013-12-24 2017-04-11 International Business Machines Corporation Collaborative health management in a storage system
US10255304B2 (en) * 2014-09-30 2019-04-09 International Business Machines Corporation Removal of garbage data from a database
US10338817B2 (en) * 2014-12-30 2019-07-02 Sandisk Technologies Llc Systems and methods for storage recovery
US20160306822A1 (en) * 2015-04-17 2016-10-20 Samsung Electronics Co., Ltd. Load balancing of queries in replication enabled ssd storage
US10254998B2 (en) * 2015-11-03 2019-04-09 Samsung Electronics Co., Ltd. Coordinated garbage collection of flash devices in a distributed storage system
CN106708912B (zh) * 2015-11-18 2021-06-25 中兴通讯股份有限公司 垃圾文件识别及管理方法、识别装置、管理装置和终端
US10540095B1 (en) * 2016-08-12 2020-01-21 Pure Storage, Inc. Efficient garbage collection for stable data
US10783136B1 (en) * 2017-02-28 2020-09-22 Virtuozzo International Gmbh Management of garbage data in distributed systems
US10764045B2 (en) * 2017-06-30 2020-09-01 Microsoft Technology Licensing, Llc Encrypting object index in a distributed storage environment
US10659225B2 (en) * 2017-06-30 2020-05-19 Microsoft Technology Licensing, Llc Encrypting existing live unencrypted data using age-based garbage collection
CN110222039B (zh) * 2019-05-07 2023-09-29 平安科技（深圳）有限公司 数据存储及垃圾数据清理方法、装置、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107066604A (zh) * 2017-04-25 2017-08-18 努比亚技术有限公司 一种垃圾文件清理方法和终端
US20190057090A1 (en) * 2017-08-17 2019-02-21 Samsung Electronics Co., Ltd. Method and device of storing data object
CN107818136A (zh) * 2017-09-26 2018-03-20 华为技术有限公司 用于回收垃圾对象数据的方法和装置
CN108415986A (zh) * 2018-02-11 2018-08-17 杭州朗和科技有限公司 一种数据处理方法、装置、系统、介质和计算设备
CN111177075A (zh) * 2019-12-26 2020-05-19 浪潮电子信息产业股份有限公司 一种垃圾数据的识别方法、系统、电子设备及存储介质

Also Published As

Publication number Publication date
CN111177075B (zh) 2022-04-22
US20220374394A1 (en) 2022-11-24
CN111177075A (zh) 2020-05-19
US11687489B2 (en) 2023-06-27

Similar Documents

Publication Publication Date Title
US11799959B2 (en) Data processing method, apparatus, and system
CN106294190B (zh) 一种存储空间管理方法及装置
CN111125447A (zh) 一种元数据访问方法、装置、设备及可读存储介质
WO2021128893A1 (zh) 一种垃圾数据的识别方法、系统、电子设备及存储介质
CN101315628A (zh) 内存数据库系统及实现内存数据库的方法和装置
CN103577483B (zh) 数据存储方法和系统以及数据访问的方法和系统
CN109669622B (zh) 一种文件管理方法、文件管理装置、电子设备及存储介质
US9514170B1 (en) Priority queue using two differently-indexed single-index tables
CN109240607B (zh) 一种文件读取方法和装置
CN102541969B (zh) 基于fat文件系统的文件保护方法、系统及存储器
CN110888878A (zh) 一种面向业务的主数据管理方法及系统
CN107704475B (zh) 多层分布式非结构化数据存储方法、查询方法及装置
CN110688065A (zh) 一种存储空间管理方法、系统、电子设备及存储介质
CN117271531B (zh) 一种数据存储方法、系统、设备及介质
CN109101608A (zh) 一种数据存储方法、数据查询方法及装置
CN113157600A (zh) 一种叠瓦式硬盘的空间分配方法、文件存储系统及服务器
CN109325023B (zh) 一种数据处理方法及装置
US20070168938A1 (en) Maintaining object referential integrity for abstract objects
CN111143418A (zh) 一种数据库读取数据方法、装置、设备及存储介质
CN110096453B (zh) 数据处理方法及装置
CN115437579B (zh) 一种元数据管理方法、装置、计算机设备及可读存储介质
CN111752941B (zh) 一种数据存储、访问方法、装置、服务器及存储介质
CN112114738B (zh) 一种存储数据的方法及存储设备
CN109388613B (zh) 一种文件管理方法和电子设备
CN113609076A (zh) 一种文件存储方法和文件读取方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20906956

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20906956

Country of ref document: EP

Kind code of ref document: A1