WO2021147360A1 - Method, apparatus and device for processing asynchronous io data, and medium - Google Patents

Method, apparatus and device for processing asynchronous io data, and medium Download PDF

Info

Publication number
WO2021147360A1
WO2021147360A1 PCT/CN2020/117021 CN2020117021W WO2021147360A1 WO 2021147360 A1 WO2021147360 A1 WO 2021147360A1 CN 2020117021 W CN2020117021 W CN 2020117021W WO 2021147360 A1 WO2021147360 A1 WO 2021147360A1
Authority
WO
WIPO (PCT)
Prior art keywords
asynchronous
data
request
storage request
tag information
Prior art date
Application number
PCT/CN2020/117021
Other languages
French (fr)
Chinese (zh)
Inventor
翟亚永
胡永刚
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2021147360A1 publication Critical patent/WO2021147360A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • G06F16/1824Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G06F16/183Provision of network file services by network file servers, e.g. by using NFS, CIFS

Definitions

  • This application relates to the field of distributed object storage technology, and more specifically, to an asynchronous IO data processing method, device, equipment, and computer-readable storage medium.
  • the distributed object storage system is mounted through the NFS (Network File System) export directory to realize the file system access of the upper application and the distributed object storage mechanism of the underlying data.
  • NFS Network File System
  • the NFS protocol access based on object storage supports two forms of synchronous IO (Iuput/Output, input/output) and asynchronous IO.
  • the out-of-sequence IO data fragments in the asynchronous IO data are reordered in the cache to obtain the normal sequence of asynchronous IO data, and make subsequent
  • the asynchronous IO data is stored and read according to the reordered IO data fragments.
  • the purpose of this application is to provide an asynchronous IO data processing method, device, device, and computer-readable storage medium for improving the processing efficiency of asynchronous IO data.
  • An asynchronous IO data processing method including:
  • asynchronous IO data storage request Receiving an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
  • the tag information Including the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
  • obtaining the tag information of each of the asynchronous IO data fragments includes:
  • first tag information of each of the asynchronous IO data fragments is acquired; wherein the first tag information includes the ID (Identity Document, identity of the asynchronous IO data storage request). Identification) information, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
  • the method before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, the method further includes:
  • the metadata information of each asynchronous IO data segment is parsed, and the object name of the asynchronous IO data storage request is obtained from the metadata information.
  • receiving an asynchronous IO data storage request includes:
  • An asynchronous IO data processing device including:
  • the receiving module is configured to receive an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
  • the first acquisition module is used to acquire the tag information of each of the asynchronous IO data fragments, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data fragments in the data pool ;
  • the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
  • the second obtaining module is configured to obtain the object name of the asynchronous IO data reading request when the asynchronous IO data reading request is received;
  • the reading module is used to obtain all the tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset and asynchronous of the asynchronous IO data segment in the tag information
  • the storage location of the IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
  • the first acquisition module includes:
  • the obtaining unit is configured to obtain the first tag information of each of the asynchronous IO data fragments when the asynchronous IO data storage request is received; wherein the first tag information includes the ID of the asynchronous IO data storage request Information, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
  • the changing unit is configured to change the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request when the asynchronous IO data storage request ends, so as to obtain Tag information of each of the asynchronous IO data fragments.
  • it also includes:
  • the parsing module is used for parsing the metadata of each asynchronous IO data segment before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request Data information, obtaining the object name of the asynchronous IO data storage request from the metadata information.
  • the receiving module includes:
  • the receiving unit is configured to receive the asynchronous IO data storage request under the NFS protocol.
  • An asynchronous IO data processing device including:
  • Memory used to store computer programs
  • the processor is used to implement the steps of the asynchronous IO data processing method as described in any one of the above when executing the computer program.
  • This application provides an asynchronous IO data processing method, device, device, and computer-readable storage medium, where the method includes: receiving an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes multiple asynchronous IO data fragments ; Obtain the tag information of each asynchronous IO data segment, and store the tag information in the pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; where the tag information includes the asynchronous IO data storage request The object name, the offset of the asynchronous IO data fragment, the storage location of the asynchronous IO data fragment; when an asynchronous IO data read request is received, the object name of the asynchronous IO data read request is obtained; and the data tag storage pool is obtained.
  • the asynchronous IO data stores all the tag information corresponding to the object name of the request, and reads the asynchronous IO data corresponding to the asynchronous IO data read request according to the offset of the asynchronous IO data fragment in the tag information and
  • the above-mentioned solution disclosed in this application obtains the tag information of each asynchronous IO data segment, and stores the tag information in a pre-created data tag storage pool, and when an asynchronous IO data read request is received, the data tag is stored Obtain the tag information corresponding to the object name of the asynchronous IO data read request from the pool, and read the asynchronous IO data according to the offset of the asynchronous IO data fragment in the tag information and the storage location of the asynchronous IO data fragment, so that there is no need to correct or wrong
  • the sequential asynchronous IO data fragments are reordered, so that the processing time of asynchronous IO data can be shortened, and the processing efficiency and processing performance of asynchronous IO data can be improved.
  • FIG. 1 is a flowchart of an asynchronous IO data processing method provided by an embodiment of the application
  • FIG. 2 is a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the application.
  • FIG. 3 is a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the application.
  • FIG. 1 shows a flowchart of an asynchronous IO data processing method provided by an embodiment of the present application.
  • the asynchronous IO data processing method provided by an embodiment of the present application may include:
  • S11 Receive an asynchronous IO data storage request; where the asynchronous IO data storage request includes multiple asynchronous IO data fragments.
  • the object gateway when asynchronous IO data needs to be processed, the object gateway receives an asynchronous IO data storage request, where the asynchronous IO data storage request may include multiple asynchronous IO data fragments, and these asynchronous IO data The fragments may not be arranged in the order of the asynchronous IO data, that is, the asynchronous IO data fragments are arranged out of order.
  • S12 Obtain the tag information of each asynchronous IO data segment, and store the tag information in the pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; where the tag information may include asynchronous IO data storage The requested object name, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment.
  • asynchronous IO data fragments can be stored in the data pool.
  • the data tag storage pool can be pre-created at the bottom layer of the distributed object storage system to store tag information of asynchronous IO data fragments.
  • the tag information mentioned here can specifically include the object name of the asynchronous IO data storage request (specifically, the object name of the asynchronous IO data to be stored included in the asynchronous IO data storage request, and it belongs to the same asynchronous IO data.
  • the data fragments have the same object name), the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment, so that the object name of the asynchronous IO data storage request in the tag information can be used to determine the object to which it belongs.
  • the offset of the asynchronous IO data segment in determines the location of the asynchronous IO data segment in the asynchronous IO data, and the storage location of the asynchronous IO data segment determines the location where the asynchronous IO data segment is stored.
  • the object name of the asynchronous IO data read request can be obtained (specifically in the asynchronous IO data read request) Contains the object name of the asynchronous IO data to be read), so that the asynchronous IO data can be read through the object name of the asynchronous IO data read request.
  • S14 Obtain all tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and read according to the offset of the asynchronous IO data segment and the storage location of the asynchronous IO data segment in the tag information The asynchronous IO data corresponding to the asynchronous IO data read request.
  • all tag information corresponding to the object name of the asynchronous IO data read request can be obtained in the data tag storage pool to ensure that complete asynchronous IO data can be obtained.
  • the asynchronous IO data corresponding to the asynchronous IO data read request can be read according to all the acquired tag information. Specifically, the position of each asynchronous IO data segment in the asynchronous IO data can be determined according to the obtained offset of the asynchronous IO data segment in each tag information, and the position of each asynchronous IO data segment in the asynchronous IO data can be determined according to the asynchronous IO data segment in each tag information.
  • the storage location of the IO data fragment determines the location where the asynchronous IO data fragment is stored, and then reads the asynchronous IO data fragments sequentially and sequentially in the order of the offset of the asynchronous IO data fragment and the storage location of the asynchronous IO data fragment, thereby Improve the efficiency of reading asynchronous IO data.
  • the above solution disclosed in this application obtains the tag information of each asynchronous IO data segment and stores the tag information in a pre-created data tag storage pool, and when an asynchronous IO data read request is received, the data tag is stored Obtain the tag information corresponding to the object name of the asynchronous IO data read request from the pool, and read the asynchronous IO data according to the offset of the asynchronous IO data fragment in the tag information and the storage location of the asynchronous IO data fragment, so that there is no need to correct or wrong
  • the sequential asynchronous IO data fragments are reordered, so that the processing time of asynchronous IO data can be shortened, and the processing efficiency and processing performance of asynchronous IO data can be improved.
  • An asynchronous IO data processing method provided by an embodiment of the present application, which obtains label information of each asynchronous IO data fragment, may include:
  • the first label information of each asynchronous IO data segment is acquired; where the first label information may include the ID information of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the asynchronous The storage location of the IO data fragment;
  • the ID information of the asynchronous IO data storage request in the first label information is changed to the object name of the asynchronous IO data storage request to obtain the label information of each asynchronous IO data segment.
  • the label information of each asynchronous IO data fragment can be obtained in the following ways:
  • the corresponding request ID will be assigned to the asynchronous IO data storage request. Therefore, when the asynchronous IO data storage request is received, the first tag information of each asynchronous IO data segment can be obtained , Where the first tag information includes the ID information of the asynchronous IO data storage request (asynchronous IO data fragments that belong to the same asynchronous IO data have the same request ID information), the offset of the asynchronous IO data fragment, and the storage of the asynchronous IO data fragment When the asynchronous IO data is stored, the request ID information will be released.
  • the asynchronous IO in the first tag information can be changed
  • the ID information of the data storage request is changed to the object name of the asynchronous IO data storage request, so that the label information of each asynchronous IO data fragment can be obtained through the object name.
  • An asynchronous IO data processing method provided by an embodiment of the present application may further include: before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request:
  • the object gateway Before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, the object gateway can parse out the metadata information of each asynchronous IO data segment, and extract it from the metadata information Obtain the object name of the asynchronous IO data storage request, so as to use the obtained object name to change the request ID information in the first tag information.
  • An asynchronous IO data processing method provided by an embodiment of the present application, receiving an asynchronous IO data storage request, may include:
  • it can receive asynchronous IO data storage requests under the NFS protocol to realize the distributed object storage system's support for asynchronous IO data under the NFS protocol, while also improving the NFS protocol access efficiency of object storage and the object storage docking NFS interface Storage performance.
  • FIG. 2 shows a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the present application, which may include:
  • the receiving module 21 is configured to receive an asynchronous IO data storage request; wherein, the asynchronous IO data storage request may include multiple asynchronous IO data fragments;
  • the first obtaining module 22 is configured to obtain the tag information of each asynchronous IO data segment, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; wherein, the tag information It can include the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment;
  • the second obtaining module 23 is configured to obtain the object name of the asynchronous IO data reading request when the asynchronous IO data reading request is received;
  • the reading module 24 is used to obtain all the tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset of the asynchronous IO data segment and the asynchronous IO data segment in the tag information The storage location of reads the asynchronous IO data corresponding to the asynchronous IO data read request.
  • the first obtaining module 22 may include:
  • the acquiring unit is used to acquire the first tag information of each asynchronous IO data segment when the asynchronous IO data storage request is received; wherein the first tag information may include the ID information of the asynchronous IO data storage request and the information of the asynchronous IO data segment Offset, storage location of asynchronous IO data fragments;
  • the change unit is used to change the ID information of the asynchronous IO data storage request in the first label information to the object name of the asynchronous IO data storage request when the asynchronous IO data storage request ends, so as to obtain the label of each asynchronous IO data segment information.
  • the parsing module is used to analyze the metadata information of each asynchronous IO data segment before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, from the metadata information Get the object name of the asynchronous IO data storage request.
  • the receiving module 21 may include:
  • the receiving unit is used to receive an asynchronous IO data storage request under the NFS protocol.
  • FIG. 3 shows a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the present application, which may include:
  • the memory 31 is used to store computer programs
  • the processor 32 is configured to execute the computer program stored in the memory 31 and can implement the following steps:
  • the asynchronous IO data storage request includes multiple asynchronous IO data fragments; obtain the label information of each asynchronous IO data fragment, and store the label information in the pre-created data label storage pool, and Store asynchronous IO data fragments in the data pool; where the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment; when asynchronous IO data is received, read When requesting, get the object name of the asynchronous IO data read request; get all the tag information corresponding to the object name of the asynchronous IO data storage request in the data tag storage pool, and according to the offset of the asynchronous IO data fragment in the tag information And the storage location of the asynchronous IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program When the computer program is executed by a processor, the following steps can be implemented:
  • the asynchronous IO data storage request includes multiple asynchronous IO data fragments; obtain the label information of each asynchronous IO data fragment, and store the label information in the pre-created data label storage pool, and Store asynchronous IO data fragments in the data pool; where the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment; when asynchronous IO data is received, read When requesting, get the object name of the asynchronous IO data read request; get all the tag information corresponding to the object name of the asynchronous IO data storage request in the data tag storage pool, and according to the offset of the asynchronous IO data fragment in the tag information And the storage location of the asynchronous IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
  • the computer-readable storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes Medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A method, apparatus and device for processing asynchronous IO data, and a medium. The method for processing asynchronous IO data comprises: receiving an asynchronous IO data storage request; acquiring label information of each asynchronous IO data segment, storing the label information in a pre-created data label storage pool, and storing the asynchronous IO data segment in a data pool; when an asynchronous IO data reading request is received, acquiring an object name of the asynchronous IO data reading request; and acquiring, from the data label storage pool, all the label information corresponding to the object name of the asynchronous IO data reading request, and reading, according to an offset of the asynchronous IO data segment in the label information and the storage position of the asynchronous IO data segment, asynchronous IO data corresponding to the asynchronous IO data reading request. Disordered asynchronous IO data segments do not need to be re-sorted, so that the processing efficiency of asynchronous IO data is improved.

Description

一种异步IO数据处理方法、装置、设备及介质Asynchronous IO data processing method, device, equipment and medium
本申请要求于2020年01月21日提交中国专利局、申请号为202010071608.9、发明名称为“一种异步IO数据处理方法、装置、设备及介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on January 21, 2020, the application number is 202010071608.9, and the invention title is "An asynchronous IO data processing method, device, equipment, and medium". The entire content of the application is approved The reference is incorporated in this application.
技术领域Technical field
本申请涉及分布式对象存储技术领域,更具体地说,涉及一种异步IO数据处理方法、装置、设备及计算机可读存储介质。This application relates to the field of distributed object storage technology, and more specifically, to an asynchronous IO data processing method, device, equipment, and computer-readable storage medium.
背景技术Background technique
在分布式对象存储系统中,通过NFS(Network File System,网络文件系统)导出目录挂载分布式对象存储系统,以实现上层应用的文件系统访问和底层数据的分布式对象存储机制。其中,基于对象存储的NFS协议访问支持同步IO(Iuput/Output,输入/输出)和异步IO这两种形式。In the distributed object storage system, the distributed object storage system is mounted through the NFS (Network File System) export directory to realize the file system access of the upper application and the distributed object storage mechanism of the underlying data. Among them, the NFS protocol access based on object storage supports two forms of synchronous IO (Iuput/Output, input/output) and asynchronous IO.
目前,对于异步IO数据的处理,具体是在接收到异步IO数据之后,将异步IO数据中错序的IO数据片段在缓存中重新进行排序,以得到正常序列的异步IO数据,并使得后续可以按照重新排序的IO数据片段来对异步IO数据进行存储和读取。上述这种对异步IO数据进行处理的方式虽然可以保证异步IO数据能够得到正常处理,但是,由于需要对错序的IO数据片段进行重新排序,因此,则会降低异步IO数据的处理效率。At present, for the processing of asynchronous IO data, after receiving the asynchronous IO data, the out-of-sequence IO data fragments in the asynchronous IO data are reordered in the cache to obtain the normal sequence of asynchronous IO data, and make subsequent The asynchronous IO data is stored and read according to the reordered IO data fragments. Although the above-mentioned method of processing asynchronous IO data can ensure that asynchronous IO data can be processed normally, it will reduce the processing efficiency of asynchronous IO data due to the need to reorder the IO data fragments that are out of order.
综上所述,如何提高异步IO数据的处理效率,是目前本领域技术人员亟待解决的技术问题。In summary, how to improve the processing efficiency of asynchronous IO data is a technical problem urgently to be solved by those skilled in the art.
发明内容Summary of the invention
有鉴于此,本申请的目的是提供一种异步IO数据处理方法、装置、设备及计算机可读存储介质,用于提高异步IO数据的处理效率。In view of this, the purpose of this application is to provide an asynchronous IO data processing method, device, device, and computer-readable storage medium for improving the processing efficiency of asynchronous IO data.
为了实现上述目的,本申请提供如下技术方案:In order to achieve the above objectives, this application provides the following technical solutions:
一种异步IO数据处理方法,包括:An asynchronous IO data processing method, including:
接收异步IO数据存储请求;其中,所述异步IO数据存储请求中包括 多个异步IO数据片段;Receiving an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
获取每个所述异步IO数据片段的标签信息,并将所述标签信息存储在预先创建的数据标签存储池中,且将所述异步IO数据片段存储在数据池中;其中,所述标签信息包括所述异步IO数据存储请求的对象名、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;Obtain the tag information of each of the asynchronous IO data segments, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data segments in the data pool; wherein, the tag information Including the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
当接收到异步IO数据读取请求时,获取所述异步IO数据读取请求的对象名;When an asynchronous IO data read request is received, obtain the object name of the asynchronous IO data read request;
在所述数据标签存储池中获取与所述异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与所述异步IO数据读取请求对应的异步IO数据。Obtain all tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset of the asynchronous IO data segment and the storage location of the asynchronous IO data segment in the tag information Read the asynchronous IO data corresponding to the asynchronous IO data read request.
优选的,获取每个所述异步IO数据片段的标签信息,包括:Preferably, obtaining the tag information of each of the asynchronous IO data fragments includes:
在接收到所述异步IO数据存储请求时,获取每个所述异步IO数据片段的第一标签信息;其中,所述第一标签信息包括所述异步IO数据存储请求的ID(Identity Document,身份标识)信息、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;When the asynchronous IO data storage request is received, first tag information of each of the asynchronous IO data fragments is acquired; wherein the first tag information includes the ID (Identity Document, identity of the asynchronous IO data storage request). Identification) information, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
当所述异步IO数据存储请求结束时,将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名,以获取每个所述异步IO数据片段的标签信息。When the asynchronous IO data storage request ends, change the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request to obtain each asynchronous IO data storage request. The tag information of the IO data fragment.
优选的,在将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名之前,还包括:Preferably, before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, the method further includes:
解析每个所述异步IO数据片段的元数据信息,从所述元数据信息中获取所述异步IO数据存储请求的对象名。The metadata information of each asynchronous IO data segment is parsed, and the object name of the asynchronous IO data storage request is obtained from the metadata information.
优选的,接收异步IO数据存储请求,包括:Preferably, receiving an asynchronous IO data storage request includes:
接收NFS协议下的所述异步IO数据存储请求。Receive the asynchronous IO data storage request under the NFS protocol.
一种异步IO数据处理装置,包括:An asynchronous IO data processing device, including:
接收模块,用于接收异步IO数据存储请求;其中,所述异步IO数据存储请求中包括多个异步IO数据片段;The receiving module is configured to receive an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
第一获取模块,用于获取每个所述异步IO数据片段的标签信息,并将 所述标签信息存储在预先创建的数据标签存储池中,且将所述异步IO数据片段存储在数据池中;其中,所述标签信息包括所述异步IO数据存储请求的对象名、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;The first acquisition module is used to acquire the tag information of each of the asynchronous IO data fragments, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data fragments in the data pool ; Wherein, the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
第二获取模块,用于当接收到异步IO数据读取请求时,获取所述异步IO数据读取请求的对象名;The second obtaining module is configured to obtain the object name of the asynchronous IO data reading request when the asynchronous IO data reading request is received;
读取模块,用于在所述数据标签存储池中获取与所述异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与所述异步IO数据读取请求对应的异步IO数据。The reading module is used to obtain all the tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset and asynchronous of the asynchronous IO data segment in the tag information The storage location of the IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
优选的,所述第一获取模块包括:Preferably, the first acquisition module includes:
获取单元,用于在接收到所述异步IO数据存储请求时,获取每个所述异步IO数据片段的第一标签信息;其中,所述第一标签信息包括所述异步IO数据存储请求的ID信息、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;The obtaining unit is configured to obtain the first tag information of each of the asynchronous IO data fragments when the asynchronous IO data storage request is received; wherein the first tag information includes the ID of the asynchronous IO data storage request Information, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
更改单元,用于当所述异步IO数据存储请求结束时,将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名,以获取每个所述异步IO数据片段的标签信息。The changing unit is configured to change the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request when the asynchronous IO data storage request ends, so as to obtain Tag information of each of the asynchronous IO data fragments.
优选的,还包括:Preferably, it also includes:
解析模块,用于在将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名之前,解析每个所述异步IO数据片段的元数据信息,从所述元数据信息中获取所述异步IO数据存储请求的对象名。The parsing module is used for parsing the metadata of each asynchronous IO data segment before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request Data information, obtaining the object name of the asynchronous IO data storage request from the metadata information.
优选的,所述接收模块包括:Preferably, the receiving module includes:
接收单元,用于接收NFS协议下的所述异步IO数据存储请求。The receiving unit is configured to receive the asynchronous IO data storage request under the NFS protocol.
一种异步IO数据处理设备,包括:An asynchronous IO data processing device, including:
存储器,用于存储计算机程序;Memory, used to store computer programs;
处理器,用于执行所述计算机程序时实现如上述任一项所述的异步IO数据处理方法的步骤。The processor is used to implement the steps of the asynchronous IO data processing method as described in any one of the above when executing the computer program.
一种计算机可读存储介质,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如上述任一项所述的异步IO数据处理方法的步骤。A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the asynchronous IO data processing method as described in any one of the above are implemented.
本申请提供了一种异步IO数据处理方法、装置、设备及计算机可读存储介质,其中,该方法包括:接收异步IO数据存储请求;其中,异步IO数据存储请求中包括多个异步IO数据片段;获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且将异步IO数据片段存储在数据池中;其中,标签信息包括异步IO数据存储请求的对象名、异步IO数据片段的偏移量、异步IO数据片段的存储位置;当接收到异步IO数据读取请求时,获取异步IO数据读取请求的对象名;在数据标签存储池中获取与异步IO数据存储请求的对象名对应的所有的标签信息,并根据标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与异步IO数据读取请求对应的异步IO数据。This application provides an asynchronous IO data processing method, device, device, and computer-readable storage medium, where the method includes: receiving an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes multiple asynchronous IO data fragments ; Obtain the tag information of each asynchronous IO data segment, and store the tag information in the pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; where the tag information includes the asynchronous IO data storage request The object name, the offset of the asynchronous IO data fragment, the storage location of the asynchronous IO data fragment; when an asynchronous IO data read request is received, the object name of the asynchronous IO data read request is obtained; and the data tag storage pool is obtained The asynchronous IO data stores all the tag information corresponding to the object name of the request, and reads the asynchronous IO data corresponding to the asynchronous IO data read request according to the offset of the asynchronous IO data fragment in the tag information and the storage location of the asynchronous IO data fragment .
本申请公开的上述方案,通过获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且在接收到异步IO数据读取请求时,在数据标签存储池中获取与异步IO数据读取请求的对象名对应的标签信息,并根据标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取异步IO数据,以无需再对错序的异步IO数据片段进行重新排序处理,从而可以缩短异步IO数据的处理时间,提高异步IO数据的处理效率和处理性能。The above-mentioned solution disclosed in this application obtains the tag information of each asynchronous IO data segment, and stores the tag information in a pre-created data tag storage pool, and when an asynchronous IO data read request is received, the data tag is stored Obtain the tag information corresponding to the object name of the asynchronous IO data read request from the pool, and read the asynchronous IO data according to the offset of the asynchronous IO data fragment in the tag information and the storage location of the asynchronous IO data fragment, so that there is no need to correct or wrong The sequential asynchronous IO data fragments are reordered, so that the processing time of asynchronous IO data can be shortened, and the processing efficiency and processing performance of asynchronous IO data can be improved.
附图说明Description of the drawings
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.
图1为本申请实施例提供的一种异步IO数据处理方法的流程图;FIG. 1 is a flowchart of an asynchronous IO data processing method provided by an embodiment of the application;
图2为本申请实施例提供的一种异步IO数据处理装置的结构示意图;2 is a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the application;
图3为本申请实施例提供的一种异步IO数据处理设备的结构示意图。FIG. 3 is a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.
参见图1,其示出了本申请实施例提供的一种异步IO数据处理方法的流程图,本申请实施例提供的一种异步IO数据处理方法,可以包括:Referring to FIG. 1, it shows a flowchart of an asynchronous IO data processing method provided by an embodiment of the present application. The asynchronous IO data processing method provided by an embodiment of the present application may include:
S11:接收异步IO数据存储请求;其中,异步IO数据存储请求中包括多个异步IO数据片段。S11: Receive an asynchronous IO data storage request; where the asynchronous IO data storage request includes multiple asynchronous IO data fragments.
在分布式对象存储系统中,当需要对异步IO数据进行处理时,对象网关接收异步IO数据存储请求,其中,该异步IO数据存储请求中可以包括多个异步IO数据片段,且这些异步IO数据片段可能并未按照其在异步IO数据的顺序进行排列,也就是说,这些异步IO数据片段是错序排列的。In a distributed object storage system, when asynchronous IO data needs to be processed, the object gateway receives an asynchronous IO data storage request, where the asynchronous IO data storage request may include multiple asynchronous IO data fragments, and these asynchronous IO data The fragments may not be arranged in the order of the asynchronous IO data, that is, the asynchronous IO data fragments are arranged out of order.
S12:获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且将异步IO数据片段存储在数据池中;其中,标签信息可以包括异步IO数据存储请求的对象名、异步IO数据片段的偏移量、异步IO数据片段的存储位置。S12: Obtain the tag information of each asynchronous IO data segment, and store the tag information in the pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; where the tag information may include asynchronous IO data storage The requested object name, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment.
在接收到异步IO数据存储请求中包含的多个异步IO数据片段时,获取每个异步IO数据片段的标签信息,并将这些异步IO数据片段的标签信息存储在预先创建的数据标签存储池中,同时可以将异步IO数据片段存储在数据池中。其中,该数据标签存储池可以预先创建在分布式对象存储系统的底层,以用于存储异步IO数据片段的标签信息。When receiving multiple asynchronous IO data fragments included in the asynchronous IO data storage request, obtain the label information of each asynchronous IO data fragment, and store the label information of these asynchronous IO data fragments in the pre-created data label storage pool At the same time, asynchronous IO data fragments can be stored in the data pool. Among them, the data tag storage pool can be pre-created at the bottom layer of the distributed object storage system to store tag information of asynchronous IO data fragments.
另外,这里提及的标签信息具体可以包括异步IO数据存储请求的对象名(具体为异步IO数据存储请求中包含的要进行存储的异步IO数据的对象名,且同属一个异步IO数据的异步IO数据片段具有相同的对象名)、异步IO数据片段的偏移量及异步IO数据片段的存储位置,以便于通过标 签信息中的异步IO数据存储请求的对象名确定其所属的对象,通过标签信息中的异步IO数据片段的偏移量确定该异步IO数据片段在异步IO数据中所属的位置,并通过异步IO数据片段的存储位置确定该异步IO数据片段所存储的位置。In addition, the tag information mentioned here can specifically include the object name of the asynchronous IO data storage request (specifically, the object name of the asynchronous IO data to be stored included in the asynchronous IO data storage request, and it belongs to the same asynchronous IO data. The data fragments have the same object name), the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment, so that the object name of the asynchronous IO data storage request in the tag information can be used to determine the object to which it belongs. The offset of the asynchronous IO data segment in determines the location of the asynchronous IO data segment in the asynchronous IO data, and the storage location of the asynchronous IO data segment determines the location where the asynchronous IO data segment is stored.
由上述可知,其并没有对所接收到的异步IO数据片段进行重新排序,然后,再进行存储,因此,可以缩短异步IO数据存储所花费的时间,从而可以提高异步IO数据的存储效率,而且通过创建数据标签存储池来对异步IO数据片段的标签信息进行存储的方式可以保证异步IO数据的完整性和一致性,从而可以提高异步IO数据存储的性能。It can be seen from the above that it does not reorder the received asynchronous IO data fragments and then store them. Therefore, the time spent on asynchronous IO data storage can be shortened, thereby improving the storage efficiency of asynchronous IO data, and By creating a data tag storage pool to store the tag information of the asynchronous IO data fragments, the integrity and consistency of the asynchronous IO data can be guaranteed, and the performance of the asynchronous IO data storage can be improved.
S13:当接收到异步IO数据读取请求时,获取异步IO数据读取请求的对象名。S13: When receiving the asynchronous IO data reading request, obtain the object name of the asynchronous IO data reading request.
在完成对异步IO数据片段标签信息和异步IO数据的存储之后,若接收到异步IO数据读取请求,则可以获取该异步IO数据读取请求的对象名(具体为异步IO数据读取请求中包含的要进行读取的异步IO数据的对象名),以便于通过异步IO数据读取请求的对象名对异步IO数据进行读取。After completing the storage of the asynchronous IO data fragment label information and the asynchronous IO data, if an asynchronous IO data read request is received, the object name of the asynchronous IO data read request can be obtained (specifically in the asynchronous IO data read request) Contains the object name of the asynchronous IO data to be read), so that the asynchronous IO data can be read through the object name of the asynchronous IO data read request.
S14:在数据标签存储池中获取与异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与异步IO数据读取请求对应的异步IO数据。S14: Obtain all tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and read according to the offset of the asynchronous IO data segment and the storage location of the asynchronous IO data segment in the tag information The asynchronous IO data corresponding to the asynchronous IO data read request.
在获取到异步IO数据读取请求的对象名时,可以在数据标签存储池中获取与异步IO数据读取请求的对象名对应的所有的标签信息,以保证可以获取到完整的异步IO数据。之后,则可以根据所获取到所有的标签信息来读取与异步IO数据读取请求对应的异步IO数据。具体地,可以根据所获取到的每个标签信息中的异步IO数据片段的偏移量分别确定每个异步IO数据片段在异步IO数据中所处的位置,并根据每个标签信息中的异步IO数据片段的存储位置确定异步IO数据片段所存储的位置,然后,按照异步IO数据片段的偏移量的顺序及异步IO数据片段的存储位置顺序地且依次地读取异步IO数据片段,从而提高异步IO数据的读取效率。When the object name of the asynchronous IO data read request is obtained, all tag information corresponding to the object name of the asynchronous IO data read request can be obtained in the data tag storage pool to ensure that complete asynchronous IO data can be obtained. After that, the asynchronous IO data corresponding to the asynchronous IO data read request can be read according to all the acquired tag information. Specifically, the position of each asynchronous IO data segment in the asynchronous IO data can be determined according to the obtained offset of the asynchronous IO data segment in each tag information, and the position of each asynchronous IO data segment in the asynchronous IO data can be determined according to the asynchronous IO data segment in each tag information. The storage location of the IO data fragment determines the location where the asynchronous IO data fragment is stored, and then reads the asynchronous IO data fragments sequentially and sequentially in the order of the offset of the asynchronous IO data fragment and the storage location of the asynchronous IO data fragment, thereby Improve the efficiency of reading asynchronous IO data.
本申请公开的上述方案,通过获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且在接收到异步IO 数据读取请求时,在数据标签存储池中获取与异步IO数据读取请求的对象名对应的标签信息,并根据标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取异步IO数据,以无需再对错序的异步IO数据片段进行重新排序处理,从而可以缩短异步IO数据的处理时间,提高异步IO数据的处理效率和处理性能。The above solution disclosed in this application obtains the tag information of each asynchronous IO data segment and stores the tag information in a pre-created data tag storage pool, and when an asynchronous IO data read request is received, the data tag is stored Obtain the tag information corresponding to the object name of the asynchronous IO data read request from the pool, and read the asynchronous IO data according to the offset of the asynchronous IO data fragment in the tag information and the storage location of the asynchronous IO data fragment, so that there is no need to correct or wrong The sequential asynchronous IO data fragments are reordered, so that the processing time of asynchronous IO data can be shortened, and the processing efficiency and processing performance of asynchronous IO data can be improved.
本申请实施例提供的一种异步IO数据处理方法,获取每个异步IO数据片段的标签信息,可以包括:An asynchronous IO data processing method provided by an embodiment of the present application, which obtains label information of each asynchronous IO data fragment, may include:
在接收到异步IO数据存储请求时,获取每个异步IO数据片段的第一标签信息;其中,第一标签信息可以包括异步IO数据存储请求的ID信息、异步IO数据片段的偏移量、异步IO数据片段的存储位置;When an asynchronous IO data storage request is received, the first label information of each asynchronous IO data segment is acquired; where the first label information may include the ID information of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the asynchronous The storage location of the IO data fragment;
当异步IO数据存储请求结束时,将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名,以获取每个异步IO数据片段的标签信息。When the asynchronous IO data storage request ends, the ID information of the asynchronous IO data storage request in the first label information is changed to the object name of the asynchronous IO data storage request to obtain the label information of each asynchronous IO data segment.
具体可以通过以下方式获取每个异步IO数据片段的标签信息:Specifically, the label information of each asynchronous IO data fragment can be obtained in the following ways:
考虑到在接收到异步IO数据存储请求初期,会为异步IO数据存储请求分配对应的请求ID,因此,在接收到异步IO数据存储请求时,可以获取每个异步IO数据片段的第一标签信息,其中,第一标签信息包括异步IO数据存储请求的ID信息(同属一个异步IO数据的异步IO数据片段具有相同的请求ID信息)、异步IO数据片段的偏移量、异步IO数据片段的存储位置,而当异步IO数据完成存储之后,请求ID信息即会释放,为了便于后续可以准确地读取异步IO数据,则当异步IO数据存储请求结束时,可以将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名,以便于通过对象名获取每个异步IO数据片段的标签信息。Considering that in the initial stage of receiving the asynchronous IO data storage request, the corresponding request ID will be assigned to the asynchronous IO data storage request. Therefore, when the asynchronous IO data storage request is received, the first tag information of each asynchronous IO data segment can be obtained , Where the first tag information includes the ID information of the asynchronous IO data storage request (asynchronous IO data fragments that belong to the same asynchronous IO data have the same request ID information), the offset of the asynchronous IO data fragment, and the storage of the asynchronous IO data fragment When the asynchronous IO data is stored, the request ID information will be released. In order to facilitate the subsequent accurate reading of the asynchronous IO data, when the asynchronous IO data storage request ends, the asynchronous IO in the first tag information can be changed The ID information of the data storage request is changed to the object name of the asynchronous IO data storage request, so that the label information of each asynchronous IO data fragment can be obtained through the object name.
本申请实施例提供的一种异步IO数据处理方法,在将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名之前,还可以包括:An asynchronous IO data processing method provided by an embodiment of the present application may further include: before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request:
解析每个异步IO数据片段的元数据信息,从元数据信息中获取异步 IO数据存储请求的对象名。Parse the metadata information of each asynchronous IO data fragment, and obtain the object name of the asynchronous IO data storage request from the metadata information.
在将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名之前,对象网关可以解析出每个异步IO数据片段的元数据信息,并从元数据信息中获取异步IO数据存储请求的对象名,以便于利用所获取到的对象名对第一标签信息中的请求ID信息进行更改。Before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, the object gateway can parse out the metadata information of each asynchronous IO data segment, and extract it from the metadata information Obtain the object name of the asynchronous IO data storage request, so as to use the obtained object name to change the request ID information in the first tag information.
本申请实施例提供的一种异步IO数据处理方法,接收异步IO数据存储请求,可以包括:An asynchronous IO data processing method provided by an embodiment of the present application, receiving an asynchronous IO data storage request, may include:
接收NFS协议下的异步IO数据存储请求。Receive asynchronous IO data storage requests under the NFS protocol.
具体可以接收NFS协议下的异步IO数据存储请求,以在实现分布式对象存储系统对NFS协议下异步IO数据的支持的同时,还提高了提高对象存储的NFS协议访问效率及对象存储对接NFS接口的存储性能。Specifically, it can receive asynchronous IO data storage requests under the NFS protocol to realize the distributed object storage system's support for asynchronous IO data under the NFS protocol, while also improving the NFS protocol access efficiency of object storage and the object storage docking NFS interface Storage performance.
本申请实施例还提供了一种异步IO数据处理装置,参见图2,其示出了本申请实施例提供的一种异步IO数据处理装置的结构示意图,可以包括:An embodiment of the present application also provides an asynchronous IO data processing device. See FIG. 2, which shows a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the present application, which may include:
接收模块21,用于接收异步IO数据存储请求;其中,异步IO数据存储请求中可以包括多个异步IO数据片段;The receiving module 21 is configured to receive an asynchronous IO data storage request; wherein, the asynchronous IO data storage request may include multiple asynchronous IO data fragments;
第一获取模块22,用于获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且将异步IO数据片段存储在数据池中;其中,标签信息可以包括异步IO数据存储请求的对象名、异步IO数据片段的偏移量、异步IO数据片段的存储位置;The first obtaining module 22 is configured to obtain the tag information of each asynchronous IO data segment, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data segment in the data pool; wherein, the tag information It can include the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment;
第二获取模块23,用于当接收到异步IO数据读取请求时,获取异步IO数据读取请求的对象名;The second obtaining module 23 is configured to obtain the object name of the asynchronous IO data reading request when the asynchronous IO data reading request is received;
读取模块24,用于在数据标签存储池中获取与异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与异步IO数据读取请求对应的异步IO数据。The reading module 24 is used to obtain all the tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset of the asynchronous IO data segment and the asynchronous IO data segment in the tag information The storage location of reads the asynchronous IO data corresponding to the asynchronous IO data read request.
本申请实施例提供的一种异步IO数据处理装置,第一获取模块22可以包括:In an asynchronous IO data processing device provided by an embodiment of the present application, the first obtaining module 22 may include:
获取单元,用于在接收到异步IO数据存储请求时,获取每个异步IO数据片段的第一标签信息;其中,第一标签信息可以包括异步IO数据存储请求的ID信息、异步IO数据片段的偏移量、异步IO数据片段的存储位置;The acquiring unit is used to acquire the first tag information of each asynchronous IO data segment when the asynchronous IO data storage request is received; wherein the first tag information may include the ID information of the asynchronous IO data storage request and the information of the asynchronous IO data segment Offset, storage location of asynchronous IO data fragments;
更改单元,用于当异步IO数据存储请求结束时,将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名,以获取每个异步IO数据片段的标签信息。The change unit is used to change the ID information of the asynchronous IO data storage request in the first label information to the object name of the asynchronous IO data storage request when the asynchronous IO data storage request ends, so as to obtain the label of each asynchronous IO data segment information.
本申请实施例提供的一种异步IO数据处理装置,还可以包括:The asynchronous IO data processing device provided by the embodiment of the present application may further include:
解析模块,用于在将第一标签信息中的异步IO数据存储请求的ID信息更改为异步IO数据存储请求的对象名之前,解析每个异步IO数据片段的元数据信息,从元数据信息中获取异步IO数据存储请求的对象名。The parsing module is used to analyze the metadata information of each asynchronous IO data segment before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request, from the metadata information Get the object name of the asynchronous IO data storage request.
本申请实施例提供的一种异步IO数据处理装置,接收模块21可以包括:In an asynchronous IO data processing apparatus provided by an embodiment of the present application, the receiving module 21 may include:
接收单元,用于接收NFS协议下的异步IO数据存储请求。The receiving unit is used to receive an asynchronous IO data storage request under the NFS protocol.
本申请实施例还提供了一种异步IO数据处理设备,参见图3,其示出了本申请实施例提供的一种异步IO数据处理设备的结构示意图,可以包括:An embodiment of the present application also provides an asynchronous IO data processing device. See FIG. 3, which shows a schematic structural diagram of an asynchronous IO data processing device provided by an embodiment of the present application, which may include:
存储器31,用于存储计算机程序;The memory 31 is used to store computer programs;
处理器32,用于执行存储器31存储的计算机程序时可实现如下步骤:The processor 32 is configured to execute the computer program stored in the memory 31 and can implement the following steps:
接收异步IO数据存储请求;其中,异步IO数据存储请求中包括多个异步IO数据片段;获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且将异步IO数据片段存储在数据池中;其中,标签信息包括异步IO数据存储请求的对象名、异步IO数据片段的偏移量、异步IO数据片段的存储位置;当接收到异步IO数据读取请求时,获取异步IO数据读取请求的对象名;在数据标签存储池中获取与异步IO数据存储请求的对象名对应的所有的标签信息,并根据标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与异步IO数据读取请求对应的异步IO数据。Receive an asynchronous IO data storage request; where the asynchronous IO data storage request includes multiple asynchronous IO data fragments; obtain the label information of each asynchronous IO data fragment, and store the label information in the pre-created data label storage pool, and Store asynchronous IO data fragments in the data pool; where the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment; when asynchronous IO data is received, read When requesting, get the object name of the asynchronous IO data read request; get all the tag information corresponding to the object name of the asynchronous IO data storage request in the data tag storage pool, and according to the offset of the asynchronous IO data fragment in the tag information And the storage location of the asynchronous IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
本申请实施例还提供了一种计算机可读存储介质,计算机可读存储介质上存储有计算机程序,计算机程序被处理器执行时可实现如下步骤:The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps can be implemented:
接收异步IO数据存储请求;其中,异步IO数据存储请求中包括多个异步IO数据片段;获取每个异步IO数据片段的标签信息,并将标签信息存储在预先创建的数据标签存储池中,且将异步IO数据片段存储在数据池中;其中,标签信息包括异步IO数据存储请求的对象名、异步IO数据片段的偏移量、异步IO数据片段的存储位置;当接收到异步IO数据读取请求时,获取异步IO数据读取请求的对象名;在数据标签存储池中获取与异步IO数据存储请求的对象名对应的所有的标签信息,并根据标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与异步IO数据读取请求对应的异步IO数据。Receive an asynchronous IO data storage request; where the asynchronous IO data storage request includes multiple asynchronous IO data fragments; obtain the label information of each asynchronous IO data fragment, and store the label information in the pre-created data label storage pool, and Store asynchronous IO data fragments in the data pool; where the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data fragment, and the storage location of the asynchronous IO data fragment; when asynchronous IO data is received, read When requesting, get the object name of the asynchronous IO data read request; get all the tag information corresponding to the object name of the asynchronous IO data storage request in the data tag storage pool, and according to the offset of the asynchronous IO data fragment in the tag information And the storage location of the asynchronous IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
该计算机可读存储介质可以包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The computer-readable storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes Medium.
本申请实施例提供的一种异步IO数据处理装置、设备及计算机可读存储介质中相关部分的说明可以参见本申请实施例提供的一种异步IO数据处理方法中对应部分的详细说明,在此不再赘述。For descriptions of related parts in an asynchronous IO data processing apparatus, equipment, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the asynchronous IO data processing method provided in the embodiments of the present application. No longer.
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。另外,本申请实施例提供的上述技术方案中与现有技术中对应技术方案实现原理一致的部分并未详细说明,以免过多赘述。It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. Moreover, the terms "including", "including" or any other variants thereof are intended to cover non-exclusive inclusions, so as to include elements inherent in a process, method, article, or equipment of a series of elements. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, article, or equipment that includes the element. In addition, the parts of the foregoing technical solutions provided by the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail, so as to avoid redundant description.
对所公开的实施例的上述说明,使本领域技术人员能够实现或使用本 申请。对这些实施例的多种修改对本领域技术人员来说将是显而易见的,本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下,在其它实施例中实现。因此,本申请将不会被限制于本文所示的这些实施例,而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, this application will not be limited to the embodiments shown in this document, but should conform to the widest scope consistent with the principles and novel features disclosed in this document.

Claims (10)

  1. 一种异步IO数据处理方法,其特征在于,包括:An asynchronous IO data processing method, characterized in that it comprises:
    接收异步IO数据存储请求;其中,所述异步IO数据存储请求中包括多个异步IO数据片段;Receiving an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
    获取每个所述异步IO数据片段的标签信息,并将所述标签信息存储在预先创建的数据标签存储池中,且将所述异步IO数据片段存储在数据池中;其中,所述标签信息包括所述异步IO数据存储请求的对象名、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;Obtain the tag information of each of the asynchronous IO data segments, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data segments in the data pool; wherein, the tag information Including the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
    当接收到异步IO数据读取请求时,获取所述异步IO数据读取请求的对象名;When an asynchronous IO data read request is received, obtain the object name of the asynchronous IO data read request;
    在所述数据标签存储池中获取与所述异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与所述异步IO数据读取请求对应的异步IO数据。Obtain all tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset of the asynchronous IO data segment and the storage location of the asynchronous IO data segment in the tag information Read the asynchronous IO data corresponding to the asynchronous IO data read request.
  2. 根据权利要求1所述的异步IO数据处理方法,其特征在于,获取每个所述异步IO数据片段的标签信息,包括:The asynchronous IO data processing method according to claim 1, wherein acquiring the label information of each of the asynchronous IO data fragments comprises:
    在接收到所述异步IO数据存储请求时,获取每个所述异步IO数据片段的第一标签信息;其中,所述第一标签信息包括所述异步IO数据存储请求的ID信息、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;When receiving the asynchronous IO data storage request, obtain the first tag information of each of the asynchronous IO data segments; wherein the first tag information includes the ID information of the asynchronous IO data storage request, the asynchronous The offset of the IO data segment, and the storage location of the asynchronous IO data segment;
    当所述异步IO数据存储请求结束时,将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名,以获取每个所述异步IO数据片段的标签信息。When the asynchronous IO data storage request ends, change the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request to obtain each asynchronous IO data storage request. The tag information of the IO data fragment.
  3. 根据权利要求2所述的异步IO数据处理方法,其特征在于,在将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名之前,还包括:The asynchronous IO data processing method according to claim 2, wherein before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request ,Also includes:
    解析每个所述异步IO数据片段的元数据信息,从所述元数据信息中获取所述异步IO数据存储请求的对象名。The metadata information of each asynchronous IO data segment is parsed, and the object name of the asynchronous IO data storage request is obtained from the metadata information.
  4. 根据权利要求1所述的异步IO数据处理方法,其特征在于,接收 异步IO数据存储请求,包括:The asynchronous IO data processing method according to claim 1, wherein receiving an asynchronous IO data storage request comprises:
    接收NFS协议下的所述异步IO数据存储请求。Receive the asynchronous IO data storage request under the NFS protocol.
  5. 一种异步IO数据处理装置,其特征在于,包括:An asynchronous IO data processing device, characterized in that it comprises:
    接收模块,用于接收异步IO数据存储请求;其中,所述异步IO数据存储请求中包括多个异步IO数据片段;The receiving module is configured to receive an asynchronous IO data storage request; wherein the asynchronous IO data storage request includes a plurality of asynchronous IO data fragments;
    第一获取模块,用于获取每个所述异步IO数据片段的标签信息,并将所述标签信息存储在预先创建的数据标签存储池中,且将所述异步IO数据片段存储在数据池中;其中,所述标签信息包括所述异步IO数据存储请求的对象名、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;The first acquisition module is used to acquire the tag information of each of the asynchronous IO data fragments, and store the tag information in a pre-created data tag storage pool, and store the asynchronous IO data fragments in the data pool ; Wherein, the tag information includes the object name of the asynchronous IO data storage request, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
    第二获取模块,用于当接收到异步IO数据读取请求时,获取所述异步IO数据读取请求的对象名;The second obtaining module is configured to obtain the object name of the asynchronous IO data reading request when the asynchronous IO data reading request is received;
    读取模块,用于在所述数据标签存储池中获取与所述异步IO数据读取请求的对象名对应的所有的标签信息,并根据述标签信息中异步IO数据片段的偏移量及异步IO数据片段的存储位置读取与所述异步IO数据读取请求对应的异步IO数据。The reading module is used to obtain all the tag information corresponding to the object name of the asynchronous IO data read request in the data tag storage pool, and according to the offset and asynchronous of the asynchronous IO data segment in the tag information The storage location of the IO data fragment reads the asynchronous IO data corresponding to the asynchronous IO data read request.
  6. 根据权利要求5所述的异步IO数据处理装置,其特征在于,所述第一获取模块包括:The asynchronous IO data processing device according to claim 5, wherein the first acquisition module comprises:
    获取单元,用于在接收到所述异步IO数据存储请求时,获取每个所述异步IO数据片段的第一标签信息;其中,所述第一标签信息包括所述异步IO数据存储请求的ID信息、所述异步IO数据片段的偏移量、所述异步IO数据片段的存储位置;The obtaining unit is configured to obtain the first tag information of each of the asynchronous IO data fragments when the asynchronous IO data storage request is received; wherein the first tag information includes the ID of the asynchronous IO data storage request Information, the offset of the asynchronous IO data segment, and the storage location of the asynchronous IO data segment;
    更改单元,用于当所述异步IO数据存储请求结束时,将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名,以获取每个所述异步IO数据片段的标签信息。The changing unit is configured to change the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request when the asynchronous IO data storage request ends, so as to obtain Tag information of each of the asynchronous IO data fragments.
  7. 根据权利要求6所述的异步IO数据处理装置,其特征在于,还包括:The asynchronous IO data processing device according to claim 6, further comprising:
    解析模块,用于在将所述第一标签信息中的所述异步IO数据存储请求的ID信息更改为所述异步IO数据存储请求的对象名之前,解析每个所述 异步IO数据片段的元数据信息,从所述元数据信息中获取所述异步IO数据存储请求的对象名。The parsing module is used for parsing the metadata of each asynchronous IO data segment before changing the ID information of the asynchronous IO data storage request in the first tag information to the object name of the asynchronous IO data storage request Data information, obtaining the object name of the asynchronous IO data storage request from the metadata information.
  8. 根据权利要求5所述的异步IO数据处理装置,其特征在于,所述接收模块包括:The asynchronous IO data processing device according to claim 5, wherein the receiving module comprises:
    接收单元,用于接收NFS协议下的所述异步IO数据存储请求。The receiving unit is configured to receive the asynchronous IO data storage request under the NFS protocol.
  9. 一种异步IO数据处理设备,其特征在于,包括:An asynchronous IO data processing device, which is characterized in that it comprises:
    存储器,用于存储计算机程序;Memory, used to store computer programs;
    处理器,用于执行所述计算机程序时实现如权利要求1至4任一项所述的异步IO数据处理方法的步骤。The processor is configured to implement the steps of the asynchronous IO data processing method according to any one of claims 1 to 4 when the computer program is executed.
  10. 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1至4任一项所述的异步IO数据处理方法的步骤。A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the asynchronous IO data according to any one of claims 1 to 4 is realized Processing method steps.
PCT/CN2020/117021 2020-01-21 2020-09-23 Method, apparatus and device for processing asynchronous io data, and medium WO2021147360A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010071608.9A CN111241036B (en) 2020-01-21 2020-01-21 Asynchronous IO data processing method, device, equipment and medium
CN202010071608.9 2020-01-21

Publications (1)

Publication Number Publication Date
WO2021147360A1 true WO2021147360A1 (en) 2021-07-29

Family

ID=70864992

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/117021 WO2021147360A1 (en) 2020-01-21 2020-09-23 Method, apparatus and device for processing asynchronous io data, and medium

Country Status (2)

Country Link
CN (1) CN111241036B (en)
WO (1) WO2021147360A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241036B (en) * 2020-01-21 2023-01-10 苏州浪潮智能科技有限公司 Asynchronous IO data processing method, device, equipment and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170318094A1 (en) * 2016-04-29 2017-11-02 Netapp, Inc. Self-organizing storage system for asynchronous storage service
CN108399050A (en) * 2018-02-02 2018-08-14 新华三技术有限公司 A kind of data processing method and device
CN108763473A (en) * 2018-05-29 2018-11-06 郑州云海信息技术有限公司 A kind of the native object storage method and device of distributed storage
CN108959500A (en) * 2018-06-26 2018-12-07 郑州云海信息技术有限公司 A kind of object storage method, device, equipment and computer readable storage medium
US20190020652A1 (en) * 2017-07-14 2019-01-17 EMC IP Holding Company LLC System and method for local data ip based network security for preventing data breach attempts in a multi-tenant protection storage deployment
CN111241036A (en) * 2020-01-21 2020-06-05 苏州浪潮智能科技有限公司 Asynchronous IO data processing method, device, equipment and medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101488104B (en) * 2009-02-26 2011-05-04 北京云快线软件服务有限公司 System and method for implementing high-efficiency security memory
CN103106233A (en) * 2012-11-02 2013-05-15 北京邮电大学 Asynchronous index and read-write method of massive files applied to search engine
CN105391649B (en) * 2015-10-16 2018-09-04 浪潮(北京)电子信息产业有限公司 A kind of array dispatching method and device
CN107277128B (en) * 2017-06-15 2020-09-22 苏州浪潮智能科技有限公司 Method and device for requesting processing order preservation in distributed storage protocol
CN107516049A (en) * 2017-07-31 2017-12-26 广东美的制冷设备有限公司 Method for reading data, device, safety chip and computer equipment
CN107797945A (en) * 2017-10-31 2018-03-13 郑州云海信息技术有限公司 A kind of storage system and its date storage method, device, system and equipment
CN110570075B (en) * 2019-07-18 2022-04-05 北京邮电大学 Power business edge calculation task allocation method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170318094A1 (en) * 2016-04-29 2017-11-02 Netapp, Inc. Self-organizing storage system for asynchronous storage service
US20190020652A1 (en) * 2017-07-14 2019-01-17 EMC IP Holding Company LLC System and method for local data ip based network security for preventing data breach attempts in a multi-tenant protection storage deployment
CN108399050A (en) * 2018-02-02 2018-08-14 新华三技术有限公司 A kind of data processing method and device
CN108763473A (en) * 2018-05-29 2018-11-06 郑州云海信息技术有限公司 A kind of the native object storage method and device of distributed storage
CN108959500A (en) * 2018-06-26 2018-12-07 郑州云海信息技术有限公司 A kind of object storage method, device, equipment and computer readable storage medium
CN111241036A (en) * 2020-01-21 2020-06-05 苏州浪潮智能科技有限公司 Asynchronous IO data processing method, device, equipment and medium

Also Published As

Publication number Publication date
CN111241036A (en) 2020-06-05
CN111241036B (en) 2023-01-10

Similar Documents

Publication Publication Date Title
US11086962B2 (en) Webpage loading method, client and server
US9715507B2 (en) Techniques for reconciling metadata and data in a cloud storage system without service interruption
AU2010221620B2 (en) Content rendering on a computer
US6754799B2 (en) System and method for indexing and retrieving cached objects
US20190146681A1 (en) Data writing and reading method and apparatus, and distributed object storage cluster
US9298625B2 (en) Read and write requests to partially cached files
US11687488B2 (en) Directory deletion method and apparatus, and storage server
US9881015B2 (en) Method and system for previewing file information
US8620990B2 (en) Parsing contents of an e-Form
KR19990064246A (en) A system and method for positioning a page on the World Wide Web and positioning a document from a network computer
US10572544B1 (en) Method and system for document similarity analysis
US20190146882A1 (en) Journal-less recovery for nested crash-consistent storage systems
CN103150395B (en) Directory path analysis method of solid state drive (SSD)-based file system
WO2019076102A1 (en) Data rollback method and system, device, and computer readable storage medium
US10489346B2 (en) Atomic update of B-tree in a persistent memory-based file system
US20180349422A1 (en) Database management system, database server, and database management method
WO2021147360A1 (en) Method, apparatus and device for processing asynchronous io data, and medium
US11853284B2 (en) In-place updates with concurrent reads in a decomposed state
WO2015057463A1 (en) File handlers supporting dynamic data streams
CN111290700A (en) Distributed data reading and writing method and system
US20150286672A1 (en) Information processing method, information processing device, and medium
JP4219125B2 (en) Full-text search device, full-text search method, program, and recording medium
US10795875B2 (en) Data storing method using multi-version based data structure
US10706012B2 (en) File creation
WO2021017655A1 (en) Method, apparatus, and computing device for obtaining inode number, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20915191

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20915191

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20915191

Country of ref document: EP

Kind code of ref document: A1