WO2016086819A1 - Method and apparatus for writing data into shingled magnetic record smr hard disk - Google Patents

Method and apparatus for writing data into shingled magnetic record smr hard disk Download PDF

Info

Publication number
WO2016086819A1
WO2016086819A1 PCT/CN2015/095997 CN2015095997W WO2016086819A1 WO 2016086819 A1 WO2016086819 A1 WO 2016086819A1 CN 2015095997 W CN2015095997 W CN 2015095997W WO 2016086819 A1 WO2016086819 A1 WO 2016086819A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
chunk
record
hard disk
mark
Prior art date
Application number
PCT/CN2015/095997
Other languages
French (fr)
Chinese (zh)
Inventor
张雷
肖垒
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2016086819A1 publication Critical patent/WO2016086819A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647Migration mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023Free address space management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A method and apparatus for writing data into a shingled magnetic record SMR hard disk relate to the technical field of computers. The method for writing data into a shingled magnetic record SMR hard disk comprises: receiving a write data command, wherein the write data comprises a key, data and a flag, the key is used for identifying the data, the flag is a write flag, a deletion flag or a modification flag (201); according to the length of a data record, applying to chunk which is being used for a space, wherein the data record comprises the key, data and flag in the write data command (202); writing the data record into the applied space (203); and saving addresses in the chunk which is being used where the key, the flag and the data are written (204). By writing the data record comprising the key, the data and the flag into the chunk which is being used, the method and apparatus mentioned above solve the problem that the layout scheme of a distributed storage system cannot be directly transplanted onto the SMR hard disk; the effect of improving the flexibility in writing data into an SMR hard disk is realized.

Description

将数据写入叠瓦状磁记录SMR硬盘的方法及装置Method and device for writing data into shingled magnetic recording SMR hard disk
本申请要求于2014年12月5日提交中国专利局、申请号为201410736477.6、发明名称为“将数据写入叠瓦状磁记录SMR硬盘的方法及装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority to Chinese Patent Application No. 201410736477.6, entitled "Method and Apparatus for Writing Data into a Stacked Magnetic Recording SMR Hard Disk", on December 5, 2014, all of which is filed on December 5, 2014. The content is incorporated herein by reference.
技术领域Technical field
本发明涉及计算机技术领域,特别涉及一种将数据写入叠瓦状磁记录SMR硬盘的方法及装置。The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for writing data into a stacked magnetic recording SMR hard disk.
背景技术Background technique
SMR(Shingled Magnetic Record,叠瓦状磁记录)技术可以通过对现有硬盘磁道进行部分重叠覆盖来减小磁道宽度,从而提高记录密度。SMR (Shingled Magnetic Record) technology can reduce the track width by partially overlapping the existing hard disk tracks, thereby increasing the recording density.
分布式Key-Value存储系统具有查询速度快、存放数据量大、支持高并发等优点,非常适合通过主键进行查询。The distributed Key-Value storage system has the advantages of fast query speed, large amount of data storage, and high concurrency support. It is very suitable for querying through the primary key.
在实现本发明的过程中,发明人发现现有技术至少存在以下问题:由于SMR硬盘的磁道很窄,在将数据写入某一个磁道中时将会对其相邻一定数量的磁道中的数据产生写覆盖,从而导致相邻磁道中的数据失效。因此,不能直接将Key-Value存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低。In the process of implementing the present invention, the inventors have found that the prior art has at least the following problem: since the track of the SMR hard disk is very narrow, data in a certain number of tracks adjacent thereto will be written when data is written into a certain track. Write coverage is generated, causing data in adjacent tracks to fail. Therefore, the Key-Value storage system layout scheme cannot be directly transplanted to the SMR hard disk, resulting in low write performance of the SMR hard disk.
发明内容Summary of the invention
为了解决不能直接将Key-Value存储系统布局方案移植到SMR硬盘上的问题,本发明实施例提供了一种将数据写入叠瓦状磁记录SMR硬盘的方法及装置。所述技术方案如下:In order to solve the problem that the Key-Value storage system layout solution cannot be directly transplanted to the SMR hard disk, the embodiment of the present invention provides a method and device for writing data into the stacked magnetic recording SMR hard disk. The technical solution is as follows:
第一方面,提供了一种将数据写入叠瓦状磁记录SMR硬盘的方法,所述SMR硬盘包括多个根据磁轨带band虚拟化后得到的chunk,所述方法包括:In a first aspect, a method for writing data to a shingled magnetic recording SMR hard disk is provided. The SMR hard disk includes a plurality of chunks obtained by virtualizing a track band, the method comprising:
接收写数据指令,所述写数据指令包括键、数据和标记,所述键用于标识所述数据,所述标记为写标记、删除标记或修改标记,所述写标记用于表示将所述数据写入所述SMR硬盘中,所述删除标记用于表示删除所述SMR硬盘中 具有所述键的数据,所述修改标记用于表示利用所述数据修改具有相同键且已被写入所述SMR硬盘的数据;Receiving a write data instruction, the write data instruction including a key, a data, and a mark, the key being used to identify the data, the mark being a write mark, a delete mark, or a modify mark, the write mark being used to indicate that the Data is written into the SMR hard disk, and the delete flag is used to delete the SMR hard disk. Data having the key, the modified mark being used to indicate that data having the same key and having been written to the SMR hard disk is modified with the data;
根据数据记录的长度向正在使用的chunk申请空间,所述数据记录包括所述写数据指令中的所述键、所述数据和所述标记;Requiring a space to the chunk being used according to the length of the data record, the data record including the key, the data and the token in the write data instruction;
将所述数据记录写入申请的所述空间中;Writing the data record into the space of the application;
保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址。The address is saved, the tag and the data are written to the address in the chunk being used.
在第一方面的第一种可能的实施方式中,所述正在使用的chunk中包含用于存储数据记录的数据区,所述根据数据记录的长度向正在使用的chunk申请空间,包括:In a first possible implementation manner of the first aspect, the chunk being used includes a data area for storing a data record, and the space is applied to the chunk being used according to the length of the data record, including:
根据所述数据记录的长度向所述正在使用的chunk中的所述数据区申请空间。A space is requested from the data area in the chunk being used according to the length of the data record.
结合第一方面或者第一方面的第一种可能的实施方式,在第二种可能的实施方式中,所述正在使用的chunk中包含用于存储索引记录的索引区,所述保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址,包括:With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation, the chunk being used includes an index area for storing an index record, where the key is saved The tag and the data are written to the address in the chunk being used, including:
将所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址组成索引记录;Forming an index record by the key, the mark, and an address in which the data is written into the chunk being used;
将所述索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在所述正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区。Saving the index record to a first storage area in memory corresponding to the chunk being used; or, when the remaining space in the chunk being used is insufficient to allocate data to be written in the write data instruction, The index record in the first storage area in the memory is stored in the index area in the chunk being used.
结合第一方面、第一方面的第一种可能的实施方式或者第一方面的第二种可能的实施方式,在第三种可能的实施方式中,所述将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区之后,还包括:With reference to the first aspect, the first possible implementation of the first aspect, or the second possible implementation of the first aspect, in a third possible implementation, the After the index record in a storage area is stored in the index area in the chunk being used, the method further includes:
将所述正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;Modify the attribute of the chunk being used to used, and apply for an unused chunk as the used chunk for the next write data;
将所述第一存储区域内的所述索引记录存储至所述内存中与已使用的chunk对应的第二存储区域;And storing the index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
清除所述第一存储区域内的所述索引记录。Clearing the index record in the first storage area.
结合第一方面、第一方面的第一种可能的实施方式、第一方面的第二种可 能的实施方式或者第一方面的第三种可能的实施方式,在第四种可能的实施方式中,所述方法还包括:Combining the first aspect, the first possible implementation manner of the first aspect, the second In a fourth possible implementation manner, the method further includes:
在重启所述硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;When the hard disk is restarted, the used chunk and the chunk being used are determined according to the attributes of the chunk;
将所述已使用的chunk中存储的索引记录,写入所述内存中与已使用的chunk对应的第二存储区域;And storing an index record stored in the used chunk into a second storage area in the memory corresponding to the used chunk;
根据所述正在使用的chunk中存储的数据记录,重建与所述数据记录对应的索引记录,并将重建得到的所述索引记录写入所述内存中与正在使用的chunk对应的第一存储区域。Reconstructing an index record corresponding to the data record according to the data record stored in the chunk being used, and writing the reconstructed index record into the first storage area in the memory corresponding to the chunk being used .
结合第一方面、第一方面的第一种可能的实施方式至第一方面的第四种可能的实施方式中的任一种实施方式,在第五种可能的实施方式中,所述重建得到的所述索引记录包括:所述数据记录中数据的键、与所述数据对应的标记以及所述数据被写入所述正在使用的chunk中的地址。With reference to the first aspect, the first possible implementation of the first aspect, to any one of the fourth possible implementation manners of the first aspect, in the fifth possible implementation, the reconstruction is obtained The index record includes: a key of data in the data record, a tag corresponding to the data, and an address in which the data is written in the chunk being used.
第二方面,提供了一种将数据写入叠瓦状磁记录SMR硬盘的装置,所述SMR硬盘包括多个根据磁轨带band虚拟化后得到的chunk,所述装置包括:In a second aspect, there is provided an apparatus for writing data to a shingled magnetic recording SMR hard disk, the SMR hard disk comprising a plurality of chunks obtained by virtualizing a track band, the device comprising:
接收模块,用于接收写数据指令,所述写数据指令包括键、数据和标记,所述键用于标识所述数据,所述标记为写标记、删除标记或修改标记,所述写标记用于表示将所述数据写入所述SMR硬盘中,所述删除标记用于表示删除所述SMR硬盘中具有所述键的数据,所述修改标记用于表示利用所述数据修改具有相同键且已被写入所述SMR硬盘的数据;a receiving module, configured to receive a write data instruction, where the write data instruction includes a key, a data, and a mark, wherein the key is used to identify the data, and the mark is a write mark, a delete mark, or a modify mark, and the write mark is used by And indicating that the data is written into the SMR hard disk, the deletion mark is used to indicate deleting data having the key in the SMR hard disk, and the modification mark is used to indicate that the same key is modified by using the data and Data that has been written to the SMR hard disk;
申请模块,用于根据数据记录的长度向正在使用的chunk申请空间,所述数据记录包括所述写数据指令中的所述键、所述数据和所述标记;An application module, configured to apply for a space to a chunk being used according to a length of the data record, where the data record includes the key, the data, and the mark in the write data instruction;
第一写入模块,用于将所述数据记录写入申请的所述空间中;a first writing module, configured to write the data record into the space of the application;
保存模块,用于保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址。And a saving module, configured to save the key, the mark, and the address in which the data is written in the chunk being used.
在第二方面的第一种可能的实施方式中,所述正在使用的chunk中包含用于存储数据记录的数据区,所述申请模块用于根据所述数据记录的长度向所述正在使用的chunk中的所述数据区申请空间。In a first possible implementation manner of the second aspect, the chunk being used includes a data area for storing a data record, and the application module is configured to use the length according to the length of the data record. The data area application space in the chunk.
结合第二方面或者第二方面的第一种可能的实施方式,在第二种可能的实施方式中,所述正在使用的chunk中包含用于存储索引记录的索引区,所述保 存模块,包括:With reference to the second aspect, or the first possible implementation manner of the second aspect, in a second possible implementation, the chunk being used includes an index area for storing an index record, where Storage module, including:
组成单元,用于将所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址组成索引记录;a component unit, configured to form an index record of the key, the mark, and an address in which the data is written into the chunk being used;
保存单元,用于将所述索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在所述正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区。a saving unit, configured to save the index record to a first storage area in the memory corresponding to the chunk being used; or, the remaining space in the chunk being used is insufficient to allocate a write data instruction to be written And storing, in the data, the index record in the first storage area in the memory to the index area in the chunk being used.
结合第二方面、第二方面的第一种可能的实施方式或者第二方面的第二种可能的实施方式,在第三种可能的实施方式中,所述装置还包括:With reference to the second aspect, the first possible implementation of the second aspect, or the second possible implementation of the second aspect, in a third possible implementation, the apparatus further includes:
修改模块,用于将所述正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;a modifying module, configured to modify the attribute of the used chunk to be used, and apply an unused chunk as a used chunk for writing data next time;
存储模块,用于将所述第一存储区域内的所述索引记录存储至所述内存中与已使用的chunk对应的第二存储区域;a storage module, configured to store the index record in the first storage area to a second storage area in the memory corresponding to the used chunk;
清除模块,用于清除所述第一存储区域内的所述索引记录。And a clearing module, configured to clear the index record in the first storage area.
结合第二方面、第二方面的第一种可能的实施方式、第二方面的第二种可能的实施方式或者第二方面的第三种可能的实施方式,在第四种可能的实施方式中,所述装置还包括:With reference to the second aspect, the first possible implementation of the second aspect, the second possible implementation of the second aspect, or the third possible implementation of the second aspect, in a fourth possible implementation The device further includes:
确定模块,用于在重启所述硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;a determining module, configured to determine, according to the attribute of the chunk, the used chunk and the chunk being used when the hard disk is restarted;
第二写入模块,用于将所述已使用的chunk中存储的索引记录,写入所述内存中与已使用的chunk对应的第二存储区域;a second writing module, configured to write an index record stored in the used chunk to a second storage area in the memory corresponding to the used chunk;
重建模块,用于根据所述正在使用的chunk中存储的数据记录,重建与所述数据记录对应的索引记录,并将重建得到的所述索引记录写入所述内存中与正在使用的chunk对应的第一存储区域。a reconstruction module, configured to reconstruct an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the memory to correspond to a chunk being used The first storage area.
结合第二方面、第二方面的第一种可能的实施方式至第二方面的第四种可能的实施方式中的任一种实施方式,在第五种可能的实施方式中,所述重建得到的所述索引记录包括:所述数据记录中数据的键、与所述数据对应的标记以及所述数据被写入所述正在使用的chunk中的地址。With reference to the second aspect, the first possible implementation manner of the second aspect, to any one of the fourth possible implementation manners of the second aspect, in the fifth possible implementation manner, the reconstruction is obtained The index record includes: a key of data in the data record, a tag corresponding to the data, and an address in which the data is written in the chunk being used.
第三方面,提供了一种将数据写入叠瓦状磁记录SMR硬盘的装置,应用于包含有所述SMR硬盘的设备中,所述SMR硬盘包括多个根据磁轨带band虚拟 化后得到的chunk,所述装置包括:接收机和处理器,其中处理器与接收机耦合;In a third aspect, an apparatus for writing data into a stacked magnetic recording SMR hard disk is provided, which is applied to a device including the SMR hard disk, wherein the SMR hard disk includes a plurality of virtual bands according to a track band The resulting chunk, the apparatus comprising: a receiver and a processor, wherein the processor is coupled to the receiver;
所述接收机,用于接收写数据指令,所述写数据指令包括键、数据和标记,所述键用于标识所述数据,所述标记为写标记、删除标记或修改标记,所述写标记用于表示将所述数据写入所述SMR硬盘中,所述删除标记用于表示删除所述SMR硬盘中具有所述键的数据,所述修改标记用于表示利用所述数据修改具有相同键且已被写入所述SMR硬盘的数据;The receiver is configured to receive a write data instruction, the write data instruction includes a key, a data and a mark, the key is used to identify the data, and the mark is a write mark, a delete mark or a modify mark, and the write Marking is used to indicate that the data is written into the SMR hard disk, and the delete flag is used to delete data having the key in the SMR hard disk, and the modified mark is used to indicate that the data modification has the same Key and data that has been written to the SMR hard disk;
所述处理器,用于根据数据记录的长度向正在使用的chunk申请空间,所述数据记录包括所述写数据指令中的所述键、所述数据和所述标记;The processor, configured to apply for a space to a chunk being used according to a length of the data record, where the data record includes the key, the data, and the mark in the write data instruction;
所述处理器,还用于将所述数据记录写入申请的所述空间中;The processor is further configured to write the data record into the space of the application;
所述处理器,还用于保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址。The processor is further configured to save the key, the mark, and an address in which the data is written into the chunk being used.
在第三方面的第一种可能的实施方式中,所述正在使用的chunk中包含用于存储数据记录的数据区,所述处理器还用于根据所述数据记录的长度向所述正在使用的chunk中的所述数据区申请空间。In a first possible implementation manner of the third aspect, the chunk being used includes a data area for storing a data record, and the processor is further configured to use the length according to the length of the data record The data area application space in the chunk.
结合第三方面或者第三方面的第一种可能的实施方式,在第二种可能的实施方式中,所述正在使用的chunk中包含用于存储索引记录的索引区,所述处理器,还用于将所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址组成索引记录;With reference to the third aspect, or the first possible implementation manner of the third aspect, in a second possible implementation, the chunk being used includes an index area for storing an index record, and the processor further Forming an index record for the address, the mark, and an address in which the data is written into the chunk being used;
所述处理器,还用于将所述索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在所述正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区。The processor is further configured to save the index record to a first storage area in a memory corresponding to a chunk being used; or, the remaining space in the chunk being used is insufficient to allocate a write data instruction When the data is written, the index record in the first storage area in the memory is stored to the index area in the chunk being used.
结合第三方面、第三方面的第一种可能的实施方式或者第三方面的第二种可能的实施方式,在第三种可能的实施方式中,所述处理器,还用于将所述正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;With reference to the third aspect, the first possible implementation manner of the third aspect, or the second possible implementation manner of the third aspect, in a third possible implementation, the processor is further configured to: The attribute of the chunk being used is modified to be used, and an unused chunk is requested as the used chunk for the next write of the data;
所述处理器,还用于将所述第一存储区域内的所述索引记录存储至所述内存中与已使用的chunk对应的第二存储区域;The processor is further configured to store the index record in the first storage area to a second storage area in the memory corresponding to a used chunk;
所述处理器,还用于清除所述第一存储区域内的所述索引记录。The processor is further configured to clear the index record in the first storage area.
结合第三方面、第三方面的第一种可能的实施方式、第三方面的第二种可 能的实施方式或者第三方面的第三种可能的实施方式,在第四种可能的实施方式中,所述处理器,还用于在重启所述硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;Combining the third aspect, the first possible implementation manner of the third aspect, and the second A possible implementation or a third possible implementation of the third aspect. In a fourth possible implementation, the processor is further configured to determine, according to an attribute of the chunk, the used Chunk and the chunk being used;
所述处理器,还用于将所述已使用的chunk中存储的索引记录,写入所述内存中与已使用的chunk对应的第二存储区域;The processor is further configured to write an index record stored in the used chunk to a second storage area in the memory corresponding to the used chunk;
所述处理器,还用于根据所述正在使用的chunk中存储的数据记录,重建与所述数据记录对应的索引记录,并将重建得到的所述索引记录写入所述内存中与正在使用的chunk对应的第一存储区域。The processor is further configured to reconstruct an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the memory and in use The chunk corresponds to the first storage area.
结合第三方面、第三方面的第一种可能的实施方式至第三方面的第四种可能的实施方式中的任一种实施方式,在第五种可能的实施方式中,所述重建得到的所述索引记录包括:所述数据记录中数据的键、与所述数据对应的标记以及所述数据被写入所述正在使用的chunk中的地址。With reference to the third aspect, the first possible implementation manner of the third aspect, or any one of the fourth possible implementation manners of the third aspect, in the fifth possible implementation manner, the reconstruction is obtained The index record includes: a key of data in the data record, a tag corresponding to the data, and an address in which the data is written in the chunk being used.
本发明实施例提供的技术方案带来的有益效果是:The beneficial effects brought by the technical solutions provided by the embodiments of the present invention are:
通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。By writing data records containing keys, data, and tags into the chunks being used; since the data records generated according to the write data instructions contain keys and data, it is solved that the distributed storage system layout scheme cannot be directly transplanted to On the SMR hard disk, the write performance of the SMR hard disk is low. Since the chunk is obtained according to the band virtualization, data can be randomly written between the chunks, thereby achieving the flexibility of writing data on the SMR hard disk. effect.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.
图1是本发明一实施例中提供的SMR硬盘的空间布局示意图;1 is a schematic diagram showing the spatial layout of an SMR hard disk provided in an embodiment of the present invention;
图2是本发明一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图;2 is a flow chart of a method for writing data into a stacked magnetic recording SMR hard disk according to an embodiment of the present invention;
图3A是本发明另一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图;3A is a flow chart of a method for writing data into a stacked magnetic recording SMR hard disk according to another embodiment of the present invention;
图3B是本发明一实施例中提供的chunk的切换过程的示意图; FIG. 3B is a schematic diagram of a process of switching a chunk provided in an embodiment of the present invention; FIG.
图4是本发明一实施例中提供的垃圾清理过程的示意图;4 is a schematic diagram of a garbage cleaning process provided in an embodiment of the present invention;
图5是本发明再一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图;5 is a flow chart of a method for writing data into a stacked magnetic recording SMR hard disk according to still another embodiment of the present invention;
图6是本发明一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的装置的结构方框图;6 is a block diagram showing the structure of an apparatus for writing data into a stacked magnetic recording SMR hard disk according to an embodiment of the present invention;
图7是本发明另一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的装置的结构方框图;7 is a block diagram showing the structure of an apparatus for writing data into a stacked magnetic recording SMR hard disk according to another embodiment of the present invention;
图8是本发明一实施例中提供的包含有SMR硬盘的设备的结构方框图;FIG. 8 is a block diagram showing the structure of an apparatus including an SMR hard disk according to an embodiment of the present invention; FIG.
图9是本发明另一实施例中提供的包含有SMR硬盘的设备的结构方框图。FIG. 9 is a block diagram showing the structure of an apparatus including an SMR hard disk according to another embodiment of the present invention.
具体实施方式detailed description
为使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明实施方式作进一步地详细描述。The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
请参考图1,其示出了本发明一实施例中提供的SMR硬盘的空间布局示意图,该SMR硬盘在逻辑上可以划分为系统元数据区110和记录区120。Please refer to FIG. 1 , which illustrates a spatial layout of an SMR hard disk provided in an embodiment of the present invention. The SMR hard disk may be logically divided into a system metadata area 110 and a recording area 120 .
系统元数据区110包括至少一个超级记录块130,超级记录块130包括至少一个超级块140,超级块140用于记录记录区120中各个chunk150的属性和使用信息。The system metadata area 110 includes at least one super record block 130, and the super record block 130 includes at least one super block 140 for recording attributes and usage information of each chunk 150 in the record area 120.
由于超级块140需要频繁地更新记录区120中各个chunk150的属性和使用信息,如果固定使用一个超级块140进行记录,可能导致超级块140所在磁盘空间故障。因此,需要多个超级块140来记录记录区120中各个chunk150的属性和使用信息。超级块140的大小可以根据记录区120中信息的大小来确定,超级块140的个数可以根据超级块140能被重复写入的次数来确定,且超级块140能被重复写入的次数越多,超级块140的个数可以设置得越少。而超级记录块130的个数则可以根据超级块140的个数以及大小来确定。在实际应用场景中,超级记录块130中的超级块140的个数,以及超级块140中包含的信息可以根据实际使用时的情况进行变化。Since the super block 140 needs to frequently update the attributes and usage information of each chunk 150 in the recording area 120, if a super block 140 is used for recording, the disk space of the super block 140 may be faulty. Therefore, a plurality of super blocks 140 are required to record the attributes and usage information of each chunk 150 in the recording area 120. The size of the super block 140 can be determined according to the size of the information in the recording area 120. The number of the super blocks 140 can be determined according to the number of times the super block 140 can be repeatedly written, and the more the super block 140 can be repeatedly written. More, the number of super blocks 140 can be set smaller. The number of super record blocks 130 can be determined according to the number and size of the super blocks 140. In the actual application scenario, the number of super blocks 140 in the super record block 130 and the information contained in the super block 140 may be changed according to the actual use.
可选的,超级记录块130中的超级块140还可以包括第一全局变量,该第一全局变量用于标识各个超级块140的先后使用顺序。举例来讲,可以在每申 请一个超级块140后,将第一全局变量加1后作为申请到的超级块140的第一全局变量。比如,当前,在申请到超级块140后,查找到前一次申请的超级块140的第一全局变量为100,则将该第一全局变量加1得到新的第一全局变量101,将该第一全局变量101作为当前超级块140的第一全局变量。需要说明的是,第一全局变量每次增加的数值也可以是其它值,比如2,3等,只需要保证在后申请的超级块140的第一全局变量大于在前申请的超级块140的第一全局变量。Optionally, the super block 140 in the super record block 130 may further include a first global variable, where the first global variable is used to identify the sequential use order of each super block 140. For example, you can apply for each After a super block 140, the first global variable is incremented by 1 as the first global variable of the applied super block 140. For example, currently, after applying to the super block 140, if the first global variable of the super application 140 of the previous application is found to be 100, the first global variable is incremented by 1 to obtain a new first global variable 101. A global variable 101 is taken as the first global variable of the current super block 140. It should be noted that the value added by the first global variable may also be other values, such as 2, 3, etc., only need to ensure that the first global variable of the super-block 140 applied later is greater than that of the super-block 140 of the previous application. The first global variable.
记录区120包括至少一个chunk150,这里的chunk150是根据SMR硬盘中的磁轨带band虚拟化后得到的。根据chunk150的使用情况,可以将chunk150分为已使用的chunk、正在使用的chunk和未使用的chunk。记录区120用于存储数据记录以及与数据记录对应的索引记录。The recording area 120 includes at least one chunk 150, and the chunk 150 here is obtained by virtualizing the track band in the SMR hard disk. According to the usage of chunk150, chunk 150 can be divided into used chunks, chunks in use, and unused chunks. The recording area 120 is for storing data records and index records corresponding to the data records.
需要说明的是,记录区120中chunk150的种类可以根据实际使用时的情况进行变化,可选的,记录区120中还可以包含其它种类的chunk。It should be noted that the type of the chunk 150 in the recording area 120 may be changed according to the actual use. Optionally, the recording area 120 may further include other types of chunks.
可选的,记录区120中的chunk150包括数据区160和索引区170。Optionally, the chunk 150 in the recording area 120 includes a data area 160 and an index area 170.
数据区160可以用于存储至少一条数据记录,每条数据记录至少包括键、标记和数据中的前两种或全部。The data area 160 can be used to store at least one data record, each data record including at least the first two or all of the keys, tags, and data.
数据记录中包含的信息种类可以根据数据记录的标记类型来确定。比如,当数据记录的标记是删除标记时,该删除记录内可以只包括键和标记;当数据记录的标记类型是写标记或修改标记时,该写入记录或修改记录内可以包括键、标记和数据。The type of information contained in the data record can be determined based on the type of tag of the data record. For example, when the mark of the data record is a delete mark, the delete record may include only the key and the mark; when the mark type of the data record is a write mark or a modified mark, the write record or the modified record may include a key and a mark. And data.
可选的,数据记录还可以包括其他信息,比如,校验信息、键长度信息和数据长度信息等,本实施例对数据记录中包含的其他信息不作具体限定。Optionally, the data record may further include other information, such as the check information, the key length information, and the data length information. The other information included in the data record is not specifically limited in this embodiment.
可选的,数据记录还可以包括第二全局变量,该第二全局变量用于标识各条数据记录的被写入的先后顺序。举例来讲,在每写一条数据记录时,可以将前一次写入的数据记录的第二全局变量加1后作为当前写入的数据记录的第二全局变量。比如,在写入当前的数据记录时,查找到前一次写入的数据记录的第二全局变量为200,则将该第二全局变量加1得到新的第二全局变量201,将该第二全局变量201作为当前数据记录的第二全局变量。需要说明的是,第二全局变量每次增加的数值也可以是其它值,比如2,3等,只需要保证在后写入的数据记录的第二全局变量大于在前写入的数据记录的第二全局变量。 Optionally, the data record may further include a second global variable, where the second global variable is used to identify a sequence in which each data record is written. For example, each time a data record is written, the second global variable of the previously written data record can be incremented by one as the second global variable of the currently written data record. For example, when writing the current data record, if the second global variable of the data record written last time is 200, the second global variable is incremented by 1 to obtain a new second global variable 201, and the second The global variable 201 acts as the second global variable of the current data record. It should be noted that the value added by the second global variable may also be other values, such as 2, 3, etc., only need to ensure that the second global variable of the data record written later is greater than the data record of the previous write. The second global variable.
需要说明的是,数据记录中的键、标记和数据的相对位置可以根据实际使用时的情况进行变化。It should be noted that the relative positions of the keys, the markers, and the data in the data record may be changed according to the actual use.
索引区170用于存储与数据记录一一对应的索引记录,与数据记录对应的索引记录至少包括与数据记录中相同的键、标记以及数据记录在SMR硬盘上的地址。The index area 170 is configured to store index records corresponding to the data records one by one, and the index records corresponding to the data records include at least the same keys, tags, and addresses recorded in the data records on the SMR hard disk.
数据区160中的数据记录与索引区170中的索引记录是一一对应关系,举例来讲,如图1所示,数据记录1对应索引记录1,数据记录2对应索引记录2。需要说明的是,图1中数据记录1、数据记录2、索引记录1和索引记录2的相对位置仅是示意,本实施例对此不作限定。The data records in the data area 160 have a one-to-one correspondence with the index records in the index area 170. For example, as shown in FIG. 1, the data record 1 corresponds to the index record 1, and the data record 2 corresponds to the index record 2. It should be noted that the relative positions of the data record 1, the data record 2, the index record 1, and the index record 2 in FIG. 1 are only schematic, which is not limited in this embodiment.
可选的,索引记录还可以包括与数据记录对应的第二全局变量。且索引记录中的键、标记以及地址的相对位置可以根据实际使用时的情况进行变化。Optionally, the index record may further include a second global variable corresponding to the data record. The relative positions of the keys, tags, and addresses in the index record can be changed according to the actual use.
需要说明的是,系统元数据区110和记录区120在SMR硬盘上的相对位置可以根据实际使用时的情况进行变化。It should be noted that the relative positions of the system metadata area 110 and the recording area 120 on the SMR hard disk may be changed according to the actual use.
综上所述,本实施例提供的SMR硬盘,通过将SMR硬盘划分成系统元数据区和记录区,记录区用于存储数据记录以及与数据记录对应的索引记录,系统元数据区用于记录记录区中各个chunk的属性和使用信息;由于为SMR硬盘中每条数据记录都建立了索引记录,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the SMR hard disk provided in this embodiment divides the SMR hard disk into a system metadata area and a recording area, and the recording area is used for storing data records and index records corresponding to the data records, and the system metadata area is used for recording. The attribute and usage information of each chunk in the recording area; since the index record is established for each data record in the SMR hard disk, the solution of the distributed storage system layout cannot be directly transplanted to the SMR hard disk, resulting in the writing of the SMR hard disk. The problem of lower performance is achieved; since the chunk is obtained according to the band virtualization, data can be randomly written between the chunks, thereby achieving the effect of improving the flexibility of writing data on the SMR hard disk.
请参考图2,其示出了本发明一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图,该将数据写入叠瓦状磁记录SMR硬盘的方法可以用于包含有SMR硬盘的设备中,该将数据写入叠瓦状磁记录SMR硬盘的方法,可以包括:Please refer to FIG. 2, which is a flowchart of a method for writing data into a stacked magnetic recording SMR hard disk according to an embodiment of the present invention. The method for writing data into a stacked magnetic recording SMR hard disk can be used for In a device including an SMR hard disk, the method for writing data into a stacked magnetic recording SMR hard disk may include:
步骤201,接收写数据指令,写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将该数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有该键的数据,修改标记用于表示利用该数据修改具有相同键且已被写入SMR硬盘的数据;Step 201: Receive a write data instruction, where the write data instruction includes a key, a data, and a mark, the key is used to identify the data, and the mark is a write mark, a delete mark, or a modify mark, and the write mark is used to indicate that the data is written into the SMR hard disk, and is deleted. The mark is used to indicate that the data having the key in the SMR hard disk is deleted, and the modification mark is used to indicate that the data having the same key and written to the SMR hard disk is modified by using the data;
步骤202,根据数据记录的长度向正在使用的chunk申请空间,数据记录包 括写数据指令中的键、数据和标记;Step 202: Apply space to the chunk being used according to the length of the data record, and record the data packet. Including the keys, data, and tags in the data instructions;
步骤203,将数据记录写入申请的空间中; Step 203, writing a data record into the space of the application;
步骤204,保存键、标记与数据被写入正在使用的chunk中的地址。In step 204, the save key, the tag, and the data are written to the address in the chunk being used.
综上所述,本实施例提供的将数据写入叠瓦状磁记录SMR硬盘的方法,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the method for writing data into the shingled magnetic recording SMR hard disk provided by the embodiment is to write the data record containing the key, the data and the mark into the chunk being used; since the data is generated according to the write data instruction. The data record contains keys and data. Therefore, it solves the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization, Data can be randomly written between chunks, thus achieving the flexibility of writing data on the SMR hard disk.
在实际使用时,为了便于读取SMR硬盘中存入的数据,SMR所在的设备需要对chunk的索引记录进行持久化,具体请参考图3A,其示出了本发明另一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图,该将数据写入叠瓦状磁记录SMR硬盘的方法可以用于包含有SMR硬盘的设备中,该将数据写入叠瓦状磁记录SMR硬盘的方法,可以包括:In actual use, in order to facilitate the reading of the data stored in the SMR hard disk, the device in which the SMR is located needs to persist the index record of the chunk. For details, please refer to FIG. 3A, which shows another embodiment of the present invention. A flow chart of a method of writing data into a stacked magnetic recording SMR hard disk, the method of writing data into a stacked magnetic recording SMR hard disk can be used in a device including an SMR hard disk, which writes data into a stacked magnetic field The method of recording the SMR hard disk may include:
步骤301,接收写数据指令;Step 301: Receive a write data instruction.
这里的写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将该数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有该键的数据,修改标记用于表示利用该数据修改具有相同键且已被写入SMR硬盘的数据。The write data instructions here include keys, data, and tags. The keys are used to identify data, marked as write marks, delete marks, or modified marks. Write marks are used to indicate that the data is written to the SMR hard disk, and the delete mark is used to indicate that the SMR is deleted. The data in the hard disk has the key, and the modification flag is used to indicate that the data having the same key and written to the SMR hard disk is modified by the data.
步骤302,根据数据记录的长度向正在使用的chunk中的数据区申请空间;Step 302: Apply space to the data area in the chunk being used according to the length of the data record;
数据记录包括写数据指令中的键、数据和标记。Data records include keys, data, and tags in a write data instruction.
可选的,在步骤302之前,通常还可以包括根据写数据指令生成数据记录。Optionally, before step 302, it may also generally include generating a data record according to the write data instruction.
SMR所在的设备在接收到写数据指令后,可以根据该写数据指令按照数据记录的格式生成一条数据记录。当写数据指令中不包括数据或包含的数据为空时,数据记录可以只包括键和标记,比如该写数据指令的标记为删除标记时,该写数据指令通常不包括数据。After receiving the write data command, the device where the SMR is located can generate a data record according to the data record format according to the write data command. When the data command does not include data or the included data is empty, the data record may include only keys and tags. For example, when the write data instruction is marked as a delete tag, the write data instruction usually does not include data.
SMR所在的设备在接收到写数据指令后,确定该写数据指令的标记类型,根据该写数据指令的标记类型生成相应的数据记录,具体实现可以参见以下三种实现方式。 After receiving the write data command, the device in which the SMR is located determines the type of the write data command, and generates a corresponding data record according to the mark type of the write data command. For the specific implementation, refer to the following three implementation manners.
在一种实现方式中,当写数据指令包括的标记为删除标记时,按照数据记录的格式,将写数据指令中携带的键和标记组合成一条数据记录。In one implementation, when the write data instruction includes a mark as a delete mark, the key and the mark carried in the write data command are combined into one data record according to the format of the data record.
在另一种实现方式中,当写数据指令包括的标记为写标记时,按照数据记录的格式,将写数据指令中携带的数据、键以及标记组合成一条数据记录。In another implementation, when the write data instruction includes a mark as a write mark, the data, keys, and tags carried in the write data command are combined into one data record according to the format of the data record.
在还一种实现方式中,当写数据指令包括的标记为修改标记时,按照数据记录的格式,将写数据指令中携带的数据、键以及标记组合成一条数据记录。In still another implementation, when the write data instruction includes a mark as a modification mark, the data, the key, and the mark carried in the write data command are combined into one data record according to the format of the data record.
也就是说,当写数据指令包括的标记为删除标记时,根据该写数据指令生成的数据记录中通常包含键和标记;而当写数据指令包括的标记为写标记或修改标记时,根据该写数据指令生成的数据记录中通常包含键、标记和数据。That is, when the write data instruction includes a mark as a delete mark, the data record generated according to the write data instruction usually includes a key and a mark; and when the write data instruction includes a mark as a write mark or a modified mark, according to the Data records generated by write data instructions typically contain keys, tags, and data.
SMR所在的设备可以计算数据记录的长度,并通过SMR硬盘中系统元数据区内的超级块来获取记录区中各个chunk的属性,根据各个chunk的属性确定出当前正在使用的chunk,并在内存中查找到与该正在使用的chunk中的数据记录对应的索引记录,根据查找到的索引记录中的最后被写入数据的地址以及最后被写入数据的长度,确定该正在使用的chunk最后被写入的位置。这里确定的最后被写入的位置是指最后被写入数据的末尾位置。The device where the SMR is located can calculate the length of the data record, and obtain the attributes of each chunk in the record area through the super block in the system metadata area of the SMR hard disk, and determine the chunk currently being used according to the attributes of each chunk, and in the memory. Finding an index record corresponding to the data record in the chunk being used, determining, according to the address of the last written data in the found index record and the length of the last written data, determining that the chunk being used is finally The location of the write. The last determined position to be written here refers to the last position at which data is last written.
SMR所在的设备根据生成的数据记录的长度从当前正在使用的chunk中最后被写入的位置开始分配与生成的数据记录的长度相同的空间。The device in which the SMR is located allocates the same space as the length of the generated data record from the last written position in the chunk currently being used, based on the length of the generated data record.
需要说明的是,当正在使用的chunk中的剩余空间小于生成的数据记录的长度时,需要申请一个未使用的chunk来进行空间分配。It should be noted that when the remaining space in the chunk being used is smaller than the length of the generated data record, it is necessary to apply for an unused chunk for space allocation.
步骤303,将数据记录写入申请的空间中; Step 303, writing a data record into the space of the application;
步骤304,将键、标记与数据被写入正在使用的chunk中的地址组成索引记录; Step 304, forming an index record by combining the key, the tag and the address in which the data is written into the chunk being used;
这里的键和标记与数据记录里的键和标记相同,SMR所在的设备将键、标记和数据被写入正在使用的chunk中的地址的组合确定为该数据记录的索引记录。The keys and tags here are the same as the keys and tags in the data record. The device where the SMR is located determines the combination of the keys, tags, and addresses written to the chunks being used as the index record for the data record.
需要说明的是,本实施例对索引记录中的键、标记和地址的相对位置不作限定。It should be noted that the relative position of the key, the mark, and the address in the index record is not limited in this embodiment.
步骤305,将索引记录保存至内存中与正在使用的chunk对应的第一存储区域;Step 305: Save the index record to a first storage area in the memory corresponding to the chunk being used;
这里的第一存储区域用于临时存储当前正在使用的chunk中写入的各条数 据记录所对应的索引记录,这里的内存可以是SMR所在的设备的系统内存。The first storage area here is used to temporarily store the number of pieces written in the chunk currently being used. According to the index record corresponding to the record, the memory here can be the system memory of the device where the SMR is located.
在实际使用时,当检测到正在使用的chunk中的空间被分配完毕后,SMR所在的设备需要对该正在使用的chunk的索引记录进行持久化,并需要申请新的正在使用的chunk以进行后续的数据写入,对应的,还需要申请新的超级块以记录相应记录信息,具体流程请参见下述步骤306至309。In actual use, when it is detected that the space in the chunk being used is allocated, the device where the SMR is located needs to persist the index record of the chunk being used, and needs to apply for a new chunk being used for subsequent The data is written, and correspondingly, a new super block needs to be applied to record the corresponding record information. For the specific process, refer to steps 306 to 309 below.
步骤306,在正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将内存中的第一存储区域内的索引记录存储至正在使用的chunk中的索引区; Step 306, when the remaining space in the chunk being used is insufficient to allocate the data to be written in the write data instruction, storing the index record in the first storage area in the memory to the index area in the chunk being used;
当正在使用的chunk的数据区中剩余的空间不足以存储当前接收到的写数据指令中所对应的数据时,SMR所在的设备则确定该正在使用的chunk的空间被分配完毕。此时,SMR所在的设备将内存中的第一存储区域内的索引记录存储至正在使用的chunk中的索引区。When the remaining space in the data area of the chunk being used is insufficient to store the data corresponding to the currently received write data instruction, the device in which the SMR is located determines that the space of the chunk being used is allocated. At this time, the device where the SMR is located stores the index record in the first storage area in the memory to the index area in the chunk being used.
步骤307,将正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk; Step 307, modify the attribute of the chunk being used to be used, and apply for an unused chunk as the used chunk for the next write data;
SMR所在的设备在检测到正在使用的chunk的空间被分配完毕,并将第一存储区域中存储的各条索引记录写入到该正在使用的chunk的索引区后,该正在使用的chunk不再继续被写入数据记录,因此,此时可以将该正在使用的chunk的属性更改为已使用,进而使得该正在使用的chunk变成已使用的chunk。After the SMR device detects that the space of the chunk being used is allocated, and writes each index record stored in the first storage area to the index area of the chunk being used, the chunk being used is no longer used. The data record continues to be written, so the property of the chunk being used can be changed to used at this time, so that the chunk being used becomes the used chunk.
同时,SMR所在的设备需要在系统元数据区中申请下一个超级块,并在申请到的超级块中记录当前记录区中各个chunk的属性和使用信息。At the same time, the device where the SMR is located needs to apply for the next super block in the system metadata area, and records the attributes and usage information of each chunk in the current recording area in the applied super block.
举例来讲,请参见图3B,其是本发明一实施例中提供的chunk的切换过程的示意图。如图3B所示,SMR所在的设备在检测到正在使用的chunk320的数据区321被分配完毕后,将与该正在使用的chunk320的数据区321的数据记录所对应的索引记录从内存中的第一存储区域存储至该正在使用的chunk320的索引区322中,并将该正在使用的chunk320的属性更改为已使用,进而变成已使用的chunk330。同时,SMR所在的设备申请一块未使用的chunk340,将该未使用的chunk340的属性更改为正在使用,进而变成正在使用的chunk350,并在该正在使用的chunk350的数据区351中开始分配空间。SMR所在的设备在将正在使用的chunk320和未使用的chunk340的属性更改后,申请一块超级块360,并在该新申请的超级块360中记录当前记录区中各个chunk的属性和使用信息。 For example, please refer to FIG. 3B , which is a schematic diagram of a process of switching a chunk provided in an embodiment of the present invention. As shown in FIG. 3B, after detecting that the data area 321 of the chunk 320 being used is allocated, the device in which the SMR is located records the index corresponding to the data record of the data area 321 of the chunk 320 being used from the memory. A storage area is stored in the index area 322 of the chunk 320 being used, and the attribute of the chunk 320 being used is changed to used, and becomes the used chunk 330. At the same time, the device in which the SMR is located requests an unused chunk 340, changes the attribute of the unused chunk 340 to be used, becomes the chunk 350 being used, and starts to allocate space in the data area 351 of the chunk 350 being used. After the SMR is located, the device changes the attributes of the chunk 320 and the unused chunk 340 that are being used, and applies a super block 360, and records the attributes and usage information of each chunk in the current recording area in the newly applied super block 360.
步骤308,将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域; Step 308, storing an index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
这里的第二存储区域用于存储SMR硬盘中所有已使用的chunk中的索引记录。The second storage area here is used to store index records in all used chunks in the SMR hard disk.
为了便于查找SMR硬盘中保存的有效数据,SMR所在的设备在将当前的第一存储区域中存储的各条索引记录写入到正在使用的chunk的索引区后,还需要将这些索引记录写入到内存中的第二存储区域。In order to find the valid data saved in the SMR hard disk, the device where the SMR is located needs to write the index records after the index records stored in the current first storage area are written to the index area of the chunk being used. Go to the second storage area in memory.
在将第一存储区域中的索引记录写入内存中的第二存储区域时,SMR所在的设备需要依次从第一存储区域中读取将要被写入第二存储区域的索引记录,并根据读取到的索引记录中的标记进行相应操作,具体请参见下述三种情况:When the index record in the first storage area is written into the second storage area in the memory, the device in which the SMR is located needs to sequentially read the index record to be written into the second storage area from the first storage area, and according to the read The tag in the obtained index record is operated accordingly. For details, please refer to the following three cases:
第一种情况,当读取的索引记录中的标记为删除标记时,删除第二存储区域中键与读取的索引记录中的键相同的索引记录,并丢弃读取的索引记录;In the first case, when the mark in the read index record is a delete mark, the same index record in the second storage area as the key in the read index record is deleted, and the read index record is discarded;
第二种情况,当读取的索引记录中的标记为修改标记时,删除第二存储区域中键与读取的索引记录中的键相同的索引记录,并将读取的索引记录写入第二存储区域中;In the second case, when the mark in the read index record is a modified mark, the same index record in the second storage area as the key in the read index record is deleted, and the read index record is written into the first In the second storage area;
第三种情况,当读取的索引记录中的标记为写标记时,将读取的索引记录写入第二存储区域中。In the third case, when the mark in the read index record is a write mark, the read index record is written in the second storage area.
需要说明的是,当第一存储区域中的所有索引记录均读取完并保存至第二存储区域之后,第二存储区域中各个索引记录所对应的数据记录均为有效数据。此时,SMR所在的设备可以根据各个chunk中未保存到第二存储区域中的索引记录,确定该chunk中无效数据所占的比例。It should be noted that, after all the index records in the first storage area are read and saved to the second storage area, the data records corresponding to the respective index records in the second storage area are valid data. At this time, the device where the SMR is located may determine the proportion of invalid data in the chunk according to an index record in each chunk that is not saved in the second storage area.
步骤309,清除第一存储区域内的索引记录; Step 309, clearing an index record in the first storage area;
SMR所在的设备在将内存中第一存储区域中的索引记录写入内存中的第二存储区域后,需要清除该第一存储区域内的索引记录,以减少内存的占用率。After the index record in the first storage area of the memory is written to the second storage area in the memory, the index record in the first storage area needs to be cleared to reduce the memory usage.
也就是说,第一存储区域被申请后通常用于存储当前正在使用的chunk中生成的索引记录,如果将第一存储区域中记录的索引记录均持久化到该正在使用的chunk之后,该第一存储区域内的索引记录通常会被清除掉。这样,被申请的第一存储区域通常对应一个正在使用的chunk。That is to say, after the first storage area is applied, it is usually used to store the index record generated in the chunk currently being used, and if the index record recorded in the first storage area is persisted to the chunk that is being used, the first storage area Index records in a storage area are usually cleared. Thus, the first storage area being requested typically corresponds to a chunk being used.
综上所述,本实施例提供的将数据写入叠瓦状磁记录SMR硬盘的方法,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写 数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the method for writing data into the shingled magnetic recording SMR hard disk provided by this embodiment is to write the data record containing the key, the data and the mark into the chunk being used; The data record generated by the data instruction contains keys and data. Therefore, the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk is solved, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization The data can be randomly written between the various chunks, thus achieving the effect of improving the flexibility of writing data on the SMR hard disk.
另外,通过将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;由于将SMR硬盘中已使用的chunk的索引记录持久化到内存中,因此,解决了顺序读取SMR硬盘时的读取效率较低的问题;达到了提高SMR硬盘的读取效率的效果。In addition, the index record in the first storage area is stored in the second storage area corresponding to the used chunk in the memory; since the index record of the used chunk in the SMR hard disk is persisted into the memory, the solution is solved. The problem of low reading efficiency when sequentially reading the SMR hard disk is achieved; the effect of improving the reading efficiency of the SMR hard disk is achieved.
在实际使用时,为了保证SMR硬盘的高效使用,SMR所在的设备需要对SMR硬盘进行垃圾清理。具体请参考下述步骤:In actual use, in order to ensure the efficient use of the SMR hard disk, the device where the SMR is located needs to clean the SMR hard disk. Please refer to the following steps for details:
第一,当SMR硬盘的空间使用率达到指定使用阈值时,查找第一全局变量最大的超级块;First, when the space usage rate of the SMR hard disk reaches the specified usage threshold, the super block with the largest global variable is found;
需要说明的是,SMR硬盘中的超级块还可以包括第一全局变量,通常来讲,在后被申请的超级块的第一全局变量大于在前被申请的超级块的第一全局变量。举例来讲,当SMR所在的设备在系统元数据区中申请下一个超级块时,会将当前的第一全局变量加1并作为申请得到的超级块第一全局变量。It should be noted that the super block in the SMR hard disk may further include a first global variable. Generally, the first global variable of the super block applied later is greater than the first global variable of the super block that was previously applied. For example, when the device where the SMR is located requests the next super block in the system metadata area, the current first global variable is incremented by 1 and used as the first global variable of the super block obtained by the application.
超级块中的使用信息可以包括每个已使用chunk中无效数据记录的占用比例,无效数据记录为索引记录未记载在内存的第二存储区域中的数据记录。The usage information in the super block may include an occupation ratio of an invalid data record in each used chunk, and the invalid data record is a data record in which the index record is not recorded in the second storage area of the memory.
需要说明的是,这里的无效数据记录的占用比例的详细获取过程可以参见图3A对应实施例中的步骤308,这里不再累述。It should be noted that the detailed acquisition process of the proportion of the invalid data records herein can be referred to step 308 in the corresponding embodiment of FIG. 3A, and is not described here.
SMR所在的设备检测当前SMR硬盘的空间使用率,当检测到SMR硬盘的空间使用率高于指定使用阈值时,在SMR硬盘的系统元数据区内查找第一全局变量最大的超级块。由于在后被申请的超级块的第一全局变量大于在前被申请的超级块的第一全局变量,因此该第一全局变量最大的超级块中记录了当前SMR硬盘中的记录区内各个chunk的属性和使用信息。The device where the SMR is located detects the space usage of the current SMR hard disk. When it is detected that the space usage rate of the SMR hard disk is higher than the specified use threshold, the super block with the largest global variable is searched in the system metadata area of the SMR hard disk. Since the first global variable of the super block applied later is greater than the first global variable of the super block that was previously applied, the maximum chunk of the first global variable records each chunk in the record area of the current SMR hard disk. Properties and usage information.
举例来讲,指定使用阈值为85%,SMR所在的设备检测到当前SMR硬盘的空间使用率为90%,则此时SMR硬盘的空间使用率高于指定使用阈值。SMR所在的设备将在SMR硬盘的系统元数据区内查找第一全局变量最大的超级块。For example, if the specified usage threshold is 85%, the device where the SMR is located detects that the current SMR hard disk space usage is 90%, and then the SMR hard disk space usage rate is higher than the specified usage threshold. The device where the SMR is located will find the super block with the largest global variable in the system metadata area of the SMR hard disk.
可选的,由于该垃圾清理过程可能需要频繁地写入和读出,因此,为了不 影响SMR所在的设备的正常使用,SMR所在的设备可以先检测系统的使用情况,当检测到系统空闲时,执行该垃圾清理过程。Optionally, since the garbage cleaning process may need to be frequently written and read, therefore, in order not to Affects the normal use of the device where the SMR is located. The device where the SMR resides can detect the usage of the system first. When the system is detected to be idle, the garbage cleaning process is performed.
第二,从查找到的超级块中的使用信息中,读取每个已使用的chunk中无效数据记录的占用比例;Second, from the usage information in the found super block, read the occupation ratio of the invalid data record in each used chunk;
SMR所在的设备可以根据查找到的超级块获取每个已使用的chunk的使用信息,根据每个已使用的chunk的使用信息确定每个已使用的chunk中无效数据记录的占用比例。The device where the SMR is located can obtain the usage information of each used chunk according to the found super block, and determine the occupation ratio of the invalid data record in each used chunk according to the usage information of each used chunk.
第三,确定出无效数据记录的占用比例达到预定比例阈值的已使用的chunk,对该已使用的chunk进行垃圾清理。Third, it is determined that the used chunk of the invalid data record reaches a predetermined ratio threshold, and the used chunk is garbage-cleaned.
SMR所在的设备在确定出无效数据记录的占用比例达到预定比例阈值的已使用的chunk后,确定出已使用的chunk中的索引记录,在内存中的第二存储区域中查找该索引记录,若查找到该索引记录,则将该索引记录对应的数据记录写入当前的正在使用的chunk中,若未查找到该索引记录,则对该索引记录对应的数据记录进行垃圾清理。The device in which the SMR is located determines the index record in the used chunk after determining that the occupied proportion of the invalid data record reaches the predetermined threshold, and searches for the index record in the second storage area in the memory. If the index record is found, the data record corresponding to the index record is written into the current chunk being used. If the index record is not found, the data record corresponding to the index record is garbage-cleaned.
举例来讲,请参见图4,其是本发明一实施例中提供的垃圾清理过程的示意图。如图4所示,SMR所在的设备在确定出已使用的chunk420中的无效数据记录的占用比例达到预定比例阈值后,将该已使用的chunk420的数据区中的有效数据记录0、有效数据记录2和有效数据记录4写入正在使用的chunk430的数据区中,生成与有效数据记录0、有效数据记录2和有效数据记录4对应的索引记录,并将这些索引记录保存到内存的第一存储区域中。SMR所在的设备在将已使用的chunk420中有效数据全部写入到正在使用的chunk430后,将该已使用的chunk420的属性更改为未使用,进而变成未使用的chunk440,并申请一块超级块450,在该超级块450中记录当前记录区中各个chunk的属性和使用信息。For example, please refer to FIG. 4, which is a schematic diagram of a garbage cleaning process provided in an embodiment of the present invention. As shown in FIG. 4, after the device in which the SMR is located determines that the occupied proportion of the invalid data record in the used chunk 420 reaches a predetermined ratio threshold, the valid data record in the data area of the used chunk 420 is 0, and the valid data record is recorded. 2 and the valid data record 4 is written in the data area of the chunk 430 being used, and an index record corresponding to the valid data record 0, the valid data record 2, and the valid data record 4 is generated, and these index records are saved to the first storage of the memory. In the area. After the SMR device writes all the valid data in the used chunk420 to the chunk430 being used, the attribute of the used chunk420 is changed to unused, and then becomes an unused chunk440, and a super block 450 is applied. The attribute and usage information of each chunk in the current recording area is recorded in the super block 450.
在实际使用时,当SMR硬盘重启后,为了保证SMR硬盘的正常使用,SMR所在的设备需要在内存中重建数据,具体请参考图5,其示出了本发明再一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的方法流程图,该将数据写入叠瓦状磁记录SMR硬盘的方法可以用于包含有SMR硬盘的设备中,该将数据写入叠瓦状磁记录SMR硬盘的方法,可以包括:In actual use, after the SMR hard disk is restarted, in order to ensure the normal use of the SMR hard disk, the device in which the SMR is located needs to reconstruct data in the memory. For details, please refer to FIG. 5, which illustrates that it will be provided in another embodiment of the present invention. A flow chart of a method for writing data into a stacked magnetic recording SMR hard disk, the method of writing data into a stacked magnetic recording SMR hard disk can be used in a device including an SMR hard disk, which writes data into a stacked magnetic recording The method of the SMR hard disk may include:
步骤501,接收写数据指令; Step 501: Receive a write data instruction.
步骤502,根据数据记录的长度向正在使用的chunk中的数据区申请空间;Step 502: Apply space to the data area in the chunk being used according to the length of the data record;
步骤503,将数据记录写入申请的空间中; Step 503, writing a data record into the space of the application;
步骤504,将键、标记与数据被写入正在使用的chunk中的地址组成索引记录; Step 504, forming an index record by combining the key, the tag, and the address in which the data is written into the chunk being used;
步骤505,将索引记录保存至内存中与正在使用的chunk对应的第一存储区域;Step 505: Save the index record to a first storage area in the memory corresponding to the chunk being used;
步骤506,在正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将内存中的第一存储区域内的索引记录存储至正在使用的chunk中的索引区; Step 506, when the remaining space in the chunk being used is insufficient to allocate the data to be written in the write data instruction, storing the index record in the first storage area in the memory to the index area in the chunk being used;
步骤507,将正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk; Step 507, modifying the attribute of the chunk being used to be used, and applying an unused chunk as the used chunk for the next write data;
步骤508,将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;Step 508: Store an index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
步骤509,清除第一存储区域内的索引记录;Step 509: Clear an index record in the first storage area.
上述步骤501至509是SMR所在的设备向SMR硬盘内写入数据记录以及对chunk的索引记录进行持久化的过程,详细描述请参考与图3A对应的实施例,这里就不再累述。The foregoing steps 501 to 509 are processes in which the device in which the SMR is located writes a data record to the SMR hard disk and persists the index record of the chunk. For details, refer to the embodiment corresponding to FIG. 3A, and the description thereof will not be repeated here.
在实际使用时,当硬盘重启后,为了保证SMR硬盘的正常使用,SMR所在的设备需要在内存中重建数据,具体流程请参见下述步骤510至512。In actual use, after the hard disk is restarted, in order to ensure the normal use of the SMR hard disk, the device where the SMR is located needs to rebuild data in the memory. For details, see steps 510 to 512 below.
步骤510,在重启硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk; Step 510, when restarting the hard disk, determining the used chunk and the chunk being used according to the attribute of the chunk;
当SMR硬盘重启后,SMR所在的设备可以根据chunk的属性确定已使用的chunk和正在使用的chunk。After the SMR hard disk is restarted, the device where the SMR is located can determine the used chunk and the chunk being used according to the attributes of the chunk.
可选的,SMR所在的设备可以从SMR硬盘中系统元数据区内的最新的超级块中来获取各个chunk的属性,从而确定已使用的chunk和正在使用的chunk。Optionally, the device where the SMR is located can obtain the attributes of each chunk from the latest super block in the system metadata area of the SMR hard disk, thereby determining the used chunk and the chunk being used.
可选的,超级块还可以包括第一全局变量,在后被申请的超级块的第一全局变量大于在前被申请的超级块的第一全局变量。举例来讲,当SMR所在的设备在系统元数据区中申请下一个超级块时,会将当前的第一全局变量加1并作为申请得到的超级块第一全局变量。SMR所在的设备在检测到硬盘重新启动后,在SMR硬盘中的系统元数据区内查找第一全局变量最大的超级块。该第一全局 变量最大的超级块中记录了当前SMR硬盘中的记录区内各个chunk的属性和使用信息。SMR所在的设备根据超级块中记录的各个chunk的属性和使用信息确定已使用的chunk和正在使用的chunk。Optionally, the super block may further include a first global variable, and the first global variable of the super block applied later is greater than the first global variable of the super block that was previously applied. For example, when the device where the SMR is located requests the next super block in the system metadata area, the current first global variable is incremented by 1 and used as the first global variable of the super block obtained by the application. After detecting the hard disk restart, the SMR device searches for the super block with the largest global variable in the system metadata area of the SMR hard disk. The first global The super-block with the largest variable records the attributes and usage information of each chunk in the record area of the current SMR hard disk. The device where the SMR is located determines the used chunk and the chunk being used according to the attributes and usage information of each chunk recorded in the superblock.
步骤511,将已使用的chunk中存储的索引记录,写入内存中与已使用的chunk对应的第二存储区域;Step 511: Write an index record stored in the used chunk to a second storage area in the memory corresponding to the used chunk;
需要说明的是,索引记录还包括与数据记录唯一对应的第二全局变量,在后写入的索引记录的第二全局变量大于在前写入的索引记录的第二全局变量。It should be noted that the index record further includes a second global variable uniquely corresponding to the data record, and the second global variable of the index record written later is greater than the second global variable of the previously recorded index record.
SMR所在的设备在获取已使用的chunk中的索引记录后,根据各条索引记录的第二全局变量,将已使用的chunk中的索引记录依次写入内存的第二存储区域。SMR所在的设备在依次写入索引记录时,提取各条索引记录的标记,并根据标记的类型执行相应操作,具体可以分为下述三种情况。After acquiring the index records in the used chunks, the SMR records the index records in the used chunks into the second storage area of the memory according to the second global variable recorded in each index. When the device where the SMR is located writes the index records in sequence, the tags of each index record are extracted, and corresponding operations are performed according to the types of the tags, which can be classified into the following three cases.
第一种情况,SMR所在的设备提取出的索引记录的标记为删除标记时,丢弃键与提取的索引记录的键相同且第二全局变量小于提取的索引记录的第二全局变量的索引记录,直到所有键与提取的索引记录的键相同且第二全局变量小于提取的索引记录的第二全局变量的索引记录均被丢弃后,丢弃提取的索引记录。In the first case, when the index record extracted by the device where the SMR is located is marked as a delete mark, the drop key is the same as the key of the extracted index record, and the second global variable is smaller than the index record of the second global variable of the extracted index record. The extracted index record is discarded until all the keys are identical to the extracted index record and the second global variable is smaller than the index record of the second global variable of the extracted index record.
第二种情况,SMR所在的设备提取出的索引记录的标记为修改标记时,丢弃键与提取的索引记录的键相同且第二全局变量小于提取的索引记录的第二全局变量的索引记录,直到所有键与提取的索引记录的键相同且第二全局变量小于提取的索引记录的第二全局变量的索引记录均被丢弃后,将提取的索引记录写入第二存储区域。In the second case, when the index of the index record extracted by the device where the SMR is located is a modified mark, the drop key is the same as the key of the extracted index record, and the second global variable is smaller than the index record of the second global variable of the extracted index record. The extracted index record is written to the second storage area until all the keys are identical to the extracted index record and the second global variable is smaller than the index record of the second global variable of the extracted index record.
第三种情况,SMR所在的设备提取出的索引记录的标记为写标记时,将提取的索引记录写入第二存储区域。In the third case, when the index of the index record extracted by the device where the SMR is located is a write mark, the extracted index record is written into the second storage area.
步骤512,根据正在使用的chunk中存储的数据记录,重建与该数据记录对应的索引记录,并将重建得到的索引记录写入内存中与正在使用的chunk对应的第一存储区域。Step 512: Rebuild an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the first storage area in the memory corresponding to the chunk being used.
重建得到的索引记录可以包括:数据记录中数据的键、与该数据对应的标记以及数据被写入正在使用的chunk中的地址。The reconstructed index record may include: a key of data in the data record, a tag corresponding to the data, and an address in which the data is written in the chunk being used.
综上所述,本实施例提供的将数据写入叠瓦状磁记录SMR硬盘的方法,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写 数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the method for writing data into the shingled magnetic recording SMR hard disk provided by this embodiment is to write the data record containing the key, the data and the mark into the chunk being used; The data record generated by the data instruction contains keys and data. Therefore, the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk is solved, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization The data can be randomly written between the various chunks, thus achieving the effect of improving the flexibility of writing data on the SMR hard disk.
另外,通过将已使用的chunk中存储的索引记录写入内存中的第二存储区域,并将重建得到的索引记录写入内存中的第一存储区域;由于能够在硬盘重启后及时重建数据,因此,解决了SMR硬盘重启后无法正常工作的问题;到达了提高SMR硬盘的使用性能的效果。In addition, by writing the index record stored in the used chunk to the second storage area in the memory, and writing the reconstructed index record to the first storage area in the memory; since the data can be reconstructed in time after the hard disk is restarted, Therefore, the problem that the SMR hard disk cannot work normally after being restarted is solved; the effect of improving the performance of the SMR hard disk is achieved.
请参考图6,其示出了本发明一实施例中提供的将数据写入叠瓦状磁记录SMR硬盘的装置的结构方框图,该将数据写入叠瓦状磁记录SMR硬盘的装置可以用于包含有SMR硬盘的设备中,该将数据写入叠瓦状磁记录SMR硬盘的装置,可以包括:接收模块601、申请模块602、第一写入模块603和保存模块604;Please refer to FIG. 6 , which is a structural block diagram of an apparatus for writing data into a stacked magnetic recording SMR hard disk according to an embodiment of the present invention. The device for writing data into a stacked magnetic recording SMR hard disk can be used. In the device including the SMR hard disk, the device for writing data into the stacked magnetic recording SMR hard disk may include: a receiving module 601, an application module 602, a first writing module 603, and a saving module 604;
接收模块601,用于接收写数据指令,写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有键的数据,修改标记用于表示利用数据修改具有相同键且已被写入SMR硬盘的数据;The receiving module 601 is configured to receive a write data instruction, where the write data instruction includes a key, a data and a mark, the key is used to identify the data, the mark is a write mark, a delete mark or a modify mark, and the write mark is used to indicate that the data is written into the SMR hard disk. The delete flag is used to delete the data having the key in the SMR hard disk, and the modification mark is used to indicate that the data having the same key and written to the SMR hard disk is modified by using the data;
申请模块602,用于根据数据记录的长度向正在使用的chunk申请空间,数据记录包括写数据指令中的键、数据和标记;The application module 602 is configured to apply for a space to the chunk being used according to the length of the data record, where the data record includes a key, a data, and a mark in the write data instruction;
第一写入模块603,用于将数据记录写入申请的空间中;a first writing module 603, configured to write a data record into the space of the application;
保存模块604,用于保存键、标记与数据被写入正在使用的chunk中的地址。The save module 604 is configured to save the address of the key, the mark, and the data written in the chunk being used.
综上所述,本实施例提供的将数据写入叠瓦状磁记录SMR硬盘的装置,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the device for writing data into the shingled magnetic recording SMR hard disk provided by the embodiment is written into the chunk being used by writing the data record containing the key, the data and the mark; The data record contains keys and data. Therefore, it solves the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization, Data can be randomly written between chunks, thus achieving the flexibility of writing data on the SMR hard disk.
请参考图7,其示出了本发明另一实施例中提供的将数据写入叠瓦状磁记录 SMR硬盘的装置的结构方框图,该将数据写入叠瓦状磁记录SMR硬盘的装置可以用于包含有SMR硬盘的设备中,该将数据写入叠瓦状磁记录SMR硬盘的装置,可以包括:接收模块701、申请模块702、第一写入模块703和保存模块704;Please refer to FIG. 7, which illustrates writing data into a tile-like magnetic record according to another embodiment of the present invention. A structural block diagram of a device for writing a SMR hard disk, the device for writing data to a stacked magnetic recording SMR hard disk can be used in a device including an SMR hard disk, and the device for writing data into the stacked magnetic recording SMR hard disk can include : receiving module 701, application module 702, first writing module 703 and saving module 704;
接收模块701,用于接收写数据指令,写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有键的数据,修改标记用于表示利用数据修改具有相同键且已被写入SMR硬盘的数据;The receiving module 701 is configured to receive a write data instruction, where the write data instruction includes a key, a data and a mark, the key is used to identify the data, the mark is a write mark, a delete mark or a modify mark, and the write mark is used to indicate that the data is written into the SMR hard disk. The delete flag is used to delete the data having the key in the SMR hard disk, and the modification mark is used to indicate that the data having the same key and written to the SMR hard disk is modified by using the data;
申请模块702,用于根据数据记录的长度向正在使用的chunk申请空间,数据记录包括写数据指令中的键、数据和标记;The application module 702 is configured to apply for a space to the chunk being used according to the length of the data record, where the data record includes a key, a data, and a mark in the write data instruction;
第一写入模块703,用于将数据记录写入申请的空间中;a first writing module 703, configured to write the data record into the space of the application;
保存模块704,用于保存键、标记与数据被写入正在使用的chunk中的地址。The save module 704 is configured to save the address of the key, the tag, and the data written in the chunk being used.
在本实施例中的第一种可能的实现方式中,该申请模块702用于根据数据记录的长度向正在使用的chunk中的数据区申请空间。In the first possible implementation manner in this embodiment, the application module 702 is configured to apply for a space to a data area in a chunk being used according to the length of the data record.
在本实施例中的第二种可能的实现方式中,该保存模块704可以包括:组成单元704a和保存单元704b;In a second possible implementation manner in this embodiment, the saving module 704 may include: a component unit 704a and a saving unit 704b;
该组成单元704a,用于将键、标记与数据被写入正在使用的chunk中的地址组成索引记录;The component unit 704a is configured to form an index record by inserting a key, a tag, and an address in which the data is written into the chunk being used;
该保存单元704b,用于将索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将内存中的第一存储区域内的索引记录存储至正在使用的chunk中的索引区。The saving unit 704b is configured to save the index record to the first storage area in the memory corresponding to the chunk being used; or, when the remaining space in the chunk being used is insufficient to allocate the data to be written in the write data instruction The index record in the first storage area in the memory is stored in the index area in the chunk being used.
在本实施例中的第三种可能的实现方式中,该装置还可以包括:修改模块705、存储模块706和清除模块707。In a third possible implementation manner in this embodiment, the apparatus may further include: a modification module 705, a storage module 706, and a clearing module 707.
该修改模块705,用于将正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;The modifying module 705 is configured to modify the attribute of the chunk being used to be used, and apply an unused chunk as the used chunk for the next data to be written;
该存储模块706,用于将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;The storage module 706 is configured to store an index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
该清除模块707,用于清除第一存储区域内的索引记录。The clearing module 707 is configured to clear an index record in the first storage area.
在本实施例中的第四种可能的实现方式中,该装置还可以包括:确定模块 708、第二写入模块709和重建模块710。In a fourth possible implementation manner in this embodiment, the apparatus may further include: determining a module 708. The second write module 709 and the reconstruction module 710.
该确定模块708,用于在重启硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;The determining module 708 is configured to determine, according to the attribute of the chunk, the used chunk and the chunk being used when the hard disk is restarted;
该第二写入模块709,用于将已使用的chunk中存储的索引记录,写入内存中与已使用的chunk对应的第二存储区域;The second writing module 709 is configured to write the index record stored in the used chunk to the second storage area in the memory corresponding to the used chunk;
该重建模块710,用于根据正在使用的chunk中存储的数据记录,重建与数据记录对应的索引记录,并将重建得到的索引记录写入内存中与正在使用的chunk对应的第一存储区域。The reconstruction module 710 is configured to reconstruct an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the first storage area in the memory corresponding to the chunk being used.
在本实施例中的第五种可能的实现方式中,重建得到的索引记录包括:数据记录中数据的键、与数据对应的标记以及数据被写入正在使用的chunk中的地址。In a fifth possible implementation manner in this embodiment, the reconstructed index record includes: a key of data in the data record, a mark corresponding to the data, and an address in which the data is written in the chunk being used.
综上所述,本实施例提供的将数据写入叠瓦状磁记录SMR硬盘的装置,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the device for writing data into the shingled magnetic recording SMR hard disk provided by the embodiment is written into the chunk being used by writing the data record containing the key, the data and the mark; The data record contains keys and data. Therefore, it solves the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization, Data can be randomly written between chunks, thus achieving the flexibility of writing data on the SMR hard disk.
另外,通过将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;由于将SMR硬盘中已使用的chunk的索引记录持久化到内存中,因此,解决了顺序读取SMR硬盘时的读取效率较低的问题;达到了提高SMR硬盘的读取效率的效果。In addition, the index record in the first storage area is stored in the second storage area corresponding to the used chunk in the memory; since the index record of the used chunk in the SMR hard disk is persisted into the memory, the solution is solved. The problem of low reading efficiency when sequentially reading the SMR hard disk is achieved; the effect of improving the reading efficiency of the SMR hard disk is achieved.
另外,通过将已使用的chunk中存储的索引记录写入内存中的第二存储区域,并将重建得到的索引记录写入内存中的第一存储区域;由于能够在硬盘重启后及时重建数据,因此,解决了SMR硬盘重启后无法正常工作的问题;到达了提高SMR硬盘的使用性能的效果。In addition, by writing the index record stored in the used chunk to the second storage area in the memory, and writing the reconstructed index record to the first storage area in the memory; since the data can be reconstructed in time after the hard disk is restarted, Therefore, the problem that the SMR hard disk cannot work normally after being restarted is solved; the effect of improving the performance of the SMR hard disk is achieved.
需要说明的是:上述实施例提供的将数据写入叠瓦状磁记录SMR硬盘的装置在将数据写入叠瓦状磁记录SMR硬盘时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将SMR所在的设备的内部结构划分成不同的功能模块,以完成以上描述的全 部或者部分功能。另外,上述实施例提供的将数据写入叠瓦状磁记录SMR硬盘的装置与将数据写入叠瓦状磁记录SMR硬盘的方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。It should be noted that the device for writing data into the stacked magnetic recording SMR hard disk provided by the above embodiment is only illustrated by the division of the above functional modules when writing data into the stacked magnetic recording SMR hard disk. In the application, the above function assignment can be completed by different functional modules as needed, that is, the internal structure of the device in which the SMR is located is divided into different functional modules to complete the above description. Part or part of the function. In addition, the device for writing data to the stacked magnetic recording SMR hard disk provided by the above embodiment belongs to the same concept as the method for writing data into the stacked magnetic recording SMR hard disk. For the specific implementation process, refer to the method embodiment. I won't go into details here.
请参见图8,其示出了本发明一实施例中提供的包含有SMR硬盘的设备的结构方框图,该设备可以包括:接收机802和处理器804,其中处理器804和接收机802耦合。Referring to FIG. 8, which is a block diagram showing the structure of an apparatus including an SMR hard disk provided in an embodiment of the present invention, the apparatus may include a receiver 802 and a processor 804, wherein the processor 804 and the receiver 802 are coupled.
接收机802,用于接收写数据指令,写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有键的数据,修改标记用于表示利用数据修改具有相同键且已被写入SMR硬盘的数据;The receiver 802 is configured to receive a write data instruction, where the write data instruction includes a key, a data and a mark, the key is used to identify the data, the mark is a write mark, a delete mark or a modify mark, and the write mark is used to indicate that the data is written into the SMR hard disk. The delete flag is used to delete the data having the key in the SMR hard disk, and the modification mark is used to indicate that the data having the same key and written to the SMR hard disk is modified by using the data;
处理器804,用于根据数据记录的长度向正在使用的chunk申请空间,数据记录包括写数据指令中的键、数据和标记;The processor 804 is configured to apply for a space to the chunk being used according to the length of the data record, where the data record includes a key, a data, and a mark in the write data instruction;
处理器804,还用于将数据记录写入申请的空间中;The processor 804 is further configured to write the data record into the space of the application;
处理器804,还用于保存键、标记与数据被写入正在使用的chunk中的地址。The processor 804 is further configured to save the address of the key, the tag, and the data written in the chunk being used.
综上所述,本实施例提供的包含有SMR硬盘的设备,通过将包含有键、数据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the device including the SMR hard disk provided in this embodiment writes data records including keys, data, and tags into the chunk being used; since the data record generated according to the write data instruction includes the key and The data, therefore, solves the problem that the distributed storage system layout solution cannot be directly transplanted to the SMR hard disk, resulting in low write performance of the SMR hard disk; since the chunk is obtained according to the band virtualization, it can be randomly written between the chunks. The data thus achieves the effect of increasing the flexibility of writing data on the SMR hard disk.
请参见图9,其示出了本发明另一实施例中提供的包含有SMR硬盘的设备的结构方框图,该设备可以包括:接收机902、处理器904和存储器906,其中处理器904分别与接收机902和存储器906耦合,存储器906中存储有至少一种计算机软件,处理器904可以根据存储器906存储的计算机软件执行相关的操作。Referring to FIG. 9, which is a structural block diagram of an apparatus including an SMR hard disk provided in another embodiment of the present invention, the device may include: a receiver 902, a processor 904, and a memory 906, wherein the processor 904 respectively Receiver 902 is coupled to memory 906, which stores at least one type of computer software, and processor 904 can perform related operations in accordance with computer software stored in memory 906.
接收机902,用于接收写数据指令,写数据指令包括键、数据和标记,键用于标识数据,标记为写标记、删除标记或修改标记,写标记用于表示将数据写入SMR硬盘中,删除标记用于表示删除SMR硬盘中具有键的数据,修改标记 用于表示利用数据修改具有相同键且已被写入SMR硬盘的数据;The receiver 902 is configured to receive a write data instruction, where the write data instruction includes a key, a data and a mark, the key is used to identify the data, the mark is a write mark, a delete mark or a modify mark, and the write mark is used to indicate that the data is written into the SMR hard disk. , delete mark is used to delete the data with the key in the SMR hard disk, modify the mark Used to indicate that data having the same key and having been written to the SMR hard disk is modified by using the data;
处理器904,用于根据数据记录的长度向正在使用的chunk申请空间,数据记录包括写数据指令中的键、数据和标记;The processor 904 is configured to apply for a space to the chunk being used according to the length of the data record, where the data record includes a key, a data, and a mark in the write data instruction;
处理器904,还用于将数据记录写入申请的空间中;The processor 904 is further configured to write the data record into the space of the application;
处理器904,还用于保存键、标记与数据被写入正在使用的chunk中的地址。The processor 904 is further configured to save the address of the key, the tag, and the data written in the chunk being used.
在本实施例中的第一种可能的实现方式中,处理器904,还用于根据数据记录的长度向正在使用的chunk中的数据区申请空间。In the first possible implementation manner in this embodiment, the processor 904 is further configured to apply for a space to the data area in the chunk being used according to the length of the data record.
在本实施例中的第二种可能的实现方式中,处理器904,还用于将键、标记与数据被写入正在使用的chunk中的地址组成索引记录;In a second possible implementation manner in this embodiment, the processor 904 is further configured to form an index record by using an address in which the key, the tag, and the data are written into the chunk being used;
处理器904,还用于将索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将内存中的第一存储区域内的索引记录存储至正在使用的chunk中的索引区。The processor 904 is further configured to save the index record to the first storage area in the memory corresponding to the chunk being used; or, when the remaining space in the chunk being used is insufficient to allocate the data to be written in the write data instruction The index record in the first storage area in the memory is stored in the index area in the chunk being used.
在本实施例中的第三种可能的实现方式中,处理器904,还用于将正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;In a third possible implementation manner in this embodiment, the processor 904 is further configured to modify the attribute of the chunk being used to be used, and apply for an unused chunk as the next data to be used. Chunk
处理器904,还用于将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;The processor 904 is further configured to store the index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
处理器904,还用于清除第一存储区域内的索引记录。The processor 904 is further configured to clear an index record in the first storage area.
在本实施例中的第四种可能的实现方式中,处理器904,还用于在重启硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;In a fourth possible implementation manner in this embodiment, the processor 904 is further configured to: when restarting the hard disk, determine the used chunk and the chunk being used according to the attribute of the chunk;
处理器904,还用于将已使用的chunk中存储的索引记录,写入内存中与已使用的chunk对应的第二存储区域;The processor 904 is further configured to write an index record stored in the used chunk to a second storage area in the memory corresponding to the used chunk;
处理器904,还用于根据正在使用的chunk中存储的数据记录,重建与该数据记录对应的索引记录,并将重建得到的索引记录写入内存中与正在使用的chunk对应的第一存储区域。The processor 904 is further configured to reconstruct an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the first storage area in the memory corresponding to the chunk being used. .
在本实施例中的第五种可能的实现方式中,重建得到的索引记录包括:数据记录中数据的键、与数据对应的标记以及数据被写入正在使用的chunk中的地址。In a fifth possible implementation manner in this embodiment, the reconstructed index record includes: a key of data in the data record, a mark corresponding to the data, and an address in which the data is written in the chunk being used.
综上所述,本实施例提供的包含有SMR硬盘的设备,通过将包含有键、数 据和标记的数据记录写入正在使用的chunk中;由于根据写数据指令生成的数据记录包含有键和数据,因此,解决了不能直接将分布式存储系统布局方案移植到SMR硬盘上,导致SMR硬盘的写入性能较低的问题;由于chunk根据band虚拟化得到,在各个chunk之间可以随机写入数据,因此达到了可以提高在SMR硬盘写入数据的灵活性的效果。In summary, the device including the SMR hard disk provided in this embodiment includes a key and a number. According to the marked data record written in the chunk being used; since the data record generated according to the write data instruction contains keys and data, it solves the problem that the distributed storage system layout scheme cannot be directly transplanted to the SMR hard disk, resulting in SMR The problem of low write performance of the hard disk; since the chunk is obtained according to the band virtualization, data can be randomly written between the chunks, thereby achieving the effect of improving the flexibility of writing data on the SMR hard disk.
另外,通过将第一存储区域内的索引记录存储至内存中与已使用的chunk对应的第二存储区域;由于将SMR硬盘中已使用的chunk的索引记录持久化到内存中,因此,解决了顺序读取SMR硬盘时的读取效率较低的问题;达到了提高SMR硬盘的读取效率的效果。In addition, the index record in the first storage area is stored in the second storage area corresponding to the used chunk in the memory; since the index record of the used chunk in the SMR hard disk is persisted into the memory, the solution is solved. The problem of low reading efficiency when sequentially reading the SMR hard disk is achieved; the effect of improving the reading efficiency of the SMR hard disk is achieved.
另外,通过将已使用的chunk中存储的索引记录写入内存中的第二存储区域,并将重建得到的索引记录写入内存中的第一存储区域;由于能够在硬盘重启后及时重建数据,因此,解决了SMR硬盘重启后无法正常工作的问题;到达了提高SMR硬盘的使用性能的效果。In addition, by writing the index record stored in the used chunk to the second storage area in the memory, and writing the reconstructed index record to the first storage area in the memory; since the data can be reconstructed in time after the hard disk is restarted, Therefore, the problem that the SMR hard disk cannot work normally after being restarted is solved; the effect of improving the performance of the SMR hard disk is achieved.
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,可以仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit may be only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为 单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated as The components displayed by the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外,在本发明各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .
以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应所述以权利要求的保护范围为准。 The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims (12)

  1. 一种将数据写入叠瓦状磁记录SMR硬盘的方法,其特征在于,所述SMR硬盘包括多个根据磁轨带band虚拟化后得到的chunk,所述方法包括:A method for writing data into a stacked magnetic recording SMR hard disk, wherein the SMR hard disk includes a plurality of chunks obtained by virtualizing a track band, the method comprising:
    接收写数据指令,所述写数据指令包括键、数据和标记,所述键用于标识所述数据,所述标记为写标记、删除标记或修改标记,所述写标记用于表示将所述数据写入所述SMR硬盘中,所述删除标记用于表示删除所述SMR硬盘中具有所述键的数据,所述修改标记用于表示利用所述数据修改具有相同键且已被写入所述SMR硬盘的数据;Receiving a write data instruction, the write data instruction including a key, a data, and a mark, the key being used to identify the data, the mark being a write mark, a delete mark, or a modify mark, the write mark being used to indicate that the Data is written into the SMR hard disk, the deletion flag is used to indicate deletion of data having the key in the SMR hard disk, and the modification flag is used to indicate that the same key is modified with the data and has been written The data of the SMR hard disk;
    根据数据记录的长度向正在使用的chunk申请空间,所述数据记录包括所述写数据指令中的所述键、所述数据和所述标记;Requiring a space to the chunk being used according to the length of the data record, the data record including the key, the data and the token in the write data instruction;
    将所述数据记录写入申请的所述空间中;Writing the data record into the space of the application;
    保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址。The address is saved, the tag and the data are written to the address in the chunk being used.
  2. 根据权利要求1所述的方法,其特征在于,所述正在使用的chunk中包含用于存储数据记录的数据区,所述根据数据记录的长度向正在使用的chunk申请空间,包括:The method according to claim 1, wherein the chunk being used includes a data area for storing a data record, and the requesting space for the chunk being used according to the length of the data record comprises:
    根据所述数据记录的长度向所述正在使用的chunk中的所述数据区申请空间。A space is requested from the data area in the chunk being used according to the length of the data record.
  3. 根据权利要求2所述的方法,其特征在于,所述正在使用的chunk中包含用于存储索引记录的索引区,所述保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址,包括:The method according to claim 2, wherein said chunk being used contains an index area for storing an index record, said saving said key, said mark and said data being written to said The address in the chunk used, including:
    将所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址组成索引记录;Forming an index record by the key, the mark, and an address in which the data is written into the chunk being used;
    将所述索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在所述正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区。 Saving the index record to a first storage area in memory corresponding to the chunk being used; or, when the remaining space in the chunk being used is insufficient to allocate data to be written in the write data instruction, The index record in the first storage area in the memory is stored in the index area in the chunk being used.
  4. 根据权利要求3所述的方法,其特征在于,在所述将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区之后,还包括:The method according to claim 3, wherein after said storing said index record in said first storage area in said memory to said index area in said chunk being used, Also includes:
    将所述正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;Modify the attribute of the chunk being used to used, and apply for an unused chunk as the used chunk for the next write data;
    将所述第一存储区域内的所述索引记录存储至所述内存中与已使用的chunk对应的第二存储区域;And storing the index record in the first storage area into a second storage area in the memory corresponding to the used chunk;
    清除所述第一存储区域内的所述索引记录。Clearing the index record in the first storage area.
  5. 根据权利要求2至4中任一所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 2 to 4, wherein the method further comprises:
    在重启所述硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;When the hard disk is restarted, the used chunk and the chunk being used are determined according to the attributes of the chunk;
    将所述已使用的chunk中存储的索引记录,写入所述内存中与已使用的chunk对应的第二存储区域;And storing an index record stored in the used chunk into a second storage area in the memory corresponding to the used chunk;
    根据所述正在使用的chunk中存储的数据记录,重建与所述数据记录对应的索引记录,并将重建得到的所述索引记录写入所述内存中与正在使用的chunk对应的第一存储区域。Reconstructing an index record corresponding to the data record according to the data record stored in the chunk being used, and writing the reconstructed index record into the first storage area in the memory corresponding to the chunk being used .
  6. 根据权利要求5所述的方法,其特征在于,所述重建得到的所述索引记录包括:所述数据记录中数据的键、与所述数据对应的标记以及所述数据被写入所述正在使用的chunk中的地址。The method according to claim 5, wherein said index record obtained by said reconstruction comprises: a key of data in said data record, a mark corresponding to said data, and said data is written to said index The address in the chunk used.
  7. 一种将数据写入叠瓦状磁记录SMR硬盘的装置,其特征在于,所述SMR硬盘包括多个根据磁轨带band虚拟化后得到的chunk,所述装置包括:An apparatus for writing data into a stacked magnetic recording SMR hard disk, wherein the SMR hard disk includes a plurality of chunks obtained by virtualizing a track band, the device comprising:
    接收模块,用于接收写数据指令,所述写数据指令包括键、数据和标记,所述键用于标识所述数据,所述标记为写标记、删除标记或修改标记,所述写标记用于表示将所述数据写入所述SMR硬盘中,所述删除标记用于表示删除所述SMR硬盘中具有所述键的数据,所述修改标记用于表示利用所述数据修改具有相同键且已被写入所述SMR硬盘的数据;a receiving module, configured to receive a write data instruction, where the write data instruction includes a key, a data, and a mark, wherein the key is used to identify the data, and the mark is a write mark, a delete mark, or a modify mark, and the write mark is used by And indicating that the data is written into the SMR hard disk, the deletion mark is used to indicate deleting data having the key in the SMR hard disk, and the modification mark is used to indicate that the same key is modified by using the data and Data that has been written to the SMR hard disk;
    申请模块,用于根据数据记录的长度向正在使用的chunk申请空间,所述 数据记录包括所述写数据指令中的所述键、所述数据和所述标记;An application module, configured to apply for a space to a chunk being used according to a length of the data record, The data record includes the key, the data, and the flag in the write data instruction;
    第一写入模块,用于将所述数据记录写入申请的所述空间中;a first writing module, configured to write the data record into the space of the application;
    保存模块,用于保存所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址。And a saving module, configured to save the key, the mark, and the address in which the data is written in the chunk being used.
  8. 根据权利要求7所述的装置,其特征在于,所述正在使用的chunk中包含用于存储数据记录的数据区,所述申请模块用于根据所述数据记录的长度向所述正在使用的chunk中的所述数据区申请空间。The apparatus according to claim 7, wherein said chunk being used includes a data area for storing a data record, and said application module is configured to use said chunk according to a length of said data record The data area in the application space.
  9. 根据权利要求8所述的装置,其特征在于,所述正在使用的chunk中包含用于存储索引记录的索引区,所述保存模块,包括:The device according to claim 8, wherein the chunk being used includes an index area for storing an index record, and the saving module comprises:
    组成单元,用于将所述键、所述标记与所述数据被写入所述正在使用的chunk中的地址组成索引记录;a component unit, configured to form an index record of the key, the mark, and an address in which the data is written into the chunk being used;
    保存单元,用于将所述索引记录保存至内存中与正在使用的chunk对应的第一存储区域;或者,在所述正在使用的chunk中的剩余空间不足以分配写数据指令中需要写入的数据时,将所述内存中的所述第一存储区域内的所述索引记录存储至所述正在使用的chunk中的所述索引区。a saving unit, configured to save the index record to a first storage area in the memory corresponding to the chunk being used; or, the remaining space in the chunk being used is insufficient to allocate a write data instruction to be written And storing, in the data, the index record in the first storage area in the memory to the index area in the chunk being used.
  10. 根据权利要求9所述的装置,其特征在于,所述装置还包括:The device according to claim 9, wherein the device further comprises:
    修改模块,用于将所述正在使用的chunk的属性修改为已使用,并申请一个未使用的chunk作为下次写入数据的正在使用的chunk;a modifying module, configured to modify the attribute of the used chunk to be used, and apply an unused chunk as a used chunk for writing data next time;
    存储模块,用于将所述第一存储区域内的所述索引记录存储至所述内存中与已使用的chunk对应的第二存储区域;a storage module, configured to store the index record in the first storage area to a second storage area in the memory corresponding to the used chunk;
    清除模块,用于清除所述第一存储区域内的所述索引记录。And a clearing module, configured to clear the index record in the first storage area.
  11. 根据权利要求8至10中任一所述的装置,其特征在于,所述装置还包括:The device according to any one of claims 8 to 10, wherein the device further comprises:
    确定模块,用于在重启所述硬盘时,根据chunk的属性确定已使用的chunk和正在使用的chunk;a determining module, configured to determine, according to the attribute of the chunk, the used chunk and the chunk being used when the hard disk is restarted;
    第二写入模块,用于将所述已使用的chunk中存储的索引记录,写入所述 内存中与已使用的chunk对应的第二存储区域;a second writing module, configured to write an index record stored in the used chunk into the a second storage area in memory corresponding to the used chunk;
    重建模块,用于根据所述正在使用的chunk中存储的数据记录,重建与所述数据记录对应的索引记录,并将重建得到的所述索引记录写入所述内存中与正在使用的chunk对应的第一存储区域。a reconstruction module, configured to reconstruct an index record corresponding to the data record according to the data record stored in the chunk being used, and write the reconstructed index record into the memory to correspond to a chunk being used The first storage area.
  12. 根据权利要求11所述的装置,其特征在于,所述重建得到的所述索引记录包括:所述数据记录中数据的键、与所述数据对应的标记以及所述数据被写入所述正在使用的chunk中的地址。 The apparatus according to claim 11, wherein said index record obtained by said reconstructing comprises: a key of data in said data record, a mark corresponding to said data, and said data is written to said index The address in the chunk used.
PCT/CN2015/095997 2014-12-05 2015-11-30 Method and apparatus for writing data into shingled magnetic record smr hard disk WO2016086819A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410736477.6 2014-12-05
CN201410736477.6A CN104461390B (en) 2014-12-05 2014-12-05 Write data into the method and device of imbricate magnetic recording SMR hard disks

Publications (1)

Publication Number Publication Date
WO2016086819A1 true WO2016086819A1 (en) 2016-06-09

Family

ID=52907519

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/095997 WO2016086819A1 (en) 2014-12-05 2015-11-30 Method and apparatus for writing data into shingled magnetic record smr hard disk

Country Status (2)

Country Link
CN (1) CN104461390B (en)
WO (1) WO2016086819A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394493B2 (en) 2017-06-30 2019-08-27 Seagate Technology Llc Managing shingled magnetic recording (SMR) zones in a hybrid storage device
CN111506251A (en) * 2019-01-30 2020-08-07 杭州海康威视系统技术有限公司 Data processing method, data processing device, SMR storage system and storage medium
CN112947856A (en) * 2021-02-05 2021-06-11 彩讯科技股份有限公司 Memory data management method and device, computer equipment and storage medium
CN113296697A (en) * 2021-03-17 2021-08-24 阿里巴巴新加坡控股有限公司 Data processing system, data processing method and device
CN112947856B (en) * 2021-02-05 2024-05-03 彩讯科技股份有限公司 Memory data management method and device, computer equipment and storage medium

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104461390B (en) * 2014-12-05 2017-10-24 华为技术有限公司 Write data into the method and device of imbricate magnetic recording SMR hard disks
CN106293497B (en) * 2015-05-27 2019-03-19 华为技术有限公司 Watt record filesystem-aware in junk data recovery method and device
CN105491101B (en) * 2015-11-17 2018-09-21 华为技术有限公司 The treating method and apparatus of data
CN105677243B (en) * 2015-12-31 2018-12-28 华为技术有限公司 Data transfer apparatus and method
CN107315533A (en) * 2016-04-26 2017-11-03 杭州海康威视数字技术股份有限公司 A kind of date storage method and device
CN107526691B (en) * 2016-06-21 2020-06-02 深圳市中兴微电子技术有限公司 Cache management method and device
CN106201353A (en) * 2016-07-12 2016-12-07 曹蕊 A kind of data transfer apparatus for key assignments storage system and method
CN106201355B (en) * 2016-07-12 2018-12-11 腾讯科技(深圳)有限公司 Data processing method and device and storage system
CN108021513B (en) * 2016-11-02 2021-09-10 杭州海康威视数字技术股份有限公司 Data storage method and device
CN108255408B (en) * 2016-12-28 2021-03-23 中国电信股份有限公司 Data storage method and system
CN111399762B (en) * 2019-11-27 2021-06-18 杭州海康威视系统技术有限公司 Data storage method, device and storage system
CN111459412B (en) 2020-03-30 2023-07-04 北京百度网讯科技有限公司 Disk management method and device and electronic equipment
CN113741794B (en) * 2020-05-29 2024-04-05 浙江宇视科技有限公司 SMR disk array reconstruction method, system, equipment and computer medium
CN111651127B (en) * 2020-06-10 2023-05-02 杭州海康威视数字技术股份有限公司 Monitoring data storage method and device based on shingled magnetic recording disk

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6631478B1 (en) * 1999-06-18 2003-10-07 Cisco Technology, Inc. Technique for implementing high performance stable storage hierarchy in a computer network
CN103955433A (en) * 2014-05-09 2014-07-30 华为技术有限公司 Shingled magnetic recording hard disk, and method and device for writing data in shingled magnetic recording hard disk
CN104049908A (en) * 2013-03-15 2014-09-17 希捷科技有限公司 Dynamic granule-based intermediate storage
US20140304525A1 (en) * 2013-04-01 2014-10-09 Nexenta Systems, Inc. Key/value storage device and method
CN104156172A (en) * 2013-01-17 2014-11-19 西部数据技术公司 Data management for a data storage device
CN104461390A (en) * 2014-12-05 2015-03-25 华为技术有限公司 Method and device for writing data into imbricate magnetic recording SMR hard disk

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010006099A1 (en) * 2008-07-11 2010-01-14 Marvell World Trade, Ltd. Access point rotation for sharing power load

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6631478B1 (en) * 1999-06-18 2003-10-07 Cisco Technology, Inc. Technique for implementing high performance stable storage hierarchy in a computer network
CN104156172A (en) * 2013-01-17 2014-11-19 西部数据技术公司 Data management for a data storage device
CN104049908A (en) * 2013-03-15 2014-09-17 希捷科技有限公司 Dynamic granule-based intermediate storage
US20140304525A1 (en) * 2013-04-01 2014-10-09 Nexenta Systems, Inc. Key/value storage device and method
CN103955433A (en) * 2014-05-09 2014-07-30 华为技术有限公司 Shingled magnetic recording hard disk, and method and device for writing data in shingled magnetic recording hard disk
CN104461390A (en) * 2014-12-05 2015-03-25 华为技术有限公司 Method and device for writing data into imbricate magnetic recording SMR hard disk

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394493B2 (en) 2017-06-30 2019-08-27 Seagate Technology Llc Managing shingled magnetic recording (SMR) zones in a hybrid storage device
CN111506251A (en) * 2019-01-30 2020-08-07 杭州海康威视系统技术有限公司 Data processing method, data processing device, SMR storage system and storage medium
CN111506251B (en) * 2019-01-30 2023-06-02 杭州海康威视系统技术有限公司 Data processing method and device, SMR storage system and storage medium
CN112947856A (en) * 2021-02-05 2021-06-11 彩讯科技股份有限公司 Memory data management method and device, computer equipment and storage medium
CN112947856B (en) * 2021-02-05 2024-05-03 彩讯科技股份有限公司 Memory data management method and device, computer equipment and storage medium
CN113296697A (en) * 2021-03-17 2021-08-24 阿里巴巴新加坡控股有限公司 Data processing system, data processing method and device
CN113296697B (en) * 2021-03-17 2024-04-19 阿里巴巴创新公司 Data processing system, data processing method and device

Also Published As

Publication number Publication date
CN104461390A (en) 2015-03-25
CN104461390B (en) 2017-10-24

Similar Documents

Publication Publication Date Title
WO2016086819A1 (en) Method and apparatus for writing data into shingled magnetic record smr hard disk
US10649910B2 (en) Persistent memory for key-value storage
US10031675B1 (en) Method and system for tiering data
US10628378B2 (en) Replication of snapshots and clones
US20200150890A1 (en) Data Deduplication Method and Apparatus
US10175894B1 (en) Method for populating a cache index on a deduplicated storage system
US8799601B1 (en) Techniques for managing deduplication based on recently written extents
US10466932B2 (en) Cache data placement for compression in data storage systems
KR101813786B1 (en) System and method for copy on write on an ssd
US8924664B2 (en) Logical object deletion
US10877898B2 (en) Method and system for enhancing flash translation layer mapping flexibility for performance and lifespan improvements
US8112607B2 (en) Method and system for managing large write-once tables in shadow page databases
US10503424B2 (en) Storage system
CN109804359A (en) For the system and method by write back data to storage equipment
US9922039B1 (en) Techniques for mitigating effects of small unaligned writes
JP2017079053A (en) Methods and systems for improving storage journaling
CN109902034B (en) Snapshot creating method and device, electronic equipment and machine-readable storage medium
CN113568582B (en) Data management method, device and storage equipment
WO2019223377A1 (en) File processing method, apparatus and device, and storage medium
US10416901B1 (en) Storage element cloning in presence of data storage pre-mapper with multiple simultaneous instances of volume address using virtual copies
CN110134551B (en) Continuous data protection method and device
US9798793B1 (en) Method for recovering an index on a deduplicated storage system
US9535796B2 (en) Method, apparatus and computer for data operation
US11163446B1 (en) Systems and methods of amortizing deletion processing of a log structured storage based volume virtualization
US10204002B1 (en) Method for maintaining a cache index on a deduplicated storage system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15865496

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15865496

Country of ref document: EP

Kind code of ref document: A1