TW201933121A - Data storage system and method for writing object of key-value pair - Google Patents

Abstract

A data storage system includes: a plurality of data storage devices for storing a plurality of objects of a key-value pair; and a virtual storage layer that applies different data reliability schemes including a data replication scheme and an erasure coding scheme based on a size of an object of the plurality of objects. The plurality of objects includes a first object having a first size and a second object having a second size that is larger than the first size. The virtual storage layer classifies the first object as a small object, applies the data replication scheme, and stores the small object across one or more of the plurality of data storage devices. The virtual storage layer classifies the second object as a huge object, splits the huge object into one or more chunks of a same size, applies the erasure encoding scheme, and distributedly stores the one or more chunks across the plurality of data storage devices. A method for writing an object of a key-value pair is also provided.

Description

System and method for storing large key-value objects

The present invention generally relates to a data storage system, and more particularly, to a method for storing a maximum key value object in a data storage system.

Data reliability is a key requirement for data storage systems. Data reliability using conventional block devices has been fully studied and implemented through various data replication technologies such as Redundant Array of Independent Disk (RAID) and erasure coding. RAID propagates (or copies) data through a collection of data storage drives to prevent permanent data loss for a particular drive. RAID is mainly divided into two categories: a complete image of the data is stored on the second drive, or parity blocks are added to the data so that the failed block can be recovered in the event of a failure. Erasure coding uses a complex algorithm to add a cluster of co-located blocks that provides powerful data protection and recovery that can tolerate high-level failures. For example, erasure coding virtualized physical drives to create virtual drives that can be spread over more physical drives for fast recovery. Data replication using RAID may be too expensive for copying large objects, and erasure coding may waste storage space for small objects.

Key-value solid-state drive (KV SSD) is a new type of storage device that is comparable to conventional block devices such as hard disk drive (HDD) and solid-state drive (SSD) In contrast, the new storage devices have different interfaces and semantics. KV SSD can directly store data values of key-value pairs. The value of the data stored in the KV SSD can be larger or smaller depending on the characteristics of the application and the data. There is a need for an effective data reliability model for efficiently storing objects of different sizes without performance bottlenecks and space constraints.

According to one embodiment, a data storage system includes: a plurality of data storage devices for storing a plurality of objects of a key-value pair; and a virtual storage layer that applies different data reliability schemes based on the size of an object among the plurality of objects The different data reliability schemes include data replication schemes and erasure coding schemes. The plurality of objects include a first object having a first size and a second object having a second size, and the second size is larger than the first size. The virtual storage layer classifies the first object as a small object, applies a data copying scheme, and stores the small object on one or more of a plurality of data storage devices. The virtual storage layer classifies the second object as an oversized object, splits the oversized object into one or more chunks of the same size, applies an erasure coding scheme, and stores one or more chunks distributed across multiple data stores Device.

According to another embodiment, a method for writing an object of a key-value pair includes: receiving a plurality of objects of the key-value pair, wherein the plurality of objects include a first object having a first size and a second object having a second size , The second size is larger than the first size; classifying the first object as a small object; applying a data replication scheme to the small object; storing the small object on one or more of a plurality of data storage devices; classifying the second object as Oversized objects; splitting oversized objects into one or more blocks of the same size; applying an erasure coding scheme to oversized objects; and decentrally storing one or more blocks on multiple data storage devices.

The above and other preferred features, including combinations of various novel details and events of the embodiments, will now be described more specifically with reference to the accompanying drawings and pointed out in the claims. It should be understood that the specific systems and methods described herein are shown by way of illustration only and not as a limitation. As those skilled in the art would understand, the principles and features described herein may be employed in various and numerous embodiments without departing from the scope of the invention.

Each of the features and teachings disclosed herein can be used individually or in combination with other features and teachings to provide a data storage system for efficiently storing objects of different sizes and a data storage system Object methods. Representative examples of utilizing many of these additional features and teachings individually and in combination are described in further detail with reference to the accompanying drawings. This detailed description is intended only to teach those skilled in the art other details for practicing the preferred aspects of the teachings and is not intended to limit the scope of the claims. Therefore, the combination of features disclosed above in the detailed description may not necessarily practice the teaching content in the broadest sense, but is taught only to specifically describe the representative examples of the teaching content.

In the following description, specific nomenclature is set forth for the purpose of explanation only to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that these specific details are not necessary for practicing the embodiments of the present invention.

Algorithms and symbolic representations of the operation of data bits in computer memory present some sections described in detail herein. Those skilled in the data processing arts use these algorithmic descriptions and representations to effectively convey the substance of their work to others skilled in the art. In this paper, and algorithms are generally conceived as a self-consistent sequence of steps that produce the desired result. The steps are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. Mainly for common reasons, it has proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc.

It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless stated explicitly as otherwise as apparent from the following discussion, it should be understood that throughout the description, use such as "processing", "computing", "calculating", "determining ( The discussion of "determining", "displaying" or similar terms refers to manipulating and transforming data representing physical (electronic) quantities in the register and memory of a computer system into similar representations of computer system memory The actions and processes of a computer system or similar electronic computing device such as a computer or similar electronic computing device that stores, transmits, or displays other data of physical quantities in a device or register, or other such information.

Furthermore, the various features of the representative examples and the appended claims may be combined in ways that are not specifically and explicitly enumerated to provide additional applicable embodiments of the present teachings. It should also be explicitly noted that for the purpose of the original disclosure, and for the purpose of limiting the claimed subject matter, all value ranges or indications of the group of entities disclose every possible intermediate value or intermediate entity. It should also be explicitly noted that the size and shape of the components shown in the drawings are designed to help understand how to practice this teaching, but are not intended to limit the size and shape shown in the examples.

The invention describes a data storage system and a method for storing large key-value objects in the data storage system. The data storage system of the present invention can store data in one or more data storage devices with high reliability. Specifically, the data storage system of the present invention can store objects in different ways based on the size of the objects, so as to achieve high reliability while reducing costs and storage space. The data associated with large objects can be split into smaller pieces and stored in one or more data storage devices. In this article, when the data stored in the data storage device is a data array associated with a key-value pair, the data storage device may also be referred to as a key-value solid state drive (KV SSD).

Ability to split an object (such as the value of a key-value pair) into multiple segments or chunks of the same size. The size of the chunks can be determined dynamically at execution time based on each item. Depending on the size of the tiles, the object may have tiles of different numbers and sizes.

A group is defined as a collection of data storage devices to achieve target data reliability. A group may be contained within a main case (eg, a chassis or rack) or one or more data storage devices in multiple main cases, and may be structured in a hierarchical or non-hierarchical manner.

The data storage system of the present invention includes a virtual storage layer, which manages grouping of one or more data storage devices and presents the group as a whole to a user application as a single virtual storage unit. The virtual storage layer can be used to manage multiple drives that control one or more data storage devices. The number of data storage devices managed by the virtual storage layer can be configured based on reliability goals. For erasure coding, the total number of data storage devices may be the sum of the data device (D) and the co-located device (P) to tolerate P faults. For replication, the total number of P fault-tolerant data storage devices may be P + 1. The storage capacity of a data storage device may be approximately similar in erasure coding or copying. The storage capacity of the virtual storage can be determined by the total data storage space of all data storage devices in the group.

According to one embodiment, the virtual storage layer manages a group of one or more data storage devices in a stateless manner. That is, the virtual storage layer does not (and does not need to) maintain any key information or mapping information between the object and the data storage device. However, the virtual storage layer can dynamically cache memory and maintain basic metadata of one or more data storage devices, such as the number of objects, available storage capacity, and / or the like, at runtime.

Data storage devices such as KV SSDs have implementation-specific constraints on objects and operations on objects that they can process. The virtual storage layer of the data storage system can know the minimum and maximum sizes that each data storage device can support, and can determine the minimum and maximum sizes.

for example, Is the minimum size of the i-th KV SSD. The minimum size of the virtual storage (VMIN_VS) can be defined by the maximum of all the minimum sizes of the individual KV SSDs in the group.
Formula 1)

Similarly, Is the maximum size of the i-th KV SSD. The maximum size of the virtual storage (VMAX_VS) can be defined by the minimum of all the maximum sizes of the individual KV SSDs in the group.
Equation (2)

In one example, a Reed-Solomon (RS) -encoded maximum distance separable (MDS) algorithm can be used for erasure coding.

According to one embodiment, the system and method of the present invention provides a hybrid data reliability mechanism that utilizes both copy and erasure coding based on the size of an object. Data replication is faster for small objects and can be lightweight. However, data copying may consume more storage space for large objects than erasure coding. Compared with data copy, erasure coding may consume less storage space for large objects, and erasure coding can reconstruct these large objects by using multiple data storage devices. However, erasure coding usually involves cumbersome calculations and may take longer to reconstruct small objects because it requires multiple chunks from multiple data storage devices. This is a non-hierarchical method that copies the material after it has been erasure-coded. In fact, the system and method of the present invention presents flexible decisions between data copying and erasure coding based on the size of the object. In other words, the system and method of the present invention can determine data reliability decisions at execution time based on the size of the object and the total space overhead of storing the object, ie, whether to apply data copy or erasure coding.

According to one embodiment, the system and method of the present invention does not have read-modify-write overhead when updating an object. Conventional erasure coding or data copying for block devices has a high penalty value for local updates. If the small data segment is updated, all blocks used for erasure coding must be read and updated, and after the update, the parity blocks must be re-calculated and written back to the data storage device. In other words, updating an object requires a series of read-modify-write processes because blocks can be shared with multiple objects. Conversely, the systems and methods of the present invention can provide reliability based on an object and its characteristics (eg, size). If the object is updated, then only the updated object needs to be rewritten without incurring read-modify-write overhead.

According to one embodiment, the system and method of the present invention supports a wide range of object sizes in a unified framework. If the object is very large (or oversized) beyond the storage capacity limit of the data storage device, the data storage device may not be able to store the object as a whole. In this case, the object needs to be split into small pieces, referred to herein as chunks. The present invention discusses how an object can be divided into blocks based on the performance of the data storage device and the object can be reconstructed using the divided blocks. For example, the system and method of the present invention support multiple split and reconstruction mechanisms based on the group feature of the data storage device, which will be discussed in detail below.

The system and method of the present invention provides the reliability of the object and is not based on fixed blocks. Data duplication and erasure coding can be mixed to achieve the target reliability of objects of a single disk set by forking objects based on their size. This method differs from conventional data reliability techniques that have a hierarchical structure (that is, a copy used to erase coded objects). The system and method of the present invention take space efficiency as a primary concern, and performance is a secondary metric to determine a reliability mechanism suitable for a particular object. Object storage is stateless. Copying or erasing codes does not require storing additional information. Regardless of the size of the object, updates do not require read-modify-write overhead.

In this article, an object refers to static data, which has a fixed value during input and output (input and output (I / O) operations). Objects can be associated with keys of key-value pairs. In this case, the object corresponds to the value of the key-value pair. Objects can be split into multiple blocks of the same size, and the size of the blocks can be dynamically determined based on each object at execution time. When each object is saved to one or more data storage devices, each object may have a different number and size of tiles.

In this article, the smallest set of data chunks for all objects stored on one or more data storage devices (such as KV SSDs) at once without data reliability is called fragments. For example, a fragment corresponds to the number of chunks of an object that split one or more data storage devices. The smallest set of data chunks for an object that achieves data reliability (such as copying or erasure coding) is called a band. Due to co-located blocks, the number of blocks in a band may be greater than the number of blocks in a segment. A collection of pieces of an object stored in a group to one of the data storage devices may be referred to as a split.

A snippet can contain only one block (of the target object) for data reproduction (ie, a duplicate copy of the original object). For erasure coding, a segment may contain D tiles including the target object. On the other hand, a band may contain one original block and P copy blocks for data copying, or D data blocks and P parity blocks for erasure coding. The number of segments or bands corresponds to the total number of tiles in the split. The split size (that is, the number of tiles stored in one data storage device) and the tape size (that is, the number of tiles stored in one or more data storage devices) can vary based on the original object and the size of the tiles . For example, the split size may be the size of the object used for data copying, but the split size is the block size multiplied by the number of blocks per band used for erasure coding.

FIG. 1 is a schematic diagram of objects stored in an example data storage system according to an embodiment. The object 110 may be split into a plurality of pieces. For example, the object 110 is divided into 3S blocks, and Chunk 1 to 3S, where S is a natural number. It should be noted that the total number of blocks 3S is only an example, and the total number of blocks of the object 110 may be any number, and the object 110 may not necessarily be split into multiples of three. Objects 110 may be classified as very large or oversized objects, as will be discussed in more detail below. Those split blocks can be distributed in virtual storage on one or more data storage devices. In this example, the virtual storage includes D data storage devices (disk 1 to disk D) and P parity devices (disk D + 1 to disk D + P). In this example, S is the same as D.

According to one embodiment, the virtual storage layer of the data storage system can be split-first scheme (split-first distribution) or partitioned with a band-first scheme (with priority distribution). In the split priority scheme, in virtual storage 120, the virtual storage layer stores block 1, block 2, and block 3 on disk 1, and stores block 4, block 5, and block 6 on magnetic disks. In the disc 2, the blocks 3D-2, 3D-1, and the block 3D are stored in the magnetic disk D. The band 150 includes data block 1, data block 4 to data block 3D-2, and co-located block 1 to co-located block P. In the band-priority scheme, in the virtual storage 121, the virtual storage layer stores blocks 1 to D on disk 1 to disk D, and stores blocks D + 1 to 2D on disk respectively. 1 to disk D until blocks 2D + 1 to 3D are stored in disk 1 to disk D, respectively. The split 151 includes a data block 1, a data block D + 1, and a data block 2D + 1. Parity blocks (parity 1 to parity P) are stored on parity disks (disk D + 1 to disk D + P). Although, in this example, it is shown that for convenience, the virtual storage 120 and the virtual storage 121 both store parity blocks with a priority scheme, it should be understood that the priority can be split without departing from the scope of the present invention The scheme performs the storage of parity blocks.

No matter which block distribution method is used, I / O operations can be performed with a priority scheme. In this case, even for the split-first scheme, the I / O operations of block 1, block 4 to co-located P are stored in parallel.

According to one embodiment, erasure coding is applied to an object based on its size. For having size The i-th object , The number of blocks per device (ie, split) ( It is defined by the following formula 3. For replication, the number of tiles per device can be 1 (ie .
Equation (3)

Number of blocks per device Is the minimum number of chunks per split, used when using the largest chunk size ( ) When the object Stored on data disks (ie, data storage devices called disks 1 through D). If the object size is not aligned with the allocation or alignment unit of the data storage device, the extra space allocated for the objects in the tape can be filled with zeros.

If you use the maximum tile size, you tend to waste too much storage space. So the object The actual block size of is determined more strictly by Equation 4. If additional metadata is stored with each chunk of data, then Is the metadata size. If the data storage device supports group characteristics, some metadata such as a group identifier (ID) and the total number of blocks can be stored in each block. If the data storage device does not support group characteristics, no metadata is stored (i.e. = 0). For replication, the actual chunk size can be equal to its original object size (ie ).
Equation (4)

Equation 4 determines the range of block sizes between versus Between, but close to . The block size determined by Equation 4 minimizes the number of I / O operations and maximizes I / O bandwidth. Subsequently, the amount of data (ie, the split size) stored by each data storage device is defined by Equation 5.
Equation (5)

Finally, the total amount of data (ie, tape size) written on the data storage device at one time is defined by Equation 6. For replication, D can be equal to 1.
Equation (6)

As described above, the data storage device can limit the size of objects that the data storage device can store. Some data storage devices may not support very large or very small objects. In order to achieve reliable and efficient storage of objects with different sizes, the data storage system of the present invention uses different data reliability schemes for storage based on the size of the objects. According to one embodiment, the virtual storage layer of the data storage system of the present invention can classify the objects into four types based on the size of the objects, namely, oversized, large, medium, and small. If you use multiple ribbons to store objects, classify the objects as oversized. If you use an object almost exclusively to store objects, classify the objects as large. If you use only a small part of the band to store objects, classify the objects as small. Finally, if the objects can be classified as small or large, classify the objects as medium. Therefore, different size blocks can coexist not only in the same virtual storage, but also in separate data storage devices forming a virtual storage.

If the space overhead for copying an object is less than the space overhead for erasure coding of the object, classify the object as small. In this case, replication is preferred because it provides better performance for reads and better ability to handle updates than complex erasure coding schemes. From the observations that the application of relay data is often small, this is also reasonable. In one embodiment, having Small items Meet the following formula 7:
Equation (7)

If the space overhead for erasure coding of an object is less than the space overhead for data copying of an object, then classify the object as large. In this case, erasure coding is preferred because it takes up less space. Specifically, the large object satisfies the following formula 8:
Equation (8)

Large objects can be constructed similarly to FIG. 1 , but they can have only one band or one block in a split.

If an object has more than one block within a split, the object is classified as oversized. In this case, erasure coding is preferred. Specifically, the oversized object satisfies the following formula 9:
Equation (9)

An oversized object can be constructed similarly to Figure 1 and it can have multiple bands or more than one block in a split.

There may be a range of object sizes that can be classified as small or large. Classify objects that meet the following inequality as medium:
Equation (10)

In this case, you can use data copy or erasure coding. If performance is more important and objects are updated frequently, data replication can be a better choice. In this case, the medium items can be classified as small. If space efficiency is more important, then erasure coding can be used. In this case, medium items can be classified as large.

The virtual storage layer may need to split oversized objects into small data chunks to store the objects, and then reconstruct the objects with the split data chunks to retrieve the oversized objects. For this purpose, internal keys generated from user (such as user application running on the host) keys can be used to make the virtual storage layer stateless. According to one embodiment, the virtual storage layer reserves a few bytes of key space for internally used device support for distributed partitioning and exposes the rest of the key space to the user. In this case, the user-specified object key renders one or more split and grouped internal keys of the object.

FIG. 2 illustrates an example internal key including a user key according to one embodiment. The internal key 200 includes a first part of the user key 201 and a second part with an identification code (ID) 202. The internal key 200 may be used to identify an entire block group or a part of a block for a corresponding object. In this case, the object corresponds to the value of a key-value pair, which contains the internal key 200 as the key of the key-value pair. In this example, the maximum key length supported by the virtual storage layer and / or the data storage device is L, and the number of bytes reserved for the group specification is G. The maximum key length that users of the virtual storage layer can use is LG.

For small or large objects, G bytes with ID 202 can be filled with 0 by default. For very large objects, the virtual storage layer can count the number of bands of the object. The internal key 201 can be used to identify individual bands. A tape may be written to one or more data storage devices assigned to store items one by one according to a split-first scheme or a tape-first scheme.

According to one embodiment, the data storage device may support group features. The virtual storage layer may identify a split stored in the data storage device by specifying a group based on the user key 201. In this case, no additional metadata may be needed (SZ_meta = 0). The virtual storage layer can retrieve the chunks of the object by broadcasting the user key 201 and the ID 202 filled with "don't care" bits (multi-bits of arbitrary data, such as 0xFF) to all data storage devices. If the ID is "don't care", then the ID field is ignored. It can be assumed that the data storage device effectively implements the group feature. For example, the trie structure can easily identify an object with a given first code of the user key 201, but the hash table can only be found in the hash bucket using the user key when the metadata field is fixed object. The virtual storage layer may sort the returned object chunks in ascending order based on the belt ID 202 of each device, reconstruct the belt and then reconstruct the object, and return the object with the user key 201.

FIG. 3 illustrates an example of object retrieval using group features according to one embodiment. Each of the disks, that is, disk 1 to disk D and disk D + 1 to disk P, the data storage device assigned to store objects supports the group feature. In this case, the band ID 302 is set to "don't care", indicating that the band ID 302 is ignored. The virtual storage layer uses the user key 301 to collect the blocks belonging to the band 350 (ie, data block 1, data block 2, ..., co-located block 1 to co-located block P), and uses the band 350 reconstruction to include the block 1 Go to the first fragment of block D. Subsequently, the virtual storage layer collects the remaining chunks and reconstructs the remaining fragments in turn. Once all the fragments are constructed, the virtual storage layer uses the fragments to reconstruct an object 310 containing data blocks 1 to 3D. In the case of erasure coding, the virtual storage layer further uses the co-located block 1 to the co-located block P to further reconstruct the co-located block. This example illustrates a band-priority scheme for distributing blocks; however, it should be understood that the split-priority scheme can be applied to block distribution and object retrieval without departing from the scope of the present invention.

FIG. 4 illustrates an example of object retrieval without group features according to an embodiment. In this case, the virtual storage layer attaches additional metadata to large or oversized objects (that is, SZ_meta ≠ 0) because the data storage device assigned to the storage object does not support the group feature. Each tile can be identified by a band ID 402 having a length of 1 byte. In this example, there are three bands, band 0, band 1, and band 2, so that the number of bands can be adapted to a 1-byte length. The virtual storage layer can use the ID 402 to build fragments one by one. First, the virtual storage layer broadcasts a user key 401 with a band ID (BID = 0) equal to 0 to all data storage devices. The virtual storage layer receives the block with band 0 from the data storage device, and retrieves the band information using the block from the received block belonging to band 0. Based on the received tape information, the virtual storage layer knows the number of tapes to retrieve objects. If the object is large, then there may be only one band, so the virtual storage layer can use the blocks in the band to reconstruct the entire object. If there are more than one zone for an oversized object, the virtual storage layer needs to retrieve more zones one by one (such as zone 1 and zone 2). In this case, the virtual storage layer broadcasts the retrieval request by adjusting the band ID (for example, BID = 1 or BID = 2) until it retrieves all the blocks. Once the virtual storage layer constructs all the fragments, the virtual storage layer can reconstruct the object 410. It should be noted that gadgets may not have metadata regardless of whether the device supports group features. By checking the block size, the virtual storage layer can use inequality (7) and inequality (10) to determine whether the object 410 is small.

For writing oversized objects to one or more data storage devices, the oversized objects can be split into × D blocks of the same size, ie Clips. Considering the alignment requirements, the last data block can be filled with zeros (for example, data 4a of object 510a and data 4b of object 510b in Figure 5 ), and P parity can be generated from D data blocks of each segment Block.

FIG. 5 illustrates an example of erasure coding without a dedicated parity device according to one embodiment. FIG. 6 illustrates an example of erasure coding with one or more dedicated parity devices according to one embodiment. A total of (D + P) blocks containing D data blocks and P co-located blocks in each band are distributed on one or more data storage devices to write all Band. Co-located block may be distributed on D + P devices (e.g. FIG. 5 SSD 4 to SSD 6), or P may be stored on a dedicated apparatus (e.g., in FIG. 6 SSD 5 and SSD 6). When there is no ID on the data storage device, the hash value of the user key (hereinafter referred to as "Hash (user key)") can be used to select the main data storage device. All may be selected (D + P) th device (D + P) th subset of devices in the example of FIG. 5, a device D and may be selected in the example of FIG. 6. If there is no dedicated peer device, the starting device can be determined by Hash (user key)% (D + P), or if there is a dedicated peer device, it can be determined by Hash (user key)% D. Subsequent blocks can be sequentially written to the next device, such as (Hash (user key) + 1)% (D + P), (Hash (user key) + 2)% (D + P), ..., (Hash (user key) + D + P-1)% (D + P) or (Hash (user key) + 1)% D, (Hash (user key) + 2)% D, ..., (Hash ( user key) + D-1)% D. This operation is performed on each zone, and the virtual storage layer This process is repeated for each band to write pieces of the object. A hash of the user key may need to be calculated once for each object.

If the group does not support data storage device wherein, the block having the additional information with the relay ID and the total number of bands, as shown in FIG. 4. The number of bands can be determined by equation (3). A block in a band can have a pair (with ID, ) As metadata.

Referring to FIG. 5 , the virtual storage layer stores data blocks (data 1a to 4a) and parity blocks (parity 1a and parity 2a) of the object 510a on the data storage device SSD1 to the data storage device SSD6. The virtual storage layer stores data blocks (data 1b to data 4b) and parity blocks (parity 1b and parity 2b) of another object 510b on the data storage device SSD1 to the data storage device SSD6. The starting device (eg, SSD1 of object 510a and SSD6 of object 510b) can be determined by the hash value of the user key, as discussed above. Because there is no dedicated parity device, the virtual storage layer can distribute data blocks and parity blocks on the data storage device SSD1 to the data storage device SSD6 without distinguishing data blocks from parity blocks. In this example, SSD4 and SSD6 include data blocks and co-located blocks.

Referring to FIG. 6 , the virtual storage layer stores data blocks (data 1a to 4a) and parity blocks (parity 1a and parity 2a) of the object 610a on the data storage device SSD1 to the data storage device SSD6. Similarly, the virtual storage layer stores data blocks (data 1b to 4b) and parity blocks (parity 1b and parity 2b) of the object 610b on the data storage device SSD1 to the data storage device SSD6. This is because SSD5 and SSD6 are assigned as co-located devices, and SSD 5 and SSD 6 only contain co-located blocks.

For writing large objects to one or more data storage devices, the large objects can be split into × D blocks of the same size. Large objects can be handled similarly to oversized objects, except that there may be only one band of objects, namely .

For storing small objects, you can create (P + 1) duplicate objects for the object. Taking into account the alignment with padding, duplicate objects can be distributed across (P + 1) devices. The primary device can be selected on (D + P) devices using the hash value of the user key. The P duplicates can be deterministically selected based on various factors such as storage organization, performance, and the like. For example, the copied object can be simply stored in (Hash (key) +1)% (D + P), (Hash (key) +2)% (D + P), ..., (Hash (key) + P)% (D + P), or on different nodes and racks, regardless of whether there is no dedicated co-location device. If there is a dedicated peer device or node, the copied object can be stored on (Hash (key) +1)% D, (Hash (key) +2)% D, ..., (Hash (key) + P)% D . Regardless of device performance, gadgets may not have metadata.

FIG. 7 illustrates an example replication scheme for small objects on one or more data storage devices without a co-located device, according to one embodiment. The virtual storage layer can store small objects 710a (object 1) on the data storage device SSD1, the data storage device SSD2, and the data storage device SSD3. The virtual storage layer can store small objects 710b (object 2) in blocks on the data storage device SSD3, the data storage device SSD4, and the data storage device SSD5. It should be noted that the small objects 710a and 710b are not split into smaller data chunks. The starting device for storing the object 710a and the object 710b may be determined by the hash value of the corresponding user key. In this example, the starting device of the object 710a is SSD1, while the starting device of the object 710b is SSD3. In this example, the total number (P + 1) of duplicated objects of each of the object 710a and the object 710b is 3 (ie, P = 2).

FIG. 8 illustrates an example flowchart for writing an object according to one embodiment. The virtual storage layer of the data storage system receives a write request to write an object (801). A write request received from a user (or a user application running on a host) can include a user key. The virtual storage layer uses, for example, inequality (8) and inequality (9) to determine whether an object is oversized or large (802). For large objects or very large objects, the virtual storage layer uses equations (3) and (4) to determine the block size and number of blocks (811) for each split and each band, and writes data to the data One or more tapes on the storage device (812), and complete the write process (815).

If the virtual storage layer determines that the object is neither large nor oversized, the virtual storage layer further determines whether the object is small (803), for example, using formula (7). For small objects, the virtual storage layer determines one or more data storage devices to store the data containing the original data and the copied data based on the distribution strategy (813), writes the data to the tape on one or more devices (814), and Complete the write process (815). For example, the virtual storage layer can adopt a priority policy (distribute data across multiple data storage devices). The virtual storage layer may use the hash value of the user key to determine the starting device.

If the virtual storage layer determines that the object is neither oversized, large, or small, then the virtual storage layer treats the object as medium (804), and determines one or more data storage devices to store the original data and copy based on the distribution strategy. The data of the data (813), writing the data to the tape on one or more devices (814), and completing the writing process (815).

The process 812 of writing data to one or more bands on a data storage device may include several sub-processes. First, the virtual storage layer determines whether there are any fragments to be written (820). If there are no fragments to be written, the virtual storage layer saves the error log, terminates the operation, and notifies the user application and / or the host's operating system running on the host that sent the request to write the object. The virtual storage layer creates fragments for objects and internal keys for the tiles in the fragments (821). The virtual storage layer checks if this is the last fragment to be written (822). For the last segment, if the segment cannot completely fill the band, the virtual storage layer may add padding to the data segment (823) and use an erasure coding algorithm to calculate the P parity segments of the segment ( 824). For segments other than the last segment, the virtual storage layer may skip the fill process 823. The virtual storage layer further determines a device for data partitioning and parity partitioning based on the distribution strategy (825). For example, the distribution strategy may be a split-first strategy or a band-first strategy. The virtual storage layer writes data chunks and parity chunks in a band with internal keys to one or more data storage devices (826), the internal keys being generated in 821.

The virtual storage layer may not know whether the object to be read is small, large, or oversized, because the virtual storage layer does not maintain object metadata such as user keys and value sizes. Therefore, the virtual storage layer uses the user keys of the object to broadcast read requests to all data storage devices (such as (D + P) data storage devices) in the virtual storage, and determines the appropriate way to reconstruct the object based on the classification of the object size. Depending on the group characteristics supported by the data storage device, the virtual storage layer can perform read operations and write operations in different ways.

Very large objects can be read in different ways based on the data storage device's support for group characteristics. If the data storage device in the virtual storage supports the group feature, the virtual storage layer broadcasts a user key with BID = "don't care" to all data storage devices. In response, if there is no error in the response to the broadcast, then each of the (D + P) data storage devices returns its Blocks. The virtual storage layer then classifies the received chunks into each Bands, and sort the bands in ascending order of ID. As long as the virtual storage layer receives any D tiles of the same size for each band, the virtual storage layer can reconstruct the band. If the total number of received blocks of the tape is less than the number of data storage devices D or the sizes of the blocks are not the same, an error occurs. Errors can be caused by reading non-existent objects with a NOT_EXIST error or an unrecoverable error returned by all data storage devices. The virtual storage layer reconstructs the strips one by one, and then uses the strips to reconstruct the objects.

If the data storage device in the virtual storage does not support the group feature, then the virtual storage layer broadcasts a user key with BID = 0 to all data storage devices to read oversized objects. In response, each of the (D + P) data storage devices returns a block in the first band (Band 0). The virtual storage layer can identify the number of bands of an object by examining metadata stored with any of the received tiles. The virtual storage layer then reconstructs all the bands one by one with increasing power of the band ID, and uses the band reconstruction objects. In some embodiments, the virtual storage layer may request all remaining objects that are similar to the group support situation by asynchronously requesting.

The reading process for large objects can be similar to the reading process for very large objects, except that it has only one band. If the data storage device supports the group feature, the virtual storage layer broadcasts a user key with BID = "don't care" to all data storage devices. In response, each of the (D + P) data storage devices returns a block in its split. As long as the virtual storage layer receives D blocks of the same size, the virtual storage layer can reconstruct objects. If the total number of received blocks of the band is less than the number of data blocks D or the size of the blocks is not the same, an error will occur. Errors can be caused by reading non-existent objects with a NOT_EXIST error or an unrecoverable error returned by all data storage devices.

If the data storage device in the virtual storage does not support the group feature, the virtual storage layer broadcasts a user key with BID = 0 to all data storage devices to read large objects. In response, each of the (D + P) data storage devices returns a block in the band (BID = 0). The virtual storage layer recognizes that there is only one band for a large object by examining the metadata stored with any of the received chunks, and reconstructs the object by reconstructing the band.

The reading process of small objects can be similar to the reading process of large objects, except that it is by means of copying. If the data storage device supports the group feature, the virtual storage layer broadcasts a user key with BID = "don't care" to all data storage devices. In response, each of the (D + P) data storage devices returns a response. Because the objects are small, some data storage devices can return NOT_EXIST errors, while other data storage devices return blocks. The virtual storage layer receives any valid chunk of the object that can be reconstructed using the chunks received from one or more data storage devices. If all data storage devices return a NOT_EXIST error, the object identified by the read request may be a non-existent object, or an unrecoverable error may occur.

If the data storage device in the virtual storage does not support the group feature, the virtual storage layer broadcasts a user key with BID = 0 to all small data storage devices to read small objects. In response, each of the (D + P) data storage devices returns a response. The virtual storage layer can identify objects as small using any valid block received from any data storage device. It should be noted that small objects do not hold extra metadata.

FIG. 9 illustrates an example flowchart for reading an object according to one embodiment. The virtual storage layer receives a read request of the object (901). A read request received from a user (or a user application running on a host) may include a user key. The virtual storage layer uses one or more data storage devices that store one or more chunks of the object to determine whether the group feature is supported (902). If the group feature is supported, the virtual storage layer broadcasts a read request with an internal key with BID = "don't care" to all data storage devices (903), otherwise, sets BID = 0 (904) for all data storage devices. The virtual storage layer receives blocks from one of the data storage devices (905). If there are no errors in the received block (906), the virtual storage layer determines whether the group feature is supported (907) and retrieves metadata in the block (908), otherwise, the virtual storage layer continues to determine the size of the object.

If the virtual storage layer determines that the object is oversized or large (909), the virtual storage layer further checks whether the entire object is reconstructed (910) and continues to reconstruct fragments one by one using the blocks received from one or more data storage devices until Until the entire object is reconstructed (912), and the reading process is completed (913). For small objects, the virtual storage layer uses chunked reconstructed objects received from one or more data storage devices (911).

The process 911 of reconstructing the gadget may include several child processes. The virtual storage layer first checks whether the received chunk is small (921). When the virtual storage layer expects small chunks of small objects but receives large chunks, the virtual storage layer generates an error (924). If the virtual storage layer determines that the received block is valid (922), the virtual storage layer reconstructs the small object with the received block (923); otherwise, the virtual storage layer receives the block from another data storage device ( 925).

The process 912 of reconstructing a segment may include several child processes. The virtual storage layer checks whether all D tiles used to reconstruct the segment are received (931). If so, then the virtual storage layer reconstructs the segment using the D block using an erasure coding algorithm (935). If not, the virtual storage layer further checks whether all the tiles in the current band have been received (932). If all the tiles in the band are received, the virtual storage layer generates an error (936), because at least one of the received tiles may not be valid. If there are more chunks to be received, the virtual storage layer continues to receive the chunks from another data storage device (933) and repeats the process until all D chunks for the reconstructed segment are received. If any of the received chunks are neither large nor oversized (eg, chunks are for small objects), the virtual storage layer generates an error (936).

According to one embodiment, a data storage system includes: a plurality of data storage devices for storing a plurality of objects of a key-value pair; and a virtual storage layer that applies different data reliability schemes based on the size of an object among the plurality of objects The different data reliability schemes include data replication schemes and erasure coding schemes. The plurality of objects include a first object having a first size and a second object having a second size, and the second size is larger than the first size. The virtual storage layer classifies the first object as a small object, applies a data copying scheme, and stores the small object on one or more of a plurality of data storage devices. The virtual storage layer classifies the second object as an oversized object, splits the oversized object into one or more blocks of the same size, applies an erasure coding scheme, and stores one or more blocks in multiple data stores in a distributed manner. Device.

The distribution scheme may be a split-first distribution, in which a virtual storage layer splits one or more blocks of an oversized object into each of a plurality of data storage devices and divides each of the one or more blocks into A block is stored on multiple data storage devices.

The distribution scheme may be a band-first distribution scheme, in which a virtual storage layer stores one block of an oversized object to each of a plurality of data storage devices until one or more blocks of the oversized object are completely stored in one or more data To the storage device.

The virtual storage layer can further classify a third object with a third size into a large object, split the large object into one or more blocks of the same size, apply an erasure coding scheme, and decentralize one or more blocks. The ground is stored on a plurality of data storage devices, wherein the third size is larger than the first size and smaller than the second size, and wherein the large object has only one band or only one block within a split.

The virtual storage layer may further classify a fourth object having a fourth size into a medium object, where the fourth size is larger than the first size and smaller than the third size, and wherein the virtual storage layer applies one of a data copying scheme and an erasure coding scheme .

User keys can be used to identify objects, and the virtual storage layer can create internal keys with user codes and oversized objects with identification codes, where the belt identification code is used to identify one of the multiple tapes, and each of the multiple tapes The band contains one of one or more blocks distributed across multiple data storage devices.

The virtual storage layer may use a hash value of a user key for writing or reading a first block of one or more blocks to identify a starting data storage device among the plurality of data storage devices.

The plurality of data storage devices may include one or more dedicated parity data storage devices that store parity blocks associated with oversized objects.

Multiple data storage devices can support group characteristics, and the virtual storage layer can broadcast to multiple data storage devices, where the identification code is set to multiple bits of arbitrary data when reading oversized objects.

Multiple data storage devices may not support the group feature, and the virtual storage layer may broadcast to multiple data storage devices, where the identification code is set as the unique identification code when reading oversized objects.

Each of the plurality of data storage devices may be a key-value solid state drive (KV SSD).

According to another embodiment, a method for writing an object of a key-value pair includes: receiving a plurality of objects of the key-value pair, wherein the plurality of objects include a first object having a first size and a second object having a second size , The second size is larger than the first size; classifying the first object as a small object; applying a data replication scheme to the small object; storing the small object on one or more of a plurality of data storage devices; classifying the second object as Oversized objects; splitting oversized objects into one or more blocks of the same size; applying an erasure coding scheme to oversized objects; and decentrally storing one or more blocks on multiple data storage devices.

The method may further include: receiving a write request for the object; determining that the object is an oversized object; determining the block size and number of blocks of the oversized object; and dividing one or more of the oversized object based on the block size and the number of blocks Write to multiple data storage devices.

The method may further include creating a segment containing a segment for each of the plurality of data storage devices among the one or more segments, and using a user key to create an internal key that is additionally used in the segment A band identification code for each of the contained one or more blocks; a band is created using an erasure coding scheme, the band containing one or more co-located blocks and one or more blocks corresponding to the segment ; Determining multiple data storage devices to store tapes based on a distribution scheme; and writing one or more chunks in a tape with internal keys.

The method may further include: receiving a write request for the object; determining that the object is a small object; determining a subset of the plurality of data storage devices to store one or more blocks of the small object based on the distribution scheme; and Multiple blocks are written to a subset of multiple data storage devices.

The method may further include: receiving a read request for an object containing a user key; determining whether a plurality of data storage devices support a group feature; and broadcasting a read request with an internal key to the plurality of data storage devices.

If the group feature is supported, the ID with the internal key can be set to multiple bits of any data.

If the group feature is not supported, the ID with internal key can be set as the unique ID.

The method may further include: receiving at least one chunk from each of the plurality of data storage devices; and using an erasure coding scheme to reconstruct the segment using the at least one chunk received from the plurality of data storage devices.

Each of the plurality of data storage devices may be a key-value solid state drive (KV SSD).

The above example embodiments have been described above to illustrate various embodiments of implementing a data storage system capable of efficiently storing objects having different sizes and a method of storing those objects in the data storage system. Those of ordinary skill in the art will make various modifications and departures from the disclosed example embodiments. The subject matter which is intended to be within the scope of the invention is set forth in the appended claims.

110, 310, 410, 510a, 510b, 610a, 610b, 710a, 710b‧‧‧ objects

120, 121, 321, 421‧‧‧ Virtual storage

150, 350‧‧‧ belt

151‧‧‧ split

200‧‧‧ Internal key

201, 301, 401‧‧‧ user keys

202, 302, 402‧‧‧ with identification code

801, 802, 803, 804, 811, 812, 813, 814, 815, 820, 821, 822, 823, 824, 825, 826, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 921, 922, 923, 924, 925, 931, 932, 933, 934, 935, 936‧‧‧ steps

G‧‧‧Bytes

L, L-G‧‧‧ maximum key length

SSD1 ～ SSD6‧‧‧ Data Storage Device

The accompanying drawings, which are included as part of this specification, illustrate preferred embodiments of the present invention and, together with the general description given above and the detailed description of the preferred embodiments given below, are used to clarify and impart the principles described herein .

FIG. 1 is a schematic diagram of objects stored in an example data storage system according to an embodiment.

FIG. 2 illustrates an example internal key including a user key according to one embodiment.

FIG. 3 illustrates an example of object retrieval using group features according to one embodiment.

FIG. 4 illustrates an example of object retrieval without group features according to an embodiment.

FIG. 5 illustrates an example of erasure coding without a dedicated parity device according to one embodiment.

FIG. 6 illustrates an example of erasure coding with one or more dedicated parity devices according to one embodiment.

FIG. 7 illustrates an example replication scheme for small objects on one or more data storage devices without a co-located device, according to one embodiment.

FIG. 8 illustrates an example flowchart for writing an object according to one embodiment.

FIG. 9 illustrates an example flowchart for reading an object according to one embodiment.

The drawings are not necessarily drawn to scale, and throughout the drawings, for the purpose of illustration, elements of similar structure or function are generally denoted by the same reference numerals. The drawings are only intended to facilitate the description of the various embodiments described herein. The drawings do not describe every aspect of the teachings disclosed herein and do not limit the scope of the claims.

Claims (20)

A data storage system including: Multiple data storage devices for storing multiple objects of key-value pairs; and The virtual storage layer applies different data reliability schemes based on the size of one of the plurality of objects. The different data reliability schemes include a data replication scheme and an erasure coding scheme. The plurality of objects include a first object having a first size and a second object having a second size, the second size is larger than the first size, The virtual storage layer classifies the first object as a small object, applies the data copying scheme, and stores the small object on one or more of the plurality of data storage devices, and The virtual storage layer classifies the second object into an oversized object, splits the oversized object into one or more blocks of the same size, applies the erasure coding scheme, and divides the one or more Blocks are stored on the plurality of data storage devices in a distributed manner. The data storage system according to item 1 of the scope of patent application, wherein the virtual storage layer stores the one or more blocks on the plurality of data storage devices in a split-first manner, and in a split-first manner Wherein the virtual storage layer stores some of the one or more chunks of the oversized object to one of the plurality of data storage devices and stores other ones of the one or more chunks Stored to another one of said plurality of data storage devices. The data storage system according to item 1 of the scope of patent application, wherein the virtual storage layer stores the one or more blocks on the plurality of data storage devices in a band priority distribution scheme, and the band storage priority In the distribution scheme, the virtual storage layer stores a block of the oversized object to each of the plurality of data storage devices until the one or more blocks of the oversized object are completely stored in the Multiple data storage devices. The data storage system according to item 1 of the scope of patent application, wherein the virtual storage layer further classifies a third object having a third size into a large object, and splits the large object into one or more of the same size. Chunking, applying the erasure coding scheme, and storing the one or more chunks on the plurality of data storage devices in a distributed manner, wherein the third size is larger than the first size and smaller than the The second size, and wherein the large object has only one band or only one block within one split. The data storage system according to item 4 of the scope of patent application, wherein the virtual storage layer further classifies a fourth object having a fourth size into a medium object, wherein the fourth size is larger than the first size and smaller than The third size is described, and wherein the virtual storage layer applies one of the data copying scheme and the erasure coding scheme. The data storage system according to item 1 of the scope of patent application, wherein a user key is used to identify the object, and the virtual storage layer creates an internal key with an identification code including the user key and the oversized object, where The band identification code is used to identify one of a plurality of bands, and each of the plurality of bands includes the one or more blocks distributed on the plurality of data storage devices. One block. The data storage system according to item 6 of the patent application scope, wherein the virtual storage layer uses a hash of the user key for writing or reading a first one of the one or more blocks Value to identify a starting data storage device among the plurality of data storage devices. The data storage system according to item 1 of the scope of patent application, wherein the plurality of data storage devices include one or more dedicated parity data storage devices, and the dedicated parity data storage devices store parity associated with the oversized object Block. The data storage system according to item 6 of the patent application scope, wherein the plurality of data storage devices support a group feature, and when reading the oversized object, the virtual storage layer provides the data storage device to the plurality of data storage devices. The band identification code set as a multi-bit of arbitrary data is broadcast. The data storage system according to item 6 of the scope of patent application, wherein the plurality of data storage devices do not support a group feature, and when reading the oversized object, the virtual storage layer stores data to the plurality of data. The device broadcasts the tape identification code set as the unique tape identification code. The data storage system according to item 1 of the patent application scope, wherein each of the plurality of data storage devices is a key-value solid state drive. A method for writing an object of a key-value pair, the method includes: Receiving a plurality of objects of key-value pairs, wherein the plurality of objects include a first object having a first size and a second object having a second size, and the second size is larger than the first size; Classifying the first object as a small object; Applying a data copying scheme to said small objects; Storing the small object on one or more of a plurality of data storage devices; Classifying the second object as an oversized object; Splitting the oversized object into one or more pieces of the same size; Applying an erasure coding scheme to the oversized object; and The one or more blocks are stored on the plurality of data storage devices in a distributed manner. The method described in item 12 of the patent application scope further includes: Receiving a write request for the object; Determining that the object is the oversized object; Determining the block size and number of blocks of the oversized object; and Writing the one or more blocks of the oversized object to the plurality of data storage devices based on the block size and the number of blocks. The method described in item 13 of the patent application scope further includes: Creating a segment containing a segment for each of the plurality of data storage devices among the one or more segments, and using a user key to create an internal key, the user key is additionally used for the segment A band identification code for each of the one or more sub-blocks included in; Creating a band using the erasure coding scheme, the band including one or more co-located blocks and the one or more blocks corresponding to the segment; Determining the plurality of data storage devices to store the tape based on a distribution scheme; and The one or more blocks in the band having the internal key are written. The method described in item 12 of the patent application scope further includes: Receiving a write request for the object; Determining that the object is the small object; Determining a subset of the plurality of data storage devices to store one or more chunks of the gadget based on a distribution scheme; and Write the one or more chunks of the gadget to the subset of the plurality of data storage devices. The method described in claim 14 of the patent application scope further includes: Receiving a read request for the object containing the user key; Determining whether the plurality of data storage devices support a group feature; and The read request having the internal key is broadcast to the plurality of data storage devices. The method according to item 16 of the scope of patent application, wherein if the group feature is supported, the identification code of the internal key is set to a multi-bit of arbitrary data. The method according to item 16 of the scope of patent application, wherein if the group feature is not supported, the band identification code of the internal key is set as a unique band identification code. The method described in claim 16 of the scope of patent application further includes: Receiving at least one chunk from each of the plurality of data storage devices; and Using the erasure coding scheme to reconstruct a segment using the at least one chunk received from the plurality of data storage devices. The method of claim 12, wherein each of the plurality of data storage devices is a key-value solid state drive.