KR102067440B1 - Method and Apparatus for hierarchical RAID with erasure coding accelerating based on virtual GPU - Google Patents

Abstract

Disclosed are a method and an apparatus for a hierarchical RAID applied with erasure code acceleration in a virtualization GPU environment. The method for a hierarchical RAID applied with erasure code acceleration in a virtualization GPU environment of the present invention comprises the steps of: allowing a master virtual machine to divide input data into data chunks; generating an external parity disk in a host from the input data; allowing a GPU kernel to calculate external parity data to transfer the same to a CPU memory; allowing the master virtual machine to store the external parity disk and the external parity data in an external parity node; generating an internal parity disk in an internal data node; allowing a slave virtual machine to calculate internal parity data to transfer the same to a GPU memory from the CPU memory; allowing the GPU kernel to call a multi-GPU thread in order to generate internal parity data in parallel; and allowing the slave virtual machine to store the internal parity data and the internal parity disk in the internal data node.

Description

Method and Apparatus for Hierarchical RAID with Eraser Code Acceleration in Virtualized GPU Environments {Method and Apparatus for hierarchical RAID with erasure coding accelerating based on virtual GPU}

The present invention relates to a data loss recovery method and apparatus applying the hierarchical RAID method. More specifically, the present invention relates to a layered method and apparatus to which erasure code acceleration is applied in a virtualized GPU environment.

In cloud storage systems, erasure code is used to create a parity disk or to recover a lost data disk. To cope with these loss problems, storage systems require erasure code. The encoding scheduler creates parity disks, while the decoding scheduler uses redundant parity disks to recover lost data disks. Parity disks take up less storage space compared to replicated data disks such as RAID 1. However, the erasure code must perform many XOR operations, which can increase write overhead because encoding and decoding erasure codes can occupy the CPU for a long time and cause data bottlenecks.

In parallel operations using GPGPU, data bottlenecks occur during host-to-device transfer. In order to move data from the host system to the GPU, the data must be copied from the host buffer to the GPU buffer, and the results must be similarly copied again. For this reason, parallel computing using large amounts of data causes data bottlenecks due to buffer size constraints.

An object of the present invention is to provide a data loss recovery method and apparatus using a hierarchical RAID method. By suggesting the RAID 55 and RAID 66 schemes, we have constructed a system that can recover more than three data loss while having higher availability than the existing schemes. To reduce the phenomenon.

In one aspect, the layered method to which the erasure code acceleration is applied in the virtualized GPU environment proposed by the present invention, the master virtual machine divides the input data into data chunks, and generates an external parity disk in the host from the input data A step in which the GPU kernel calculates and passes external parity data to the CPU memory, the master virtual machine stores the external parity disk and the external parity data in the external parity node, generates an internal parity disk in the internal data node, Slave virtual machine computes internal parity data and passes it from CPU memory to GPU memory; GPU kernel calls multi-GPU threads to generate internal parity data in parallel; slave virtual machine uses internal parity data and Internal parity disk And storing the section data node.

In the step of creating an external parity disk in the host from the input data, the master virtual machine distributes all data chunks to corresponding internal data nodes via the slave virtual machine.

The step in which the GPU kernel calculates and passes external parity data to the CPU memory is calculated by performing XOR operation between corresponding data chunks in various internal data nodes.

To improve the performance of external parity data generation, the master virtual machine passes all data chunks from CPU memory to GPU memory, and the GPU kernel creates multiple-GPU threads to generate external parity data in parallel. Call

When the slave virtual machine stores the internal parity data and the internal parity disk in the internal data node, when the internal parity data is lost, the corresponding slave virtual machine uses the internal parity disk to recover the lost internal parity data.

The recovered internal parity data is transferred from GPU memory to CPU memory, and the recovered internal parity data is stored on a spare disk in the same internal data node.

In another aspect, the layered device to which the erasure code acceleration is applied in the virtualized GPU environment proposed by the present invention divides the input data into data chunks to generate an external parity disk in the host from the input data, and the GPU kernel It computes the external parity data and transfers it to the CPU memory, creates a master virtual machine that stores the external parity disk and the external parity data in the external parity node and an internal parity disk in the internal data node, and calculates the internal parity data in the CPU memory. It delivers to GPU memory and includes a slave virtual machine that the GPU kernel calls multi-GPU threads to generate internal parity data in parallel, and stores internal parity data and internal parity disks in internal data nodes.

The master virtual machine distributes all data chunks to corresponding internal data nodes through the slave virtual machine, and external parity data is calculated by performing XOR operations between corresponding data chunks in various internal data nodes, and external parity data. All data chunks are passed from CPU memory to GPU memory to improve generation performance, and the GPU kernel invokes multiple-GPU threads to generate external parity data in parallel.

When the internal parity data is lost, the slave virtual machine uses an internal parity disk to recover the lost internal parity data, and the recovered internal parity data is transferred from GPU memory to CPU memory. Internal parity data is stored on a spare disk in the same internal data node.

According to the embodiments of the present invention, unlike the simple data replication method of the existing mirroring method, parity may be generated in units of nodes in which several disks in the system are bundled to secure high storage availability. In addition, pass-through GPU virtualization prevents performance degradation by effectively handling a large amount of XOR operations that can occur with parity operations on all nodes.

1 is a flowchart illustrating a layered method to which erasure code acceleration is applied in a virtualized GPU environment according to an embodiment of the present invention.
2 is a hierarchical RAID structure to which pass-through GPUrk is applied according to an embodiment of the present invention.
3 is a view for explaining a single disk loss according to an embodiment of the present invention.
4 is a diagram illustrating a multiple disk loss according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a recovery algorithm for node loss occurring in a hierarchical RAID according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a parity calculation matrix of a layer RAID 55 according to an embodiment of the present invention.

The present invention proposes a data loss recovery scheme using a hierarchical RAID method. By suggesting the RAID 55 and RAID 66 schemes, we have constructed a system that can recover more than three data loss while having higher availability than the existing schemes. Reduced the phenomenon. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a layered method to which erasure code acceleration is applied in a virtualized GPU environment according to an embodiment of the present invention.

In the proposed layered method using the eraser code acceleration in the virtualized GPU environment, the master virtual machine divides the input data into data chunks (110), generates an external parity disk in the host from the input data (120), In step 130, the GPU kernel calculates and transmits the external parity data to the CPU memory, in which the master virtual machine stores the external parity disk and the external parity data in the external parity node (140), and stores the internal parity disk in the internal data node. In step 150, the slave virtual machine calculates the internal parity data and passes it from the CPU memory to the GPU memory 160.The GPU kernel calls the multi-GPU thread to generate the internal parity data in parallel. Step 170 and slave virtual machine internal internal parity data and internal parity disk And a step 180 of storing the data node.

In step 110, the master virtual machine divides the input data into data chunks, and in step 120, creates an external parity disk in the host from the input data. At this time, the master virtual machine distributes all data chunks to corresponding internal data nodes through the slave virtual machine.

In step 130, the GPU kernel calculates and transmits the external parity data to the CPU memory. External parity data is computed by performing XOR operations between corresponding data chunks within the various internal data nodes. To improve external parity data generation performance, the master virtual machine passes all chunks of data from CPU memory to GPU memory, and the GPU kernel creates multiple-GPU threads to generate external parity data in parallel. Call

In step 140, the master virtual machine stores the external parity disk and external parity data in the external parity node.

In step 150, an internal parity disk in an internal data node is created, and in step 160, a slave virtual machine computes and passes internal parity data from CPU memory to GPU memory.

In step 170, the GPU kernel invokes a multi-GPU thread to generate internal parity data in parallel, and in step 9180 the slave virtual machine stores the internal parity data and the internal parity disk in an internal data node. At this time, if the internal parity data is lost, the corresponding slave virtual machine uses the internal parity disk to recover the lost internal parity data. The recovered internal parity data is transferred from GPU memory to CPU memory, and the recovered internal parity data is stored on a spare disk in the same internal data node.

2 is a hierarchical RAID structure to which pass-through GPUrk is applied according to an embodiment of the present invention.

Referring to FIG. 2, each node has a plurality of storages, and each node is configured with RAID 5. In addition, by creating an external parity node stored by performing parity operation for each node, the RAID of the hierarchical structure is constructed.

데이터 청크인

로 분할하며, 여기서 n은 슬레이브 VM의 수를, p를 각 슬레이브 VM을 위한 전용 데이터 디스크의 수를 나타낸다. 마스터 VM은 모든 데이터 청크들을 대응하는 내부 데이터 노드들(internal data nodes)로 슬레이브 VM을 통해 분배한다. 마스터 VM은 입력 데이터로부터 호스트 내에 외부 패리티 디스크(external parity disk)

를 생성한다. 외부 패리티 데이터(external parity data)

는 다양한 내부 데이터 노드 내에서 대응하는 데이터 청크

간의 XOR 연산을 수행함으로써 계산된다. 외부 패리티 데이터 생성 성능을 향상시키기 위하여, 마스터 VM은 모든 데이터 청크들을 CPU 메모리에서 GPU 메모리로 전달한다. GPU 커널은 외부 패리티 데이터를 병렬적으로 생성하기 위하여 다중-GPU 쓰레드(multiple-GPU threads)를 호출한다. 그 이후, GPU 커널은 외부 패리티 데이터를 CPU 메모리로 다시 전달한다. 마지막으로, 마스터 VM은 외부 패리티 데이터를 외부 패리티 노드에 저장한다. We show a proposed layered RAID using pass-through GPUs to improve encoding performance in cloud storage systems. One master virtual machine (VM) and several slave virtual machines (VM) are used to construct the proposed layer RAID, which is composed of vertical and horizontal stripes. The encoding process is described using the proposed layer RAID as follows. For encoding, assume that the client requests write input data. Then, the master VM

Data Chunk In

Where n is the number of slave VMs and p is the number of dedicated data disks for each slave VM. The master VM distributes all data chunks to corresponding internal data nodes through the slave VM. The master VM has an external parity disk in the host from the input data.

Create External parity data

The corresponding data chunks within the various internal data nodes

Calculated by performing an XOR operation on the liver. To improve external parity data generation performance, the master VM passes all data chunks from CPU memory to GPU memory. The GPU kernel calls multiple-GPU threads to generate external parity data in parallel. After that, the GPU kernel passes external parity data back to the CPU memory. Finally, the master VM stores external parity data in the external parity node.

는 내부 데이터 노드 내에서 데이터 청크

간의 XOR 연산을 수행함으로써 계산된다. 내부 패리티 데이터 생성 성능을 향상시키기 위하여, 슬레이브 VM은 데이터를 CPU 메모리에서 GPU 메모리로 전달한다. 그 이후, GPU 커널은 내부 패리티 데이터를 병렬적으로 생성하기 위하여 다중-GPU 쓰레드를 호출한다. GPU 커널은 내부 패리티 데이터를 CPU 메모리로 다시 전달한다. 마지막으로, 슬레이브 VM은 데이터 디스크 및 내부 패리티 디스크에 데이터 및 내부 패리티 데이터를 저장한다. 내부 데이터 디스크가 손실되면(fails), 대응하는 슬레이브 VM은 손실된 내부 데이터 디스크를 복구하기 위하여 내부 패리티 디스크를 사용한다. One internal data node consists of internal data disks and parity disks. Internal parity data

Chunks of data within internal data nodes

Calculated by performing an XOR operation on the liver. To improve internal parity data generation performance, the slave VM transfers data from CPU memory to GPU memory. After that, the GPU kernel calls the multi-GPU thread to generate internal parity data in parallel. The GPU kernel passes internal parity data back to the CPU memory. Finally, the slave VM stores data and internal parity data on the data disk and internal parity disk. If the internal data disk fails, the corresponding slave VM uses the internal parity disk to recover the lost internal data disk.

3 is a view for explaining a single disk loss according to an embodiment of the present invention.

4 is a diagram illustrating a multiple disk loss according to an embodiment of the present invention.

3 and 4 show a recovery algorithm for the disk loss caused by the proposed layer RAID method. In the case of the loss in the disk inside the node, the lost disk is recovered by utilizing the disk data and parity data in the node.

FIG. 5 is a diagram illustrating a recovery algorithm for node loss occurring in a hierarchical RAID according to an embodiment of the present invention.

If a whole node is lost, the entire internal node is recovered by using the external parity data and the data of the nodes except the lost node.

디스크 손실이 발생하면, 대응하는 슬레이브 VM이 생존 디스크들과 내부 패리티 디스크를 함께 읽는다. 단일-디스크 손실에서 디코딩 성능을 향상시키기 위하여, 슬레이브 VM은 남아있는 내부 데이터 디스크들 및 패리티 디스크의 생존 데이터(survival data)를 CPU 메모리에서 GPU 메모리로 전달한다. 그 이후, GPU 커널은 병렬적으로 손실된 디스크를 복구하기 위하여 다중-GPU 쓰레드를 호출한다. 수학식(1)에 나타난 바와 같이, 손실된 디스크

을 복구하기 위해서 XOR 연산들이 수행된다:3, 4 and 5 show the proposed layer RAID using a pass-through GPU to improve decoding performance in cloud storage systems. There are three types of data loss in cloud storage systems. In the case of 1-disk loss, the corresponding slave VM uses surviving disks and internal parity disks to recover the lost disks within the same node. first,

If a disk loss occurs, the corresponding slave VM reads the surviving disks together with the internal parity disk. To improve decoding performance at single-disk loss, the slave VM transfers surviving data of the remaining internal data disks and parity disks from CPU memory to GPU memory. After that, the GPU kernel invokes multiple-GPU threads to recover the lost disks in parallel. As shown in equation (1), the lost disk

XOR operations are performed to recover:

(1)

(One)

은 GPU 메모리에서 CPU 메모리로 전달된다. 마지막으로, 재생된 데이터는 동일한 내부 데이터 노드 내의 예비 디스크(spare disk) 상에 저장된다.After that, the survived data

Is passed from GPU memory to CPU memory. Finally, the reproduced data is stored on a spare disk in the same internal data node.

및

디스크 손실이 발생하면, 영향을 받지 않은 슬레이브 VM들이 생존 데이터 디스크들을 읽고, 마스터 VM이 외부 패리티 디스크들을 함께 읽는다. 다중-디스크 손실에서 디코딩 성능을 향상시키기 위하여, 각 슬레이브 VM은 데이터를 대응하는 슬레이브 VM에서 마스터 VM으로 전달한다. 마스터 VM은 남아있는 내부 데이터 디스크들 및 외부 패리티 디스크들의 생존 데이터를 CPU 메모리에서 GPU 메모리로 전달한다. 그 이후, GPU 커널은 손실된 디스크들을 병렬적으로 복구하기 위하여 다중-GPU 쓰레드를 호출한다. 수학식(2)에 나타난 바와 같이, 손실된 디스크

및

를 복구하기 위하여 XOR 연산을 수행한다.In case of multi-disk loss, the master VM uses external parity disks to recover lost disks and the unaffected slave VMs use surviving data disks. priority,

And

If a disk loss occurs, the unaffected slave VMs read the surviving data disks and the master VM reads the external parity disks together. To improve decoding performance in multi-disk loss, each slave VM transfers data from the corresponding slave VM to the master VM. The master VM transfers surviving data of remaining internal data disks and external parity disks from CPU memory to GPU memory. After that, the GPU kernel invokes multi-GPU threads to recover lost disks in parallel. As shown in equation (2), the lost disk

And

Perform XOR operation to recover.

(2)

(2)

와

는 GPU 메모리에서 CPU메모리로 전달된다. 최종적으로, 재생된 데이터는 대응하는 내부 데이터 노드 내 예비 디스크 내에 저장된다.After that, the survived data

Wow

Is passed from GPU memory to CPU memory. Finally, the reproduced data is stored in the spare disk in the corresponding internal data node.

Finally, in case of node loss, the unaffected slave VMs use surviving data disks and the master VM uses external parity disks to recover the lost node. First, if an internal data Node 1 loss occurs, the unaffected slave VMs read the surviving data disks and the master VM reads the external parity disks together. In order to improve decoding performance in case of node loss, each slave VM transfers data from the corresponding slave VM to the master VM. The master VM transfers surviving data of remaining internal data disks and external parity disks from CPU memory to GPU memory. After that, the GPU kernel calls the multi-GPU threads to recover the lost disks in parallel. As shown in equation (3), an XOR operation is performed to recover the lost node:

(3)

(3)

는 GPU 메모리에서 CPU 메모리로 전달된다. 최종적으로 재생된 데이터는 대응하는 내부 데이터 노드 내 예비 디스크들에 저장된다. 그러나, 노드 손실의 경우, 대응하는 슬레이브 VM은 내부 패리티 데이터를 복구해야 한다. 따라서, 슬레이브 VM은 수학식(4)에 나타난 바와 같이, 손실된 내부 패리티

을 복구하기 위하여 XOR 연산을 수행한다:After that, the survived data

Is passed from GPU memory to CPU memory. The finally reproduced data is stored on spare disks in the corresponding internal data node. However, in case of node loss, the corresponding slave VM must recover the internal parity data. Therefore, the slave VM has lost internal parity as shown in equation (4).

Perform an XOR operation to recover

(4)

(4)

The proposed layer RAID using the GPU has three advantages. First, because tiered RAID uses RAID 55 or 66, it can provide storage space efficiency, so the eraser codes of the proposed tiered RAID consume less storage space than the replication of traditional tiered RAID. . Second, because data chunking occurs within the master VM and uses multiple slave VMs, it can reduce more GPU overhead than traditional layer RAID. Finally, while traditional hierarchical RAID cannot tolerate node loss, this approach can tolerate data node loss through surviving data nodes and external parity nodes.

FIG. 6 is a diagram illustrating a parity calculation matrix of a layer RAID 55 according to an embodiment of the present invention.

Parity stored in each node generates parity using only data used by one virtual machine, and external parity data generates parity by acquiring some of data used by each virtual machine from all virtual machines. .

TABLE 1

개의 데이터 청크인

로 나눠진다. 라인 (2-6)에서, 마스터 VM 인코더는 p 개의 외부 패리티 청크

를 병렬적으로 생성하기 위하여 GPU 커널을 호출한다. GPU 커널은 외부 패리티 청크들을 생성하기 위하여 모든 데이터 청크들과 외부 코딩 행렬 H 에 대한 액세스를 요구한다. 라인 (7)에서, p 개의 외부 패리티 청크들은 마스터 VM 내 CPU 메모리로 전달된다. 마지막으로, 마스터 VM은 라인(8)에서 외부 패리티 청크들을 외부 패리티 디스크들에 쓴다. 도 6은 표 1에 나타난 바와 같이 외부 패리티

를 생성하기 위해 사용되는 이레이져 코딩 방식을 보여준다.Table 1 shows the pseudo-code of the master VM encoder using a pass-through GPU for the proposed layer RAID 55. In line 1, the input data is by the master VM

Chunks of data

Divided by. In line (2-6), the master VM encoder is p outer parity chunks

The GPU kernel is called to generate The GPU kernel requires access to all data chunks and the outer coding matrix H to generate outer parity chunks. In line 7, p external parity chunks are passed to CPU memory in the master VM. Finally, the master VM writes external parity chunks to external parity disks in line 8. 6 shows external parity as shown in Table 1

It shows the erasure coding scheme used to generate the.

TABLE 2

가 슬레이브 VM 내 CPU 메모리에 복사된다. 라인 (2-4)에서, 슬레이브 VM 인코더는 내부 패리티 청크 P _k 를 병렬적으로 생성하기 위하여 GPU 커널을 호출한다. GPU 커널은 내부 패리티 청크를 생성하기 위하여 오직 p 개의 데이터 청크들 및 내부 코딩 행렬 X 에 대한 액세스를 요구한다.Table 2 shows the pseudo-code of k slave VM encoders using a pass-through GPU for the proposed layer RAID 55. In line (1), p data chunks are

Is copied to CPU memory in the slave VM. In line 2-4, the slave VM encoder is internal parity chunk P _k The GPU kernel is called to generate The GPU kernel only requires access to the p data chunks and the inner coding matrix X to generate an inner parity chunk.

와 내부 패리티 청크 P _k 를 대응하는 내부 패리티 디스크에 쓴다.In line 5, an internal parity chunk is passed to the CPU memory. Finally, at line 6-7, the slave VM is given p data chunks

And internal parity chunks P _k are written to the corresponding internal parity disks.

을 생성하기 위하여 사용되는 이레이져 코딩 방식을 보여준다.6 is the internal parity shown in Table 2

We show the erasure coding scheme used to generate

TABLE 3

Table 3 shows the pseudo-code of master VM decoding using pass-through GPU for proposed layer RAID 55. First, the master VM reads data chunks from the data disks in the slave VMs. If the number of slave VM losses exceeds one loss, it is called a disaster failure and it cannot be recovered. The master VM allocates corresponding survival data and parity data to ds in CPU memory. Then, the Mater_ VM _decoder function is called. The coding matrix B is generated by deleting the columns of the outer coding matrix H corresponding to the lost slave VM, and the inverse coding matrix B ^{- 1} is stored in the GPU shared memory in the master VM.

At line 3-5, the GPU kernel is called to recover the lost data d ^' using the inverse coding matrix B ^-1 and the survival data and parity chunks ds . Finally, the master VM writes the recovery data d ^' to the spare disk.

TABLE 4

Table 4 shows the pseudo-code of slave VM decoding using pass-through GPU for the proposed layer RAID 55. First, the slave VM reads data chunks from the data disks in the corresponding slave VM. If the number of disk losses exceeds one loss, it is called a disaster loss and cannot be recovered. The slave VM sends corresponding survival data and parity data in the CPU memory ds. Assign to After that, Slave_VM_Decoder is called. The coding matrix D is generated by deleting the columns of the inner coding matrix X corresponding to the lost disk, and the inverse code matrix D ^-1 is stored in the GPU shared memory in the slave VM. In line (3-5), the GPU kernel adds the inverse code matrix B ^-1 and the survival data and parity chunks ds. Called to recover lost data d ^' using. Finally, the slave VM writes recovery data d ^' to the spare disk.

The proposed layered method and device with Eraser Code Acceleration in the virtualized GPU environment is essential to provide stable services in the cloud environment. The present invention ensures high storage availability and at the same time recovers three disk and node losses, thereby improving cloud performance.

High storage availability in cloud environments can lower resource utilization on physical servers. Conventional mirroring-based raids generate high resource consumption, but have been used for data recovery. Therefore, it is determined that through the present invention, cloud reliability and data reliability can be secured to expand the high reliability, high availability cloud technology and the market.

Unlike the simple data replication method of the existing mirroring method, parity is generated in the unit of node that binds several disks in the system to secure high storage availability. In addition, pass-through GPU virtualization prevents performance degradation by effectively handling large amounts of XOR operations that can occur with parity operations on all nodes.

The present invention is expected to be applied to a cloud server that manages a large amount of data, while reducing cloud physical resource usage and at the same time ensuring a very stable data reliability.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. It can be embodied in. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (9)

The master virtual machine dividing the input data into data chunks;
Creating an external parity disk in the host from the input data;
Computing, by the GPU kernel, external parity data to the CPU memory;
Storing, by the master virtual machine, the external parity disk and the external parity data in the external parity node;
Creating an internal parity disk in an internal data node;
A slave virtual machine calculating internal parity data and transferring the internal parity data from the CPU memory to the GPU memory;
Calling, by the GPU kernel, a multi-GPU thread to generate internal parity data in parallel; And
The slave virtual machine storing the internal parity data and the internal parity disk in the internal data node
Including,
Generating an external parity disk in the host from the input data,
The master virtual machine distributes all data chunks to the corresponding internal data nodes through the slave virtual machine.
Tiered raid method. delete The method of claim 1,
Computing the external parity data to the CPU memory by the GPU kernel,
External parity data is computed by performing XOR operations between corresponding chunks of data within the various internal data nodes.
Tiered raid method. The method of claim 3,
To improve external parity data generation performance, the master virtual machine passes all data chunks from CPU memory to GPU memory, and the GPU kernel creates multiple-GPU threads to generate external parity data in parallel. To call
Tiered raid method. The method of claim 1,
The slave virtual machine stores the internal parity data and the internal parity disk in an internal data node,
If the internal parity data is lost, the corresponding slave virtual machine uses the internal parity disk to recover the lost internal parity data.
Tiered raid method. The method of claim 5,
Recovered internal parity data is transferred from GPU memory to CPU memory, and recovered internal parity data is stored on a spare disk within the same internal data node.
Tiered raid method. The master divides the input data into data chunks to create an external parity disk in the host from the input data, the GPU kernel computes the external parity data and passes it to the CPU memory, and stores the external parity disk and external parity data in the external parity node. Virtual machine; And
Creates an internal parity disk within an internal data node, computes and passes internal parity data from CPU memory to GPU memory, the GPU kernel calls multi-GPU threads to generate internal parity data in parallel, and internal parity data Virtual machine for storing internal parity disks and internal parity disks
Including,
The master virtual machine,
All data chunks are distributed to corresponding internal data nodes through a slave virtual machine, and external parity data is calculated by performing XOR operations between corresponding data chunks in various internal data nodes,
To improve external parity data generation performance, all data chunks are passed from CPU memory to GPU memory, and the GPU kernel calls multiple-GPU threads to generate external parity data in parallel.
Tiered device. delete The method of claim 7, wherein
The slave virtual machine,
If the internal parity data is lost, the corresponding slave virtual machine uses the internal parity disk to recover the lost internal parity data, the recovered internal parity data is transferred from GPU memory to CPU memory, and the recovered internal parity data is Stored on a spare disk within the same internal data node
Tiered device.

Images