WO2016065556A1

WO2016065556A1 - Software-defined storage system and method, and centralized control device thereof

Info

Publication number: WO2016065556A1
Application number: PCT/CN2014/089799
Authority: WO
Inventors: 周文
Original assignee: 北京麓柏科技有限公司
Priority date: 2014-10-29
Filing date: 2014-10-29
Publication date: 2016-05-06

Abstract

A software-defined storage system implementation method and a centralized control device thereof. The centralized control device comprises a control unit (807), a front end interface (802) connected to a front end virtual machine, and a back end interface (803) connected to a back end distributed storage service unit. The control unit is used for performing the following processing according to a user storage service configuration parameter sent by a storage service configuration unit: performing corresponding processing on a storage service request of the storage service unit that is sent by the storage service configuration unit to the centralized control device; and/or performing corresponding processing on a storage service request of the storage service unit that is sent by the virtual machine to the centralized control device. The storage service unit is provided with a computing unit. The computing unit is used for implementing a storage service that is forwarded by the centralized control device and is corresponding to the storage service request. High-efficiency software-defined storage can be implemented, and the storage service quality between virtual machines can be dynamically adjusted and allocated.

Description

Software defined storage system, method and centralized control device thereof

[Technical Field]

The present invention relates to the field of storage, and in particular to a storage system, a method, and a centralized control device thereof.

【Background technique】

With the development of virtualization technology and cloud technology, current and future data centers will serve thousands of users. The data center infrastructure evolved into the era of software definition. Software-defined data centers include software-defined computing, software-defined networking, and software-defined storage. Data center operators provide computing services and storage services to tenants. Different tenants have different storage service requirements and different application scenarios. Different sub-accounts (users) under the same tenant have different storage service requirements and different application scenarios. These different storage service requirements include, but are not limited to, capacity, performance (IOPS, BandWidth), latency, backup level, durability, encryption, deduplication, compression, snapshots, physical isolation. And these storage service service requirements are dynamically changeable as needed. A storage system that meets these massive, dynamically changing, and personalized storage service requirements is called a software-defined storage system. Current storage system architectures (Figures 1A and 1B) do not meet the diverse requirements and performance requirements of software-defined storage requirements. In the centralized storage system shown in FIG. 1A, the centralized storage system is composed of a storage control head 104 and a storage resource 103. The storage resource 103 does not have computing power, and all computing operations related to the data service are completed by the storage control head 104. The storage service policies (or software defined storage requirements) from the control cluster 106 are all passed to the storage control head 104 via the channel 101. There are two main problems with centralized storage systems. One problem is limited by the computing power of 104 storage control heads. 104 storage heads can provide limited types of storage service requirements, while IO operations of each virtual machine 105 are stored. A large amount of computational work is generated on the control head 104, and the performance (IOPS, Bandwidth) that the storage control head 104 can provide is limited; another problem is that there is no lateral expansion capability. These two problems lead to centralized storage systems that do not have the personalized quality of service control capabilities and seamless capacity expansion required by software-defined storage systems. In the distributed storage system shown in FIG. 1B, the storage cluster is composed of a storage unit 113 having computing resources, and has a horizontal expansion capability, and the storage unit 113 may be a commercial x86 server. The plurality of storage units 113 receive the IO requests from the virtual machine 115 in parallel, and the centralized storage system controls the head 104 to limit the computing power, thereby providing higher performance. The storage service policy from the control cluster (116) is handed over to each virtual machine 115 via channel 111. The main problem of the distributed storage system is that the execution of the storage service policy is dispersed in each virtual machine 115, and the service level distinction based on the tenant, the logical disk and the virtual machine cannot be provided, and the software defined storage service for different service levels cannot be satisfied. Option control.

With the development of semiconductor technology, the storage density of high-speed nonvolatile memory devices NAND Flash is getting higher and higher. It has been widely used as a data access acceleration device in data centers. Compared to mechanical disks, NAND Flash has faster random access speeds. Compared to DRAM, NAND Flash is a non-volatile memory device that continues to hold data after the power is turned off. In order to take advantage of the high-speed access of NAND Flash, new module specifications and new access protocols for NAND Flash have been developed. Because of the nature of mechanical disk (HDD), such as the need for block erasure, traditional x86-based servers can not better play the performance of storage resources using Nand Flash as a storage medium, and need to design a new storage architecture and Storage device.

[Summary of the Invention]

In order to overcome the inability of existing storage systems to meet the requirements of software-defined storage, the present invention provides software-defined storage systems, methods, and centralized control devices thereof to implement software-defined storage requirements for software-defined storage systems for software-defined storage diversity, and performance. Requirements.

A centralized control device for a software-defined storage system, comprising: a control unit, a front-end interface connected to the front-end virtual machine, and a back-end interface connected to the back-end distributed storage service unit, the control unit is configured to use the storage service The user storage service configuration parameters sent by the configuration unit are processed as follows:

Transmitting, by the storage service configuration unit, the storage service request of the storage service unit to the centralized control device for corresponding processing;

And/or performing corresponding processing on the storage service request of the storage service unit by sending the virtual machine to the centralized control device;

The storage service unit has a computing unit, and the computing unit is configured to complete a storage service corresponding to the storage service request forwarded by the centralized control device.

In one embodiment, the storage service configuration parameter includes a user permission parameter, and the control unit is configured to send the storage service configuration unit to the centralized control device according to the user permission parameter to the storage service. The unit's storage service request is processed accordingly.

In one embodiment, the storage service configuration parameter includes a storage quality of service parameter, and the control unit controls quality of a storage service between the virtual machine and the storage service unit according to the storage quality of service parameter .

In one embodiment, the storage quality of service parameter includes: a bandwidth between the virtual machine and the storage service unit, a number of read and write operations per second, a storage service unit capacity or data layering, data encryption, and Heavy, compressed, and physically isolated.

In an embodiment, the control unit is further configured to dynamically allocate the storage quality of service between the plurality of virtual machines according to the storage quality of service parameter.

In an embodiment, the method further includes a buffer, and the control unit is further configured to determine according to the storage service configuration parameter:

Whether to store data required by the virtual machine to complete the storage service to the cache;

Or the aging speed of the data required by the cached virtual machine to complete the storage service;

Or whether the cached partial space is used as a fixed hierarchical storage space to store data required by the virtual machine to complete the storage service.

In one embodiment, the control unit uses the virtual machine in the cache to complete data required for storing the service, and increases the number of read/write operations per second between the virtual machine and the storage service unit.

The present invention also provides a software defined storage system, including a virtual machine, further comprising a distributed storage service unit, a storage service configuration unit, and the centralized control device;

The storage service configuration unit is configured to generate a storage service configuration parameter of the user, and send the storage service configuration parameter to the centralized control device;

The virtual machine is configured to send a storage service request to the centralized control device;

The storage service unit is configured to receive a storage service task corresponding to the storage service request sent by the centralized control device, and complete the storage service task.

In an embodiment, the storage service configuration unit is further configured to query the centralized control device for current storage service configuration parameters and operation information.

The invention also provides a software definition storage system implementation method, comprising the following steps:

The centralized control device receives the user storage service configuration parameter generated by the storage service configuration unit;

The centralized control device generates a corresponding service control parameter according to the storage service configuration parameter;

Receiving, by the centralized control device, a storage service request sent by a virtual machine or a storage service configuration unit;

The centralized control device performs corresponding processing on the storage service request according to the service control parameter;

And if the storage service request is legal, the centralized control device forwards the storage service corresponding to the storage service request to a storage service unit, where the storage service unit has a calculation unit, and the calculation unit is configured to complete the concentration Controlling, by the device, a storage service corresponding to the storage service request.

The centralized control device controls the storage service request sent by the virtual machine and the storage service configuration unit according to the storage service configuration parameter sent by the storage service configuration unit, and sends the specific storage service to the distributed storage service unit that has completed the corresponding storage service ( The completion of the cluster can not only satisfy the user's various definition storage requirements, but also the centralized control device can not become the bottleneck node of the entire software-defined storage system because there is not a large number of specific storage services, so it can also ensure that the entire software-defined storage system has Better software-defined storage service processing capabilities.

The centralized control device dynamically adjusts the storage service quality of the plurality of virtual machines according to the storage service configuration parameters, and ensures the utilization of the storage service quality of the plurality of virtual machines as much as possible.

The centralized control device can control the storage service quality between the virtual machine and the distributed storage service unit (cluster) according to the storage service configuration parameter, and centrally control the cache of the device, and ensure that the virtual machine achieves a higher storage service as much as possible. quality.

[Description of the Drawings]

1a is a schematic diagram of a centralized storage system architecture in the prior art;

1b is a schematic structural diagram of a distributed storage system in the prior art;

2 is a schematic structural diagram of a software defined storage system according to an embodiment of the present invention;

3 is a schematic structural diagram of storage service configuration parameters according to an embodiment of the present invention;

4a is a schematic diagram of a storage service process in a prior art distributed storage system;

4b is a schematic diagram of a storage service process in a scenario of a software-defined storage system according to an embodiment of the present invention;

4c is a schematic diagram of a storage service process of a software-defined storage system in another scenario according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of the composition of a software defined storage system according to an embodiment of the present invention; FIG.

6 is a schematic diagram of the composition of a centralized control device according to an embodiment of the present invention;

7 is a schematic diagram of the composition of a storage service unit according to an embodiment of the present invention;

FIG. 8 is a schematic overall flow chart of software definition storage according to an embodiment of the present invention; FIG.

FIG. 9 is a schematic flowchart of a storage service configuration unit providing a storage service configuration according to an embodiment of the present invention.

【detailed description】

Preferred embodiments of the invention are described in further detail below.

As shown in FIG. 2, the system architecture of a software-defined storage system of an embodiment may be referred to as centralized distribution. The storage system architecture includes a storage service configuration cluster (or a control cluster), a centralized control device 204, a distributed storage service cluster, and a computing cluster, where the storage service configuration cluster includes multiple storage service configuration units, and the computing cluster includes multiple The virtual machine, the distributed storage service cluster includes a plurality of storage service units, and the centralized control devices are respectively connected to the storage service configuration unit, the distributed storage service unit (cluster), and the virtual machine (cluster).

The storage service configuration unit generates storage service configuration parameters (or software-defined storage requirements), the storage service configuration unit sends the storage service configuration parameters to the centralized control device 204, and the centralized control device 204 receives the storage service configuration parameters, and according to these Parameters to control the virtual machine's access to the distributed storage service cluster, or the storage service request sent by the virtual machine, corresponding processing according to the storage service configuration parameters; or the storage sent from the storage service configuration unit The service request is processed according to the storage service configuration parameters. If the storage service request sent by the virtual machine meets the storage service configuration parameter, the centralized control device may send the storage service request to the distributed storage service cluster, and the storage service unit in the distributed storage service cluster has the storage service request completed. Corresponding to the computing power of the specific storage service, the centralized control device 204 itself does not need to provide complex storage service calculations. Therefore, the centralized control device 204 is difficult to become a performance bottleneck node of the entire software-defined storage system.

The storage service configuration parameters can be divided into user permission parameters and storage quality of service parameters. The permissions can be the number of virtual machines and logical disks, access to a virtual machine or logical disk, clone (for example, clone a virtual machine), and virtual Snapshots (such as number, name) stored by the machine, logical disk, etc., for example, basic user rights include having one virtual machine and one logical disk, and higher-level user rights are multiple virtual machines (virtual units) and multiple logical disks. (Logical disk group), user rights with multiple virtual machines and multiple logical disks can be referred to as a tenant.

The storage quality of service parameters may include, but are not limited to, storage capacity or data tiering, bandwidth between the virtual machine and the distributed storage service cluster, number of read and write operations per second (IOPS), latency (eg, data) The delay from the virtual machine to the storage service unit), the backup level (such as the number of data backups in the logical disk), storage data encryption, data deduplication, data compression, physical isolation, and so on. The centralized control device can dynamically allocate the storage service quality between the multiple virtual machines under the same tenant according to the storage service quality parameter, so as to improve the utilization efficiency of the storage service quality of the tenant at a certain moment.

As shown in FIG. 3, it is a classification method for storing service configuration parameters, which includes a storage service level and a storage service option. Storage service levels are primarily based on having virtual machines and logical disks, including but not limited to capacity, performance (such as IOPS and bandwidth), latency, backup levels, encryption, snapshots, cloning, deduplication, compression, and physical isolation. . An example of a specific storage service configuration parameter is as follows: Example 1, tenant A has a capacity of 100T, performance is 500KIOPS, 2GB/s bandwidth, three backups, encryption option is off, snapshot option is on, g The option is off, the option is off, the compression option is off, the physical isolation option is turned on; for example, the performance of virtual machine I belonging to tenant B is 100KIOPS, 3 backup, encryption option is on, snapshot option is on, clone option is off, deduplication The option is turned off and the compression option is turned on.

In an embodiment, the centralized control device may determine whether the storage service request of the user is legal according to the storage service configuration parameter sent from the storage service configuration unit, and if not, prevent data operation on the distributed storage service cluster. Or operate on the corresponding data in the cache attached to the centralized control device. For example, the user sends a storage service request for cloning a virtual machine to the centralized control device through the storage service configuration unit, and if the centralized control device determines that the user has the permission, sends the clone service request to the distributed storage service cluster, and The service of cloning the virtual machine is completed by the distributed storage service cluster; if it is determined that the user does not have the privilege, the centralized control device does not send a corresponding request to the distributed storage service cluster, and may feed back related information to the storage service configuration unit. For example, a warning.

In an embodiment, the centralized control device may further control the quality of the storage service between the virtual machine and the distributed storage service cluster according to the storage quality of service parameter. For example, the virtual machine A sends a storage service request to the centralized control device, and the centralized control device queries the relevant service quality parameter of the virtual machine A, and then allocates the corresponding service quality to the virtual machine A, for example, the corresponding IOPS, the storage capacity, and the like.

In an embodiment, the centralized control device may dynamically adjust and allocate storage quality (including IOPS, bandwidth, capacity, and the like) of multiple virtual machines according to storage quality of service parameters, and ensure multiple virtual machines as a whole as possible. The quality of storage services will not decline.

For example, a tenant has two virtual machines, and the tenant has a total IOPS set (ie, the total IOPS limit of two virtual machines at any time), when two virtual machines respectively send storage service requests to the centralized control device, and When the corresponding IOPS is required, the centralized control device allocates the corresponding IOPS to the corresponding virtual machine according to the total IOPS, so as to avoid the waste of IOPS by the tenant and improve the overall IOPS utilization of the tenant. The centralized control device can dynamically adjust and allocate the corresponding IOPS to the corresponding virtual machine according to different storage service quality requirements of the two virtual machines at different times.

As shown in Figures 4a to 4c, a comparison of the application of an IO access on the distributed storage system and the software-defined storage system is given. As shown in Figure 4a, it is the IO access between the virtual machine and the storage service unit in the distributed storage system. Both virtual machine A and virtual machine B are sub-accounts of the tenant NetBric, and the virtual machine A on the physical server A is calculated. Or look up the table, the content that needs to be accessed is on the storage service unit A, the virtual machine A directly issues an IO request to the storage service unit A; the virtual machine B on the physical server B undergoes calculation or table lookup, and the content to be accessed is in the storage service. On unit B, virtual machine B initiates an IO request directly to storage service unit B. If the total IOPS quota of the tenant NetBric is 500K, then the virtual IOPS of virtual machine A and virtual machine B can only be static. State allocation, for example, virtual machine A is allocated 300K IOPS, virtual machine B is allocated 200K IOPS, at some point, virtual machine B's IOPS demand is only 100K, and virtual machine A's IOPS demand is 400K, virtual machine A is actually Only 300K IOPS can be allocated, so the tenant NetBric can achieve only IOPS of 400K. Compared with the total amount of 500K IOPS, 100K is wasted idle.

As shown in FIG. 4b, it is the IO access in the software-defined storage system of this embodiment. It is assumed that the virtual machine A and the virtual machine B are also sub-accounts of the tenant NetBric, and the total IOPS quota of the tenant NetBric is 500K, and the virtual machine A The IO request with the virtual machine B is sent to the centralized control device, and the centralized control device distributes the IO request to the corresponding storage service unit A and storage service unit B. The centralized control device can be dynamically updated within a total range of 500K IOPS. Adjust the IOPS allocation of virtual machine A and virtual machine B. When virtual machine B's IOPS is only 100K, allow virtual machine A's IOPS to reach 400K. When virtual machine B's IOPS is 200K, limit virtual machine A's IOPS to 300K. Therefore, the storage service quality is dynamically allocated between multiple virtual machines under the same tenant, and the utilization efficiency of the storage service quality of the tenant at a certain moment is improved.

If the tenant NetBric upgrades its total IOPS quota to 5M and the IOPS assigned to virtual machine A is 1M, the centralized control device judges that the back-end distributed storage service cluster cannot provide 1M IOPS to virtual machine A. The access path can be as shown in Figure 4c. The centralized control device accelerates the access of the virtual machine A through the corresponding algorithm, and tries to ensure that the IO access content of the virtual machine A can be found in the cache in the centralized control device, thereby The IO access provides acceleration.

In an embodiment, the centralized control device may further include a cache, and the control unit of the centralized control device may determine, according to the storage service configuration parameter, whether to store data required by the virtual machine to complete the storage service to the cache; for example, When the tenant's authority is high or the quality of service is high, you need to centrally control the device to cache related data to reduce the delay. For example, a tenant has a lower delay parameter. When a centralized control device finishes forwarding data to a tenant's virtual machine, the related data is stored in the cache for use in the next use.

In an embodiment, the control unit of the centralized control device may further determine, according to the storage service configuration parameter, that the cache stores the aging speed of data required by the virtual machine to complete the storage service; for example, the rights of a tenant are compared. When the service quality is high or the service quality is high, the aging speed of the related data is slower. Otherwise, the aging speed can be faster, so as to reduce the storage service delay and ensure the remaining space of the cache.

In an embodiment, the control unit of the centralized control device may further determine, according to the storage service configuration parameter, whether the cached partial space is used as a fixed tiered storage space to store the virtual machine required to complete the storage service. The data. For example, when a tenant has a high authority or a high quality of service, a part of the space may be opened in the cache as a fixed hierarchical storage space for buffering the virtual machine of the tenant to perform data generated by the storage service.

As shown in FIG. 5, it is a schematic diagram of the composition of a software-defined storage system of one embodiment. As shown in FIG. 6, it is a schematic structural diagram of a centralized control device of an embodiment. The storage service client is used to provide a data interface to the virtual machine, and the virtual machine sends the data to the storage service client. The storage service client and the centralized control device exchange data through the S5 protocol, and between the centralized control device and the storage service unit. The data exchange is completed through the S5 protocol, and the storage service configuration unit sends the storage service configuration parameters to the centralized control device through a software-defined storage protocol.

The functions of the centralized control device include, but are not limited to, providing high performance (high IOPS and high bandwidth) data storage services, performing storage quality control according to storage service configuration parameters, and distributing specific storage according to storage service configuration parameters and specific storage service requests. Operate to the backend distributed storage service cluster.

The centralized control device may include: a computing and total control module 801, a front end interface module 802, a backend interface module 803, an acceleration module 804, a quality of service (QoS) control module 805, a storage control switching module 806, a MegaNAND control module 807, and a NANDFlash. Module module 808.

The calculation and total control module 801 and the storage service configuration unit exchange information through a software-defined storage protocol, receive storage service configuration parameters from the storage configuration unit configuration, and send the generated control parameters to corresponding modules within the centralized control unit through calculation. This module runs the daemon software of the centralized control device, which is responsible for monitoring the state of the centralized control device and processing the data processing request sent by the centralized control device to the back-end distributed storage service cluster. At the same time, this module runs client software with a distributed storage system.

The front-end interface module 802 can communicate with the storage service client cluster through interfaces such as an Ethernet port, an FC network port, and an iSCSI network port, and receive a storage service request from the storage service client. The front-end interface module can implement hardware resolution protocols such as TCP/IP, FC, and iSCSI, and deliver the parsed storage service protocol package to the acceleration module 804.

The back-end interface module 803 can communicate with the storage service unit cluster through an Ethernet port, an FC network port, an iSCSI network port, and the like. The back-end interface module can implement hardware parsing protocols such as TCP/IP, FC, iSCSI, and pass the parsed storage service protocol package to the acceleration module 804.

The acceleration module 804 is responsible for hardware parsing the S5 protocol packet, generating corresponding hardware operation control instructions, and sending the subsequent modules.

The quality of service (QoS) control module 805 is responsible for adjusting the statistics generated by the quality of service (QoS) control module 805 based on the control parameters from the computing and total control module 801, and the information (eg, identity ID) contained in the S5 protocol packet. Quality of service for storage services from different service levels. Quality of service includes but is not limited to IOPS, data bandwidth (Bandwidth), capacity and data layering.

The storage control switch module 806 receives hardware control instructions from the quality of service (QoS) control module 805 and the acceleration module 804, generates corresponding actions, or passes the data directly to the backend distributed storage service cluster. Or generate a hardware operation request to the MegaNAND control module 807.

The MegaNAND control module 807 virtualizes a large number of NANDFlash module modules 808 (for example, hundreds of terabytes) into a storage resource pool of NAND Flash, and uses it as a smart cache of the back-end storage service unit cluster, and completes data redundancy, deduplication, encryption, and the like. operating. This module is the main module to improve the centralized control device IOPS and data bandwidth. The characteristics of the intelligent cache include: determining whether to cache relevant data according to the information of the quality of service control module; determining the aging speed of the cached data according to the information of the quality of service control module; determining whether to use part of the cache space according to the information of the quality of service control module Fixed tiered storage space to use.

The NANDFlash module module 808 is responsible for organizing the NANDFlash device into a basic NANDFlash memory function unit, providing basic NANDFlash memory services.

FIG. 8 is a block diagram of a storage service unit according to an embodiment of the present invention, including a network interface 901, a computing unit 902, and a storage medium 903. The network interface 901 implements a connection between the computing storage service unit and the network; the computing unit 902 can be an x86 CPU or an ARM CPU, and runs simple calculations; the storage medium 903 can be an SSD or an HDD. A plurality of storage service units constitute a cluster of storage service units, and a distributed storage service cluster (system) is composed of software running on the computing unit 902. This distributed storage service cluster completes a specific data storage service.

As shown in FIG. 9, the workflow of the software-defined storage system according to an embodiment of the present invention includes the following steps:

Step 501: The management account configures a storage service configuration parameter (software defined storage requirement) on the storage service configuration unit.

Step 502: The storage service configuration unit sends the storage service configuration parameter to the centralized control device through a software-defined storage protocol. This step uses a portion of the software-defined storage protocol. A software defined storage protocol is used for message transfer between a storage service hive (eg, Openstack's console, VMWare console) and the centralized control device disclosed herein. Software-defined storage protocols include, but are not limited to, the following:

Set performance, delay, service level, encryption, compression, deduplication, number of backups, and physical location distribution related parameters.

Query the current setting status of the storage service configuration unit or centralized control device, that is, the status of each parameter set above.

Query current statistics, such as virtual machine-based IOPS, bandwidth (width), latency (Latency), and hit rate for virtual machine access acceleration (for virtual machines that store users with higher quality of service, concentrate) The control device can provide accelerated access to the virtual machine, and the centralized control device can count the proportion of the successful acceleration of the virtual machine, that is, the hit rate mentioned above.

Manage tenants and sub-accounts of tenants for billing purposes.

Query the running status of the system, including the CPU, memory, disk load status, and health status of each node of the entire system.

The statistics are queried by the number of resources occupied by the tenant and the sub-account of the tenant for billing.

Virtual disk operations, including creating virtual disks, deleting virtual disks, setting parameters of virtual disks, performing snapshots, cloning, and copying.

The software-defined storage protocol uses the TCP/HTTP network protocol as the transport protocol to transport between the storage service configuration unit and the centralized control device. You can use XML as the base description format.

Step 503: The centralized control device generates corresponding software and hardware control parameters according to the storage service configuration parameter. This step can be calculated and generated by the calculation and total control module 801.

Step 504: The storage service client or the storage service configuration unit issues a storage service request to the centralized control device through the S5 protocol. In some embodiments, a particular storage service request may be accomplished by the client agent software disclosed herein or by the storage service application interface (API) disclosed herein. The storage service client agent software or application interface can provide storage services based on block, object, and file. The storage hive agent software or application programming interface (API) can provide storage service requests such as clones, snapshots, and so on.

Step 505: The centralized control device queries the software and hardware control parameters generated by the storage service policy, determines the legality and rationality of the storage service request, and performs service quality control. This step can be done by quality of service (QoS) control module 805.

Step 506: The centralized control device generates a corresponding data processing request according to the storage service request.

Step 507: The centralized control device confirms whether the storage service request will generate a data processing request with the backend distributed storage service cluster.

Step 508: The centralized control device completes the data processing operation through the S5 protocol and the backend distributed storage service cluster.

Step 509: The centralized control device returns a storage service response to the storage service client, including but not limited to an illegal request, delayed retry, normal completion, and the like.

Step 501, step 502, and step 503 are non-recurring steps, which are usually performed when the user's account is set up or the storage service configuration parameter is changed. Step 504, step 505, step 506, step 507, step 508, step 509 are recurring steps that are typically performed each time the user performs a storage service operation.

In some embodiments, the S5 protocol is a protocol used between a centralized control device and a storage service client and a distributed storage service cluster. The contents of the agreement include but are not limited to:

The specific read/write operation instruction is used to instruct the centralized control device to perform acceleration, and is used to check a heartbeat command of the network, and is used to instruct an instruction of the logical disk operation.

Instructions for obtaining information such as snapshots, statistics, data lengths, data locations, data usage, and more.

An instruction for determining data identity (including but not limited to transport ID, user ID, resource pool ID, Image ID).

An instruction for determining information such as data version, modification time, and the like.

An instruction to establish information about a session (such as status, listening port, etc.).

An instruction for verifying information for improving transmission reliability.

As shown in FIG. 9, a storage service configuration parameter configuration process according to an embodiment of the present invention. The entire configuration process is provided by software running on the storage service configuration unit, which is part of the software defined storage system disclosed herein.

Step 701: The super administrator (for example, the S5 administrator) creates a tenant account according to the tenant requirement, and sets the tenant to store service configuration parameters, such as a permission parameter and a storage quality of service parameter, such as capacity, IOPS, bandwidth, encryption, deduplication, compression.

Step 702: The tenant account creates a management account with different management rights as needed. Different permissions include but are not limited to whether you can create a logical disk, whether you can set backup properties, and so on. Step 702 is not a required step.

Step 703: The tenant account or the tenant's management account creates different virtual machines and logical disks according to the internal requirements of the tenant. It can be set that some virtual machines and logical disks can only be used by some accounts of one tenant.

Step 704: The tenant account or the tenant's management account creates different service option sets according to the tenant internal demand categories. Including but not limited to IOPS, bandwidth, number of backups, physical isolation. The type of service selection available is limited by the storage service options of the tenant account.

Step 705: The tenant account or the tenant's management account associates the storage service option set configured in step 704 with the virtual machine, virtual machine group, logical disk, logical disk group, and the like configured in step 703 to form a complete storage service configuration parameter.

Software running on the storage service hive also provides statistical results as well as status information. For example, current capacity consumption, IOPS consumption, and the like.

In one embodiment, the application flow of the software defined storage system in Figure 4c is as follows.

The data center provider creates a tenant NetBric account through the storage configuration unit. The associated software-defined storage options are 500K IOPS, 2GB/s bandwidth, 100T capacity, up to 3 backups, data encryption allowed, snapshots allowed, clone allowed, and deduplication allowed. Allow compression and allow physical isolation. NetBric accounts are configured through storage services The unit creates virtual machine A and virtual machine B. The software defined storage option associated with virtual machine A is 400KIOPS, 1GB/s bandwidth, 20T capacity, 3 backup, data encryption, snapshot, deduplication, compression, and physical isolation. The NetBric account attempts to associate virtual machine B with software-defined options of 300K IOPS, 1GB/s bandwidth, 20T capacity, 4 backups, data encryption, snapshots, deduplication, compression, no physical isolation, and storage service hive queries for NetBrick software. After defining the storage options, it is recommended to modify the virtual machine B's illegal software-defined storage option 300K IOPS, 4 backup to 100K IOPS, 3 backup, and give a warning. After the NetBric account is confirmed, the storage hive sends the software-defined storage requirements to the centralized control device through the software-defined storage protocol. After receiving the software-defined storage requirements of the tenant NetBric, the computing and total control module 801 generates corresponding control parameters (eg, IOPS quota, identity information, bandwidth quota).

The read data request of virtual machine A and the write data request of virtual machine B are processed by front end interface module 802 and acceleration module 804, and finally arrive at quality of service (QoS) control module 805. The quality of service (QoS) control module 805 checks the previous statistics and the corresponding control parameters to confirm that the virtual machine B's IOPS quota is not exhausted, and the backend distributed storage service cluster can meet the virtual machine B 100K IOPS service requirements, NANDFlash There is no data required by virtual machine B in module module 808. The quality of service (QoS) control module 805 generates control instructions to the storage control switching module 806 and the backend interface module 803, and writes data requests and partial software defined storage options of the virtual machine B (including not limited to 3 backups, data encryption, snapshot sequences). Number, compression) is distributed to the back-end distributed storage system, and the corresponding data encryption, compression, main data update and backup data update are completed by the back-end distributed storage system. For the read data request of the virtual machine A, the quality of service (QoS) control module 805 checks the previous statistical result and the corresponding control parameters, and confirms that the IOPS quota of the virtual machine A is not exhausted, and the NANDFlash module module 808 already has the virtual The machine A reads the data request related data, and the quality of service (QoS) control module 805 generates a corresponding control command to the storage control switch module 806, which is shared by the storage control switch module 806, the MegaNAND control module 807, and the NANDFlash module module 808. The reading of the read data is completed and returned to the virtual machine A.

The tenant account NetBric can also issue a storage service request for cloning virtual machine A to the centralized controller's computing and total control module 801 through the storage service configuration unit, and the computing and total control module 801 will clone the service through the acceleration module 804 and the backend interface module 803. The request is sent to the back-end distributed storage service cluster, and the clone service is completed by the back-end distributed storage service cluster.

The software defined storage service disclosed by the present invention is not limited to the above examples, and any combination of the storage service level and the storage service option in the software defined storage requirements disclosed in the present invention is included in the scope of the software defined storage service disclosed by the present invention.

The above is a further detailed description of the present invention in connection with the specific preferred embodiments, and the specific embodiments of the present invention are not limited to the description. For those of ordinary skill in the art to which the present invention pertains, A number of simple derivations or substitutions may be made without departing from the spirit of the invention, and should be considered as belonging to the scope of the invention as defined by the appended claims.

Claims

A centralized control device for a software-defined storage system, comprising: a control unit, a front-end interface connected to the front-end virtual machine, and a back-end interface connected to the back-end distributed storage service unit, wherein the control unit uses Perform the following processing on the user storage service configuration parameters sent according to the storage service configuration unit:

Transmitting, by the storage service configuration unit, the storage service request of the storage service unit to the centralized control device for corresponding processing;

And/or performing corresponding processing on the storage service request of the storage service unit by sending the virtual machine to the centralized control device;

The storage service unit has a computing unit, and the computing unit is configured to complete a storage service corresponding to the storage service request forwarded by the centralized control device.
The centralized control device according to claim 1, wherein the storage service configuration parameter comprises a user permission parameter, and the control unit is configured to send the storage service configuration unit to the user according to the user permission parameter. The centralized control device performs corresponding processing on the storage service request of the storage service unit.
The centralized control device according to claim 1, wherein the storage service configuration parameter comprises a storage quality of service parameter, and the control unit compares the virtual machine and the storage service unit according to the storage quality of service parameter The quality of the storage service is controlled between.
The centralized control device according to claim 3, wherein the storage quality of service parameter comprises: a bandwidth between the virtual machine and the storage service unit, a number of read/write operations per second, and a capacity of a storage service unit. Or data tiering, data encryption, deduplication, compression, and physical isolation.
The centralized control device according to claim 3, wherein the control unit is further configured to dynamically allocate the storage quality of service among the plurality of virtual machines according to the storage quality of service parameter.
The centralized control device according to claim 1, further comprising a cache, wherein the control unit is further configured to: according to the storage service configuration parameter:

Whether to store data required by the virtual machine to complete the storage service to the cache;

Or the aging speed of the data required by the cached virtual machine to complete the storage service;

Or whether the cached partial space is used as a fixed hierarchical storage space to store data required by the virtual machine to complete the storage service.
The centralized control device according to claim 6, wherein the control unit uses the virtual machine in the cache to complete data required for storing a service, and improves between the virtual machine and the storage service unit. The number of read and write operations per second.
A software-defined storage system, comprising a virtual machine, characterized by further comprising: a distributed storage service unit, a storage service configuration unit, and the centralized control device according to any one of claims 1 to 7;

The storage service configuration unit is configured to generate a storage service configuration parameter of the user, and send the storage service configuration parameter to the centralized control device;

The virtual machine is configured to send a storage service request to the centralized control device;

The storage service unit is configured to receive a storage service task corresponding to the storage service request sent by the centralized control device, and complete the storage service task.
A software defined storage system according to claim 8 wherein:

The storage service configuration unit is further configured to query the centralized control device for current storage service configuration parameters and operation information.
The software-defined storage system according to claim 8, wherein the storage service configuration parameter comprises a user permission parameter, and the control unit is configured to send the storage service configuration unit to the user according to the user permission parameter. The centralized control device performs corresponding processing on the storage service request of the storage service unit.
A software-defined storage system according to claim 8, wherein said storage service configuration parameter comprises a storage quality of service parameter, said control unit, said virtual machine and said storage service according to said storage quality of service parameter The quality of the storage service between the units is controlled.
The defined storage system of claim 11, wherein the control unit is further configured to dynamically allocate the storage quality of service among the plurality of virtual machines according to the storage quality of service parameter.
A software defined storage method, comprising the steps of:

The centralized control device receives the user storage service configuration parameter generated by the storage service configuration unit;

The centralized control device generates a corresponding service control parameter according to the storage service configuration parameter;

Receiving, by the centralized control device, a storage service request sent by a virtual machine or a storage service configuration unit;

The centralized control device performs corresponding processing on the storage service request according to the service control parameter;

And if the storage service request is legal, the centralized control device forwards the storage service corresponding to the storage service request to a storage service unit, where the storage service unit has a calculation unit, and the calculation unit is configured to complete the concentration Controlling, by the device, a storage service corresponding to the storage service request.