WO2017177953A1

WO2017177953A1 - Method and apparatus for setting data processing path for application in storage server

Info

Publication number: WO2017177953A1
Application number: PCT/CN2017/080480
Authority: WO
Inventors: 李夫路
Original assignee: 华为技术有限公司
Priority date: 2016-04-14
Filing date: 2017-04-13
Publication date: 2017-10-19
Also published as: CN107301090A; CN107301090B

Abstract

A method and apparatus for setting a data processing path for an application in a storage server, which belong to the field of data storage. The method comprises: receiving a target parameter value, sent by an application server, of a target data processing path set for an application; determining a parameter value, corresponding to the target parameter value, of each data processing path in a storage server; and setting a data processing path corresponding to the determined parameter value closest to the target parameter value as the target data processing path of the application. In the method, by means of acquiring a target parameter value embodying the requirements for the performance, cost, etc. of an application, and setting a data processing path corresponding to a parameter value closest to the target parameter value as a target data processing path of the application, the target data processing path can satisfy the requirements for the performance, cost, etc. of the application, thereby improving the data I/O performance of the application.

Description

Method and device for setting data processing path for application in storage server

Technical field

The present invention relates to the field of data storage, and in particular, to a method and apparatus for setting a data processing path for an application in a storage server.

Background technique

With the popularity of the Internet and the development of social information, people's lives and work have been inseparable from a variety of applications, such as database applications, multimedia applications, virtual desktop applications. The storage system can provide a data storage service for a plurality of applications, where the storage system includes a storage server, and the storage server can set a corresponding data processing path for the application based on various software and hardware resources included in the storage system, so that the application is in the data I. /O (Input Output, input and output), can be carried out according to the set data processing path.

In the prior art, the storage server may randomly set a data processing path for an application based on existing hardware and software resources. However, the existing random allocation mechanism, although capable of setting a data processing path for an application, enables an application to perform basic data processing, but different applications have different requirements for performance and cost of data processing, for example, virtual desktop application for data processing. The delay requirements are high, and multimedia applications have lower latency requirements and higher cost requirements. The data processing path set based on the random allocation mechanism described above cannot meet the performance and cost requirements of the application, and the data I/O performance of the application is affected.

Summary of the invention

In order to solve the problems of the prior art, the present application provides a method and apparatus for setting a data processing path for an application in a storage server. The technical solution is as follows:

A first aspect provides a method for setting a data processing path for an application in a storage server, where the storage server is provided with a plurality of data processing paths, where the data processing path is an application server completing data of the application. O. Multiple tasks that need to be performed; the methods include:

Receiving, by the application server, a target parameter value of the target data processing path set for the application; determining a parameter value corresponding to the target parameter value in each data processing path in the storage server; and determining the determined closest value to the target parameter value The data processing path corresponding to the parameter value is set as the target data processing path of the application. By obtaining the target parameter value that reflects the performance, cost, and the like of the application, and setting the data processing path corresponding to the parameter value closest to the target parameter value as the target data processing path of the application, the target data processing path can satisfy the performance of the application. , cost and other requirements, improve the application's data I / O performance.

With reference to the first aspect, in a first possible implementation manner of the first aspect, each task in the storage server includes at least one processing manner, and selecting one processing from at least one processing manner of each task The manner forms the plurality of data processing paths. By providing multiple processing methods for each task, the diversity of data processing path selection is increased, enabling the storage server to better set the data processing path for the application to meet the requirements.

In conjunction with the first aspect, in a second possible implementation of the first aspect, the determining The parameter value corresponding to the target parameter of the data processing path includes: acquiring a plurality of tasks corresponding to each data processing path and at least one processing manner of each task; and according to the multiple tasks and each task And at least one processing manner, calculating parameter values corresponding to the target parameter values of each of the data processing paths. By calculating the parameter values corresponding to the target parameter values for each data processing path, it is possible to more precisely set the data processing path that satisfies the requirements for the application.

In conjunction with the second possible implementation of the first aspect, in a third possible implementation manner of the first aspect, the calculating, according to the multiple tasks, and the processing manner of each of the tasks, And the parameter value corresponding to the target parameter value includes: generating a directed acyclic graph DAG according to the multiple tasks and at least one processing manner of each task, where the DAG is used to indicate the application And a plurality of data processing paths; and calculating, according to the DAG, parameter values corresponding to the target parameter values of each of the data processing paths. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

With reference to the first aspect, in a fourth possible implementation manner of the first aspect, the data processing path corresponding to the determined parameter value that is closest to the target parameter value is set as target data processing of the application The path includes: determining whether a parameter value corresponding to the target parameter value of each data processing path is not greater than the target parameter value; if there is a data processing path, the parameter value corresponding to the target parameter value is not greater than the a target parameter value, the data processing path is set as the target data processing path of the application; if there are at least two data processing paths and the parameter value corresponding to the target parameter value is not greater than the target parameter value, then And acquiring, in the at least two data processing paths, a first data data processing path having a smallest parameter value, and setting the first data data processing path as a target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the determining, by the data processing path, a parameter value corresponding to the target parameter value is not greater than After the target parameter value, the method further includes: if the parameter value corresponding to each of the data processing path and the target parameter value is greater than the target parameter value, multiple data processing paths in the application And obtaining a second data processing path having the smallest parameter value; setting the second data processing path as the target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

With reference to the first aspect, in a sixth possible implementation manner of the first aspect, the data processing path corresponding to the determined parameter value that is closest to the target parameter value is set as target data processing of the application After the path, the method further includes: when receiving the data write request of the application, storing the data of the application according to the target data processing path. The data I/O performance of the application is improved by storing the application data according to the target data processing path that satisfies the application performance, cost, and the like.

In a second aspect, an apparatus for setting a data processing path for an application in a storage server is provided, the apparatus comprising a plurality of functional modules for performing the method of the first aspect described above. In a possible implementation, the apparatus further includes other functional modules for performing the methods described in the various possible implementations of the first aspect described above. By obtaining the target parameter value that reflects the performance, cost, etc. of the application, and the parameter value closest to the target parameter value The corresponding data processing path is set as the target data processing path of the application, so that the target data processing path can meet the performance, cost and the like requirements of the application, and the data I/O performance of the application is improved.

In a third aspect, a server is provided, comprising a processor, a memory for storing processor-executable instructions; wherein the processor is configured to:

With reference to the third aspect, in a first possible implementation manner of the third aspect, each task in the storage server includes at least one processing manner, and selecting one processing from at least one processing manner of each task The manner forms the plurality of data processing paths. By providing multiple processing methods for each task, the diversity of data processing path selection is increased, enabling the storage server to better set the data processing path for the application to meet the requirements.

With reference to the third aspect, in a second possible implementation manner of the third aspect, the determining, by the determining, the parameter value corresponding to the target parameter in each data processing path of the storage server includes: obtaining, corresponding to each data processing path, And a plurality of tasks and at least one processing manner of each task; calculating, according to the plurality of tasks and at least one processing manner of each task, a parameter value corresponding to each of the data processing paths and the target parameter value . By calculating the parameter values corresponding to the target parameter values for each data processing path, it is possible to more precisely set the data processing path that satisfies the requirements for the application.

In conjunction with the second possible implementation of the third aspect, in a third possible implementation manner of the third aspect, the processor is configured to: process according to the multiple tasks and at least one of each task In a manner, a directed acyclic graph DAG is generated, where the DAG is used to indicate a plurality of data processing paths of the application; and the parameter values corresponding to the target parameter values are calculated according to the DAG. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

With reference to the third aspect, in a fourth possible implementation manner of the third aspect, the processor is configured to: determine whether a parameter value corresponding to the target parameter value of each data processing path is not greater than a target parameter value; if there is a data processing path and the parameter value corresponding to the target parameter value is not greater than the target parameter value, setting the one data processing path as the target data processing path of the application; And obtaining, by the two data processing paths, a parameter value corresponding to the target parameter value that is not greater than the target parameter value, acquiring, in the at least two data processing paths, a first data data processing path with a minimum parameter value, and The first data data processing path is set as the target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

In conjunction with the fourth possible implementation of the third aspect, in a fifth possible implementation manner of the third aspect, the processor is configured to: if each of the data processing paths corresponds to the target parameter value The parameter value is greater than the target parameter value, and in the multiple data processing paths of the application, the second data location with the smallest parameter value is obtained. The second data processing path is set as the target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

With reference to the third aspect, in a sixth possible implementation manner of the third aspect, the processor is configured to: when receiving a data write request of the application, according to the target data processing path The data of the application is stored. The data I/O performance of the application is improved by storing the application data according to the target data processing path that satisfies the application performance, cost, and the like.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.

1 is a schematic structural diagram of a storage system according to an embodiment of the present invention;

2 is a flowchart of a method for setting a data processing path for an application in a storage server according to an embodiment of the present invention;

3 is a DAG diagram of a data processing path according to an embodiment of the present invention;

4 is a block diagram of an apparatus for setting a data processing path for an application in a storage server according to an embodiment of the present invention;

FIG. 5 is a block diagram of an apparatus for setting a data processing path for an application in a storage server according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic structural diagram of a storage system according to an embodiment of the present invention. Referring to Figure 1, the storage server and storage medium should be included. In addition, the storage system is also connected to at least one application server through a network. Only one application server is shown in FIG. 1. The number of application servers is not specifically limited in the embodiment of the present invention.

Each application server of the at least one application server includes a central processing unit (CPU), a hardware resource such as a memory, a cache, and a software module such as an application and an operating system, and is used to acquire a target of the target data processing path of the application. The parameter value is sent to the storage server through the specified interface, so that the storage server can set a data processing path that satisfies the target parameter value for the application in the multiple data processing paths of the application according to the target parameter value of the application. An application server can provide services for one or more applications, including database applications, Virtual Desktop Infrastructure (VDI) applications, multimedia applications, and the like; operating systems include file systems, volume management, multipathing software, and the like. The database application may include an online transaction processing (OLTP) type database application, an online analytical processing (OLAP) type database application.

Among them, the storage medium includes Solid State Drives (SSD) and Hard Disk Drive (Hard Disk Drive, HDD), used to store data for an application. The storage server includes hardware resources such as a central processing unit (CPU), memory, and cache. The storage server is used to provide data storage services to applications in the application server. The storage server is provided with a plurality of data processing paths, and the data processing path is a plurality of tasks that the application server needs to perform to complete the data I/O of the application, for example, a storage medium type selection task, a data cache task, a data deduplication task, and a data. Compression tasks, data encoding tasks, data redundancy tasks, data addressing tasks, and more. The storage server is configured to receive a target parameter value of the application sent by the application server, and select a data processing path that meets the target parameter value in the multiple data processing paths of the application as the data processing path of the application, to supply the data through the setting. The processing path accesses data in the storage server, such as data reading, writing, and deleting.

FIG. 2 is a flowchart of a method for setting a data processing path for an application in a storage server according to an embodiment of the present invention. Referring to FIG. 2, the following steps are included:

201. The storage server receives a target parameter value of a target data processing path set by the application server and set for the application.

It should be noted that when the application performs data I/O, the application server usually needs to perform multiple tasks, such as storage medium type selection task, data cache task, data deduplication task, data compression task, data encoding task, and data redundancy. Remaining tasks, data addressing tasks, etc. The physical environment of the storage system is fixed for a certain period of time. The physical environment includes the storage space, the type and size of the storage medium, and the like. A plurality of data processing paths are set in the storage server, and the data processing path is a plurality of tasks that the application server needs to perform to complete the data I/O of the application.

The storage server provides at least one processing manner for each task, that is, each task includes at least one processing manner. For example, for a data compression task, the storage server is the data. The compression task provides two methods of data compression: data compression mode 1 and data compression mode 2. Selecting one processing mode from at least one processing manner of each task forms a plurality of data processing paths.

Among them, in the process of performing data I/O, each processing mode corresponds to multiple processing consumption, such as cost consumption, delay consumption, data reliability consumption, and the like. In the embodiment of the present invention, the multiple processing consumption corresponding to each processing mode is referred to as a parameter value corresponding to each processing mode, and the parameter value may include multiple parameter values, such as a cost parameter value, a delay parameter value, and a data reliability. Parameter values, etc. Correspondingly, the parameter values of each processing mode are used to indicate the multiple processing consumption corresponding to each processing mode. For example, the cost parameter value corresponds to the cost consumption, the delay parameter value corresponds to the delay consumption, and the data reliability parameter value corresponds to the data reliability. Consumption of multiple treatments such as sexual consumption.

It should be noted that each data processing path is formed by selecting one processing manner in at least one processing manner of each task, that is, each data processing path includes multiple processing manners. Correspondingly, the parameter value corresponding to the data I/O of each data processing path refers to the sum of each parameter value corresponding to multiple data processing modes in each data processing path.

The target data processing path refers to a data processing path set by the storage server for the application, so that when the application performs data storage, the data is processed according to the processing manner included in the target data processing path.

In the embodiment of the present invention, the application administrator may preset a target parameter value of the target data processing path for the application by using the application server, where the target parameter value is used to indicate various processing consumptions expected by the application when performing data I/O.

Specifically, the application server may send the target parameter value to the storage server through the specified interface, and the storage server selects the data processing path that satisfies the target parameter value as the target data processing path among the plurality of data processing paths according to the target parameter value. Among them, the target parameter values include target cost, target delay, and target data reliability. Its The target cost is used to indicate the cost consumption of the data processing path, and may include a target storage cost of the application data, such as a storage medium storing the application data, a storage space, and the like; the target delay is used to indicate the delay consumption of the data processing path, Refers to the target time interval from when the application sends a data I/O request to the storage device to when the data I/O response is received; the target data reliability is used to indicate the data reliability consumption of the data processing path, that is, the application-to-data I/O The requirement to request the integrity or consistency of the requested data, for example, the integrity requirement of the data requested for the data I/O request is 95%, that is, the data I/O request is tolerated to a maximum of 5% data loss, then The target data reliability consumption can be 5%.

It should be noted that, as the target parameter value of the application may include the parameter value of the plurality of target parameters, one or more of the target parameter values may be set when the target parameter value is set, and the embodiment of the present invention does not specifically limited. The user can be an application developer or an application administrator.

In the embodiment of the present invention, the application server may provide a configuration interface for the target parameter value of the application, and in the configuration interface of the target parameter value, the user may set the target cost, target delay, and target data reliability of the application. set. The configuration interface of the target parameter value may be displayed during the process of installing the application by the user, or displayed after the user triggers the configuration operation of the application, or embedded in the software dedicated to configuring the processing consumption target of the application, in the embodiment of the present invention This is not specifically limited.

202. The storage server determines a parameter value corresponding to each target data processing path and the target parameter value in the storage server.

In the embodiment of the present invention, after determining the target parameter value of the target data processing path of the application, the parameter value corresponding to each data processing path and the target parameter value may be correspondingly obtained according to the multiple target parameters corresponding to the target parameter value. That is, the parameter values corresponding to the plurality of target parameters in each data processing path are obtained. Since each task of the application corresponds to multiple processing modes, different processing modes are adopted, and the parameter values of the application are different, so that the parameter values corresponding to different data processing paths are different.

The process of determining a parameter value corresponding to each data processing path and the target parameter value in the storage server may be: acquiring multiple tasks corresponding to each data processing path and at least one processing mode of each task; And at least one processing manner of each task, and calculating parameter values corresponding to the target parameter values of each data processing path.

It should be noted that when performing data I/O, the execution order of multiple tasks of the application is different, so that the priority corresponding to each task is different. For example, if the compression task is prioritized over the execution of the cache task, the priority of the compression task is prioritized. The level is higher than the cache task. After selecting one processing mode in at least one processing mode of each task, the priority corresponding to the task may be considered as the priority of the selected processing mode.

Specifically, in order to be able to calculate the parameter value of each data processing path more conveniently and quickly, the storage server may perform DAG (Directed Acyclic Graph) according to multiple data processing paths of the application in the embodiment of the present invention. Modeling so that each path in the DAG indicates a data processing path. Then, calculating a parameter value corresponding to the target parameter value of each data processing path of the application according to the established model, the process may be: generating a directed acyclic graph DAG according to the plurality of tasks and at least one processing manner of each task The DAG is used to indicate a plurality of data processing paths of the application; and the parameter values corresponding to the target parameter values of each data processing path are calculated according to the DAG.

Specifically, the generating process of the DAG may be: acquiring a priority of each task, and determining each processing manner corresponding to each task as a node of the DAG; determining, according to the priority of each task, each task Each processing mode corresponds to a hierarchy of nodes in the DAG, wherein the higher the priority of the task, the higher the level of the node corresponding to the processing mode; the initial node and the ending node of the DAG are determined, and the initial node is each The starting point of the data processing path and the highest level in the DAG, the end node being the end of each data processing path and at the DAG The middle level is the lowest; the edge is added between the nodes of the adjacent level, and the directed edge points to the lower level node in the adjacent level, that is, the processing method of connecting two different priorities for each directed edge The weight of the directed edge can be set to the processing consumption of the processing method pointed by the directed edge, and finally a directed acyclic graph with the processing parameter as the node and the processing mode as the directed edge weight is formed. The parameter value of the processing mode corresponds to the target parameter value. When the target parameter value includes the parameter value corresponding to the multiple parameter, the parameter value of the processing mode also includes the parameter value corresponding to the multiple parameter. Correspondingly, each of the directed edge weights in the DAG is a parameter value corresponding to a plurality of parameters of the processing mode.

It should be noted that the initial node pair of the DAG should be applied to the data I/O starting point when performing data storage, such as the time when the data I/O enters the storage system from the application layer (that is, the storage system receives the application data). The time of the I/O); the end node of the DAG should be applied to the end of the data I/O when the data is stored, such as the data I/O is returned from the storage system to the application layer (ie, the application receives the return from the storage system) Time of data I/O).

The following describes the generation process of the DAG in combination with the above-mentioned DAG generation method. For example, the application includes four tasks, and the four tasks correspond to four priorities. The first priority task is a data compression task, and the data compression task corresponding to the processing method includes: compression algorithm A, compression algorithm B, no compression, The de-duplication algorithm A, the de-duplication algorithm B, and the non-deduplication; the second priority task is a data caching task, and the corresponding processing manner of the data caching task includes: SSD cache, NVRAM (Non-Volatile Random Access Memory, non-volatile Scaling random access memory) cache, memory cache, no cache; the third priority task is data redundancy task, the data redundancy task corresponding processing methods include: DHT (Distributed Hash Table), EC (Erasure Code, data redundancy coding), three copies, no redundancy; the fourth priority task is a data backup task, and the corresponding processing methods of the data backup task include: dual-active, global backup, block service, file service, Object service.

Suppose, the initial node of the DAG is P0 and the end node is Pe. In the DAG, P0 has the highest level and Pe has the lowest level. In the specific implementation, the initial node can be set to a level of 0 by default. According to different priorities of the foregoing tasks, the first priority corresponds to the first level in the DAG, the second priority corresponds to the second level in the DAG, and the third priority corresponds to the third level in the DAG, and the fourth priority corresponds to the third level in the DAG. Four levels, the end node has a level of 5. For convenience of description, multiple processing modes corresponding to the above four priority tasks are numbered in corresponding nodes in the DAG, and the specific correspondence is as follows: compression algorithm A (P11), compression algorithm B (P12), and no compression C ( P13), deduplication algorithm A (P14), deduplication algorithm B (P15), no deduplication (P16); SSD cache (P21), NVRAM buffer (P22), memory cache (P23), no cache (P24); DHT (P31), EC (P32), three copies (P33), no redundancy (P34), live (P41), global backup (P42), block service (P43), file service (P44), object service ( P45). The content in "()" is the node number corresponding to the processing method.

According to the above processing manner, in the hierarchy in the DAG, a directed edge pointing to a lower hierarchical node is added between nodes of two adjacent hierarchical levels, and a DAG map constructing a processing manner as shown in FIG. 3 can be obtained. In Figure 3, the DAG shows a plurality of data processing paths for the application, where each path included in the DAG corresponds to each data processing path that should be applied.

It should be noted that the priorities of the foregoing different tasks are only exemplified, and the embodiment of the present invention does not specifically limit this.

In the embodiment of the present invention, different tasks include different tasks, and different tasks correspond to different processing manners. Therefore, DAGs for different applications may be generated according to processing consumption targets of different applications. When the processing mode of the storage server changes, for example, a new processing mode is added to the task included in the application, the DAG of the application may be automatically generated according to the changed processing mode and the parameter value of each processing mode after the change. And regained Take the data processing path that satisfies the target parameter value of the application. Or, after adding a new processing mode, prompting the user to re-set the target parameter value of the application, that is, resetting the target cost, the target delay, and the reliability of the target data; if the user resets After the target parameter value of the application is determined, the DAG of the application is regenerated according to the changed processing mode and the parameter value of each processing mode after the change, and the new target parameter is re-acquired. The data processing path for the value. By automatically regenerating the DAG of the application according to the changed processing manner and re-acquiring the data processing path that satisfies the target parameter value, the flexibility of the data processing path setting is increased, and the efficiency of the data processing path setting is improved.

In the embodiment of the present invention, after the DAG is generated, the parameter values of each data processing path may be calculated, and the calculation process is as follows:

Get all the paths from the initial node to the end node, sum the weights of the directed edges contained in each path, and get the parameter values of each data processing path.

203. The storage server sets the determined data processing path corresponding to the parameter value closest to the target parameter value as the target data processing path of the application.

In the embodiment of the present invention, the process of setting the data processing path corresponding to the parameter value closest to the target parameter value to the target data processing path of the application in the plurality of data processing paths may be: first, determining each data processing path and Whether the parameter value corresponding to the target parameter value is not greater than the target parameter value. Secondly, according to the difference of judgment results, there are three possible situations:

First, if there is a data processing path and the parameter value corresponding to the target parameter value is not greater than the target parameter value, then a data processing path is set as the target data processing path of the application.

Secondly, if there are at least two data processing paths and the parameter values corresponding to the target parameter values are not greater than the target parameter values, then in at least two data processing paths, the first data data processing path with the smallest parameter value is obtained, and The first data data processing path is set to the target data processing path of the application.

Third, if the parameter value corresponding to each data processing path and the target parameter value is greater than the target parameter value, the second data processing path with the smallest parameter value is obtained in the multiple data processing paths of the application; The processing path is set to the target data processing path of the application.

It should be noted that, in the foregoing third case, a minimum parameter value threshold may be set. If the calculated parameter value of the second data processing path is still greater than the minimum parameter value threshold, the current storage system cannot satisfy the user setting. The target parameter value can display the prompt information for resetting the target parameter value to the user.

In the embodiment of the present invention, the problem of obtaining the data processing path corresponding to the parameter value closest to the target parameter value by the DAG model is transformed into the problem of obtaining the shortest path in the DAG.

Among them, the basic principle of calculating the shortest path in DAG is to adopt the method of dynamic programming: V represents the set of all nodes in the constructed DAG, S represents the set of nodes that have obtained the shortest path, and VS=T indicates that the shortest has not been determined. The set of nodes for the path. Initially, S={V0}, V0 is the starting point, T={other nodes}, for any Vi (Vi belongs to T), if there is a direct connection of V0 to Vi <V0, Vi>, <V0, Vi> is The weight on the arc, if not present, <V0, Vi> is set to infinity. Select a node W whose value is the smallest distance from V0, and add S to modify the distance value of other nodes in T: if W is added as the intermediate node, the distance value from V0 to Vi is less than W. If the path is short, modify the distance value. Repeat the above steps until all nodes are included in S, ie S=V.

204. The storage server stores the data of the application according to the target data processing path when receiving the data write request of the application.

In the embodiment of the present invention, after determining the target data processing path of the application, the multiple tasks corresponding to the target data processing path, the processing manner of each task, and the parameter values of the target data processing path may be output, so that the user can be clear The data processing path of the application and the parameter value of the data processing path are obtained, that is, the processing cost, the delay consumption, the data reliability consumption, and the like are consumed by the data processing path. If the user determines that the output target data processing path is the final data processing path of the application, when the storage server receives the data write request of the application, the application data is stored according to the determined target data processing path.

It should be noted that, when the application interacts with the storage system, not only the foregoing write operation but also operations such as reading and deleting are included, which is not specifically limited in the embodiment of the present invention. For different operations corresponding to different data I/Os, the storage server receives the corresponding operation request of the application, and performs operations such as reading, deleting, and the like through the storage device according to the corresponding operation request, and returns the processing result to the application.

It should be noted that the method provided by the embodiment of the present invention may be applied to a high-end storage array, a mass storage cloud storage system, a Doradoa V5 all-flash array storage system, and the like, which is not specifically limited in the embodiment of the present invention.

The method provided by the embodiment of the present invention obtains the target parameter value of the target data processing path, and sets the data processing path corresponding to the parameter value closest to the target parameter value as the application target in the multiple data processing paths of the application. The data processing path realizes the processing of the applied data by using the optimal processing method under the constraints of resources, performance and cost. Further, the problem of obtaining the optimal data processing path is transformed into the problem of obtaining the shortest path in the DAG model, which greatly simplifies the calculation of the data processing, and makes the acquisition process of the optimal data processing path easier and faster.

FIG. 4 is a block diagram of an apparatus for setting a data processing path for an application in a storage server according to an example of the present invention, including: a receiving module 401, an obtaining module 402, and a setting module 403.

The receiving module 401 is configured to receive a target parameter value of the target data processing path set by the application server and set by the application;

The obtaining module 402 is configured to determine a parameter value corresponding to each target data parameter in the storage server and the target parameter value;

The setting module 403 is configured to set the determined data processing path corresponding to the parameter value closest to the target parameter value as the target data processing path of the application.

Optionally, each task in the storage server includes at least one processing manner, and selecting one processing manner from at least one processing manner of each task to form a plurality of data processing paths. By providing multiple processing methods for each task, the diversity of data processing path selection is increased, enabling the storage server to better set the data processing path for the application to meet the requirements.

Optionally, the acquiring module is configured to acquire multiple tasks corresponding to each data processing path and at least one processing manner of each task; and calculate each data processing according to multiple tasks and at least one processing manner of each task The parameter value corresponding to the path and the target parameter value. By calculating the parameter values corresponding to the target parameter values for each data processing path, it is possible to more precisely set the data processing path that satisfies the requirements for the application.

Optionally, the obtaining module is configured to generate a directed acyclic graph DAG according to the plurality of tasks and at least one processing manner of each task, where the DAG is used to indicate multiple data processing paths of the application; and each data processing is calculated according to the DAG. The parameter value corresponding to the path and the target parameter value. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of data processing is greatly simplified, and the optimal data processing path is obtained. The process is quicker and easier.

Optionally, the setting module is configured to determine whether the parameter value corresponding to each data processing path and the target parameter value is not greater than the target parameter value; if there is a data processing path corresponding to the target parameter value, the parameter value is not greater than the target parameter value, Setting a data processing path as the target data processing path of the application; if there are at least two data processing paths and the parameter value corresponding to the target parameter value is not greater than the target parameter value, obtaining the minimum parameter value in at least two data processing paths The first data data processing path sets the first data data processing path to the target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

Optionally, the setting module is configured to: if the parameter value corresponding to each of the data processing path and the target parameter value is greater than the target parameter value, obtain the second data processing path with the smallest parameter value in the multiple data processing paths of the application; The second data processing path is set to the target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

Optionally, the device further includes:

The storage module is configured to store the data of the application according to the target data processing path when receiving the data write request of the application. The data I/O performance of the application is improved by storing the application data according to the target data processing path that satisfies the application performance, cost, and the like.

The device provided by the embodiment of the present invention obtains a target parameter value that reflects performance, cost, and the like of the application, and sets a data processing path corresponding to the parameter value closest to the target parameter value as the target data processing path of the application, so that the target The data processing path can meet the requirements of application performance, cost, etc., and improve the data I/O performance of the application.

It should be noted that, in the data processing path obtaining device in the storage system provided by the foregoing embodiment, when the data processing path is obtained, only the division of each functional module is used as an example. In an actual application, the foregoing function may be set as needed. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the fault-tolerant processing device and the fault-tolerant processing method are provided in the same embodiment, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.

FIG. 5 is a block diagram of an apparatus for setting a data processing path for an application in a storage server according to an embodiment of the present invention. For example, the apparatus 500 may be provided as a server. Referring to FIG. 5, apparatus 500 includes a processing component 522 that further includes one or more processors, and memory resources represented by memory 532 for storing instructions executable by processing component 522, such as an application. An application stored in memory 532 can include one or more modules each corresponding to a set of instructions. Further, the processing component 522 is configured to execute instructions to perform the data processing path acquisition method in the storage system described above.

Apparatus 500 can also include a power supply component 526 configured to perform power management of apparatus 500, a wired or wireless network interface 550 configured to connect apparatus 500 to the network, and an input/output (I/O) interface 558. Device 500 can operate based on an operating system stored in memory 532, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

The processing component 522 is configured to execute an instruction to execute the data processing path obtaining method in the foregoing storage system, including the following steps:

Optionally, each task in the storage server includes at least one processing manner, and selecting one processing manner from at least one processing manner of each task to form the multiple data processing paths. By providing multiple processing methods for each task, the diversity of data processing path selection is increased, enabling the storage server to better set the data processing path for the application to meet the requirements.

Optionally, the determining, by the storage server, the parameter value corresponding to the target parameter in each data processing path includes: acquiring a plurality of tasks corresponding to each data processing path and at least one processing manner of each task; And calculating, according to the plurality of tasks and at least one processing manner of each task, parameter values corresponding to the target parameter values of each of the data processing paths. By calculating the parameter values corresponding to the target parameter values for each data processing path, it is possible to more precisely set the data processing path that satisfies the requirements for the application.

Optionally, calculating, according to the multiple tasks and at least one processing manner of each task, calculating parameter values corresponding to the target parameter values of each of the data processing paths, including: according to the multiple tasks and At least one processing manner of each task, generating a directed acyclic graph DAG, the DAG is used to indicate a plurality of data processing paths of the application; and calculating each of the data processing paths and the target according to the DAG The parameter value corresponding to the parameter value. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

Optionally, setting the determined data processing path corresponding to the parameter value closest to the target parameter value to the target data processing path of the application comprises: determining each of the data processing paths and the target Whether the parameter value corresponding to the parameter value is not greater than the target parameter value; if there is a data processing path and the parameter value corresponding to the target parameter value is not greater than the target parameter value, the one data processing path is set as the The target data processing path of the application; if there are at least two data processing paths and the parameter value corresponding to the target parameter value is not greater than the target parameter value, the acquired parameter value is the smallest among the at least two data processing paths a first data data processing path and setting the first data data processing path to a target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

Optionally, after the determining whether the parameter value corresponding to the target parameter value is not greater than the target parameter value, the method further includes: if each of the data processing paths and the If the parameter value corresponding to the target parameter value is greater than the target parameter value, acquiring, in the plurality of data processing paths of the application, a second data processing path having the smallest parameter value; setting the second data processing path to The target data processing path of the application. By transforming the acquisition problem of the optimal data processing path into the shortest path in the DAG model, the computational complexity of the data processing is greatly simplified, and the acquisition process of the optimal data processing path is more convenient and faster.

Optionally, after the determined data processing path corresponding to the determined parameter value closest to the target parameter value is set as the target data processing path of the application, the method further includes: when the application is received When the data is written to the request, the data of the application is stored according to the target data processing path. By satisfying the application The target data processing path for performance, cost, etc. stores the application data, improving the data I/O performance of the application.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

A method for setting a data processing path for an application in a storage server, where the storage server is provided with a plurality of data processing paths, where the data processing path is required for the application server to complete data I/O of the application. Task

The method is characterized in that: the method comprises:

Receiving, by the application server, a target parameter value of a target data processing path set for the application;

Determining, by the parameter value corresponding to the target parameter value, each data processing path in the storage server;

A data processing path corresponding to the determined parameter value closest to the target parameter value is set as the target data processing path of the application.
The method according to claim 1, wherein each task in the storage server comprises at least one processing manner, and selecting one processing manner from at least one processing manner of each task to form the plurality of processing Data processing path.
The method according to claim 1, wherein the determining the parameter values corresponding to each of the data processing paths in the storage server and the target parameters comprises:

Obtaining a plurality of tasks corresponding to each of the data processing paths and at least one processing manner of each of the tasks;

And calculating, according to the plurality of tasks and at least one processing manner of each task, parameter values corresponding to the target parameter values of each of the data processing paths.
The method according to claim 3, wherein the parameter values corresponding to the target parameter values are calculated according to the plurality of tasks and at least one processing manner of each task include:

Generating a directed acyclic graph DAG for indicating a plurality of data processing paths of the application according to the plurality of tasks and at least one processing manner of each task;

Calculating, according to the DAG, a parameter value corresponding to each of the data processing paths and the target parameter value.
The method according to claim 1, wherein the setting of the determined data processing path corresponding to the parameter value closest to the target parameter value to the target data processing path of the application comprises:

Determining whether the parameter value corresponding to each of the data processing paths and the target parameter value is not greater than the target parameter value;

If there is a data processing path and the parameter value corresponding to the target parameter value is not greater than the target parameter value, setting the one data processing path as the target data processing path of the application;

If the parameter value corresponding to the at least two data processing paths and the target parameter value is not greater than the target parameter value, acquiring, in the at least two data processing paths, a first data data processing path having a minimum parameter value, And setting the first data data processing path to a target data processing path of the application.
The method according to claim 5, wherein said determining said each data processing path is After the parameter value corresponding to the target parameter value is not greater than the target parameter value, the method further includes:

If the parameter value corresponding to the target parameter value is greater than the target parameter value, the second data processing path with the smallest parameter value is obtained in the multiple data processing paths of the application;

The second data processing path is set to a target data processing path of the application.
The method according to claim 1, wherein after the determined data processing path corresponding to the parameter value closest to the target parameter value is set as the target data processing path of the application, the method Also includes:

When receiving a data write request of the application, storing data of the application according to the target data processing path.
An apparatus for setting a data processing path for an application in a storage server, where the storage server is provided with a plurality of data processing paths, where the data processing path is required for the application server to complete data I/O of the application. Task

The device is characterized in that:

a receiving module, configured to receive a target parameter value of a target data processing path set by the application server and set by the application;

An obtaining module, configured to determine a parameter value corresponding to each target data parameter in the storage server and the target parameter value;

And a setting module, configured to set the determined data processing path corresponding to the parameter value closest to the target parameter value as the target data processing path of the application.
The apparatus according to claim 8, wherein each task in the storage server comprises at least one processing manner, and selecting one processing manner from at least one processing manner of each task forms the plurality of processing Data processing path.
The device according to claim 8, wherein the obtaining module is configured to acquire a plurality of tasks corresponding to each data processing path and at least one processing mode of each task; At least one processing manner of each task, calculating a parameter value corresponding to each of the data processing paths and the target parameter value.
The apparatus according to claim 10, wherein the obtaining module is configured to generate a directed acyclic graph DAG according to the plurality of tasks and at least one processing manner of each task, wherein the DAG is used to indicate a plurality of data processing paths of the application; calculating, according to the DAG, parameter values corresponding to the target parameter values of each of the data processing paths.
The device according to claim 8, wherein the setting module is configured to determine whether a parameter value corresponding to each of the data processing paths and the target parameter value is not greater than the target parameter value; if a data exists And if the parameter value corresponding to the target parameter value is not greater than the target parameter value, setting the one data processing path to the target data processing path of the application; if there are at least two data processing paths and the Head The parameter value corresponding to the parameter value is not greater than the target parameter value, and in the at least two data processing paths, acquiring a first data data processing path with the smallest parameter value, and setting the first data data processing path The target data processing path for the application.
The device according to claim 12, wherein the setting module is configured to: if the parameter value corresponding to each of the data processing paths and the target parameter value is greater than the target parameter value, then the application is And acquiring, in the plurality of data processing paths, a second data processing path having the smallest parameter value; and setting the second data processing path as the target data processing path of the application.
The device according to claim 8, wherein the device further comprises:

a storage module, configured to store data of the application according to the target data processing path when receiving a data write request of the application.