KR101924099B1 - Method and Apparatus of performing Policy-Driven Storage Service Federation - Google Patents

Abstract

A method of performing a policy based storage service federation includes: receiving a request from a client and receiving a request interceptor for identifying whether the policy is applicable to the request; step; A policy acquiring step of acquiring a policy by evaluating request information and policy status of the request; And a storage service set acquisition step of acquiring a set of customized storage services based on the policy, and can provide a framework corresponding to the storage federation layer for rapid application development.

Description

METHOD AND APPARATUS FOR PERFORMANCE-BASED STORAGE SERVICE FEDERATION FIELD OF THE INVENTION [0001]

The present invention relates to a policy-based storage service federation performing method and apparatus. And more particularly, to a method and apparatus for policy-based storage service federation using a policy-based storage service federation framework.

In the IT industry, data of a large size and a large size are generated rapidly, but the processing speed of the IT industry is relatively low, and it is necessary to store and manage the data.

In the storage and management of such data, as the huge amount of data is generated, there are advantages in data analysis field and data storage. These big data are so large that it is difficult to store them all in one repository. In addition, big datasets and datastores have the problem of reducing data access performance. Therefore, there is a need for a system that can automatically and precisely determine how data is stored and its location, while providing scalability in keeping with the exponential growth in data generation.

In this regard, Hewlett Packard Enterprise 3PAR Peer Motion software allows the federation of multiple third-party Store Server systems. However, the biggest problem is that the federation function is incompatible with other systems.

Next, there is research on several storage federation methods. In most cases, however, it focuses on the union of cloud storage to prevent vendor dependence and to simplify the management of multiple accounts. The problem with this approach is that it concentrates only on one service type (eg cloud storage).

Finally, with respect to the discussion on the service selection approach, currently existing methods have the problem that there are limitations in providing only statically specified numerical services. Also, most storage federation still lacks policy-based service selection.

Accordingly, an object of the present invention is to provide a framework corresponding to a storage federation layer for rapid application development.

It is another object of the present invention to provide a policy-based storage service framework capable of compatibility with other systems and capable of policy-based service selection.

According to another aspect of the present invention, there is provided a policy based storage service federation method comprising: receiving a request from a client by a request interceptor to apply a policy to the request; policy applicable) whether the policy is applicable or not. The policy-based storage service association performing method further includes a policy acquiring step of acquiring a policy by evaluating the request information and the policy status of the request. The policy-based storage service association execution method includes a storage service set acquisition step of acquiring a collection of customized storage services based on the policy.

According to one embodiment, the policy-based storage service association performing method includes: a service credential request step of requesting a service credential for the set of storage services; Performing an authentication for the service credential; And a request transmitting step of adding the service information and the authority authentication to the request, and transmitting the request to the storage manager.

According to an embodiment, the policy-based storage service association performing method includes: a service information transmitting step in which a virtual storage resource interface transmits the service information to an instance manager; A service implementation class search step of searching for a service implementation class for each service based on the service information; And generating and transmitting a service implementation class instance based on the discovered service implementation class and transmitting the service implementation class instance to the virtual storage resource interface. The policy-based storage service federation method may further include transmitting the request, the service privilege associated with the privilege authentication, and the request / service privilege / service implementation class instance transferring the service implementation class instances to the task manager by the virtual storage resource interface Step < / RTI >

According to an embodiment, the policy-based storage service association execution method includes: a task process / response merge step of processing a task for a plurality of services accessed asynchronously by the task manager and merging responses of the services; And a final response generation step of generating a final response so that the client can receive a response received from the task manager by the virtual storage resource interface.

According to one embodiment, the storage service profile for each service may be defined by a respective metric corresponding to availability, write performance, read performance, price, and used space. In addition, the above-mentioned policy is classified into P1 to P4, P1 corresponds to a case where correction and read / write operations are mainly performed, P2 corresponds to a case where only a reading operation is performed without performing a write operation . In addition, P3 corresponds to a case where writing and reading operations are hardly performed, and P4 corresponds to a case where data is less important. It may be characterized as being used only for storage although there is little possibility of use.

According to an exemplary embodiment, the service profile includes S1 to S4, and a normalized weight may be given to each of the S1 to S4 by the service profile. For each of the P1 to P4, a priority order is determined for each of the metrics, and the order of selecting the S1 to S4 for each of the P1 to P4 based on the normalized weight and the priority can be determined have.

According to another aspect of the present invention, an apparatus for performing a policy-based storage service association comprises: a request receiving unit for receiving a request from a client, identifying whether the request is policy applicable, And a request interceptor for acquiring a policy by evaluating the status and obtaining a set of customized storage services based on the policy. The policy-based storage service federation performing apparatus further includes a matching policy verification unit that receives a policy request from the request interceptor, acquires an active policy, and returns a matching policy according to the request information and the policy status to the request interceptor ≪ / RTI >

According to one embodiment, the policy-based storage service federation performing device comprises: a meta data manager for obtaining the active policy according to the policy request and transmitting the active policy to the matching policy verifier; And a service selector for obtaining a service profile based on the policy and generating a set of customized storage services based on the service profile.

According to one embodiment, the request interceptor may request service credentials for the set of storage services, perform authentication on the service credentials, and add service information and authorization credentials to the request. The policy-based storage service federation performing apparatus further includes an authentication manager (OAuth manager) for receiving a request for the service credential from the request interceptor and providing a token for the authentication to the request interceptor .

According to an embodiment, the policy-based storage service federation performing apparatus may further include a virtual storage resource interface for receiving the service information and the authorization-added request from the request interceptor. The policy-based storage service association performing device searches for a service implementation class for each service based on the service information received from the virtual storage resource interface, An instance manager for creating an instance, and transmitting the service implementation class instance to the virtual storage resource interface. In addition, the policy-based storage service federation execution device may receive the request, the service privilege associated with the privilege authentication and the service implementation class instances from the virtual storage resource interface, and perform operations on a plurality of services that access asynchronously , An operation manager that processes the response, merges the responses of the services, and transmits the response to the virtual storage resource interface. Meanwhile, the virtual storage resource interface may generate a final response so that the client can receive a response received from the task manager.

According to one embodiment, the policy-based storage service federation performing device includes a plurality of storage service interfaces for transmitting the request from the instance manager and the task manager to a plurality of storage devices each performing a plurality of storage services, As shown in FIG. At this time, the plurality of storage devices may include cloud storage, distributed file system, and object storage.

According to another aspect of the present invention, an apparatus for performing policy-based storage service federation comprises: a client for sending a request for policy-based storage service federation; And means for receiving the request from the client, identifying whether the request is policy applicable, obtaining a policy by evaluating request information and policy status for the request, And a policy engine that obtains a set of services.

According to an embodiment, the policy-based storage service federation performing apparatus may further include a policy management unit that receives the service information and the request to which the rights authentication is added from the policy engine, And a storage service manager for searching for an implementation class and creating a service implementation class instance based on the searched service implementation class.

According to one embodiment, the storage service manager processes a task for a plurality of services accessed asynchronously by the task manager, based on the request, the service authorization associated with the authorization, and the service implementation class instances. Combine the responses of the services, and send the merged response to the client.

The policy-based storage service federation method according to the present invention improves data throughput and performance through horizontal and vertical scaling and parallel processing techniques, and provides a framework corresponding to a storage federation layer for rapid application development. Can be provided.

A policy-based storage service federation method according to the present invention is a policy-based storage service framework capable of being compatible with other systems and selecting a policy-based service by a policy-based storage service through a service profile framework can be provided.

1 shows a detailed configuration diagram of an apparatus for performing a policy-based storage service association according to the present invention.
2 is a flow diagram of a method for performing a policy-based storage service federation according to the present invention.
3 illustrates a storage service profile according to an example of the present invention.
4 illustrates a normalized storage service profile according to an example of the present invention.
FIG. 5 illustrates a comparison between a normalized service profile and a normalized service profile according to the present invention.
FIG. 6 is a diagram that defines the ranking of each metric for a policy according to an exemplary embodiment of the present invention.
FIG. 7 is a bar graph showing the ratios of respective metrics for a policy according to an exemplary embodiment of the present invention.
FIG. 8 is a diagram illustrating a policy-based service distance in accordance with an exemplary embodiment of the present invention.
FIGS. 9A to 9C show throughputs according to a file size and a block size in performing a write operation, a read operation, and a delete operation according to an example of the present invention.
10 illustrates a configuration of an algorithm for policy-based storage service selection according to an exemplary embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It will be possible. The present invention is capable of various modifications and various forms, and specific embodiments are illustrated in the drawings and described in detail in the text. It is to be understood, however, that the invention is not intended to be limited to the particular forms disclosed, but on the contrary, is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

Like reference numerals are used for similar elements in describing each drawing.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The term "and / or" includes any combination of a plurality of related listed items or any of a plurality of related listed items.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Should not.

The suffix "module "," block ", and "part" for components used in the following description are given or mixed in consideration of ease of specification only and do not have their own distinct meanings or roles .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a method of performing a policy-based storage service association and a method of performing a policy-based storage service association according to the present invention will be described. In this regard, the purpose of the framework presented in the present invention is to provide a storage federation layer for rapid application development. The main purpose of this framework is to provide a federated storage layer for rapid application development while providing a sufficient amount of required resources. It can operate on any file-based storage service by implementing a storage service interface. This allows the client to integrate desired services into the framework instead of providing a limited number of implementation services.

In this regard, FIG. 1 illustrates a detailed configuration diagram of an apparatus for performing a policy-based storage service association according to the present invention. 1, the storage service federation performing apparatus 1000 includes a client 10, a REST API 100, an authentication controller 200, and a storage Federation Framework 300 do. The storage federation framework 300 includes a policy engine 400, a meta data manager 500, an authentication manager 600, a storage manager 700, And an interface 800.

The client 10 provides a request for policy-based storage service federation to the storage federation framework 300 via the REST API 100. [ At this time, the request may be provided to the storage federation framework 300 through the authentication controller 200.

The REST API 100 receives a request for the storage service association from the client 10 and performs a storage service association through the framework 300. [ That is, the REST API 100 plays a role of converting the framework into a stand-alone service solution. It can also be extended to architectures and domain models.

The authentication controller 200 component is integrated to provide a token-based authentication procedure. This additional component provides an authentication token for client registration, authentication, and API endpoint access. The domain model, on the other hand, is extended to define the client's own service authentication, policy, and associated files.

The policy engine 400 receives the request from the client 10 and identifies whether the request is policy applicable to the request. In addition, the policy engine 400 obtains a policy by evaluating request information and policy status of the request, and acquires a set of customized storage services based on the policy.

More specifically, the policy engine 400 serves as a brain of the framework 300. They are responsible for decision making, service selection, verification requests, and supplementing requests. Policy-based data distribution algorithms are implemented to make automated decisions. Two policy models have been implemented. The first model defines the importance of service profile metrics. The second model defines the number of replicas of a given condition. This model allows the client to define the number of copies of the data separately. The policy engine 400 includes a request interceptor 410, a matching policy verifier 40, and a service selector 430.

The request interceptor 410 receives a request from the client 10, identifies whether or not the policy can be applied to the request, evaluates the request information and the policy status to obtain a policy, And acquires a set of customized storage services based on the policy.

In addition, the request interceptor 410 may request service credentials for the set of storage services, perform authentication for the service credentials, and add service information and authorization to the request .

That is, the storage manager 700 is responsible for verifying and supplementing each request before sending it to the storage manager 700. To complement the request, the service credentials must be included. Therefore, all requests must be validated by the request interceptor 410.

The matching policy verifier 420 receives a policy request from the request interceptor 420, obtains an active policy, and returns a matching policy according to the policy information to the request interceptor 420 do. That is, the matching policy verifier 420 is responsible for checking whether all the policy conditions match the current file.

The service selector 430 obtains a service profile based on the policy, and generates a set of customized storage services based on the service profile. That is, the service selector 430 plays a role of selecting the best service group according to a set of policies and replication factors.

The meta data manager 500 obtains the active policy according to the policy request and transmits the active policy to the matching policy verifier 420. More specifically, the meta data manager 500 defines a hierarchical domain model of storage services, policies, and files. It also provides an interface that allows other components to be searched, generated, and modified in the model. The metadata manager 500 may use a code-first approach of the Microsoft Entity Framework. This is a domain-based design technique focused on creating domain classes instead of any database design. Using the Entity Framework, you can easily extend the domain model by creating new domain classes or by overwriting existing ones.

The authentication manager 600 receives a request for the service credential from the request interceptor 410 and provides a token for the authentication to the request interceptor 410. More specifically, the authentication manager 600 is configured to use OAuth 2.0 authentication. It typically targets cloud storage and consists of two detailed components. That is, the metadata manager 600 includes an authentication URL creator 610 and a token manager 620.

The authentication URL creator 610 configures an authentication URL and a redirect URL based on a profile of a given service. Also, the authentication URL creator 610 checks whether the configured URL is valid. The token manager 620 plays a role of canceling the request, update and service access token of the token.

The storage manager 700 receives the service information and the request to which the authorization authentication is added from the policy engine 400. In addition, the storage manager 700 searches for a service implementation class for each service based on the received service information, and creates a service implementation class instance based on the searched service implementation class.

On the other hand, the storage manager 700 processes operations for a plurality of services accessed asynchronously by the task manager 730, based on the request, the service privilege associated with the authorization, and the service implementation class instances do. Also, the storage manager 700 merges the responses of the services and transmits the merged response to the client 10.

That is, the storage manager 700 is a component that manages resources of a central repository. The storage manager 700 provides an interface for accessing a plurality of storage service resources from a single point. In addition, parallel processing technology is implemented to improve execution time and access multiple services at the same time. The storage manager 700 dynamically creates reflection classes and uses reflection-oriented programming to provide access services. The storage manager 700 is divided into the following three detailed components.

That is, the storage manager 700 includes a virtual storage resource interface 710, a storage service instance manager 720, and a task manager 730 .

The virtual storage resource interface 710 receives the service information and the request to which the rights authentication is added from the request interceptor 410. That is, the virtual storage resource interface 710 transforms all the separate storages in the storage for which seamless services are provided to provide a single access point.

The instance manager 720 searches for a service implementation class for each service based on the service information received from the virtual storage resource interface 710 and generates a service implementation class instance based on the discovered service implementation class . In addition, the instance manager 720 transmits the service implementation class instance to the virtual storage resource interface 710. That is, the instance manager 720 performs discovery and instance creation of a service implementation class. The instance manager 720 searches for a class in which each service is implemented according to a unique name defined in the service domain model and storage service interface inheritance through reflection-oriented programming.

The task manager 730 receives the request, the service rights associated with the authorization, and the service implementation class instances from the virtual storage resource interface 710. In addition, the task manager 730 processes tasks for a plurality of asynchronously accessed services, merges the responses of the services, and sends the responses to the virtual storage resource interface 710. At this time, the virtual storage resource interface 710 generates a final response so that the client 10 can receive a response received from the task manager 730. That is, the task manager 730 manages a plurality of services and responses accessed in parallel.

The storage service interface 800 transmits the request from the instance manager 720 and the operation manager 730 to a plurality of storage devices 800 that perform each of a plurality of storage services. At this time, the plurality of storage devices 800 may include cloud storage, distributed file system, and object storage as shown. That is, the storage service interface 800 must be implemented so that each storage service can access the present framework. In addition, implementations of the interface should dynamically discover and enable instances of the service class using reflection-oriented programming.

In the above, we have seen devices that perform policy-based storage service federation. In the following, we will look at how to implement policy-based storage service federation. In this regard, FIG. 2 is a flow diagram of a method for performing a policy-based storage service federation in accordance with the present invention.

As shown in FIG. 2, the method for performing the policy-based storage service association comprises the steps of: i. A service federation process performed by the Policy Engine and ii. And a service association process by a storage manager (Storage Manager).

I. Policy Engine Identification and Complementation of Request (Policy Engine verifies and complements the request)

A. Policy Applicability Identification Step : The request interceptor receives the request from the client and identifies whether the request is policy applicable to the request. That is, the request interceptor identifies whether this request is policy-applicable. If it is applicable (policy-applicable), for example if you want to create a file or search for a duplicated file, go to step B. Conversely, if not applicable, the request will be sent to all services or to specific services. For example, if a file that has not been cloned is read or space is available, go to step D.

B. Policy Acquisition Step : The policy information is obtained by evaluating the request information and the policy status of the request. That is, the connection with the policy is made through the Matching Policy Verifier. It receives the Active policy from the Metadata Manager, evaluates the compilation and request information and policy status, and returns when there is a matching policy.

C. Storage Service Set Acquisition Step : Obtain a set of customized storage services based on the above policy. That is, a collection of customized storage services based on policies is obtained through a service selector. This brings the storage service profile from the Metadata Manager. Then apply a policy-based algorithm to find the most appropriate service for the requested procedure.

In this regard, a storage service profile for each service may be defined by a respective metric corresponding to availability, write performance, read performance, price, and used space. In addition, the policy may be classified as P1 to P4. In this case, P1 corresponds to the case of performing the correction and read / write operations, and P2 corresponds to the case where only the reading operation is performed without performing the writing operation. In addition, P3 corresponds to a case where writing and reading operations are hardly performed, and P4 corresponds to a case where data is less important. It may be characterized as being used only for storage although there is little possibility of use.

Meanwhile, the service profile includes S1 through S4, and a normalized weight is assigned to each of the S1 through S4 according to the service profile. At this time, for each of the P1 to P4, priority is determined for each metric, and the order of selecting the S1 to S4 for each of the P1 to P4 based on the normalized weight and the priority is determined . This is discussed in more detail below.

D. Service Credential Request Step : Request service credentials for the set of storage services. That is, the request interceptor requests a service credential from a metadata manager. At this time, if the service uses the OAuth protocol, perform step E.

E. Authentication step : Authentication is performed on the service credential. In other words, authentication of OAuth is performed through OAuth Manager (OAuth Manager). If the token expires when it is refreshed, the permissions are updated in the Metadata Manager and a new token is issued to the Request Interceptor.

F. Request Transmission Step : Adds service information and authorization to the request, and sends the request to the storage manager. In other words, service information and authorization are added to the client's request and sent to the Storage Manager.

Ii. Storage Manager delivers the request to the matching storage service. The storage manager also generates a response to the client.

A. Service Information Transmission Step : The virtual storage resource interface transmits service information to the Storage Service Instance Manager.

B. Service Implementation Class Search Step : The service implementation class for each service is searched based on the service information. That is, the Storage Service Instance Manager uses a reflection-oriented programming approach to find the implementation class of each service according to a unique name defined in storage service information and storage service interface inheritance.

C. Service Implementation Class Instance Creation / Transmission Step : Creates a service implementation class instance based on the discovered service implementation class, and transmits the service implementation class instance to the virtual storage resource interface. In other words, each implemented service class instance is created and sent to the Virtual Storage Resource Interface.

D. Request / Service Authority / Service Implementation Class Instance Transfer Phase : Client requests, service privileges and instances of service classes are sent to the Task Manager.

E. Work Process / Reply Merge Step : The virtual storage resource interface generates a final response so that the client can receive a response received from the task manager. That is, the Task Manager uses parallel processing techniques to process all services that are accessed asynchronously. And then merges the responses of all services and transmits them to the virtual storage resource interface.

F. Final Reply transmission step : The Virtual Storage Resource Interface transmits the final response in response to the client's request.

The above-described policy-based storage service association execution method and apparatus have the following differences.

(A) This framework presents the possibility for a number of file-based heterogeneous storage services that are integrated. For example, data from storage such as cloud storage, distributed file systems, object storage, and ftp can be managed through a single point of control in a single virtual storage.

I. Parallel processing techniques and data throughput and performance can be improved.

Ii. Horizontal and vertical scaling

Iii. Automated decision

(B) A policy-based algorithm that defines the number of storage areas to provide clients with the most precise requirements.

(C) a simple policy model (eg, availability, cost, space, performance) based on storage service metrics that allows clients to autonomously define data distribution rules;

In addition, the method and apparatus for performing the policy-based storage service association described above can be applied to the following application fields.

Application field (Application field)

     (1) Big Data Storage (Big Data Storage)

     (2) Cloud Federation

     (3) Storage as a Service (SaaS)

     (4) Parallel Data Processing

     (5) Enterprise Storage Solutions

 In addition, the method and apparatus for performing the policy-based storage service association described above have the following expected effects.

(1) Choice of policy-based storage services

I. Storage service profile definition : Four kinds of service profiles are defined as S1, S2, S3, S4, and five storage metrics are defined when each information is generated. All metrics are expressed as good, with lower values. Based on this, we redefine the metrics.

A. Availability: The percentage of time during which the service was not normally provided during the entire usage time of the service, expressed as a percentage.

B. Write and Read Performance: Represents the average time taken to process 1MB of data.

C. Cost: Cost of using 1 GB storage

D. Used space: Percentage of space currently in use

In this regard, FIG. 3 illustrates a storage service profile according to an example of the present invention. Figure 4 also shows a normalized storage service profile according to an example of the present invention. 5 is a diagram showing a comparison between a normalized service profile and a normalized service profile according to the present invention.

As shown in FIG. 3, the service profiles S1 to S4 have different weights for write performance, read performance, cost, used space, and availability. .

Ii. Storage Service Profile Normalization: As shown in FIG. 4, the service profile should perform normalization to fairly determine the contribution of each metric in the selection process. On the other hand, the normalized service profile shown on the right side in FIG. 5 indicates a write performance, a read performance, a cost, a used space, and a availability, It is easier to determine what kind of tendency there is.

As described above, the present invention seeks to distribute and replicate data to a plurality of storage services based on a dynamic and personalized policy. Each storage service is defined as a group of metrics. Policies specify the importance of each metric to store specific data. Finally, many replicas are provided. Story services have a finite set of defined metrics. There are a tremendous amount of other metrics to analyze how to maximize the storage efficiency of data. Nevertheless, their value is not only a different range, but even the meanings of the values are opposite, which adds to confusion when compared. For example, the higher the availability, the better the performance. The lower the cost, the better. With this availability, users will want as much availability as possible. Conversely, you will want to pay the minimum price. Because of these characteristics, it is necessary to normalize all the values between 0 and 1 for objective comparison. Therefore, in the present patent, it is assumed that a smaller value is a better service in all the measured values. For example, availability defines the rate at which services are not provided properly. Therefore, the lower the value, the better the availability.

Meanwhile, the present invention has a set of policies for determining the most suitable storage service group. Each policy is defined by two characteristics. First, the condition must correspond to the policy along with the data. Second, we define the weighting method for each service according to the order of importance. This means that the first order has the highest weight.

Iii. Policy definition: Four policies are defined by the following conditions.

A. P1, mainly to perform the correction and read / write operations. In this case, availability is paramount. For example, when you are working on a document.

B. P2, if you do not perform a write operation but mainly read. As a result, the most important metrics are read performance and availability. For example, when using media files or PDF documents.

C. P3, if you rarely write and read. For example, data created for backup purposes. Therefore, the most important characteristic is the available storage space and price.

D. P4, less important data. It is rarely used, but only for storage. In this case, price is the most important metric.

iv. Storage service selection: Finally, we apply a defined policy-based selection algorithm to find the best selection order. This is shown in FIG.

A. Storage service selection order of P1 and P2: [S4, S2, S1, S3].

B. Storage service selection order of P3: [S2, S1, S3, S4].

C. Storage service selection order of P4: [S4, S3, S2, S1].

Next, as described above, let us consider a mathematical definition for performing policy-based storage service federation.

Define a set of storage services as S = {s1, s2, ..., sn} and set a set of policies as P = {p1, p2, ... , pm}, and replication factor r . We want to select a subset R from r to S that minimizes the distance between P and R.

I. Storage service model

Each storage service can be defined as a vector:

The value of the Jth metric for service i is given by:

,

,

Ii. Policy Model

Each policy can be defined as a vector:

The importance of the jth metric based on policy i is given as follows.

,

,

Iii. Metrics weight

The metric weights are distributed exponentially in order of importance. (The weight of metric) The statistical weight j based on the policy i is expressed as follows.

The statistical weight j for all policies P is expressed as follows.

On the other hand, in order to solve the service selection problem, we used the k nearest neighbor algorithm. In general, k-NN is used to classify query points according to neighboring neighbors. To apply to this problem, the query point Q is at the center of the space that maximizes all storage metrics, and k (the number of neighbors) is determined by r (replication factor). Euclidean distance was used for distance measurement. Each attribute reflects the weight of the policy.

In this regard, Figure 6 is a diagram that defines the ranking of each metric for a policy according to the example of the present invention. 7 is a bar graph showing the ratio of each metric to the policy according to an exemplary embodiment of the present invention. FIG. 8 is a diagram illustrating a policy-based service distance in accordance with an exemplary embodiment of the present invention.

 Referring to FIG. 6, Availability is the most important for P1 and P2, Used space is most important for P3, and Cost is the most important for P4.

Referring to FIG. 7, the importance of the write performance, the read performance, the cost, the used space, and the availability are defined with respect to P1 to P4, Can be understood at a glance.

(2) Parallel processing performance

 To evaluate the performance of parallel processing of striped files and no-striped files in this framework, we used several file sizes with different stripe blocks sizes. In this regard, FIGS. 9A to 9C show the throughput according to the file size and the block size in performing the write operation, the read operation, and the delete operation according to the example of the present invention. As shown in FIGS. 9A to 9C, the read and write operations for each block size were performed and evaluated. We confirmed that the combination of striping and parallel processing improves performance and data throughput.

In the foregoing, a policy-based service federation framework and a policy-based service federation execution method and apparatus using the framework have been described. The federation execution device in which the framework is installed is a terminal, a user equipment, and can be connected to a plurality of storage service devices (for example, cloud storage, distributed file system, and object storage) through the terminal and user equipment.

Lt; / RTI > Further, the federation performing apparatus in which the framework is installed is a gateway, and can be connected to a plurality of storage service apparatuses (for example, cloud storage, distributed file system, and object storage) through the gateway.

10 illustrates a configuration of an algorithm for policy-based storage service selection according to an exemplary embodiment of the present invention. As shown in FIG. 10, the storage service set is S, the number of metrics is defined as k, and the replication factor is defined as r . That is, as described above, a set of storage services is defined as S = {s1, s2, ..., sn}, and a set of policies is defined as P = {p1, p2, ... , pm}, and replication factor r . We want to select a subset R from r to S that minimizes the distance between P and R.

This framework provides a federation layer component for easy application development. Meanwhile, the independent service can be provided based on the implemented REST API.

According to at least one embodiment of the present invention, data throughput and performance can be improved through horizontal and vertical scaling and parallel processing techniques to provide a framework corresponding to the storage federation layer for rapid application development .

In addition, according to at least one embodiment of the present invention, a policy-based storage service framework capable of compatibility with other systems and capable of policy-based service selection by a policy-based storage service through a service profile It has the advantage that it can provide.

1000: Storage service federation device
10: Client 100: REST API
200: authentication controller 300: storage federation framework
400: Policy Engine 410: Request Interceptor
420: matching policy validator 430: service selector
500: Metadata manager 600: Authentication manager
610: Authentication URL creator 620: Token manager
700: Storage Manager 710: Virtual Storage Resource Interface
720: Instance Manager 730: Operation Manager
800: Storage service interface

Claims (14)

A method for performing a policy based storage service federation performed by a device that performs a policy-based storage service association,
A policy applicability identification step of receiving a request from a client and receiving a request interceptor to identify whether or not the request is policy applicable;
A policy acquiring step of acquiring a policy matching the request information and the policy status of the request; And
And acquiring a set of storage services customized based on the policy. The method according to claim 1,
A service credential request step of requesting a service credential for the set of storage services;
Performing an authentication for the service credential; And
Further comprising a request transmission step of adding the service information and the rights authentication, which is information for specifying the respective storage services included in the plurality of storage services, to the request, and transmitting the request to the storage manager. How to perform federation. 3. The method of claim 2,
A service information transmission step in which the virtual storage resource interface transmits the service information to the instance manager;
A service implementation class search step of searching for a service implementation class for each service based on the service information;
Generating a service implementation class instance based on the searched service implementation class and transmitting the service implementation class instance to the virtual storage resource interface; And
Further comprising a step of transmitting the request, a service authorization associated with the authorization, and a request / service authorization / service implementation class instance transferring the service implementation class instances to the operations manager by the virtual storage resource interface. Way. The method of claim 3,
A task processing / response merging step of processing a task for a plurality of services accessed asynchronously by the task manager, and merging responses of the services; And
Further comprising a final response generation step of generating a final response such that the client receives a response received from the task manager by the virtual storage resource interface. The method of claim 3,
The storage service profile for each service is defined by a respective metric corresponding to availability, write performance, read performance, price, and used space, and
The policy is classified as P1 to P4,
P1 corresponds to a case where a correction and a read / write operation are performed,
P2 corresponds to a case where only a reading operation is performed without performing a writing operation,
P3 corresponds to a case in which writing and reading are not performed,
P4 is less important data. Wherein the policy-based storage service association is for use only for archiving purposes. 6. The method of claim 5,
The service profile includes S1 through S4,
For each of S1 to S4, a normalized weight is assigned to each of the service profiles,
For each of P1 to P4, a priority is determined for each of the metrics,
Based on the normalized weight and the priority, determines the order of selecting the S1 to S4 for each of the P1 to P4. 1. An apparatus for performing a policy-based storage service association,
The method comprising: receiving a request from a client; identifying whether a policy is applicable to the request; obtaining a matching policy according to the request information for the request and the policy status; A request interceptor for obtaining a set; And
A matching policy validator for receiving a policy request from the request interceptor, obtaining an active policy, and returning a matching policy to the request interceptor according to the request information and the policy status. . 8. The method of claim 7,
A metadata manager for obtaining the active policy according to the policy request and transmitting the active policy to the matching policy verifier; And
Further comprising a service selector for obtaining a service profile based on the policy and creating a collection of customized storage services based on the service profile. 8. The method of claim 7,
Wherein the request interceptor requests service credentials for the set of storage services, performs authentication for the service credentials, and generates service information, which is information for specifying each of the storage services included in the plurality of storage services, Adds authorization to the request, and
Further comprising an authentication manager (OAuth manager) for receiving a request for the service credential from the request interceptor and providing a token for the authentication to the request interceptor. 10. The method of claim 9,
A virtual storage resource interface for receiving the service information and the authorization request added from the request interceptor;
Searching for a service implementation class for each service based on the service information received from the virtual storage resource interface, creating a service implementation class instance based on the searched service implementation class, An instance manager that sends to the virtual storage resource interface;
Receiving from the virtual storage resource interface the service request, the service authorization associated with the authorization, and the service implementation class instances, processing operations for a plurality of services accessing asynchronously, merging responses of the services, And an operation manager for transmitting the response to the virtual storage resource interface,
Wherein the virtual storage resource interface generates a final response so that the client can receive a response received from the task manager. 11. The method of claim 10,
Further comprising a plurality of storage service interfaces transmitting the request from the instance manager and the task manager to a plurality of storage devices performing each of a plurality of storage services,
Wherein the plurality of storage devices comprises cloud storage, distributed file system, and object storage. 1. An apparatus for performing a policy-based storage service association,
A client for sending a request for policy-based storage service federation; And
Receiving the request from the client, identifying whether the request is policy applicable, acquiring a policy consistent with the request information for the request and the policy status, And a policy engine to obtain a set of services. 13. The method of claim 12,
A storage medium storing a program for causing a computer to receive, from a policy engine, service information, which is information for identifying each storage service included in a plurality of storage services, and the request to which authorization authentication is added, Further comprising a storage service manager for searching for a service implementation class for the service implementation class and for creating a service implementation class instance based on the searched service implementation class. 14. The method of claim 13,
Wherein the storage service manager processes work for a plurality of services accessed asynchronously by the task manager based on the request, the service privilege associated with the authorization, and the service implementation class instances, And send the merged response to the client. ≪ Desc / Clms Page number 24 >

Images