WO2013001214A1

WO2013001214A1 - Method and system for the distributed storage of information with optimized management of resources

Info

Publication number: WO2013001214A1
Application number: PCT/FR2012/051431
Authority: WO
Inventors: Anthony Foulonneau; Philippe Raipin Parvedy
Original assignee: France Telecom
Priority date: 2011-06-28
Filing date: 2012-06-22
Publication date: 2013-01-03
Also published as: FR2977337A1

Abstract

The invention relates to a method of storage by distribution of sets of information (Cl, C2,...Cj,...Cn) within a plurality of equipments (Sl,...Sm,...Sp) furnished with storage means (MM1,...MMm,...MMp) and implanted within a communication network NWK. The method according to the invention makes provision for tracking the values of transmission bit rate (Drtl,...Drtm,...Drtp) that each of said equipments (Sl,...Sm,...Sp), is capable of producing, in addition to a tracking of the values (Sdl,...Sdm,...Sdp) of their available storage volume. Such a distribution based on tracking of the two essential resources, namely the available volume of data in each equipment Si (for i=l to p) and the available transmission bit rate Drti at the level of this equipment Si will make it possible to optimize the utilization of these two resources and hence potentially to decrease the number of equipments necessary for storing all the sets of information.

Description

Method and system for distributed storage of information with optimized resource management

1. Field of the invention

The invention relates to a storage method for distributing sets of information in a plurality of devices provided with storage means.

Such devices are commonly used to store large amounts of information, either as part of archiving applications, or as part of other more specific applications that make use of such information.

2. Prior Art

In the current state of the art, the sets of information to be stored are generally related to a given application and are then all of the same kind, so that their distribution within the different equipment is achieved by means of a monitoring of amounts of data stored in these devices. In case of saturation of an equipment, that is to say when this equipment no longer has enough storage volume to accommodate a new set of information, a transfer of information sets from this equipment to a or several other equipment is automatically realized, in order to distribute the information to be stored in a substantially homogeneous manner while limiting as much as possible the number of equipment necessary for storing all the sets of information.

3. Objectives of the invention

The telecommunications sector is currently undergoing a transformation from a mode of operation in which end users, whether they are humans or machines, retain their own information and applications. to another mode of operation in which the information of the end users will not be stored locally, in their own equipment, but remotely, in remote equipment located within communication infrastructures allowing users to communicate between them. This other mode of operation is called today "cloud computing", the English expression "Cloud Computing" well known to those skilled in the art.

The inventors have observed that this other mode of operation reveals difficulties that the distribution storage methods known from the state of the art can not overcome entirely satisfactorily.

In fact, the remote storage of user information requires the distribution of different equipment heterogeneous information sets, such as images, audiovisual sequences, text files or executable programs.

It thus appeared that a distribution management solely based on a monitoring of the storage volumes available in the various equipment could not make it possible to obtain a satisfactory quality of service for the users. In particular, the inventors have noted that it may happen that, although a given equipment still has enough storage space to accommodate one or more additional sets of information, it is actually in a saturation situation because that restitution of the information already stored requires a transmission rate higher than that which this equipment is actually capable of producing. This can happen, for example, if a large proportion of the information already stored consists of video type data.

The present invention proposes a solution that does not have this drawback, by proposing a method in which the management of the distribution of the sets of information to be stored is no longer carried out exclusively on the basis of a monitoring of the storage volumes available in the various equipment.

4. Presentation of the invention

Indeed, according to a functional aspect, the invention relates to a method according to the introductory paragraph, characterized in that it includes a step of determining a transmission rate value specific to each of said equipment, and a distribution step of said sets of information among said equipment based on the transmission rate values specific to these devices.

By providing a follow-up of the transmission rate values of the different devices, the invention makes it possible to take into account an instantaneous capacity of each device to restore the sets of information that it contains, and thus ensures a distribution that is no longer performed solely on the basis of the storage capacity of these devices, but also on their ability to provide a certain quality of service expected by users. A distribution based on a follow-up of the two essential resources, namely the volume of data available in each equipment and the transmission rate available at the level of this equipment, will make it possible to optimize the exploitation of these two resources and thus potentially reduce the number of resources. equipment needed to store all information sets.

Various implementations are possible to exploit the transmission rate values offered by the different equipment. According to one embodiment of the invention, a method as described above includes a step of evaluating a diagnostic parameter specific to each device and simultaneously representing a volume of data stored in this equipment and the transmission rate available at this equipment. The use of such a diagnostic parameter is advantageous in that this parameter is at the same time representative of the volume of storage available within given equipment, and of the instantaneous flow rate that it is capable of providing, so that dispatch management based on the monitoring of this unique diagnostic parameter will simultaneously take into account the two essential aspects of the available storage volume and available throughput.

The diagnostic parameter can be defined in various ways, provided that it allows to observe variations of available flow and storage volume. It could thus be formed by a difference or a product between these two quantities.

In a particular embodiment of the invention, the evaluation step of the equipment-specific diagnostic parameter includes a step of calculating a quotient between the volume of data stored in this equipment and the transmission rate. available at this equipment.

In order to optimize the number of equipment required for distributed information storage, an advantageous embodiment of the invention provides for a step of identifying a critical resource by comparing a specific diagnostic parameter. to all the equipment with a diagnosis parameter specific to a union of all the sets of information to be stored, said critical resource can be a storage volume or a transmission rate available.

The identification of the critical resource will make it possible to detect situations of saturation of certain equipment, which will then be no longer intended to accommodate new sets of information, but instead serve as a source to feed other equipment identified as who not yet saturated.

In such an implementation mode, when no shortage of critical resource is detected at one of the equipment, an additional set of information is transmitted to this equipment from equipment at which a shortage of critical resource has been previously identified. Before ordering the transfer, the system will take care to verify that the equipment intended to receive the additional set of information has a sufficient available memory space to store it.

To optimize the distribution, it will be advantageous to identify the kind of information present in a device that will have reached saturation. Indeed, if it is for example mainly information requiring a large amount of storage but little transmission rate, it will be desirable to relieve the equipment saturated by transferring some of such information to equipment still having a large storage capacity but having only a low availability in terms of instantaneous transmission rate.

To this end, when a critical resource shortage is detected at one of the equipment, a comparison is made between the value of the diagnostic parameter specific to this equipment and a reference value representative of a quotient between the total data volume corresponding to a union of all the sets of information to be stored and a data rate of transmission associated with a predetermined quality of service expected by at least one user of the method.

Equipment that is in a situation of critical resource shortage can thus be described as "stable" if the value of its diagnostic parameter is close to the reference value, of "source of information with a low diagnostic parameter" if a difference or a ratio between the reference value and the value of the diagnostic parameter of the equipment is greater than a first threshold, or "source of information with high diagnostic parameter" if a difference or a ratio between the value of the diagnostic parameter of the equipment and the reference value is greater than a second threshold, which may also be identical to the first threshold.

Thus, equipment in a situation of critical resource shortage can be qualified as a "low diagnostic parameter information source" if a quotient between the reference value and the diagnostic parameter value of this equipment is greater than 80%, and "high diagnostic parameter information source" if a quotient between the value of its diagnostic parameter and the reference value is greater than 120%.

On the basis of such equipment qualifications in a situation of shortage, it will then be possible to optimize the distribution of the sets of information to be stored by ensuring that, when the value of the diagnostic parameter specific to the equipment to be received the additional set of information is greater than the reference value, said additional set of information is taken from another equipment that has been identified as a "low diagnostic parameter information source". Conversely, when the value of the diagnostic parameter specific to the equipment that is to receive the additional set of information is less than the reference value, said additional set of information will preferably be taken from another equipment that has been identified. as a "source of information with high diagnostic parameters".

In order to allow an exact census of the types of equipment which are in a situation of shortage of critical resource and can thus serve as source to the other equipment, any collection of a set of information from a piece of equipment is followed by the execution a new step of evaluation of the diagnostic parameter specific to this equipment.

The optimization principles described above will advantageously be transposed to situations in which new sets of information will have to be stored in the plurality of devices. Such a situation will then advantageously following steps :

. a step of determining a diagnostic parameter value corresponding to a new set of information to be added to the already stored sets,

. a step of comparing the value of this diagnostic parameter with a reference value representative of a quotient between the total data volume corresponding to a union of all the sets of information to be stored and a transmission rate associated with a quality predetermined service expected by at least one user of the method, and

if the diagnostic parameter is greater than the reference value,

. a step of searching for at least one device at which no shortage of critical resource has been detected, whose value of the diagnostic parameter is lower than the reference value, with a view to transferring the new set of information, or

if the diagnostic parameter is lower than the reference value,

. a step of searching for at least one device at which no shortage of critical resource has been detected, whose value of the diagnostic parameter is greater than the reference value, with a view to transferring the new set of information.

Finally, in the event that the search step has revealed within the plurality of equipment already used no equipment as desired, a new equipment will be added to said plurality in order to memorize the new set of information.

Such a process of adding equipment, conditioned on a finding of incapacity of the equipment included in the existing plurality to respect the conditions guaranteeing an optimal distribution of information, makes it possible to limit the number of equipment necessary for the implementation of implementation of the invention while providing end users with a quality of service that meets their expectations.

According to a first of its material aspects, the invention also relates to a communication system able to provide distributed storage of sets of information within a plurality of equipment provided with storage means for receiving, characterized in that it includes means for determining a transmission rate value specific to each of said equipment, and means for distributing said sets of information among said equipment based on the transmission rate values specific to these devices.

According to another of its material aspects, the invention also concerns, as an essential means for its implementation, a computer program characterized in that it comprises program code instructions for the implementation of a method according to the foregoing description when this program is executed by a processor. 5. List of figures

Other characteristics and advantages of the invention will appear more clearly on reading the following description of a particular embodiment, given as a simple illustrative and non-limiting example, in relation to FIG. form of a functional diagram a communication system in which the invention is implemented.

6. Description of an embodiment of the invention

6.1 General principle

The general principle of the invention is based on a management of the distribution of sets of information to be stored which is no longer performed exclusively on the basis of a monitoring of the storage volumes available in the various equipment. By providing for a follow-up of the flow rate values that the different equipments are capable of producing, the invention makes it possible to take into account an instantaneous capacity of each equipment item to restore the information sets it contains, and thus ensures a distribution that 'is no longer executed solely on the basis of the storage capacities of these devices, but also on their ability to provide a certain quality of service expected by users. A distribution based on a follow-up of the two essential resources, namely the volume of data available in each equipment and the transmission rate available at the level of this equipment, will make it possible to optimize the exploitation of these two resources and thus potentially reduce the number of resources. equipment needed to store all information sets.

6.2 Description of an embodiment

FIG. 1 illustrates a communication system SYST capable of ensuring distribution storage of sets of information (C1, C2,... Cj,... Cn) within a plurality of pieces of equipment (Sl, ... Sm, ... Sp) provided with storage means (MM1, ... MMm, ... MMp) and implanted within an NWK communication network. These sets of information may be from a terminal UT1, for example a telephone terminal held by a user UT1 and whose content is intended to be stored remotely in the NWK network.

The SYST system further includes an STMN storage manager including a CPU processor adapted to execute a tracking program extracted from a central memory not shown in this figure. At regular time intervals, the processor CPU sends requests (DrtRq (Sl), ... DrtRq (Sm), ... DrtRq (Sp)) to input / output means (I01, ... IOm , ... IOp) which are equipped with the equipment (Sl, ... Sm, ... Sp), in order to obtain transmission rate values (Drtl, ... Drtm, ... Drtp) that are capable of producing each of said equipment (Sl, ... Sm, ... Sp). In response to these requests, the devices (Sl, ... Sm, ... Sp) produce instantaneous values of transmission rate (Drtl, ... Drtm, ... Drtp) which are stored by the processor CPU in a table TAB in association with the volumes of storage (Sdl, ... Sdm, ... Sdp) still available within said equipment (Sl, ... Sm, ... Sp) and the data volumes (Vl, ..., V2, ... Vj, ... Vn) respectively corresponding to the sets of information (C1, C2, ... Cj, ... Cn). The processor CPU also dynamically records in said table TAB the devices (SI, Sl,... Sm,... Sp) in which the sets of information (C1, C2,... ... Cn).

It should be noted that, although in the example illustrated by FIG. 1, the STMN storage manager and the TAB table are implemented centrally within the NWK network, these elements can be distributed in the NWK network in the network. other embodiments of the invention.

Various implementations are possible to exploit the transmission rate values (Drtl, ... Drtm, ... Drtp) offered by the different equipment. According to one embodiment of the invention, the processor CPU calculates for each equipment Si (for i = 1 to p) the value R (Si) of a diagnostic parameter specific to each device and simultaneously representative of a the amount of data stored in this equipment and the available transmission rate at that equipment. In the present example, the diagnostic parameter R (Si) is defined as being a quotient between the volume of stored data Sdi in the equipment Si and the transmission rate Drti available at this equipment Si, and therefore written R (Si) = Sdi / DrTI.

In order to optimize the number of equipment required for distributed information storage, an advantageous embodiment of the invention provides a step of identifying a critical resource by comparing a diagnostic parameter R (S) specific to all the equipment with a reference value R (C) specific to a union of all the sets of information to be stored.

The value of the parameter R (S) is given by a quotient between the sum of the storage volumes of the equipment used and the sum of their transmission rates.

The value of the reference value R (C) is given by a quotient between the sum of the data volumes of the different sets of information to be stored and a predetermined rate value DrtO corresponding to the quality of service expected by the user. This DrtO bit rate value will for example be derived from a quantity called "Service Level Agreement", which is usually specified by the user to a storage service operator.

In a situation where R (S)> R (C), it is the transmission rate that will be considered as the critical resource, whereas, in a situation where R (S) <R (C), it is the storage capacity that will be considered as the critical resource

The identification of the critical resource makes it possible to detect saturation situations of certain equipment, which will then no longer be used to accommodate new equipment. sets of information, but instead serve as a source to feed other equipment identified as not yet saturated.

When a shortage of critical resource is detected at one of the equipment Si (for i = 1 to p), a comparison is made between the value of the diagnostic parameter R (Si) = Sdi / Drti specific to this equipment If and the reference value R (C).

An equipment Si which is in a situation of shortage of critical resource can thus be qualified as "stable" if the value of its diagnostic parameter R (Si) is close to the reference value, of "source of information to parameter of weak diagnosis "if a difference or a ratio between the reference value R (C) and the diagnostic parameter value R (Si) of the equipment Si is greater than a first threshold, or of a" parameter information source " high diagnostic value "if a difference or a ratio between the value of the diagnostic parameter R (Si) of the equipment Si and the reference value R (C) is greater than a second threshold, which may also be identical to the first threshold.

When no shortage of critical resource is detected at one of the equipment, an additional set of information is transmitted to this equipment from equipment at which a shortage of critical resource has been previously identified.

On the basis of the previously described qualifications applied to equipment in a situation of shortage, it will then be possible to optimize the distribution of the sets of information to be stored by ensuring that, when the value of the diagnostic parameter specific to the equipment that must receive the additional set of information is greater than the reference value R (C), said additional set of information is taken from another equipment that has been identified as a "source of information with a low diagnostic parameter ". Conversely, when the value of the diagnostic parameter specific to the equipment that is to receive the additional set of information is less than the reference value R (C), said additional set of information will preferably be taken from another equipment. which has been identified as a "source of information with high diagnostic parameters".

Such an operation of the method according to the invention can be illustrated by a simple example in which the equipment SI, Sm and Sp are three identical servers each having 10 TeraOctets of storage capacity and a network interface of 1 Gigabit / second.

If, by hypothesis, the server SI stores the contents of a video streaming service, while that the Sm and Sp servers are servers that store the contents of an archiving service, the critical resource will be the transmission rate. It may therefore happen that the SI equipment is in a situation of critical resource shortage because of a limitation in available bit rate placing this equipment SI unable to restore the contents stored in it quickly enough. In such a case, the diagnostic parameter (S1) will be larger than the reference value R (C) and the equipment SI will be considered as a "source of information with a high diagnostic parameter".

On the other hand, the equipment Sm and Sp being mainly used for archiving purposes and thus using only a small part of the transmission capacities of their input-output means IOm and IOp, their values of available bit rates Drtm and Drtp will be relatively high while their available storage volume will be rather low, so the values of their diagnostic parameters will be lower than the reference value R (C).

In this situation, the processor CPU will issue a transfer request TrRq (Cl, Sm) to the equipment SI, in order to instruct it to transfer at least one set of information, in this example the set Cl, to equipment Sm.

The collection of the set of information Cl from the SI equipment will be followed by the execution of a new step of evaluation of the diagnostic parameter R (S1) specific to this SI equipment, in order to allow an exact census of the types of equipment that are in a critical resource shortage and can serve as a source for other equipment.

The optimization principles described above will advantageously be transposed to situations in which new sets of information will have to be stored within the plurality of devices (Sl, ... Sm, ... Sp). Such a situation will advantageously then lead to the following steps:

. a step of determining a diagnostic parameter value R (Cn + 1) corresponding to a new set of information Cn + 1 to be added to the sets (C1, C2, ... Cj, ... Cn) already memorized,

. a step of comparing the value of this diagnostic parameter R (Cn + 1) with the reference value R (C), and

if the diagnostic parameter R (Cn + 1) is greater than the reference value R (C),

. a step of searching for at least one equipment Si (for i = 1 to p) at which no shortage of critical resource has been detected, whose value of the diagnostic parameter R (Si) is less than the value of reference R (C), in order to transfer the new set of information Cn + 1, or

if the diagnostic parameter R (Cn + 1) is smaller than the reference value R (C), . a step of searching for at least one equipment Si (for i = 1 to p) at which no shortage of critical resource has been detected, whose value of the diagnostic parameter R (Si) is greater than the value of reference R (C), for transferring the new set of information Cn + 1.

Finally, in the event that the search step has revealed within the plurality of equipment (Sl, ... Sm, ... Sp) already used no equipment If as desired, a new equipment Sp +1 will be added to said plurality (Sl, ... Sm, ... Sp) to store therein the new set of information Cn + 1.

Claims

A method for distributing storage of information sets in a plurality of devices provided with storage means, characterized in that it includes a step of determining a transmission rate value specific to each said equipment, and a step of distributing said sets of information among said equipment according to the transmission rate values specific to these devices.

2. Storage method according to claim 1, characterized in that it includes a step of evaluating a diagnostic parameter specific to each device and simultaneously representative of a volume of data stored in this equipment and the transmission rate available at this equipment.

3. Storage method according to claim 2, characterized in that the evaluation step of the equipment-specific diagnostic parameter includes a step of calculating a quotient between the volume of data stored in this equipment and the flow rate of the equipment. transmission available at this equipment.

4. Storage method according to claim 3, characterized in that it includes a step of identifying a critical resource by comparing a diagnostic parameter specific to all equipment with a diagnostic parameter specific to a joining all the sets of information to be stored, said critical resource being a storage volume or a transmission rate available.

5. Storage method according to claim 4, characterized in that, when no shortage of critical resource is detected at one of the equipment, an additional set of information is transmitted to this equipment from a device at the level of which a shortage of critical resource was previously detected.

6. Storage method according to claim 5, characterized in that, when a shortage of critical resource is detected at one of the equipment, a comparison is made between the value of the diagnostic parameter specific to this equipment and a a reference value representative of a quotient between the total data volume corresponding to a union of all sets of information to be stored and a transmission rate associated with a predetermined quality of service expected by at least one user of the method.

7. Storage method according to claim 6, characterized in that, when the value of the diagnostic parameter specific to the equipment that is to receive the additional set of information is greater than the reference value, said additional set of information is taken from another equipment for which a difference between the reference value and the value the diagnostic parameter specific to this other device is greater than a first predetermined threshold.

8. Storage method according to claim 6, characterized in that, when the value of the diagnostic parameter specific to the equipment that is to receive the additional set of information is less than the reference value, said additional set of information is taken from another equipment for which a difference between the value of the diagnostic parameter specific to this other equipment and the reference value is greater than a second predetermined threshold.

9. Transmission method according to claim 6, characterized in that a collection of a set of information from a device is followed by the execution of a new evaluation step of the diagnostic parameter specific to this equipment.

10. The storage method according to claim 4, including:

if the diagnostic parameter is greater than the reference value,

if the diagnostic parameter is lower than the reference value,

11. Storage method according to claim 10, characterized in that, if the search step has revealed within the plurality of equipment already used no equipment as desired, new equipment is added to said plurality for storing the new set of information therein.

12. A communication system capable of distributing storage of sets of information in a plurality of equipment provided with storage means, characterized in that it includes means for determining a bit rate value. transmission means specific to each of said equipment, and means for distributing said sets of information among said equipment based on the transmission rate values specific to these devices.

13. Computer program characterized in that it comprises program code instructions for the implementation of a method according to claim 1 when the program is executed by a processor.