JP2017130143A - Data distribution device, data distribution method, and data distribution program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the utilization efficiency of a distributed storage.SOLUTION: Provided is a secrecy distribution/data distribution system 1 with which it is possible to fragment a file and preserve distributedly in a plurality of storage providing terminals 5 and restore an original file using a smaller number of fragment files than the total number of fragments of the file, wherein the storageable capacity of at least one of the plurality of storage providing terminals 5 is different from the other, said system comprising: a file conversion information management unit 14 for calculating the number of fragments of a file in accordance with the storageable capacity of all of the plurality of storage providing terminals 5; a file-fragment file conversion unit 12 for fragmenting a file by the calculated number of fragments of the file; and a fragment file transfer unit 13 for transferring some of a plurality of fragmented fragment files that differ from each other to each of the storage providing terminals 5 in accordance with the storageable capacity of each storage providing terminal 5.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for fragmenting data and distributing and storing it in a plurality of storages.

  Currently, there is a technique for fragmenting data and distributing and storing it in a plurality of storages. For example, secret sharing technology and data sharing technology. Such existing data distribution technology is considered on the assumption that a plurality of storages having the same capacity are used.

Japanese Patent No. 5765416

  When constructing a large-scale storage system, it is relatively easy to use a plurality of storages having the same capacity. On the other hand, when a small user such as an office constructs a storage system using the above data distribution technique, it is conceivable to use a storage of a handheld terminal such as a notebook personal computer connected to the LAN in the office. However, in the latter case, since there is a difference in storage capacity and availability between terminals, only a part of the entire capacity can be used, and use of storage resources is wasted. That is, in the existing data distribution technology, when data fragmented in storage having different capacities is distributed and stored, a method that can use the entire storage capacity without waste has not been considered.

  The present invention has been made in view of the above problems, and an object thereof is to improve the utilization efficiency of distributed storage.

  In order to solve the above-mentioned problem, the data distribution apparatus according to claim 1, fragments the data, distributes and stores the data in a plurality of storages, and decrypts the original data using a number of pieces of fragment data equal to or less than the total number of pieces of data. In a possible data distribution device, at least one of the plurality of storages has a storage capacity different from the other, and a management unit that calculates the number of pieces of data according to all the storage capacity of the plurality of storages, and a calculation A conversion unit that fragments the data by the number of pieces of the data, a transfer unit that transfers different parts of the plurality of fragmented pieces of data to each storage according to the storage capacity of each storage, It is a summary to provide.

  Further, the data distribution apparatus according to claim 2 is the data distribution apparatus according to claim 1, wherein the management unit is configured such that the maximum common divisor of the storable capacities of the plurality of storages or the plurality of storages The gist is that the size of the fragment data is the greatest common divisor of values obtained by rounding each storable capacity into a unit of a certain size.

  The data distribution apparatus according to claim 3 is the data distribution apparatus according to claim 2, wherein the management unit divides all storable capacities of the plurality of storages by the greatest common divisor. The number of pieces of data is calculated, and the value obtained by subtracting the storage capacity of a predetermined storage from all the storage capacity of the plurality of storages is divided by the greatest common divisor to obtain the fragment data necessary for decoding the original data. The gist is to calculate the number.

  The data distribution apparatus according to claim 4 is the data distribution apparatus according to claim 2, wherein the management unit includes a storage that may be stopped and a storage that should not be stopped among the plurality of storages. Is used to calculate the number of pieces of fragment data necessary for decoding the original data by subtracting the value obtained by dividing all the storage capacity of the storage that may be stopped by the greatest common divisor from the number of pieces of the data. The gist is to do.

  Further, in the data distribution apparatus according to claim 5, in the data distribution apparatus according to claim 4, the management unit sets a variable for each number of the storage that may be stopped and the storage that should not be stopped. The calculation result of the calculation formula is maximized by using a calculation formula for calculating an available storage capacity by integrating the number of fragment data necessary for decoding the original data and the greatest common divisor. The gist is to calculate the respective numbers of the storage that may be stopped and the storage that should not be stopped.

  Further, in the data distribution apparatus according to claim 6, in the data distribution apparatus according to claim 5, the management unit sets a variable for each number of the storage that may be stopped and the storage that should not be stopped. And subtracting from 1 the maximum probability of simultaneous failure of the storage to which the number of pieces of fragment data equal to or greater than the value obtained by dividing all the storage capacity of the storage that may be stopped by the greatest common divisor is transferred. On the other hand, using the calculation formula that calculates the storage operation rate by integrating the integrated values of all the operation rates of the storage that should not be stopped, the stop is performed so that the calculation result of the calculation formula is maximized. The gist is to calculate the number of storages that may be stored and the storage that should not be stopped.

  Further, the data distribution apparatus according to claim 7 is a data distribution apparatus according to claim 5, further comprising: a linkage unit that notifies a storage that increases storage utilization efficiency based on the available storage capacity as an additional recommended storage; The gist is to provide further.

  Further, the data distribution apparatus according to claim 8 is the data distribution apparatus according to claim 7, wherein the cooperation unit is a server apparatus having a large storage capacity when the storage utilization efficiency is given priority. If the storage operation rate is given priority, the storage of the client terminal having the same storage capacity as the storage with the smallest storage capacity among the storages that may be stopped is notified. And

  The data distribution method according to claim 9 is a data distribution apparatus capable of fragmenting data, distributing and storing the data in a plurality of storages, and decoding original data using a number of pieces of fragment data equal to or less than the total number of pieces of data. In the data distribution method to be performed, at least one of the plurality of storages has a storable capacity different from the others, and a management step of calculating the number of pieces of data according to all storable capacity of the plurality of storages, A conversion step of fragmenting the data by the number of pieces of data, and a transfer step of transferring different parts of the fragmented pieces of fragment data to each storage according to the storage capacity of each storage, The gist is to provide.

  According to a ninth aspect of the present invention, there is provided a data distribution program for causing a computer to function as the data distribution apparatus according to any one of the first to eighth aspects.

  According to the present invention, it is possible to provide a distributed storage service that can be applied to both large-scale users and small-scale users and can use the prepared storage without waste.

It is a figure which shows the functional block structure of a secret sharing / data sharing system. It is a figure which shows the utilization efficiency and availability of storage. It is a figure which shows the utilization efficiency and availability of storage. It is a figure which shows the utilization efficiency and availability of storage. It is a figure which shows the calculation formula of file conversion information. It is a figure which shows storage information. It is a figure which shows the influence which the capacity | capacitance and classification of storage have on the utilization efficiency of storage. It is a figure which shows the proposal processing flow of an additional storage. It is a figure which shows the processing flow at the time of initialization. It is a figure which shows the processing flow at the time of secret sharing / data sharing. It is a figure which shows the processing flow at the time of file provision.

  In order to solve the above-mentioned problems, the present invention is a method that can use all the storage capacity even if the storage capacity varies between storages, and the entire system even if the availability (operation rate) varies between storages. We propose a method that can define availability. Specifically, (Method 1) In a distributed storage system using existing data distribution technology, intentionally restricts storage availability {= sacrificing storage availability (for example, RAID or redundancy) Ignoring down or disconnection of the storage using the power supply)}, increasing the number of pieces of fragment data stored in the storage, thereby improving the use efficiency of the storage. At that time, the maximum utilization efficiency: availability ratio required is realized by letting the user set the storage utilization efficiency and availability priority. (Method 2) The utilization efficiency of the storage is further improved by automatically obtaining the availability requirement set by the method 1 and determining the optimum setting value. (Method 3) When adding storage to the distributed storage system of Method 1, the influence of the addition of storage on the entire distributed storage system is analyzed, and the type and capacity of effective storage to be added are proposed. Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings.

<First Embodiment: Method 1>
In the first embodiment, method 1 will be described. In each embodiment, file data is used as an example of distributed storage target data.

  First, the existing data distribution technology will be further outlined. Secret sharing technology and data sharing technology divide a file into n pieces, store them in multiple storages, and can decrypt the original file using k pieces (k ≦ n) of n pieces Technology. More specifically, data distribution technology divides a file into multiple pieces to ensure storage availability, when a specific storage goes down or a network with a specific storage is disconnected However, it is a technology that enables the original file to be decrypted from the remaining fragment files. The secret sharing technique is a kind of data sharing technique that provides security in addition to availability by performing encryption so that an original plaintext file cannot be estimated from a fragment file.

  In such an existing data distribution technique, generally, “number of file fragments n = total number of storages”, “number of file fragments necessary for decryption k = minimum operation to provide availability” The number of storages required. On the other hand, in the present embodiment, focusing on the fact that the use efficiency of the storage is increased as the value of k / n approaches 1, and the bias of the distributed arrangement of the fragment files with respect to the storage is allowed (= in the storage operation rate). By providing a limitation, it is possible to use (n, k) pairs larger than the total number of storages, thereby realizing efficient and efficient use of storage resources.

  FIG. 1 is a diagram showing a functional block configuration of a secret sharing / data sharing system according to the present embodiment. In each embodiment, a secret sharing technique is used as an example of a data sharing technique. The secret sharing / data sharing system 1 includes a file transmission / reception unit 11, a file / fragment file conversion unit 12, a fragment file transfer unit 13, a file conversion information management unit 14, a storage information management unit 15, and system user cooperation. Unit 16.

  The file transmission / reception unit 11 receives the plaintext file to be secret shared from the storage use client terminal 3. Further, the decrypted original plain text file is transmitted to the storage use client terminal 3.

  The file / fragment file conversion unit 12 fragments the plaintext file using the number n of file fragments determined by the file conversion information management unit 14 and the number k of file fragments necessary for decryption, and each fragmented plaintext fragment Encrypt each file. Also, each encrypted fragment file collected from each storage providing terminal 5 is decrypted, and decrypted into the original plaintext file using each decrypted plaintext fragment file.

  The fragment file transfer unit 13 transfers each encrypted fragment file to each storage providing terminal 5 based on the file distribution method determined by the file conversion information management unit 14. Further, each encrypted fragment file is collected from each storage providing terminal 5.

  The file conversion information management unit 14 uses the parameters (total number T, capacity S, operation rate K, type C, etc.) of the storage providing terminal 5 connected to the secret sharing / data sharing system 1 for the storage providing terminal 5 The number n of file fragments and the number k of file fragments necessary for decryption are calculated so as to allow a bias in the distribution of encrypted fragment files, and the capacity D of the distributed storage that can be used in that case and the storage utilization efficiency R Are calculated together, and a distributed arrangement method is further determined.

  The storage information management unit 15 collects the total number T, capacity S, operation rate K, and type C of the storage providing terminals 5 connected or not connected to the secret sharing / data sharing system 1 from each storage providing terminal 5, Storage management is performed in association with each storage providing terminal 5. The storage capacity S refers to the capacity that can be stored in the storage. Instead of the operation rate K, a storage error rate E (≈1−operation rate K) may be used.

  The system user cooperation unit 16 provides the storage user terminal 3 with a screen used by the user when secret sharing is performed. For example, a screen showing the connection configuration / operation state of the storage providing terminal 5, a screen for allowing the user to input storage parameters (the number of storages A that can be stopped simultaneously, the number of storages B that ignores the risk of stoppage), and file fragments A screen for displaying the number n and its calculation formula, a screen for displaying additional recommended storage, and the like.

  The above is the functional example and functional configuration example of the secret sharing / data sharing system 1. Supplementally, the storage use client terminal 3 is a use terminal of a user who wants to utilize the secret sharing / data sharing system 1. A shared folder access program for accessing the file server program of the secret sharing / data sharing system 1 is operating. The storage providing terminal 5 is a distributed storage for storing and storing encrypted fragment files in a distributed manner. It is only necessary that at least one of the plurality of storages is different from the other storage capacity. Specifically, it is a server device such as NAS or cloud, or a client terminal such as a personal computer connected to the office via LAN. A storage sharing program is running inside.

  Next, a method for calculating the number n of file fragments, the number k of file fragments necessary for decryption, the capacity D of the distributed storage, the storage utilization efficiency R, and the method for determining the distributed arrangement will be described. Specifically, as described above, it is requested to allow a bias in the distributed arrangement of the encrypted fragment file with respect to the storage providing terminal 5. According to the secret sharing / data sharing system 1 which is a computer, “the number n of file fragments is determined according to all the storable capacities of the plurality of storage providing terminals 5, and a plurality of encryption methods are used for the distributed arrangement method. It can be said that a part of the fragment data different from each other is determined to be transferred to each storage providing terminal 5 according to the storage capacity of each storage providing terminal 5. Hereinafter, the storage providing terminal 5 is simply referred to as storage, and the encrypted fragment file is referred to as a fragment file.

(Conventional)
For example, as shown in the upper side of FIG. 2, when a 1 TB storage S 1, a 100 GB storage S 2, and a 10 GB storage S 3 can be used, if the existing data distribution technology is used, “file fragment number n = total number of storages” “T = 3” and “number of pieces of file necessary for decryption k = T−m = 2 (m: number of storages allowed to be down or disconnected)” are stored in the three storages S1 to S3, respectively. I was letting. In this case, “the capacity D of the distributed storage that can be used D = k × min (S) = 2 × 10 G = 20 GB” and “the storage use efficiency R = D / total (S) = 20 GB / 1110 GB = 2%”. The utilization efficiency of was low.

(Method a)
On the other hand, in the present embodiment, as shown in the lower side of FIG. 2, the value obtained by dividing the total storage capacity by the minimum storage capacity is defined as the number n of file fragments, which is 1 for the minimum storage capacity. Save one fragment file so that it corresponds. That is, the size of the minimum storage is set as the size of the fragment file. Specifically, “the number of file fragments n = total (S) / min (S) = 111” and the distributed arrangement method is “store 100 fragment files out of 111 in the storage S1, 10 fragments “Save file in storage S2 and save one fragment file in storage S3”. The number k of file fragments required for decryption is a value obtained by adding 1 to the value obtained by dividing the maximum storage capacity by the minimum storage capacity so that the maximum storage alone cannot be decrypted. That is, “max (S) / min (S) + 1 = 101” is set to the number k of file fragments necessary for decoding. In this case, “capacity D of available distributed storage D = k × min (S) = 101 × 10G = 1010 GB”, “storage utilization efficiency R = D / total (S) = 1010 GB / 1110 GB = 90%” It is possible to improve the storage usage efficiency. However, the number of file fragments k necessary for decryption increases from the conventional level, and if the maximum storage of 1 TB for storing many fragment files is stopped, decryption cannot be performed.

(Method b)
In the case of method a, since “1” is added because it is a condition that the maximum storage alone cannot be decrypted, there is a possibility that the newly added storage cannot be used effectively. Therefore, in the method b, as shown in the lower side of FIG. 3, decryption cannot be performed with the maximum storage alone, and the storage utilization efficiency is improved from the viewpoint of securing availability when m storages are down or disconnected. That is, the number k of file fragments required for decryption is a value obtained by subtracting a specific general-purpose storage capacity from the total storage capacity and dividing the subtraction value by the minimum storage capacity. Specifically, “(total (S) −100 GB × 1) / min (S) = 121” is set as the number k of file fragments necessary for decryption. Here, it is assumed that the storage of 100 GB is a specific general-purpose storage. As a result, the number k of file fragments necessary for decryption increases, so that the capacity D of the distributed storage that can be used and the storage utilization efficiency R can be improved as compared with the method a.

(Method c)
In the methods a and b, the size of the minimum storage is the size of the fragment file. On the other hand, in the method c, as shown in the lower side of FIG. 4, the largest common divisor of the capacity of each storage is set as the size of the fragment file. In other words, the value obtained by dividing the total storage capacity by the greatest common divisor of the storage is defined as the number n of file fragments. Specifically, “number of file fragments n = total (S) / gcd (S1, S2, S3, S4, S5) = 1440 GB / 40 = 36” and the distributed arrangement method is “25 out of 36. It is stored in 1 TB storage S1, three are stored in three 120 GB storages S2 to S4, respectively, and two are stored in 80 GB storage S5. In addition, for the number of file fragments k required for decryption, m storage used in method b is used as it is to ensure availability when down or disconnected, and the capacity of a specific general-purpose storage from the total capacity of the storage The minutes are subtracted, and the subtracted value is divided by the greatest common divisor. Specifically, “(total (S) −120 GB × 1) / gcd (S1, S2, S3, S4, S5) = 33” is set as the number k of file fragments necessary for decoding. By using the greatest common divisor of the capacity between the storages, the number k of file fragments required for decryption further increases, so that the capacity D of the distributed storage that can be used and the utilization efficiency R of the storage can be improved as compared with the method b. Can do. Note that gcd (S1, S2, S3, S4, S5) is the greatest common divisor of the capacities of the storages S1 to S5. In the methods a and b, the size of the fragment file is limited to the size of the minimum storage, but in the method c, (n, k) is derived using the greatest common divisor of the capacity between the storages. Can be suppressed.

  Next, calculation formulas for each parameter (n, k, D, R) used in method c will be described. Here, it is assumed that the user inputs the number A of storages that can be stopped simultaneously among the plurality of storages and the number of storages B (ignoring the risk of stopping) (= the number of storages whose operation is trusted). The storage number A may be all except the storage number B from all storages, or may be a part of a plurality of storages excluding the storage number B from all storages.

As shown in FIG. 5, the input information includes the total number of storages T, the number of storages A that may be stopped at the same time, the number of storages B that ignores the risk of stopping, and the capacity S of each storage that provides resources S = {S ₁ , S ₂ , S ₃ ,..., S _i ,... S _T } (∴for ∀S _i where i> B S _i ≧ S _{i + 1} , where B storages ignoring the risk of stopping are arranged at the top, and B + 1 Assuming that the subsequent storages are arranged in descending order of capacity), the file fragment number n is represented by equation (1), the file fragment number k required for decryption is represented by equation (2), and the available distributed storage capacity D is represented by equation (1). 3) The storage utilization efficiency R can be expressed as in equation (4).

  That is, the file fragment number n is calculated by dividing all the capacities of the plurality of storages by the greatest common divisor, as shown in equation (1). As for the number k of file fragments necessary for decryption, as shown in equation (2), the value obtained by dividing all the storage capacities that may be stopped simultaneously by the greatest common divisor is calculated from the number n of data fragments. Calculate by subtraction. The available distributed storage capacity D is calculated by integrating the number k of file fragments required for decryption and the greatest common divisor, as shown in equation (3). Further, the storage utilization efficiency R is calculated by dividing the available distributed storage capacity D by all the capacity of a plurality of storages, as shown in equation (4).

  Next, the processing operation of the secret sharing / data sharing system 1 performed in the present embodiment will be described.

  First, the system user cooperation unit 16 displays the storage information of the storage providing terminal 5 that can be used in the secret sharing / data sharing system 1 on the screen of the storage using client terminal 3, and the number of storages A that may be stopped simultaneously, The user is prompted to input the number of storages B that ignores the risk of stoppage (step S101). At this time, it is instructed to input the number of storages (A, B) in consideration of storage utilization efficiency and availability priority. Note that the system user cooperation unit 16 may calculate the number of storages (A, B) by allowing the user to select and specify a storage that may be stopped simultaneously and a storage that ignores the risk of the stop. In addition, it may be instructed to specify the number B of storages that ignore the risk of stoppage in descending order of storage capacity.

  Next, when the user inputs each storage number (A, B), the file conversion information management unit 14 uses each of the above-described methods a to c to set each parameter (n, k, D, R). ) And a file distribution method is determined (step S102). Note that which of the methods a to c is used may be based on setting information designated in advance, or may be selected by the user.

  Subsequently, the system user cooperation unit 16 calculates the calculation results of the parameters (n, k, D, R), their calculation formulas, the distributed arrangement method, and the number of storages (A, B) input by the user. It is displayed on the screen of the storage use client terminal 3 (step S103).

  After that, if the user performs OK processing with the contents, the file / fragment file conversion unit 12 uses the number n of file fragments calculated in step S102 and the number k of file fragments necessary for decryption to determine the secret sharing target. The plaintext file is fragmented, and each fragmented plaintext fragment file is encrypted (step S104). On the other hand, when the user performs an NG process on the display contents because the capacity D of the distributed storage that can be used is insufficient or the storage use efficiency R is poor, the process is performed without performing fragmentation and encryption. Exit.

  Finally, the fragment file transfer unit 13 transfers each encrypted fragment file to each storage providing terminal 5 based on the fragment file distribution method determined in step S102 (step S105).

  As described above, according to the present embodiment, it is allowed to bias the distributed arrangement of encrypted fragment files with respect to the storage providing terminal 5, that is, the number of file fragments according to all the storage capacity of the plurality of storage providing terminals 5. n is calculated, and different portions of the plurality of encrypted fragment data are transferred to each storage providing terminal 5 according to the storage capacity of each storage providing terminal 5, so that it is larger than the total number of storages (n, Since the pair k) can be used, the storage resource can be used efficiently, and the data size of the plain text file that can be stored in the distributed storage can be increased. Further, even if there is a deviation or variation in the storage capacity of the storage resource, the storage resource can be efficiently used in an aggregated manner. As a result, a personal computer or the like in the office can be used as storage.

  Further, according to the present embodiment, since (n, k) is calculated using the greatest common divisor of the storable capacity of the plurality of storage providing terminals 5, it is possible to use storage resources more efficiently, The bias and variation of the encrypted fragment file can be suppressed.

  Further, according to the present embodiment, (n, k) is calculated using the number of storages A that may be stopped simultaneously and the number of storages B that ignores the risk of stopping, so that storage resources can be more efficiently used. Can be used.

  Further, according to the present embodiment, the calculation result of each parameter (n, k, D, R), the calculation formula thereof, the distributed arrangement method, and the number of storages (A, B) input by the user are also obtained. Since it is visualized, the influence of each parameter (n, k, A, B) on the overall storage utilization efficiency can be clearly presented to the user.

  In the present embodiment, the maximum common divisor of the capacities of the plurality of storage providing terminals 5 is set as the fragment data size, but this is only an example. Usually, the capacity of the storage providing terminal 5 is, for example, 10,657 bytes or bit units, and is poor. In this case, the greatest common divisor becomes extremely small and the number of fragment files increases, which may reduce the distribution efficiency. Therefore, the capacity of each storage providing terminal 5 may be rounded to a unit of a certain size such as kilobytes or megabytes, and the greatest common divisor may be used using the rounded value. For example, in the case of the above 10,657 bytes, a value rounded to 10 kilobytes is used. As a result, the size of the greatest common divisor can be kept at a certain level, and the dispersion efficiency can be improved as compared with the case of using the actual capacity value. Note that “rounding” is synonymous with rounding, and means that a given numerical value is replaced with a numerical value that is an integral multiple of a certain rounding width. Generally, a power of 10 is a rounding width. Specifically, approximation to a certain size granularity, rounding down, rounding up, rounding, etc. are performed.

<Second Embodiment: Method 2>
In the second embodiment, method 2 will be described. In Method 2, the secret sharing / data sharing system 1 automatically derives the parameters (A, B) to be determined by the user by calculation of the optimization problem. Specifically, the parameters (A, B) are obtained so that the capacity D of the distributed storage that can be used shown by the equation (3) is maximized. Here, two methods will be described.

(Method a)
Simply, the file conversion information management unit 14 uses the brute force method, for example, when the total number of storages is 10, (A, B) = {(1, 9), (2, 8), ( 3, 7),..., (9, 1)} and substituting each combination pattern into equation (3) to obtain the value of D, and the value of D is maximized (A, B) The combination is determined as a parameter (A, B). Accordingly, it is possible to derive parameters (A, B) that maximize the capacity D of the distributed storage that can be used.

(Method b)
A method of deriving parameters (A, B) so as to ensure system availability (= operation rate) is also conceivable. Specifically, the storage information management unit 15 collects the operating rate K in addition to the capacity S of the storage providing terminal 5 when using the system, and stores and manages it as shown in FIG. Then, the file conversion information management unit 14 derives a parameter (A, B) by further using the collected operating rate K of the storage providing terminal 5. Specifically, for example, the operation rate K ′ of the entire storage is defined as shown in Expression (5), and the parameter ((5) and (3) is optimized as an optimization problem that maximizes either or both of Expression (5) and Expression (3). A, B) is calculated.

Formula (5) is demonstrated. On the right side, the variable ∂ is multiplied by the integrated value of all the utilization rates of storage that ignores the risk of outage. The variable ∂ is defined as “∂ = 1−max {(Σ _{i = B + 1} ^{B + 1 + A} S _i / gcd (S)) or more probabilities that a storage having a fragment file or more will simultaneously fail}”. That is, the maximum probability that the storage providing terminal 5 that can transfer all the pieces of storable capacity of the storage providing terminal 5 that may be stopped at the same time and the number of pieces of encrypted fragment data equal to or greater than the greatest common divisor will fail at the same time The value is subtracted from 1. Then, the “probability that a storage having fragment files equal to or larger than“ max {(Σ _{i = B + 1} ^{B + 1 + A} S _i / gcd (S)) ”simultaneously” is “Σ _{i = 1} ^T−B C _{(T−B, i)} Among combinations, there are equal to {1- (1-K ₁ ) × (1-K ₂ ) ×... × (1-K _i ) that maximizes} ”. The parameters (A, B) can be calculated using existing algorithms such as brute force method, dynamic programming, and greedy method.

  Accordingly, it is possible to derive parameters (A, B) that maximize the capacity D of the distributed storage that can be used while ensuring system availability (= operation rate). At this time, it is possible to impose a condition that A is large and B is small.

  As described above, according to the present embodiment, the secret sharing / data sharing system 1 automatically derives the parameters (A, B) to be determined by the user by calculating the optimization problem. The method can be determined. In addition, since the user does not need to be aware of the operating rate of each storage, the burden on the user can be reduced.

<Third Embodiment: Method 3>
In the third embodiment, method 3 will be described. In Method 3, when a new storage is added, the influence of the addition of the storage on the entire distributed storage system is analyzed, and an effective storage type and capacity to be added are proposed. Here, “effective” means that the storage utilization efficiency is high. That is, a condition for additional storage that improves the use efficiency R of the storage represented by Expression (4) is obtained.

From equation (4), it can be understood that various properties of the storage affect the utilization efficiency R. Therefore, the capacity of the added storage is defined as S _α , the operation rate as K _α , and the type as C _α as the contribution rate to the utilization efficiency R according to the property. For occupancy rates, it is considered to vary according to the type of storage, the average operating ratio K of all the storage of the same type to be existing based on the type C _alpha.

The effects of the capacity S _α and the type T _α on the storage usage efficiency can be summarized as shown in the table of FIG. That is, when the type C _alpha is server device such as NAS and cloud like, in general, comprises a storage capacity of the large capacity, since it is less likely the down or cut, the capacity S _alpha storage to add size Regardless of the storage, it can be used as a storage that does not assume a stop. This is equivalent to an increase in the portion of Σ _{i = 1} ^B S _i (= storage that can always be connected) in the equation (4), and the use efficiency R of the storage is improved by increasing the numerator value.

On the other hand, when the type C _α is a client terminal such as a personal computer or a BYOD (Bring your own device), it generally has a small storage capacity, so the storage usage efficiency depends on the capacity S _α. The effect on R is different. Therefore, for example, when the capacity S _α is larger than the capacity S _{B + 1 + A of the} existing storage, the number A of storages that can be stopped at the same time increases, so the storage utilization efficiency R decreases. This is equivalent to an increase in the portion of Σ _{i = 1} ^T S _{i in} the equation (4), and the storage utilization efficiency R decreases as the denominator value increases.

On the other hand, when the capacity S _α is equal to or less than the capacity S _{B + 1 + A of the} existing storage, the storage utilization efficiency R is improved by simply increasing the number of storages. This is equivalent to _{Σ i = A + B + 2} T S i part of the equation (4) increases, the storage of utilization R by increasing the value of the numerator is improved. At this time, the larger the capacity S _{α, the} better. Therefore, the largest capacity S _{B + 1 + A} among the conditions of “S _{B + 1 + A} ≧ S _α ” (= about the same capacity as the smallest capacity storage among the storages that may be stopped simultaneously) Is preferred.

Accordingly, the storage to be added in order to improve the use efficiency R of the storage has a large capacity server storage with a low possibility of being down or disconnected, or has a capacity of about S _{B + 1 + A} with a possibility of being down or disconnected. Client storage. Therefore, if the emphasis is placed on utilization efficiency and not on availability (operation rate), a large-capacity server storage is recommended. On the other hand, when importance is placed on availability (operation rate), the addition of a large-capacity server storage affects the availability of the entire system, so a client storage having a capacity of about S _{B + 1 + A} is recommended.

That is, as shown in FIG. 8, the storage information management unit 15 determines whether the priority of the storage addition by the user is storage utilization efficiency or storage availability (operation rate) (step S201), and determines the storage utilization efficiency. When giving priority, the system user cooperation unit 16 notifies the storage of the server apparatus having a large storage capacity as additional recommended storage (step S202), and when giving priority to the operation rate of the storage, about S _{B + 1 + A} The storage of the client terminal having the storable capacity is notified as the additional recommended storage (step S203). Specifically, for example, the screen of the storage-use client terminal 3 is displayed with a screen such as “It is possible to improve the use efficiency of storage by adding a large-capacity server device or a client terminal having a capacity of about S _{B + 1 + A} as storage”. To display.

As described above, according to the present embodiment, a large-capacity server storage with a low possibility of down or disconnection, or a client storage with a capacity of about _{SB + 1 + A} with a possibility of down or disconnection is proposed as an additional storage. As a result, the storage utilization efficiency can be improved.

  Next, the overall operation performed in the secret sharing / data sharing system 1 will be described with reference to FIGS. First, a processing flow at the time of initial setting will be described with reference to FIG.

  First, when storage information such as capacity is notified from the storage providing terminal 5, the storage information management unit 15 stores and manages the notified storage information (step S301). Specifically, the management is performed as shown in FIG. As for the operation rate, in particular, as described above, it may be the average value of the operation rates of all existing storages of the same type, or using the past connection status (for example, connection time per day) by the connected storage. It may be calculated.

  Next, when receiving the secret sharing request from the storage use client terminal 3, the storage information management unit 15 stores the storage information (storage number, capacity, type, operation) of the storage currently connected to the secret sharing / data sharing system 1 Response) (step S302).

  Subsequently, the system user cooperation unit 16 confirms whether or not the user selects / specifies automatically the number of storages A that may be stopped simultaneously and the number of storages B that ignores the risk of stoppage. Is transmitted (step S303).

  Thereafter, when the automatic setting is input in response to the confirmation request, the file conversion information management unit 14 itself sets the storage parameters (A, B) based on the method described in the second embodiment (method 2). An optimum value is calculated (step S304).

  On the other hand, when selection / designation is input by the user in response to the confirmation request, the file conversion information management unit 14 receives the storage parameters (A, B) input by the user (step S305). As will be described again, the number of storages A and B may be calculated by allowing the user to select and specify a storage that may be stopped simultaneously and a storage that ignores the risk of stopping. Further, it may be instructed to specify the number of storages B ignoring the risk of stoppage in descending order of capacity.

  Next, the file conversion information management unit 14 uses the storage parameters (A, B) calculated or input based on the method described in the first embodiment (method 1), and the file fragment number n, The number k of file fragments necessary for decryption, the capacity D of the distributed storage that can be used, and the storage utilization efficiency R are calculated, and the distributed arrangement method is determined (step S306).

  Thereafter, the system user cooperation unit 16 notifies the calculated available distributed storage capacity D as the data capacity that can be stored in the entire current storage (= the total capacity of fragment files necessary for decryption) (step S307). After that, all the parameter information calculated and determined in step S306 is stored (step S308).

  Finally, the user refers to the notified data capacity. For example, when the user performs NG processing because the data size of the secret sharing target file exceeds the data capacity, the storage information management unit 15 performs the third embodiment. Based on the method described in the form (method 3), an additional recommended storage that improves the use efficiency of the storage is calculated (step S309), and the system user cooperation unit 16 notifies the calculation result (step S310). On the other hand, when the user performs the OK process, this process ends.

  Next, a processing flow at the time of secret sharing (at the time of file writing) will be described with reference to FIG.

  First, when a plaintext secret sharing target file is transmitted from the storage use client terminal 3, the fragment file transfer unit 13 searches for an active storage that can be used with reference to the connection status / operation status of the storage providing terminal 5. (Step S401).

  After that, the file / fragment file conversion unit 12 determines the storage destination of the secret sharing target file based on the storage active response from the storage providing terminal 5 (step S402), and uses the parameters stored in step S308 for the secret. The distribution target file is fragmented and each plaintext fragment file is encrypted (step S403). After that, the fragment file transfer unit 13 transfers each encrypted fragment file to the storage destination storage providing terminal 5, and stores and manages each storage destination in association with the identifier of the secret sharing target file (step S404).

  Finally, when receiving the fragment file reception response including the hash value of the encrypted fragment file from the storage providing terminal 5 as the transfer destination, the file / fragment file conversion unit 12 receives the received hash value and the encrypted fragment before the transfer. A hash value calculated by itself is compared using a file, and if both hash values match, a storage completion response is returned to the storage use client terminal 3 (steps S405 and S406).

  Next, a processing flow at the time of file provision (file reading) will be described with reference to FIG.

  First, when a file acquisition request is transmitted from the storage use client terminal 3, the fragment file transfer unit 13 confirms the storage destination of the target file (step S501), and the fragment file is sent to each storage providing terminal 5 of the storage destination. Each provision request is transmitted (step S502).

  After that, when a certain number of encrypted fragment files cannot be collected because the storage providing terminal 5 is down or disconnected, or the provision of fragment files according to client authority is refused, the file / fragment file conversion unit 12 The original plaintext file is decrypted using the file (step S503) and transmitted to the storage use client terminal 3 (step S504).

  Finally, in each of the embodiments, the secret sharing technique has been described as an example. However, since it is sufficient if data can be distributed and stored, a general data distribution technique may be used. In addition, it can also be applied to distributed storage services such as Hadoop and SheepDog that construct a virtual large-capacity storage by storing fragment files in a large number of storages using data distribution.

  The secret sharing / data sharing system 1 according to the present embodiment can be realized by a computer having a calculation function such as a CPU and a storage function such as a memory. In addition, a program (software product) for causing the computer to function is constructed, or a storage medium (hardware product) for the program is generated, installed and executed on the computer, and distributed via a communication or distribution network. It is also possible.

1 ... Secret sharing / data sharing system (data sharing device)
11. File transmission / reception unit 12. File / fragment file conversion unit (conversion unit)
13: Fragment file transfer unit (transfer unit)
14 ... File conversion information management unit (management unit)
15. Storage information management unit (management unit)
16 ... System user cooperation part (cooperation part)
3 ... Storage use client terminal 5 ... Storage providing terminal S101 to S105, S201 to S203, S301 to S310, S401 to S406, S501 to S504 ... step

Claims (10)

In a data distribution device capable of fragmenting data and distributing and storing it in a plurality of storages, and decrypting the original data using a number of pieces of fragment data equal to or less than the total number of pieces of data,
At least one of the plurality of storages has a storage capacity different from the others,
A management unit that calculates the number of pieces of data according to the total storable capacity of the plurality of storages;
A conversion unit for fragmenting data by the calculated number of fragments of the data;
A transfer unit that transfers different pieces of fragmented pieces of data to each storage according to the storage capacity of each storage;
A data distribution apparatus comprising: The management unit
The largest common divisor of each storable capacity of the plurality of storages, or the greatest common divisor of values obtained by rounding each storable capacity of the plurality of storages into a unit of a certain size is used as the size of the fragment data. The data distribution apparatus according to claim 1. The management unit
The number of pieces of the data is calculated by dividing all the storage capacity of the plurality of storages by the greatest common divisor, and the storage capacity of the predetermined storage is subtracted from all storage capacity of the plurality of storages 3. The data distribution apparatus according to claim 2, wherein the number of fragment data necessary for decoding the original data is calculated by dividing a value by the greatest common divisor. The management unit
Using the storage that may be stopped and the storage that should not be stopped among the plurality of storages, a value obtained by dividing all the storable capacity of the storage that may be stopped by the greatest common divisor The data distribution apparatus according to claim 2, wherein the number of fragment data necessary for decoding the original data is calculated by subtracting from the number of fragments. The management unit
Storage that can be used by integrating the number of pieces of fragment data necessary for decoding of the original data and the greatest common divisor with the numbers of the storage that may be stopped and the storage that should not be stopped as variables. The number of storages that may be stopped and storage that should not be stopped is calculated using a calculation formula for calculating capacity so that the calculation result of the calculation formula is maximized. The data distribution apparatus according to claim 4. The management unit
The number of pieces of the storage that can be stopped and the storage that should not be stopped is a variable, and the number of fragments equal to or greater than the value obtained by dividing all the storage capacity of the storage that can be stopped by the greatest common divisor The storage operation rate is calculated by adding the integrated value of all the operation rates of the storages that should not be stopped to the value obtained by subtracting from 1 the maximum probability of failure of the storage to which data can be transferred simultaneously. The number of storages that may be stopped and storage that should not be stopped is calculated using a calculation formula so that a calculation result of the calculation formula becomes maximum, respectively. The data distribution device described. A linkage unit for notifying, as an additional recommended storage, a storage having high storage utilization efficiency based on the available storage capacity;
The data distribution apparatus according to claim 5, further comprising: The cooperation unit
When the storage utilization efficiency is prioritized, the storage of the server device having a large storage capacity is notified, and when the storage operation rate is prioritized, the most storable capacity among the storages that may be stopped The data distribution apparatus according to claim 7, wherein storage of a client terminal having the same storage capacity as that of a small storage is notified. In a data distribution method in which data is fragmented and distributed and stored in a plurality of storages, and the data distribution apparatus is capable of decrypting the original data using the number of pieces of fragment data equal to or less than the total number of pieces of data
At least one of the plurality of storages has a storage capacity different from the others,
A management step of calculating the number of pieces of data according to all the storable capacities of the plurality of storages;
A conversion step of fragmenting the data with the calculated number of fragments of the data;
A transfer step of transferring different parts of the fragmented pieces of fragment data to each storage according to the storage capacity of each storage;
A data distribution method comprising:   A data distribution program for causing a computer to function as the data distribution apparatus according to claim 1.

Images