JP6686495B2 - Grid storage control device, grid storage system, grid storage control method, and program - Google Patents

Description

  The present invention relates to a grid storage control device, a grid storage system, a grid storage control method, and a program, and in particular, reads a fragment from a plurality of storages and reconstructs a written data block, a grid storage control device, a grid storage system. , A grid storage control method, and a program.

  FIG. 1 is a diagram showing an outline of data writing to a storage node in a grid storage. FIG. 2 is a diagram showing an outline of reading data from a storage node in the grid storage.

  The grid storage divides the input data block into n (n is a plurality: n = 9 in FIGS. 1 and 2) data fragments (A in FIG. 1)). In FIG. 1, each solid square indicated by A) is a data fragment.

  After that, the grid storage calculates p (p is 1 or more: p = 3 in FIGS. 1 and 2) redundant fragments from the n data fragments and adds them to the n data segments (see FIG. 1). B)). In FIG. 1, each dashed square indicated by B) is a redundant fragment.

  Finally, the grid storage distributes and stores n data fragments and p redundant fragments in n + p storage nodes (C in FIG. 1). This allows the grid storage to have both improved fault tolerance and improved performance due to parallel access.

  At the time of reading, the grid storage reads at least n fragments from the n + p fragments distributed and stored (D in FIG. 2)), and reconstructs data from the read n fragments (E in FIG. 2)). . The n fragments may be a mixture of data fragments and redundant fragments.

  When there is a data read request from the client, the grid storage will read the minimum required n fragments or all n + p fragments including redundant data in order to restore the data block. , Either one of them is selected.

  When the required minimum number of n fragments are read from the storage node, the load and communication volume of the entire grid storage can be suppressed. However, when there is a concentration of access to a particular storage node, the data block read by the client to the grid storage is delayed by the particular storage node.

  FIG. 3 shows a state in which there is a bias in access among the storage nodes in the grid storage. In FIG. 3, a horizontally long rectangle is a data block read request issued by the client, and a straight line emitted from the rectangle is a fragment read request derived from each data block read request.

  FIG. 3 shows that two or three fragment read requests are derived from the six data block read requests. Of these, three fragment read requests are generated in the storage node 1 and the storage node 4, whereas six fragment read requests are concentrated in the storage node 3, and the storage node 3 It is a bottleneck.

  On the other hand, when reading fragments for all n + p storage nodes, the original data can be restored when all n fragments are available, so the client can shorten the time to read the data from the grid storage. However, this causes an increase in the communication volume in the grid storage and the load on the entire storage node.

  Patent Document 1 discloses a disk device that performs prefetching when reading a data block. This disk device changes the number of read-ahead data blocks according to the number of data blocks designated by the read command.

  Patent Document 2 discloses a storage control device that controls a storage device including a first disk that is operating and a second disk that is stopped. The storage control device issues a rotation start command to the second disk when the response time of the first disk is longer than a predetermined threshold value.

JP-A-6-180633 International Publication No. 2012/108040

  It is desirable to appropriately control the balance between the suppression of the data block read delay and the suppression of the increase in the total load of the grid storage according to the operating status of the grid storage. However, neither the technique disclosed in Patent Document 1 nor the technique disclosed in Patent Document 2 can enable such control. Both technologies are not related to the system that reads fragments from multiple storages and reconstructs the written data blocks.

  It is an object of the present invention to provide a grid storage control device, a grid storage system, a grid storage control method, and a program that solve the above problems.

  A grid storage control device according to an embodiment of the present invention stores n pieces of storage means for storing data fragments obtained by dividing a data block into n pieces (n is a plurality of pieces), and p (p) generated from the n pieces of the fragments. p is 1 or more) based on at least one of the characteristics of the read request of the input data block connected to the p storage means for storing redundant fragments and the load state of the storage means. , N or more and n + p or less are determined, and a read number determination means for issuing a read request for the fragment to the r storage means and n data fragments or parity fragments are read in total. Data reconstruction means for reconstructing and outputting the data block.

  A grid storage control method according to an embodiment of the present invention includes n storage means for storing data fragments obtained by dividing a data block into n (n is a plurality) pieces, and p (p (s) generated from the n pieces of the fragments. p is 1 or more), p storage means for storing redundant fragments, and a request for reading the data block from the storage means, the characteristics of the received read request of the block, and the load of the storage means. Based on at least one of the states, r of n or more and n + p or less is determined, a read request of the fragment is issued to r storage means, and a total of n data fragments or parity fragments are obtained. When read, the data block is reconstructed and output.

  A program according to an embodiment of the present invention includes n storage means for storing a data fragment obtained by dividing a data block into n (n is a plurality) pieces, and p (p is 1) generated from the n pieces of the fragments. Based on at least one of the characteristics of the read request of the data block input to the computer connected to the p storage means for storing the redundant fragments and the load state of the storage means. , N or more and n + p or less are determined, and a read number determination process of issuing a read request for the fragment to the r storage means and a total of n data fragments or parity fragments are read. , A data reconstruction process for reconstructing and outputting the data block.

  The grid storage control device according to the present invention appropriately adjusts the balance between the suppression of the data block read delay and the suppression of the increase in the total load of the grid storage system according to the operating status of the grid storage system.

FIG. 1 is a diagram showing an outline of data writing to a storage node in a grid storage. FIG. 2 is a diagram showing an outline of reading data from a storage node in the grid storage. FIG. 3 shows a state in which there is a bias in access among the storage nodes in the grid storage. FIG. 4 is a block diagram showing the configuration of the grid storage system 6 according to the first embodiment. FIG. 5 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes only the response delay allowance determination unit 21. FIG. 6 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes both the response delay allowance determination unit 21 and the load determination unit 22. Is. FIG. 7 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes only the load determination unit 22. FIG. 8 is an operation flowchart of the queue monitoring unit 41. FIG. 9 is a configuration diagram of the computer device 50. FIG. 10 is a block diagram showing the configuration of the grid storage control device 5 according to the second embodiment.

<First Embodiment>
<Outline>
The grid storage control device 5 according to the present embodiment changes the number of fragments read from the storage node 4 from n to n + p according to the characteristics of the read request of the data block received from the client. Here, n is the number of data fragments into which the data block is divided, and p is the number of parity fragments.

  The grid storage control device 5 also changes the number of fragments to be read depending on the load status of the grid storage system 6.

  With these mechanisms, the grid storage control device 5 avoids the response delay to the client even if there is a storage node 4 in which access concentration occurs and the response from the storage node 4 is delayed. At the same time, the grid storage control device 5 suppresses the number of fragments to suppress the load on the grid storage system 6 as a whole.

<Structure>
FIG. 4 is a block diagram showing the configuration of the grid storage system 6 according to the first embodiment.

  The grid storage system 6 includes a grid storage control device 5 and at least n + p storage nodes 4 connected to the device.

  In the grid storage system 6, the grid storage control device 5 receives a data block write request from a client and divides the data block into n (plural) data fragments. Furthermore, the grid storage control device 5 generates p (1 or more) parity fragments from n data fragments, and distributes these fragments to the n + p storage nodes 4 for writing.

  When the grid storage control device 5 accepts a data block read request from the client, it reads at least n fragments, reconstructs the data block, and returns it.

  The following description focuses on reading data blocks in the grid storage system 6.

  The grid storage control device 5 includes a request reception unit 1, a read number determination unit 2, and a data reconstruction unit 3. The read number determination unit 2 includes at least one of a response delay allowance determination unit 21 and a load determination unit 22. The load determination unit 22 includes a parity read flag 23 associated with each storage node 4.

  The request receiving unit 1 receives a data block read request from a client.

  The read number determination unit 2 activates at least one of the response delay allowance determination unit 21 and the load determination unit 22 to determine the fragment read number r between n and n + p in response to the input data block read request. Then, the reading of the number of fragments is executed.

  The response delay allowance determination unit 21 determines the fragment read number r based on the characteristics of the read request of the input data block.

  The load determination unit 22 determines the fragment read number r based on the load state of the storage node 4. Specifically, the load determination unit 22 increases the number r of read fragments when there is a storage node 4 with a high load. The reason for doing this is to enable rapid data reconstruction using fragments that have been read in early, and to suppress response delays due to variations in the load on the storage node 4.

  The parity read flag 23 of the load determination unit 22 indicates whether or not the load of the corresponding storage node 4 is equal to or greater than a predetermined threshold.

  The data reconstruction unit 3 reconstructs a data block from at least n fragments read from the storage node 4 and outputs it.

  Each storage node 4 includes a queue storage unit 41 and a read queue 42 together with a fragment storage area (not shown).

  The read queue 42 is a queue for processing a fragment read request issued by the grid storage controller 5.

  The queue monitoring unit 41 turns on the parity read flag 23 associated with the storage node 4 if the number of fragment read requests retained in the read queue 42 of a certain storage node 4 is greater than or equal to a predetermined threshold. If it is less than the predetermined threshold, it is set to off.

  The queue monitoring unit 41 and the read queue 42 may be included in the load determination unit 22.

  The request receiving unit 1, the read number determining unit 2, the response delay allowance determining unit 21, the load determining unit 22, the data reconfiguring unit 3, and the queue monitoring unit 41 are configured by a logic circuit.

  The request receiving unit 1, the read number determining unit 2, the response delay allowance determining unit 21, the load determining unit 22, the data reconstructing unit 3, and the queue monitoring unit 41 are executed by the grid storage control device 5, which is also the computer device 50. It may be realized by the program 53.

  FIG. 9 is a configuration diagram of the computer device 50. The computer device 50 includes a processor 51 and a main storage unit 52 which are interconnected by a bus 55. Here, the main memory unit 52 is a semiconductor memory device. The main storage unit 52 stores a program 53.

  The program 53 is executed by the processor 51, thereby causing the processor 51 to receive the request reception unit 1, the read number determination unit 2, the response delay allowance determination unit 21, the load determination unit 22, the data reconfiguration unit 3, and the queue monitoring unit. Function as 41. The main storage unit 52 stores the read queue 42.

<Operation>
FIG. 5 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes only the response delay allowance determination unit 21.

  When the request receiving unit 1 receives a data block read request (S1), the response delay allowance determination unit 21 determines the response level of this data read request and determines the number r of fragments to be read (S2).

The response delay allowance determination unit 21 determines the response level based on one of the following items or a combination of a plurality of items, and determines the number r of read fragments within a range of n or more and n + p or less.
a) Attributes of the client device that issued the read request, such as communication address, login user name b) Attributes of the virtual machine that issued the read request, such as communication address, name, type, and priority c) Applications that issued the read request Attribute of program, for example, name, type, priority d) Attribute of file to be accessed, for example, name, type, position on directory Response delay allowance determination unit 21 has a high response level (determines an early response) ), The number of read fragments r is increased.

  The conversion from the value of each attribute described above to the read fragment number r is performed based on, for example, a mapping rule registered in advance.

  The mapping rules are, for example, r = n if the application program type is batch processing, r = n + 1 if online response transaction processing, and r = n + 3 if online high response transaction processing. Good things. The mapping rule is, for example, if the login user is a normal user, the value obtained by adding 1 to the read number determined from other attributes is set as r, and if the privileged user is the read number determined from other attributes. The value obtained by adding 2 may be set to r.

  After the response delay allowance determination unit 21 determines the number r of fragments to be read (S2), the read number determination unit 2 issues a fragment read request to the number r of storage nodes 4 (S3). When the first n fragments are read, the data reconstructing unit 3 reconstructs a data block from the read fragments (S4) and responds to the client (S5).

  FIG. 6 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes both the response delay allowance determination unit 21 and the load determination unit 22. Is.

  When the request receiving unit 1 receives a data block read request (S11), the response delay allowance determination unit 21 determines the response level of the data read request and determines the number r1 of fragments to be read (S12). The response delay allowance determination unit 21 determines r1 in the same procedure as S2 in FIG.

  Next, the load determination unit 22 checks each parity read flag 23, and if there is one that is on (Yes in S13), the larger one of r1 and the number r2 of n + of the parity read flag 23 that is on is selected. , The number r of fragments to be read is determined (S14). When there is no parity read flag 23 turned on (No in S13), the load determination unit 22 determines r1 determined by the response delay allowance determination unit 21 as the number of fragments r to be read.

  After determining the number r of fragments to be read, the read number determination unit 2 issues a fragment read request to the storage nodes 4 of the number r (S15). When the first n fragments are read, the data reconstruction unit 3 reconstructs a data block from the read fragments (S16) and responds to the client (S17).

  FIG. 7 is an operation flowchart when the grid storage control device 5 receives a data block read request when the read number determination unit 2 includes only the load determination unit 22.

  When the request accepting unit 1 accepts a data block read request (S21), the load determining unit 22 checks each parity read flag 23, and if one is on (Yes in S22), n + parity read flag 23. The number r2 of ON of is determined as the number r of fragments to be read (S23). When there is no parity read flag 23 that is on (No in S22), the load determination unit 22 determines n to be the number r of fragments to be read (S27).

  After determining the number r of fragments to be read, the read number determination unit 2 issues a fragment read request to the storage nodes 4 of the number r (S24). When the first n fragments are read, the data reconstruction unit 3 reconstructs a data block from the read fragments (S25) and responds to the client (S26).

  FIG. 8 is an operation flowchart of the queue monitoring unit 41. The queue monitoring unit 41 is activated when the storage node 4 is initially set, and thereafter operates at regular intervals.

  The queue monitoring unit 41 of one storage node 4 monitors the number of fragment read requests accumulated in the read queue 42 of the storage node 4 (S31). When the number of staying read requests exceeds the predetermined threshold (No in S32), the queue monitoring unit 41 turns on the parity read flag 23 corresponding to the storage node 4 (S34).

  On the contrary, if the threshold is less than or equal to the predetermined threshold (Yes in S32), the queue monitoring unit 41 turns off the parity read flag 23 corresponding to the storage node 4 (S33).

<Effect>
The grid storage control device 5 appropriately adjusts the balance between the suppression of the data block read delay and the suppression of the increase in the total load of the grid storage system 6 according to the operating status of the grid storage system 6.

  The reason is that the read number determination unit 2 adjusts the read number of fragments according to the characteristics of the read request of data input from the client and the load status of the storage node 4.

<Second Embodiment>
FIG. 10 is a block diagram showing the configuration of the grid storage control device 5 according to the second embodiment.

  The grid storage control device 5 stores n storage units that store data fragments obtained by dividing a data block into n (n is a plurality), and p (p is 1 or more) redundant pieces generated from the n fragments. And p storage units for storing the fragments.

  The grid storage control device 5 includes a read number determination unit 2 and a data reconstruction unit 3.

  The read number determination unit 2 determines r of n or more and n + p or less, based on at least one of the characteristics of the input read request of the data block and the load state of the storage unit, and stores r storages. Issue a fragment read request to the department.

  When a total of n data fragments or parity fragments are read, the data reconstructing unit 3 reconstructs a data block and outputs it.

  The grid storage control device 5 appropriately adjusts the balance between the suppression of the data block read delay and the suppression of the increase in the total load of the grid storage system 6 according to the operating status of the grid storage.

  The reason is that the read number determination unit 2 adjusts the read number of fragments according to the characteristics of the read request of data input from the client and the load status of the storage node 4.

  Although the present invention has been described with reference to the exemplary embodiments, the present invention is not limited to the above exemplary embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

1 request reception unit 2 read number determination unit 3 data reconstruction unit 4 storage node 5 grid storage control device 6 grid storage system 21 response delay allowance determination unit 22 load determination unit 23 parity read flag 41 queue monitoring unit 42 read queue 50 computer device 51 processor 52 main memory 53 program 55 bus

Claims (10)

Connected to multiple storage means,
Said plurality of storage means, and said n memory means for storing a data block n (n is plural) number to the divided data fragment, p generated from said n data fragments (p is 1 or more) It includes a p number of said storage means for storing parity fragments, and
Characteristics of read requests of the inputted data block and, based on at least one of the load state of the storage means, n +1 or more and, to determine the n + p less r, the previously stored Symbol storage means And a read number determining means for issuing a fragment read request for requesting the data fragment or the parity fragment to the r storage means ,
Of n of the r said storage means receives said fragment read requests, at the time when the data fragment or the parity fragment requested is read, data reconstruction means for outputting to reconstructing the data block And a grid storage control device comprising: The read number determination means, an attribute of readings narrowing request issuing source client device of said data blocks, the attribute of the data block readings narrowing request issuing source of the application program, readings narrowing request issuing source of the data block The grid storage control device according to claim 1, further comprising: a response delay allowance determination unit that determines r based on at least one of an attribute of a virtual machine and an attribute of a file to which the data block belongs. The read number determining means, the number of said memory means the number of requests narrowing seen the fragment preparative reading has a pending exceeds a predetermined threshold value and r1, comprises the load determining means for determining n + r1 to r, The grid storage control device according to claim 1.   The read number determining means sets at least one of the attribute of the application program of the read request issuing source of the data block, the attribute of the virtual machine of the read request issuing source of the data block, and the attribute of the file to which the data block belongs. On the basis of r2 + n and r1, the response delay allowance determining means for determining r1 and the number of storing means for which the number of fragment read requests waiting to be processed exceed a predetermined threshold are defined as r2 + n and r1. The grid storage control device according to claim 1, further comprising a load determination unit that determines a larger one as r. A grid storage control device according to any one of claims 1 to 4,
A grid storage system including n + p storage means. A computer connecting multiple storage means
Data Block n (n is a plurality) stores n number of said storage means for storing the data fragment is divided into pieces, the n-number of the p (p is 1 or more) generated from the data fragment parity fragment and the p number of the storage means receives the read- request of the data block from,
Characteristics of read requests for said received data block, and, based on at least one of the load state of the storage means, n +1 or more and, to determine the n + p less r, the previously stored Symbol storage means Issuing a fragment read request for requesting the data fragment or the parity fragment to the r storage means ,
Of n of the r said storage means receives said fragment read requests, at the time when the data fragment or the parity fragment was requested has been read, and outputs the reconstructing the data block, grid storage control Method. Computer
Attributes of readings narrowing request issuing source client device of the data block, the data block attribute of the application program readings narrowing request issuing of an attribute of the data block readings narrowing request issuing source virtual machine and, The grid storage control method according to claim 6, wherein r is determined based on at least one of attributes of a file to which the data block belongs. Computer
The number of requests narrowing seen the fragments collected by reading be processed waiting a number of said storage means exceeds a predetermined threshold value and r1, grid storage control method according to claim 6 for determining the n + r1 to r. Computer
R1 is determined based on at least one of the attribute of the application program that issued the data block read request, the attribute of the virtual machine that issued the data block read request, and the attribute of the file to which the data block belongs. ,
7. The grid storage according to claim 6, wherein r2 is the number of storage units in which the number of fragment read requests waiting to be processed exceeds a predetermined threshold value, and r is the larger of r2 + n and r1. Control method. By a computer connected to multiple storage means,
Data Block n (n is a plurality) stores n number of said storage means for storing the data fragment is divided into pieces, the n-number of the p (p is 1 or more) generated from the data fragment parity fragment in the case where the p number of said storage means, from receiving the read request of said data blocks,
Characteristics of read requests of the inputted data block and, based on at least one of the load state of the storage means, n +1 or more and, to determine the n + p less r, the previously stored Symbol storage means And a read number determination process for issuing a fragment read request for requesting the data fragment or the parity fragment to the r storage means ,
Wherein the n number of r pieces of the storage means which receives the fragment read requests, at the time when the data fragment or the parity fragment requested is read, the data reconstruction processing and outputting the reconstructing the data block And a program to execute.

Images